Advertisement

Validation of an Integrated Risk Tool, Including Polygenic Risk Score, for Atherosclerotic Cardiovascular Disease in Multiple Ethnicities and Ancestries

Open AccessPublished:March 02, 2021DOI:https://doi.org/10.1016/j.amjcard.2021.02.032

      Highlights

      • A new integrated risk tool combines clinical and polygenic risk scores.
      • Prediction is significantly enhanced in multiple ethnicities and ancestries.
      • Ethnicities include Black and White, ancestries include African and European.
      • This is the first positive cross-ethnicity cross-ancestry validation of such a tool.
      The American College of Cardiology / American Heart Association pooled cohort equations tool (ASCVD-PCE) is currently recommended to assess 10-year risk for atherosclerotic cardiovascular disease (ASCVD). ASCVD-PCE does not currently include genetic risk factors. Polygenic risk scores (PRSs) have been shown to offer a powerful new approach to measuring genetic risk for common diseases, including ASCVD, and to enhance risk prediction when combined with ASCVD-PCE. Most work to date, including the assessment of tools, has focused on performance in individuals of European ancestries. Here we present evidence for the clinical validation of a new integrated risk tool (IRT), ASCVD-IRT, which combines ASCVD-PCE with PRS to predict 10-year risk of ASCVD across diverse ethnicity and ancestry groups. We demonstrate improved predictive performance of ASCVD-IRT over ASCVD-PCE, not only in individuals of self-reported White ethnicities (net reclassification improvement [NRI]; with 95% confidence interval = 2.7% [1.1 to 4.2]) but also Black / African American / Black Caribbean / Black African (NRI = 2.5% [0.6–4.3]) and South Asian (Indian, Bangladeshi or Pakistani) ethnicities (NRI = 8.7% [3.1 to 14.4]). NRI confidence intervals were wider and included zero for ethnicities with smaller sample sizes, including Hispanic (NRI = 7.5% [−1.4 to 16.5]), but PRS effect sizes in these ethnicities were significant and of comparable size to those seen in individuals of White ethnicities. Comparable results were obtained when individuals were analyzed by genetically inferred ancestry. Together, these results validate the performance of ASCVD-IRT in multiple ethnicities and ancestries, and favor their generalization to all ethnicities and ancestries.

      Introduction

      Current US guidelines for the primary prevention of cardiovascular disease are based on the quantification of an individual's predicted risk of atherosclerotic cardiovascular disease (ASCVD) over the following 10 years using the ASCVD pooled cohort equations tool (ASCVD-PCE).
      • Goff DC
      • Lloyd-Jones DM
      • Bennett G
      • Coady S
      • D'Agostino RB
      • Gibbons R
      • Greenland P
      • Lackland DT
      • Levy D
      • O'Donnell CJ
      • Robinson JG
      • Schwartz JS
      • Shero ST
      • Smith SC
      • Sorlie P
      • Stone NJ
      • 2013 Wilson PWF.
      ACC/AHA Guideline on the assessment of cardiovascular risk.
      • Grundy SM
      • Stone NJ
      • Bailey AL
      • Beam C
      • Birtcher KK
      • Blumenthal RS
      • Braun LT
      • Ferranti S de
      • Faiella-Tommasino J
      • Forman DE
      • Goldberg R
      • Heidenreich PA
      • Hlatky MA
      • Jones DW
      • Lloyd-Jones D
      • Lopez-Pajares N
      • Ndumele CE
      • Orringer CE
      • Peralta CA
      • Saseen JJ
      • Smith Jr, SC
      • Sperling L
      • Virani SS
      • Yeboah J.
      2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the management of blood cholesterol.
      • Arnett DK
      • Blumenthal RS
      • Albert MA
      • Buroker AB
      • Goldberger ZD
      • Hahn EJ
      • Himmelfarb CD
      • Khera A
      • Lloyd-Jones D
      • McEvoy JW
      • Michos ED
      • Miedema MD
      • Muñoz D
      • Smith SC
      • Virani SS
      • Williams KA
      • Yeboah J
      • Ziaeian B.
      2019 ACC/AHA Guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.
      The tool combines information from multiple clinical risk factors including age, sex, blood lipid levels, blood pressure, history of diabetes, smoking or anti-hypertensive treatment, and racial identity (entered as “White,” “African American” or “Other,” where “Other” is treated algorithmically as “White”). Although genetics is a known risk factor,
      • Khera AV
      • Kathiresan S.
      Genetics of coronary artery disease: discovery, biology and clinical translation.
      it is not directly included in ASCVD-PCE (family history of ASCVD is assessed separately, outside of the tool). Polygenic risk scores (PRSs), which combine information across thousands of common genetic variants in the human genome, can be added to the ASCVD-PCE tool, but previous studies have reported variable predictive performance.
      • Riveros-Mckay F
      • Weale M
      • Moore R
      • Selzam S
      • Krapohl E
      • Sivley RM
      • Tarran W
      • Sørensen P
      • Lachapelle A
      • Griffiths J
      • Saffari A
      • Deanfield J
      • Spencer C
      • Hippisley-Cox J
      • Hunter D
      • O'Sullivan J
      • Ashley E
      • Plagnol V
      • Donnelly P
      An integrated polygenic tool substantially enhances coronary artery disease prediction.
      • Elliott J
      • Bodinier B
      • Bond T
      • Chadeau-Hyam M
      • Evangelou E
      • Moons K
      • Dehghan A
      • Muller DC
      • Elliott P
      • Tzoulaki I.
      Predictive accuracy of a polygenic risk score–enhanced prediction model vs a clinical risk score for coronary artery disease.
      • Mosley J
      • Gupta D
      • Tan J
      • Yao J
      • Wells QS
      • Shaffer CM
      • Kundu S
      • Robinson-Cohen C
      • Psaty BM
      • Rich SS
      • Post WS
      • Guo X
      • Rotter JI
      • Roden DM
      • Gerszten RE
      • Wang TJ.
      Predictive accuracy of a polygenic risk score compared with a clinical risk score for incident coronary heart disease.
      • Mars N
      • Koskela JT
      • Ripatti P
      • Kiiskinen TTJ
      • Havulinna AS
      • Lindbohm JV
      • Ahola-Olli A
      • Kurki M
      • Karjalainen J
      • Palta P
      • Neale BM
      • Daly M
      • Salomaa V
      • Palotie A
      • Widén E
      • Ripatti S.
      Polygenic and clinical risk scores and their impact on age at onset and prediction of cardiometabolic diseases and common cancers.
      Additionally, these studies focused primarily on individuals with European ancestries, but it is known that the predictive accuracy of a PRS tends to attenuate in individuals with non-European ancestries.
      • Martin AR
      • Kanai M
      • Kamatani Y
      • Okada Y
      • Neale BM
      • Daly MJ.
      Clinical use of current polygenic risk scores may exacerbate health disparities.
      • Duncan L
      • Shen H
      • Gelaye B
      • Meijsen J
      • Ressler K
      • Feldman M
      • Peterson R
      • Domingue B.
      Analysis of polygenic risk score usage and performance in diverse human populations.
      • Wang Y
      • Guo J
      • Ni G
      • Yang J
      • Visscher PM
      • Yengo L.
      Theoretical and empirical quantification of the accuracy of polygenic scores in ancestry divergent populations.
      We therefore undertook a clinical validation study of a new 10-year ASCVD risk prediction tool (ASCVD-IRT) that integrates a PRS for ASCVD with the established ASCVD-PCE tool, paying particular attention to its predictive performance in non-European ancestries and non-White ethnicities.

      Methods

      Following recent guidelines on the use and reporting of race, ethnicity and ancestry,
      • Khan A
      • McHugh C
      • Conomos MP
      • Gogarten SM
      • Nelson SC
      Race and Genetics Discussion Group
      Guidelines on the use and reporting of race, ethnicity, and ancestry in the NHLBI Trans-Omics for Precision Medicine (TOPMed) program.
      we clarify that in this study we use the term “ethnicity” to refer to social categories including both race and ethnicity, and we do not distinguish “race” from “ethnicity.” In the testing cohorts described below, data on ethnicity were collected from questionnaire, census and other self-identification data, allowing us to infer that ethnicity was self-reported. We use the term “ancestry” or “genetic ancestry” to refer to inferences from genetic data. We note that the concepts of ethnicity and ancestry are correlated but not synonymous.
      Race Ethnicity and Genetics Working Group
      The use of racial, ethnic, and ancestral categories in human genetics research.
      To infer ancestry, we used a method based on principal components derived from genetic data to infer membership to one of 5 high-level ancestry groups conforming to those used by the 1000 Genomes Project
      The 1000 Genomes Project Consortium. A global reference for human genetic variation.
      (Sub-Saharan African [AFR], Native/Indigenous American [AMR], East Asian [EAS], European [EUR], and South Asian [SAS]; see Supplementary Materials for details, OTHER indicates mixed inferred ancestry). We note that this method is more accurately described as producing “ancestry-like” genetic similarity relationships.
      • Mathieson I
      • Scally A.
      What is ancestry?.
      We present performance evaluations for both ethnicity and ancestry groups, as the former relate to important social categories while the latter allow us to examine the ancestry-specific PRS attenuation issue
      • Martin AR
      • Kanai M
      • Kamatani Y
      • Okada Y
      • Neale BM
      • Daly MJ.
      Clinical use of current polygenic risk scores may exacerbate health disparities.
      • Duncan L
      • Shen H
      • Gelaye B
      • Meijsen J
      • Ressler K
      • Feldman M
      • Peterson R
      • Domingue B.
      Analysis of polygenic risk score usage and performance in diverse human populations.
      • Wang Y
      • Guo J
      • Ni G
      • Yang J
      • Visscher PM
      • Yengo L.
      Theoretical and empirical quantification of the accuracy of polygenic scores in ancestry divergent populations.
      and its effect on IRT performance.
      We tested the performance of the ASCVD-IRT using data from the Atherosclerosis Risk in Communities (ARIC) cohort, the Multi-Ethnic Study of Atherosclerosis (MESA), and UK Biobank (UKB). All individuals in these studies gave informed consent. Legal and ethical approval for our use of ARIC and MESA data is provided by the Western Institutional Review Board (Study Number 1264897, IRB Tracking Number 20192201). See Supplementary Materials for UKB approval information.
      All 3 cohorts are prospective, contain participants that were extensively examined at baseline, and have continuing follow-up (via annual phone calls for ARIC and MESA, via linkage to national electronic healthcare and death records for UKB). ARIC comprises over 15,000 adults from predominantly 2 study-defined racial/ethnic groups (“Black” and “White”), from defined populations in 4 sites in the USA, aged 45 to 64 years when recruited between 1987 and 1989.
      The ARIC Investigators
      The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives.
      MESA comprises over 6,000 adults from 4 study-defined racial/ethnic groups (“African American,” “Chinese American,” “Hispanic,” and “White/Caucasian”), recruited primarily via phone call invitation to 6 sites in the USA, aged 45 to 84 years and free of cardiovascular disease when recruited between 2000 and 2002.
      • Bild DE
      • Bluemke DA
      • Burke GL
      • Detrano R
      • Diez Roux A V.
      • Folsom AR
      • Greenland P
      • Jacobs DR
      • Kronmal R
      • Liu K
      • Nelson JC
      • O'Leary D
      • Saad MF
      • Shea S
      • Szklo M
      • Tracy RP
      Multi-ethnic study of atherosclerosis: objectives and design.
      ,
      • Olson JL
      • Bild DE
      • Kronmal RA
      • Burke GL.
      Legacy of MESA.
      UKB comprises over 500,000 adults, recruited via postal invitation to 22 sites in the UK, aged 40 to 69 when recruited between 2006 and 2010.
      • Sudlow C
      • Gallacher J
      • Allen N
      • Beral V
      • Burton P
      • Danesh J
      • Downey P
      • Elliott P
      • Green J
      • Landray M
      • Liu B
      • Matthews P
      • Ong G
      • Pell J
      • Silman A
      • Young A
      • Sprosen T
      • Peakman T
      • Collins R.
      UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age.
      ,
      • Bycroft C
      • Freeman C
      • Petkova D
      • Band G
      • Elliott LT
      • Sharp K
      • Motyer A
      • Vukcevic D
      • Delaneau O
      • O'Connell J
      • Cortes A
      • Welsh S
      • Young A
      • Effingham M
      • McVean G
      • Leslie S
      • Allen N
      • Donnelly P
      • Marchini J
      The UK Biobank resource with deep phenotyping and genomic data.
      We carefully selected 88,666 UKB individuals from multiple ethnicities (labels defined from questionnaire data) for the “IRT testing” subgroup, in order to ensure that the IRT testing subgroup was maximally enriched for incident ASCVD cases from non-European ethnicities and ancestries, and to ensure the independence of these individuals from other UKB subgroups that were used to construct and train the IRT (see Supplementary Materials for further details).
      We excluded individuals with cardiovascular disease or on cholesterol lowering medication at baseline, and also those related to others in the cohort at greater than third degree relative level according to the genetic inference method described in Bycroft et al.
      • Bycroft C
      • Freeman C
      • Petkova D
      • Band G
      • Elliott LT
      • Sharp K
      • Motyer A
      • Vukcevic D
      • Delaneau O
      • O'Connell J
      • Cortes A
      • Welsh S
      • Young A
      • Effingham M
      • McVean G
      • Leslie S
      • Allen N
      • Donnelly P
      • Marchini J
      The UK Biobank resource with deep phenotyping and genomic data.
      A separate bridging analysis was performed on individuals free of cardiovascular disease who were on cholesterol lowering medication at baseline, which indicated similar performance for this subgroup (Supplementary Figure 1). White participants in MESA and ARIC were excluded from testing due to sample overlap with GWASs used to construct the ASCVD PRS (MEGASTROKE_EUR
      • Malik R
      • Chauhan G
      • Traylor M
      • Sargurupremraj M
      • Okada Y
      • Mishra A
      • Rutten-Jacobs L
      • Giese AK
      • Van Der Laan SW
      • Gretarsdottir S
      • Anderson CD
      • Chong M
      • Adams HHH
      • Ago T
      • Almgren P
      • Amouyel P
      • Ay H
      • Bartz TM
      • Benavente OR
      • Bevan S
      • Boncoraglio GB
      • Brown RD
      • Butterworth AS
      • Carrera C
      • Carty CL
      • Chasman DI
      • Chen WM
      • Cole JW
      • Correa A
      • Cotlarciuc I
      • Cruchaga C
      • Danesh J
      • Bakker PIW De
      • Destefano AL
      • Den Hoed M
      • Duan Q
      • Engelter ST
      • Falcone GJ
      • Gottesman RF
      • Grewal RP
      • Gudnason V
      • Gustafsson S
      • Haessler J
      • Harris TB
      • Hassan A
      • Havulinna AS
      • Heckbert SR
      • Holliday EG
      • Howard G
      • Hsu FC
      • Hyacinth HI
      • Ikram MA
      • Ingelsson E
      • Irvin MR
      • Jian X
      • Jiménez-Conde J
      • Johnson JA
      • Jukema JW
      • Kanai M
      • Keene KL
      • Kissela BM
      • Kleindorfer DO
      • Kooperberg C
      • Kubo M
      • Lange LA
      • Langefeld CD
      • Langenberg C
      • Launer LJ
      • Lee JM
      • Lemmens R
      • Leys D
      • Lewis CM
      • Lin WY
      • Lindgren AG
      • Lorentzen E
      • Magnusson PK
      • Maguire J
      • Manichaikul A
      • McArdle PF
      • Meschia JF
      • Mitchell BD
      • Mosley TH
      • Nalls MA
      • Ninomiya T
      • O'Donnell MJ
      • Psaty BM
      • Pulit SL
      • Rannikmäe K
      • Reiner AP
      • Rexrode KM
      • Rice K
      • Rich SS
      • Ridker PM
      • Rost NS
      • Rothwell PM
      • Rotter JI
      • Rundek T
      • Sacco RL
      • Sakaue S
      • Sale MM
      • Salomaa V
      • Sapkota BR
      • Schmidt R
      • Schmidt CO
      • Schminke U
      • Sharma P
      • Slowik A
      • Sudlow CLM
      • Tanislav C
      • Tatlisumak T
      • Taylor KD
      • Thijs VNS
      • Thorleifsson G
      • Thorsteinsdottir U
      • Tiedt S
      • Trompet S
      • Tzourio C
      • Van Duijn CM
      • Walters M
      • Wareham NJ
      • Wassertheil-Smoller S
      • Wilson JG
      • Wiggins KL
      • Yang Q
      • Yusuf S
      • AFGen Consortium
      • Cohorts for Heart and Aging Research in Genomic Epidemiology (CHARGE) Consortium
      • International Genomics of Blood Pressure (iGEN-BP) Consortium
      • INVENT Consortium
      • STARNET
      • Bis JC
      • Pastinen T
      • Ruusalepp A
      • Schadt EE
      • Koplev S
      • Björkegren JLM
      • Codoni V
      • Civelek M
      • Smith NL
      • Trégouët DA
      • Christophersen IE
      • Roselli C
      • Lubitz SA
      • Ellinor PT
      • Tai ES
      • Kooner JS
      • Kato N
      • He J
      • Harst P van der
      • Elliott P
      • Chambers JC
      • Takeuchi F
      • Johnson AD
      • BioBank Japan Cooperative Hospital Group
      • COMPASS Consortium
      • EPIC-CVD Consortium
      • EPIC-InterAct Consortium, International Stroke Genetics Consortium (ISGC)
      • METASTROKE Consortium
      • Neurology Working Group of the CHARGE Consortium
      • NINDS Stroke Genetics Network (SiGN)
      • UK Young Lacunar DNA Study
      • MEGASTROKE Consortium
      • Sanghera DK
      • Melander O
      • Jern C
      • Strbian D
      • Fernandez-Cadenas I, Jr WTL
      • Rolfs A
      • Hata J
      • Woo D
      • Rosand J
      • Pare G
      • Hopewell JC
      • Saleheen D
      • Stefansson K
      • Worrall BB
      • Kittner SJ
      • Seshadri S
      • Fornage M
      • Markus HS
      • Howson JMM
      • Kamatani Y
      • Debette S
      • Dichgans M
      Multiancestry genome-wide association study of 520,000 subjects identifies 32 loci associated with stroke and stroke subtypes.
      and CARDIoGRAMplusC4D
      The CARDIoGRAMplusC4D Consortium
      A comprehensive 1,000 Genomes-based genome-wide association meta-analysis of coronary artery disease.
      respectively - see Supplementary Table 1). Black ARIC individuals were included, but it should be noted that they also formed part of the cohort data used to train the ASCVD-PCE model (outside of this study).
      • Goff DC
      • Lloyd-Jones DM
      • Bennett G
      • Coady S
      • D'Agostino RB
      • Gibbons R
      • Greenland P
      • Lackland DT
      • Levy D
      • O'Donnell CJ
      • Robinson JG
      • Schwartz JS
      • Shero ST
      • Smith SC
      • Sorlie P
      • Stone NJ
      • 2013 Wilson PWF.
      ACC/AHA Guideline on the assessment of cardiovascular risk.
      It may therefore be expected that the absolute prediction performance of both ASCVD-PCE and ASCVD-IRT is somewhat elevated in this group. However, our validation focused on the comparative performance of these 2 tools, which is not expected to be biased. All individuals in our testing cohorts were selected so as to be independent and unrelated to any individuals used in the training of the PRS or the IRT model.
      To define ASCVD cases, we used outcomes that closely matched the ASCVD-PCE tool. In MESA we used “CARDIOVASCULAR DISEASE (CVD), HARD,” and in ARIC we used a union of “MI (myocardial infarction), heart attack, or fatal CHD (coronary heart disease) by December 31, 2004” and “Definite or probable ischemic incident stroke by December 31, 2004.” The ASCVD definition for UK Biobank is described in Supplementary Materials.
      The ASCVD-IRT tool is a function of the score obtained from the currently established ASCVD-PCE tool
      • Goff DC
      • Lloyd-Jones DM
      • Bennett G
      • Coady S
      • D'Agostino RB
      • Gibbons R
      • Greenland P
      • Lackland DT
      • Levy D
      • O'Donnell CJ
      • Robinson JG
      • Schwartz JS
      • Shero ST
      • Smith SC
      • Sorlie P
      • Stone NJ
      • 2013 Wilson PWF.
      ACC/AHA Guideline on the assessment of cardiovascular risk.
      • Grundy SM
      • Stone NJ
      • Bailey AL
      • Beam C
      • Birtcher KK
      • Blumenthal RS
      • Braun LT
      • Ferranti S de
      • Faiella-Tommasino J
      • Forman DE
      • Goldberg R
      • Heidenreich PA
      • Hlatky MA
      • Jones DW
      • Lloyd-Jones D
      • Lopez-Pajares N
      • Ndumele CE
      • Orringer CE
      • Peralta CA
      • Saseen JJ
      • Smith Jr, SC
      • Sperling L
      • Virani SS
      • Yeboah J.
      2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the management of blood cholesterol.
      • Arnett DK
      • Blumenthal RS
      • Albert MA
      • Buroker AB
      • Goldberger ZD
      • Hahn EJ
      • Himmelfarb CD
      • Khera A
      • Lloyd-Jones D
      • McEvoy JW
      • Michos ED
      • Miedema MD
      • Muñoz D
      • Smith SC
      • Virani SS
      • Williams KA
      • Yeboah J
      • Ziaeian B.
      2019 ACC/AHA Guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.
      and a PRS for ASCVD, trained from multiple datasets that each represent individuals from multiple ancestry groups and from different geographies. Ten GWAS datasets for different ASCVD subtraits were meta-analyzed to derive the PRS, and an additional 4 cohorts were used to train the PRS effect size. Further details regarding the construction of ASCVD-IRT are provided in Supplementary Materials.
      We assessed predictive performance of ASCVD-IRT via relative performance comparisons to ASCVD-PCE, focusing in particular on differences in sensitivity and specificity and on the sum of these differences, also known as the Net Reclassification Improvement.
      • Pencina MJ
      • D'Agostino RB
      • D'Agostino RB
      • Vasan RS
      Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.
      We used ASCVD events in the following 10 years to define cases and we used scores above and below a certain risk threshold to define positive and negative results. This relative performance approach is justified because ASCVD-PCE provides a strong basis for comparison, being the currently established and recommended tool for 10-year ASCVD risk prediction in the US.
      • Goff DC
      • Lloyd-Jones DM
      • Bennett G
      • Coady S
      • D'Agostino RB
      • Gibbons R
      • Greenland P
      • Lackland DT
      • Levy D
      • O'Donnell CJ
      • Robinson JG
      • Schwartz JS
      • Shero ST
      • Smith SC
      • Sorlie P
      • Stone NJ
      • 2013 Wilson PWF.
      ACC/AHA Guideline on the assessment of cardiovascular risk.
      • Grundy SM
      • Stone NJ
      • Bailey AL
      • Beam C
      • Birtcher KK
      • Blumenthal RS
      • Braun LT
      • Ferranti S de
      • Faiella-Tommasino J
      • Forman DE
      • Goldberg R
      • Heidenreich PA
      • Hlatky MA
      • Jones DW
      • Lloyd-Jones D
      • Lopez-Pajares N
      • Ndumele CE
      • Orringer CE
      • Peralta CA
      • Saseen JJ
      • Smith Jr, SC
      • Sperling L
      • Virani SS
      • Yeboah J.
      2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the management of blood cholesterol.
      • Arnett DK
      • Blumenthal RS
      • Albert MA
      • Buroker AB
      • Goldberger ZD
      • Hahn EJ
      • Himmelfarb CD
      • Khera A
      • Lloyd-Jones D
      • McEvoy JW
      • Michos ED
      • Miedema MD
      • Muñoz D
      • Smith SC
      • Virani SS
      • Williams KA
      • Yeboah J
      • Ziaeian B.
      2019 ACC/AHA Guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.
      We note that absolute values of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) are difficult to interpret on their own. ASCVD-IRT and ASCVD-PCE are not diagnostic tests but risk predictors of an uncertain future event, where even under the best of conditions a “positive” result has a measured probability of being a noncase and a “negative” result has a measured probability of being a case. For example, we label an individual with a risk of 8% a “positive” result, but they only have an 8% chance of becoming an ASCVD case, assuming the model is correct. But while absolute values are difficult to interpret, relative improvements remain good indicators of performance improvements. In particular, simultaneous improvements in both sensitivity and specificity are strongly indicative of superior test performance.
      There is also a logical choice for which threshold to use for binary compartmentalization into “positive” (high-risk) and “negative” (low-risk) results, as required for metrics such as sensitivity and specificity. The same guidelines recommending ASCVD-PCE for risk prediction also lay out recommendations for actionable risk thresholds above which intervention is advised. This threshold is 7.5% risk over 10 years for most individuals, but is reduced to 5% for individuals presenting with additional risk factors, one of which is South Asian ethnicity.
      • Grundy SM
      • Stone NJ
      • Bailey AL
      • Beam C
      • Birtcher KK
      • Blumenthal RS
      • Braun LT
      • Ferranti S de
      • Faiella-Tommasino J
      • Forman DE
      • Goldberg R
      • Heidenreich PA
      • Hlatky MA
      • Jones DW
      • Lloyd-Jones D
      • Lopez-Pajares N
      • Ndumele CE
      • Orringer CE
      • Peralta CA
      • Saseen JJ
      • Smith Jr, SC
      • Sperling L
      • Virani SS
      • Yeboah J.
      2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the management of blood cholesterol.
      ,
      • Arnett DK
      • Blumenthal RS
      • Albert MA
      • Buroker AB
      • Goldberger ZD
      • Hahn EJ
      • Himmelfarb CD
      • Khera A
      • Lloyd-Jones D
      • McEvoy JW
      • Michos ED
      • Miedema MD
      • Muñoz D
      • Smith SC
      • Virani SS
      • Williams KA
      • Yeboah J
      • Ziaeian B.
      2019 ACC/AHA Guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.
      We therefore applied the same thresholds for our clinical performance assessment, applying the 5% threshold for individuals of South Asian ethnicities or ancestries and 7.5% for all others.
      Further details on the statistical methods used to calculate performance metrics and their confidence intervals are provided in Supplementary Materials.

      Results

      Tables 1 and 2 show the subgroup sample sizes, after exclusions, in each IRT testing cohort. Figure 1 and Supplementary Table 2 summarize the relative performance improvements of ASCVD-IRT over the currently established tool (ASCVD-PCE). When meta-analyzed across all testing cohorts, ASCVD-IRT shows significantly improved performance (as measured by 95% CI) across NRI, sensitivity, and specificity (combined results [with 95% CI]: NRI = 3.0% [1.7 to 4.3]; delta-sensitivity [equivalent to NRI in cases] = 1.5% [0.2 to 2.8]; delta-specificity [equivalent to NRI in noncases] = 1.5% [1.3 to 1.7]). Positive predictive value (PPV) and negative predictive value (NPV) were also significantly improved (Supplementary Table 2). The NRI is also significantly positive in all 3 cohorts, with the largest point estimates seen in the 2 US cohorts (Figure 1a, Supplementary Table 2). The within-cohort point estimates for changes in sensitivity, specificity, log(PPV), log(NPV) and Harrell's C
      • Harrell FE
      • Califf RM
      • Pryor DB
      • Lee KL
      • Rosati RA.
      Evaluating the yield of medical tests.
      are also all positive, albeit in some cases with 95% confidence intervals that are large and overlap zero (Figure 1b-c, Supplementary Table 2).
      Table 1IRT testing cohort sample numbers (and percentage of cohort) by sex, age at recruitment, case status and self-reported ethnicity
      CohortSelf-reported ethnicityAge at recruitmentWomenMen
      CasesNoncasesCasesNoncases
      ARICBlack40-5447 (2.4%)762 (39%)38 (1.9%)422 (21%)
      55-6940 (2.0%)383 (19%)40 (2.0%)233 (12%)
      MESAChinese American40-540 (0.0%)0 (0.0%)0 (0.0%)0 (0.0%)
      55-690 (0.0%)1 (0.1%)1 (0.1%)3 (0.2%)
      70-790 (0.0%)0 (0.0%)0 (0.0%)0 (0.0%)
      African American40-544 (0.2%)191 (9.9%)8 (0.4%)154 (8.0%)
      55-6910 (0.5%)252 (13%)20 (1.0%)216 (11%)
      70-7915 (0.8%)102 (5.3%)16 (0.8%)109 (5.7%)
      Hispanic40-542 (0.1%)134 (7.0%)1 (0.1%)146 (7.6%)
      55-6910 (0.5%)159 (8.3%)25 (1.3%)186 (9.7%)
      70-798 (0.4%)59 (3.1%)9 (0.5%)84 (4.4%)
      UKB (IRT testing)White40-54135 (0.2%)18191 (21%)308 (0.4%)14167 (16%)
      55-69588 (0.7%)25162 (28%)1048 (1.2%)18098 (20%)
      Indian, Bangladeshi or Pakistani40-5412 (0.01%)1211 (1.4%)53 (0.1%)1136 (1.3%)
      55-6917 (0.02%)652 (0.7%)54 (0.1%)553 (0.6%)
      Black Caribbean or Black African40-5414 (0.02%)1628 (1.8%)17 (0.02%)1194 (1.4%)
      55-6916 (0.02%)633 (0.7%)18 (0.02%)415 (0.5%)
      Chinese40-541 (0.001%)390 (0.4%)0 (0.0%)233 (0.3%)
      55-694 (0.01%)238 (0.3%)1 (0.001%)112 (0.1%)
      Other40-5410 (0.01%)948 (1.1%)8 (0.01%)624 (0.7%)
      55-6913 (0.01%)501 (0.6%)14 (0.02%)249 (0.3%)
      IRT = Integrated Risk Tool.
      Table 2.IRT testing cohort sample numbers (and percentage of cohort) by sex, age at recruitment, case status and genetically inferred ancestry
      CohortGenetic ancestryAge at recruitmentWomenMen
      CasesNoncasesCasesNoncases
      ARICAFR40-5446 (2.3%)748 (38%)37 (1.9%)410 (21%)
      55-6939 (2.0%)370 (19%)40 (2.0%)223 (11%)
      EUR40-540 (0.0%)4 (0.2%)0 (0.0%)2 (0.1%)
      55-690 (0.0%)1 (0.1%)0 (0.0%)3 (0.2%)
      OTHER40-541 (0.1%)10 (0.5%)1 (0.1%)10 (0.5%)
      55-691 (0.1%)12 (0.6%)0 (0.0%)7 (0.4%)
      MESAAFR40-544 (0.2%)202 (10%)8 (0.4%)156 (8.1%)
      55-6910 (0.5%)246 (13%)21 (1.1%)217 (11%)
      70-7914 (0.7%)106 (5.5%)14 (0.7%)105 (5.5%)
      AMR40-540 (0.0%)41 (2.1%)0 (0.0%)62 (3.2%)
      55-694 (0.2%)55 (2.9%)8 (0.4%)91 (4.7%)
      70-792 (0.1%)21 (1.1%)4 (0.2%)26 (1.4%)
      EAS40-540 (0.0%)0 (0.0%)0 (0.0%)0 (0.0%)
      55-690 (0.0%)2 (0.1%)1 (0.1%)3 (0.2%)
      70-790 (0.0%)0 (0.0%)0 (0.0%)0 (0.0%)
      EUR40-542 (0.1%)39 (2.0%)0 (0.0%)39 (2.0%)
      55-695 (0.3%)38 (2.0%)6 (0.3%)45 (2.3%)
      70-793 (0.2%)14 (0.7%)2 (0.1%)28 (1.5%)
      OTHER40-540 (0.0%)43 (2.2%)1 (0.1%)43 (2.2%)
      55-691 (0.1%)71 (3.7%)10 (0.5%)49 (2.5%)
      70-794 (0.2%)20 (1.0%)5 (0.3%)34 (1.8%)
      UKB (IRT testing)AFR40-5420 (0.02%)1876 (2.1%)19 (0.02%)1359 (1.5%)
      55-6918 (0.02%)722 (0.8%)19 (0.02%)462 (0.5%)
      EAS40-543 (0.003%)705 (0.8%)0 (0.0%)363 (0.4%)
      55-6910 (0.01%)412 (0.5%)3 (0.003%)152 (0.2%)
      EUR40-54135 (0.2%)18182 (21%)307 (0.3%)14148 (16%)
      55-69588 (0.7%)25146 (28%)1048 (1.2%)18090 (20%)
      SAS40-5411 (0.01%)802 (0.9%)26 (0.03%)714 (0.8%)
      55-6912 (0.01%)492 (0.6%)33 (0.04%)353 (0.4%)
      OTHER40-543 (0.003%)803 (0.9%)34 (0.04%)770 (0.9%)
      55-6910 (0.01%)414 (0.5%)32 (0.04%)370 (0.4%)
      IRT = Integrated Risk Tool; AFR = Sub-Saharan African; AMR = Native/Indigenous American; EAS = East Asian; EUR = European; SAS = South Asian.
      Figure 1
      Figure 1Relative performance of ASCVD-IRT over the currently established ASCVD-PCE tool, split by cohort and combined via meta-analysis across cohorts. (a) Net Reclassification Improvement (NRI), with 95% confidence intervals (CI). (b) Delta-sensitivity (equivalent to NRI in cases) with 95% CI. (c) Delta-specificity (equivalent to NRI in noncases) with 95% CI. Vertical dotted lines at zero indicate the null hypothesis of no change.
      We proceeded to investigate relative performance patterns within ethnicity (Figure 2, Supplementary Table 3) and ancestry (Supplementary Figure 2, Supplementary Table 4) groups (groups with >50 ASCVD cases across men and women are shown, metrics for groups with fewer cases can be found in Supplementary Tables 3 to 4). Both figures show similar patterns. The overall NRI remains significantly positive when individuals are meta-analyzed for the 2 largest groups we have data for – those corresponding to White or Black / African American / Black Caribbean / Black African self-reported ethnicities and EUR or AFR genetic ancestries. The significantly positive NRI results for Black / African American / Black Caribbean / Black African ethnicities (combined NRI = 2.5% [0.6 to 4.3]) and AFR ancestry (combined NRI = 2.2% [0.4 to 4.1]) are especially noteworthy, given the reported attenuation of PRS performance in individuals of African genetic ancestries.
      • Martin AR
      • Kanai M
      • Kamatani Y
      • Okada Y
      • Neale BM
      • Daly MJ.
      Clinical use of current polygenic risk scores may exacerbate health disparities.
      • Duncan L
      • Shen H
      • Gelaye B
      • Meijsen J
      • Ressler K
      • Feldman M
      • Peterson R
      • Domingue B.
      Analysis of polygenic risk score usage and performance in diverse human populations.
      • Wang Y
      • Guo J
      • Ni G
      • Yang J
      • Visscher PM
      • Yengo L.
      Theoretical and empirical quantification of the accuracy of polygenic scores in ancestry divergent populations.
      The sample sizes and case numbers for other ethnicities and ancestries are lower, meaning that in some contexts it is not possible to reject the null hypothesis that there is no change in NRI. We note that there are no instances of significantly negative performance, whereas there are several instances of significantly positive NRI performance (for MESA-AMR, UKB “Indian, Bangladeshi, or Pakistani,” and UKB-SAS), and point estimates are also generally positive. The changes in sensitivity (equivalent to NRI in cases), specificity (equivalent to NRI in noncases), log(PPV), log(NPV) and Harrell's C are either significantly positive or not significantly different from zero (Figure 2b, c, Supplementary Figure 2b, c, Supplementary Tables 3, 4). We also find that ASCVD-IRT has, across the same large ethnicity and ancestry groups, a larger NRI than that of a tool constructed in the same way as ours, but using an alternative PRS previously shown to have good cross-ancestry performance (the coronary artery disease PRS of Inouye et al,
      • Inouye M
      • Abraham G
      • Nelson CP
      • Wood AM
      • Sweeting MJ
      • Dudbridge F
      • Lai FY
      • Kaptoge S
      • Brozynska M
      • Wang T
      • Ye S
      • Webb TR
      • Rutter MK
      • Tzoulaki I
      • Patel RS
      • Loos RJF
      • Keavney B
      • Hemingway H
      • Thompson J
      • Watkins H
      • Deloukas P
      • Di Angelantonio E
      • Butterworth AS
      • Danesh J
      • Samani NJ
      Genomic risk prediction of coronary artery disease in 480,000 adults: implications for primary prevention.
      ,
      • Dikilitas O
      • Schaid DJ
      • Kosel ML
      • Carroll RJ
      • Chute CG
      • Denny JA
      • Fedotov A
      • Feng Q
      • Hakonarson H
      • Jarvik GP
      • Lee MTM
      • Pacheco JA
      • Rowley R
      • Sleiman PM
      • Stein CM
      • Sturm AC
      • Wei WQ
      • Wiesner GL
      • Williams MS
      • Zhang Y
      • Manolio TA
      • Kullo IJ.
      Predictive utility of polygenic risk scores for coronary heart disease in three major racial and ethnic groups.
      Supplementary Figure 3).
      Figure 2
      Figure 2Relative performance improvements of ASCVD-IRT over the currently established ASCVD-PCE tool, split by self-reported ethnicities. (a) Net Reclassification Improvement (NRI), with 95% confidence intervals (CI). (b) Delta-sensitivity (equivalent to NRI in cases) with 95% CI. (c) Delta-specificity (equivalent to NRI in noncases) with 95% CI. Vertical dotted lines at zero indicate the null hypothesis of no change.
      Performance metrics for smaller groups (with fewer than 50 cases across men and women) are reported in Supplementary Tables 3 and 4. As the sample sizes are small, it is to be expected that most of the reported CIs in these smaller groups overlap zero. For example, UKB contains a small group of Chinese ethnicities (n = 979, with 6 cases), and MESA contains an even smaller group of Chinese American ethnicities (n = 5, with 1 case). The case numbers in these 2 groups are too low to provide an accurate estimate of performance.
      We next proceeded to investigate relative performance patterns in 4 sex-by-age subgroups (Figure 3, Supplementary Table 5). This analysis reiterates patterns previously reported for individuals of European ancestries in UKB and using a different IRT built from a coronary artery disease PRS.
      • Riveros-Mckay F
      • Weale M
      • Moore R
      • Selzam S
      • Krapohl E
      • Sivley RM
      • Tarran W
      • Sørensen P
      • Lachapelle A
      • Griffiths J
      • Saffari A
      • Deanfield J
      • Spencer C
      • Hippisley-Cox J
      • Hunter D
      • O'Sullivan J
      • Ashley E
      • Plagnol V
      • Donnelly P
      An integrated polygenic tool substantially enhances coronary artery disease prediction.
      The overall NRI performance in 3 of the 4 subgroups is significantly positive, with the strongest performance seen in younger middle-aged men (40 to 54 year old) (NRI = 10.3% [5.7 to 15.0]). The NRI estimates within cohorts are either significantly positive (for ARIC 40 to 54 year old men and women) or not significantly different from zero. It is noteworthy that the largest NRI point estimate in ARIC, comprising individuals self-reporting as Black, is also for younger middle-aged men.
      Figure 3
      Figure 3Relative performance improvements of ASCVD-IRT over the currently established ASCVD-PCE tool, split by 4 sex-by-age subgroups. (a) Net Reclassification Improvement (NRI), with 95% confidence intervals (CI). (b) Delta-sensitivity (equivalent to NRI in cases) with 95% CI. (c) Delta-specificity (equivalent to NRI in noncases) with 95% CI. Vertical dotted lines at zero indicate the null hypothesis of no change.
      UKB sex-by-age subgroups vary in their sensitivity and specificity patterns (Figure 3b, c, Supplementary Table 5), with younger middle-aged men and women (40 to 54 year old) showing significant increases in sensitivity and smaller but significant decreases in specificity, while older middle-aged men (55 to 69 year old) show the opposite pattern. The 2 US cohorts are more balanced, with no significantly negative performance estimates in any subgroup, while some effects remain significantly positive (delta-specificities for ARIC 40 to 54 year old men and women, ARIC 55 to 69 year old women, and MESA 55 to 69 year old men and women).
      Detailed tables of sensitivity, specificity, PPV, NPV, Harrell's C, and their comparative differences to ASCVD-PCE (deltas and NRI) are provided in Supplementary Tables 2 to 5). We also provide an equivalent Supplementary Table 6 for an analysis carried out using a different version of the IRT that used the same PRS and PRS coefficients but was integrated with the QRISK2 score that is recommended for use in the UK.
      • Hippisley-Cox J
      • Coupland C
      • Vinogradova Y
      • Robson J
      • Minhas R
      • Sheikh A
      • Brindle P.
      Predicting cardiovascular risk in England and Wales: Prospective derivation and validation of QRISK2.
      ,
      National Institute for Health and Care Excellence
      Cardiovascular disease: risk assessment and reduction, including lipid modification (Clinical guideline [CG181]).

      Discussion

      Our results indicate that ASCVD-IRT, a new tool for estimating 10-year ASCVD risk that incorporates a PRS for ASCVD, outperforms the existing standard-of-care tool ASCVD-PCE, and that this improvement extends across ethnicities and genetic ancestries. The 2 US-based cohorts used in our validation (ARIC and MESA) are drawn, in part, from minority US ethnic groups, allowing us to demonstrate that, in addition to individuals of White ethnicities or European ancestries, the significant improvement of ASCVD-IRT is also seen in individuals with Black or African American self-reported ethnicities and African genetic ancestries.
      Data in other ethnicities and ancestries are more limited. However, the UKB contains a reasonably large group (n = 3,688, with 136 cases) of individuals of South Asian (“Indian, Bangladeshi, or Pakistani”) ethnicities, and a significantly improved performance of ASCVD-IRT is seen in this group as well. MESA contains a smaller group of Hispanic ethnicities (n = 823, with 55 cases), and although the NRI point estimate (7.5%) is positive, there was insufficient power to reject the null hypothesis of no change. Groups of East Asian ethnicities are even smaller in the IRT testing data (n = 979, with 6 cases, of Chinese ethnicities in UKB; n = 5, with 1 case, of Chinese American ethnicities in MESA), and this resulted in poor estimation of NRI and wide 95% CIs that extend to either side of zero.
      The extent to which the results from larger groups can be generalized to support the clinical use of ASCVD-IRT in individuals from ethnicities and ancestries that are poorly represented in the IRT testing data is an important question. Three lines of evidence support such a generalization. First, we have data from additional cohorts (described in Supplementary Materials) that indicate the ASCVD PRS has an effect size in individuals of Hispanic and East Asian ethnicities that is positive, significantly different from zero and comparable in size to that seen in individuals of White ethnicities, with an equivalent pattern also seen across genetically inferred ancestries (Supplementary Figure 4). Although these cohorts lack the necessary longitudinal and covariate information to calculate ASCVD-PCE at baseline, and therefore could not be used for IRT testing, these results permit the inference that the strong predictive performance seen at the PRS level should transfer to the IRT level. Second, in line with previous work,
      • Riveros-Mckay F
      • Weale M
      • Moore R
      • Selzam S
      • Krapohl E
      • Sivley RM
      • Tarran W
      • Sørensen P
      • Lachapelle A
      • Griffiths J
      • Saffari A
      • Deanfield J
      • Spencer C
      • Hippisley-Cox J
      • Hunter D
      • O'Sullivan J
      • Ashley E
      • Plagnol V
      • Donnelly P
      An integrated polygenic tool substantially enhances coronary artery disease prediction.
      ,
      • Mars N
      • Koskela JT
      • Ripatti P
      • Kiiskinen TTJ
      • Havulinna AS
      • Lindbohm JV
      • Ahola-Olli A
      • Kurki M
      • Karjalainen J
      • Palta P
      • Neale BM
      • Daly M
      • Salomaa V
      • Palotie A
      • Widén E
      • Ripatti S.
      Polygenic and clinical risk scores and their impact on age at onset and prediction of cardiometabolic diseases and common cancers.
      we observe a low (and statistically nonsignificant) correlation between PRS values and ASCVD-PCE scores in the IRT testing cohorts (ARIC: r = 0.026, 95% CI −0.018 to 0.070 [Fisher's z-method]; MESA: r = 0.002, 95% CI −0.043 to 0.047; UKB: r = 0.002, 95% CI −0.005 to 0.008). This increases our confidence that the ASCVD PRS acts largely independently of ASCVD-PCE, and strengthens the inference that PRS results should therefore transfer to the IRT level. Third, both population genetic theory and prior data indicate that individuals of African ancestries should be most affected by attenuation in PRS effect size.
      • Martin AR
      • Kanai M
      • Kamatani Y
      • Okada Y
      • Neale BM
      • Daly MJ.
      Clinical use of current polygenic risk scores may exacerbate health disparities.
      • Duncan L
      • Shen H
      • Gelaye B
      • Meijsen J
      • Ressler K
      • Feldman M
      • Peterson R
      • Domingue B.
      Analysis of polygenic risk score usage and performance in diverse human populations.
      • Wang Y
      • Guo J
      • Ni G
      • Yang J
      • Visscher PM
      • Yengo L.
      Theoretical and empirical quantification of the accuracy of polygenic scores in ancestry divergent populations.
      Thus, other non-European ancestries should be intermediate in attenuation.
      Just as ASCVD-PCE is itself an improvement over previous risk estimators,
      • Goff DC
      • Lloyd-Jones DM
      • Bennett G
      • Coady S
      • D'Agostino RB
      • Gibbons R
      • Greenland P
      • Lackland DT
      • Levy D
      • O'Donnell CJ
      • Robinson JG
      • Schwartz JS
      • Shero ST
      • Smith SC
      • Sorlie P
      • Stone NJ
      • 2013 Wilson PWF.
      ACC/AHA Guideline on the assessment of cardiovascular risk.
      ,
      • Wilson PWF
      • D'Agostino RB
      • Levy D
      • Belanger AM
      • Silbershatz H
      • Kannel WB
      Prediction of coronary heart disease using risk factor categories.
      so we conclude the addition of a PRS to ASCVD-PCE should lead to further predictive enhancement. But for clinical utility, the ASCVD-IRT tool requires gains in performance that are not only statistically significant but also clinically meaningful.
      • Khan SS
      • Cooper R
      • Greenland P.
      Do polygenic risk scores improve patient selection for prevention of coronary artery disease?.
      On this latter point, we note that large gains are seen especially in younger middle-aged men (40 to 54 year old), not only in this study (overall NRI=10.3% [95% CI 5.7 to 15.0]), but also in a previous study on individuals of European ancestry in UK Biobank, where an NRI of 15.4% (95% CI 11.6 to 19.3) was observed for coronary artery disease outcomes.
      • Riveros-Mckay F
      • Weale M
      • Moore R
      • Selzam S
      • Krapohl E
      • Sivley RM
      • Tarran W
      • Sørensen P
      • Lachapelle A
      • Griffiths J
      • Saffari A
      • Deanfield J
      • Spencer C
      • Hippisley-Cox J
      • Hunter D
      • O'Sullivan J
      • Ashley E
      • Plagnol V
      • Donnelly P
      An integrated polygenic tool substantially enhances coronary artery disease prediction.
      Furthermore, a large effect is also seen in younger middle-aged ARIC men of Black ethnicity in the current study (NRI = 8.5% [95% CI 0.4 to 16.7]), suggesting that this effect generalizes to other ethnicities and ancestries. We note that ASCVD is a more heterogenous condition than coronary artery disease, which may explain the observed drop in NRI.
      There are limitations to this study. Tools that predict future risk, such as ASCVD-PCE and ASCVD-IRT, require larger datasets for validation than diagnostic clinical tools that typically have high sensitivity, specificity, PPV, and NPV. Thus, our conclusions remain data limited. We demonstrate significant gains in performance in some ethnicities and ancestries, but generalization to other groups requires additional inferential steps. More data should be collected to further validate and optimize ASCVD-IRT, improve risk prediction, further incorporate variable genetic ancestry among individuals, and assess performance gains in different subgroups. We also note this study does not address the question of analytical validation, and additional evidence is required to demonstrate the value of specific sampling and genotyping or sequencing protocols for the accurate computation of risk scores.
      In conclusion, using data from multiple ethnicities and ancestries, we have shown improved predictive performance of the ASCVD-IRT tool over the currently established ASCVD-PCE tool in multiple cohorts, multiple ethnicities, and multiple ancestries. To our knowledge, this is the first time, for any disease, that an integrated risk tool combining a current clinical risk tool and a PRS has been successfully validated across multiple ethnicities and ancestries.

      Author Contributions

      Michael E. Weale: Writing - Original draft preparation, Methodology, Investigation, Supervision. Fernando Riveros-Mckay: Writing - Review & Editing, Methodology, Formal analysis, Software, Validation, Investigation, Visualization. Saskia Selzam, Eva Gradovich: Data Curation, Software, Resources. Priyanka Seth, Rachel Moore, Carla Giner-Delgado, Duncan Palmer, Daniel Wells, Ayden Saffari: Methodology, Software, Resources. William A. Tarran, R. Michael Sivley: Software, Resources, Validation. Alexander S. Lachapelle: Conceptualization, Project administration. Hannah Wand, Lee Shoa Long Clarke, Joshua W. Knowles, Jack W O'Sullivan, Euan A Ashley: Writing - Review & Editing. Gil McVean: Conceptualization, Writing - Review & Editing, Project administration. Vincent Plagnol, Peter Donnelly: Conceptualization, Writing - Review & Editing, Supervision, Project administration.

      Acknowledgments

      We acknowledge and thank the participants and investigators of all the datasets used in this study. Please see Supplementary Materials for detailed acknowledgements.

      Disclosures

      Peter Donnelly and Gil McVean are partners in Peptide Groove LLP. All other authors declare no competing interests.

      Appendix. Supplementary materials

      References

        • Goff DC
        • Lloyd-Jones DM
        • Bennett G
        • Coady S
        • D'Agostino RB
        • Gibbons R
        • Greenland P
        • Lackland DT
        • Levy D
        • O'Donnell CJ
        • Robinson JG
        • Schwartz JS
        • Shero ST
        • Smith SC
        • Sorlie P
        • Stone NJ
        • 2013 Wilson PWF.
        ACC/AHA Guideline on the assessment of cardiovascular risk.
        Circulation. 2014; 129: S49-S73
        • Grundy SM
        • Stone NJ
        • Bailey AL
        • Beam C
        • Birtcher KK
        • Blumenthal RS
        • Braun LT
        • Ferranti S de
        • Faiella-Tommasino J
        • Forman DE
        • Goldberg R
        • Heidenreich PA
        • Hlatky MA
        • Jones DW
        • Lloyd-Jones D
        • Lopez-Pajares N
        • Ndumele CE
        • Orringer CE
        • Peralta CA
        • Saseen JJ
        • Smith Jr, SC
        • Sperling L
        • Virani SS
        • Yeboah J.
        2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the management of blood cholesterol.
        Circulation. 2018; 139: 1082-1143
        • Arnett DK
        • Blumenthal RS
        • Albert MA
        • Buroker AB
        • Goldberger ZD
        • Hahn EJ
        • Himmelfarb CD
        • Khera A
        • Lloyd-Jones D
        • McEvoy JW
        • Michos ED
        • Miedema MD
        • Muñoz D
        • Smith SC
        • Virani SS
        • Williams KA
        • Yeboah J
        • Ziaeian B.
        2019 ACC/AHA Guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.
        J Am Coll Cardiol. 2019; 74: e177-e232
        • Khera AV
        • Kathiresan S.
        Genetics of coronary artery disease: discovery, biology and clinical translation.
        Nat Rev Genet. 2017; 18: 331-344
        • Riveros-Mckay F
        • Weale M
        • Moore R
        • Selzam S
        • Krapohl E
        • Sivley RM
        • Tarran W
        • Sørensen P
        • Lachapelle A
        • Griffiths J
        • Saffari A
        • Deanfield J
        • Spencer C
        • Hippisley-Cox J
        • Hunter D
        • O'Sullivan J
        • Ashley E
        • Plagnol V
        • Donnelly P
        An integrated polygenic tool substantially enhances coronary artery disease prediction.
        Circulation: Genomic and Precision Medicine. 2021; (in press).
        • Elliott J
        • Bodinier B
        • Bond T
        • Chadeau-Hyam M
        • Evangelou E
        • Moons K
        • Dehghan A
        • Muller DC
        • Elliott P
        • Tzoulaki I.
        Predictive accuracy of a polygenic risk score–enhanced prediction model vs a clinical risk score for coronary artery disease.
        J Am Med Assoc. 2020; 323: 636-645
        • Mosley J
        • Gupta D
        • Tan J
        • Yao J
        • Wells QS
        • Shaffer CM
        • Kundu S
        • Robinson-Cohen C
        • Psaty BM
        • Rich SS
        • Post WS
        • Guo X
        • Rotter JI
        • Roden DM
        • Gerszten RE
        • Wang TJ.
        Predictive accuracy of a polygenic risk score compared with a clinical risk score for incident coronary heart disease.
        J Am Med Assoc. 2020; 323: 627-635
        • Mars N
        • Koskela JT
        • Ripatti P
        • Kiiskinen TTJ
        • Havulinna AS
        • Lindbohm JV
        • Ahola-Olli A
        • Kurki M
        • Karjalainen J
        • Palta P
        • Neale BM
        • Daly M
        • Salomaa V
        • Palotie A
        • Widén E
        • Ripatti S.
        Polygenic and clinical risk scores and their impact on age at onset and prediction of cardiometabolic diseases and common cancers.
        Nat Med. 2020; 26: 549-557
        • Martin AR
        • Kanai M
        • Kamatani Y
        • Okada Y
        • Neale BM
        • Daly MJ.
        Clinical use of current polygenic risk scores may exacerbate health disparities.
        Nat Genet. 2019; 51: 584-591
        • Duncan L
        • Shen H
        • Gelaye B
        • Meijsen J
        • Ressler K
        • Feldman M
        • Peterson R
        • Domingue B.
        Analysis of polygenic risk score usage and performance in diverse human populations.
        Nat Commun. 2019; 103328
        • Wang Y
        • Guo J
        • Ni G
        • Yang J
        • Visscher PM
        • Yengo L.
        Theoretical and empirical quantification of the accuracy of polygenic scores in ancestry divergent populations.
        Nat Commun. 2020; 11: 3865
        • Khan A
        • McHugh C
        • Conomos MP
        • Gogarten SM
        • Nelson SC
        • Race and Genetics Discussion Group
        Guidelines on the use and reporting of race, ethnicity, and ancestry in the NHLBI Trans-Omics for Precision Medicine (TOPMed) program.
        2020 (Available at:) (Accessed December 10, 2020)
        • Race Ethnicity and Genetics Working Group
        The use of racial, ethnic, and ancestral categories in human genetics research.
        Am J Hum Genet. 2005; 77: 519-532
      1. The 1000 Genomes Project Consortium. A global reference for human genetic variation.
        Nature. 2015; 526: 68-74
        • Mathieson I
        • Scally A.
        What is ancestry?.
        PLoS Genet. 2020; 16e1008624
        • The ARIC Investigators
        The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives.
        Am J Epidemiol. 1989; 129: 687-702
        • Bild DE
        • Bluemke DA
        • Burke GL
        • Detrano R
        • Diez Roux A V.
        • Folsom AR
        • Greenland P
        • Jacobs DR
        • Kronmal R
        • Liu K
        • Nelson JC
        • O'Leary D
        • Saad MF
        • Shea S
        • Szklo M
        • Tracy RP
        Multi-ethnic study of atherosclerosis: objectives and design.
        Am J Epidemiol. 2002; 156: 871-881
        • Olson JL
        • Bild DE
        • Kronmal RA
        • Burke GL.
        Legacy of MESA.
        Glob Heart. 2016; 11: 269-274
        • Sudlow C
        • Gallacher J
        • Allen N
        • Beral V
        • Burton P
        • Danesh J
        • Downey P
        • Elliott P
        • Green J
        • Landray M
        • Liu B
        • Matthews P
        • Ong G
        • Pell J
        • Silman A
        • Young A
        • Sprosen T
        • Peakman T
        • Collins R.
        UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age.
        PLoS Med. 2015; 12e1001779
        • Bycroft C
        • Freeman C
        • Petkova D
        • Band G
        • Elliott LT
        • Sharp K
        • Motyer A
        • Vukcevic D
        • Delaneau O
        • O'Connell J
        • Cortes A
        • Welsh S
        • Young A
        • Effingham M
        • McVean G
        • Leslie S
        • Allen N
        • Donnelly P
        • Marchini J
        The UK Biobank resource with deep phenotyping and genomic data.
        Nature. 2018; 562: 203-209
        • Malik R
        • Chauhan G
        • Traylor M
        • Sargurupremraj M
        • Okada Y
        • Mishra A
        • Rutten-Jacobs L
        • Giese AK
        • Van Der Laan SW
        • Gretarsdottir S
        • Anderson CD
        • Chong M
        • Adams HHH
        • Ago T
        • Almgren P
        • Amouyel P
        • Ay H
        • Bartz TM
        • Benavente OR
        • Bevan S
        • Boncoraglio GB
        • Brown RD
        • Butterworth AS
        • Carrera C
        • Carty CL
        • Chasman DI
        • Chen WM
        • Cole JW
        • Correa A
        • Cotlarciuc I
        • Cruchaga C
        • Danesh J
        • Bakker PIW De
        • Destefano AL
        • Den Hoed M
        • Duan Q
        • Engelter ST
        • Falcone GJ
        • Gottesman RF
        • Grewal RP
        • Gudnason V
        • Gustafsson S
        • Haessler J
        • Harris TB
        • Hassan A
        • Havulinna AS
        • Heckbert SR
        • Holliday EG
        • Howard G
        • Hsu FC
        • Hyacinth HI
        • Ikram MA
        • Ingelsson E
        • Irvin MR
        • Jian X
        • Jiménez-Conde J
        • Johnson JA
        • Jukema JW
        • Kanai M
        • Keene KL
        • Kissela BM
        • Kleindorfer DO
        • Kooperberg C
        • Kubo M
        • Lange LA
        • Langefeld CD
        • Langenberg C
        • Launer LJ
        • Lee JM
        • Lemmens R
        • Leys D
        • Lewis CM
        • Lin WY
        • Lindgren AG
        • Lorentzen E
        • Magnusson PK
        • Maguire J
        • Manichaikul A
        • McArdle PF
        • Meschia JF
        • Mitchell BD
        • Mosley TH
        • Nalls MA
        • Ninomiya T
        • O'Donnell MJ
        • Psaty BM
        • Pulit SL
        • Rannikmäe K
        • Reiner AP
        • Rexrode KM
        • Rice K
        • Rich SS
        • Ridker PM
        • Rost NS
        • Rothwell PM
        • Rotter JI
        • Rundek T
        • Sacco RL
        • Sakaue S
        • Sale MM
        • Salomaa V
        • Sapkota BR
        • Schmidt R
        • Schmidt CO
        • Schminke U
        • Sharma P
        • Slowik A
        • Sudlow CLM
        • Tanislav C
        • Tatlisumak T
        • Taylor KD
        • Thijs VNS
        • Thorleifsson G
        • Thorsteinsdottir U
        • Tiedt S
        • Trompet S
        • Tzourio C
        • Van Duijn CM
        • Walters M
        • Wareham NJ
        • Wassertheil-Smoller S
        • Wilson JG
        • Wiggins KL
        • Yang Q
        • Yusuf S
        • AFGen Consortium
        • Cohorts for Heart and Aging Research in Genomic Epidemiology (CHARGE) Consortium
        • International Genomics of Blood Pressure (iGEN-BP) Consortium
        • INVENT Consortium
        • STARNET
        • Bis JC
        • Pastinen T
        • Ruusalepp A
        • Schadt EE
        • Koplev S
        • Björkegren JLM
        • Codoni V
        • Civelek M
        • Smith NL
        • Trégouët DA
        • Christophersen IE
        • Roselli C
        • Lubitz SA
        • Ellinor PT
        • Tai ES
        • Kooner JS
        • Kato N
        • He J
        • Harst P van der
        • Elliott P
        • Chambers JC
        • Takeuchi F
        • Johnson AD
        • BioBank Japan Cooperative Hospital Group
        • COMPASS Consortium
        • EPIC-CVD Consortium
        • EPIC-InterAct Consortium, International Stroke Genetics Consortium (ISGC)
        • METASTROKE Consortium
        • Neurology Working Group of the CHARGE Consortium
        • NINDS Stroke Genetics Network (SiGN)
        • UK Young Lacunar DNA Study
        • MEGASTROKE Consortium
        • Sanghera DK
        • Melander O
        • Jern C
        • Strbian D
        • Fernandez-Cadenas I, Jr WTL
        • Rolfs A
        • Hata J
        • Woo D
        • Rosand J
        • Pare G
        • Hopewell JC
        • Saleheen D
        • Stefansson K
        • Worrall BB
        • Kittner SJ
        • Seshadri S
        • Fornage M
        • Markus HS
        • Howson JMM
        • Kamatani Y
        • Debette S
        • Dichgans M
        Multiancestry genome-wide association study of 520,000 subjects identifies 32 loci associated with stroke and stroke subtypes.
        Nat Genet. 2018; 50: 524-537
        • The CARDIoGRAMplusC4D Consortium
        A comprehensive 1,000 Genomes-based genome-wide association meta-analysis of coronary artery disease.
        Nat Genet. 2015; 47: 1121-1130
        • Pencina MJ
        • D'Agostino RB
        • D'Agostino RB
        • Vasan RS
        Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.
        Stat Med. 2008; 27: 157-172
        • Harrell FE
        • Califf RM
        • Pryor DB
        • Lee KL
        • Rosati RA.
        Evaluating the yield of medical tests.
        J Am Med Assoc. 1982; 247: 2543-2546
        • Inouye M
        • Abraham G
        • Nelson CP
        • Wood AM
        • Sweeting MJ
        • Dudbridge F
        • Lai FY
        • Kaptoge S
        • Brozynska M
        • Wang T
        • Ye S
        • Webb TR
        • Rutter MK
        • Tzoulaki I
        • Patel RS
        • Loos RJF
        • Keavney B
        • Hemingway H
        • Thompson J
        • Watkins H
        • Deloukas P
        • Di Angelantonio E
        • Butterworth AS
        • Danesh J
        • Samani NJ
        Genomic risk prediction of coronary artery disease in 480,000 adults: implications for primary prevention.
        J Am Coll Cardiol. 2018; 72: 1883-1893
        • Dikilitas O
        • Schaid DJ
        • Kosel ML
        • Carroll RJ
        • Chute CG
        • Denny JA
        • Fedotov A
        • Feng Q
        • Hakonarson H
        • Jarvik GP
        • Lee MTM
        • Pacheco JA
        • Rowley R
        • Sleiman PM
        • Stein CM
        • Sturm AC
        • Wei WQ
        • Wiesner GL
        • Williams MS
        • Zhang Y
        • Manolio TA
        • Kullo IJ.
        Predictive utility of polygenic risk scores for coronary heart disease in three major racial and ethnic groups.
        Am J Hum Genet. 2020; 106: 707-716
        • Hippisley-Cox J
        • Coupland C
        • Vinogradova Y
        • Robson J
        • Minhas R
        • Sheikh A
        • Brindle P.
        Predicting cardiovascular risk in England and Wales: Prospective derivation and validation of QRISK2.
        Br Med J. 2008; 336: 1475-1482
        • National Institute for Health and Care Excellence
        Cardiovascular disease: risk assessment and reduction, including lipid modification (Clinical guideline [CG181]).
        2016 (Available at:)
        • Wilson PWF
        • D'Agostino RB
        • Levy D
        • Belanger AM
        • Silbershatz H
        • Kannel WB
        Prediction of coronary heart disease using risk factor categories.
        Circulation. 1998; 97: 1837-1847
        • Khan SS
        • Cooper R
        • Greenland P.
        Do polygenic risk scores improve patient selection for prevention of coronary artery disease?.
        J Am Med Assoc. 2020; 323: 614-615