External Validation of a Periodontal Prediction Model for Identification of Diabetes among Saudi Adults
Arwa A Talakey, Francis Hughes, Eduardo Bernabé
Citation Information :
Talakey AA, Hughes F, Bernabé E. External Validation of a Periodontal Prediction Model for Identification of Diabetes among Saudi Adults. J Contemp Dent Pract 2020; 21 (10):1176-1181.
Aim and objective: To externally validate the performance of a novel periodontal prediction model (PPM) for identification of diabetes among Saudi adults. Materials and methods: The study was carried out among 150 adults attending primary care clinics in Riyadh (Saudi Arabia). The study adopted a temporal external validation approach, where the performance of the PPM was evaluated in the same location as the development study, but at a later time to allow for some variation between samples. A case-control approach was adopted, where diabetes status was first ascertained, followed by the completion of the Finnish Diabetes Risk Score (FINDRISC), Canadian Diabetes Risk (CANRISK) tools, and periodontal examinations. Results: The area under the curve (AUC) of the PPM (based on the number of missing teeth, the proportion of sites with pocket probing depth ≥6 mm, and mean pocket probing depth) was 0.514 (95% CI: 0.385, 0.642). The FINDRISC and CANRISK tools had AUC values of 0.871 (95% CI: 0.811–0.931) and 0.927 (95% CI: 0.884–0.971), respectively. The addition of the PPM did not improve the AUC of FINDRISC (p = 0.479) or CANRISK (p = 0.920). The decision curve analysis showed that there was no clinical benefit in adding the PPM to either tool. The PPM was updated with an overall adjustment factor for all existing predictors and three more periodontal measures. Conclusion: In an external sample, the PPM had poor performance for identification of diabetes and no added value when combined with FINDRISC and CANRISK. The performance of the PPM improved after recalibration and extension. Clinical significance: The results underscore the value of externally validating prediction models before applying them in clinical dental practice.
Borrell LN, Kunzel C, Lamster I, et al. Diabetes in the dental office: using NHANES III to estimate the probability of undiagnosed disease. J Periodontal Res 2007;42(6):559–565. DOI: 10.1111/j.1600-0765.2007.00983.x.
Lalla E, Kunzel C, Burkett S, et al. Identification of unrecognized diabetes and pre-diabetes in a dental setting. J Dent Res 2011;90(7):855–860. DOI: 10.1177/0022034511407069.
Lalla E, Cheng B, Kunzel C, et al. Dental findings and identification of undiagnosed hyperglycemia. J Dent Res 2013;92(10):888–892. DOI: 10.1177/0022034513502791.
Talakey A, Hughes F, Almoharib H, et al. The added value of periodontal measurements for identification of diabetes among Saudi adults. J Periodontol 2020. DOI: 10.1002/JPER.20-0118.
Debray TP, Vergouwe Y, Koffijberg H, et al. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 2015;68(3):279–289. DOI: 10.1016/j.jclinepi.2014.06.018.
Collins GS, de Groot JA, Dutton S, et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol 2014;14(1):40. DOI: 10.1186/1471-2288-14-40.
Wynants L, Collins GS, Van Calster B. Key steps and common pitfalls in developing and validating risk models. BJOG 2017;124(3):423–432. DOI: 10.1111/1471-0528.14170.
Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. BMC Med 2015;13(1):1. DOI: 10.1186/s12916-014-0241-z.
Moons KGM, Kengne AP, Grobbee DE, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012;98(9):691–698. DOI: 10.1136/heartjnl-2011-301247.
Siontis GC, Tzoulaki I, Castaldi PJ, et al. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J Clin Epidemiol 2015;68(1):25–34. DOI: 10.1016/j.jclinepi.2014.09.007.
Abbasi A, Peelen LM, Corpeleijn E, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ 2012;345(sep18 2):e5900. DOI: 10.1136/bmj.e5900.
Cohen JF, Korevaar DA, Altman DG, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open 2016;6(11):e012799. DOI: 10.1136/bmjopen-2016-012799.
Moons KGM, Altman DG, Reitsma JB, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162(1):W1–W73. DOI: 10.7326/M14-0698.
Linnet K, Bossuyt PM, Moons KG, et al. Quantifying the accuracy of a diagnostic test or marker. Clin Chem 2012;58(9):1292–1301. DOI: 10.1373/clinchem.2012.182543.
American Diabetes Association. 2. Classification and diagnosis of diabetes: Standards of medical care in diabetes—2018. Diabetes Care 2018;41(Suppl. 1):S13–S27. DOI: 10.2337/dc18-S002.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143(1):29–36. DOI: 10.1148/radiology.143.1.7063747.
Eke PI, Dye BA, Wei L, et al. Self-reported measures for surveillance of periodontitis. J Dent Res 2013;92(11):1041–1047. DOI: 10.1177/0022034513505621.
Lindström J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 2003;26(3):725–731. DOI: 10.2337/diacare.26.3.725.
Kaczorowski J, Robinson C, Nerenberg K. Development of the CANRISK questionnaire to screen for prediabetes and undiagnosed type 2 diabetes. Canadian J Diabetes 2009;33(4):381–385. DOI: 10.1016/S1499-2671(09)34008-3.
Newman MG, Takei H, Klokkevold PR, et al. Carranza's clinical periodontology-e-book: Expert consult: Online. Elsevier Health Sciences; 2014.
Beltrán-Aguilar ED, Eke PI, Thornton-Evans G, et al. Recording and surveillance systems for periodontal diseases. Periodontol 2000 2012;60(1):40–53. DOI: 10.1111/j.1600-0757.2012.00446.x.
Control CfD, Prevention. National Health and Nutrition Examination Survey (NHANES). 2013–14. Retrieved August. 2016.
Holtfreter B, Albandar JM, Dietrich T, et al. Standards for reporting chronic periodontitis prevalence and severity in epidemiologic studies: Proposed standards from the joint EU/USA periodontal epidemiology working group. J Clin Periodontol 2015;42(5):407–412. DOI: 10.1111/jcpe.12392.
Page RC, Eke PI. Case definitions for use in population-based surveillance of periodontitis. J Periodontol 2007;78(Suppl 7S): 1387–1399. DOI: 10.1902/jop.2007.060264.
Tonetti MS, Greenwell H, Kornman KS. Staging and grading of periodontitis: framework and proposal of a new classification and case definition. J Clin Periodontol 2018;45(Suppl 20):S149–S161. DOI: 10.1111/jcpe.12945.
Moons KG, de Groot JA, Linnet K, et al. Quantifying the added value of a diagnostic test or marker. Clin Chem 2012;58(10):1408–1417. DOI: 10.1373/clinchem.2012.182550.
Pencina MJ, D'Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27(2):157–172. DOI: 10.1002/sim.2929.
Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J 2014;35(29):1925–1931. DOI: 10.1093/eurheartj/ehu207.
Steyerberg EW. Clinical prediction models. Springer International Publishing; 2019.
Kyrou I, Tsigos C, Mavrogianni C, et al. Sociodemographic and lifestyle-related risk factors for identifying vulnerable groups for type 2 diabetes: a narrative review with emphasis on data from Europe. BMC Endocr Disord 2020;20(1):1–13. DOI: 10.1186/s12902-019- 0463-3.
Altman DG, Vergouwe Y, Royston P, et al. Prognosis and prognostic research: validating a prognostic model. BMJ 2009;338(may28 1):b605. DOI: 10.1136/bmj.b605.
