Title
Measuring agreement between decision support reminders: the cloud vs. the local expert.
Abstract
A cloud-based clinical decision support system (CDSS) was implemented to remotely provide evidence-based guideline reminders in support of preventative health. Following implementation, we measured the agreement between preventive care reminders generated by an existing, local CDSS and the new, cloud-based CDSS operating on the same patient visit data.Electronic health record data for the same set of patients seen in primary care were sent to both the cloud-based web service and local CDSS. The clinical reminders returned by both services were captured for analysis. Cohen's Kappa coefficient was calculated to compare the two sets of reminders. Kappa statistics were further adjusted for prevalence and bias due to the potential effects of bias in the CDS logic and prevalence in the relative small sample of patients.The cloud-based CDSS generated 965 clinical reminders for 405 patient visits over 3 months. The local CDSS returned 889 reminders for the same patient visit data. When adjusted for prevalence and bias, observed agreement varied by reminder from 0.33 (95% CI 0.24 - 0.42) to 0.99 (95% CI 0.97 - 1.00) and demonstrated almost perfect agreement for 7 of the 11 reminders.Preventive care reminders delivered by two disparate CDS systems show substantial agreement. Subtle differences in rule logic and terminology mapping appear to account for much of the discordance. Cloud-based CDSS therefore show promise, opening the door for future development and implementation in support of health care providers with limited resources for knowledge management of complex logic and rules.
Year
DOI
Venue
2014
10.1186/1472-6947-14-31
BMC Med. Inf. & Decision Making
Keywords
Field
DocType
decision support systems,health informatics
Data mining,Decision support system,Patient Visit,Knowledge management,Health services research,Clinical decision support system,Guideline,Health informatics,Medicine,Cloud computing,The Internet
Journal
Volume
Issue
ISSN
14
1
1472-6947
Citations 
PageRank 
References 
2
0.37
13
Authors
5
Name
Order
Citations
PageRank
Brian E. Dixon110024.46
Linas Simonaitis212912.24
Susan M. Perkins3173.61
Adam Wright420.37
Blackford Middleton5916112.88