Data Phenotype & Chart Review SOP
A. Purpose
To provide a standardized approach for conducting a chart review to ensure data accuracy, consistency in documentation to achieve identification of surrogative label (silver label) for machine learning-driven research for the Patient-Focused Collaborative Hospital Repository Uniting Standards for Equitable AI (thereafter CHoRUS) members.
B. Scope
This SOP applies to all cliniciancs and research assistants who will be involved in the chart review process.
C. Definitiions
Chart Review: The process of examining a patient's medical record for relevant infomration.
EHR (Electonic Health Record): Digital version of a patient's paper chart, which contaiins medical history, diagnoses, treatment plans, immunization dates, allergies, radiology images, and lab/test result.
D. Responsibilities
Chart Reviewer (CR): Responsible for accessing, reviewing, and accurately recording information from patient charts as per this SOP. Reviewers should have critical care work experience (actual patient care in the intensive care unit or equivalent space in hospital) and will be limited to individual with medical (M.D or equivalent) degree or nursing/PA degree (R.N. or eq) for current round.
Point Person (PP): A member of CHoRUS, designated by the individual site PI to direct the review process. He or she is responsible for the pre-review training of all chart reviewers and supervision of the quality of the results from the chart review. The PP also is responsible for the reporting spreadsheet generation and act as a point person for the institution.
E. Procedure
E.1. Numbers of charts to be reviewed: A total of 10% of all charts submitted to CHoRUS central (at least 200+ charts per center) will be reviewed for 8 different concept labels, to identify and ascertain the existence of concept in EHR.
The pilot stage will consist of reviewing 20 charts - each covered by two (2) CRs. The very first 5 charts by two CRs will not be counted towards the statistics (which leaves 15 charts to be included in the final analysis) but to be used for other internal review.
The next 15 charts after the first 5 charts will also be read by two (2) CRs. This will allow the calculation of interrater reliability (IRR) by Data Annotation Team. Interim decision will be made by the Team when and how to proceed. If IRR is within reasonable level, the rest of charts will be read by 1 CR per chart. Otherwise, the DA team will determine whether more charts need to be annotated by 2 CRs per chart.
E.2. The REDCap Survey protocol
a. The PP accesses the redcap survey page (with the application through the University of Pittsburgh REDCap system: https://redcap-std.hs.pitt.edu/redcap/surveys/?s=injPIG).
b. The PP reviews the REDCap questions and familiarize the path.
c. The PP selects CRs and educate the survey response process.
d. Each CR reviews 5 charts and annotate each of the concept.
e. PP meets with CRs to review each concept and receives feedback to ascertain standardized workflow.
f. Two CR review the next 15 charts and decide the concept presence, date and time, and the confidence of such decision (with the scale of 1 to 5, 1 being the least and 5 being the most confident). If 2 CRs have scale distance more than three (3), a third CR or PP could review the specific decision for annotation.
g. We encourage the initially assigned CRs to do the entire chart review, if possible (to minimize repetitive training and utilize learning curves), but partial review with change in CR could be possible under PP's supervision for quality assurance.
h. The PP can gather the conclusions of the survey. The contents include the answers for each element, decision on the concept, the confidence (scaled by CR) for each concept, and time spent for the entire survey (note the time spent will not be visible to the CRs).
E.3. Available CHoRUS Support
a. CHoRUS Data Phenotyping Team will hold an office hour at the start of the weekly Data Acquisition Team meeting (usually every Tuesday 2-3PM EST) for 10-15 minutes.
b. CHoRUS Data Phenotyping Team can support individual SI through a separate meeting to check each center's progression (detailed schedule TBA).
F. Reporting and Publication
The Data Phenotyping team will report to CHoRUS executive committee on its progress every 3 months. The team will remain inclusive for anyone from CHoRUS to participate with the team's work. Results and evaluation metrics, along with its downstream products will be disseminated at scientific venue or through publications. The team will abide by the authorship guideline of the CHoRUS for those activities.
G. References
HIPPA Compliance Manual
Institution's EHR System Guide
H. Workflow
