This article describes how facilities can use the ACR’s Dose Index Registry (DIR) to perform both incident identification and external benchmarking required by the Joint Commission’s (TJC) Diagnostic Imaging Element of Performance PI.02.01.01 issued August 10, 2015. PI.02.01.01 requires organizations to:
- Review and analyze incidents where the radiation dose from a diagnostic CT examinations exceeded the facility’s expected dose range they established for that specific protocol; and then
- Compare these incidents to external benchmarks.
Incident identification’s basic purpose is to identify individually aberrant cases where the dose is so high it merits further investigation. TJC requires facilities to establish “expected dose ranges” for each protocol used at their facility and then compare actual doses imparted at the end of each study against the expected dose range. TJC categorizes any exam whose total dose exceeded the expected dose as an “Incident” and requires the facility to review and analyze them, and compare them against an external benchmark. The joint commission does not proscribe what each dose threshold should be – they leave it up to each facility to determine. The American Association of Physicists in Medicine (AAPM) published dose ranges for a subset of protocols, but not for all of them.
There are several third-party applications that will help facilities establish dose ranges for each protocol and provide an external benchmark. However, many of them are expensive to purchase and/or carry significant on-going costs. In their defense, these third-party applications provide easy to use interfaces and perform other functions. However, not all facilities may be able to afford the system or need the other capabilities.
What options are available for those who find many of the third-party solutions prohibitively expensive? Answer: The American College of Radiology’s Dose Index Registry (DIR). By participating in the ACR’s Dose Index Registry facilities can meet TJC requirement to create protocol specific dose ranges act as an external benchmarks in a very cost effective manner.
The DIR is an inexpensive yet powerful tool which provides regular reports to the enrolling facility comparing the facility’s aggregated doses by exam category to doses reported by other registry participants. Participants receive an Executive Summary as well as more detailed exam level reports including Box and Whisker Plots and dose trends over time. Participants also receive excel spreadsheets showing how their facilities compare to the 25th, median, and 75th percentiles. These spreadsheets are color coded to indicate if the facilities doses are above the 75th percentiles or below the 25th percentiles, making it easy to discern where potential problem areas lie. Together, these reports allow facilities to identify general areas for improvement. With proper documenting and review this satisfies the TJC’s external benchmarking requirement. The Joint Commission recently issued a written response to an inquiry acknowledging the DIR meets the benchmarking requirements under PI.02.01.01. This response can be found on the ACR’s DIR website.
Using DIR Data to Satisfy TJC’s Incident Review Requirement
TJC also requires facilities to identify incidents when the exam dose exceeded an expected dose range for that protocol. Externally derived benchmarks aren’t necessarily the best “source” for facilities to use to develop their expected ranges (using others’ data risks over- or under-identifying such incidents.)
Fortunately, facilities can use their DIR data to both establish an expected dose range for each protocol, and then identify individual studies that exceed it. While beneficial to all sites, it may be particularly beneficial to non-XR-29 compliant sites that are unable to use the Dose Alert and Dose Notification features. The question non-XR-29 compliant Joint Commission sites must ask themselves is “How will I develop expected dose ranges and how will I compare actual doses to the expected dose?” Under the recently issued rules, if you are a TJC-accredited facility you must have the capability.
Establishing Facility Thresholds
The requirement that each facility establishes a process to review and analyze incidents where the radiation dose for any particular protocol exceeds expected dose index ranges is probably one of the most difficult of the new TJC requirements. How does a facility establish meaningful “expected dose index ranges” for each protocol so it can identify when an “incident” occurred?
Fortunately, the DIR allows facilities to export their data for further analysis – analysis particularly well suited to the establishment of a specific dose threshold for protocol. Using the DIR’s export function, facilities can export their dose information by exam over a range of dates to Excel. The results include the Study Instance ID, scanner Study name, Institution, Scanner name, Scanner model, Total CTDIvol and Total DLP, and Study Date. From this information it is possible to calculate a mean dose and its standard deviation for each study.
As mentioned earlier, the purpose for identifying “incidents” is to identify those individually aberrant cases where the dose is so unusually high that it merits further investigation. The challenge then becomes how to identify those aberrancies. Fortunately, we can apply the statistical theory underpinning the Normal Distribution to help us identify such incidents.
In a Normal Distribution most samples occur about the mean then taper off in both directions, ending with a tiny percentage of samples at the “tails.” For our purposes, the “incidents” we are interested in are represented by those studies at the right end of the Normal Curve which, in the accompanying chart, is represented by the area to the right of the Dose Threshold line. Studies that fall to the right of this line are the true aberrancies, or outliers, and would be considered to have exceeded a facility’s expected dose index range. In TJC terms, these are “dose incidents.”
The Dose Threshold line, as shown in the accompanying Normal Curve graph, is be determined by calculating am an upper dose threshold – a “tail” – for each exam type. Generally speaking, most facilities will want to establish their Dose Threshold somewhere between the 2nd or 3rd standard deviation from the mean. Using this method, studies whose doses are more than 2 or 2 standard deviations from the mean would be considered an outlier, or an “Incident” in TJC’s guidelines, and subject to review and analysis. This calculation needs to be made for each protocol in use.
Once thresholds are established each facility is required to compare each study’s total dose against its threshold limits. There are two likely methods for incident identification using DIR data. A facility could either:
- Calculate Dose Thresholds for each exam and then provide a table of thresholds to the staff and have them manually check each end dose against the table of threshold doses and log it; or,
- Use a database query designed to use the exported DIR data to automatically calculate dose thresholds for each exam and identify only those studies which exceed it.
Option 1 above would be a largely manual process with 98%-99% or more of the recording keeping done for studies that are of no concern (i.e., those left of the Dose Threshold line). Option 2 would be preferable as it only requires staff involvement for the 1-2% of cases that are true “Incidents” with no work involved for the rest. As with external benchmarking, with proper documenting and review, both Options 1 and 2 should satisfy the TJC’s incident identification, review, and analysis requirement. In either case, when an incident is found to have occurred, the dose is compared to the ACR’s DIR results as well as reviewed according to whatever policy the facility and its physicist set forth.
The biggest challenge most facilities will have using the DIR to meet the two TJC requirements described above is identifying someone with the time and skills to perform the steps and analysis described. Generally speaking, it will take someone with moderate to advanced Microsoft Access© and Excel© skills.
Cost to enroll in the DIR
The costs to enroll in the DIR are nominal: $500 to enroll with annual costs determined by the number of radiologists and/or sites, e.g., $500 for 1-5 sites with 1-5 radiologists; $1,000 for 1-5 sites with 16-25 radiologists; $1,000 for 6-15 sites with 1-5 radiologists, etc. Click here for more information on costs for enrolling in the DIR. The software is free, but facilities will need to set up a small server to use with the software. If a facility has already enrolled in the ACR’s Lung Cancer Screening Registry or other ACR Registry some or all of these fees may have already been paid.
The DIR is a viable and cost effective option for facilities to consider as they evaluate how to meet two of the most problematic TJC’s requirements – the requirement to identify and analyze dose incidents and compare them to an external benchmark. With moderate to advanced Microsoft Access© and Excel© skills facilities can meet TJC’s new requirements without significant time or expense. Even if the facility must contract out for the protocol development and analysis (through their physicist or other independent contractor) the cost to implement this solution is almost certain to be far less than many of the third-party options available.
Michael Bohl, MHA
Dose Registry Support Services