OBJECTIVE: To compare documentation, referral practices, and appropriateness of telephone advice provided by RNs using computerized telephone triage protocols with mat of pediatric residents not employing such support. METHODS: We randomly selected 100 Primary Care Clinic resident calls and 100 Pediatric Health Information Line RN calls for comparison. We noted documentation of: patient's name and age, phone number, date and time of call, chief complaint, history, disposition, and person giving advice. Dispositions of the calls were divided into 4 categories: see immediately, office visit within 24 hours, non-urgent office visit, and home management Three faculty pediatricians rated each call using a 3-item questionnaire assessing documentation of history, completeness of documented advice, and appropriateness of medical advice. RESULTS: The two groups were similar in patient age and chief complaint RNs recorded the patient's phone number more frequently man residents (87% vs 66% p<0.01). No other differences in recording pertinent patient information were noted. Residents advised home management more often than RNs (62% vs 39% p<0.01), and RNs more often recommended an office visit within 24 hours (27% vs 15% p=0.05). Inter-rater agreement among the physician evaluators was low (kappa=.15-.23). In their assessment of appropriateness of medical advice, evaluators agreed with one another more often regarding RN calls than resident calls(51% vs 23%, 95% CI of the difference 15-42%). When all 3 evaluators were in agreement, the call was usually judged to have been managed appropriately. CONCLUSIONS: In our sample, residents performed as well as RNs in recording patient information with the exception of phone numbers. Residents recommended home management more often than RNs. The low level of inter-rater agreement in our sample illustrates the difficulty of evaluating telephone encounters based on written documentation alone. This information will be important in the development of strategies for resident education and quality assurance of telephone triage systems.
