Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
MOH - TCE 2002 5/1/02 9:21 AM Page i Toward Clinical Excellence An Introduction to Clinical Audit, Peer Review and Other Clinical Practice Improvement Activities MOH - TCE 2002 5/1/02 9:21 AM Page ii Published in April 2002 by the Ministry of Health PO Box 5013, Wellington, New Zealand ISBN 0-478-27039-9 (Book) ISBN 0-478-27040-2 (Internet) HP 3513 This document is available on the Ministry of Health’s website: http://www.moh.govt.nz MOH - TCE 2002 5/1/02 9:21 AM Page iii Foreword Toward Clinical Excellence is a series of publications developed to assist practitioners and managers as they work to improve clinical quality in the New Zealand public health system. This publication, the third in this series, provides a handbook for practitioners developing expertise in peer review and clinical audit as part of their service quality plan. The systematic critical review of the quality of clinical practice by a multidisciplinary team is the key to improving outcomes for service users. Clinical audit, peer review and evidence based practice methodologies provide practitioners with knowledge to better understand the nature of their clinical practice. The most important factors driving the success of the move towards improved clinical quality, however, reside within District Health Boards themselves. Key success factors within their organisational culture include a focus on improving outcomes for service users, a commitment to learning organisation principles and a system of identifying and rewarding clinical quality improvement ‘champions’ to provide clinical leadership at both organisational and service levels. iii Foreword At a policy level there is strong support for this development from the Minister of Health. The New Zealand Health Strategy (King 2000), along with the recently introduced Health and Disability Sector (Safety) Act 2001 and health professionals competency assurance legislation, provides clear direction toward quality improvement in health care. In addition, the new Clinical Services Directorate in the Ministry of Health has assembled key staff to provide more effective support and co-ordination for national quality improvement projects. MOH - TCE 2002 5/1/02 9:21 AM Page iv The task of implementing activities for clinical quality improvement provides a number of challenges for health professionals, not least of which is prioritising the competing demands on available time and resources. Achieving a clinical environment in which all health professionals understand the concepts and tools required to be effective in their clinical audit activities, however, is well worth this investment. As a result, we can look forward to improved outcomes and satisfaction for service users and the maintenance of professional integrity at both practitioner and service levels. Toward Clinical Excellence Dr Colin Feek Deputy Director-General Clinical Services Directorate iv MOH - TCE 2002 5/1/02 9:21 AM Page v Contents Foreword iii Using This Handbook viii Glossary of Key Terms ix Section One: Introduction 1 Using a systems approach to clinical practice improvement The clinical quality improvement ‘team’ Overview of this handbook Section Two: Clinical Audit Section Three: Peer Review Introduction Peer review meetings Self-evaluation Section Four: Morbidity and Mortality Review 7 7 9 11 12 16 27 29 31 33 35 37 40 41 41 43 44 45 Introduction 45 Objectives 45 Key points 46 v Contents Introduction The audit cycle A practical guide for clinical audit Step 1: Select an audit topic Step 2: Plan the audit Step 3: Test the audit methodology and tool Step 4: Conduct the audit Step 5: Analyse the data Step 6: Review Step 7: Develop solutions Step 8: Implement solutions Step 9: Re-audit 3 4 5 MOH - TCE 2002 5/1/02 9:21 AM Page vi Section Five: Clinical Practice Guidelines 47 Introduction Should new or existing guidelines be used? Guideline implementation Ongoing guideline evaluation 47 47 50 50 Section Six: Clinical Pathways Introduction Benefits of clinical pathway use Quality improvement and risk management potential Variance Evaluation criteria Section Seven: Clinical Indicators Introduction Four features of indicators Uses of clinical indicators Method for selecting effective indicators 51 51 53 54 54 54 55 55 58 58 59 Appendices Toward Clinical Excellence Appendix A: Members of the Working Group 61 Appendix B: Similarities and Differences between Audit and Research 62 Appendix C: Online Resources for Clinical Audit 64 Appendix D: National Health Information Datasets held by the New Zealand Health Information Service 66 Appendix E: Determining Levels of Evidence 72 Appendix F: Uses of Retrospective, Prospective and Concurrent Audit 75 Appendix G: Balanced Scorecard Approach to Performance Management 76 Appendix H: Directory of New Zealand Health Professional Groups 78 Appendix I: Sources of Seed Guidelines and International Best Practice Websites 82 References vi 84 MOH - TCE 2002 5/1/02 9:21 AM Page vii List of Figures Figure 1: Clinical data as the basis for appropriate health service decision-making Figure 2: Relationship between clinical practice improvement and evidence based practice Figure 3: Classical model for quality improvement Figure 4: Percentage of patients receiving an asthma action plan by shift Figure 5: Percentage of patients receiving an asthma action plan by day of the week 2 3 9 32 32 List of Tables 41 48 63 64 65 69 75 vii Contents Table 1: Purposes and benefits of peer review Table 2: A comparison between developing new guidelines and using existing guidelines Table B1:Differences between research and audit Table C1:New Zealand audit resources on the Internet Table C2:Overseas audit resources on the Internet Table D1:Standard set of indicators Table F1: A comparison of retrospective, prospective and concurrent audits MOH - TCE 2002 5/1/02 9:21 AM Page viii Using This Handbook Toward Clinical Excellence: An introduction to clinical audit, peer review and other clinical practice improvement activities is the work of a group of health professionals listed in Appendix A. Although other quality tools and processes are introduced, its purpose is to provide a simple explanation of clinical audit and peer review to encourage health professionals to develop these skills to achieve ongoing improvement of clinical care. It is a handbook, not a textbook. That is, it offers a practical, ‘how to’ guide for those with little experience of peer review or clinical audit, and identifies resources that offer more in-depth information. Toward Clinical Excellence The focus of this publication is on practitioners in a multidisciplinary team. Hence the generic terms practitioner and health professional are used. Similarly health service is used to denote a health and disability support service delivery entity, department or practice. In general, service user or consumer are used to describe the people for whom health services are provided; in some instances patient is used. We expect that, over time, publications such as this will change in content and detail as we become more accustomed to evidence based practice. The Ministry of Health is developing a website that will serve as an adjunct to this document, supplying updated information on clinical practice improvement. In the meantime, we would appreciate your feedback on the usefulness of this handbook. Comments and suggestions should be addressed to: Gillian Bohm Principal Advisor, Quality Improvement and Audit Ministry of Health PO Box 5013 WELLINGTON viii MOH - TCE 2002 5/1/02 9:21 AM Page ix Glossary of Key Terms Some key terms related to clinical practice improvement activities have a number of meanings. In this handbook, they are used based on the following definitions. Clinical audit The systematic peer evaluation of an aspect of patient care. The process, which may be multidisciplinary, involves a cycle of continuous improvement of care based on explicit and measurable indicators of quality. These indicators include a service user perspective. Clinical pathway A ‘road map’ outlining a course of care provided to a patient. It is a combination of clinical practices that result in the most effective, resource-efficient, appropriate treatment for a specific condition, procedure or symptom. Clinical pathways are a ‘point of service’ tool used to disseminate and implement clinical guidelines. ix Glossary of Key Terms Clinical indicator An evaluative criterion providing an objective measure of either process or outcome in qualitative terms that is comparable across similar services. Indicators are generally reported in percentages or ratios. Although primarily for clinical use, indicators are intended to provide useful information to other people in addition to clinicians. For example, they can enable consumers to make informed choices about treatment options, tell providers how they are functioning in comparison with others and enable funders to make appropriate policy and funding decisions. MOH - TCE 2002 5/1/02 9:21 AM Page x Toward Clinical Excellence Clinical practice The evidence based process of improving clinical improvement practice, using tools such as clinical pathways, (CPI) outcome and performance indicators, clinical measurement and review in a continuous quality improvement cycle supported by appropriate information systems. Criteria The measurable key components of a standard. Criteria specify what is to be measured in a clinical audit, such as the appropriateness of specific health care decisions, the effectiveness of specific processes of care, or the acceptability of specific outcomes (National Centre for Clinical Audit 1997b). Evidence based practice (EBP) An approach that incorporates best available evidence, based on scientific research, into the clinical decision-making process using tools such as clinical practice guidelines, peer reviewed clinical research and direct clinical measurement (adapted from Royal Australasian College of Physicians website www.racp.edu.au). Guideline Has a very specific meaning in modern medical language. A clinical guideline provides an evidence based summary of the benefits, risks and contraindications for investigation, treatment and ongoing management of a particular condition or disease. At a service delivery level a guideline is used as a tool to close the gap between how we currently practise (and the outcomes associated with current practice) and other alternative practices (and the outcomes associated with those practices). It informs decisions for individual patients and for organisations by making clear the benefits, harm and costs of different treatment options. x MOH - TCE 2002 5/1/02 9:21 AM Page xi Morbidity and A form of peer review that involves the mortality review multidisciplinary team and focuses on processes or systems of care within a service that led to less than ideal patient outcomes. An evaluation of the performance of individuals or groups of practitioners by members of the same profession whose status is similar to the status of those delivering the care. It may be formal or informal and can include any occasion in which practitioners are in learning situations with other colleagues. Peer review should also be used in the context of multidisciplinary teams to incorporate feedback from ‘peers’ of other health professions who are members of the team. Standard A measurable statement about performance describing the quality of care to be achieved based on the best available evidence. It may describe (a) minimum performance or results (b) excellent performance or results or (c) a range of acceptable performances or results (Grimshaw and Russell 1993). xi Glossary of Key Terms Peer review MOH - TCE 2002 5/1/02 9:21 AM Page xii MOH - TCE 2002 5/1/02 9:21 AM Page 1 Section One: Introduction The central concern for health professionals and provider organisations is to ensure that clinical care is safe, effective and efficient, as well as appropriate to the needs of a particular service user. The challenge for health professionals is not just to identify inadequate care, but to make changes that improve clinical practice and health care service delivery. Meeting this challenge is primarily a health service responsibility involving a range of health professionals represented in the team who contribute to patient outcomes. There is also an individual professional responsibility to ensure ongoing competence within an identified scope of practice. Introduction Concern with improving clinical care is not new. Codes of practice in medicine date back thousands of years – perhaps the best known being the Hippocratic Oath. As a nurse in the Crimea, Florence Nightingale used clinical audit in the mid 19th century. Similarly, Abraham Flexner’s documentation of the quality of surgery in the United States in the early 20th century led to major reforms and contributed to the development of the American College of Surgeons accrediting process in 1913 (Lembeke 1967). In essence, health care providers today need to answer three questions: 1 What are we trying to accomplish? 2 What changes can we make that will result in an improvement? 3 How will we know a change is an improvement? (Institute for Health Care Improvement, www.ihi.com). 1 MOH - TCE 2002 5/1/02 9:21 AM Page 2 To answer these questions we need clinical data that are both valid and reliable, enabling appropriate decision-making with both clinical and economic benefits. This process of improvement based on sound data is demonstrated in Figure 1. Figure 1: Clinical data as the basis for appropriate health service decision-making Improved outcomes for patients (clinical benefits) Appropriate decision-making Toward Clinical Excellence Knowledge Information Clinical Data 2 Improved use of resources (economic benefits) MOH - TCE 2002 5/1/02 9:21 AM Page 3 Using a systems approach to clinical practice improvement The key to improving clinical care is to take a systematic approach. Each service should have a quality plan that sets out the activities to be undertaken, documents the processes to be used and identifies individual accountabilities. In developing your service quality plan, use the skills of those people in your organisation responsible for quality improvement systems so that you can be sure it is consistent with the organisational plan. Start simple – complexity is not necessarily better. The first step is to systematise the quality activities the service does now and identify areas for development. Figure 2 demonstrates how some of the pieces of the ‘jigsaw’ for clinical quality improvement – combining tools of clinical practice improvement (CPI) with those of evidence based practice (EBP) – might fit together in a health service. Introduction Figure 2: Relationship between clinical practice improvement and evidence based practice CPI/EBP Practice CPI Tools Small scale strategic research Clinical pathways Outcome and performance indicators Clinical audit and peer review Community participation in health care planning and evaluation EBP Tools Plan Act Information Systems Study Do Systematic appraisal of the best available evidence Clinical Practice Guidelines Peer reviewed clinical research Cochrane Collaboration Community values and expectations Source: Adapted from Berwick 1994 3 MOH - TCE 2002 5/1/02 9:21 AM Page 4 The clinical quality improvement ‘team’ The focus of this document is on the practitioner and the multidisciplinary team engaged in clinical practice improvement activities. There are other important parties in these activities whose roles need to be clarified at the outset. These include service users, management and professional bodies. Role of service users Toward Clinical Excellence One of the seven principles of the New Zealand Health Strategy is to have ’active involvement by consumers and communities at all levels’ (King 2000). The role of the service user therefore needs to be explicit. Clinical quality improvement activities are enhanced by input from service users. This input may be broad, such as the perspective of an independent consumer or an ‘expert’ service user in service planning and evaluation, or more specific, providing information about an individual’s experience of a particular episode of care. Role of management The role of management in clinical quality improvement is critical. A quality improvement focus and the involvement of consumers in service evaluation are strategies that need to be supported and suitably resourced by the Board and management team. This requires an organisational climate in which improving outcomes for the patient is the central purpose, practitioners are valued as partners, and resources appropriately prioritised. 4 MOH - TCE 2002 5/1/02 9:21 AM Page 5 Role of professional organisations, colleges and specialist societies Practitioners in a range of disciplines in health are increasingly being required by their professional bodies to engage in continuing education and activities to maintain professional standards as a requirement for ongoing registration. These activities include clinical audit and peer review. In an environment of increasingly complex health care and associated technology and the subsequent development of sub specialisation and advanced practice, it makes good sense to develop a closer relationship between the practitioner, the organisation and professional bodies. This relationship has the potential to focus on key clinical quality improvement issues, reduce ambiguity for the practitioner and enable consumers to contribute at both a professional and organisational level. Overview of this handbook Introduction The next section constitutes the bulk of this publication, providing a step by step approach to clinical audit. Sections Three to Seven look at the tools and processes that are used in clinical audit, peer review and other clinical practice improvement activities. For a guide to the terms used throughout this document, see the Glossary of Key Terms. A considerable amount of material is contained in the appendices. In addition, the Ministry of Health is developing a webpage (www.moh.govt.nz/quality) that will serve as an adjunct to this document, supplying updated information on clinical practice improvement. 5 MOH - TCE 2002 5/1/02 9:21 AM Page 6 MOH - TCE 2002 5/1/02 9:21 AM Page 7 Section Two: Clinical Audit Introduction The principle of all clinical audit activity is that it leads to improvements in clinical practice, resulting in improved outcomes for patients. It allows for the systematic, critical review of the quality of clinical practice by a multidisciplinary team. It includes the procedures used for diagnosis, treatment and care of patients, the associated use of resources and the effect of care on the outcome and quality of life for the patient. Audit compares actual practice to a standard of practice. As a result of this comparison, any deficiencies in actual practice may be identified and rectified. Similarities and differences between audit and research For further discussion about the similarities and differences between audit and research, see Appendix B. 7 Clinical Audit Clinical audit is sometimes confused with research because the two activities have many similarities, but they also differ in some significant ways. Audit has been described as determining whether current knowledge, skills and resources are being properly used (www.ubht.org.uk). In contrast, research is concerned with generating new knowledge that will have general application, for example determining whether a new treatment is superior to an existing one. Put another way, ‘research discovers the right thing to do; audit ensures it is done right’. MOH - TCE 2002 5/1/02 9:21 AM Page 8 Development of clinical audit The last 20 years have seen considerable development and increasing use of clinical audit. This development has been significantly influenced by the requirement that health professionals participate in audit by their colleges and professional bodies, their employing organisations, and purchasers and funders of health care. Clinical audit can make a powerful contribution to clinical quality improvement. From a professional development perspective, audit may expose health professionals to new information and knowledge, while service based audit involving the multidisciplinary team assists in ‘breaking down barriers’ among professional groups. The practice of health care often lags behind the science. Audit activities initiated, developed and conducted by individual practitioners or clinical teams can therefore be instrumental in changing practice, adjusting resource allocation and improving standards of patient care. Toward Clinical Excellence Organisational requirements for effective clinical audit The following are examples of criteria that will determine to a large extent the ability of health professionals within a service to develop effective clinical audit processes: • board and management ‘champions’ for ongoing clinical quality improvement • a professional ‘champion’ for clinical audit activities • knowledge of and commitment to clinical practice improvement among clinicians • appropriately resourced service quality plans that are developed from DHB annual plans • dedicated personnel – health professionals whose primary role is to facilitate audit activities • appropriate support such as availability of meeting rooms and secretarial support, information systems and technical support. 8 MOH - TCE 2002 5/1/02 9:21 AM Page 9 The audit cycle The essential features of audit are embodied in the audit cycle. The classical model of quality improvement illustrated in Figure 3 has two parts: • three key questions, which can be addressed in any order • the Plan-Do-Study-Act (PDSA) Cycle to test and implement changes in real work settings. Figure 3: Classical model for quality improvement What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? Clinical Audit Plan Act Do Study Source: Adapted from IHI Quality Improvement Resources 9 MOH - TCE 2002 5/1/02 9:22 AM Page 10 As well as dealing with the key questions in this model, more detailed questions such as the following will need to be answered in planning an audit project. • What is the clinical outcome that you are aiming to improve? • How can you review this outcome in a systematic way? • How is confidentiality to be maintained for the patients whose provision of care is being reviewed and for the clinicians delivering the care? • Are all of the professionals whose care is being audited members of the planning team? • How will the information be collected? • What standards are available, or need to be developed, against which to audit? • At what stage will implementation be possible? Toward Clinical Excellence • How should changes be introduced so that implementation is effective? • When will you review the implemented changes to determine whether the outcome has improved? 10 MOH - TCE 2002 5/1/02 9:22 AM Page 11 A practical guide for clinical audit Clinical audit is a process rather than a static event. This process involves a cycle of continuous improvement of care against evidence based standards. A number of different versions of this ‘audit cycle’ have been suggested; the information in this handbook is based on a nine-step cycle: 1 Select an audit topic. 2 Plan the audit. 3 Test the audit methodology and tool. 4 Conduct the audit. 5 Analyse the data. 6 Review the results and rank problems that are identified. 7 Develop solutions. 8 Implement solutions. 9 Re-audit. Throughout this section resources are referred to which provide further detail on a range of aspects of clinical audit; additional resources are contained in Appendix C. In addition, Appendix D details services provided by the New Zealand Health Information Service (NZHIS), a group within the Ministry of Health responsible for the collection and dissemination of health-related information. Audit data collected at local department, ward or service level, particularly if they relate to an identifiable cohort of patients, can be compared with data from other sources that have been collected, validated and collated by the NZHIS. 11 Clinical Audit To demonstrate certain aspects of the audit cycle, examples are given from medical, physiotherapy and nursing perspectives. Although each example may have a particular professional focus, there are common elements that could form the foundation of a multidisciplinary team audit. MOH - TCE 2002 5/1/02 9:22 AM Page 12 Step 1: Select an audit topic 1 2 Pl an • Who should be involved? 3 Test 8 dit au Re Implem ent 9 Select audit topic • What should be audited? op vel s De lution so Co n audduct it • Prioritise audit topics. 7 Rev resuiew lts 6 4 • Make a final decision about your audit topic. lyse Ana ta da 5 Toward Clinical Excellence Who should be involved? Clinical care is multidisciplinary. When an activity is undertaken to improve care, the process should involve all members of the clinical team. This is not to suggest that individual professional audit is not a key part of a departmental or practice audit programme. Focusing all members of the team on key areas in a co-ordinated way, however, will generally lead to greater improvements. Team ownership of the audit process makes improvements easier to identify and implement in all aspects of care. 12 MOH - TCE 2002 5/1/02 9:22 AM Page 13 What should be audited? Because health resources are limited, it is important to determine the priority topics for an audit in a particular service area. The following questions may be useful in identifying audit topics. • What things are done frequently? • Are there areas where problems have been identified? • What do we do that involves significant risk (to patients, staff or the organisation)? • What do consumers think we should audit? • Is there a particular area of practice where complaints have been received? • Where is there clear potential for improving the service delivery? • Where do national standards or guidelines exist? • Is there evidence about clinical effectiveness? • What do we do that is particularly costly? • What are the personal interests of the health team members? Answering the questions above will identify a number of potential audit topics that may need to be prioritised. One or more of the following criteria may help to rank the topics in order of priority: • clinical concern • concern of service users • high risk • high volume • clinician interest • complex or difficult management • high cost • availability of national or professional standards. 13 Clinical Audit Prioritise audit topics MOH - TCE 2002 5/1/02 9:22 AM Page 14 One way of prioritising topics is to score each topic based on frequency, risk and level of concern to practitioners. Topics that score highly on all of these criteria are high priority for audit. Alternatively, scoring each topic using different criteria such as availability of evidence, patient concern and cost may produce a different priority. It is important that the team agree on the method used for prioritising the topics because the criteria chosen will influence the result, as shown in the examples below. Audit prioritisation examples Example 1 identifies asthma management as the highest priority for audit by scoring each potential audit topic out of 10 and then comparing totals for the three criteria the team chooses. Example 1 Toward Clinical Excellence Topic Frequency Risk Practitioner concern Total Asthma management 7/10 6/10 8/10 21/30 Chest pain management 6/10 8/10 5/10 19/30 Heart failure management 5/10 6/10 4/10 15/30 Using the scoring system in Example 2, chest pain management is the area that scores highest and is the priority for audit. Example 2 Topic Evidence Patient concern Cost Total Asthma management 7/10 4/10 5/10 16/30 Chest pain management 9/10 7/10 8/10 24/30 Heart failure management 6/10 3/10 7/10 16/30 14 MOH - TCE 2002 5/1/02 9:22 AM Page 15 Levels of evidence An important part of selecting an audit topic is agreement by the team as to the level of evidence that will be required to justify any proposed clinical practice improvement as a result of the study. Three accepted ways to determine levels of evidence are Sackett’s five-level approach, the more recent revised SIGN grading system, and the Joanna Briggs Quality of Evidence ratings. All these approaches are outlined in Appendix E. Make a final decision about your audit topic Prior to finalising your audit topic it is a good idea to review the process to date and ask the following questions. • Is the topic important? • Is it ‘do-able’? • Are all the important stakeholders involved? • Is there a ‘champion’ in the department for the audit and improvements? Clinical Audit • Has the issue of confidentiality been addressed? • Will ethical approval be required? 15 MOH - TCE 2002 5/1/02 9:22 AM Page 16 Step 2: Plan the audit 1 2 Pl an • Set aims and objectives. 3 Test 8 dit au Re Implem ent 9 Select audit topic • Develop best practice standards. op vel s De lution so Co n audduct it • Select cases for audit. 7 Rev resuiew lts Toward Clinical Excellence 6 4 • Develop the audit tool and method of data collection. lyse Ana ta da 5 Set aims and objectives for the audit After choosing an audit topic it is important for the practitioner or team to agree on exactly what the project is trying to achieve or establish. This will ensure that the audit stays focused, making the most efficient use of time and resources. The audit should concentrate on collecting specific data, using the simplest method in the shortest possible time. This approach makes audit easy and effective. The aim and objectives of the audit should follow on from the question you are trying to answer (see example below). 16 MOH - TCE 2002 5/1/02 9:22 AM Page 17 The aim is a broad statement of intent. Always try to express the aim using positive language. Objectives break the audit aim into components that are measurable and time limited. Examples of the question that the audit is to answer Question Audit aim Example 1 Are we managing patients with asthma in the emergency department based on best practice standards? Improve the management of patients with asthma in the emergency department Example 2 Are we managing elderly people who fall based on best practice standards? Improve the management of elderly patients who fall Example 3 Clinical Audit Are we using best practice standards to assess patients with leg ulcers? Improve the assessment of ambulatory patients with leg ulcers 17 MOH - TCE 2002 5/1/02 9:22 AM Page 18 Develop best practice standards Once the aims of the audit have been established, the team develops standards that are measurable statements of best practice for these goals, derived from guidelines. The guidelines and standards you use for the audit should represent, where possible, evidence based best practice. The guidelines will have evidence of specific interventions that are proven to be effective and should be used to develop standards for the audit. (References on guideline development including information from the New Zealand Guidelines Group and the United Bristol Healthcare National Health Service Trust are listed in Appendix C.) Examples of standards and guidelines Audit aim Guidelines Standard ‘Patients should be given a written action plan based on signs and symptoms and/or PEF (Peak Expiratory Flow); this is especially important for patients with moderate-tosevere persistent asthma or a history of severe exacerbation.’ All patients discharged from the emergency department with asthma receive and have explained to them a written action plan. Example 1 Toward Clinical Excellence To improve the management of patients with asthma in the emergency department US National Guidelines Clearing House 18 MOH - TCE 2002 5/1/02 9:22 AM Audit aim Page 19 Guidelines Standard ‘Elderly patients who have fallen and are admitted to hospital should have the safety of their home environment assessed prior to discharge.’ All elderly persons who have fallen and are admitted to hospital have documented evidence that the safety of their home environment has been assessed prior to discharge. Example 2 To improve the management of elderly patients who fall (Mawson and McCreadie 1998) Example 3 To improve the management of ambulatory patients with leg ulcers ‘Assessment and clinical investigation should be undertaken by a health care professional trained in leg ulcer management.’ All ambulatory patients with a leg ulcer of six weeks’ duration are assessed by a trained nurse practitioner. RCN Institute (1998) A useful acronym in developing new standards or using existing standards is S.M.A.R.T. It is a reminder that evidence based guidelines and standards should be: • Specific – use precise language. • Measurable – identify a target standard to measure practice against. • Achievable – use performance levels that can actually be used in practice. • Related – to the aims and objectives of your project. • Theoretically sound (based on best practice) and Time-bound. 19 Clinical Audit From the range of guidelines available, it is important to find one that is suited to your environment and adequately evidence based. Three different ways of assessing the evidence associated with the guideline’s standards are detailed in Appendix E. MOH - TCE 2002 5/1/02 9:22 AM Page 20 Select cases for audit In selecting an audit sample the team must first identify the target population whose care is being reviewed. The population needs to be defined as exactly as possible, since the results will be applicable only to the population you choose. Examples of selected study population Audit aim Study population Example 1 To improve the management of asthma in the emergency department Patients with asthma presenting to the emergency department, who have not been referred by the general practitioner, are 16 years or over and were discharged home from the emergency department. Example 2 Toward Clinical Excellence To improve the management of elderly patients who fall People of 65 years or over who have fallen and subsequently received physiotherapy and/or occupational therapy while they were inpatients and for whom the fall was the key reason for therapy intervention. Example 3 To improve the management of ambulatory patients with leg ulcers Patients receiving ambulatory care whose leg ulcer shows no improvement for a period of six weeks or more. These above examples illustrate how specifically the target population is defined. • The patients with asthma in the first audit used the emergency department for primary care and were discharged; therefore they should have pre-discharge education and a written action plan. This audit does not review the department’s management of patients with acute severe asthma admitted to hospital in the same period. 20 MOH - TCE 2002 5/1/02 9:22 AM Page 21 • The second audit identifies elderly people who receive physiotherapy or occupational therapy after a fall. Elderly patients not reviewed or referred to physiotherapy or occupational therapy after a fall, or for whom a fall was not the primary reason for the referral, are excluded. • In the third example, patients receiving ambulatory care for a lower leg ulcer of more than six weeks’ duration, which is healing, are not included in this study. Sampling In deciding if it is necessary to review every patient who meets the criteria in the audit, consider the following points. • It is not practical to review every patient record in a large population. • A timeframe that is too short or a sample that is too small will introduce bias. • Where conditions are seasonal, estimates of frequency will vary widely depending on the time of year that the sample is taken. You may choose one of the following methods of sampling. Systematic sampling uses every nth case, eg, every 10th case. Choose the starting point at random, eg, based on a random number between 1 and 10 (use a calculator or computer). Then review only the cases identified in the sampling. 21 Clinical Audit For a population of 100 patients you can sample 80 and be 95 percent confident that your sample reflects the population. That is, there is a one in 20 chance that the your results will not be representative. Using a sample is acceptable as long as everyone is aware of the greater chance that the sample may not be representative and agrees that improvements can be made to local management based on the results. MOH - TCE 2002 5/1/02 9:22 AM Page 22 Simple random sampling gives each person in the population an equal chance of being drawn. A random number generator (on a scientific calculator or in a statistical tables book) can be used to choose a population sample. A number of readily available texts explain this process (see Beaglehole et al 1993). In stratified random sampling, you may specify certain conditions for the sample before taking the sample. For example, if your population is 80 percent European and 20 percent Måori and you wish to ensure these populations are appropriately represented, you can separate the two groups and sample of each group at random. Develop the audit tool and method of data collection Once the topic, aims, standards and population have been defined, you can design a tool to collect the appropriate data. Ask the following questions about data collection. Toward Clinical Excellence 1 What data do you need to collect? Data collection should include relevant information about the sample of consumers audited, which may include: • basic demographic data – eg, patient age, sex, ethnicity • clinical data – eg, peak flow readings and medication. The specific information you collect will depend on what data analysis you will be conducting to determine whether you are meeting your audit goals. 2 When will data be collected? Depending on the availability of data, your audit may be: • retrospective, if data has already been collected. It is important to check if the data have been collected accurately and completely before commencing a full audit. As treatments and recommendations change over time, and a retrospective audit over many years may not represent current practice; 22 MOH - TCE 2002 5/1/02 9:22 AM Page 23 • prospective or concurrent audit, if no data or insufficient data have been collected or if practice has changed • concurrent, if data is collected as care is provided. The advantages and disadvantages of these three types of audit are outlined in Appendix F. 3 Who will collect the data? To determine who would be the most appropriate and reliable person to collect the data, consider the relative importance of the following: • understanding the audit goals • being familiar with the condition • being familiar with the case notes structure • being someone who may need to change practice as a result of the audit. Finally, ensure the person or people collecting the data are allocated sufficient time and training for this task. Sufficient time is especially important where data collection may compete with a clinical caseload. 23 Clinical Audit A multidisciplinary audit might require more than one person to complete parts of the data collection form. In this case it is important to define the responsibilities for each person involved. MOH - TCE 2002 5/1/02 9:22 AM Page 24 4 Will data collection be manual or electronic? Choice of manual or electronic format will depend on a number of factors, such as the availability of appropriate information technology systems and support. Among the advantages of electronic data collection are that it: • is efficient – data are collected and entered in one step • is accessible – an electronic collection form on a network can be accessed from any terminal • allows immediate manipulation of large amounts of data • avoids loss of data collection forms • reduces transcribing error. Both manual and electronic data collection raise privacy issues. Ensure data are anonymised and used only for the purpose for which they were collected. Further, although the potential for remote access through electronic collection can be advantageous, it is important to address associated privacy concerns. Toward Clinical Excellence 5 What is an appropriate design for the data collection form? To ensure consistent collection of data, the data collection form should be simple and unambiguous. For each of the standards defined for the audit, there should be at least one question with clear options for the answer. Ensure you can identify which goal each question addresses. In designing a manual form consider the following points. • Keep the questions short and simple. • Where possible use forced choice options such as tick boxes to simplify data recording and analysis. • If brief guidance notes are required put them after the question. • Ideally the form will be no longer than one A4 sheet. 24 MOH - TCE 2002 5/1/02 9:22 AM Page 25 Example of a manual form for asthma Identification code Question Yes No Comment Name of Practitioner 1 Peak flow Documented on arrival in ED (within 10 minutes) 2 Best peak flow Documented (or no previous peak flow) Predicted peak flow documented 3 Action plan Documented as given and explained An electronic data collection form is an alternative to a paper data collection form. Consider the following points in its design. • Use only one screen on which the respondent reads and answers the questions. • Ask short, unambiguous questions. Clinical Audit • Have explanations of individual questions available. • Numbers, options or tick boxes are the best structures for responses (not free text). 25 MOH - TCE 2002 5/1/02 9:22 AM Page 26 Example of an electronic form for asthma Patient 1 Asthma Electronic Audit Observations Yes Peak flow (< 10 minutes of arrival) Best peak flow (or ‘no previous peak flow’) predicted documented Documented action plan Comment ¤ ¤ Respiration rate Waiting Dr < 10 minutes No ¤ Patient triage code 3 ¤ ¤ Where electronic data already exist, a form is not needed. Instead, simply sort the data appropriately and check for validity. Toward Clinical Excellence 6 How will completed data collection forms be stored? In prospective and concurrent manual audits particularly, everyone must be aware of where the forms are kept. Ideally the storage place will be near where the forms are going to be used. Everyone must also be aware of where to send the forms after completing them. A reminder on the form is a good idea. 26 MOH - TCE 2002 5/1/02 9:22 AM Page 27 Step 3: Test the audit methodology and tool 1 2 Pl an • Seek comment from colleagues. 3 op vel s De lution so Co n audduct it Test 8 it ud -a e R Implem ent 9 Select audit topic 7 Rev resuiew lts 6 4 • Pilot the audit methodology and tool. • Undertake final modifications. lyse Ana ta da 5 Seek comment from colleagues Another advantage of seeking feedback is that it publicises the project to other members of staff. Such awareness may be useful later when making recommendations for change and subsequently implementing them. 27 Clinical Audit As a first task in pretesting the tool, ask for feedback from colleagues who have not been involved in the process of development. Check that they interpret the wording in the way that is intended. They can also give feedback on the validity of the tool by questioning the rationale for including or excluding certain questions. MOH - TCE 2002 5/1/02 9:22 AM Page 28 Pilot the audit methodology and tool After seeking comment from colleagues, pilot the tool on a few cases that will not be included in the sample (around 10 percent of the sample size). The results of the pilot should indicate that the data you are about to collect will enable you to meet the audit goals. The people who are going to perform the audit should check the results for any misinterpretations and determine if any modifications are required before commencing the audit proper. If the audit is concurrent or prospective, have a team member available to answer any questions and consider whether either the methodology or the tool require modification. Undertake final modifications – tool, staff training and timeframe Piloting the tool and subsequent feedback may highlight: • where further training of those performing the audit is required Toward Clinical Excellence • a need to define more clearly some of the questions or terms in the audit tool • time constraints to be allowed for, and which the data collection schedule must accommodate. 28 MOH - TCE 2002 5/1/02 9:22 AM Page 29 Step 4: Conduct the audit 1 2 Pl an • Audit co-ordination. 3 op vel s De lution so Co n audduct it Test 8 dit au Re Implem ent 9 Select audit topic 7 Rev resuiew lts 6 • Avoid errors that lead to poor data collection and collation. • Collect the data. 4 lyse Ana ta da 5 Step 4 involves the collection of the audit data and then collation of the audit forms. Issues may arise that need clarifying so it is useful to have a member of the team available to answer questions from those collecting the data and to follow up incomplete data. Identifying and resourcing an audit co-ordinator is a key consideration. Data collection is one of the most labour-intensive steps of the audit process. In addition, the accuracy and timeliness of data collection are critical – they influence analysis and interpretation of data and thereby the outcome of the audit. 29 Clinical Audit Audit co-ordination MOH - TCE 2002 5/1/02 9:22 AM Page 30 Avoid errors that lead to poor data collection and collation It is the role of the audit co-ordinator to minimise human error in collecting and collating data, particularly where data are being collected manually. Measures to limit human error include: • appropriate staff training, including an understanding of the purpose of the audit • a set process for answering queries related to data collection (for a concurrent or prospective audit) • an unambiguous process for collecting and collating data • strategies to ensure inter-rater reliability • strategies to reduce transcribing errors • a set process for dealing with missing data (retrospective study). Collect the data Toward Clinical Excellence After all preliminary steps have been taken and appropriate procedures are in place, you are ready to collect the data. 30 MOH - TCE 2002 5/1/02 9:22 AM Page 31 Step 5: Analyse the data 1 2 Pl an • Collate the results. 3 op vel s De lution so Co n audduct it Test 8 it ud -a e R Implem ent 9 Select audit topic 7 Rev resuiew lts 6 • Identify gaps between standards and performance. • Use graphical displays of data. 4 lyse Ana ta da 5 Use the standards that you identified prior to the audit in analysing the data. Where possible, use simple statistical methods to analyse the results against the agreed standards to highlight the areas requiring improvement. If more sophisticated methods are required, it may be necessary to seek external statistical or analytical expertise. It is important, however, that it is the team who interprets the results. 31 Clinical Audit Collate the results MOH - TCE 2002 5/1/02 9:22 AM Page 32 Identify gaps between standards and performance Where benchmarks are established for monitoring points, it is important to identify variance or gaps between the expected standards and actual delivery of care. Use graphical displays of data Using graphs is an effective way of presenting the results. The examples below demonstrate how histograms may be used. Run-charts that show variation across time can be used to illustrate changes since the previous audit. Examples of histograms 100 100 80 80 60 40 20 0 Note: 32 Figure 5: Percentage of patients receiving an asthma action plan by day of the week Percentage Percentage Toward Clinical Excellence Figure 4: Percentage of patients receiving an asthma action plan by shift 60 40 20 AM shift PM shift Night shift 0 Mon Tue Wed Thu Fri Sat Both figures illustrate how data compare with the audit standard of achieving 100% compliance in patients receiving an asthma education plan. Figure 4 analyses the results by shift, Figure 5 by day of the week. Sun MOH - TCE 2002 5/1/02 9:22 AM Page 33 Step 6: Review 1 2 Pl an • Interpret the results. 3 op vel s De lution so Co n audduct it Test 8 it ud -a e R Implem ent 9 Select audit topic 7 Rev resuiew lts 6 4 • Recognise bias and variance. • Rank problems according to priority for resolution. lyse Ana ta da 5 Interpret the results Clinical Audit Those providing clinical care should interpret and validate the results. They have the greatest understanding of the process and will be able to identify the most appropriate corrective actions. Interpreting results requires individual reflection and discussion with colleagues. This task may be time consuming, but it is important not to jump to conclusions. Recognise bias and variance Despite careful planning, problems that arise during data collection can introduce bias that must be acknowledged. Bias may arise from errors in sampling, in data collection and collation or in the design of the tool itself. 33 MOH - TCE 2002 5/1/02 9:22 AM Page 34 Variation is present in all aspects of life. Walter Shewhart’s concept of variation (Langley et al 1966) suggests that it should be viewed in one of two ways: 1 as variation that indicates something has changed or is not right 2 as random variation, which is similar to variation that has occurred in the past and does not indicate change. Decisions therefore should be based on the nature of the variation. Individual instances of variation may not be relevant whereas a pattern of variation will warrant further attention. Audit results are reviewed against the aims and objectives set in Step 2. Your review may include the following questions: • How do the standards that you set at the start of the audit compare to the results? • How has the performance of other departments or hospitals compared to the standards? • Are there established benchmarks being met elsewhere? Toward Clinical Excellence • How do the results compare to other published studies? Rank problems according to priority for resolution Again, those members of the team providing clinical care must have input in ranking the problems identified in the review of results. During this process you will need to refer back to the key questions identified in relation to the audit aim (Step 2). 34 MOH - TCE 2002 5/1/02 9:22 AM Page 35 Step 7: Develop solutions 1 2 Pl an 3 op vel s De lution so Co n audduct it Test 8 dit au Re Implem ent 9 Select audit topic 7 Rev resuiew lts 6 • Form an action plan to select the appropriate solution. • Document required changes and remedial action. 4 lyse Ana ta da 5 Form an action plan to select the appropriate solution • list all identifiable causes for variance • prioritise these causes • define actions to address these causes • where there is more than one possible solution, first trial the solution that seems the most appropriate. If no improvement is evident by the time of re-audit, you will need to try other possible solutions, so do not discard this information. 35 Clinical Audit Once the areas requiring improvement have been identified, the first task is to form an action plan. As mentioned in the introduction, the challenge is not just to identify inadequate care, but to change clinical practice to improve patient care. To this end, an action plan will: MOH - TCE 2002 5/1/02 9:22 AM Page 36 Document required changes and remedial action Toward Clinical Excellence When an audit is completed it is important to present the results to all those participating in the care process and to document them for wider circulation. The recommended actions and required changes to policy or procedure should be documented. The person responsible for the audit and for ensuring these changes occur should present and interpret the data for other groups involved including management. Reminders are also useful to reinforce the findings and recommendations of the audit. 36 MOH - TCE 2002 5/1/02 9:22 AM Page 37 Step 8: Implement solutions 1 2 Pl an • Undertake diagnostic analysis. 3 op vel s De lution so Co n audduct it Test 8 dit au Re Implem ent 9 Select audit topic 7 Rev resuiew lts 6 4 • Develop a dissemination and implementation strategy. • Plan to monitor and evaluate. lyse Ana ta da 5 Undertake diagnostic analysis In your analysis of the available information, identify: 1 all groups affected by or influencing the proposed change 2 potential internal and external barriers to change, including whether practitioners are willing to change 3 enabling factors for change such as resources and skills. 37 Clinical Audit Changing clinical practice requires a systematic approach and strategic planning. This requires diagnostic analysis, development of a dissemination and implementation strategy and a plan to monitor and evaluate the impact of any changes made. MOH - TCE 2002 5/1/02 9:22 AM Page 38 Develop a dissemination and implementation strategy Raising awareness about the reasons for change, while important, is unlikely to be sufficient to change behaviour. Below is a brief review of the evidence concerning benefits and pitfalls of implementation strategies to improve clinical practice. The following features contribute to an effective implementation strategy. • Timing is important when introducing change. • Health professionals need to receive information that is easy to understand. It should also give them confidence that the proposed changes are based on valid and reliable information and will have a beneficial impact on patient outcomes. • A personal approach works best – take time to talk with individual practitioners and teams. • Successful solutions will take account of particular service or service team needs. Toward Clinical Excellence Audit and feedback are effective mechanisms of change, but not on their own. Strategies that enhance their effectiveness include: • using opinion leaders to generate enthusiasm • providing reminders and information on progress • using peer comparisons within the service • linking changes in practice with patient outcomes • targeting trainees • benchmarking with other organisations. 38 MOH - TCE 2002 5/1/02 9:22 AM Page 39 Ineffective strategies include: • distribution of printed material as a strategy on its own • didactic education. For more information on evidence based implementation strategies refer to: www.york.ac.uk/inst/crd/ehc51.pdf or NHS Centre for Review and Dissemination (1999). Plan to monitor and evaluate Your planning for monitoring and evaluation culminates in the final step of the audit cycle – re-audit. Re-audit evaluates the effectiveness of the change, the degree of compliance and develops strategies to maintain and reinforce change. Clinical Audit 39 MOH - TCE 2002 5/1/02 9:22 AM Page 40 Step 9: Re-audit 1 9 2 Pl an 3 op vel s De lution so Co n audduct it Test 8 -a t Implem ent Re i ud Select audit topic 7 Re v resuiew lts 6 • Has improvement been achieved? 4 ly s e A n a ta da 5 Toward Clinical Excellence Has improvement been achieved? The final step is to re-audit. This important step shows whether the changes implemented have improved care or whether further changes are required. Re-audit is usually limited to those areas highlighted as requiring improvement. If you find no, or only limited improvement, you may need to: • review some of the other solutions that were not used after the initial audit • find completely new solutions • undo some of the changes if they are not successful – but you needed to try them to gain evidence of their effect. CONGRATULATIONS! This audit cycle is now complete. Audit follow-up Take some time to reflect on: • what you learned from this audit • what you could have done better • how to put this learning into practice with your next project. 40 MOH - TCE 2002 5/1/02 9:22 AM Page 41 Section Three: Peer Review Introduction The peer review process is designed to foster individual accountability for professional development and practice, as well as group accountability for the overall quality of professional practice in a particular service. Peer review is considered to be a hallmark of professional practice. It is a professional tool with multiple uses. It contributes to clinical audit and other activities such as professional development, information for credentialling and service planning and evaluation. The overall purpose of peer review is to inform others about one’s own practice in relation to that of the peer group. Its more specific purposes and benefits are summarised in Table 1. Table 1: Purposes and benefits of peer review Purpose Benefits • Acknowledges professional expertise • Allows feedback on individual performance • Provides a ‘safe’ environment to admit mistakes Ongoing education • Helps individuals to identify their strengths and areas needing improvement Awareness of standards and quality of performance • Encourages information sharing • Promotes a professional approach to problems that is centred on the service user • Helps to organise information Individual and team accountability for professional development and practice • Provides reassurance of personal and/ or team competence • Gives support during periods of risk taking, conflict and role transition Improvement of teamwork • Recognises group members as resources, providing different perspectives • Creates a feeling of equality • Takes pressure off individuals 41 Peer Review Self-regulation within the profession MOH - TCE 2002 5/1/02 9:22 AM Page 42 Types of peer review Peer review may be regular and structured or occur informally. Informal peer review happens on a daily basis often without being recognised as such; for example, ‘handover’ between shifts and discussion with colleagues about investigations or drug therapy. ‘Grand rounds’ are an example of a more formal approach. Toward Clinical Excellence Structured assessments such as questionnaires also provide useful feedback for practitioners about their performance. Such assessments may be completed by professional peers or by a range of respondents. For example, the Physician Achievement Review (PAR) developed in Alberta, Canada requires practitioners to have their performance assessed by a range of stakeholders every five years. Questionnaires are completed by 25 patients, eight medical peers and eight practitioners from other health professions. These respondents comment on five categories of performance (clinical knowledge and skills, communication skills, psychosocial management, administrative skills and collegiality) (www.par-program.org/PAR-History2.htm). A formal peer review process increases objectivity and provides a structured opportunity to give significant feedback. Effective peer review demands commitment and consultative skills. It also requires an understanding of small group processes, good listening skills, skills in facilitation and problem solving, and an understanding of the principles of effective feedback. How to introduce peer review Peer review is a skill in which many health professionals have had no formal training. Some professional groups have developed resources that are readily available, such as the Physician Assessment Review developed by the Royal Australasian College of Physicians. 42 MOH - TCE 2002 5/1/02 9:22 AM Page 43 Before introducing a formal process of peer review, it may be useful to have a workshop to discuss the purpose and process. The following points should be included: • Comments should be precise, objective and factual, using nonjudgemental language. • Give your opinion only; do not discuss the review with others. • Excellence in one area does not mean the person is excellent in all areas. Comment both on areas of excellence and areas needing improvement. • Comments at either end of the scale must be supported by specific examples. • Consider the person’s performance over the whole time under review. It is important to avoid being swayed by one incident. • Accurate reporting is a professional responsibility. Modifying feedback to avoid damaging or enhancing egos places consumer safety and professional credibility at risk. Peer review meetings Peer Review The process, content and documentation of formal peer review meetings should be agreed in writing. Among the matters that need to be specified are: • the frequency of meetings, numbers for a quorum, election of a chair • the method of reporting key findings, which will be influenced by whether this is a protected activity under the Medical Practitioners Act 1995, Part VI. However, any method should include the essential components of peer review: – discussion of adverse events – quantitative indices of the clinical performance of the service – identification of systemic deficiencies – follow-up of previously identified matters 43 MOH - TCE 2002 5/1/02 9:22 AM Page 44 • the reporting route and responsibilities of the person receiving the report to deal with issues identified • how to manage the situations where serious concern about an individual’s performance has been identified. This would include establishing a process for reporting to the credentialling committee in the event of a clinical competence issue. Self-evaluation Like peer feedback, self-evaluation is both a skill and a professional responsibility. It is another valuable element of the peer review process that encourages a critical look at one’s own performance and clinical practice. It is aimed at providing insight, promoting expertise, and evaluating competency. New Zealand peer review documentation Toward Clinical Excellence Appendix H provides a directory of health professional organisations and associated groups in New Zealand, to assist in locating examples specific to a given health profession. In addition, a Ministry of Health webpage (www.moh.govt.nz/quality) is to be developed and will provide examples of current practice in this area. 44 MOH - TCE 2002 5/1/02 9:22 AM Page 45 Section Four: Morbidity and Mortality Review Introduction Morbidity and mortality review meetings are a core component of any service quality plan. They focus on systems and processes used in the service, but may generate information on the performance of individual practitioners. Review meetings should be held on a regular basis; the specific frequency will depend on the service, but there should be at least one review a month. Frequent meetings are important so that the cases reviewed are fresh in the memory of those involved. Objectives The objectives of morbidity and mortality meetings are to: 2 recommend improvements to processes and systems 3 action these recommendations and monitor the results. 45 Morbidity and Mortality Review 1 involve the multidisciplinary team in a critical analysis of the systems and processes leading to an outcome of care. In addition to examining deaths and adverse events within the service setting, a morbidity and mortality review may cover subsequent death outside the service setting where appropriate and may regularly review clinical indicators relevant to the service MOH - TCE 2002 5/1/02 9:22 AM Page 46 Key points Morbidity and mortality meetings are more effective where: • written Terms of Reference make explicit that the focus of discussion is on process and system change, with the aim of developing recommendations to prevent a similar adverse outcome in the future • the meeting is aimed at team learning and quality improvement. It should therefore include staff at all levels of seniority • this is not a forum to discuss individual competence. Where competence issues are apparent, the chair (or professional leader or service manager) should consider referring the matter to the appropriate credentials committee • everyone involved in the case under review should be given the opportunity to report to the meeting Toward Clinical Excellence • a brief written report should be compiled for each meeting. Action points should be noted for the next agenda and feedback must always be given to ensure that recommended changes are implemented and evaluated. 46 MOH - TCE 2002 5/1/02 9:22 AM Page 47 Section Five: Clinical Practice Guidelines Introduction Perhaps the greatest benefit that clinical practice guidelines offer to the health system is improved outcomes for service users through consistent and effective treatment. By promoting interventions with proven benefit they can reduce morbidity and mortality and improve quality of life for some conditions, thereby increasing the cost-effectiveness of care provided. Should new or existing guidelines be used? Organisations may choose to evaluate and adopt existing ‘seed’ guidelines, thus taking advantage of the work of other groups rather than developing their own guidelines. The choice of approach will depend on the resources available for clinical quality improvement and the specific needs of the service concerned. Generally the evaluation and adaptation of existing guidelines, where these exist, is the more cost-effective option for most organisations. 47 Clinical Practice Guidelines Guideline use is not universally popular with some health practitioners for whom the ‘art’ rather than the ‘science’ of healing is the driving force in their practice philosophy. Yet guideline use does not eliminate the art required in practice, as it allows for adaptation to individual cases and circumstances. What use of guidelines does achieve is to reinforce the importance of critical appraisal of practice, which in turn highlights ineffective and dangerous practices as well as practices that are not cost-effective. MOH - TCE 2002 5/1/02 9:22 AM Page 48 Table 2 compares features of both options to be considered when deciding whether to evaluate and adapt existing guidelines or develop new ones. Table 2: A comparison between developing new guidelines and using existing guidelines Developing new guidelines Resources • Resource intensive Skills required • Literature search and evaluation • Guideline development Evaluating and adapting existing guidelines • Smaller investment required • Guideline evaluation • Leadership of a small group • Leadership of a small group Advantages • Information management • Information management • Facilitates implementation • Is less resource intensive than developing new guidelines Toward Clinical Excellence • Builds an evidence based culture in the service • Encourages practitioner participation and ‘buy in’ • May facilitate mandatory updating of guidelines • Develops relationships with other organisations through accessing and regular updating of their guidelines Finding ‘seed’ guidelines Finding guidelines on a particular topic is not hard to do. Finding rigorously developed guidelines that can be used to ‘seed’ local adaptation, however, is much more difficult. As part of the selection process, an interested clinician or specialist professional group often presents guidelines for consideration. The New Zealand Guidelines Group recommends using the AGREE guidelines appraisal instrument (www.agreecollaboration.org) to determine the quality of potential seed guidelines. It will save time and effort if you limit your searching to evidence based guidelines. Library staff can help 48 MOH - TCE 2002 5/1/02 9:22 AM Page 49 search electronic databases for guidelines. Another strategy that can save time is to go to sources with mature and rigorous guideline programmes. Appendix I lists a number of sources of ‘seed’ guidelines and international best practice. Developing effective guidelines To be effective, guidelines need to be: • valid – their use leads to health gain • reliable – given the same evidence and method another guideline group should produce essentially the same recommendations • representative – all key groups are involved in their development, including consumers • flexible – exceptions to recommendations should be identified including how to take account of consumer preferences • research based. Clinical Practice Guidelines 49 MOH - TCE 2002 5/1/02 9:22 AM Page 50 Guideline implementation Implementing a new guideline in a service needs planning. It is not sufficient to issue the guideline to all potential users; it needs to be ‘marketed’ in order to gain acceptance. The following strategies may assist in this process. • Ensure that guidelines take into account local circumstances. • Provide an appropriate education programme. • Enlist senior practitioners in the team to ‘champion’ guideline introduction. • Develop practical steps to ensure the guideline is used, such as patient specific reminders. • Identify and resolve any barriers to implementation. • Build checks and fail-safes into the system. • Consider the cost implications of implementation and ensure adequate resources are available. Toward Clinical Excellence Measurable health outcomes from guideline implementation include: • clinical outcomes • improvement in the quality of care • improvement in patient satisfaction • user satisfaction with the guideline • cost-effectiveness. Ongoing guideline evaluation Guidelines have a limited ‘shelf life’. Regular evaluation and review of guidelines are essential and ongoing. 50 MOH - TCE 2002 5/1/02 9:22 AM Page 51 Section Six: Clinical Pathways Introduction Clinical pathways are evidence based, multidisciplinary plans of care. They may be for patients who have been diagnosed with a specific condition (diagnosis based), who are having a particular procedure (procedure based), or who are presenting with a particular symptom (symptom based). These time- and stage-oriented tools are used to synchronise the activities of every member of the health care team to achieve predetermined patient outcomes and provide a continuum of care for those patients whose outcomes are predictable 60 to 75 percent of the time. The aim of using a clinical pathway is to increase the probability that the required care will achieve predetermined clinical outcomes and will be provided in a timely way, minimising delays, omissions, cancellations and unnecessary costs. 51 Clinical Pathways Clinical pathways are best developed by the multidisciplinary team members who are directly or indirectly involved in the care of the patient. It is essential that sufficient flexibility be built into the pathway format so that it can be tailored to individual patient needs. Two common designs for a clinical pathway are illustrated on the following page to demonstrate the format options. Currently algorithms are less frequently used than the text format in New Zealand. 52 Excerpt from Royal College of Nursing (1998). Week 2 Week 1 Consider specialist referral Treatment Text format Activity Section of an algorithm for the management of chronic leg ulcers (Arroll, Bourchier, Gelber Jull et al 2000) Assessment History and clinical examination including: • Pain • Wound • Doppler ultrasound • Pain • Wound • Activities of daily living Ulcer recurs Doppler of Ankle Brachial Index (ABI). Consider referral to appropriate service for management (Note 2) Yes Education 9:22 AM No Is patient diabetic or rheumatoid? History and clinical exam (Note 1) Leg ulcer > six weeks’ duration Algorithm format 5/1/02 A full clinical history and physical exam should be conducted for a patient presenting with either their first or recurrent leg ulcer and should be ongoing thereafter (Strength of Evidence III). Assessment and clinical investigations should be undertaken by a health care professional trained in ulcer management (Strength of Evidence III). Assessment of leg ulcers: Guideline Toward Clinical Excellence MOH - TCE 2002 Page 52 Example of clinical pathway formats MOH - TCE 2002 5/1/02 9:22 AM Page 53 Benefits of clinical pathway use Use of clinical pathways offers benefits for both health professional and service user (see local examples below). For staff, the pathway gives a plan of care for that patient on any given day, reducing documentation time. Needs are anticipated, allowing for better organisation and sequencing of interventions. It is an ideal orientation tool for new, less experienced or relieving staff and is a self-assessment guide for the team to evaluate and monitor care provided. The valuable contribution that each health care professional provides is also acknowledged. The pathway is a printed document. An edited, ‘patient friendly’ version should be available to patients so that they can participate in and have more control over their care. Written information for patients should cover: • the disease or procedure • the health professionals involved in their care • the care they should expect and when to expect it • discharge planning. Local examples of how a clinical pathway can benefit patient care The following two local examples have been reported recently: • An Ashburton Hospital audit found that only 50 percent of patients presenting with myocardial infarction received Aspirin pre-hospitalisation or in the emergency department. After introduction of a Myocardial Infarction Pathway, a Coronary Care Department audit showed 100 percent compliance. • Hutt Valley DHB’s Chronic Obstructive Pulmonary Disease pathway reduced the 30-day readmission rate by 15 percent and the 90-day rate by 27 percent. The average length of stay was reduced by 0.8 days. 53 Clinical Pathways • expected length of stay MOH - TCE 2002 5/1/02 9:22 AM Page 54 Quality improvement and risk management potential Although legal advantages have not been formally tested, it has been suggested that properly utilised pathways provide a wealth of information and assist in risk management in the case of an adverse event. The information is documented concisely, logically and independent of literary ability or memory. Less time is required for recording, resulting in better compliance. In addition, omissions are less likely and repetition and transcribing errors are avoided. Problems with legibility, grammar, spelling and inappropriate abbreviations are also minimised. Variance Toward Clinical Excellence Variance is any deviation from the predicted ‘path’. It can be positive or negative: a positive variance occurs when a patient progresses faster than expected; a negative variance when progress is slower than expected. A negative variance will also occur if activities prescribed in the pathway are not completed. Either way, variance is an important component in analysing and improving care. By identifying variances, team members are alerted to areas of care they can learn from. The key point about variance analysis is that the information obtained should be used to improve the pathway where appropriate. Evaluation criteria Evaluation criteria represent what the team hopes to achieve through implementation of a clinical pathway. They are identified at the outset and used as a basis for post-implementation evaluation and review. Outcomes that the team expects the patient to achieve are articulated and are specific to each pathway. Examples are outcomes related to mobility, physiological parameters such as absence of pain or fever or demonstrated ability to carry out particular self-care activities safely. 54 MOH - TCE 2002 5/1/02 9:22 AM Page 55 Section Seven: Clinical Indicators Introduction To improve quality in health care we need to be able to reliably measure and report aspects of clinical practice that will help us determine current performance and identify quality improvement. As well as providing information for health professionals, clinical indicators can enable consumers to make informed choices about treatment options, tell providers how they are functioning in comparison with others and enable funders to make appropriate policy and funding decisions. For example, District Health Boards use clinical indicators, such as bloodstream infection rates, in Balanced Scorecard reporting to the DHB Funding and Performance Monitoring Directorate. (For further information about the Balanced Scorecard approach to performance measurement see Appendix G.) Clinical Indicators The following principles of measurement underpin a clinical practice reporting framework and apply to the use of clinical indicators: • All measures should be transparent. • Information should be made freely available. • Measurements should be meaningful to health professionals to ensure accurate collection and subsequent use for quality improvement. • The process of measurement should require minimal additional effort for practitioners. • Practitioners should be involved throughout the process to ensure accuracy and to minimise misinterpretation. • Measurements used should be regularly reviewed. 55 MOH - TCE 2002 5/1/02 9:22 AM Page 56 Indicators of performance are pivotal tools for quality management. A clinical indicator is an objective measure of the clinical management and outcome of patient care at a point on the process – outcome continuum. Although outcomes measurement has been emphasised, process measures can be sensitive indicators of the quality of care (Crombie and Davies 1998). The advantages of process measures over outcome measures is that they: • are readily measured and easily interpreted • indicate deficiencies of care that need to be remedied Toward Clinical Excellence • do not depend on comparisons, in contrast to outcomes monitoring. It is important that evidence links the processes of care with desirable outcomes. Clinical indicators are not exact standards. They are designed to alert health professionals to possible problems as well as opportunities for improvement in patient care. Because clinical indicators are objective they may also identify areas for further quality activities, generate ideas for new studies, or assist in assessing whether a standard in patient care is being met. They allow for comparison of performance against national aggregate data, by providing agreed definitions – for example, the numerator and denominator for rate based indicators. Ideally, a good quality indicator should: • identify cases concurrently (to provide immediate patientspecific information) and retrospectively • use data that are easily collected • identify cases with a high likelihood of having sub-standard care • identify common problems caused by factors that are predictable, manageable and preventable. (Hofer et al 1997) 56 Excerpt from Royal College of Nursing (1998). Consider specialist referral Section of an algorithm for the management of chronic leg ulcers (Arroll, Bourchier, Gelber Jull et al 2000) Clinical Indicators Ulcer recurs Yes Doppler of Ankle Brachial Index (ABI). Consider referral to appropriate service for management (Note 2) No Is patient diabetic or rheumatoid? History and clinical exam (Note 1) • All patients not referred to a specialist have a doppler of Ankle Brachial Index at their initial examination. • All patients with a chronic leg ulcer of more than six weeks’ duration are assessed by a nurse trained in chronic leg ulcer management within 14 days of referral. Example of clinical indicators for the assessment of chronic leg ulcers: Clinical indicator 9:22 AM A full clinical history and physical exam should be conducted for a patient presenting with either their first or recurrent leg ulcer and should be ongoing thereafter (Strength of Evidence III). Assessment and clinical investigations should be undertaken by a health care professional trained in ulcer management (Strength of Evidence III). Leg ulcer > six weeks’ duration Clinical pathway 5/1/02 Assessment of leg ulcers: Guideline MOH - TCE 2002 Page 57 57 MOH - TCE 2002 5/1/02 9:22 AM Page 58 Four features of indicators There are four aspects of clinical and performance indicators: normative, technical, strategic and operational. These aspects are critical to the effectiveness of any indicator set. Society as a whole wants to know many different things about the health system. Different stakeholders bring different values, priorities and norms to the process of constructing indicator sets and including particular indicators. As we move toward the adoption of indicators that will be used by multiple stakeholders, some important questions are raised in regard to their normative aspects. For example, whose values will be addressed, whose values will predominate and how will conflicting values be reconciled? Toward Clinical Excellence Technical aspects should also be considered if indicators are to be useful. Such aspects include the ability to provide meaningful comparisons across entities, validity and reliability, the ability to distinguish between high and low performers and the capacity to be appropriately risk adjusted. To be operational – that is, to be useable and thus meaningful, indicators must provide reliable and comparative data. Accompanying them must be standard protocols to collect and provide data, along with methods of packaging and disseminating data that are comprehensible, credible and accessible to the end-users. Finally, if the strategic performance of a system is being assessed, the indicators must tap into the functioning of different elements of the system and identify how they interact. Uses of clinical indicators The two main categories of clinical indicators are: • rate based indicators for which a certain number of cases are usually unfavourable. Thresholds or acceptable rates in such cases are established from preliminary audits, or from comparisons with the international literature 58 MOH - TCE 2002 5/1/02 9:22 AM Page 59 • alert indicators, which signal to managers and staff that a specific event has happened or is likely to happen. These indicators monitor ‘vital signs’ within programmes and organisations and prompt predetermined responses when they identify that the risk of disaster or dysfunction is high. Method for selecting effective indicators For an indicator to be valid strong evidence must support the relationship between the process and outcome of care. If this relationship is weak the indicator is measuring a process that has minimal impact on the outcome, or it is measuring an outcome that is not associated with the process of care. In either case, the indicator is ineffective. To ensure indicators are effective, it is important to validate them during the selection process. The following method offers an example. • Select potential indicators based on a literature review and clinical expertise. • Assess the strength of evidence supporting the use of each indicator (process – outcome relationship). • Identify and classify preventable problems in the process of care. • Have an expert panel review each potential indicator. • Test each indicator for any quality problems using a chart review of cases and controls. • Simulate each indicator in its proposed setting and evaluate its performance. (For example, ask: what sample size is needed?) For further resources to develop clinical indicators see Appendices C, D and E. 59 Clinical Indicators • Identify rates of cases provided by each indicator. MOH - TCE 2002 5/1/02 9:22 AM Page 60 MOH - TCE 2002 5/1/02 9:22 AM Page 61 Appendix A: Members of the Working Group Principal Advisor, Quality Improvement and Audit, Ministry of Health Patsi Davies Consumer perspective Dr Stephen Dee (Project Co-ordinator to October 2001) Registrar Clinical Audit, Hutt Valley District Health Board Mr Phillip Godfrey General Surgeon & Medical Director, Ashburton Hospital Dr Ian Goodwin Liaison Psychiatrist, Auckland District Health Board Assoc Prof Peter Gow (Working Group Chair) Chairman, Clinical Board, Counties Manukau District Health Board Gillian Hall Senior Teaching Fellow, School of Physiotherapy, University of Otago Dr Lynne Lane Public Health Physician, Acting GM Funding, Auckland District Health Board Dr Peter Leslie Chair, New Zealand Council of Medical Colleges Phillipa Molloy Nurse Advisor, Nelson Marlborough District Health Board Amanda Newton Account Manager, New Zealand Health Information Service Appendix A: Gillian Bohm (Project Sponsor) Members of the Working Group 61 MOH - TCE 2002 5/1/02 9:22 AM Page 62 Appendix B: Similarities and Differences between Audit and Research Both audit and research have similarities that relate to process and rigour. For example, both activities: • answer a specific question relating to quality of care • may be prospective, retrospective or concurrent in nature • involve sampling, questionnaire design and analysis • require professional leadership. Toward Clinical Excellence Table B1 outlines some of the differences between audit and research. Even with this information it is sometimes still difficult to decide whether a proposal is audit or research. If so, it may be necessary to discuss the project with your Medical Advisor or Director of Nursing, clinical board, local ethics committee and regional ethics committee. 62 MOH - TCE 2002 5/1/02 9:22 AM Page 63 Table B1: Differences between research and audit Research Clinical audit Is based on a hypothesis Measures against standards Is usually carried out on a large scale over a prolonged period Is usually carried out on a relatively small population over a short time May involve patients receiving a completely new treatment Never involves a completely new treatment May involve experiments on patients Never involves anything being done to (or withheld from) patients beyond their normal clinical management May involve patients being allocated to different treatment groups Never involves allocation of patients to different treatment groups Is based on a scientifically valid sample size (although not necessarily with a pilot study) Depending on circumstances, may be based on a sample size acceptable to clinicians Extensive statistical analysis of data is routine Some statistical analysis may be useful (simple statistics often applied) Results are generalisable and hence publishable Results are only relevant within local setting (though the audit process may be useful to others) Responsibility to act on findings is unclear Responsibility to act on findings rests with the organisation – Clinical Director/Clinical Head of Department and Management Findings influence the activities of clinical practice as a whole Findings influence activities of local clinicians and teams Always requires ethical approval May require ethical approval Always requires patient consent May require patient consent 63 Similarities and Differences between Audit and Research Answers ‘Are we following best practice?’ Appendix B: Creates new knowledge about what works and what doesn’t MOH - TCE 2002 5/1/02 9:22 AM Appendix C: Page 64 Online Resources for Clinical Audit Table C1 lists New Zealand sites that have general advice on quality activities or are repositories of standards, examples of audits or audit tools for use in secondary care. Overseas sites are listed in Table C2. Table C1: New Zealand audit resources on the Internet Toward Clinical Excellence Organisation New Zealand Guidelines Group Description Evidence based guidelines that can be used as standards for audit Section on introducing guidelines New Zealand Centre www.joannabriggs.edu.au/ Guidelines and latest evidence for Evidence Based Nursing team/nz.html in Nursing practice New Zealand Health www.nzhis.govt.nz Data standards and publications Information Service Clinical Leaders www.clanz.org.nz Role of leadership in quality Association of improvement New Zealand Health Research www.hrc.govt.nz/publicns Guidelines for health research and Council how to put research into practice Mental Health www.mhc.govt.nz Publications and progress reports Commission in implementation of Blueprint for Mental Health Services in New Zealand New Zealand Health nzhta.chmeds.ac.nz/ Clearing house of evidence Technology Assessment based health policy and outcomes Clearing House assessment National Health www.nhc.govt.nz Online publications on quality and Committee important areas in health requiring focus Health and Disability www.hdc.org.nz Code of patient rights, reports and Commissioner outcomes of cases Enigma Publishing www.enigma.co.nz/hcro_ Articles on accreditation Group articles/0006/vol4no6_002 and health care quality www.healthsite.co.nz/ Access to journals and resources practice_support/ online by speciality specialist_care/index.htm National Library of tepuna.natlib.govt.nz/web National Library collection New Zealand Health _directory/NZ/health.html of online health resources Resources Online Ministry of Health www.moh.govt.nz Ministry publications and quality initiatives 64 Internet address www.nzgg.org.nz MOH - TCE 2002 5/1/02 9:22 AM Page 65 Table C2: Overseas audit resources on the Internet Resources IHI quality improvement resources Internet address www.ihi.org/resources/qi/ index.asp Description A model of quality improvement; contains a lot of useful quality knowledge and tips Audit flags (clinical indicators) www.jr2.ox.ac.uk/bandolier/ Nine generic quality band7/b7-6.html flags/indicators from Bandolier Example of quality improvement tools and techniques used in the emergency medical service, US www.nhtsa.dot.gov/people/ Guidance in using run injury/ems/leaderguide/ charts, histograms, cause #qitat and effect diagrams, flow charts and pareto diagrams for service quality improvement Pain management in day www.jr2.ox.ac.uk/bandolier/ Project to improve post-operative case surgery: a quality ImpAct/imp08/i8-4.html analgesia and a web-based improvement project clinical guideline and flowchart Quality flags in fractured www.jr2.ox.ac.uk/bandolier/ Some flags from the UK on care neck of femur band25/b25-3.html processes for neck of femur management management Example of how to apply www.jr2.ox.ac.uk/bandolier/ Summary of a continuous quality continuous quality band83/b83-3.html improvement project focused on improvement to a the rate of planned induction of clinical problem labour from Canada (Summary Bandolier) Outcome indicators in maternal health, child health, dental health, colorectal cancer and emergency admissions National Institute of Clinical Excellence www.nice.org.uk/ catlist.asp?c=160 Guidelines Primary care audit tools, UK (University of Leicester) www.le.ac.uk/cgrdu/ protocol.html Specific audit tools for angina, asthma, heart failure, hypertension and benzodiazepine prescribing in primary care The clinical audit assessment framework: improving the effectiveness of audit (UK) www.hsmc3.bhlam.ac.uk/ hsmc/publicns/caaf In-depth frameworks for assessing audit projects and programmes 65 Online Resources for Clinical Audit www.show.scot.nhs.uk/ indicators/Publications/ OutcomesReport2000.pdf Appendix C: Scottish report on outcome indicators MOH - TCE 2002 5/1/02 9:22 AM Appendix D: Page 66 National Health Information Datasets held by the New Zealand Health Information Service The New Zealand Health Information Service (NZHIS) is a group within the Ministry of Health responsible for the collection and dissemination of health-related information. Its data cover all publicly funded inpatient and day patient events obtained from hospitals. This is a good source of information to use as part of an audit. Toward Clinical Excellence The NZHIS data can be used to analyse efficiency, effectiveness, severity and complexity, quality, coding practice and survival. This analysis can be performed at a specific level, grouped by diagnosis/procedures, admission types and discharge types. An analysis is also possible at aggregated levels by facility, District Health Board, rural area, secondary and tertiary care, or nationally. NZHIS can conduct such analysis and then feed the results back to the local area. The local service verifies and interprets these results so that it can incorporate the information into clinical and managerial decision-making and quality improvement. Before analysis NZHIS ‘cleans up’ the data by removing events with duplicates or overlapping events, coding errors, unacceptable or non-specific principal diagnoses, inappropriate diagnoses for the person (eg, age- or gender-specific diagnoses), unsuitable cause of death, or diagnoses that are unusual in New Zealand. NZHIS can also use Victorian trim points to identify events outside expected levels and to trim outliers so that they do not skew statistics. An example of a recent in-depth analysis by NZHIS is its examination of the quality of fracture of neck of femur procedures across all hospitals. This analysis was drawn from data collected in 66 MOH - TCE 2002 5/1/02 9:22 AM Page 67 the National Minimum Dataset (NMDS) for the 1999/2000 year and is being reviewed by the New Zealand Orthopaedic Association. The indicators reviewed were quality, readmission, complications/misadventures, mortality rates, access to hospital, access to surgery, and the impact of delay in time to surgery. For mortality, two patient treatments were compared – conservative versus surgical. Clinical practice was also considered – reduction and fixation, hemiarthroplasty or total hip replacement. Clinicians have been consulted throughout the analysis to ensure accurate interpretation of the data – a very important element of the analysis. On completion of the analysis a report will be written, endorsed by the New Zealand Orthopaedic Association, which will then be distributed to all DHBs. Rather than giving detailed conclusions, the report will raise questions. Only the organisations and clinicians concerned can draw the appropriate conclusions. The national collections that NZHIS maintains and manages are a vast source of data that can be used for analysis and audit. They are: • National Health Index (NHI) – a mechanism for identifying every health care user by assigning them a unique identifying number. As a registration system, the NHI includes only the information needed to identify health care users, such as name, address, date of birth, ethnicity and gender. 67 Appendix D: NHI Datasets Held by NZHIS National data from the NMDS have also been used to develop a standard set of indicators of secondary care to provide DHBs with benchmarking information at the level of Diagnosis Related Group (see Table D1). This set can be viewed at various levels of detail across different groupings. NZHIS has developed a tool that takes 12 months of NMDS data and produces indicators at a relevant level for benchmarking across hospitals. It is intended to encourage the analysis of the quality of hospital services. This first dataset has been cut to CD and distributed to all DHBs. Further datasets will be distributed in the future. It is emphasised that the indicators only give a high-level picture of each service; a full evaluation of quality of care requires an in-depth analysis. MOH - TCE 2002 5/1/02 9:22 AM Page 68 • National Minimum Dataset – a single integrated collection of secondary and tertiary health data for New Zealand. It collects information about all inpatients and day patients to both public and private hospitals. The data cover diagnoses, and diagnostic and therapeutic procedures as well as some demographic information about the health care user. • National Cancer Registry – a population based tumour register of all primary malignant diseases, except basal-cell and other ‘simple’ skin cancers. It has operated since 1948. Data cover the site, stage, and pathology of the cancer as well as demographic information. • National Mortality Registry – a register of all cause of death data based on the legal death certificate or coroner’s report, together with autopsy reports. The data cover the underlying cause of death, and demographic information about the deceased. Toward Clinical Excellence • Mental Health Information National Collection (MHINC) – a database containing information on secondary mental health and alcohol and drug services. This database is not fully populated yet so it is not used for quality analysis. • National Reporting Booking System (NBRS) – a database on all health events where a patient has received an assessment of priority for a medical or surgical service and is accepted for publicly funded treatment. Booking information can be linked to the actual procedure when it is undertaken as each hospital attaches its patient management system unique event identifier when submitting the data to NZHIS. This database is also not fully populated at this stage. At present for analytical purposes it is possible to draw on only the NHI, NMDS, Cancer Registry and Mortality Registry collections. The records in each collection can be linked to each other using the NHI number, so a person can be tracked through the cancer registry, secondary care treatment and even death. 68 MOH - TCE 2002 5/1/02 9:22 AM Page 69 NZHIS also maintains the following primary care warehouses: • Pharmhouse – pharmacy claim data. • Labs – laboratory claim data. • Immunisation – immunisation claim data. • Maternity and Newborn Information System (MNIS) – integrated data related to pregnancies from March 1998. • Hepatitis B Screening Programme (Hep B) – primary care and secondary care information for the Hepatitis B Screening Programme pilot, which is being run for a three-year period. Table D1: Standard set of indicators Heading Description Diagnosis Related Group DESCR DRG description (*=NSW tertiary DRG annotation; +=NZ tertiary DRG annotation) NO Number of events NOA Actual number of patients with the same NHI NOR Number of events per number of patients AGE Average age SEXM Sex – males SEXF Sex – females ETHNM Ethnicity – Maori (21) ETHNPI Ethnicity – Pacific Island (30–37) ETHNO Ethnicity – other (10–12 and 40–54) ETHNNS Ethnicity – not stated (99) ADMR Admission source – routine rate ADMT Admission source – transfer from another hospital facility rate ADMWN Admission type – waiting list (WN) + arranged (AA) rate ADMZ Admission type – ACC covered (ZW, ZA, ZC, ZP) rate ADMAC Admission type – acute (AC) rate Appendix D: NHI Datasets Held by NZHIS DRG 69 MOH - TCE 2002 5/1/02 9:22 AM Toward Clinical Excellence Heading Page 70 Description DISDD Discharge type – died (DD) rate DISDS Discharge type – self discharged (DI, DS) rate DISDT Discharge type – transferred to another facility (DT, DP), acute specialist facility (DA) or other department within same facility (DW) rate DISDC Discharge type – day cases (DR, DC, DL, DN and LOS=0) rate DISDR Discharge type – ended routinely (DR, DC, DL, DN and LOS>0) rate ANZDEP NZDep96 index of deprivation (interval variable mapped against domicile code) HOLOS Special circumstances (V60 code) rate DXREC Average number of codes per record MDC Average number of disease categories per record (ICD-9, total of 17) CC Average number of complications/co-morbidities per record (as defined by the ICD-9-CM classification system) OUTL Low outliers rate (below the Victorian low trim point) OUTH High outliers rate (over the Victorian high trim point) LELOS Patients with leave days rate DAYC Day cases rate LOSD Average length of stay – overall including day cases TLOSD Truncated average length of stay – overall including day cases (Victorian trim points) ACCL Average CCL (complication/co-morbidity level). Complication/co-morbidity class level (CCL) is a field calculated and output by the grouper as a result of an evaluation of all coded secondary diagnoses against the complication/comorbidity class (CC) list, the CC exclusion list and the appropriate CC tables. A list of highly correlated codes is used to exclude certain codes from CC status, hence a secondary diagnosis code that occurs on the CC list can be demoted to non-CC status if the secondary diagnosis is too closely aligned with the principal diagnosis. There are three levels for the patient’s clinical severity in each medical DRG and four levels in each surgical DRG: 70 MOH - TCE 2002 5/1/02 9:22 AM Page 71 Heading Description • Level 0 = no secondary diagnosis that is CC • Level 1 = at least one secondary diagnosis that is a minor CC (but none is more than minor) • Level 2 = at least one secondary diagnosis that is a moderate CC (but none is more than moderate) • Level 3 = at least one secondary diagnosis that is a major CC (but none is more than major) • Level 4 = at least one secondary diagnosis that is a catastrophic CC (surgical partition only). ACOMBW Average combined weights. This measure is determined for recognising the patient’s increased severity and case complexity by an adjustment methodology on the basis of CCLs, length of stay, and the patient’s cost data. It provides the assessment of inter-DRG variations, ie as higher ACOMBW, higher case complexity. Average Victorian cost weight SULOS Procedures not carried out rate (V64) RAMDC MDC related same hospital readmission rate. An admission is classified as a readmission if the patient returned to the same hospital within 30 days of the original discharge. The exception to this definition include: Appendix D: NHI Datasets Held by NZHIS ACOSTW • discharges with an event end type of DD (discharged dead) or DI/DS (self discharge) or DT/DP/DA/DW (transfers) followed by another admission within 24 hours • admissions with an admission type of WN/ZW (waiting list) or AA/ZA (arranged) or AP/ZP (private). For those events classified as readmissions, an attempt was made to distinguish between ‘related’ (at MDC and DRG levels) and unrelated readmissions. The total MDC readmission rate includes both MDC and DRG related readmissions. TCOMPL Total complication rate (996–999 range) TMISAD Total misadventure rate (E870–876 range) TDEATH Total death rate 71 MOH - TCE 2002 5/1/02 9:22 AM Appendix E: Page 72 Determining Levels of Evidence The following three methods are generally accepted approaches to determining the strength of a given body of evidence. Sackett’s criteria for the grading of evidence (Sackett 1989) A study’s evidence may be ranked at one of five levels. Level I study: Large randomised trials, statistically significant, clear-cut results (low risk of error). Toward Clinical Excellence Level II study: Small randomised trials with uncertain results: can identify trends but not significance (moderate to high risk of error). Level III study: Non-randomised, concurrent cohort comparisons of contemporaneous patients (bias toward compliance theory – patients who are compliant with treatment influence clinicians’ perceptions about clinical efficacy). Level IV study: Non-randomised historical cohort comparisons between current patients who did receive therapy and former patients who did not (from same institution or from the literature). Level V study: Case series without controls – description of a group of patients. 72 MOH - TCE 2002 5/1/02 9:22 AM Page 73 Revised SIGN grading system (2000) Levels of evidence 1++ High quality meta analyses, systematic reviews of RCTs, or RCTs with a very low risk of bias 1+ Well conducted meta analyses, systematic reviews of RCTs, or RCTs with a low risk of bias 1 – Meta analyses, systematic reviews of RCTs, or RCTs with a high risk of bias 2++ High quality systematic reviews of case-control or cohort or studies. High quality case-control or cohort studies with a very low risk of confounding, bias, or chance and a high probability that the relationship is causal 2+ Well conducted case control or cohort studies with a low risk of confounding, bias, or chance and a moderate probability that the relationship is causal 3 Non-analytic studies, eg, case reports, case series 4 Expert opinion Grades of recommendation A At least one meta analysis, systematic review, or RCT rated as 1++, and directly applicable to the target population; or A systematic review of RCTs or a body of evidence consisting principally of studies rated as 1+, directly applicable to the target population, and demonstrating overall consistency of results 73 Appendix E: Determining Levels of Evidence 2 – Case control or cohort studies with a high risk of confounding bias, or chance and a significant risk that the relationship is not causal MOH - TCE 2002 B 5/1/02 9:22 AM Page 74 A body of evidence including studies rated as 2++, directly applicable to the target population, and demonstrating overall consistency of results; or Extrapolated evidence from studies rated as 1++ or 1+ C A body of evidence including studies rated as 2+, directly applicable to the target population and demonstrating overall consistency of results; or Extrapolated evidence from studies rated as 2++ D Evidence level 3 or 4; or Extrapolated evidence from studies rated as 2+ Joanna Briggs quality of evidence ratings These ratings have been adapted from the National Health and Medical Research Council guidelines for the development and implementation of clinical practice guidelines (1995). Toward Clinical Excellence Level I Evidence obtained from a systemic review of all relevant randomised, controlled trials Level II Evidence obtained from at least one properly randomised, controlled trial Level III Evidence obtained from well designed, controlled trials without randomisation, or evidence obtained from well designed cohort or case control analytic studies, preferably from more than one centre or research group, or evidence obtained multiple time series, with or without the intervention. Dramatic results in uncontrolled experiments, such as the results of the introduction of penicillin treatment in the 1940s, could be regarded as Level III evidence Level IV Opinions of respected authorities, based on clinical experience, descriptive studies or reports of expert committees 74 MOH - TCE 2002 5/1/02 9:22 AM Appendix F: Page 75 Uses of Retrospective, Prospective and Concurrent Audit If the data are already collected a retrospective audit might be appropriate. It is important to check that the data have been collected accurately and completely before commencing the full audit. If no data or insufficient data are available or practice has changed, you may wish to start a prospective audit or concurrent audit. Table F1 outlines the advantages and disadvantages of each of these approaches. Table F1: A comparison of retrospective, prospective and concurrent audits Timing of audit Retrospective Advantages • Ability to review many records of patients and/or documents • Cost-effective option • Good data available • May not be suitable for rapidly changing treatment/ technologies • Can determine data • Potentially more to be collected, and expensive than quality of collection retrospective audit for similar-sized • Quick and accurate sample identification of • Useful for evaluating audit topics where data are not retrievable by diagnostic • Data collectors may coding methods and in high need more training cost/high risk situations • May be more accurate because data are collected as audited care is provided • Potentially more expensive than retrospective audit for similar-sized sample • Opportunity to intervene to make clinical care improvements as these are discovered • Data collectors may need more training • Used where data are not normally collected and for which no system exists 75 and Concurrent Audit cases to study. Prevents lengthy tracking after discharge Concurrent (open audit) Application • Large sample Appendix F: Uses of Retrospective, Prospective Prospective Disadvantages • Data may be incomplete or inaccurate MOH - TCE 2002 5/1/02 9:22 AM Appendix G: Page 76 Balanced Scorecard Approach to Performance Management Toward Clinical Excellence Robert Kaplan and David Norton developed the Balanced Scorecard concept in 1992. It translates an organisation’s mission and strategy into a comprehensive set of performance measures, and links improved quality with increased financial performance. The traditional key performance dimensions include finance, process and efficiency, patient and quality, and organisational health. The relevance of the Balanced Scorecard to clinical audit is that standards in audit can include a variety of process and outcome measures, including patient perceptions of care, quality measures such as clinical appropriateness and access, and financial parameters. The rationale for this approach to performance measurement in the health sector is described in detail in the Balanced Scorecard Performance Indicators for New Zealand Public Hospital and Health Indicators: Hospitals Monitoring Directorate (October 2000, www.moh.govt.nz). The Balanced Scorecard approach reflects the four quadrants of operational effectiveness: 1 finance 2 productivity, which incorporates efficiency ie effectiveness without waste 3 clinical quality, which covers functional dimensions of quality, such as safety, appropriateness, consumer participation and access 4 delivery quality, which incorporates customer satisfaction (internal and external, including general practitioners), patient satisfaction (more accurately patient perceptions of care), and the organisational health factors, including staff satisfaction and turnover. 76 MOH - TCE 2002 5/1/02 9:22 AM Page 77 The patient and quality dimensions in the Balanced Scorecard include overall patient satisfaction with the hospital, as measured in the quarterly patient satisfaction survey of the DHB Funding and Performance Monitoring Directorate. There are two clinical indicators (emergency department triage times, and hospitalacquired blood stream infections) in the patient and quality quadrant of the Ministry of Health Balanced Scorecard, which is a set of measures collected quarterly by the Ministry of Health. to Performance Management Appendix G: Balanced Scorecard Approach 77 MOH - TCE 2002 5/1/02 9:22 AM Appendix H: Page 78 Directory of New Zealand Health Professional Groups The National Library of New Zealand (tepuna.natlib.govt.nz/web_directory/NZ/healthproforgan.htm) links to most health professional organisations in New Zealand, eg, colleges, societies (see list below). Toward Clinical Excellence Organisation Web address Association of Salaried Medical Specialists (ASMS) www.asms.org.nz/ Australasian Faculty of Occupational Medicine www.racp.edu.au/ afom/index.htm Australasian Faculty of Public Health Medicine www.afphm.org.nz/ Australasian Society for Infectious Diseases (ASID) www.racp.edu.au/ open/asid1.htm Australasian Society of Blood Transfusion (ASBT) www.asbt.org.au/ Australasian Society of Clinical and Experimental Pharmacologists and Toxicologists (ASCEPT) www.racp.edu.au/ open/ascept.htm Australasian Society of Clinical Immunology and Allergy (ASCIA) www.medeserv.com.au/ ascia/ Australia and New Zealand Dialysis and Transplant Registry (ANZDATA) www.anzdata.org.au/ ANZDATA/ anzdatawelcome.htm Australian and New Zealand College of Anaesthetists (ANZCA) www.medeserv.com.au/ anzca/ Australian and New Zealand College of Mental Health Nurses (ANZCMHN) www.voyager.co.nz/ ~anzcmhn/ Australian and New Zealand Intensive Care Society (ANZICS) anzics.herston.uq.edu.au/ Australian and New Zealand Society of Occupational Medicine Inc www.anzsom.org.au/ Australian and New Zealand Society of Nephrology (ANZSN) www.nephrology.edu.au/ 78 MOH - TCE 2002 5/1/02 9:22 AM Page 79 Organisation Web address Australian and New Zealand Society of Respiratory Science Inc. (ANZSRS) www.csu.edu.au/special/ anzsrs/ANZSRS.html Aviation Medical Society of Australia and New Zealand (AMSANZ) www.avmed.org.nz/ Cardiac Society of Australia and New Zealand www.csanz.edu.au/ Clinical Leaders’ Association of New Zealand (CLANZ) www.clanz.org.nz/ College of Nurses Aotearoa (NZ) www.nurse.org.nz/ Hauora Mäori www.healthsite.co.nz/ hauora_Maori/ Health Information Association of New Zealand www.northland.ac.nz/ HIANZ/index.html Health Promotion Forum of New Zealand www.hpforum.org.nz/ HEMNZ : Health Emergency Management New Zealand www.hemnz.org.nz/ index.html Human Genetics Society of Australasia www.hgsa.com.au/ Internal Medicine Society of Australia and New Zealand (IMSANZ) www.racp.edu.au/imsanz/ Marce Society Australasian Branch www.wairua.co.nz/marce/ Maternity Services Consumer Council www.maternity.org.nz/ Medical Council of New Zealand www.mcnz.org.nz/ Mutagenesis and Experimental Pathology Society of Australasia (MEPSA) www.mepsa.org/ National Society on Alcoholism and Drug Dependence New Zealand www.nsad.org.nz/ New Zealand Association of Natural Family Planning www.natfamplan.co.nz/ New Zealand Association of Neonatal Nurses (NZANN) www.nzann.org.nz/ New Zealand Association of Optometrists (NZAO) www.nzao.co.nz/ The New Zealand Branch of the Australian and New Zealand College of Mental Health Nurses www.voyager.co.nz/ ~anzcmhn/index.html New Zealand College of Midwives www.midwives.org.nz/ Groups www.anzsom.org.nz/ 79 Appendix H: Directory of NZ Health Professional Australian and New Zealand Society of Occupational Medicine (ANZSOM) MOH - TCE 2002 5/1/02 9:22 AM Page 80 Toward Clinical Excellence Organisation Web address New Zealand Drug Foundation www.nzdf.org.nz/ New Zealand Foundation for Cosmetic Plastic Surgery www.nzcosmeticsurgery. org.nz/ New Zealand Guidelines Group www.nzgg.org.nz/ New Zealand Medical Association (NZMA) www.nzma.org.nz/ New Zealand Microbiological Society www.nzms.org.nz/ New Zealand Nurses Organisation (NZNO) www.nzno.org.nz/ index.html New Zealand Private Hospitals Association www.nzpha.org.nz/ New Zealand Rheumatology Association www.rheumatology.org.nz/ New Zealand Society of Naturopaths Inc. www.naturopath.org.nz/ New Zealand Society of Otolaryngology – Head and Neck Surgery www.orl.org.nz/ New Zealand Society of Podiatrists Inc. www.podiatry.org.nz/ New Zealand Society of Physiotherapists www.physiotherapy. org.nz/ Nursing Council of New Zealand www.nursingcouncil. org.nz/ Nursing Informatics New Zealand Inc (NINZ) www.ninz.org.nz/ Nursing Research Section of the New Zealand Nurses Organisation www.nursingresearch. co.nz/ Nutrition Society of New Zealand www.nutritionsociety. ac.nz/ The Paediatric Society of New Zealand www.paediatrics.org.nz/ Pasifika Medical Association (NZ) Inc www.pacifichealth.org.nz/ Perinatal Society of Australia and New Zealand www.128.250.239.13/psanz/ Pharmaceutical Society of New Zealand www.psnz.org.nz/ Pharmacy Guild of New Zealand Inc. www.pgnz.org.nz/ Physiological Society of New Zealand www.otago.ac.nz/PSNZ Researched Medicines Industry Association of New Zealand Inc. (RMI) www.nzhealth.co.nz/rmi/ Royal Australasian College of Physicians (RACP) www.racp.edu.au/ 80 MOH - TCE 2002 5/1/02 9:22 AM Page 81 Organisation Web address Royal Australasian College of Surgeons (RACS) www.racs.edu.au/ Royal College of Pathologists of Australasia www.rcpa.edu.au/ Royal New Zealand College of General Practitioners www.rnzcgp.org.nz/ Thoracic Society of Australia and New Zealand (TSANZ) www.thoracic.org.au/ Transplantation Society of Australia and New Zealand Inc. (TSANZ) www.racp.edu.au/tsanz /index.htm Source: National Library of New Zealand Groups Appendix H: Directory of NZ Health Professional 81 MOH - TCE 2002 5/1/02 9:22 AM Appendix I: Page 82 Sources of Seed Guidelines and International Best Practice Websites Sources of Seed Guidelines Appraisal Instrument for Clinical Guidelines NZGG – New Zealand Guidelines Group US Agency for Healthcare Policy and Research (AHCPR) US Preventive Services Task Force SIGN – Scottish Intercollegiate Guidelines Network Canadian Task Force on the Periodic Health Exam American College of Physicians Toward Clinical Excellence Managed Care Organisations with Mature Guideline Programmes – Group Health Cooperative of Puget Sound – Kaiser Permanente 82 MOH - TCE 2002 5/1/02 9:22 AM Page 83 Evidence based practice websites: Name Internet address Description Bandolier www.jr2.ox.ac.uk/bandolier Oxford Evidence Based Medicine site Cochrane reviews www.cochrane.org Cochrane database of evidence based reviews Netting the evidence www.med.unr.edu/medlib/ netting.html Finding evidence based links The Trip Database www.tripdatabase.com Evidence based reviews and research US National Guidelines Clearing House www.guideline.gov/ index.asp Guidelines that have been logged from around the world, predominantly from the US Scottish guidelines UK Guidelines database www.eguidelines.co.uk/ and audit projects eguidelinesmain/index.htm (CLIP database) Database of UK guidelines and CLIP database of clinical audits completed (giving title and contact person only) Registration free www.ihi.org/ International Best Practice Websites Institute of Health Care improvement 83 Appendix I: Sources of Seed Guidelines and Scottish Intercollegiate www.sign.ac.uk/ online Guidelines Network (SIGN) MOH - TCE 2002 5/1/02 9:22 AM Page 84 References Arroll B, Bourchier R, Gelber P, et al. 2000. Algorithm for the management of chronic leg ulcers. New Zealand Guidelines Group. Australian Council of Health Care Standards. 2001. Clinical Indicators Users’ Manual. [Online] available at www.achs.org.au (accessed 18 February 2002). Beaglehole R, Bonita R, Kjellström T. 1993. Basic Epidemiology. Geneva: World Health Organization. Berwick DM. 1994. Developing and testing changes in the delivery of care. Ann Intern Med 128: 651– 6 Berwick D, Nolan T. 1998. Physicians as leaders in improving health care. Annals of Internal Medicine 128(4): 289–92. Berwick DM. 1998. Interaction of Clinical Practice Improvement and Evidence Based Medicine. Crombie IK, Davies HTO. 1998. Beyond health outcomes: the advantages of measuring process. J Evaln Clinical Practice. 4(1): 31–8 Toward Clinical Excellence Grimshaw J, Russell I. 1993. Achieving health gain through clinical guidelines. 1: Developing scientifically valid guidelines. Quality in Health Care 2: 243–8. Hofer TP, Bernstein SJ, Hayward RA, De Monner S. 1997. Validating quality indicators for hospital care. Jt Comm J. Qual Improv. 23(9): 455–67 Humphris D, Littlejohns D. 1999. Implementing Clinical Guidelines: A practical guide. UK: Radcliffe Medical Press. Institute for Healthcare Improvement. IHI Quality Improvement Resources. [Online] available at www.ihi.org/resources/qi/index.asp (accessed 15 February 2002). King, Hon Annette. 2000. The New Zealand Health Strategy. Wellington: Ministry of Health. Langley G, Nolan K, Nolan T, et al. 1966. The Improvement Guide: A practical approach to enhancing organisational performance. San Francisco: Jossey Bass. Lembeke PA. 1967. Evolution of the Medical Audit. J Am Med Assn 199: 111–18. 84 MOH - TCE 2002 5/1/02 9:22 AM Page 85 Ministry of Health. 2000. Overview and Definitions for Balanced Scorecard Performance Indicators. Wellington: Ministry of Health Mawson SJ, McCreadie MJ. 1998. Guidelines for the collaborative rehabilitative management of elderly people who have fallen. Physiotherapy 84(4): 173–7. Ministry of Health. 2001. Toward Clinical Excellence. A Framework for the Credentialling of Senior Medical Officers in New Zealand. Wellington: Ministry of Health. NHS Centre for Review and Dissemination. 1999. Getting evidence into practice. Effective Health Care 5(1): 1–16. New South Wales Health. 2001a. Indicators, Variation and Health Care Improvement: A guide for health care professionals. New South Wales: Department of Health. New South Wales Health. 2001b. The Clinician’s Toolkit for Improving Patient Care. New South Wales: Department of Health, September. Royal Australasian College of Physicians. Official website. [Online] available at www.racp.edu.au (accessed February 2002). Sackett DL. 1989. Rates of evidence and clinical recommendations on the use of antithrombic agents. Chest 95(2), Suppl 25–45. Scottish Intercollegiate Guidelines Network. 2002. Sign 50: A guideline developer’s handbook. [Online] available at www.sign.ac.uk (accessed February 2002). United States National Guidelines Clearing House. Expert Panel Report 2: Guidelines for the diagnosis and management of asthma. United Bristol Healthcare NHS Trust. A Practical Guide to Clinical Audit. [Online] available at www.ubht.org.uk (accessed 8 December 2001). 85 References Royal College of Nursing. 1998. Clinical Practice Guidelines: The management of patients with venous leg ulcers. York: RCN Institute. Notes MOH - TCE 2002 86 5/1/02 9:22 AM Page 86 MOH - TCE 2002 5/1/02 9:22 AM Page 87 Notes 87 Notes MOH - TCE 2002 88 5/1/02 9:22 AM Page 88