Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Quality of decision-making by the Pharmaceutical Benefits Advisory Committee (PBAC) and the impact on outcomes Neville A M, Lloyd J M. Pretium (Pharmaceutical and Health Economic Consultants) Sydney, Australia. Introduction In Australia, medicines are registered and approved for use by the Therapeutic Goods Administration (TGA) and the Australian Drug Evaluation Committee (ADEC). The TGA carries out a range of assessment and monitoring activities to ensure that all therapeutic goods available in Australia are of an acceptable standard. At the same time, the TGA aims to ensure that the Australian community has access, within a reasonable time, to therapeutic advances. The ADEC was formed in 1963 and given the role of providing independent, scientific advice on new drugs, within the policy framework of the time, to the Federal Government. The ADEC is appointed by the Minister for Health and Ageing and provides advice to the Minister and the Secretary of the Commonwealth Department of Health and Ageing through the TGA, on a number of matters including medical and scientific evaluations of applications for registration of prescription drugs (e.g. new chemical entities, new forms of previously registered drugs and therapeutic variations to registered drugs). Figure 1 displays the process involved in submitting for and obtaining registration and marketing approval in Australia. Figure 1 The processes involved in submitting for and obtaining registration and marketing approval in Australia. Given the complexity of the analyses that support submissions to the PBAC, we sought to determine the quality of the PBAC evaluations (as determined by the sponsors of the applications), their effect on decision-making and the outcomes of PBAC meetings. The questionnaire was divided into three sections; each explored one area of interest. The first section (Section A) was designed to elicit examples of good decision-making by the PBAC. The survey asked respondents to provide examples of a good decision that was experienced by their company. Detail was requested on: Glossary of Terms ADEC DUSC ESC PBAC PBS PES PSC RWC TGA Australian Drug Evaluation Committee Drug Utilisation Sub-Committee (of the PBAC) Economics Sub-Committee (of the PBAC) Pharmaceutical Benefits Advisory Committee Pharmaceutical Benefits Scheme Pharmaceutical Evaluation Section Pharmaceutical Sub-Committee (of ADEC) Restriction Working Committee Therapeutic Goods Administration 1st submissions Resubmissions Total Good decisions 26 (67%) 6 (33%) 32 (56%) Poor decisions 13 (33%) 12 (67%) 25 (44%) 39 18 Total the type of economic argument used in the submission, whether a positive or a negative recommendation was the outcome, the reason/s for the decision qualifying as good, and if the price agreement post recommendation equalled the submission price. The same detail was requested in the second section (Section B), but focussing on poor decision-making by the PBAC. Respondents were asked in Section C to quantify the decisions made by the PBAC for all submissions made during the 6 meetings of interest. Specifically, quantitative detail was requested on: 1st submissions, From sections A and B of the survey, detail was requested about one or two individual submissions, one good and one poor. From this detail (from 12 companies) it is hypothesised that the quality of the PBAC decision was related to the form of economic argument taken. From the feedback received the likelihood of the evaluation being good was 69% (9/13) if the submission took a cost minimisation approach. In contrast, the likelihood of an evaluation being good given an incremental cost effectiveness approach was only 38% (3/8), Table 3. Table 3 Quality of decision making by type of economic argument used (numbers and percentages) resubmissions, and the maximum number of submissions until a medicine was successfully listed on the PBS. Results Source: Australian Draft Regulatory Guidelines for Prescription Medicines, December 2003 Figure 2 The processes involved in submitting for and obtaining public subsidy in Australia. Sponsor – lodges Major Submission Preparation of DUSC Secretariat Commentary Preparation of PBAC Secretariat Overview A total of 35 questionnaires were sent and 17 replies were received, a response rate of 48%. Five of these respondents had not prepared any submissions to the PBAC in the period of study. Thus 12 companies were able to provide details of their submissions. The 17 companies had sales in the period of analysis that represented 47% of total pharmaceutical sales [as assessed by IMS data, API and AHI]. From section C of the questionnaire, the total number of submissions made to the PBAC within the survey period was 57, which is approximately 40-45% of all submissions. Of these, 39 were first submissions and 18 were resubmissions. Of the total of 57, 35 submissions resulted in a PBS listing (61% successful). Of the total submissions, 39 were first submissions (68% of the total). Of the total successful submissions, 26 were first submissions, a success rate of 67% for this group. Resubmission had a lower success rate – 9 of 18 submissions resulted in PBS listing (50%). Preparation of PES Commentary Poor Decisions Total Cost minimisation 9 (69%) 4 (31%) 13 Cost effectiveness 3 (38%) 5 (63%) 8 Total 12 (57%) 9 (43%) PBAC Secretariat Table 4 26 (66%) 9 (50%) 35 (61%) Unsuccessful submission 13 (34%) 8 (50%) 22 (39%) 39 18 Sponsor – for Pre-sub-committee Response Total Successful submission Unsuccessful submission Cost minimisation 12 (92%) 1 (8%) 13 Cost effectiveness 5 (63%) 3 (38%) 8 17 4 Total Total Reasons for decisions being poor DUSC Meeting for DUSC Adviser Restriction Working Group for RWC Advice ESC Meeting for ESC Advice PBAC Secretariat Sponsor – for Pre-PBAC Response PBAC Secretariat PBAC Meeting Source: Source and flow of documents for major submissions to the PBAC (17 week cycle) (www.health.gov.au/pbs/pharm/listing/subchart.htm) From the detail in Section C of the questionnaire, which formed a sample of 57 submissions, industry considered that good decisions were made by the PBAC 56% of the time (32/57). This figure ranged from 67% for first submissions to only for 33% for resubmissions. Thus, industry considered that poor decisions were made by the PBAC in 44% of submissions, 33% for first submissions and 67% of the time for resubmissions, Table 2. The low quality of decision-making in the resubmission process resulted in many repeated resubmissions. One company reported that it had taken 6 submissions until a medicine was successfully listed on the PBS. The sample median was 2, where a figure was provided. “Conflicting advice [was received from the Secretariat] on appropriate comparator that led to delays in listing.” “We were forbidden from using the market leader as the comparison. We used another product as comparator; our data were not believed, then we resubmitted using a common comparator to another agent and, ultimately, were given listing at the market leader price which was not the comparator in our submission.” “PBB advised [the sponsor] that drug A was the appropriate comparator. PBAC stated that the comparator could equally have been another type of inhibitor or dual inhibitor regimens even though none of these have regulatory approval.” 2. Inter Agency Inconsistency Another major issue of concern was that decisions were incorrectly interpreted results in the favour of the Pharmaceutical Benefits Branch, even ignoring the PBAC’s own published guidelines. There were instances where very large nonrandomised trials were not considered in the evaluation, valid epidemiological data was ignored or not accepted or there were problems regarding the definition of ‘therapeutic equivalence’. These flaws could result in outcomes that contradicted Therapeutic Goods Administration findings. “TGA/ADEC had accepted that the drug was effective making some allowance for the Orphan Drug status, but on the same data the PBAC did not accept that the drug was effective.” “The quality of submission was criticised because only the preliminary report of the pivotal study was provided. This was in spite of the fact that the product was fast-track evaluated by the TGA and the same report was used to approve product for marketing.” Two main issues were apparent where the policy set out by the Department was problematic and not always optimal. The use of an ‘evidence-based’ approach to developing and evaluating the submissions could require evidence that would be impossible to prove. Also, the uncertainty over the value of Quality of Life measures and the related issue of cost utility analyses is an area that needs improvement and clarification. Others cited costing issues as a concern. “The pack size recommended was not consistent with evidence based practice.” “Quality of life results were compared to baseline levels rather than the difference between treatment groups (when the guidelines stipulate comparison should be between treatment groups).” 4. Issues of Expertise Total Successful submission “The comparator’s dose was not accepted – PBAC wanted bd versus tid (with no evidence cited by PBAC).” 3. Issues of Policy and Process It is also hypothesised that the success of a first submission was related to the form of economic argument taken. From the feedback received the likelihood of the submission being successful was 92% (12/13) if the submission took a cost minimisation approach. In contrast, the likelihood of an evaluation being successful given an incremental cost effectiveness approach was 63% (5/8), Table 4. Type of decision made by submission / resubmission (numbers analysed and percentages) 1st submissions Resubmissions PBAC Secretariat Good Decisions Probability of success by type of economic argument used (numbers and percentages) Table 1 Method A good decision was defined as one that incorporates the evidence within a sound analytical framework. In this case, the framework for analysis is the Guidelines for the Pharmaceutical Industry on Preparation of Submissions to the PBAC2. The following criteria were used to define why, when and how the decisions made were good in quality: evidence was systematically included by the evaluators, and A poor decision was defined as one where the Guidelines are not followed or were poorly interpreted. The reasons why the applicant considered the evaluation (with specific examples where possible) were provided. PBAC Secretariat A survey was conducted to determine pharmaceutical industry’s experience regarding PBAC decision-making over a period of six PBAC meetings. The survey period was stipulated as the 4 meetings of the PBAC in 2000 and the first two (March and June) of 2001. The membership of the PBAC changed after the June 2001 meeting, which would allow ongoing survey work to compare the changes over time and by different Committees. The questionnaire was designed to elicit information on good and poor decisions, and information needed to quantify the issues and their effects on submission outcomes. Type of decision made by submission / resubmission (numbers analysed and percentages) comparator/s were within guidelines and current practice. However, the Federal Government will only subsidise a drug if the Pharmaceutical Benefits Advisory Committee (PBAC) evaluation has determined the drug to be cost effective. The PBAC was established in 1954 under section 101 of the National Health Act (1953) to make recommendations and give advice to the Federal Minister of Health about which drugs and medicinal preparations should be made available as pharmaceutical benefits on the Pharmaceutical Benefits Scheme (PBS). The PBS is a national (taxpayer funded) scheme that provides subsidised access to necessary medicines. Note that medicines/medicinal preparations can only be listed on the PBS for their registered indications (as determined by the TGA & ADEC). Following an amendment to the Act in 1987, which took effect in 1993, the Committee has been required by the Act to consider the effectiveness and cost of a proposed drug or medicinal preparation compared to alternative therapies. Guidelines to assist those preparing submissions for consideration by the PBAC (in most cases, the local pharmaceutical industry) have been in existence since 1991 (with subsequent revisions). Figure 2 displays processes involved in submitting for and obtaining public subsidy (PBS listing) in Australia. The content of such submissions and their outcome (recommended, rejected or deferred) had not, until recently, been made public. However, the government reviewed the quality of pharmaceutical industry pharmacoeconomic analyses included in PBAC submission documents and reported that significant problems existed1. It is likely, however, that, given the complexity of many of the interventions and the analyses that support the submission, the PBAC evaluation process itself could contain errors. This is problematic as there is no process for appeal decisions with which sponsors disagree. Rather, resubmissions to the same Committee, following the same processes, is required. Table 2 guidelines were followed by the Pharmaceutical Benefits Branch in the evaluation of submissions, A number of reasons were identified as problematic and lowering the quality of evaluations and decision-making. These have been grouped into 4 areas by the researchers and are: Companies recognised that the submissions were sometimes complex and required the development of considerable expertise internally to understand and then present the issues. However, the same level of expertise was not always demonstrable by the evaluators. An example of when this becomes overt is in the choice of ‘old’ or out-dated surrogate outcomes that are not the best measure of the benefit of new interventions. “The primary outcome of a pivotal study was the percent of patients with undetectable virus. This is recognised by most experts, including US treatment guidelines, as the best surrogate indicator of future disease outcomes. PBAC asked the sponsor to justify why an older and now outdated marker should not have been used instead.” “[The PBAC] did not accept the comparator dosage which we submitted (tds dose) because they felt that the most commonly used dosage was bd (no evidence cited); they did not allow a cost offset for pathology testing for drug prophylaxis because they felt that such testing is not necessary (no evidence cited); they approved a pack size (5 injections) which is less than the minimum duration of treatment (6 days).” 1. Comparator Issues Many respondents cited comparator issues as a source of concern. This group ranges from issues over the choice of comparator made by the evaluators, which was often neither pragmatic nor evidence-based, to the age of comparators (when the low price of generic comparators would result in returns being inadequate given the costs of developing innovative new products). Confusion could arise when current comparator usage may not reflect approved usage, or doses were not accepted. Companies often received conflicting advice from the PBAC on the appropriate comparator to use over the course of a submission and subsequent resubmission. Also, opinion over the most appropriate comparator could differ between the PBAC and the PBPA. Conclusions Government subsidy decision-making is of variable quality, which varies depending on the method of economic argument used and affects the probability of success for submissions. A process to appeal decisions or have them reviewed by an independent body should be considered. 1 Problems With the Interpretation of Pharmacoeconomic Analyses A Review of Submissions to the Australian Pharmaceutical Benefits Scheme. Hill, SR et al. JAMA. 2000;283:2116-2121. 2 Commonwealth Department of Health and Ageing, Guidelines for the Pharmaceutical Industry on Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee, September 2002.