Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Managing for Change: The Art of Results Based Management Module 3 - Risk and Performance Measurement Mosaic Workshop - Ottawa July 14th - 18th, 2008 Enablers and Constraints… • Are forces affecting any part of your project both positively and negatively Why Identify Assumptions? • Programs/projects are implemented in a dynamic environment • Factors external to the project are often the cause of failure • Make explicit the conditions external to the program/project upon which its internal logic is dependent Test Your Design Logic OUTCOME Participating citizens increasingly challenge intolerance, racism and hatred in their communities. OUTPUT Individuals and groups gain awareness of racism and discrimination. Internal Logic What conditions are you assuming will exist in the operation’s environment? External Logic Risk • • Risk is the uncertainty that surrounds project results. It is the likelihood and impact of an event with the potential to influence the achievement of results. Risk Analysis Matrix Project Example… Internal Staff Training on Public Consultation Practice Outcome: Local Government Authority staff consistently apply public consultation guidelines in the implementation of their unit responsibilities; they do so with a solid grasp of the subject matter and with a command of the leading techniques of engagement 1. What could hamper progress? 2. What is the likelihood of a problem? 3. What effect would the problem have on the Outcome? Effects Significant (3) Moderate (2) Minor (1) Monitor Risks Risk Mitigation Required Risk Mitigation Required Acceptable Risks Monitor Risks Risk Mitigation Required Acceptable Risks Acceptable Risks Monitor Risks Low (1) Medium (2) High (3) Likelihood Risk Mitigation & Monitoring • • Risk mitigation strategies should reduce the likelihood of an undesirable event, or minimize the effect the event could have on the project. Risk areas that remain above the level of acceptable risk should be monitored. Workshop Exercise - Risk Management 1. 2. 3. 4. 5. Develop your matrix Pick one output and one outcome. Define your assumptions. Define your risks and and conduct risk analysis Consider mitigation and monitoring strategies Assumptions Assumptions, Risk Identification and Mitigation Strategy Template Risks Risk Ranking/Rating Likelihood/Effect (L/M/H) (L/M/H) Mitigation Strategy k Identification and Mitigation Strategy Template cted Result er LFA) ut: omes: ct: Assumptions Risks Risk Ranking/Rating Likelihood/Effect (L/M/H) (L/M/H) Mitigation Strategy Risk Identification and Mitigation Strategy Template Expected Result (as per LFA) Output: Outcomes: Impact: Assumptions Risk Ranking/Rating Likelihood/Effect (L/M/H) (L/M/H) Mitigation Strategy Keeping Track of Results • Shifting from the design and planning side of RBM to information gathering Monitoring and Evaluation - The Distinction • Monitoring A continuous self-assessment of progress -collecting data and comparing current performance with planned activities and results. • Evaluation Independent (often external), periodic, strategically focused assessment of a project’s continuing relevance, management, governance, results, coverage, external relationships, sustainability, etc. Keeping Track of Results… • Means paying attention to … – Indicators – Sources of Information – Methods and Frequency – Person(s) to do the Work – Who needs to know Indicators… • Point to information you need to determine if the project is creating the results you want. • They can be both quantitative and qualitative. • They should help the project make wise management decisions. • They should be cost effective to use. Quantitative Indicators Expressed as a percentage, ratio, proportion, comparison, or number that can be analyzed statistically Examples? –Number of households represented on Village Development Committees –Ratio of men to women in senior management roles in the partner organization –Proportion of households using servicable latrines –Number of community organizations committing resources to address hate crimes in ‘x’ community Quantitative Indicators Your Turn Climate Change Conference – Stakeholder Dialogue on Provincial Action Plan Outcome - Public discourse on the Province’s Climate Change Action Plan Update peaks through community organized events ? Qualitative Indicators Expressed as a change or comparison between two states of understanding or experience that can be analyzed for patterns Relies on people’s judgment or perception Examples: –Villager (men and women) stories of how their lives have changed following completion literacy training –Change in the level of rapport between local government and higher levels of government. –Change in community perceptions of the role/efficacy of Village Development Committees –Before-after comparison in the coverage of race relations issues in the local media Qualitative Indicators Your Turn Climate Change Conference – Stakeholder Dialogue on Provincial Action Plan Outcome - Public discourse on the Province’s Climate Change Action Plan Update peaks through community organized events ? Indicators, Baseline, and Target Meaning Example Indicator The information to be collected Proportion of school children routinely washing their hands after using the latrine/washroom Baseline What the indicator would tell you at the beginning 1 in 5 Target What you want the indicator to tell you at a specific point in the future 4 in 5 Qualitative Indicators - Setting out Markers of Progress • Sometimes you cannot meaningfully track progress by counting “units” at the beginning (baseline), middle and end (e.g. ratio of local government units/service areas that have institutionalized public involvement as a way of working) • You may need to set out… – Stages of completion, or – Markers of progress Qualitative Indicators - Setting out Markers of Progress • Consider the policy formulation… – Indicator is ‘Comparison of actual to planned progress in updating ‘x’ policy’ • You could say… – Baseline - ‘0’ updated policy – Target - ‘1’ updated policy • But how useful is that from a management point of view? Qualitative Indicators - Setting out Markers of Progress • Would be more useful to identify progress markers. For example: a) b) c) d) e) f) • • • Established need for updated policy Policy research underway Policy options formulated Discussion and approvals underway Updated policy approved Updated policy enacted Baseline might be ‘a’ Yr 1 target might be ‘c’ Yr 3 target might ‘e’ Indicator Selection Checklist • Validity - Does it measure the result? Yes No • Is it cost-effective to collect the information? Yes No • Does it help understand how the program or project is affecting men and women, and/or specific sub-groups of people differently? Yes No Yes No Yes No • Does it provide useful information for management decisions? • Does it communicate well to stakeholders (public, policy makers, donors)? Test Case #1 Indicator: a) Change in the average amount of production lost between field and market among participating farmers b) Proportion of participant farmers who can describe how they will replicate handling procedures following the training New skills through training - Output Participant farmers understand post harvest handling procedures and are willing to try these out on their own after the training. • Validity - Does it measure the result? Yes No • Is it cost-effective to collect the information? Yes No • Does it help understand how the program or project is affecting men and women, and/or specific sub-groups of people differently? Yes No Yes No Yes No • Does it provide useful information for management decisions? • Does it communicate well to stakeholders (public, policy makers, donors)? Test Case # 2 Climate Change Conference – Stakeholder Dialogue on Provincial Action Plan Outcome - Public discourse on the Province’s Climate Change Action Plan Update peaks through community organized events Indicator: Pick your favourite from your group’s list of quantitative and qualitative indicators. Test it… • Validity - Does it measure the result? Yes No • Is it cost-effective to collect the information? Yes No • Does it help understand how the program or project is affecting men and women, and/or specific sub-groups of people differently? Yes No Yes No Yes No • Does it provide useful information for management decisions? • Does it communicate well to stakeholders (public, policy makers, donors)? Test Case #3 Household Participation in Community Decisionmaking - Outcome Households participate more fully in community decisionmaking and improvement initiatives; they help each other out when in trouble Indicator: Reported change in the way households respond to common hardships (e.g. abusive relationships, financial shortfalls, childcare, infirmity) • Validity - Does it measure the result? Yes No • Is it cost-effective to collect the information? Yes No • Does it help understand how the program or project is affecting men and women, and/or specific sub-groups of people differently? Yes No Yes No Yes No • Does it provide useful information for management decisions? • Does it communicate well to stakeholders (public, policy makers, donors)? Performance Measurement Plan Results Levels Impact Outcome Output Indicators Base -Line Target Data Source/ Report Method Frequency & Resp. Exercise • Prepare a Matrix of the Performance Measurement Framework • Select at least one output and one outcome. • Brainstorm a total of 3 indicators per result: 1 quantitative and 1 qualitative indicator and 1 other indicator. • Develop your sources of information, methods, responsibility and frequency.