Download Measured Approach TEGV Analysis

TEGV Assignment #1
Trevor Cook
PA 745
March 12, 2015
1
Introduction:
The purpose of this paper is to examine a case study conducted by Harvard
Kennedy School about the Educational Volunteers Foundation of Turkey (TEGV) and
identify issues of nonprofit management the organization has undergone. This analysis
will be viewed under the lens of: strategic planning, or the process of producing a
strategic plan; performance management, the ongoing process of collecting and analyzing
data to track program effectiveness; program evaluation, a method used to determining
program effectiveness in achieving results; efficiency, which measures the relationship
between resources and results; and effectiveness, or how well a program is able to
provide a service to meet its objectives (LeRoux & Wright, 2010, p. 574; Worth, 2014,
pp. 135 & 160). Following this introduction, there will be a brief overview of the case
study along with problems and key issues presented within the case study. The section
following the overview will provide recommendations as to what TEGV can do in order
achieve results.
TEGV Overview
The Educational Volunteers Foundation of Turkey (TEGV) has been providing
services for 20 years with the purpose of, “supporting the primary education provided by
the government” (Varley, 2014, p. 13). TEGV has several volunteer lead programs that
provide children ages 7-16 with many opportunities go interact with each other and grow
in both a mental and emotional capacity. As of 2012, TEGV has served just over 1.43
million children and has had over 9,000 volunteers help at their sites. The organization
has made a reputation for itself by building performance measurement tools into their
programs at the programs creation. The volunteers who run the programs complete the
2
performance assessments. In order to save time and resources, TEGV constantly
evaluates their programs. If any program is not producing the outcomes they hope to see,
the program gets restructured or cut immediately and top management begin to design a
new program to take its place (p. 2)
Key Issues
The first key issue that TEGV faces is in the area of strategic planning. Though
they have had success in growing the organization and providing services to more than a
million children, top management were not on the same page as to what the goals of the
organization should be. Varley (2014) wrote that some top managers wanted the impact
to be children receiving better grades, while others wanted to focus on behavioral skills,
and even more abstractly, some thought the goal should be for children to develop
discipline, tolerance, and confidence (p. 10). Strategic planning should have been done
periodically as the organization was growing. Bryson (2011) in Worth (2014) stated that
strategic planning is, “a deliberate, disciplined approach to producing fundamental
decisions and actions that shape and guide what an organization…is, what it does, and
why” (p. 160). Had TEGV managers engaged in strategic planning, they would have
come up with organizational goals early on in the organizations life cycle. It was not until
more than 15 years after the organization had formed that they finally came to a
consensus as to what they wanted organizational outcomes to be. This concept of
strategic planning is good in framing this key issue, especially since TEGV places such a
heavy emphasis on performance management and evaluation. If components of the
strategic plan are missing (such as the goals), then the organization cannot expect to
accurately measure outcomes
3
The second key issue deals with program evaluation and outcomes. The method
TEGV uses to collect program data is through its volunteers. Varley (2014) mentioned
that volunteers were trained how to monitor children and assess them, even in abstract
areas such as self-esteem, responsibility, and creativity (p. 5). This is an issue because
organizational effectiveness is related to program effectiveness (Herman & Renz, 2008,
p. 407). This statement is not to say that program effectiveness is identical to
organizational effectiveness, but to make aware that the volunteers, who have a very high
rate of turnover, are collecting nearly all programmatic data. This means that the
organization is relying on its ability to train short-term volunteers (remember, they have
had more than 9,000 volunteers) how to collect and report data as abstract as a child’s
level of creativity or self-esteem. This data will in turn be used to evaluate the programs
ability to produce outcomes. The possibility exists that the organization is not as effective
as it could be because of the possibility they may have discontinued effective programs
based on their volunteers ability to collect data correctly. To add to the issue of program
evaluation and outcomes, the other thing to consider is the effectiveness and efficiency in
producing their stated outcomes.
In the case study, Varley (2014) noted that TEGV identified two separate
struggles that could affect the efficiency of the organization and the program
effectiveness. The first issue, briefly mentioned earlier, is the high turnover rate of
volunteers. Varley noted that, “the organization devoted considerable resources to recruit,
train, and support each new volunteer. Yet—just when TEGV was in a position to reap
the benefits of this investment—most of these volunteers left the program “ (p. 7). The
second was in collecting data on the children. The issue in trying to evaluate a single
4
program was that children often joined multiple programs, and absences were common
(p. 8). Efficiency and effectiveness measures can be very useful in identifying program
performance (LeRoux & Wright, 2010, pp. 573-574). With that said, the issue at hand is
that TEGV may be putting too many resources into their volunteers while not having
accurate ways to measures the program effects on children. The programs are not as
efficient as they could be because of high use of resources compared to the low ability to
measure results. The programs are potentially not as effective as TEGV believes because
of the inability to accurately measuring how well a program meets its objectives. With all
the evaluations, direct services being provided to children, and resources being put into
volunteer training, Varley (2014) poses a question at the end of the case study about
whether or not engaging in large impact studies— on TEGV alumni and with the
Education Reform Initiative about controversial education reforms—would be worth the
cost (p. 11).
Recommendations:
Before TEGV should engage in any large impact studies, there are a few things
they should consider doing within their organization. The first recommendation is to
engage in strategic planning. This will help TEGV to better understand where its
priorities should be and what they should be looking for in their programs. As part of
their strategic planning, they should revise or reword their mission in a way that provides
them a clear direction. With part of their mission to be to support the development of
children as, “rational, responsible, self-confident, peace-loving, inquisitive, cognizant,
creative individuals, who are against any kind of discrimination, respect diversity and are
committed to the basic principles of the Turkish Republic” (Varley, 2014, p. 13), it can
5
be very difficult to narrow in just how the organization can do that and come up with
programs that support the mission. While the mission defines, “the social value that the
organization creates” (Worth, 2014, p. 164), it should also be simple to identify how the
programs align with the mission. As stated before, top managers were not even on the
same page as to what the impact of the organization should be, which identifies a huge
gap between mission and programs. A clear mission should eliminate doubt as to the aim
of the organization and what programs should measure. In order to engage in strategic
planning, TEGV would possibly have to put a hold to program evaluation and designing
new programs as it would require extra resources, both monetary and time from top
managers.
The next recommendation is that TEGV should change how they evaluate the
effectiveness of their programs. They have admitted that they put a lot of resources into
training volunteers while having low volunteer retention (Varley, 2014, p. 7); not all
volunteers are of the same skill level in there assessment capability; and volunteers are
having to assess concepts that in general would be very difficult to assess, such as selfesteem, responsibility, and creativity (p. 5). TEGV should either revise what exactly the
program is expected to achieve in order to make it easier for all volunteers to make
accurate assessment or they should put less resources into training volunteers and use
those resources to hire professionals with the capabilities to accurately assess children
within the programs. TEGV is a leader in nonprofit volunteerism and they are very active
in encouraging other nonprofits to engage volunteers. Herman & Renz (2008) mention
that organizations are always looking for models of best practice. If TEGV promotes
these best practices for volunteers then the likelihood goes up that other nonprofits will
6
follow suit in putting many resources into volunteers without much concern for retention
and capacity for assessment.
Lastly, to answer the question Varley (2014) poses at the end of the case study
regarding whether or not they should engage in the costly impact studies, only the TEGV
alumni study would make sense in their case and even then they should not engage in the
study. TEGV only recently decided what their aim was, so putting resources into a study
would only find the impact of what they were doing before they came to a consensus, not
what they have been doing since then. Resources would be better spent on designing
more accurate ways to measure the impact of the programs that take into account that
children are involved in multiple other programs. Coming up with more accurate ways to
measure the program performance will provide accurate performance data. LeRoux &
Wright (2010) state that, “Performance data can provide managers with valuable insights
into organizational strengths and weaknesses, thereby equipping managers with
information to guide strategic decision making” (p. 575). Using these resources to assess
how to measure program performance will ultimately make the organization more
effective and efficient
Conclusion:
TEGV has had great success in growing their organization. Though they have had
over 9,000 volunteers and served over 1.43 million children, there are signs of
organizational weaknesses. First, TEGV operated for several years without having a
unified goal its managers supported. Second, TEGV relies on volunteers to collect
programmatic data though the resources used to train them are not providing them all the
skills and capacity to make such assessments. Lastly, program evaluations are not set up
7
to take into account major external factors such as child involvement in multiple
programs and child absences. The recommendations will ultimately benefit the
organization. Strategic planning will help the organization be more direct in their
mission, goals, and the programs they offer. Reallocating resources away from volunteer
training to hiring professionals with the capacity to assess programs will lead to more
accurate program evaluations. Lastly, avoiding costly impact studies will allow the
organization to put resources into designing more accurate programs and program
evaluations, which will make the programs more effective and efficient.
Reference:
Herman, R. & Renz, D. (2008). Advancing nonprofit organizational effectiveness
research and theory: Nine theses. Nonprofit Mangemetn & Leaership, 18(4) 399415. Wiley Periodical, Inc.
LeRoux, K. & Wright S. (2010). Does performance measurement improve strategic
decision making? Findings from a national survey of nonprofit social service
agencies. Nonprofit and Voluntary Sector Quarterly, 39(4) 571-587. Sage
Publications.
Varley, P. (2014). Measured approach: TEGV assesses is performance & impact on
educational enrichment programs. HKS No. 2028.0. Cambridge, MA.
Worth, M. (2014). Nonprofit management: Principles and practices [3rd Ed]. Sage
Publications.