Download submits the following comments in response to the KEMA Draft

The Division of Ratepayer Advocates Comments on the KEMA Draft Impact Evaluation Research
Plan on Upstream and Residential Downstream Lighting
The Division of Ratepayer Advocates (DRA) submits the following comments in
response to the KEMA Draft Impact Evaluation Research Plan on Upstream and
Residential Downstream Lighting (Research Plan), Work Order 28, dated October 29,
2011. The Research Plan provides an outline of Section 1 through 2.1 of the topics in the
Table of Contents, which include an evaluation overview, goals and objectives, a list of
researchable issues, research plan schedule, contact information and the list of high
impact measure groups.
The goals and objectives according to the research plan are:
(1) to verify and validate the IOU-reported energy savings and peak demand
reduction claims (including net savings and free ridership)
(2) to provide feedback to the IOUs on how well the energy efficient lighting
measures are doing relative to their established program performance metrics
(PPMs) and market transformation indicators (MTIs)
(3) To provide results that will support future program design improvements and
future program ex ante impact estimation.
The last two goals indicate that this research plan is not simply a “savings evaluation
report,” but also a report on lighting market transformation performance. DRA’s
submitted comments filed after the Commission’s November 7, 2011 market
transformation workshop recommend the application of the 13 key steps of a market
transformation program plan in the development of market transformation indicators, as
recommended by workshop participants and leaders in market transformation work –
NEEA, NYSERDA and Prahl and Keating.1 This Research Plan should be
synchronized with the development of market transformation indicators, since one of the
objectives of the Research Plan is to provide “feedback” on how well lighting measures
1
DRA Comments on the November 7, 2011 Market Transformation Indicator Workshop (DRA MTI
comments), filed on November 21, 2011 in R.09-11-014.
are doing relative to these MTIs. Following the 13-key steps is central and essential to
this process.
First, the Research Plan should define each of the appropriate market segments as the
primary step to evaluating upstream and residential downstream lighting. This should
result in several market segments, each defined in a manner that meets each of the
following definitional attributes: (1) a set of actual or potential customers, (2) for a given
set of products or services, (3) who have a common set of needs and wants, and (4) who
reference each other when making buying decisions.2 The third and fourth attributes of
defining a market are critical as they work towards ensuring the evaluations are
actionable. By actionable, DRA means helpful to program administrators and
implementers who strive to improve the effectiveness on residential lighting programs.”3
The design, development, implementation and evaluation of market transformation and
resource acquisition programs cannot be divorced from a disciplined, delimited definition
of the addressable market segment and it cannot be divorced from the product attributes,
quality and performance standards expected by customers in that market. This is
especially the case in residential lighting where customers have different expectations for
each area of the home. It is simply not enough to divide residential lighting based on
light bulb intensity and whether the bulb is dimmable or not and then assume the color
quality is essentially equal across all these bulbs, regardless of technology. They are not.
For example, the denominator in calculating penetration of basic CFLs should not simply
be non-dimmable sockets but rather non-dimmable sockets located in areas of the home
where customers can reasonably expect and accept lower color quality relative to the
superior color quality provided by incandescent bulbs which is the baseline customers
expect. In this narrower market segment, the market share of basic CFLs will be much
greater than in previous evaluations. This information becomes actionable because it will
focus investment on other promising bulb technologies that have the potential to meet the
minimum color quality standards expected for the remaining sockets in the home,
dimmable or not. Furthermore, previous lighting program evaluations have been bulbcentric (also very much CFL-centric) and do not take into account other aspects of
2
3
Geoffrey A. Moore, Crossing the Chasm, First Collins Business Edition 2006, p. 28.
Ibid, pp. 5.
lighting technologies, for example, lighting controls and sensors that can be part of a
lighting system retrofit.
Thus, one market segment may be defined as residential customers who can
reasonably accept the color quality of basic CFLs in specific areas of the home (e.g. the
garage, hallways, home office). Another market segment may be those residential
customers who would prefer other energy efficient light bulbs that have the color quality,
shapes, and sizes that are at least equal to or better than that of the diverse spectrum of
incandescent bulbs (e.g., lighting to showcase artwork and home shelf displays, or
lighting placed in a “powder room”).
Unless markets are defined carefully and in a delimited way, it is difficult to
determine the “saturation rates” of particular lighting measures. It is inaccurate to claim,
as the 2006-2008 Market Effects report states, that California CFL saturation rates
remain at approximately 21% of all sockets.4 The fact is, that CFLs are unlikely to
“saturate” market segments that have no desire for them. For example, while warm
incandescent bulbs may work to showcase artwork, basic CFLs may not work. Basic
CFLs can only therefore, address or penetrate the segment of the lighting market that
does not require warm color lighting. CFL saturation rates are measured appropriately
when the market is defined within the correct target market segment. A more accurate
way to understand CFL penetration is to view it within the appropriately-defined and sized market segment. From a market perspective, CFLs may have already reached its
saturation point. For all other market segments, other lighting products are required to
meet the specific demands of these other segments. Together, these lighting products may
“saturate” a larger part of the residential lighting market once served by incandescent
bulbs (the baseline).
The measure groups listed on page 3 of the Research Plan should be categorized
into the market segments they serve. Multiple technologies may serve a single market
segment. Their success on “how well the energy efficient lighting measures are doing
relative to their established ….market transformation indicators” should be measured on
(step 3) how well these measures remove market and institutional barriers and saturation
4
Compact Fluorescent Lamps Market Effects Final Report, The Cadmus Group, KEMA, Itron Inc, Nexus
Market Research, A. Goett Consulting for the California Public Utilities Commission, April 2010. (The
Lighting Report). Pp. 105-106.
rates within the appropriate market segments. The gap between the saturation rate and
the rest of the market will “provide results that will support future program design
improvements.” Once MTIs are developed (if they are developed within a timeframe that
is early enough to be incorporated into the Lighting Research), success of the measures
should be assessed according to other indicators, including the short-term, intermediate,
and long-term progress indicators developed through the Commission’s MTI efforts. As
DRA’s MTI workshop comments states, it will be difficult to measure progress along
MTIs if the market, leverage points, barriers, interventions, and differences from a
baseline not precisely defined.5
Further, as stated in DRA’S MTI Workshop comments, DRA believes market
transformation progress for lighting should not be measured based on units of lumens per
watts used, but rather on the average energy consumption of residential lighting. “This is
because the products available to reduce the average consumption of lighting applications
include not only bulbs but other items like sensors and controls. Incandescent bulbs
linked to sensors and controls can be quite an efficient energy efficient solution.”6 In
other words, they can reduce the amount of energy used in the residential lighting sector
for providing the same amount of “lighting energy service”. While this is not a primary
focus of the current portfolio’s upstream and downstream residential lighting programs,
progress in this area may provide useful information for next-cycle portfolio design
improvements. By contrast, “[a] “lumens per watt” metric will only reinforce investment
decisions and program designs that are bulb-centric and it will be a struggle to transition
to a whole lighting systems approach which is more consistent with the desire to achieve
persistent, long-lasting energy savings.”7
Finally, in addition to the above recommendations, DRA recommends that if
possible, the “researchable issues” in this research plan (p. 4) include an assessment of
the spillover effects or benefits of the current upstream and downstream residential
lighting program as well measure the persistence (or “decay replacement”) of CFL
lighting in sockets that were filled with prior program CFLs that have burned out. This is
5
DRA MTI comments, pp.3.
DRA MTI comments, pp. 10.
7
Ibid.
6
an important indicator of market transformation that can provide highly useful feedback
for future lighting program design.