Home Energy Notebook for FY 1998, Part 3
Performance
measurement | To Table of Contents
Performance measurement system design
| To Top of Page
The Human Services Amendments of 1994 (Public
Law 103-252) directed the U.S. Department of Health and Human
Services (HHS) to develop, in close consultation with LIHEAP grantees,
model LIHEAP performance goals and measures. LIHEAP grantees,
at their option, could then use the goals and measures to assess
their success in achieving the purposes of LIHEAP. The Administration
for Children and Families' Office of Community Services (OCS)
received extensive input from LIHEAP grantees, local administering
agencies, and other interested parties in developing the model
LIHEAP performance goals and measures. In November 1995, OCS'
Division of Energy Assistance (DEA) issued the model LIHEAP performance
goals and measures to LIHEAP grantees.
In 1997, DEA established the LIHEAP Advisory Committee
on Managing for Results. The Advisory Committee is a joint partnership
of DEA, state LIHEAP program offices, local subgrantee agencies,
other program stakeholders, and technical experts. The Advisory
Committee was established to assist DEA in its development of
annual LIHEAP performance plans under the Government Performance
and Results Act (GPRA), to identify and find solutions to LIHEAP
performance measurement implementation issues, and to furnish
guidance on technical assistance that might be needed by grantees
and subgrantees as they implement LIHEAP performance measurement
systems.
In support of the work of the Advisory Committee, DEA has sponsored a
number of studies to develop technical specifications for measuring
state performance in the area of LIHEAP targeting. One output
of those studies is that DEA has defined a set of targeting indicators
and annually furnishes information to state LIHEAP offices to
document targeting performance during the previous fiscal year.
The reports can assist them in the development of performance
measurement systems.6
LIHEAP targeting
The Human Services Amendments of 1994 reauthorized
LIHEAP through FY 1999.7 As part of this reauthorization, Congress amended the purpose
of LIHEAP (Sec. 2602(a) as amended) to clarify that LIHEAP is
"to assist low-income households, particularly those with
the lowest income, that pay a high proportion of household income
for home energy, primarily in meeting their immediate home energy
needs." Congress further indicated that LIHEAP grantees should
reassess their LIHEAP benefit structures to ensure that they are
actually targeting those low income households that have the highest
energy costs or needs.
In targeting LIHEAP benefits, grantees' targeting
options include the following:
- Eligibility Targeting: Grantees set program
eligibility requirements, within the income limits set by the
federal LIHEAP statute.8 Within those bounds, grantees can use restrictive income limits
and other program eligibility criteria to target the program
to certain eligible households.
- Outreach: Grantees conduct program outreach
to make individuals aware of LIHEAP program benefits. Outreach
activities can be designed to target certain eligible households.
- Benefit Targeting: Grantees set benefit determination
procedures. These procedures can be designed to target higher
benefits to certain eligible households.
By using these targeting mechanisms, a grantee
achieves two different targeting goals. First, it serves certain
types of households at a higher rate than other types of households.
This can be called recipiency targeting. Second, it gives higher
benefits to certain types of households. This can be called benefit
targeting. As a state designs its LIHEAP program and sets its
performance standards, it must set standards for both recipiency
targeting and benefit targeting.
_________
6 Roper Starch Worldwide
Inc. prepared this study for HHS under contract to the LIHEAP clearinghouse.
The statements, findings, conclusions, and recommendations are solely
those of analysts from Roper Starch Worldwide Inc. and do not necessarily
reflect the views of HHS or the LIHEAP Clearinghouse.
7 The Coats Human Services
Reauthorization Act of 1998 (PL 105-285) reauthorizes LIHEAP through
FY 2004.
8 Under the LIHEAP statute,
the maximum income standard is the greater of 150 percent of the
Federal Poverty Income Guidelines (poverty) and 60 percent of a
state's median income. The minimum income standard is 110 percent
of the Federal Poverty Income Guideline.
Recipiency targeting performance indicators
When a state sets a recipiency targeting performance
standard, it specifies the number or the percentage of households
in a targeted group that it hopes to serve. To set this standard,
it must have a good understanding of the number of eligible households
in a target group, the energy needs of eligible households in
the target group, and the number of target households it is serving
under its current program.
DEA has contracted for the development of a database
that can furnish information on the number of eligible and recipient
households by target groups for each state. In addition, DEA has
developed a performance indicator that states can use to measure
their program's targeting performance. The "recipiency
targeting index" for a specific group is computed by comparing
the percent of the group that received LIHEAP benefits to the
percent of all eligible households that received LIHEAP benefits.
For example, if 25 percent of eligible elderly households are
served, and 20 percent of all eligible households are served,
the recipiency targeting index for elderly households is 125 (100
times 25 divided by 20). A targeting index over 100 indicates
that a group receives LIHEAP benefits at a rate higher than the
rate for other groups.
Benefit targeting performance indicators
When a state sets benefit targeting standards,
it specifies different levels of LIHEAP benefits by target group.
To do so, it must have a good understanding of the number of households
in each target group, the energy costs and energy burdens (i.e.,
percent of income spent on energy) incurred by target households,
and the distribution of benefits to target households under the
current benefit determination procedure.
In the 1997 LIHEAP Home Energy Notebook,
two targeting performance indicators were defined to assist states
in quantifying their targeting performance. The first indicator,
the "benefit targeting index" is computed by comparing
the average LIHEAP grant for a target group of LIHEAP recipients
to the average LIHEAP grant for all LIHEAP recipient households.
For example, if elderly household recipients have an average grant
of $250 and the average grant for all households is $200, the
benefit targeting index is 125 (100 times $250 divided by $200).
A targeting index over 100 indicates that a group receives higher
LIHEAP benefits than the recipient population as a whole.
The second indicator, the 'burden reduction
targeting index' is computed by comparing the percent reduction
in the median individual energy burden for a target group of LIHEAP
recipients to the percent reduction in the median individual energy
burden for all LIHEAP recipients9. For example, if elderly household recipients have their
energy burden reduced by 25 percent (e.g., from 8 percent of income
to 6 percent of income) and all households have their energy burden
reduced by 20 percent (e.g., from 5 percent of income to 4 percent
of income), the burden reduction targeting index is 125 (100 times
25 divided by 20). A targeting index over 100 indicates that a
group has a greater burden reduction than the recipient population
as a whole. Note that a state must have access to LIHEAP recipient
energy expenditure data to compute the burden reduction targeting
index.
The benefit targeting index and the burden reduction
targeting index are both useful indicators because they measure
different aspects of benefit targeting. The benefit targeting
index is a simple measure of how benefits for a particular group
of LIHEAP recipient households compare to benefits for all LIHEAP
recipient households in the state. The burden reduction index
is affected by energy costs for the group of LIHEAP recipient
households compared to energy costs for all IHEAP recipient households,
as well as by benefit levels. A group of LIHEAP recipient households
may have a much higher benefit than the average LIHEAP recipient
household in the state and will therefore have a benefit targeting
index that is greater than 100. However, these households may
have a lower than average reduction in energy burden because of
high energy costs. In this case, the burden reduction targeting
index would be less than 100.
_________
9In general, the
mean is a preferable statistic to the median, as it is more informative.
Energy costs and benefits are not highly skewed variables; therefore
mean benefits are used to compute the benefit targeting index. Because
energy burden is a highly skewed statistic, the median energy burden,
which is less affected by outliers, is used to calculate the burden
reduction index.
Performance
measurement case studies |
To Top of Page
To support the work of the LIHEAP Committee on
Managing for Results, LIHEAP performance measurement case studies
were conducted in four states: Wisconsin, Washington, Arkansas,
and Texas. Using the performance indicators developed in earlier
case studies, this research documents each state's targeting
procedures and baseline targeting levels. The information developed
in these case studies furnishes examples of how to use performance
indicators to examine LIHEAP targeting issues.
In this study, information on each state's
LIHEAP program procedures and guidelines was developed from operations
manuals and interviews with program managers. Data on LIHEAP eligible
households and LIHEAP recipient households in each state were
developed from DEA's database of state-level information. These
data were used to compute baseline performance indicators for
each program. The analysis compared and contrasted targeting performance
for the four states.
The major findings of these case studies were
the following.
- While all of the states target benefits, they
use different methods, and they sometimes have very different
results.
- Serving the elderly appears to be the greatest
challenge for these LIHEAP programs. All four states examined
in this study had elderly targeting indexes below 100. An index
below 100 means that a state is serving the elderly at a rate
lower than the other types of households. In part, this may be
because elderly households may be reluctant to apply for LIHEAP
benefits.
- All four states have been successful in serving
households with the lowest income and poverty levels. Among the
four states, Washington has been the most aggressive in targeting
these groups. Households with incomes below 75 percent of the
poverty level had a recipiency targeting index of 279 and households
with incomes between 75 and 100 percent of the poverty level had
an index of 203.
- The examination of benefit targeting in Texas,
Washington, and Wisconsin revealed that these states were successful
in providing the greatest benefits to households with the greatest
energy burdens. However, while the households with the greatest
energy burdens received the greatest benefits and had the largest
reductions in energy burdens, they still had energy burdens that
were significantly higher than the lower burden group households
after receipt of LIHEAP benefits. (Since Arkansas did not have
energy expenditure data for LIHEAP recipient households, burden
targeting statistics were not computed for Arkansas.)
- Among the three states examined, Washington was
the most aggressive in targeting benefits towards the highest
energy burden group.
This study demonstrated how data may be used to
examine recipiency and benefit targeting performance. All states
have data available on the number of LIHEAP recipients in vulnerable
groups, and the number of households eligible for LIHEAP may be
calculated from the March Supplement of the Current Population Survey
(conducted by the Census Bureau) to calculate recipiency targeting
indexes. Data requirements to calculate benefit targeting are greater,
and many states may not have these data available. Three of the
four states examined here base benefits upon actual home energy
costs, and therefore these data were available for analysis.
While the states examined in this study had different
targeting procedures and different results, all of the states have
made significant efforts to reach vulnerable LIHEAP eligible households.
In order to refine targeting procedures, states must decide what
their goals are in terms of both recipiency and benefit targeting,
and then base their outreach and benefit formulas upon those goals.
It is then important for states to examine data such as those presented
in this study to determine whether they are meeting their goals
or whether procedures should be revised.
Performance
measurement outlier study |
To Top of Page
To support the work of the LIHEAP Committee on
Managing for Results, a performance measurement outlier study
was conducted in seven states. This study attempted to determine
how differences in state programs affect one LIHEAP performance
indicator and how targeting indexes should be calculated to account
for these differences. The performance indicator examined in this
study is the elderly household recipiency targeting index (i.e.,
a performance measure that demonstrates the extent to which LIHEAP
benefits are targeted toward elderly households). This study examined
targeting index computation procedures and values for states with
the highest and the lowest elderly household targeting indexes
in FY 1995 and FY 1998.
Methodology
The performance indicator that is examined in
this study is the targeting index for elderly LIHEAP recipients.
The index examines the percentage of elderly LIHEAP eligible households
that receive LIHEAP benefits relative to the percentage of all
LIHEAP eligible households that receive LIHEAP benefits. An index
of 100 indicates that the elderly are served at the same rate
as the LIHEAP eligible population as a whole. An index greater
than 100 indicates that a higher percentage of eligible elderly
are served than other eligible households, while a result of less
than 100 indicates that a lower percentage of elderly are served.
The elderly targeting index was calculated for
all states for FY 1995 and FY 1998.10
The four states with the highest targeting indexes in FY 1995
(Texas, Nevada, Mississippi, and Louisiana) and the three states
with the lowest targeting indexes in FY 1995 (Nebraska, Arizona,
and New Jersey) were chosen for inclusion in this study. The purpose
of this selection was to determine whether these states were actually
high (low) performers or whether the targeting index was overstating
(understating) these states' performance because of state differences
in programs and other state-specific factors.
LIHEAP plans for the seven states chosen for this
study were examined to understand state eligibility rules and
targeting procedures. State LIHEAP directors were then contacted
to obtain more detailed information about program structure, outreach
and targeting methods, and their views on why their program had
a high or low elderly targeting index.
________
10 When this study
was initiated, the most recent LIHEAP recipient data available were
from 1995. Once the 1998 data were available, the 1998 LIHEAP recipient
statistics were used to confirm that the selected states were still
outliers with respect to targeting elderly households.
Summary of findings
Discussions with LIHEAP program directors revealed
that there are many factors specific to individual states that
affect the calculation of the targeting indexes. To accurately
represent the performance of each state, targeting indexes should
be adjusted for these differences whenever possible. The main
adjustments that should be made are:
- Inclusion of recipients of other components
of the LIHEAP program: The original calculation of the targeting
index included only those LIHEAP recipient households that received
heating benefits. This number is valid in those states where households
must receive heating benefits in order to receive cooling and
crisis benefits or where the crisis benefits are a very small
component of the program. However, in many states, households
may receive cooling or crisis benefits without receiving heating
benefits. Including recipient households from these other program
elements may make a large difference in the targeting index if
different types of households are served by different components
of the programs.
- Inclusion of recipient households of other
state energy assistance programs: Some states have state-funded
programs that provide energy assistance to low income households.
Households may use these other programs as a substitute for LIHEAP
benefits. This performance indictor attempts to measure whether
households receive energy assistance, rather than the level or
type of energy assistance received. Therefore, an adjustment should
be made to account for the fact that households receive energy
assistance from a source other than LIHEAP.
- Adjustments for differences in eligibility
rules: States have different eligibility rules for receipt
of LIHEAP. One rule that may disproportionately affect the elderly
is that households living in subsidized housing and not responsible
for paying their heating bills directly are ineligible for LIHEAP.
In states where this rule is in place, the number of elderly eligible
may be overstated relative to the total number of eligible if
elderly are more likely to live in subsidized housing. This overstatement
of elderly eligible would lead to an underestimate of the targeting
index.
The study finds that, while targeting indexes
must be adjusted for differences in state LIHEAP programs and
other contextual factors, the indexes can be a useful tool to
help states measure how well they are targeting vulnerable groups.
States can use these targeting indexes to examine how their performance
varies over time, and to determine how changes in policy affect
the indexes. By looking at the program characteristics and/or
outreach efforts of states that have a higher targeting index
value, a state may be able to identify specific program changes
than can help to improve targeting performance.
The analysis of a small number of states in this
study identified some factors that appear to be related to high
targeting indexes. While only a small sample of states was studied,
there is some evidence that programs using certain targeting mechanisms
may be more successful in reaching vulnerable groups. The states
that had the highest targeting indexes had made special efforts
to reach the elderly population. The means by which the states
reached this population included having a separate component of
the program to serve that segment of the population, stressing
that the population is important when subcontracting to local
agencies, monitoring the success of local agencies in reaching
the population, working with other social service agencies that
serve the population such as the Social Security Administration,
and having an early application period for the targeted group.
To Top of Page
Page Last Updated: January 27, 2010
