1

July 13, 2011 — It’s been more than 20 years since the United Nations Development Programme (UNDP) started publishing annual Human Development Reports “with the goal of putting people at the center of development, going beyond income to assess people’s long-term well-being.” Building on this model, many countries have adopted or adapted a “human development index,“ and, last year, the Affordable Care Act called for the creation of a “key national indicators” system in the U.S.

“People think you can put these things together and publish them and then something will happen,” said a senior advisor to the Canadian Index of Wellbeing project, but “I don’t think there will be uptake unless there’s a strong communication and education effort made.”

But Congress has yet to authorize funding for the project, and simply establishing an indicator system is no guarantee that its measures will become significant considerations in the crafting of public policy. The fundamental issue: do indicator systems work by attempting to “insulate” them from politics, or by attempting to have them do battle in the political arena by challenging those who don’t accept the importance of the indicators and by promulgating specific policy recommendations? If the answer were engagement, what steps would make a key national indicator system effective?

Limitations of GDP

The limitations of using GDP as the primary measure of a society’s progress have been well documented. The figure does not differentiate between goods and services that are beneficial and those that are not. For example, producing more cigarettes (and thus more disease), and producing more gasoline for cars stuck on the expressway (and thus speeding destructive climate change), both lead to an uptick in GDP; on the contrary, the work of the stay-at-home parent who provides care for his or her child every day is treated by the GDP as zero.  And GDP neither indicates whether economic benefits are flowing disproportionately to certain groups, classes, or interests within a society, nor says much about the subjective well being of a society’s population.

In the U.S., there have been many attempts on the regional, state, and national levels to go beyond GDP. For example, on the state level, an indicator system was first put in place in Oregon in 1991. On the municipal level, one city that has used human development indicators is Jacksonville, Florida.  On a broader scope, the not-for-profit American Human Development Project has produced “The Measure of America,” its implementation of a human development index for the country. (Remapping Debate’s previous edition featured a video interview with Sarah Burd-Sharps, co-author of “The Measure of America”).

But an official system for the U.S. is still at the starting gate, and serious questions remain as to how to maximize its efficacy.

If you build it, will they come?

Chris Hoenig is a senior adviser to the presidents of the National Academy of Sciences (NAS) and president of State of the USA, a nonprofit founded in 2007 with the goal of creating a comprehensive national indicator system. After passage of the Affordable Care Act, State of the USA partnered with the NAS to prepare for implementing the key national indicator system.

If Congress appropriates funding for the indicator system and the project goes forward, the NAS will identify the human development areas to be addressed by the key national indicators, determine the measures that will be used, and choose where the data will come from (see box).

Methodology

The dimensions to be covered by the “key national indicators” for the U.S. have not yet been determined by the National Academy of Science, said Chris Hoenig of State of the USA, although he expects those dimensions to include aging, civic and cultural life, crime and justice, economy, education, energy, the environment, families and children, governance, health, housing, infrastructure, innovation, safety and security, and transportation.

Two similar programs that are currently in place are the Human Development Index (HDI) of the U.N. Development Programme and “The Measure of America” of the Social Science Research Council’s American Human Development Project.

The HDI is a composite index that factors in health, education and standard of living to arrive at a single number to represent a country’s development.

In the most recent Human Development Report, the HDI takes into account four indicators: life expectancy at birth, mean years of schooling, expected years of schooling, and gross national income per capita. Countries are then ranked according to HDI. The report also compiles an HDI that adjusts for inequality and publishes a “gender inequality index.”

Like the U.N.’s HDI, “The Measure of America” uses the three dimensions of health, education and standard of living, but employs different indicators. “The Measure of America” factors in life expectancy at birth, educational degree attainment, school enrollment, and median earnings.

An interactive map on the American Human Development Project website allows users to visualize disparities in HDI across the country and rank states and congressional districts according to composite HDI or its components, or according to a host of other indicators on the environment, housing and transportation, political participation, and security.

2

The legislation calls for the data generated to be published on a freely accessible website. Hoenig wants the site to be a place where an individual can browse information about his community and a researcher can have access to a deep array of data to conduct sophisticated analyses.

“If you can focus on and select what are the most important things to learn about, then you can begin the process of goal-setting, you can begin the process of comparing, you can begin the process of cause and effect, you can begin the process of analyzing root cause,” Hoenig said.

But other attempts to implement key indicator systems have shown that it is not sufficient to simply release data and hope for the best.

How the U.S. Fares on Human Development

Overall, the U.S. ranks fourth on the U.N. Development Programme’s Human Development Index. To dig deeper, Remapping Debate used the “do-it-yourself index” on UNDP’s website to find how the U.S. ranks according to more particularized dimensions.

Dimension Rank
Health 29 of 164
Education 9 of 152
Income 9 of 183
Inequality 42 of 133
Gender 55 of 151
Sustainability 99 of 114
Human security 38 of 50

 

Note: In each dimension, we used the maximum number of measures that included data for the U.S., and then compared all countries with equivalent data across the composite created when each of those measures are equally weighed.

Note: In each dimension, we used the maximum number of measures that included data for the U.S., and then compared all countries with equivalent data across the composite created when each of those measures are equally weighed.

Measures not available for U.S. and thus not used in comparison: for education, “adult literacy rate”; for income, “household final consumption ”; for human security, “internally displaced persons.”

Measures not available for U.S. and thus not used in comparison: for education, “adult literacy rate”; for income, “household final consumption ”; for human security, “internally displaced persons.”

“People think you can put these things together and publish them and then something will happen,” said Alex Michalos, a senior advisor to the Canadian Index of Wellbeing project, an effort to create a national index for Canada that goes beyond GDP. “But that doesn’t happen.” 

“I don’t think there will be uptake unless there’s a strong communication and education effort made,” Michalos said.

In a June report, the Government Accountability Office (GAO) enumerated several ways the systems are vulnerable to failure: it is difficult to sustain funding for them; stakeholders are liable to lose interest in the project; the systems are susceptible to accusations of bias; and they can tend to languish in obscurity without reaching a wide audience.

How will this time be different?

Hoenig says that he is hoping that the U.S. key national indicator system can overcome those problems through the quality of its data and through outreach to a wide spectrum of intended users: the public, researchers, businesses, nonprofits, the media. If the information is comprehensive, transparent, and presented in an engaging way — through interactive graphs and charts on the website that reveal relationships in the data — then, he asserts, it will earn the trust and confidence of the American public and it will be adopted as an authoritative reference.

It is also important that the indicator system not be seen as the product of a single party or branch of government, Hoenig said. He said the provision in the Affordable Care Act establishing the key national indicator system was crafted with that in mind. The NAS was chosen to select the data to be used as indicators because it is known as a nonpartisan institution, he said. And the legislation established a Commission on Key National Indicators to oversee the indicator system, with members of the commission nominated by the majority and minority leaders in both chambers of Congress.

Hoenig said that although some indicator programs have successfully integrated data gathering with interpreting that data and making recommendations, Congress did not task the key national indicator system with making policy recommendations. That would have to be left to policymakers, researchers, advocacy groups, businesses and the media.

3

Some representatives of comprehensive indicator systems told Remapping Debate that if they were to make policy recommendations it would undermine their mission.

The limitations of using GDP as the primary measure of a society’s progress have been well documented. The figure does not differentiate between goods and services that are beneficial and those that are not.

Andrea Whitsett, project manager of Arizona Indicators, a benchmark initiative managed by the Morrison Institute for Public Policy at Arizona State University, said that the need was to “build something…that becomes over time a trusted resource that is perceived as neutral and objective: then it advances that civil dialogue because people can start from a common data point.”

But can that model work in the context of human development measures?

Joseph Sirgy, a professor at Virginia Tech and the editor of the journal Applied Research in Quality of Life, explained that, historically, those who push to look beyond economic well-being as the primary indicator of the country’s development ”are labeled leftists.”

Support for a national indicators system has tended to depend on “how the pendulum has been swinging back and forth in Congress whether the Democrats have dominated the agenda versus the Republicans,” Sirgy said.

During the Carter administration there was a push for a social indicator system at the national level, but it was “swept under the rug” during subsequent Republican administrations, he said, “the reason being the conservatives labeled it as a leftist agenda.”

Developing measures is invariably intertwined with politics

Stephen Heintz, president of the Rockefeller Brothers Fund, who was nominated to sit on the Commission on Key National Indicators in December by then-House Speaker Nancy Pelosi, said it is difficult to disentangle politics from indicators. “What you measure really does drive your priorities, and people obviously know that. And so this question very quickly becomes a political question,” he said.

Heintz, who made clear that he was speaking on his own behalf and not that of the commission, said public education about indicator systems can counteract demagogic efforts to marginalize such systems by enabling people to better hold politicians accountable for their decisions. In Heintz’s view, the broad public exposure to economic data that drive policymaking (GDP, unemployment, inflation) significantly shapes citizen attitudes towards policymakers’ decisions, and influences how citizens lobby their legislators and how they vote. If human development indicators came to have equivalent exposure, then those indicators, too, Heinz claimed, would become part of a public policy dialogue.

Indicators give people “the opportunity to use them as levers of accountability,” Heintz said. “People can go testify on the Hill on legislation and say, ‘As you’re considering an economic stimulus program, have you looked at the following data that is part of the key national indicators, and how does your stimulus program respond to data sets x, y, and z?’”

“If policy is going to be in part influenced by these measures, you want people to have an understanding of that,” Heintz continued. “If they have an understanding of the indicators and the data that is driving the decision making, they will be better prepared to exercise their responsibilities as citizens.”

Jacksonville case study, part 1: design of the program

The longest-running indicators program in the country is the Jacksonville, Florida community indicators project. The program was designed to have an advisory body with the capacity to assemble policymakers, officials, institutions, and the public to study a problem and make policy recommendations based on indicator data.

Each year, the Jacksonville Community Council Inc. (JCCI), a nonpartisan civic organization that oversees the indicator program, puts out a progress report for the city with an assessment of how the community is performing in different areas. The report highlights particularly problematic trends with red flags. The 2010 progress report, for example, flagged the number of new HIV cases, which had increased to 434 from 379 the previous year.

Sometimes the issues are too large for a particular body or agency to deal with on its own, and the council will be asked to step in, said JCCI Vice President Ben Warner. He gave as an example the murder rate in Duval County, where Jacksonville is located. In the first half of 2006, the already high murder rate rose even further, and the Jacksonville sheriff asked JCCI for help.

The council held an open community meeting that included the sheriff, the mayor, the state attorney, shelter representatives, crime victims, and members of the public. JCCI then prepared a report examining factors behind the rising murder rate and made recommendations to reverse the trend based on input from a community study group.

Some of the recommendations were directed at the sheriff’s office, such as an initiative to get illegal guns off the street. But the recommendations also sought to address underlying problems that are not typically thought of as being in the jurisdiction of crime fighters, such as racism and a lack of economic opportunity for young people. After the recommendations were implemented, violent crime rates and murder rates went down, Warner said.

4

How to bring the message home

Kristen Lewis, co-director of the American Human Development Project, said that how the benchmark data is presented to the public can make a big difference in how it is received.

“If you’re trying to persuade someone and you give them a black-and-white typed report that is 50 pages long and at the back has 30 tables that make your point really well with numbers and you give that to a regular person, and you try to show them that’s why they should change the policy…they are not really going to be persuaded,” Lewis said. “You have to…use the numbers to tell a convincing story.”

The timing of how the information is released can also make a difference, Lewis said. GDP, for example, is published every quarter. But with life expectancy or other health and education indicators, “there’s nothing like that — there’s no fanfare,” Lewis said. “That’s very problematic.”

“The more media is on board and actually understands the indicators, the bigger the chance that the data will actually be reported and in such a fashion that the community can actually engage around whatever the topics are that they’re bringing forward,” said the head of a local nonprofit.

Lewis said for indicators such as life expectancy that don’t change much from year to year, it wouldn’t make sense to publish them as frequently. But other indicators, such as the birth weight of babies, are amenable to short-term policy change, she said.

Birth weight is a “very sensitive indicator for the health of the population,” Lewis said. “So if that is collected and published, and policymakers use that and talk about wanting to change it and build policy around it and actually use it to measure change, then that would be great [and] a lot would happen in terms of population health.”

Karen Hruby, executive director of Truckee Meadows Tomorrow, a nonprofit that oversees indicator data in northwestern Nevada, said the media has a crucial role to play as well. “The more media is on board and actually understands the indicators, the bigger the chance that the data will actually be reported and in such a fashion that the community can actually engage around whatever the topics are that they’re bringing forward,” she said.

The Reno Gazette-Journal has used indicator data to examine the underlying causes of problems affecting the region, Hruby said. In its reporting on the region’s economic troubles, for example, the newspaper has looked at how education and innovation factor in. “It’s nice to say we’re going to get out of economic slump here if we just create jobs, but nobody’s got the magic wand,” she said. “They started to dig deeper and say, ‘What’s the reason behind why we don’t have more jobs?’” she continued. “‘Let’s talk about what other problems do we have? Do we have an educated workforce. No. OK how do we address that?’ It starts to go from there.”

The paper held forums with members of the community to discuss solutions to the problem. Eventually, policymakers began to show up and participate at the forums, including county commissioners and city council members, Hruby said. After community members said it was important for the region to develop alternative sources of energy, Washoe County and the cities of Reno and Sparks all set renewable energy targets, she said.

Jacksonville case study, part 2: what happens when legislators don’t listen

The benchmark program is a trusted and well respected source for elected officials and stakeholders, Warner said. “Of the ones that don’t use it directly, people who go and lobby them often use it,” he said. Still, at times policymakers only refer to the benchmarks when they show support for a favored cause, and ignore the indicators if they are unflattering.

If officials refuse to address a problem, then indicators can serve as a way to spur the community into action, Warner said. For example, when JCCI education data showed sizable achievement gaps when it was broken down by income and race and ethnicity, the official school system response was, “There is no achievement gap,” Warner said.

But when JCCI presented the data to the public, “the community reaction to it was one of these a-ha moments,” Warner said. “They said we knew that there was a problem, we didn’t know the problem was this big, and we didn’t understand how important it was that we as a community address this issue, and now that we understand that we’re going to demand change.”

The next time the school board advertised for a superintendent, the primary job criterion was the ability to address the achievement gap, Warner said. “It was just simply persistent sharing of information and the community getting involved and saying, ‘This is important to me, it needs to be important to the schools.’”

Remapping Debate asked Warner if he would favor having statutes in place that would mandate that lawmakers be responsive to benchmarks when they indicate a severe problem. For example, a policy trigger could be instituted to compel lawmakers to act if an indicator reached a predetermined threshold.

Warner said he was wary of building policy around a particular number. He said he would be worried that those responsible would try to change the number rather than address the underlying problem.

For example, when Florida first instituted standardized testing, school systems trying to get a leg up started moving school start dates back earlier and earlier so they would have extra time before the test date.

“Was that in the best interest of the kids? Probably not,” Warner said. “It was trying to change the numbers rather than trying to have a real substantive change in the way kids were learning.” Warner said that similar problems had arisen when benchmarks were viewed simply as ends in themselves, “instead of something telling you about a larger issue.”

“Our role in the community is that of building collaborative partnerships,” Warner said. “If [the indicators] aren’t galvanizing the community as a whole,” he continued, “then they’re not going to galvanize your elected officials.”

5

Oregon: good intentions, but how much progress?

Oregon’s indicator program, Oregon Benchmarks, has been widely perceived as one of the preeminent indicator systems in the country since it was adopted early in the 1990s, but it has not always been supported (see box), and questions remain about how well-integrated the benchmarks have been with policy development.

According to Oregon state representative Jefferson Smith, the Oregon Benchmarks program has not been sufficiently linked as a matter of law and structure to policymaking. “We get these benchmarks, great, but it’s not doing enough to inform our policy choices.”

If officials refuse to address a problem, then indicators can serve as a way to spur the community into action, the Jacksonville Community Council’s Ben Warner said.

On July 6, Oregon governor John Kitzhaber signed a bill sponsored by Smith that directs the state’s Department of Administrative Services to integrate Oregon Benchmarks with the legislative budget and policy process. The bill does not specify how that process should be implemented, but Smith had his own ideas for how to go about it.

To raise awareness of the benchmarks among policymakers, the indicator system should have a lobbyist or several lobbyists to inform them about how policy initiatives would affect benchmarks, Smith said. 

“There needs to be talented human time applied to communicating with decision makers,” Smith said. “I can imagine it being a lobbyist who went around and met with everybody and said, ‘Hey, here’s some of the stuff that we’re thinking about, here’s what the indicators are saying now, here’s what the policy impacts of the following pieces of legislation might be.’”

Hruby from the Truckee Meadows nonprofit said she thought having a lobbyist dedicated to indicators would be helpful because she did not have time to do that work herself. “I did my fair part sending as much indicator data as I could…in our last legislative session, related to particular bills,” Hruby said. “But you’ve got to have the time to track the bills to do that, which requires a lobbyist.”

Smith offered his view that, “I don’t think one thing will make [Oregon Benchmarks] sufficiently linked to policymaking. I think it’s going to take several things.”

Among his other suggestions:

• having all incoming state legislators go through an orientation process to familiarize them with the benchmarks and thereby help them start thinking about policy ideas that could address problems identified by the indicators.

• having the benchmarks be linked to the governor’s state of the state address. (“Imagine a State of the State address that wasn’t merely a few generalized plaudits and troubles and then an announcement of the governor’s favorite next initiatives,” Smith said. “What if it also included an evaluation of where we are?”)

• having a required impact statement that would show how the proposed legislation would influence benchmarks. (Smith gave the example in the context of an appropriations bill: “If we defund higher education by another 10 percent, that’s going to reduce the number of college-educated people we have by blank percent.”)

Oregon

Duncan Wyse was executive director of the Oregon Progress Board, the body that oversees Oregon Benchmarks, when the benchmarks were instituted in the early 1990s. The governor of Oregon — at the time Barbara Roberts — served as chair of the Progress Board, and Wyse said she embraced the benchmarks and integrated them into policy proposals that she would present to the legislature. Roberts also tied the benchmarks to the budget process by promising state agencies more funding if they could show their programs had a positive influence on relevant benchmarks, according to a 2004 GAO report. The Progress Board estimated that the policy in 1993 “resulted in a shift in the budget distribution worth an estimated $130 million toward programs aimed at the lead benchmarks,” the report said.

When the Oregon benchmarks were created, Democrats held both the governorship and majorities in both chambers of the state legislature. When Republicans gained control of the legislature in the 1994 elections, the program stumbled, Wyse said. The Republican caucus, some of whom felt that the benchmarks represented a partisan agenda, did not approve a bill to reauthorize the Progress Board, according to the GAO report. But then-Governor John Kitzhaber, a Democrat, had already approved a two-year budget for the board, thus keeping the program alive.

A new executive director of the board refreshed the benchmarks and renewed support for them, which lasted until recently, Wyse said. When Republicans again gained control of the legislature in 2009, funding for the Progress Board dried up.

As the GAO put it in its June report, “In the Oregon legislature, the fact that legislative term limits were instituted exacerbated difficulties already present due to the legislature’s turnover rate, as, over time, there were fewer members who understood the purpose of the benchmarks or had a desire to use them to inform their decision making.”