Standard Errors Can Be Deadly

Copyright, The New York Times Company

Whether you like or dislike the Affordable Care Act, it helps to understand the distinction between statistical significance and practical importance.

Taxpayers want to know whether the government programs they pay for actually make a difference, so measurement is a critical part of policy evaluation. That’s where econometric and statistical analysis comes in: gather data under various policy implementations and try to measure the differences.

Take the act’s major expansion of Medicaid benefits, going to able-bodied adults living at or below 133 percent of the poverty line (without regard to asset ownership), which will occur in most states at the beginning of next year. Will the expansion make adults healthier?

An important study, published in The New England Journal of Medicine in May, set out to help answer the question by examining an Oregon-specific Medicaid expansion in 2008. In their short summary of conclusions, the authors wrote, “Medicaid coverage generated no significant improvements in measured health outcomes.” Opponents of the Medicaid expansion quote that sentence, telling us that we might not want to expand a health program that doesn’t actually make people healthier. Proponents of the expansion acknowledge the conclusion, too, and try to help readers find encouraging results elsewhere in the study.

The New Subsidy for Layoffs

Copyright, The New York Times Company

A major provision of the American Recovery and Reinvestment Act helps predict what will happen in the insurance market next year.

Employees and employers often take steps to avoid layoffs, like working hard to encourage customers and clients to continue buying the goods and services provided by the business.

Sometimes layoffs are not avoided by such efforts, which is why many employers also take steps to ease the burden of a layoff on employees and their families and to minimize disputes between them and the employer. Many businesses voluntarily offer cash severance pay, which is intended to replace the usual paycheck for several weeks or months after the layoff, depending on the length of time that the employee had been with the company. (During the time of unemployment, the former employee can many times collect both the severance pay and state unemployment insurance benefits.)

Employers may also offer to help laid-off workers continue with their health insurance during the unemployment spell. (Indeed, a laid-off worker who intended to spend some severance pay on health insurance premiums would save money on taxes if the former employer were paying those premiums, even if it meant lower severance.) Severance pay and health insurance costs for former employees are significant expenses that employers presumably take into account when they decide the number and timing of their layoffs.

During 2009 and 2010, the American Recovery and Reinvestment Act had the federal government pay some costs of layoffs, especially with its premium-assistance program, which paid 65 percent of the premiums that a laid-off employee would pay to stay on the former employer’s health plan. People who could join a spouse’s employer health plan and people without health insurance on their previous job were ineligible for the program.

Economists understand this premium assistance to be a subsidy to layoffs, making them cheaper and less of a burden. Employers saw it this way, too, according to an Urban Institute study.

“Some large‐firm interviewees reported that before A.R.R.A. they provided some amount of free or reduced cost Cobra coverage for laid‐off workers, based upon the prior duration of employment,” the study found, referring to the Consolidated Omnibus Budget Reconciliation Act, which allows workers who have lost their jobs to purchase coverage. ”Several of these companies reported that they reduced or dropped this prior benefit in reaction to A.R.R.A.”

Although we will learn more when (and if) the United States Treasury releases its final report on the premium-assistance program, it appears that program participation was high. An interim report indicated that perhaps two million households and even more individuals had their health insurance subsidized within nine or 10 months of starting the program.

The law’s premium assistance program ended in 2010, but significant amounts of premium assistance are coming next year as a part of the Affordable Care Act. For families with income between 100 and 250 percent of the poverty line, the Affordable Care Act is even more generous than the Recovery Act, because it helps them pay for both health insurance premiums and out-of-pocket costs like co-payments and deductibles. Also, health insurance is more expensive now than it was in 2009, which makes a percentage subsidy that much more valuable.

Moreover, unlike the Recovery Act’s program, next year’s program welcomes people out of work even if they left work by quitting, retiring or being fired for cause. It also welcomes people who have the possibility of joining a spouse’s plan and people who had no health insurance on their prior job.

Thus, we are about to begin a federal program that subsidizes layoffs to a degree that we have not seen before. Nevertheless, economic and budget forecasts by the Congressional Budget Office and others have yet to consider the effects of the layoff subsidy on the size of the program and the number of layoffs that will occur.

The C.B.O. has concluded that 800,000 people will take early retirements or quit as a consequence of the Affordable Care Act, not from its premium assistance, but based on the assumption that the unsubsidized nongroup health insurance market will operate better. The C.B.O. still needs to estimate how many people will be laid off as a consequence of the new subsidy to layoffs.

Over all, the C.B.O. predicts that 11 million people will receive premium assistance in 2015 (and even fewer in 2014), the vast majority of whom would be employed or dependents of an employed person. Yet the Recovery Act’s experience suggests that three million or four million people will receive premium assistance through unemployment alone, not to mention the millions more that receive it while working.

Be prepared for some unpleasant surprises over the next year or two, both as to the amount that the labor market is depressed and the unanticipated federal spending that will be needed to provide the benefits promised by the Affordable Care Act.

Subsidizing Layoffs

I noticed this in an Urban Institute report:

Some large‐firm interviewees reported that before ARRA they provided some amount of free or reduced cost COBRA coverage for laid‐off workers, based upon the prior duration of employment. ...Several of these companies reported that they reduced or dropped this prior benefit in reaction to ARRA.

In other words, these employers normally were paying for health insurance for workers they laid off, but for a while (from April 2009 to May 2010) the ARRA picked up the tab. This is yet another reason why laying people off during the recession was cheaper than layoffs normally are.

Andrew Greeley: Controversy Needs Careful Measurement

Copyright, The New York Times Company

The Rev. Andrew M. Greeley, who died on Thursday, was a creative and dedicated social scientist who taught economists and others that scholars help resolve controversies by making careful measurements.

Father Greeley was a Roman Catholic priest, well known outside of academic circles for his outspoken views on Church matters (the late Cardinal John Patrick Cody of Chicago refused to assign Father Greeley to a parish) and his best-selling mysteries and romance novels. But he also earned a doctorate in sociology at the University of Chicago in 1962 and was a senior study director at the National Opinion Research Center at Chicago.

Father Greeley actively participated in academic discourse at both of those institutions for the rest of his life. His participation included sharing wisdom and ideas with young social scientists arriving at the University of Chicago, as he did with me when I arrived in the early 1990s.

In his dissertation work, Father Greeley tried to help answer controversial questions by gathering better data. One anecdote from his graduate student days – a time when Americans were actively debating whether it would be appropriate to have a Catholic, John F. Kennedy, as president of the United States – succinctly illustrates Father Greeley’s approach.

He was interested in assertions that Catholics were not segregated from Protestants, especially in Midwestern cities like Chicago. As evidence against segregation, Father Greeley told me, many people pointed to Chicago institutions that included significant numbers of both Catholics and Protestants. The Beverly Country Club on the southwest side of the city was one of those institutions, and in fact had roughly equal numbers of Catholic and Protestant members.

Father Greeley wondered whether the club was nonetheless highly segregated on the inside, but, working long before the days of surveillance cameras and eye-recognition software, was faced with the challenge of measuring internal segregation. He approached the caddy master at the club, who kept records on which club members played golf and at what “tee time.” Up to four members could play golf together, and in doing so they would have a common tee time. Father Greeley was permitted to examine the tee sheets and found that Catholics and Protestants rarely shared a tee time: Catholics and Protestants might have been at the same club, but they were not golfing together.

(If you are wondering how tee sheets would indicate religion, Catholics in the Beverly neighborhood were primarily Irish and had distinctly Irish surnames. Moreover, Father Greeley was the assistant pastor at a Catholic parish in that neighborhood and knew many of the families).

Father Greeley’s passion to inform controversies with new and better data suited him well for his positions at the National Opinion Research Center, where in the early 1970s he helped initiate the General Social Survey, which continues today and is widely used in economics and other social-science research. For example, the survey results he examined helped overturn the belief that Catholics were less educated than the average American. He also helped fund expansions in the General Social Survey, and its international counterpart the International Social Survey Programme, to include additional topics and additional nations.

He and I talked about the welfare state, especially its differences between the United States and Western Europe. One point of view, still held today, is that Western Europeans are more compassionate toward their less fortunate neighbors. After all, the United States had traditionally devoted less of its national income to social programs than did France, Sweden and other nations. But Father Greeley noted that results from international surveys of volunteering and giving showed that Americans spent more money and time in volunteer work than Western Europeans did, so perhaps American compassion takes a different form.

Survey results are criticized when they go against the conventional wisdom. Father Greeley acknowledged that surveys have flaws but explained that conventional wisdom often has those flaws in spades.

“Everybody takes surveys,” he wrote. “Whoever makes a statement about human behavior has engaged in a survey of some sort,” adding:

The difference between the survey takers and the rest of generalizing humankind is that the former (usually) observe large numbers of people in a representative sample that reflects the total population and are precise about their methods of data collection and analysis. Precisely because the professional survey taker is honest about his methods, he becomes an easy target for loudmouth critics who appeal to ‘what everyone knows’ and ‘common sense.’