We Did Not “Lose” 1.2 Million People (Updated)

UPDATE: Nate Silver pointed me to the BLS extrapolation of their population adjustment data (Table C at the bottom of this release). Long story short: the 1.2 million change in “Not in the Labor Force” is due entirely to the population adjustment and is not (as I assumed when writing this post) a straight across-the-board proportional increase. From December-January, we actually saw a “Not in the Labor Force” decrease which would mean people returning to the workforce. This is awesome. I’ll leave this post here as a testament to my own ignorance.

BLS data came out today and, if you follow me on Twitter, you’ll know that’s kind of “my thing”. I tried to explain this in tweet form, but it requires a bigger canvas.

There was some noise made today that we say “1.2 million people drop out of the workforce”. Zero Hedge and Iowahawk, both of whom are usually really good with numbers, made this claim.

And I want to explain why that isn’t so.

First, let’s look at where that number comes from. In the BLS Employment tables, there is a stat called “Not In The Work Force”. That number rose from 86.7 million in December 2011 to 87.9 million in January 2012, a rise of 1.2 million in one month.

Usually, when I talk about “people dropping out of the labor force”, I’m talking about the actual stat “Civilian Labor Force Level” decreasing as a raw number. This isn’t what happened. In fact, the Labor Force jumped half a million between December and January (the third biggest jump since the recession began).

So… the labor force rose dramatically, but “not in the labor force” also rose dramatically. Why is that?

The answer lies in the population. Normally population increases are a super-boring statistic in the job report. They always go up about the same amount, 150K-200K per month. However, the BLS does an annual adjustment every January. The last two adjustments have been downward, but this January, the adjustment was a huge upward one.


That’s a population adjustment of 1.7 million people. That’s big. In fact, that’s second biggest population adjustment in BLS data ever.

So when the population is adjusted, everything else in the data set gets adjusted too. The Labor Force goes up, Employment goes up, and the Not in the Labor Force goes up.

Now, the concerning thing about this adjustment, and the only mote in an otherwise spotless jobs report, is the distribution of this adjustment. We have a labor force participation rate of 63.7% (which is very, very low). If we add a hundred people, ideally we’d like to see at least 64 of them added join the workforce (to keep that participation rate up). We want to see the labor force grow at least on the level of the population.

Instead, what we got was this:

The BLS added 1.6 million people to the population number but only 30% of those were added to the Labor Force. That’s upside down from what we should expect in a normal adjustment.

Now… what does this mean?

Here is my theory based on how I understand the stats work.

At the end of 2011, the BLS re-calculates their population numbers and says “Whoa… we’ve way undercounted the population, so let’s adjust that number.” They adjust that number up and all the other numbers with it go up too. So what happened was that the “Not in the labor force” count did go up, but the population adjustment amplified the increase. And the Labor Force probably went down, but the population adjustment made it look like it’s going up.

I did some rough calculations (adjusting the December population upwards so that it more closely matches the January population) and, if we smooth the population adjustment, the jobs report looks less rosy. It looks like we lost another 450K from the labor force, which accounts for nearly all of the drop in unemployment.

The complicated nature of these numbers means that this observation won’t get very much traction (and, honestly, I’m not 100% convinced I’ve accounted for everything so maybe it shouldn’t) but it fits into the overall observation that employment increases are just not keeping up with population increases. I wouldn’t say this is alarming, but it is concerning.

BLS Data to Excel Format & Source Code

Last month, I published BLS Data in Excel format. This can be helpful for anyone who ever wants to really dig into the data but doesn’t have the time to pull data out of the atrocious BLS data tables.

I’m going to try to make this something of a monthly thing, putting these data files on my website as soon as I can convert them so that the latest BLS data is always available in a helpful format.

Additionally, I’ve added the files for employment by metro. It’s only up till September, 2011, but it’s still some super cool data.

January 2012 BLS Files

A Tables (Employment/Unemployment)

B Tables (Employment By Industry)

State Employment/Unemployment

State Employment By Industry

Metro Employment up to September 2011

Brief Interruption To Beg

This took a not-insignificant amount of time and if you use it in anything resembling a professional capacity, I’d really appreciate a beer as a way of saying thank you.

 

BLS-To-Excel Application

For those of you who are a little more interested in the data and willing to follow a lot of directions, I’ve decided to publish the program I use for this so that you’re not reliant on me to publish this every month. I do mostly Microsoft development, so you’ll need Windows to run the project

BLS Data To Excel Setup

I’ve also loaded the project to github so you can go download the source code and make it better.

BLS-Data-To-CSV on github

The code is a disaster in a large part because the BLS data is something of a disaster. However, the app itself contains some helpful tutorials on how to get the data and make everything work.

It looks awful. But if you follow the directions, it works.

This will never be a professional application, but I’ll update it as I can. If you happen to have any talent in design, my “thing” is translating designs to reality. So if you want to send me even a screenshot of how you think this app should work, I’m happy to incorporate that into the next version.

BLS Data in Excel Format

If you’ve ever tried to get the data out of the Bureau of Labor Statistics (BLS) you know that can be kind of a pain in the butt. You usually have to go through the wizards to get the data and then it only gives you one kind of data per table and you have to do a lot of tedious work to get that data into a format that is actually useful for additional work.

I finally got tired of doing this, so this weekend I put together a little program that takes BLS data and turns it into csv format, which can be opened in Excel.

Finally, I would really appreciate two things. First, please mention me (Matthias Shapiro) if you use the data for professional purposes, link back here to let people know where to get it. Second, if this is actually helpful data, please consider tossing $5 or so into my digital hat. I make maybe $50 per year from this blog, so anything to let me know this is worth doing is helpful to me.





Or you could buy a copy of my book Beautiful Visualization (disclaimer: I only wrote one chapter, but I call it “my book” anyway because that makes me feel important).

Download employment status (A Tables) (BLS link)

  • 1948 – Nov 2011
  • civilian population
  • labor force
  • participation rate
  • employed
  • employment-to-population ratio
  • unemployed
  • unemployment rate
  • not in labor force
  • persons who currently want a job
  • 1939-Nov 2011
  • payroll job counts for 150 industries/sub-industries
  • csv file (headers labeled “[state] – [field]” Example: “Alabama – Unemployed”
  • xlsx file (better headers, grouping states)
  • 1976 – Oct 2011
  • labor force by state
  • employed by state
  • unemployed by state
  • unemployment rate by state
Download state payrolls by industry (BLS Link)

11 Reasons Occupy Wall Street Should Become Occupy Foreclosure

I was speaking with Brendan Loy the other day and he made the comment (I paraphrase):

Usually when you perform civil disobedience, the act you’re performing is in direct relation to the injustice you’re protesting. For example, Rosa Parks refusing to give up her seat on the bus was in direct defiance of an unjust law requiring her to do exactly that. With the Occupy protesters, they’re against corporate greed, so they’re camping in a park. I don’t get that.

And I think he’s right. There seems to be a very loose relationship between what the protesters say they want and their method of protesting.

Giving this some thought, I think there is an civil disobedience action the Occupiers can take that would make a great deal more sense. And that is occupying foreclosures.

Hear me out here… I’m not the most sympathetic toward the Occupy movement, but occupying foreclosures has the following benefits:

  1. Real shelter means fewer deaths (as long as they don’t do drugs).
  2. The action is directly related to the financial sector (although they would quickly discover that Fannie Mae and Freddie Mac are bigger culprits than Goldman Sachs).
  3. It would be genuinely disruptive to the financial sector. Don’t fool yourselves, sleeping in a park is more disruptive to a bagel shop than to a hedge fund manager.
  4. Far less impact on small businesses whose owners just want to make ends meet.
  5. They could actually get arrested for peaceful civil disobedience (trespassing) rather than for jaywalking or public indecency.
  6. Good optics if they keep the houses clean & leave when they are sold. Local news pieces would relate directly to real neighborhoods, get great pictures of people and the houses they occupy. People could go check out the movement without heading downtown… the movement is right down the street.
  7. Build excellent community standing (if they are actually good community members in these neighborhoods).
  8. A good platform for spreading their position. If people come to see the houses for purchase, they can pass out literature about the pitfalls of tricksy banks and dangerous mortgages.
  9. They can attach themselves closely to the individual stories of woe within the local community. Every foreclosure comes with a story. They could take advantage of that.
  10. If banks decided it would be better to sell foreclosures for a loss rather than risk an occupation, it might move inventory, actually help solve one of the problems.
  11. Filter out the antagonistic element from Occupy. I suspect anarchists are less interested in playing house with a half dozen people than with running down the streets smashing windows.
Of course any movement is only as good as the people who are involved with it. But this path seems more targeted, sustainable and sanitary. And it might just be the best place to go next for Occupy.
Would I support this? Meh. Probably not wholeheartedly. It is still against the law (but civil disobedience is, by definition, against the law). And I’m sure there are some unintended consequence that I’ve failed to consider (there always are).
But at least it would make some kind of sense.

How To Cherry Pick Data

In his post “Senate Republicans Block Targeted Jobs Relief for Teachers And First Responders“, Matthew Yglesias points out that “during the Obama years” private employment has rebounded while government employment has seen a “sharp contraction”.

Yglesias points to a couple of charts, but I’ve helpfully replicated his data set into a single chart, because that’s just the kind of guy I am.

As you can see, using January 2009 as our point of reference, private jobs have rebounded from a drop of 3.79% in 2010 to a drop of 1.63% in August (my data is slightly out of date, but good enough for gov’t work… get it?!?). Local gov’t employment has fallen 3.6% in that same time frame. I also added federal gov’t employment (which has fallen 2.75% since January 2009) for the heck of it.

In the comments section, Peter Schaeffer complains that Yglesias is cherry picking the data and points out that gov’t employment saw +10% gains in the decade leading up to the crash and 3-4% losses from the peak while the private sector saw slightly less than 5% gains in that time period and slightly more than 5% losses from the peak.

I thought that Schaeffer had a good point, but needed some visuals to drive it home, so I thought I’d show Yglesias’ jobs data in Schaeffer’s context.

As you can see, Yglesias’ data starts at a really handy place for his argument, since it begins measuring job losses and growth at a time when we had already seen drastic private sector losses, but no public sector losses.

Of course, the funny aspect to this data is that one could use it to say that President Obama is reigning in the public sector that George W. Bush let grow out of control. I think the only reason no one is saying this is because everyone on President Obama’s side would consider that a bad thing and everyone who opposes President Obama would consider that a good thing. Neither side really wants to attribute this trend to President Obama. In fact, President Obama is working actively to reverse this trend.

Ah, the little ironies of life.

Note: In the spirit of “never attribute to malice what can be explained by incompetence”, I wouldn’t be surprised if Yglesias unwittingly cherry-picked the data. “The Obama years” is a perfectly rational place to start looking at data and, if that was the only data you looked at, it would support his conclusion. On the other hand, Yglesias has always had a better grasp of the data than this particular post suggests, so I suspect he kind-of-sort-of knew that this was a cherry picked sample set but was OK with using it because it bolstered his argument.

How To Read Unemployment Reports

Every time a national unemployment report comes out, I tweet the many details from @politicalmath. Frequently I get a lot of the same questions, so I thought I’d jot down a quick summary on unemployment reports and numbers and where they come from.

There are 2 kinds of employment numbers, summarized here:

  1. Establishment Data (Current Employment Statistics or CES) – this survey covers 400,000 businesses and counts the number of payroll positions that are filled.
  2. Household Data (Current Population Survey or CPS) – this survey covers 60,000 households and counts the number of people who are employed and unemployed.

When an employment report comes out from the Bureau of Labor Statistics (BLS), they usually report:

  1. The unemployment rate, which is calculated using household data
  2. The number of jobs added, which comes from the establishment data

Sometimes this data can seem contradictory. For example, between March and  June 2011, we gained 290,000 jobs but the unemployment rate went up .4% (from 8.8% to 9.2%).

There can be a couple reasons for this. The first one is that, the “jobs added” number comes from subtracting last month’s establishment jobs number from this month’s establishment jobs number, but we never use either of those numbers to calculate the unemployment data.

Why?

Because the essence of the establishment jobs number is asking employers: “How many people work for you?” It gives a nice accurate number, but it doesn’t tell us anything about how many people don’t work for them. We don’t have any number on the unemployed, only a number for jobs.

For unemployment, we have to go to individuals and ask them: “Are you employed or unemployed?” Then we take the unemployed number and divide it by the total number of people who are in the labor force, which counts both the employed and the unemployed.

But even the differences between the establishment jobs number and the household jobs number can be big. According to the household jobs number (which is supposed to exclude farm workers and the self-employed), we had 139.6 million jobs in August 2011. According to the establishment jobs number, we had 131.1 million.

That’s a difference of 8.5 million jobs, and that kind pf spread is pretty normal. The variation changes a little month-to-month, but we could get a report of  jobs created from the household number and jobs lost from the establishment number. In fact, we saw something similar in August where the household number said we gained 331,000 jobs, but the establishment number said we gained 0.

So why is the establishment number reported?

Because the establishment survey is so much larger, more reliable and gives more consistent results. In the graph below , we can see that even though the establishment data counts fewer jobs, it is a less erratic count.

So… that is a quick explanation of the employment report. I dig into this data once a month, so I’m pretty familiar and I’m delighted to answer questions or explain in greater detail in the comments.