Archive for BLS Data

BLS Numbers: The Case For and Against Paranoia

At first, I thought the BLS numbers for September were weird. I thought about commenting on them, but I’m very slow in writing blog posts (what with trying to understand things and run the numbers and all) so I thought the moment had passed.

But then I saw Mickey Kaus’s post on “The Case for Paranoia” and I thought maybe I should add my 2 cents.

But first, I want to put forth my position. I hear a lot about how conservatives lack basic empathy, but I’ve been pretty frustrated at how liberals lack basic empathy over the results to this recent jobs report. Empathy is the art of seeing through the eyes of another human being, and it is a beautiful art… possibly the only true art. What I want to do here is to aid empathy. Why would someone be skeptical of the BLS numbers? Would they have any good reasons? And why might that skepticism be unwarranted?

The Case for Skepticism

If you look at this data one way, it actually looks very weird, very out of place. In September 2012 according to the BLS, the unemployment rate dropped from 8.1% to 7.8%, due largely to an increase of 873,000 jobs (as measured in the “A” Tables, which are based on a survey of individuals). However the “jobs increased” number (as measured in the “B” tables, which are based on a survey of business payrolls and is the number commonly reported) was only 114K, which is a pretty weak number. After all, just to keep up with population growth the job increase needs to be 125,000, right? So how can unemployment decrease so dramatically when the jobs number didn’t even keep pace with population growth?

Let’s look at every time in the history of modern job growth (since 1948) when we’ve seen +800K jobs increase in a month.


Something looks kind of weird here. In the last 18 years (not counting September 2012), the only times we’ve seen +800K job growth has been during January. Why is that?

It turns out every January, the BLS (Bureau of Labor Statistics) re-aligns the data to conform with population increases. So we may see employment increases that are augmented by population adjustments (and may not actually be “real” increases). So let’s take those out.


We’ve had such a huge non-adjustment employment increase only 6 times in the last 70 years. And, with those other increases, did we have similarly large corresponding “payroll jobs increases”?

It turns out this last September was the ONLY TIME IN THE HISTORY OF BLS DATA that we had an “employment” increase this large where the “payroll” increase didn’t even meet population growth.

In fact, since 1950, every single +800K employment gain has been joined by a +300K payroll gain. Outside of the census hiring in May 2010, we haven’t seen such healthy monthly payroll growth for any month since 2006.

You could even go a step further. This is also the first time we’ve seen an employment growth number this large that wasn’t preceded by 3 months of solid +200K payroll growth. So we’re looking at a fairly weird number here.

And this number, this number that is unique in the history of BLS numbers and is beneficial to the incumbent administration, just happened to come out just in time to influence an election that depends heavily on jobs numbers.

So, even if you don’t agree, I hope you can see why some people are skeptical of this jobs report.


The Case Against Skepticism

We have to keep in mind that there are 2 surveys that look at job growth. Think of the “B” tables as some guys calling employers and asking “how many people do you have on payroll?” Based on that number they come up with the “job growth” data. In September, that was an increase of 114,000 jobs. Very weak.

But you can’t call a company and ask “how many people do you NOT employ?” so to determine unemployment, they call individuals and ask “are you employed or unemployed”? This survey becomes the “A” tables and they take number of people looking for work, divide it by the number of people unemployed and get the unemployment rate.

Because of this, the two numbers (payroll increases in the B Tables and employment increases in the A Tables) can differ greatly. You could call all the companies in the Fortune 500 and ask how many they employ and cover millions of jobs. But if you call 500 individuals, that’s such a small sample size, it is basically meaningless. So the “A” tables (individuals) has a higher margin of error.

And we see that margin of error if we look at the data a little more holistically.


For those of you who (heart) some numbers, the standard deviation for the A Tables is significantly higher than the standard deviation for the B Tables (293,000 for A Tables vs. 209,000 for B Tables). This means that we’ll see a higher level of variability in the A Tables (the 873K job number) than we see in the B Tables (the 114K job number).

But looking at this data in this way, we see a couple things:

1) The September jobs report is a CLEAR outlier. It is totally reasonable to raise some eyebrows at this.

2) When we look at all the data, and not just pare it down to a few data points like we did above, we can see the September jobs report isn’t enough of an outlier to be considered unique. It could very easily be an artifact of randomness. The randomness just happens to fall  in a way some people don’t like.

And that second position is where I am. There is a lot of variability in the jobs data, especially the A Table employment data. Add into this the fact that we saw a lot of part time jobs added (600,000 of the 873,000 increase was in part time jobs) at the same time that we saw some major employers announce a shift to part-time workers in response to Obamacare and we see that maybe this report isn’t some conspiracy, maybe it is actually telling us something about the changing status of employment in the country.

UPDATE: Conn Carroll points out that part time jobs as a whole did not increase by 600,000, but instead fell by 26,000. What increased by 600,000 was the number of people working part time “out of economic necessity”, but that shouldn’t have influenced the overall job number. Only the overall number of part time workers should do that.


Is it a weird jobs report? Yes. But it isn’t unique in its weirdness and there are some very important extenuating circumstances that help explain it.

I’ve been following jobs reports very carefully for about 3 years. I’ve run through the historical numbers dozens of times, looking for averages, estimates, trends and patterns. For what it is worth, I don’t see anything that would suggest any kind of conspiracy or number tampering.

I know the numbers well enough to say that this was an odd report and I wish others would give the BLS skeptics a little bit of slack. This was a weird report, no doubt about it. But an understanding of how the report is compiled and a little bit of exploration shows that this report wasn’t so weird as to warrant particular skepticism.

BLS B Tables (Jobs By Industry) Treemap

I’m going to try something that is a little dependent on me always being on top of things. So I can tell you right now it’s a terrible idea.


I’ve been working for some time to make BLS data a little more accessible to the average person (read: the average wonk) and this something of a high point on that project.

In summary: Every month on the first Friday of the month, the Bureau of Labor Statistic releases two tables of jobs data. The A Tables contain employment, unemployment, the unemployment rate and labor force numbers. This is where we get the unemployment rate from. The B Tables contain detailed payroll data and a breakdown of payrolls by industry and sub-industry. This is where we get the “XYZ new jobs” number from. Due to the level of detail in the BLS B Tables, there is a lot of insight to be drawn from which industries are rising or falling (including public sector vs private sector jobs).

I’ve created a system where I can quickly snag all the BLS data from the most recent jobs report and display it in a treemap visualization, making it easy to explore.

So… here it is (interactive version).

And here’s a static version

The size of the boxes are proportional to the number of jobs in that industry and are colored according to the growth in that industry over a given time period. You can adjust the time period to color the boxes according to growth over the last month, the last 12 months, since Obama took office and over the last 10 years.

If you have a slower machine or are looking at it on a mobile device, you might be disappointed. It is a somewhat large visual and it is optimized for traditional desktop interaction. However, I’m hopeful that I can keep on top of this and post this visual monthly as the BLS numbers are released.

Romney, Obama, and Executive Job Records

This is one of the Goose/Gander Visualization Series.

Recently President Obama’s team has felt that attacking Romney’s jobs record in Massachusetts tests well in the sample group.

These attacks got me thinking about executive job records.  “Where” I asked myself  “would President Obama place in a ranking of US Presidents in terms of job creation?”

Job Gains By Presidential Tenure Medium

You can also download a larger version of the chart. I find it difficult to create visualizations that work well in both blog form and Facebook-sharing form. This was my attempt at a compromise.

Is this a fair comparison? Yes and no. Part of the Goose/Gander series is that I create a provocative visual and then explain in more details what is fair and isn’t fair about it.

This Isn’t Fair

President Obama hasn’t had a full term yet

This puts him at a distinct disadvantage to everyone else (except John F Kennedy) because he hasn’t had the same amount of time to grow jobs. However it also seems pretty obvious that he’s not going to get out of last place before January 2013. That would require 300K new jobs per month every month from now until then.

President Obama came into office in the middle of a recession

In fact, he came in the middle of a recession that was worse in terms of job loss than anything any other president in this chart had to deal with. Now, he did split those job losses about half-and-half with George W Bush, so it’s not as bad as it could have been for him.

Presidents only have a certain amount of control over job growth

Actually presidents (and executives in general) only have a certain amount of control over the economy, so this entire exercise is kind of tainted by that fact. But this is the part where we point out that Obama did start this by attacking Mitt Romney’s job record in a similar way.

This Is Fair

The data Is Unassailable

I’m using the Employment table from the BLS A Tables. This is not the one that most Obama proponents prefer to use. They prefer using the BLS B Tables because they give numbers that are kinder to Obama. But the B Tables undercount employment (they only count payrolls) and everyone knows this.

I counted January-January (or whenever the president left office) for each president. I did this not because it was particularly fair but because I wanted to match how Obama has assigned himself and Romney jobs responsibility. I’m following his lead to show that, if we take him at his word, he doesn’t stand up to his own standard.

If we’re going to play the presidential job visuals game…

… this is a totally fair visual to keep in mind. Depending on the metric, Obama talks about jobs in different ways. When talking raw numbers, he likes to talk about the “last 22 months” or however gets us to the low point in the recession. When talking about month-to-month change, he likes to talk about when he came into office which was the worst point of job loss in the recession, so everything else looks good in comparison.

Fairly or unfairly, Presidents and jobs are commonly linked. It’s only fair to give a proper representation of that information.

May 2012 BLS Jobs Data (BLS Friday)

I’ve been a bit of a slacker, but I’m trying to get back on wagon.

Here are the A and B Tables for the May 2012 BLS Employment report in csv format

May 2012 BLS A Tables (Household Survey – Population/Labor Force/Employment/Unemployment)

May 2012 BLS B Tables (Payroll Survey – Non-Farm/Private/Jobs by industry)

And I want to capture my initial analysis so you can see what you’re missing on Twitter (and so I can come back to it later)

And Annie Lowrey has what I thought was a fantastic summary of how this jobs report felt

We Did Not “Lose” 1.2 Million People (Updated)

UPDATE: Nate Silver pointed me to the BLS extrapolation of their population adjustment data (Table C at the bottom of this release). Long story short: the 1.2 million change in “Not in the Labor Force” is due entirely to the population adjustment and is not (as I assumed when writing this post) a straight across-the-board proportional increase. From December-January, we actually saw a “Not in the Labor Force” decrease which would mean people returning to the workforce. This is awesome. I’ll leave this post here as a testament to my own ignorance.

BLS data came out today and, if you follow me on Twitter, you’ll know that’s kind of “my thing”. I tried to explain this in tweet form, but it requires a bigger canvas.

There was some noise made today that we say “1.2 million people drop out of the workforce”. Zero Hedge and Iowahawk, both of whom are usually really good with numbers, made this claim.

And I want to explain why that isn’t so.

First, let’s look at where that number comes from. In the BLS Employment tables, there is a stat called “Not In The Work Force”. That number rose from 86.7 million in December 2011 to 87.9 million in January 2012, a rise of 1.2 million in one month.

Usually, when I talk about “people dropping out of the labor force”, I’m talking about the actual stat “Civilian Labor Force Level” decreasing as a raw number. This isn’t what happened. In fact, the Labor Force jumped half a million between December and January (the third biggest jump since the recession began).

So… the labor force rose dramatically, but “not in the labor force” also rose dramatically. Why is that?

The answer lies in the population. Normally population increases are a super-boring statistic in the job report. They always go up about the same amount, 150K-200K per month. However, the BLS does an annual adjustment every January. The last two adjustments have been downward, but this January, the adjustment was a huge upward one.

That’s a population adjustment of 1.7 million people. That’s big. In fact, that’s second biggest population adjustment in BLS data ever.

So when the population is adjusted, everything else in the data set gets adjusted too. The Labor Force goes up, Employment goes up, and the Not in the Labor Force goes up.

Now, the concerning thing about this adjustment, and the only mote in an otherwise spotless jobs report, is the distribution of this adjustment. We have a labor force participation rate of 63.7% (which is very, very low). If we add a hundred people, ideally we’d like to see at least 64 of them added join the workforce (to keep that participation rate up). We want to see the labor force grow at least on the level of the population.

Instead, what we got was this:

The BLS added 1.6 million people to the population number but only 30% of those were added to the Labor Force. That’s upside down from what we should expect in a normal adjustment.

Now… what does this mean?

Here is my theory based on how I understand the stats work.

At the end of 2011, the BLS re-calculates their population numbers and says “Whoa… we’ve way undercounted the population, so let’s adjust that number.” They adjust that number up and all the other numbers with it go up too. So what happened was that the “Not in the labor force” count did go up, but the population adjustment amplified the increase. And the Labor Force probably went down, but the population adjustment made it look like it’s going up.

I did some rough calculations (adjusting the December population upwards so that it more closely matches the January population) and, if we smooth the population adjustment, the jobs report looks less rosy. It looks like we lost another 450K from the labor force, which accounts for nearly all of the drop in unemployment.

The complicated nature of these numbers means that this observation won’t get very much traction (and, honestly, I’m not 100% convinced I’ve accounted for everything so maybe it shouldn’t) but it fits into the overall observation that employment increases are just not keeping up with population increases. I wouldn’t say this is alarming, but it is concerning.

February 2012 BLS Data in Excel (CSV) Format

Trying to get better at posting these soon after they are released.

BLS A Tables (Labor Force, Employment Rates, Discouraged Workers, etc)

BLS B Tables (Total Non-Farm Payrolls, Private Sector Jobs, Government Jobs, etc)

For more information on this ongoing project of mine.

BLS Data to Excel Format & Source Code

Last month, I published BLS Data in Excel format. This can be helpful for anyone who ever wants to really dig into the data but doesn’t have the time to pull data out of the atrocious BLS data tables.

I’m going to try to make this something of a monthly thing, putting these data files on my website as soon as I can convert them so that the latest BLS data is always available in a helpful format.

Additionally, I’ve added the files for employment by metro. It’s only up till September, 2011, but it’s still some super cool data.

January 2012 BLS Files

A Tables (Employment/Unemployment)

B Tables (Employment By Industry)

State Employment/Unemployment

State Employment By Industry

Metro Employment up to September 2011

Brief Interruption To Beg

This took a not-insignificant amount of time and if you use it in anything resembling a professional capacity, I’d really appreciate a beer as a way of saying thank you.


BLS-To-Excel Application

For those of you who are a little more interested in the data and willing to follow a lot of directions, I’ve decided to publish the program I use for this so that you’re not reliant on me to publish this every month. I do mostly Microsoft development, so you’ll need Windows to run the project

BLS Data To Excel Setup

I’ve also loaded the project to github so you can go download the source code and make it better.

BLS-Data-To-CSV on github

The code is a disaster in a large part because the BLS data is something of a disaster. However, the app itself contains some helpful tutorials on how to get the data and make everything work.

It looks awful. But if you follow the directions, it works.

This will never be a professional application, but I’ll update it as I can. If you happen to have any talent in design, my “thing” is translating designs to reality. So if you want to send me even a screenshot of how you think this app should work, I’m happy to incorporate that into the next version.