Archive for May 21, 2013

Flawed Statistical Thinking, the IRS, and Loooooooong Odds

Let’s say there’s a casino where you can’t choose to play. Everyone is forced to participate in the gamble and instead of winning money, the “winners” get punched in the face. A man plays at the dice table 30 hours in a row and never gets punched in the face once. Lucky jerk. Then he says that the casino should be under different management. Within 15 minutes of this statement, he rolls snake eyes (the lucky face-punching number) three times.

Should we consider the possibility that his punches were perhaps the result of his statement about management? They could just be random. Randomness does work that way sometimes.

What if we then discovered that another table in that casino admitted to using loaded dice. Fans of the casino might argue that just because the dice were loaded at one table doesn’t mean they’re loaded at our poor sap’s table. But I think that most of us would take the triple face-punching and the loaded dice story, put them together, and conclude that management conspired to have their critic punched.

I float this analogy in service of recent events.

A few days ago, Nate Silver published a post noting the flawed statistical thinking from Peggy Noonan regarding IRS audits of Romney supporters. Go read it for yourself, but the long story short is that Silver is (rightfully) irritated by people who hold up a few anecdotal examples as proof of a conspiracy. However I suspect in his haste to make a larger valid point, Silver got the odds wrong on of one of the cases Noonan holds up.

In summary, Frank VanderSloot donated a substantial amount of money to groups in support of Mitt Romney. He was mentioned by name by President Obama’s campaign website in an attack that must have seemed normal to lots of people but seems mildly creepy to me. During the next four months, he had to deal with 3 audits: one individual audit, one audit by the Department of Labor and one audit of his business.

In a move that is very much unlike him and that I can only attribute to a careless reading of the events or “blogging-while-buzzed” (full disclosure: that is me right now) Silver has drastically miscalculated this case. Silver looks at the odds of a single audit and maintains (as any reasonable statistician would) that the odds of a single audit of a single individual do not rise to the level of conspiracy.

One of the things I love about randomness is how it builds. Flip a coin and call the result. Get it right and no one thinks you’re anything special. Same thing if you call the second flip correctly. By the third flip, people are getting irritated, waiting for you to be wrong. By the 10th flip, they want to change the coin because they suspect it’s rigged. Every subsequent flip drives the odds higher in a way that becomes almost impossible to comprehend. By the 34th flip, the odds of calling them all right far outweigh the population of the planet.

What we’re talking about here isn’t a single audit (which is what Silver based his math on). We’re looking at three different audits within four months of being mentioned by the President’s team as a “very bad man”.

What are the odds of that?

Fortunately for us, those odds are very easy to count. The odds of someone making over $1 million per year being audited is 12%. Let’s assume it’s similar for medium sized business. Then the odds of being audited AND your business being audited in the same year drops down to 1.4%. That’s certainly lower, but not so low that we should be overly suspicious. Hey… this kind of thing just happens.

But then add in the Department of Labor audit. The Department of Labor  “conducts more than 3,000 audits each year“. For the sake of statistical generosity, let’s call that 4,000. There are 30 million businesses in the United States, so the chance of an individual business being targeted randomly is pretty low, under 1%.

Now let’s add all those odd together. The chance of having all three of these audits in one year is 1 in 520,833 or generously rounded up to 0.002%. This is the kind of thing we look at askance and say “Huh. That’s really kind of weird.”

This would look weird even if the IRS hadn’t admitted to rigging the game (albeit in a different context). Given the admissions they’ve made so far, I think any serious mathematically minded individual should look at the odds and say “That’s odd.”

I feel that, given the circumstances, the burden of proof should lay at the feet of the auditors. I want them to prove this wasn’t politically motivated because, given that they’ve admitted to so far, this particular case is extremely weird.

This Is How We Disappear

(Skip to point 5 for my personal post-mortem on the Romney campaign)

I haven’t posted in 6 months and my Twitter presence (where I’ve usually done most of my interaction) has slowed to a crawl. I missed the last #BLSFriday (my monthly data-dig into the BLS employment data) and I haven’t made a decent chart in ages. Several people have asked what happened and I hate to just disappear without any explanation, so I wanted to put something up here.

I love being able to make a contribution to the political discussion. I love digging into data and asking questions that too much of the data community prefers to ignore. I love the people I’ve met and become friends with and my chances to speak and educate. But for a variety of reasons, I’ve had to pull back. I don’t like to just disappear and leave people in the dark (we miss you @Cubachi!) so I wanted to elaborate here.

1) I moved to the west coast

I didn’t think it would make a big difference, but moving into a time zone just 1 hour further from DC has really limited my Twitter engagement. By the time I get on at night, a lot of east-coast people I like to interact with have called it a night. I still try to check in frequently (and @stephenkruiser and @politicsofamy make the evenings pretty awesome) but it’s not the same interactions that I loved.

2) My new job requires my personality

Some of you know what my new job is, but for people who don’t, I’ll just say that it requires my personality. Whereas all my previous work relied on my ability to deliver a good product, this job requires that I put my face on my work in a big way. I’ve never been super-secret about my identity, but the nature of my new job requires that I keep my name and personality squarely in the professional sphere.

3) Baby + 2 year old

We just had our 2nd kid and our 2 year old is a delightful little time suck. As much as I love digging into data, building charts, making videos and arguing with the internet, I like spending time with my kids more.

4) This is a hobby

My political data work was fun, educational, engaging, and some of the best stuff I’ve ever made. I have gotten job offers by the dozen. But I have a career in which I make money. It’s not a huge amount of money but… well, let’s just say this conversation actually happened (although it is paraphrased):

Fox News: “Hi, this is (so and so) with Fox News. We’ve seen your stuff and we love it. What do you think about doing a regular piece for (show X).”

Me: Sound great. So… compensation… I was thinking [2/3 my going rate as a programmer].

FN: Yeah, that’s never going to happen. How about [1/6 my going rate as a programmer].

Me: Ha. Ha ha ha. Ha ha ha ha. Ha.

I think I actually laughed at them on the phone. The number was really that low.

We tend to think that people in DC make stupid huge money. But that’s true for very few people (usually corporate lobbyists and maybe some organization directors or higher-ups). Bloggers, media content creators, journalists… all these people get paid crap (with the exception of the very top-tier, let’s call them the 1%).

So, if there is a blogger, writer, video creator, podcaster, Twitter personality, etc who you enjoy, donate something to them. Anything is helpful.

5) Disillusionment

OK… now for the real reasons. In the last election, I was approached by someone in the Romney campaign to do some visualization work, charts, videos, that kind of thing. We agreed upon a reasonable rate for my work and I got started working on some concepts. The first visual I produced for the team was a variation of this visual, showing job growth by presidential terms.

The version I made for them was cleaner, better designed, conceptually a bit firmer, but the point was the same. After a number of iterations, I felt I had a great visual that I’d be glad to see be a point of conversation.

And then the approval process began. We spent weeks trying to get an OK on the visual. They asked for references for my data which I gladly included. (The only time I deny references to data is when people on Twitter refuse to do basic research and I want to know they’re willing to do basic research before I engage them.) The approval process for the most basic inoffensive visual showing how mediocre Obama’s jobs record was required the approval of a vast number of message managers, PR managers, researchers, etc. A single veto would kill the iteration and I’d have to resubmit with changes. Sometimes I knew what those changes should be, sometimes I didn’t.

After enough time it dawned on me: These people didn’t believe me. They didn’t believe my numbers (even though they were the most basic BLS numbers out there). I felt (and this is just my intuition talking here) that they had bought, hook line and sinker, the Obama teams “I created X million jobs” line (easily shown to be little more than a flimsy propaganda line based on selective data). I believe they were more willing to swallow the line being promoted by the opposition than a friendly voice with a history of dedication to the truthful portrayal of data.

At a certain point I said “screw this” and gave up.

And I never got paid.

I liked Romney. I voted for Romney (which, incidentally, marked the first time the candidate I voted for didn’t win). There are all sorts of reasons we can point to about why Romney lost. But from my perspective, I saw an over-managed campaign untrustful of their own side and unwilling to take the smallest risks for fear of being butchered by the media. Which, of course, happened anyway.

There’s actually one more reason, but it requires it’s own post. Suffice to say I’d love to keep making data beautiful, engaging the issues, digging into charts and making videos, but my life has changed substantially and for the foreseeable future