I’ve always considered the response to a big data request a fair indicator of how good the story might be:
“Why do you want this? Nobody’s ever asked for that before.” Nice.

“It’s impossible to get this to you and even if we could, you wouldn’t understand it.” Even better.

“Screw the public records law. You’ll need a court order.” I’m drooling into my keyboard.

But I’ve had to rethink that philosophy following publication of “Unfit for Duty,” our nine-day series on how state and local officials handle serious incidents of misconduct by ...

Every day, thousands of Muskegon County residents pour into their favorite restaurants to dine with friends and family. Yet until now, little was known about whether those restaurants followed practices aimed at preventing people from getting sick.

While looking through more than 22,000 electronic health-inspection records spanning four years, I found numerous instances where restaurants repeatedly violated rules that help prevent foodborne illness. Schools, hospitals and food stands were cited for breaking the rules, too.

Raw chicken and crabmeat sitting out at room temperature. Food kept past its expiration date. Cockroaches, mice and fruit flies living in kitchens. Employees ...

After I learned the banks that had been bailed out by taxpayers had become the main purchasers of tax liens in Arizona, I knew I was onto a potentially big story.

I began interviewing several tax lien buyers and found that this was a common practice not only in Tucson and throughout Arizona, but all over the country. My goal was to use what was happening here as a window into the larger story.

From the beginning, I thought the key to the story was the fact that the bailed-out banks were using money to buy up tax liens rather ...

U.S. Immigration and Customs Enforcement agents deported an illegal immigrant who was a witness in a homicide case, an attorney told me. "Prosecutors are going to drop the charges."

Sure enough, the case crumbled and the U.S. citizen who had been charged with murder was set free. ICE agents had deported the witness after he was arrested on unrelated charges and booked into the county jail, court documents showed.

Local and federal authorities responded by pointing fingers at each other and insisting this was an isolated incident. The problem, a breakdown in communication, had been addressed, they promised ...

Many Texans had no idea that carcinogenic radiation was in their tap water.

For decades, the environmental agency that was supposed to protect the public from pollution had been deliberately changing radiation test data for water systems.

Not only had false data been reported to consumers, but the "lowballing" also allowed water providers to avoid breaking federal safe drinking water rules.

This all began when KHOU I-team member Keith Tomshe noticed a disclaimer on his water bill stating that small amounts of arsenic, also a carcinogen, had been found in his neighborhood’s drinking water. The disclosure is called a ...

Badger Guns and Ammo in suburban Milwaukee rose to national prominence in the 1990s when reports showed it had sold more crime guns than any other dealer in the nation.

In 2007, I learned from sources that the U.S. Bureau of Alcohol, Tobacco, Firearms and Explosives had found serious problems at the store, known then as Badger Outdoors. Such records on gun stores are shrouded in secrecy by law but I discovered the ATF was considering revoking the license – a rare move taken against only the worst gun dealers.

However, a few months later ATF’s plan evaporated. The ...

Located just south of Chicago, Indiana's Lake County has long been a hot-bed of political corruption, bloated government and patronage jobs. County government spending became so rampant that the largest corporate taxpayers in 2005 commissioned a study that recommended cutting government spending or consolidating services to save taxpayers millions. Few of the recommendations were adopted, and about two years later, the Indiana General Assembly forced a frozen tax levy specifically targeting the county's wasteful ways.

Times of Northwest Indiana reporter Bill Dolan and I spent four years collecting a decade's worth of electronic spending records for all ...

I had always wanted to write a story about police chases after I watched a crazy high-speed pursuit unfold on local television. I wondered how often these chases go bad, and how the San Antonio Police Department kept track of that information.

Law enforcement agencies usually churn out tons of paperwork. For instance, I learned that officers must fill out a pursuit-evaluation form after they chase someone. The reports have check boxes for different categories of information, such as whether someone was injured during the chase.

When you see boxes like that on a report, chances are, someone at the ...

Two colleagues approached me last summer with an intriguing pitch: They wanted to trace the meat, poultry and other food served in school cafeterias all the way back to their manufacturers. Parents, they said, were often in the dark about the quality of the food their kids eat at school — much less who supplies it — and they suspected school officials didn't know enough about the foods' sources to act when students fell ill.

How could I resist the challenge? I love food. A couple months later, though, I'd start thinking twice about having a hamburger, thanks to what ...

NOTE: This tutorial assumes a basic familiarity with the Python language and interpreter, an interactive environment for running code. It also requires Python 2.6 or later.

As a database editor, my daily routine often involves pulling data from spreadsheets and databases and processing it for a variety of other tasks: automated email alerts, visualizations, mashups with data from other sources.

The moment data leaves the confines of Microsoft Excel spreadsheets or a relational database manager, you lose all those handy column headers that help sort, filter and otherwise make sense of information. This recipe shows how, using Python, you ...