Developers will demo a beta version of the newsroom appliance at the 2012 CAR Conference.

Tucked away on reporters' computers are dozens of details that could benefit news coverage, if only other journalists knew where to look.

Newsrooms are swimming in data. Journalistic organizations big and small continue to collect data from local, state and federal governments, and dozens of other places. As the collection grows, making sense of that information can become more difficult.

That's what the PANDA project, a 2011 Knight News Challenge winner, wants to solve — make data analysis easier for journalists and make sharing ...

Read more ...

Read more ...

Your average CAR geeks - especially the old timers - follow a predictable route in tools they use for data analysis and sharing.

You start with Microsoft Excel spreadsheets, and everything’s going fine. But then someone tells you about relational databases, and suddenly you notice all of the things you can’t easily do in Excel.

Step up to Microsoft Access database manager and pretty soon you’re joining tables right and left, slipping terms like "Group By" and "normalization" into conversations and generally feeling pretty good about yourself.

But at some point, someone in your newsroom looks over your shoulder ...

Read more ...

The web now offers numerous free tools that give non-programmers the ability to create data-driven applications. Among those with the most promise, especially for journalists, is Google Fusion Tables.

Fusion Tables essentially is a free database manager in the cloud, allowing anyone to upload large data sets, merge them with other tables and create visualizations. It’s much more, though. The service has a detailed application programming interface (API), and basically can serve as a free back end to serve data and mapping applications. The service also is customizable while simple, with extra capabilities for more experienced users.

Of course ...

Read more ...

A new specialty in newsrooms is emerging that’s giving data new reach. It’s separate from computer-assisted reporting but shares much of the same DNA. Like CAR, it involves working intensively with data, but the end product is a web-based software application, not a story. A formal name for this field hasn’t completely gelled, but at ProPublica and some other newsrooms we call them “news applications.”

What are news applications? How do they relate to CAR? How can CAR nerds work with news apps nerds?

Simply put, news applications are journalism done with software development, much like photojournalism ...

Read more ...

Here’s how to use Google Spreadsheets to import RSS and ATOM data.

ImportFeed for RSS and ATOM

All sorts of data gets pushed out as RSS/ATOM feeds. You can put those in spreadsheets too. The command takes the following form:

=ImportFeed(URL, [feedQuery | itemQuery], [headers], [numItems])

• URL of the feed.
• We'll almost always use itemQuery options ("items", "items author", "items title", "items summary", "items url", or "items created"), as they return individual items in the feed while feedQuery just returns metadata about the feed.
• "Items" will be the best default option, as it returns everything you'll ...

Read more ...

By now you may have read that EveryBlock, a Knight Foundation-funded project, has released its source code to the public (here's a browsable version). Getting a chance to look under the hood is a great opportunity to see how other folks tackle some of the tasks we all face, or are likely to. The first thing to note is that the code has the GPL license, which means that if you incorporate any of it into an application you're building and then release that code, it will need to be under the terms of the GPL as well ... Read more ...

I had the privilege of speaking on a panel with Sarah Cohen and Steve Doig last week in Baltimore about the future of computer-assisted reporting. Whoever thought I even belonged in the same room as those two gave me way more credit than I deserved. But in preparing for that panel, I got to thinking: What skills and software tools are we going to be using in 10 years? What skills should we start learning now if we want to be prepared for the future? Or better yet: What types of problems in newsgathering and investigations could technology best help ... Read more ...

The process of geolocating information isn't new to journalists; producing maps has long been a key part of what we do. But when it comes to our stories, extracting mappable entities like cities from text is a relatively new concept. There are commercial services that do this task, and researchers have created software for academic pursuits as well. Widespread free availability of geolocation services, however, has been mostly wishful thinking until last month. That's when Yahoo! announced Placemaker, an API for extracting geographic locations from text. It's important to note that Placemaker isn't a geocoder; it ... Read more ...

As news outlets struggle to make bottom line, journalists might look for new ways to save some cash. Fortunately, journalists doing CAR can look toward no-cost open-source database managers as powerful alternatives to commercial software. Open-source database applications like MySQL and PostgreSQL are more resilient than commercial software applications, such as Microsoft Access, and offer more powerful capabilities for analyzing and synthesizing data. Glen McGregor, database manager for the Ottawa Citizen, used to rely on Access to query data he received from the federal government. When McGregor made a request to the federal government to release data revealing where guns ...

Read more ...