By now you may have read that EveryBlock, a Knight Foundation-funded project, has released its source code to the public (here's a browsable version). Getting a chance to look under the hood is a great opportunity to see how other folks tackle some of the tasks we all face, or are likely to. The first thing to note is that the code has the GPL license, which means that if you incorporate any of it into an application you're building and then release that code, it will need to be under the terms of the GPL as well ... Read more ...

I drive by an elementary school on my way to work every day. More than once there’s been a police cruiser idling in the school’s parking lot with lights flashing and the officer standing nearby. Although those incidents never involved a major crime, on several occasions this year the Tulsa World has chronicled arrests at schools. In January, police arrested an 18-year-old man found with a stun gun, two samurai swords and six knives in his car in a high school parking lot. In February, police arrested a 59-year-old man after he pointed a gun at students standing ...

Read more ...

The federal government launched Data.gov a little less than a month ago with raw databases, data extraction tools and widgets and pledged to bring "unprecedented access to government information." The White House said the site would allow "unfiltered access to government data streams in machine-readable formats." At launch, the site offered 73 links to demographic, economic and environmental data files and tools that had already been available on federal government Web sites, some of them for years. Now the catalog offers nearly 300. It's clear that the White House is going after the low-hanging fruit in the first ... Read more ...

The process of geolocating information isn't new to journalists; producing maps has long been a key part of what we do. But when it comes to our stories, extracting mappable entities like cities from text is a relatively new concept. There are commercial services that do this task, and researchers have created software for academic pursuits as well. Widespread free availability of geolocation services, however, has been mostly wishful thinking until last month. That's when Yahoo! announced Placemaker, an API for extracting geographic locations from text. It's important to note that Placemaker isn't a geocoder; it ... Read more ...

Some gun-rights advocates responded angrily when they found that a searchable database of Tennessee gun permit holders was posted on the Web site of the Commercial Appeal in Memphis. Many complained that their right to privacy had been violated. In turn, journalists have been debating the ethics of posting raw data on news organization Web sites and asking how those databases can serve journalism. The Commercial Appeal posted the database in December 2008. Chris Peck, then the newspaper’s editor, said in an e-mail that he didn’t foresee the rage gun rights advocates would unleash on the newspaper two ...

Read more ...

Outside hundreds of schools across the country, children are exposed to air that appears to be rife with chemicals that can exacerbate asthma or cause cancer. The reporting that led us to that astounding conclusion began with a relatively straightforward question: What's in the air outside the nation's schools? To find the answer, we turned to a variety of state and federal databases that pinpoint schools, detail the chemicals released from industrial facilities and estimate the potential severity of toxic air pollution across the country. Many newspapers have dealt extensively with the government's most basic pollution information ...

Read more ...

One of the biggest hurdles we all face for trying new software or utilities is the lack of a sandbox, a machine we can just use when we want to without having to worry if something goes wrong. This is particularly true for new open-source technologies, like the fast-growing field of open source GIS software. Sure, it would be great to try out OpenLayers or other mapping utilities, but it's not like we can just turn our main computer into a development box overnight. Luckily, this problem has been worked out for us. In the past week, I've ... Read more ...

On Feb. 20, a group of my colleagues at The New York Times gathered for a daylong series of presentations on a set of APIs that we've been releasing during the past few months. TimesOpen, as it was called, gathered about 140 developers and other folks interested in working with Times data. So what are APIs? The acronym stands for "application programming interface," but another way of describing an API is a programmatic way to access data. Rather than perform SQL queries to return the data you want, you'd use your browser or a script to retrieve data ... Read more ...

In the fall of 2007 my wife became pregnant with our first child. Like many expectant parents, one of the first things we began worrying about was daycare. There is an acute shortage of child care services in Canada. Often the only way to get your child in to one is to get your name on their waiting list as soon as you get pregnant. But when it came time decide which daycare to apply to, it was hard to choose. Government lists showed us which daycares were near our home, but we had no way of knowing which facilities ...

Read more ...

Folks doing CAR are blessed with a wealth of tools, which is both a blessing and a curse. For example, I frequently use two database programs, MySQL and PostgreSQL, at work. While similar in most respects, they have slightly different syntax for some common tasks such as string functions. You see this situation played out on the NICAR-L listserv all the time, when someone asks a question that usually starts with, "I know I've done this before, but I can't seem to remember the right syntax." Keeping snippets of SQL or Excel IF statements lying around in files ... Read more ...