In the United States, investigative journalists can often begin their work with a powerful tool: a Freedom of Information Act (FOIA) request. But access to public records isn’t created equally, especially in countries more hostile to investigative reporting.

In Yemen, there is no comparable system that reliably opens government records to journalists. There are few transparent government portals, public institutions have been fractured by more than a decade of conflict, and access to official records often depends heavily on political control and geography.

As the founder and editor-in-chief of Aljumhuriya Media Network, managing operations securely from abroad, I frequently face a difficult question: How do you build data-driven investigations when the public record is fragmented, inaccessible, or effectively nonexistent?

The answer is that in data-poor environments, data journalism is not simply about analyzing existing datasets. It is about hunting, extracting, constructing, and verifying datasets from fragmented sources.

Here is the methodology we use to turn a data void into an investigative resource.

Start with a data map, not a spreadsheet

Before collecting any data, we map where the information might exist. A government database may be unavailable, but fragments of the same information can exist elsewhere: international organizations, humanitarian assessments, local news reports, court documents, company records, academic studies, satellite imagery, shipping databases, and even archived web pages.

In Yemen, this means looking beyond official portals. Conflict datasets such as ACLED, humanitarian platforms, UN reports, and international NGO assessments provide vital starting points. Archived web pages can also preserve information that institutions later remove, revise, or make inaccessible. When one official dataset does not exist, we look for the fragments that collectively describe the same phenomenon.

Proxy data mining: Build the dataset from the outside in

When domestic sources go dark, we look outward. We rely on what I call "proxy data" — information produced for another purpose that can answer part of an investigative question.

For example, instead of asking local authorities for import records or port activity, journalists can examine maritime tracking data, international customs documents, or satellite imagery to reconstruct a supply chain. The same approach applies to public spending, displacement, food prices, and aid distribution. A humanitarian report may not be designed as a database, but dozens of reports published over several years contain enough repeated indicators to build one.

This requires a crucial shift in mindset. Do not ask, "Where is the database?" Ask, "Where are the individual observations that could become a database?"

Mine local media systematically

Local journalism is not merely a source for quotes; it can become a longitudinal dataset. In Yemen, hundreds of local reports contain small pieces of information that are never compiled centrally: clashes, arrests, fuel prices, electricity outages, or road damage.

Instead of reading these reports only as individual stories, we code them. For every relevant report, we record specific fields such as date, governorate, event type, actors involved, numbers reported, source, and verification status. Over time, this creates a structured archive that can be analyzed. The key is to treat every article as an observation, not automatically as a fact. To prevent double-counting, we merge reports that share the exact same date, location, and key actors into a single event record with multiple source links.

Crowdsourcing: Turn scattered testimony into structured evidence

In a data-poor environment, human testimony becomes the raw material for a dataset. We establish secure channels to gather reports from people on the ground. However, crowdsourcing is not simply collecting messages; the real work is designing a verification system.

Each submission is logged with standardized fields. Sensitive identifying information is minimized or separated from the analytical dataset. While a single testimony may be weak evidence, multiple genuinely independent reports pointing to the same event, location, and timeframe can provide a much stronger signal. This is where crowdsourcing becomes a structured observation system.

Build geographic data when records disappear

Some investigations become possible only when text is converted into geography. If reports mention damaged buildings, displaced populations, or destroyed infrastructure, we geocode those locations and place them on a map.

Satellite imagery can then be used to examine physical changes over time. A government statement may claim that a facility was undamaged; an earlier satellite image and a later image provide an independent physical reference point. The underlying geographic dataset can then be analyzed and visualized as a map.

Triangulate the numbers and document the disagreement

In conflict zones, numbers are highly political. Warring parties often publish competing casualty figures or damage estimates. The objective should not be to blindly choose whichever number appears more credible.

Instead, preserve the disagreement. Record who reported each figure, when it was published, what methodology was used, and what independent evidence exists. Then compare the claims with other sources: local reporting, medical records, satellite imagery, or historical datasets. Sometimes the most important finding is not discovering the "correct" number, but discovering why the numbers diverge.

Build your own records, but make them reproducible

Data journalism is often associated with analyzing massive databases that already exist. In hostile and opaque environments, it is about creation.

But a journalist-built dataset is only useful if another journalist can understand how it was constructed. For every dataset, we preserve the original source, the date collected, extraction methods, definitions for every field, missing-data notes, and the level of confidence assigned to each record. This turns a private spreadsheet into a reproducible reporting asset.

The real advantage of investigating in the dark

The biggest misconception about data-poor environments is that the journalist has no data. Often, the data exists, but it is scattered across hundreds of documents, fragmented testimonies, maps, and incompatible databases.

The investigative challenge is to assemble those fragments without losing their context or overstating their certainty. The result is not a perfect dataset. It is something more useful: a transparent evidence base that can be challenged, updated, and independently verified.

When governments do not keep reliable public records, journalists can build them—piece by piece. That is what investigating in the dark truly looks like.