Interactive Media & Services Data: Video, Streaming, Music Catalogs and Social Graphs · Head-to-head
Wayback Machine CDX Server API vs OONI Explorer – Global Internet Censorship Measurements
Which interactive media & services data: video, streaming, music catalogs and social graphs data fits your job: Wayback Machine CDX Server API, or OONI Explorer – Global Internet Censorship Measurements. API, files, or your warehouse. Daily, weekly, or hourly.
Wayback Machine CDX Server API
OONI Explorer – Global Internet Censorship Measurements
Where the fields line up
No shared field names. These two answer different questions.
| Field | Wayback Machine CDX Server API | OONI Explorer – Global Internet Censorship Measurements |
|---|---|---|
urlkey | documented | not in this set |
timestamp | documented | not in this set |
original | documented | not in this set |
mimetype | documented | not in this set |
statuscode | documented | not in this set |
digest | documented | not in this set |
length | documented | not in this set |
probe_cc | not in this set | ISO country code of the probe's location - the primary geographic slicing column. |
probe_asn | not in this set | Autonomous system number of the network the probe rode, resolving each measurement to a specific operator. |
test_name | not in this set | Which nettest produced the row: web_connectivity, whatsapp, telegram, signal, http_invalid_request_line, dash and the rest of the ten-family shelf. |
input | not in this set | URL or target the test probed; null for app-based tests, which check connectivity rather than fetch a page. |
measurement_start_time | not in this set | UTC timestamp of when the measurement began. |
Coverage, side by side
| Wayback Machine CDX Server API | OONI Explorer – Global Internet Censorship Measurements | |
|---|---|---|
| Geographic | Global public web as crawled - geography is the address space itself, not a country code | Nearly 200 countries, natively keyed by country code and network autonomous system |
| Temporal | Captures from 1996 to the present, added continuously; earliest row observed in the August 2026 verification pass dated January 17, 1999 | Continuous collection since 2012; every row carries a UTC start time |
| Granularity | One row per capture, timestamps resolving to the second; reducible to hourly or coarser timelines | One row per measurement per probe session, filterable by country, network, nettest and date |
What each contains
They tie on 1 attribute. Pick by fit, not by loyalty.
| Wayback Machine CDX Server API | OONI Explorer – Global Internet Censorship Measurements | |
|---|---|---|
| Publisher | Internet Archive | Open Observatory of Network Interference |
| Subject lens | The query layer over the web's memory: one row per capture of any archived URL | A measurement corpus over the network's interference: one row per probe test of a URL or app |
| What one row means | A single fetch of a specific URL at a 14-digit timestamp, with content type, status code, digest and compressed record length | A single probe run from one country and one network against one target, labeled anomaly, confirmed or failure and graded with blocking scores |
| Geographic coverage | Global public web as crawled - geography is the address space itself, not a country code | Nearly 200 countries, natively keyed by country code and network autonomous system |
| Temporal reach | Captures from 1996 to the present, added continuously; earliest row observed in the August 2026 verification pass dated January 17, 1999 | Continuous collection since 2012; every row carries a UTC start time |
| Finest granularity | One row per capture, timestamps resolving to the second; reducible to hourly or coarser timelines | One row per measurement per probe session, filterable by country, network, nettest and date |
| Documented fields | 7 documented fields, definitions verified | 12 documented entries including the grouped blocking-score object, definitions verified |
| Formats delivered | JSON arrays or line-oriented text | JSON plus dashboards and batch files |
| Rubric rating | 9 out of 10 (catalog average 7.81 across 1,744 datasets) | 9 out of 10 (catalog average 7.81 across 1,744 datasets) |
| Best for | What a URL said, when it changed, and proof of what it once was | Whether a URL or app is reachable now, from where, and who sits in the way |
Where they're equivalent
More than the memory-versus-interference split implies.
- Both are single-organization collection programs - each the product of one nonprofit observatory running its own apparatus at scale, not a government statistical release.
- Both are event logs: one row per observation, append-only by design, with the timestamp as the spine of any analysis.
- Both are global - one across the entire crawled address space, the other across nearly 200 countries of probing networks.
- Both dictionaries are fully verified, every definition checked rather than inferred - a standard met by 85.7 percent of the 1,744 cataloged datasets.
- Both score 9 out of 10, tied at the top mark awarded in the Interactive Media & Services slice.
- Neither carries what the other specializes in: no observer geography on the capture side, no content history on the measurement side. Any workflow wanting both buys both capabilities.
Fair questions
Do the two field dictionaries overlap?
Only in skeleton. Both reduce an observation to a target identifier, a timestamp and an outcome flag. Beyond that they diverge: CDX stays inside the capture - mimetype, status code, digest, compressed length - while OONI stays with the observer and the verdict - probe country, probe network, nettest, anomaly and confirmation flags, and blocking scores at five scopes.
Which record reaches further back?
Captures run from 1996 to the present - the earliest row observed in the August 2026 verification pass dates to January 17, 1999 - while OONI's measurements begin in 2012. For website history there is no contest; for current network conditions the longer archive is irrelevant.
Which one fits a country-specific censorship question?
OONI, by construction. Every row carries the probe's country code and autonomous system, with blocking graded separately at global, country, ISP and local scopes, across nearly 200 countries and every nettest from web connectivity to WhatsApp and Telegram checks. The capture index knows the address but not the vantage point.
Can I get both records from Datadory?
Yes - sample both, pick by fit, or take both in one feed. Each arrives normalized to its documented field dictionary with sample rows attached for validation, delivered on one schedule - daily, weekly, or hourly - your call - next to the rest of the interactive media & services catalog.