federated open-data catalog

A federated open-data catalog is one search index aggregating metadata harvested from many independent publishing portals: federal agencies plus state, county, city, university, tribal and non-profit publishers in the cataloged Data.gov records. Data.gov's catalog holds 552,271 datasets, of which about 264 match the keyword 'retail', while a restaurant-filtered search spans dozens to hundreds of records covering inspection, licensing and food-safety topics.

Because the publishers are independent, licensing and quality vary per source. The cataloged restaurant search notes that the portal aggregates open government data without registration or fee, but each record's terms come from its own city, county, state or federal owner.

• Keyword counts are a ceiling, not a deliverable. About 264 'retail' matches out of 552,271 datasets is the honest size of that slice.

• Per-source licensing is the real work. Aggregation itself is free and registration-free, but each record carries its publisher's terms.

• Staleness varies by publisher. A city inspection feed and a federal survey in the same result set can differ by years in update cadence.

1. Size the slice with your own keyword. The cataloged numbers, 552,271 total, about 264 for 'retail' and dozens to hundreds for 'restaurant', came from running real queries.

1. Read the license per record. The catalog aggregates without registration or fee, but per-dataset terms govern redistribution.

1. Check publisher type per result. City, county, state, federal, university, tribal and non-profit publishers sit in one index with different standards.

1. Plan pagination. Large result sets use cursor pagination, so bulk pulls need a resumable loop.

These entries cover the mechanics of pulling and trusting federated results:

cursor pagination is how large result sets are walked, so bulk pulls need a resumable loop rather than one big request.

dataset snapshot staleness varies by publisher when city inspection feeds and federal surveys land in one result set.

DCAT metadata record is the unit being harvested, one per dataset, carrying each owner's license and cadence.

The retail and restaurant queries described here sit in the catalog's other-specialty-retail data, restaurants data pages.