DCAT metadata schema

A metadata schema is the agreed list of fields a record must carry. In the DCAT family that means dataset-level description fields plus a distribution object per downloadable file, so that any conforming portal returns the same structure for any dataset.

The cataloged records show the schema in use. The Data.gov Catalog - Telecommunications & Carrier Datasets is summarized as 'an unauthenticated DCAT-shaped JSON search over 552,271 federal catalog records whose distributions carry direct downloadURL links'. The Communications Equipment Data.gov record lists its formats as 'DCAT-US metadata', 'JSON (Catalog API v4)', and 'CSV / XML / XLS / ZIP / KML (per-dataset resource links)' - the schema layer and the file layer named separately. The packaging search gives the query scale: 'roughly 20 results per page for the 'packaging' keyword', inside a 552,271-record catalog. Because the schema is shared, the same parser reads the telecommunications, packaging and carrier slices without per-portal code.

A shared schema is what makes one harvest script work across hundreds of publishers - and what makes schema drift expensive.

• Field presence is not guaranteed. The schema defines the fields; individual publishers fill them unevenly, so code must tolerate missing keywords or empty licence fields.

• Two schema layers, two formats. The cataloged record separates 'DCAT-US metadata' from 'JSON (Catalog API v4)' - the metadata schema describes datasets, the API schema describes search responses, and both need parsing.

• Distributions are where the schema pays off. Because each distribution object carries its own resource links ('CSV / XML / XLS / ZIP / KML'), format discovery is a field read rather than a page scrape.

• Page size shapes harvest time. At roughly 20 results per page for the packaging keyword, walking a large query means many requests - build pagination in from the start.

1. Validate records against the field list. A conforming record carries title, description, publisher, modified and issued dates, keywords and distributions - the cataloged search response enumerates exactly these.
2. Check that distributions expose direct downloadURLs. The cataloged summary highlights 'distributions carry direct downloadURL links'; without them the schema is descriptive only.
3. Distinguish the metadata schema from the API schema. The cataloged formats list names 'DCAT-US metadata' and 'JSON (Catalog API v4)' separately - map both before writing a harvester.

Harvest terms that depend on the schema:

• cursor-pagination — The paging mechanism behind the roughly 20-results-per-page search responses.
• metadata-catalog-harvesting — The process that fills the catalog with schema-conformant records.