API query etiquette rules

Etiquette rules govern how hard you may hit a shared public server: request pacing, query scoping and result-size conventions that keep the service responsive for everyone.

EIA's Weekly Natural Gas Storage Report states simply that "EIA asks that users not query its servers aggressively" - a request backed by an embargo holding releases until 10:30 a.m. eastern Thursday. ENTSOG's Transparency Platform enforces mechanically: "efficiency rules (sub-5-second responses, scoped queries, no aggressive concurrency) enforced by HTTP 500 rejection".

ENTSOG also documents the sanctioned way to move volume: "API returns paginated with limit=-1 for full result sets" - one flag instead of thousands of small calls, staying inside the 5-second guidance while pulling millions of per-interconnection-point flow, nomination and capacity records.

Politeness budgets determine whether a source can serve production pipelines or only dashboards.

• Soft rules still bind contracts. EIA's phrasing is a request, but it sits inside the licensing terms of a public-domain product - ignoring it risks access without any technical barrier warning you first.

• Hard failures are silent-looking. ENTSOG's HTTP 500 rejection reads like a bug unless your team knows it is the enforcement mechanism for oversized or concurrent queries.

• Pagination strategy is the whole game. limit=-1 exists so full result sets arrive in one compliant call; teams that loop naively generate exactly the concurrency the platform rejects.

• Embargo schedules shape freshness claims. Thursday 10:30 a.m. eastern storage releases mean 'real-time' integrations are really scheduled ones.

1. Classify the enforcement mode. Ask whether limits are advisory (EIA) or rejection-based (ENTSOG's HTTP 500); your retry logic differs fundamentally between the two.

1. Adopt the provider's bulk idiom. Use ENTSOG's limit=-1 pagination for full result sets rather than synthesizing your own paging scheme.

1. Serialize against concurrency caps. Queue jobs so overlapping workers cannot trip the no-aggressive-concurrency rule on the Transparency Platform.

1. Time runs around embargoes. Schedule EIA storage pulls after the Thursday 10:30 a.m. eastern release so jobs consume final data once, not stale data twice.

1. Log response codes as etiquette signals. Treat bursts of 500s as your own query-shape fault first; escalate to the provider only after narrowing scope and timing.

These entries cover the access models the etiquette protects: - free registered integration key — An access model where the API itself costs nothing but callers must register for a key and accept terms of service first. - registered json api — Free machine access that still requires signing up for a key first. - form eia 176 — EIA's annual company-level survey collecting supply, delivery, sales and customer data from US natural gas distribution, transmission and storage companies. - eia weekly storage report — EIA's Thursday 10:30 a.m.