Glossary
API rate limiting
API rate limiting is the traffic control a provider applies when callers exceed allowed request volumes - NHTSA runs automated rate control against bulk VIN lookups, and Best Buy enforces hard limits of 50,000 calls per day and 5 calls per second, returning HTTP 403 when either trips.
What is API rate limiting?
NHTSA's Vehicle Safety API notes that users attempting bulk VIN lookups are subject to an automated traffic rate control mechanism, and Census's API Terms of Service reserve the right to limit calls or block users exceeding limits. BEA documents approximate thresholds - roughly 100 requests per minute and 30 MB per minute, plus an error threshold that can block access temporarily.
Commercial APIs publish harder edges. Best Buy's Developer API sets hard limits of 50,000 calls per day and 5 calls per second, returning HTTP 403 on exceedance. The Data.gov Catalog API meters by key: 1,000 requests per hour on a personal key against 30 per hour and 50 per day on DEMO_KEY. RxNorm caps 20 requests per second per IP; openFDA pairs 240 req/min with 120,000 req/day keyed.
Why does API rate limiting matter when choosing a dataset?
Throttling decides your refresh window before your architecture does. A rebuild needing 200,000 Best Buy calls cannot finish in four days at 50,000 per day, and tripping NHTSA's automated control costs access at peak volume.
- Silent degradation is the failure mode. A 403 on exceedance looks like missing data downstream; contracts should require vendors to surface throttle events.
- Error rates are thresholds too. BEA can block access past its error threshold, so retry storms cause the outage they are trying to fix.
- Caching is part of compliance. Skipping RxNorm's recommended 12 to 24 hour cache multiplies load against a 20 req/s ceiling for zero benefit.
How do you evaluate API rate limiting in a data source?
Use these checks to test whether a source delivers what the concept promises:
- Get the numbers before sizing anything. Documented caps exist: Best Buy 50,000 calls/day and 5 calls/second, BEA roughly 100 requests/minute, RxNorm 20 requests/second/IP.
- Test the failure signal. Confirm your pipeline detects Best Buy's 403-on-exceedance and backs off rather than retrying into the block.
- Cache on the publisher's terms. RxNorm's recommended 12 to 24 hour cache is compliance, not optimization.
- Register for the higher tier. Data.gov's Catalog API moves from DEMO_KEY's 30 requests per hour to 1,000 with a personal key - a free 33x step.
- Ask how the vendor handles automated controls. NHTSA's mechanism and Census's right to limit or block operate without negotiation, so backoff design matters.
Where to see it in context: automotive-parts-equipment data, computer-electronics-retail data.
Related terms
Concepts that govern how limits are signaled and priced:
Frequently asked questions
What rate limits do federal data APIs publish?
BEA documents approximately 100 requests per minute and 30 MB per minute with an error threshold that can temporarily block access; Census reserves the right to limit calls or block users; NHTSA applies an automated traffic rate control to bulk VIN lookups.
Does exceeding an API limit block you permanently?
Blocks in these records are temporary - BEA says access may be temporarily blocked past its error threshold, and RIPE-style regimes trigger temporary blocks - but repeated abuse risks firewalling by other publishers.
How should a pipeline handle a Best Buy API 403?
Treat it as a throttle signal, not an error storm. Best Buy returns 403 when either limit trips - 50,000 calls per day or 5 calls per second - so back off and reschedule rather than retrying immediately.
Datasets containing this field
Datasets containing API rate limiting
6 datasets carry api rate limiting in the catalog. Open one, count the fields, judge for yourself.
BEA Personal Consumption Expenditures - Motor Vehicles and Parts
DMOTRC · DNMVRC · DNPVRC …+1 more
BLS Quarterly Census of Employment and Wages - Auto Dealers (NAICS 4411)
area_fips · own_code · industry_code …+14 more
EU Open Data Portal - Automotive Retail and Motor Trade Search
NHTSA Vehicle Safety API (Recalls, Complaints, Ratings)
NHTSACampaignNumber · NHTSAActionNumber · Manufacturer …+24 more
RockAuto Parts Catalog Data
Every listing shows the field dictionary, sample rows, and coverage before you commit. API, files, or your warehouse. Daily, weekly, or hourly.
Get sample rows