text classification benchmark

A frozen dataset with fixed train/validation/test splits used to score text-classification models across named tasks - TweetEval ships eleven configurations over seven Twitter tasks, one labeled row per tweet, with license terms set per subset.

In this catalog it appears concretely: - TweetEval Benchmark — "Unified seven-task Twitter classification benchmark from Cardiff NLP with fixed train/val/test splits covering irony, hate, offensive, stance, emoji, emotion and sentiment".

A label on a listing is not a deliverable. Teams that license on the strength of a product-name match routinely find the shipped files cover a narrower slice than they assumed, and backfilling history afterwards costs more than the license ever did.

The failure mode is concrete: the label appears in a listing, the delivered files tell a different story, and the gap surfaces mid-project when fixing it is most expensive.

You rarely have to take a vendor's word for it. 82.4% of the 1,744 datasets Datadory catalogs are free to access, and TweetEval Benchmark lets you inspect the real artifact before any budget is committed.

Treat every claim of this attribute as testable:

1. Open TweetEval Benchmark and confirm its record — "Unified seven-task Twitter classification benchmark from Cardiff NLP with fixed train/val/test splits covering irony, hate, offensive, stance, emoji, emotion and sentiment" — against the files you actually receive.
2. Pin down update cadence in writing. Across this catalog, 22.6% of 1,744 datasets refresh daily and 64 still arrive only through a manual request form, so ask exactly how fresh each release is.
3. Check whether definitions are verified at all. Field definitions are verified for 1495 of 1,744 datasets (85.7%), and any source you license should meet that bar.
4. Price the delivery route before the license. In this catalog bulk download is the most common access method (725 datasets) ahead of official APIs (574), and 379 sources still require scraping — a maintenance cost that lands on you, not the vendor.

See the term applied to real records: interactive-media-services data.

Adjacent concepts worth reading next: - social network graph dataset - upstream platform terms