Stack Exchange Data Dump

Category

AI Training Data

Provider

archive.org

Type

dataset

Level

unknown

Topic

AI Training Data

Track

n/a

Section

Open Data Directory

Format

n/a

Status

publishable

Commercial

unknown

Featured

no

Fast start

no

Sequence

n/a

Priority

n/a

Primary source

website_existing

Sources

website_existing

ID

8d05eec5f341aeef

arXiv Bulk Data Access

lib.ncsu.edu

Access: open. arXiv metadata/full text access options

Common Crawl

commoncrawl.org

Broad web crawl data; requires careful filtering + dedup.

OpenWebText2

github.com

Open web text

RedPajama-Data-v2

together.ai

Access: open. Pretraining mix

data.gov.uk

data.gov.uk

UK open data portal

Data.gov

data.gov

US government open data

Particle Data Group

pdg.lbl.gov

Access: open. Particle properties & tables

NIST Data Portal

data.nist.gov

Access: open. Broad science/engineering data

EU Open Data Portal

data.europa.eu

European Union open data

CAIDA Data Catalog

caida.org

Access: mixed. Catalog of CAIDA datasets

World Bank Open Data

data.worldbank.org

Access: open. Development indicators

NASA Open Data Portal

data.nasa.gov

Access: open. NASA datasets