commoncrawl/warc2zip-examples
1.51 GB
AI & ML interests
Crawled data and metadata
Recent Activity
Organization Card
Common Crawl
Welcome to the Common Crawl Foundation's Hugging Face page!
We aim to provide metadata and experimental versions of our latest data products here.
Useful Links
- Common Crawl's official website
- Our existing statistics webpages (GitHub repo)
- AWS infrastructure status page
Datasets
Explore our datasets hosted on Hugging Face:
- Common Crawl Citations
- Common Crawl Citations, Annotated
- Common Crawl Statistics
- EOT 2024 Host-Level Logs (only available to EOT collaborators)
We look forward to supporting the research and development community with these resources.
models 0
None public yet
datasets 10
commoncrawl/llms.txt
Viewer • Updated • 598k • 38 • 1
commoncrawl/statistics
Viewer • Updated • 642k • 765 • 29
commoncrawl/CommonLID
Viewer • Updated • 373k • 221 • 55
commoncrawl/commonlid-results
Preview • Updated • 195 • 2
commoncrawl/web-graph-testing-v1
Updated • 57
commoncrawl/citations
Viewer • Updated • 9.18k • 273 • 4
commoncrawl/gneissweb-annotation-host-testing-v1
Viewer • Updated • 617M • 267 • 1
commoncrawl/gneissweb-annotation-url-testing-v1
Viewer • Updated • 11.5B • 4.48k
commoncrawl/host-index-testing-v2
Preview • Updated • 5.46k
commoncrawl/eot2024_hostlevel_logs
Viewer • Updated • 271k • 8 • 1