DNS Census 2013 - dataset of registered domains and DNS records
DNS Census

dnscensus2013 (294 files)
2nd-level-domains/ac.txt.xz 3.63kB
2nd-level-domains/ad.txt.xz 2.15kB
2nd-level-domains/ae.txt.xz 36.26kB
2nd-level-domains/aero.txt.xz 6.89kB
2nd-level-domains/af.txt.xz 2.92kB
2nd-level-domains/ag.txt.xz 6.69kB
2nd-level-domains/ai.txt.xz 1.21kB
2nd-level-domains/al.txt.xz 14.08kB
2nd-level-domains/am.txt.xz 30.48kB
2nd-level-domains/an.txt.xz 2.56kB
2nd-level-domains/ao.txt.xz 2.28kB
2nd-level-domains/aq.txt.xz 0.28kB
2nd-level-domains/ar.txt.xz 694.98kB
2nd-level-domains/arpa.txt.xz 0.32kB
2nd-level-domains/as.txt.xz 7.68kB
2nd-level-domains/asia.txt.xz 54.49kB
2nd-level-domains/at.txt.xz 1.31MB
2nd-level-domains/au.txt.xz 2.17MB
2nd-level-domains/aw.txt.xz 2.17kB
2nd-level-domains/ax.txt.xz 2.23kB
2nd-level-domains/az.txt.xz 15.58kB
2nd-level-domains/ba.txt.xz 28.30kB
2nd-level-domains/bb.txt.xz 3.09kB
2nd-level-domains/bd.txt.xz 9.15kB
2nd-level-domains/be.txt.xz 1.14MB
2nd-level-domains/bf.txt.xz 1.05kB
2nd-level-domains/bg.txt.xz 42.71kB
2nd-level-domains/bh.txt.xz 2.02kB
2nd-level-domains/bi.txt.xz 2.48kB
2nd-level-domains/biz.txt.xz 3.94MB
2nd-level-domains/bj.txt.xz 1.38kB
2nd-level-domains/bm.txt.xz 14.78kB
2nd-level-domains/bn.txt.xz 2.71kB
2nd-level-domains/bo.txt.xz 8.92kB
2nd-level-domains/br.txt.xz 2.04MB
2nd-level-domains/bs.txt.xz 0.98kB
2nd-level-domains/bt.txt.xz 2.47kB
2nd-level-domains/bw.txt.xz 4.52kB
2nd-level-domains/by.txt.xz 49.81kB
2nd-level-domains/bz.txt.xz 16.56kB
2nd-level-domains/ca.txt.xz 2.65MB
2nd-level-domains/cat.txt.xz 55.72kB
2nd-level-domains/cc.txt.xz 101.80kB
2nd-level-domains/cd.txt.xz 2.01kB
2nd-level-domains/cf.txt.xz 0.23kB
2nd-level-domains/cg.txt.xz 0.32kB
2nd-level-domains/ch.txt.xz 2.37MB
2nd-level-domains/ci.txt.xz 6.52kB
2nd-level-domains/ck.txt.xz 1.26kB
Too many files! Click here to view them all.
Type: Dataset

title= {DNS Census 2013 - dataset of registered domains and DNS records},
keywords= {},
journal= {},
author= {DNS Census},
year= {2013},
url= {http://dnscensus2013.neocities.org/},
license= {},
abstract= {==Introduction
Probably the last time any person or entity had a complete list of all hostnames on the Internet was in the mid-1980s, when the Domain Name System (DNS) replaced the old, centralized DoD Internet Host table. Some domain registries like Verisign have zone file access programs, which offer a way to download a complete zone file. But many country-code Top Level Domains (ccTLDs) do not offer such programs. Various companies have, usually through crawling large parts of the web, collected a huge number of DNS records, but none have a complete list of all domains.

And even those incomplete lists are in the hands of few companies like Google, Microsoft (Bing) and DomainTools LLC (whois.sc), presumably most major ISPs, and some research organizations like DNS-OARC, where they are treated as closely guarded company secrets. Other than the zone files provided by Verisign and some other registries, there are few datasets freely available for research or data mining.

The DNS Census 2013 is an attempt to provide a public dataset of registered domains and DNS records. It was inspired by the Internet Census 2012 which showed that releasing data anonymously via BitTorrent is a good thing to do. The dataset contains about 2.5 billion DNS records gathered in the years 2012-2013.

==The data
All data is compressed using xz/LZMA2.

==DNS records
DNS records are written into CSV files. There is one file for each DNS record type (A/AAAA/CNAME/DNAME/MX/NS/SOA/TXT). The records are sorted lexicographically by hostname and by time.

==Registered domains lists
If you only care about currently registered domains (e.g. example.com, nominet.org.uk, ?) but not subdomains (e.g. www.example.com, nom-ns1.nominet.org.uk., ?), the registered domains lists are for you. The lists contain only those hostnames for which a DNS record has been observed on or after January 1st, 2013. Each file contains the hostnames registered under one top-level domain. The lists are sorted lexicographically.},
tos= {},
superseded= {},
terms= {}

Send Feedback