DNS Census 2013 - dataset of registered domains and DNS records
DNS Census

folder dnscensus2013 (294 files)
file2nd-level-domains/ac.txt.xz 3.63kB
file2nd-level-domains/ad.txt.xz 2.15kB
file2nd-level-domains/ae.txt.xz 36.26kB
file2nd-level-domains/aero.txt.xz 6.89kB
file2nd-level-domains/af.txt.xz 2.92kB
file2nd-level-domains/ag.txt.xz 6.69kB
file2nd-level-domains/ai.txt.xz 1.21kB
file2nd-level-domains/al.txt.xz 14.08kB
file2nd-level-domains/am.txt.xz 30.48kB
file2nd-level-domains/an.txt.xz 2.56kB
file2nd-level-domains/ao.txt.xz 2.28kB
file2nd-level-domains/aq.txt.xz 0.28kB
file2nd-level-domains/ar.txt.xz 694.98kB
file2nd-level-domains/arpa.txt.xz 0.32kB
file2nd-level-domains/as.txt.xz 7.68kB
file2nd-level-domains/asia.txt.xz 54.49kB
file2nd-level-domains/at.txt.xz 1.31MB
file2nd-level-domains/au.txt.xz 2.17MB
file2nd-level-domains/aw.txt.xz 2.17kB
file2nd-level-domains/ax.txt.xz 2.23kB
file2nd-level-domains/az.txt.xz 15.58kB
file2nd-level-domains/ba.txt.xz 28.30kB
file2nd-level-domains/bb.txt.xz 3.09kB
file2nd-level-domains/bd.txt.xz 9.15kB
file2nd-level-domains/be.txt.xz 1.14MB
file2nd-level-domains/bf.txt.xz 1.05kB
file2nd-level-domains/bg.txt.xz 42.71kB
file2nd-level-domains/bh.txt.xz 2.02kB
file2nd-level-domains/bi.txt.xz 2.48kB
file2nd-level-domains/biz.txt.xz 3.94MB
file2nd-level-domains/bj.txt.xz 1.38kB
file2nd-level-domains/bm.txt.xz 14.78kB
file2nd-level-domains/bn.txt.xz 2.71kB
file2nd-level-domains/bo.txt.xz 8.92kB
file2nd-level-domains/br.txt.xz 2.04MB
file2nd-level-domains/bs.txt.xz 0.98kB
file2nd-level-domains/bt.txt.xz 2.47kB
file2nd-level-domains/bw.txt.xz 4.52kB
file2nd-level-domains/by.txt.xz 49.81kB
file2nd-level-domains/bz.txt.xz 16.56kB
file2nd-level-domains/ca.txt.xz 2.65MB
file2nd-level-domains/cat.txt.xz 55.72kB
file2nd-level-domains/cc.txt.xz 101.80kB
file2nd-level-domains/cd.txt.xz 2.01kB
file2nd-level-domains/cf.txt.xz 0.23kB
file2nd-level-domains/cg.txt.xz 0.32kB
file2nd-level-domains/ch.txt.xz 2.37MB
file2nd-level-domains/ci.txt.xz 6.52kB
file2nd-level-domains/ck.txt.xz 1.26kB
Too many files! Click here to view them all.
Type: Dataset

title= {DNS Census 2013 - dataset of registered domains and DNS records},
keywords= {},
journal= {},
author= {DNS Census},
year= {2013},
url= {http://dnscensus2013.neocities.org/},
license= {},
abstract= {==Introduction
Probably the last time any person or entity had a complete list of all hostnames on the Internet was in the mid-1980s, when the Domain Name System (DNS) replaced the old, centralized DoD Internet Host table. Some domain registries like Verisign have zone file access programs, which offer a way to download a complete zone file. But many country-code Top Level Domains (ccTLDs) do not offer such programs. Various companies have, usually through crawling large parts of the web, collected a huge number of DNS records, but none have a complete list of all domains.

And even those incomplete lists are in the hands of few companies like Google, Microsoft (Bing) and DomainTools LLC (whois.sc), presumably most major ISPs, and some research organizations like DNS-OARC, where they are treated as closely guarded company secrets. Other than the zone files provided by Verisign and some other registries, there are few datasets freely available for research or data mining.

The DNS Census 2013 is an attempt to provide a public dataset of registered domains and DNS records. It was inspired by the Internet Census 2012 which showed that releasing data anonymously via BitTorrent is a good thing to do. The dataset contains about 2.5 billion DNS records gathered in the years 2012-2013.

==The data
All data is compressed using xz/LZMA2.

==DNS records
DNS records are written into CSV files. There is one file for each DNS record type (A/AAAA/CNAME/DNAME/MX/NS/SOA/TXT). The records are sorted lexicographically by hostname and by time.

==Registered domains lists
If you only care about currently registered domains (e.g. example.com, nominet.org.uk, ?) but not subdomains (e.g. www.example.com, nom-ns1.nominet.org.uk., ?), the registered domains lists are for you. The lists contain only those hostnames for which a DNS record has been observed on or after January 1st, 2013. Each file contains the hostnames registered under one top-level domain. The lists are sorted lexicographically.},
tos= {},
superseded= {},
terms= {}

Hosted by users:

Send Feedback