dnmarchives (360 files)
1776.tar.xz 26.37MB
2015-sr2doug-claimedsr2leaks.tar.xz 73.38MB
2017-03-25-dnstats.sql.xz 110.85MB
abraxas-forums.tar.xz 46.86MB
abraxas.tar.xz 2.43GB
agape.tar.xz 2.81MB
agora-forums-20140421-whom-astorposts.tar.xz 11.72MB
agora-forums-2014093020141016-rasmusandersen.tar.xz 62.26MB
agora-forums.tar.xz 869.22MB
agora.tar.xz 5.87GB
alpaca.tar.xz 29.66MB
alphabay.tar.xz 938.82MB
amazondark.tar.xz 10.23MB
anarchia.tar.xz 132.43MB
andromeda-forums.tar.xz 2.94MB
andromeda.tar.xz 238.89MB
area51.tar.xz 75.61MB
armory.tar.xz 26.36MB
assassinationmarket.tar.xz 399.93kB
atlantis-20130921-christin.tar.xz 1.32MB
blackbankmarket-forums.tar.xz 80.41MB
blackbankmarket.tar.xz 815.64MB
blackgoblin.tar.xz 2.76MB
blackmarketreloaded-20131017-userlist.sql.xz 21.12MB
blackmarketreloaded-20131225-feedback-wousd.sql.xz 10.73MB
blackmarketreloaded-elpresidente.tar.xz 2.90MB
blackmarketreloaded-forums.tar.xz 57.76MB
blackservicesmarket.tar.xz 32.82MB
bloomsfield.tar.xz 2.30MB
bluesky.tar.xz 14.27MB
breakingbad.tar.xz 335.66kB
bungee54-forums.tar.xz 6.64MB
bungee54.tar.xz 980.92kB
buyitnow.tar.xz 860.10kB
cannabisroad.tar.xz 304.04kB
cannabisroad2-forums.tar.xz 8.16MB
cannabisroad2.tar.xz 734.96MB
cannabisroad3-forums.tar.xz 2.21MB
cannabisroad3.tar.xz 8.86MB
cantina.tar.xz 2.67MB
cloudnine.tar.xz 1.18GB
cryptomarket.tar.xz 1.22GB
cryuserv.tar.xz 158.90MB
darkbay-forums.tar.xz 794.78kB
darkbay.tar.xz 22.26MB
darklist.tar.xz 3.66MB
darknetheroes-forums.tar.xz 2.42MB
darknetheroes.tar.xz 6.38MB
darknetnation.tar.xz 9.02MB
dbay.tar.xz 946.70kB
deepzon.tar.xz 4.02MB
diabolus-cminfoleak-20150220-20150311.tar.xz 442.79kB
diabolus-forums.tar.xz 10.82MB
diabolus.tar.xz 1.53GB
dnmarchives_meta.sqlite 201.73kB
dnmarchives_meta.xml 1.97kB
dnstats-20150712.sql.xz 35.95MB
dnstats-20151109.sql.xz 49.29MB
documents.tar.xz 1.14GB
dogeroad-forums.tar.xz 301.61kB
dogeroad.tar.xz 33.41MB
dreammarket.tar.xz 124.98MB
drugslist.tar.xz 7.16MB
eastindiacompany.tar.xz 90.32MB
ecc.par2 79.32kB
ecc.vol000+01.par2 24.79MB
ecc.vol001+02.par2 49.57MB
ecc.vol003+04.par2 99.06MB
ecc.vol007+08.par2 197.96MB
ecc.vol015+16.par2 395.69MB
ecc.vol031+32.par2 791.06MB
ecc.vol063+64.par2 1.58GB
ecc.vol127+73.par2 1.80GB
ecc.vol200+00.par2 79.32kB
evolution-forums-2014093020141016-rasmusandersen.tar.xz 25.17MB
evolution-forums.tar.xz 1.18GB
evolution.tar.xz 3.63GB
freebay.tar.xz 11.52MB
freedommarketplace.tar.xz 139.30kB
freemarket.tar.xz 22.49MB
galaxy.tar.xz 44.54MB
gobotal-20140818-20141102.tar.xz 23.60MB
grams-20150714-20160417.tar.xz 67.79MB
grams.tar.xz 69.74MB
grandtrunk.tar.xz 249.81kB
greyroad-forums.tar.xz 48.28kB
greyroad.tar.xz 937.41kB
havanaabsolem-forums.tar.xz 1.13MB
havanaabsolem.tar.xz 15.54MB
haven.tar.xz 53.87MB
horizon.tar.xz 12.74MB
hydra-forums.tar.xz 971.30kB
hydra.tar.xz 1.09GB
ironclad.tar.xz 53.70MB
kingdom-forums.tar.xz 19.91MB
kiss-forums.tar.xz 2.49MB
kiss.tar.xz 164.55MB
middleearth.tar.xz 444.40MB
modafinil.tar.xz 79.70MB
mrniceguy-forums.tar.xz 223.42kB
Too many files! Click here to view them all.
Type: Dataset

title= {Darknet Market Archives 2013-2015 (dnmarchives) },
journal= {},
author= {Gwern Branwen and Nicolas Christin and David Décary-Hétu and              Rasmus Munksgaard Andersen and StExo and El Presidente and Anonymous              and Daryl Lau and Sohhlz, Delyan Kratunov and Vince Cakic and Van Buskirk              and Whom and Michael McKenna and Sigi Goode},
url= {https://www.gwern.net/DNM-archives},
type= {dataset},
year= {2015},
month= {July},
abstract= {Dark Net Markets (DNM) are online markets typically hosted as Tor hidden services providing escrow services between buyers & sellers transacting in Bitcoin or other cryptocoins, usually for drugs or other illegal/regulated goods; the most famous DNM was Silk Road 1, which pioneered the business model in 2011.

From 2013–2015, I scraped/mirrored on a weekly or daily basis all existing English-language DNMs as part of my research into their usage, lifetimes/​characteristics, & legal riskiness; these scrapes covered vendor pages, feedback, images, etc. In addition, I made or obtained copies of as many other datasets & documents related to the DNMs as I could.

This uniquely comprehensive collection is now publicly released as a 50GB (~1.6TB uncompressed) collection covering 89 DNMs & 37+ related forums, representing <4,438 mirrors, and is available for any research.

This page documents the download, contents, interpretation, and technical methods behind the scrapes.

There are ~89 markets, >37 forums and ~5 other sites, representing <4,438 mirrors of >43,596,420 files in ~49.4GB of 163 compressed files, unpacking to >1548GB; the largest single archive decompresses to <250GB. (It can be burned to 3 25GB BDs or 2 50GB BDs; if the former, it may be worth generating additional FEC.)

These archives are xz-compressed tarballs (optimized with the sort-key trick); typically each subfolder is a single date-stamped (YYYY-MM-DD) crawl using wget, with the default directory/file layout. The majority of the content is HTML, CSS, and images (typically photos of item listings); images are space-intensive & omitted from many crawls, but I feel that images are useful to allow browsing the markets as they were and may be highly valuable in their own right as research material, so I tried to collect images where applicable. (Child porn is not a concern as all DNMs & DNM forums ban that content.) Archives sourced from other people follow their own particular conventions. Mac & Windows users may be able to uncompress using their built-in OS archiver, 7zip, Stuffit, or WinRAR; the PAR2 error-checking can be done using par2, QuickPar, Par Buddy, MultiPar or others depending on one’s OS.

If you don’t want to uncompress all of a particular archive, as they can be large, you can try extracting specific files using archiver-specific options; for example, a SR2F command targeting a particular old forum thread:

tar --verbose --extract --xz --file='silkroad2-forums.tar.xz' --no-anchored --wildcards '*topic=49187*'

## Citation

Gwern Branwen, Nicolas Christin, David Décary-Hétu, Rasmus Munksgaard Andersen, StExo, El Presidente, Anonymous, Daryl Lau, Sohhlz, Delyan Kratunov, Vince Cakic, Van Buskirk, Whom, Michael McKenna, Sigi Goode. “Dark Net Market archives, 2011–2015”, 12 July 2015. Web.

    author = {Gwern Branwen and Nicolas Christin and David Décary-Hétu and
              Rasmus Munksgaard Andersen and StExo and El Presidente and Anonymous
              and Daryl Lau and Sohhlz, Delyan Kratunov and Vince Cakic and Van Buskirk
              and Whom and Michael McKenna and Sigi Goode},
title = {Dark Net Market archives, 2011-2015},
howpublished=  {\url{https://www.gwern.net/DNM-archives}},
url = {https://www.gwern.net/DNM-archives},
type = {dataset},
year = {2015},
month = {July},
timestamp = {2015-07-12},
note = {Accessed: DATE} }
keywords= {},
terms= {},
license= {https://creativecommons.org/about/cc0},
superseded= {}

10 day statistics (3 downloads)

Average Time 3 hrs, 05 mins, 24 secs
Average Speed 4.67MB/s
Best Time 50 mins, 19 secs
Best Speed 17.21MB/s
Worst Time 6 hrs, 05 mins, 57 secs
Worst Speed 2.37MB/s