Understanding the Deep & Dark Web

The misinterpretation of terms such as the Darknet, Dark Web, and Deep Web has always been prevalent. In the cybersecurity world, these terms have come to light with the exposure of techniques used by cybercriminals to communicate, collaborate, and participate in malicious activities. This report aims to help in understanding these terms and shed light on relevant topics within them.

THE SURFACE WEB

The World Wide Web was designed as a platform for billions of people to interact on the Internet. The most common area of the web that is publicly accessible is called the Surface Web. This includes all websites or pages that can be found using search engines such as Google, Yahoo, and Bing. Example: When a user is looking for information about a particular topic and proceeds to search online. The user would then type a few keywords about what they are looking for and will be presented with many links to various websites that relate to the keywords.

What allows these websites to be found by search engines are specially programmed scripts called web crawlers. These crawlers traverse many common directories of websites and use the links present on those pages to find more websites. Search engines have advanced web crawlers that gather all links from hundreds of billions of web pages daily and indexes each link for search optimization. The indexing process categorizes each website and their pages according to the content and text at each location. To summarize, the surface web includes searchable, indexed websites and web pages.

THE DEEP WEB

A step further into the World Wide Web, we find the Deep Web. Websites or web pages that prevent traditional search engines from indexing them are characterized as the Deep or hidden web. This prevention occurs when web crawlers cannot access and gather links from those sites; therefore, they would not be searchable. This might be an intentional decision by the website owner or an effect due to the nature of the website. There are several methods a website owner can use to keep their website hidden from web crawlers and prevent pages from being indexed. Some of these methods include making sure no surface websites contain links to their pages, limit access to pages through technical means (e.g., CAPTCHA’s) or requiring login to access pages.

An example of the Deep Web includes the large amount of content that is posted on Pastebin. Due to the extremely high frequency of information being published, most pages do not have links redirecting to the source and can only be found by using specific search tools. Another example is portals created for one particular group of people, who can only access the web pages by using their login credentials.

THE DARK WEB & DARKNETS

The Dark Web is used to describe an encrypted network built on top of the internet which can only be accessed using specialized software. The term Darknets can also be used to describe these networks. Websites found here are not indexed; therefore, the dark web is within the deep web. These networks are described as dark, due to the characteristics that assist users in hiding their identities and the popularity of supporting illegal activities. Darknets can either be in the form of privacy networks, such as Tor or Freenet, or in the as peer-to-peer networks, such as I2P or friend-to-friend networks. These forms of networks rely on routing traffic over the network through layers of encryption to support anonymity of the users.

The most common method of accessing the darknet is by accessing a private network which establishes restrictions to ensure a high level of security for its users. The most popular choice is The Onion Router (Tor) browser, which was created by the U.S. Navy to allow for internet anonymity worldwide. Tor attained its popularity due to its high level of security, ease of use and free for all software that can be downloaded and set up in minutes. Anonymity can be achieved by routing the network traffic through various Tor servers located globally. This means that if any of the packets in transmission are intercepted, the sender and receiver would appear to be random nodes on the Tor network, making it nearly impossible to identify what a user is doing on the network. Unfortunately, routing through many nodes means that the network connection speed is significantly slower than the traditional internet.

Many websites that exist can only be accessed by using Tor. Such sites have a special ‘.onion’ Top Level Domain (TLD) instead of having the commonly used ones like ‘.com’ or ‘.org’. Connections to these websites are only accepted by browsers or apps that run on the Tor network, such as the Orbot and Orfox mobile apps, or web browsers extensions.

There are several alternative methods to access a darknet, such as using the Invisible Internet Project (I2P), a peer-to-peer service which works similarly to Tor but can run within web browsers. What differentiates the methods of accessing darknets are their primary purpose of use, which is generally focused on browsing, messaging, or sharing anonymously; another critical aspect will be the level of security, which depends on their protocols of encryption and routing that is used.

A friend-to-friend (F2F) network is a type of peer-to-peer network that allows for specific rings of IP addresses to be connected and enables users to block any other IPs from knowing their existence on the network. In addition to this, F2F users can also encrypt their exchanges on the network to further enhance their security and anonymity.

INFORMATION FOUND ON THE DEEP WEB & DARKNET

Since there is a lot of personal information used and published online, it is a common occurrence that hackers get their hands on some of it. This information includes credit card numbers, images of identification cards, passports, medical records, and many types of account credentials such as email addresses, social media accounts and even subscription accounts such as Netflix.

The darknet has proven itself useful as a perfect location to host a marketplace of information hackers steal and want to sell while keeping their identities protected. Due to the nature of these sites, hackers can easily find individuals who would be interested in buying such kind of information. As soon as cryptocurrencies were available to the public, they were quickly adopted as a favored choice of payment for illegal transactions, due to their ability to hide the identities of their users. According to one cybersecurity company, it was estimated in 2016 that banking credentials would sell for 1% to 5% of the balance of the account, medical records for around $50 per record, and $12 for a U.S. credit card with its information. In the case where this information is not being sold, it is a regular occurrence for it be posted on the deep web, mainly paste sites such as Pastebin or Github.

It has become common for organizations to pay these hackers to protect themselves and/or their customers to secure the information being sold. According to a report by a prominent, the amount of breaches has gone down from 2016 to 2017, but the size of the data gathered from the breaches has increased. Furthermore, the average cost for a company to retrieve each stolen record was $141 in 2017.

It is important to note that there is no guarantee of the legitimacy of any information on marketplaces found on the darknet. However, the usual practice of these marketplaces is to allow the ability to rate a vendor, which aids them to gain a reputation based on the reliability of their sales and interactions.

CONCLUSION

To summarize, the part of the internet that is publicly searchable and accessible is called the surface web. The deep web, however, is an area of the internet where websites are not indexed and cannot be found through regular search engines or might require authorization for access. The Darknet, a.k.a. The “Dark Web,” is a part of the Deep Web that needs specific security mechanisms supporting anonymity to be in place to allow access to its sites and pages; for instance, privacy networks like Tor, or peer-to-peer networks like I2P. It is important to acknowledge the thriving markets for sensitive information on the Darknets, and also, the dispersion of this information on the Deep Web. To completely protect all aspects of an organization, the deep and darknet must be included in the threat scopes.

Understanding the Deep & Dark Web was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.