Bit torrent

BitTorrent logo

BitTorrent (of bits (smallest data unit) and English torrent , torrent 'or' torrent 'of Latin torrens ) is a collaborative file sharing - Protocol , which is particularly suitable for the rapid distribution of large amounts of data. In contrast to other file-sharing techniques, BitTorrent does not rely on a comprehensive file-sharing network, but rather sets up a separate distribution network for each file.

Bit torrent technology

Technically, the protocol is assigned to OSI layer 7, i.e. the application layer, and is based on the TCP / IP reference model . The reference implementation ( Bit Torrent client ) by the inventor Bram Cohen was originally made in the Python programming language . A number of alternative programs are now available that implement the BitTorrent protocol.

BitTorrent reduces the load on the server, since peers exchange information with one another, even with incomplete files. As the colored bars show on some clients, the file is exchanged piece by piece in random order instead of being loaded completely.

Compared to conventional downloading of a file using HTTP or FTP , Bit Torrent technology uses the (otherwise unused) upload capacities of the downloaders, even if they have not yet completely downloaded the file. Files are not only distributed by a server, they are also passed on from user to user ( peer-to-peer or P2P). This puts less load on the server and the provider saves costs. Overall, the download load is not lower, it is simply shifted to the individual users. In the case of popular files, this technique prevents the network from collapsing as a result of the provider's capacity limits being exceeded.

function

In order to participate in the distribution of the data of a torrent, the client usually uses a torrent file ( filename extension .torrent or .tor ). This contains the IP address (or host name ) of the tracker as well as the file name, size and a list of checksums of segments of the data to be downloaded (one or more files). Torrent files can be created with many available bit torrent clients. It must the initial seeding - Peer ( engl. "Seeder" = seeder ) the link to the torrent file to be offered to establish and keep them available.

For finding other peers (who are interested in a certain file) there is now a number of other options, in which the contact is mediated by special (web) servers - the trackers . Normally, the tracker only has a unique ID for the torrent locally, to which the IP addresses of the peers that keep the file available are assigned. Quite often, peers connect to the tracker in order to be able to react quickly to changes in the available peers.

Torrent files are usually a few dozen kilobytes in size and are made available for download on the provider's website or via index sites (for example The Pirate Bay , Mininova or isoHunt ). Without a tracker, other methods must be used to find remote stations ( DHT , PEX, ...), or it can only be exchanged with known remote stations.

The client software receives from the tracker a list of remote stations that have or are interested in the data or parts of it. As soon as a peer has received a segment ("chunk") of the file and has verified the checksum , it reports this to the tracker and can now pass this file on to the other peers. The set of all peers interested in the same torrent is called a swarm . Peers who are in possession of the complete contents of the torrents are, and thus download anything from other clients, but only distribute data is called seeding (of English. To seed: sow). The peers who do not yet have the entire torrent content and who download additional segments are generally referred to as leechers (from English : leech: leeches). (Mainly in other contexts, peers are sometimes disparagingly referred to as “leechers” who only download without redistributing segments that have already been downloaded and thus violate the P2P principle.) Peer generally refers to another client connected to a client.

In contrast to other known file-sharing systems, not any files from the participants' holdings are exchanged. Rather, each swarm only distributes the files that the author of the torrent file has explicitly provided for downloading. The operator of the tracker also determines which downloads it should manage. The individual trackers are not connected to each other, so there is no common network, but a separate one for each individual torrent. This enables providers to distance themselves more easily from third-party, possibly illegal content.

Official protocol extensions

The development of the Bit torrent protocols has been handed over to the community. The BitTorrent Enhancement Proposals (BEP) collect new functions that were usually introduced and now standardized by the authors of various BitTorrent software.

Extension for peers to transfer metadata

The extension described in BEP 9 allows peers to transmit metadata in order to e.g. B. to resolve magnet links . A peer can use this to download a torrent without a .torrent file and join the corresponding swarm for this purpose.

DHT protocol

BEP 5 describes the use of a distributed hash table (DHT) based on the Kademlia algorithm for "trackerless" operation. This extension is currently still in the design stage, but its use is already widespread. The tracker function is taken over by the client software. This avoids some of the previous problems (for example the lack of reliability of the tracker), even if tracker-supported operation will still be possible. It also makes it easier to advertise, as trackers are the most expensive part of BitTorrent. Since version 4.2.0, which appeared in November 2005, the reference client has supported trackerless operation and relies on the Kademlia algorithm. Similar to the Kademlia network, the tracker can be stored and managed as a distributed hash table on the clients themselves.

Modern BitTorrent clients can therefore do without a tracker and work decentrally ("trackerless"). However, they still use different protocols so that communication between different clients on a DHT basis is not always possible. BitComet , Deluge , µTorrent , KTorrent , qBittorrent , rTorrent , Transmission and the reference client are currently compatible with one another . The developers are working harder to establish compatibility.

UDP tracker protocol

With the extension from BEP 15, a tracker can be queried via UDP , which can greatly reduce the volume of data traffic on the tracker. This extension is currently still in the drafting stage.

Super seeding

BEP 16 describes what is known as super seeding ( also called initial seeding for µTorrent ), in which, in the event that the respective peer is the only seed in a still data-free swarm, the dispatch is optimized so that a (distributed ) A copy of the data is created. In addition, in this mode the seed shows up in the swarm as an incomplete source and only releases its parts gradually. For the time being, he only delivers each part once and releases further parts once he has received feedback that an uploaded part has been distributed. He tries to find out which peers provide the fastest retransmission and uploads to them preferentially. The client BitComet has become known for abusing this procedure for its own benefit.

Superseeding was introduced in the BitTornado client . The extension is currently still in the drafting stage.

HTTP / FTP seeding

BEP 17 and 19 describe methods for using HTTP or FTP web servers as complete sources (seeds) in a BitTorrent swarm.

See article Webseed

Private torrents

In contrast to the normal case, the public torrents, access is restricted with private torrents. With a flag in the .torrent file (which here only contains a single tracker), the client is required to limit itself to the one tracker specified in the file when locating sources and to only connect to clients that this provides.

BEP 27 describes the procedure.

Announcing and finding peers using Zeroconf

With the extension described in BEP 26 (still in the draft stage), peers can find themselves in a local network using DNS Service Discovery ( Zeroconf ) . As a rule, significantly higher transmission rates are possible within a local network and the effort for the transmission is lower. With this extension, peers in the same local network who are interested in the same torrent can now use this to relieve the swarm and the common external connection and possibly more peers can be found. For this, a Zeroconf daemon must be running on the respective systems .

Finding BitTorrent caches

The extension described in BEP 22 (still in the drafting stage) enables the detection of nodes that are close to the network topology, such as the buffer storage of an Internet service provider .

This can be useful in many ways: By temporarily storing traffic from torrents that are popular on their network and feeding clients operated on their network from the cache, more expensive external data traffic can be saved. For customers, this results in an improvement in torrent reliability and download speed. The data from torrent transfers can thus take more straightforward routes through the Internet.

This extension was introduced with version 4.20.0 of the original client (mainline).

Protocol obfuscation

Many clients now offer encryption of the data stream with the uncertain algorithm RC4 ( stream cipher ) and the "Infohash" of the respective torrents as the key to disguise the protocol of the data stream and Internet service providers , the traffic shaping more difficult.

The original developer of the Bit torrent protocol, Bram Cohen, thinks this approach is bad and the corresponding BEP (BEP 8) has been postponed.

For customers of some Internet service providers, it is the only way to avoid a selective throttling of their data traffic; on the other hand, it also robs the providers of the option of operating BitTorrent caches and requires more computing power.

µTorrent and Azureus introduced the extension together, which suddenly made the majority of all peers compatible with encrypted connections.

Peer exchange

Peer Exchange (PEX) is an alternative method of finding peers. This means that peers can exchange their peer lists with one another and thus additional peers can be obtained from connected peers. Although the corresponding BEP 11 has not yet been adopted, the procedure is already widespread.

Internet cache

From version 4.20.0, BitTorrent supports the Internet Cache Protocol (ICP) in order to protect the data transfer rate of the provider. The additional protocol Cache Discovery Protocol was defined for this purpose. However, the technology has not yet been used. Since the Digital Millennium Copyright Act (DMCA) does not require caching providers to be liable for the data traffic passed through, there are no legal problems at least in the United States .

Anti-Leech-Tracker (ALT)

BitTorrent is based - like all file sharing networks - on the "give and take" principle. Quite a few participants deactivate their own send function ("upload") and thereby disregard the principle of all file sharing networks. In response to this exploitation, so-called “anti-leech trackers” (ALT) have been established that constantly monitor the behavior of the participants and exclude participants who only download or do so in an unfair relationship from the network as soon as predefined rules are violated . The so-called ratio (“file ratio”, “overall ratio” or, in the case of µTorrent, “share rate”) is used as the benchmark , which is calculated from the quotient of the amount of data sent and received.

In this system, however, participants with a lower data transmission rate are at a disadvantage. It can happen that such users are banned from the ALT if they cannot keep their ratio in time. Therefore, on some ALTs, the so-called "only upload" is used to counteract this problem. With "Only Upload" only the amount of uploaded data is taken into account. This problem is also increasingly countered with a so-called “seed bonus”, in which points are awarded for the period of the sending process (“upload”), which in turn can be exchanged for “upload credit”.

Another problem: The ALT is dependent on reports from the participating clients to calculate the ratio. These can easily be falsified, either on purpose through manipulation by cheaters or inadvertently through incorrectly configured client software. In the meantime, however, methods have also been developed here that recognize such potential attempts at fraud and thus lead to immediate exclusion.

Future developments

In the future, the further development of the protocol will no longer be carried out by Bram Cohen alone, but also by other developers. BitTorrent users have recently been increasingly (sometimes unknowingly) affected by the traffic shaping of their providers. If packets are classified as P2P packets, the upload is severely throttled in many cases. In response, the client implemented Vuze (formerly Azureus ) and uTorrent an end-to-end encryption (with RC4 to bypass -Stromchiffre) to traffic shaping. BitComet , rTorrent , Transmission and KTorrent also support the new protocol encryption.

history

The Bit torrent protocol was designed in April 2001 by the programmer Bram Cohen for the bootlegger online community etree . On July 2, 2001, he published his first implementation, which is now being looked after by BitTorrent, Inc. , a company he founded . After a short time, the file-sharing scene began to use the technology, among other things, for exchanging unlicensed copies of copyrighted data. Over time, BitTorrent developed into the largest (content, users, traffic) file sharing network. At times it produced a third of all global Internet traffic.

At the end of February 2007, the US company called BitTorrent converted its website of the same name into an online shop, where a lot of media content can only be downloaded for a fee.

In 2018, Bram Cohen left BitTorrent, Inc.

Areas of application, advantages and disadvantages

BitTorrent is particularly well suited for the rapid distribution of large files for which there is currently great demand . For files that are a little older or less popular, downloading via FTP or HTTP may be preferable because of the insufficient number of uploaders. This disadvantage can be overcome by using so-called webseeds . The process of data distribution with BitTorrent differs fundamentally from the way in which the well-known file-sharing file-sharing platforms work, in which each participant offers their own files in parallel. However, due to the increasing implementation of trackerless technologies and search functions in the clients, these can also be used like traditional file sharing networks.

Since the size of the data transfer rate that each downloader can get depends on the number of finished and unfinished downloads (peers) in the network and the upload rates of the users, depending on the number of providers (seeders), downloaders (leecher ) and the distribution rate come to more or less kilobytes per second. Since a user with a full download practically does not need any download rate, the maximum available data transmission rate for the entire network increases because this finished user only uploads (seed). Conversely, downloaders require data transfer rates, but contribute themselves with their upload rate. The following (only very theoretical) example is intended to illustrate the effect of different amounts of pure providers (seeders) and downloaders (leechers):

5 full downloads, each of the connections has an average upload rate of 30 kilobytes per second: 5 × 30 = 150 KByte / s
10 incomplete downloads, each of the connections has an average upload rate of 30 kilobytes per second: 10 × 30 = 300 KByte / s

The total data transfer rate in the network is 150 KByte / s plus 300 KByte / s, for a total of 450 KByte / s. Since 10 users download at the same time, this available data transfer rate is divided into 10 × an average of 45 KByte / s.

In reality, however, such a value is usually only achieved on large, public trackers with popular torrents, since the data transfer rate is also lost due to waiting times, assignments, different accessibility, etc. In addition, different connection speeds and priorities are responsible for more or less fast downloading in practice. In addition, many users have incorrectly configured their client or firewall.

Use in practice

Red Hat and Novell were among the first Linux distributors to use BitTorrent to distribute their distributions. All the major Linux distributions and FreeBSD are now available through BitTorrent. The game developer Blizzard distributed the World of Warcraft - Beta via BitTorrent and is now, as id Software , the larger patches for the final version also as torrents ready. Even OpenOffice.org and LibreOffice can be downloaded via BitTorrent. Facebook and Twitter distribute the software on their servers via BitTorrent.

There are also specific applications outside of the software area: The British BBC is currently undertaking pilot tests to make current programs and documentaries available online via BitTorrent immediately after they have been broadcast. The free music project Jamendo distributes albums via BitTorrent. The podcasts of the Chaosradio network have also been distributed via BitTorrent since 2010.

Others

According to a study by ipoque GmbH from February 2009, data transmission via the BitTorrent protocol makes up 37% of all data traffic in Germany and thus occupies the top position, by far behind HTTP with a share of 15%. The representativeness of this study is doubted, however, as only selected customers of ipoque GmbH were examined.

Furthermore, ipoque states that BitTorrent is the only file-sharing protocol that is used significantly for legal content.

In the meantime, Google has not displayed the word bittorrent via the autocomplete function.

Legal

The BitTorrent protocol itself is completely legal. Problems arise when infringing content is transmitted.

The legality of running torrent sites or tracker servers with torrents, which can be used to distribute copyrighted material without permission, is controversial, as the servers themselves do not offer files, but only connect the peers to one another.

The Motion Picture Association of America (MPAA) first took action against torrent sites in 2003, which led to the closure of the Torrentse and Sharelive sites in July 2003 . In the past, there were also more frequent seizures of servers that offered torrents. The first large-scale operation took place in December 2004 when the Finnish police raided Finreactor's servers . The torrents have not been offered since then and the process has been discontinued. Probably the largest torrent site to date, Suprnova , was also closed in December 2004 after the Slovenian authorities carried out a search of their provider. The site LokiTorrent , presumably the largest torrent search engine based on Suprnova at the time, closed its service shortly afterwards in February 2005. According to the MPAA, the administrator paid Edward Webber a fine and was forced to hand over log files. In May 2005, the US authorities closed the FBI and the Department of Homeland Security subordinate Immigration and Customs Enforcement , the side Elite Torrents .

Probably the most spectacular action so far took place in May 2006 when the world's largest bit torrent tracker ThePirateBay.org was confiscated by the Swedish police under pressure from the US Department of Justice , but was back online a few days later. In February 2009, four of the operators of The Pirate Bay were indicted by the Swedish judiciary and sentenced in April to one-year prison terms and millions in damages. This is important insofar as it was a matter of providing copyright-protected content, which, however, is not actually on the operator's servers. The verdict is not yet legally binding. At the beginning of December 2014, the Swedish police started the biggest raid against The Pirate Bay and confiscated servers and other technology from a data center. The site has been taken offline. However, on the morning of February 1, 2015, the site went online again.

In this context, there is now often talk of P2P “privacy” or P2P privacy. In principle, anyone can put a BitTorrent tracker server online, including a copyright protection association or a criminal investigation authority. In addition, the aforementioned organizations and institutions can of course also take part in the actual peering (data exchange) and thus obtain “proof” of the files transferred or exchanged in the P2P network. Since it is difficult for the user on the BitTorrent client side to understand which tracker servers or peers (IPs) are merely "listening in", so to speak, there are efforts and projects aimed at achieving better protection of privacy in P2P networks . One approach for this is to simply block certain IP numbers via IP lists and thus exclude them from the respective torrent. An example is PeerGuardian .

According to a study by the British University of Birmingham , your own IP address is likely to be logged within three hours when files are shared with popular files on the BitTorrent network. However , no surveillance was detected on torrents that were not among the top 100 most popular on The Pirate Bay . However, the IP address can be hidden by anonymization services such as proxy servers and virtual private networks (VPNs).

Further information on the legal situation can be found under file sharing .