Let's Explore Peer-To-Peer Networks — BitTorrent — Deep Dive

Distributing a massive file from a single server is an infrastructure bottleneck waiting to happen. BitTorrent solves this by decentralizing the load, transforming every downloader into an active upload node. In this deep dive, we will explore the raw mechanics of Peer-to-Peer (P2P) networking, break down the BitTorrent architecture using real-world analogies, and write the core Go logic required to manage piece selection, tracker communication, and decentralized swarm routing.

Check out the implementation repository: github.com/mildlybrutal/bitbit
Research paper reference: Incentives Build Robustness in BitTorrent (Bram Cohen)

Why P2P?

Increased robustness and resource provision—such as bandwidth, storage space, and computing power—are achieved through peers. That is why it outscales the traditional client-server architecture.

File distribution: Eliminates single-origin bandwidth exhaustion.
P2P networking: Symmetric communication where nodes share the burden of distribution.

P2P Networking Fundamentals

A communication model in which each party has equivalent capabilities, and either party can initiate a communication session. This stands in contrast to the traditional client/server architecture with a single server handling multiple clients.

Formal Definition & The Servant Concept

A distributed network architecture may be called a Peer-to-Peer (P2P) network if the participants share a part of their own hardware resources (processing power, storage capacity, network link capacity).

These shared resources are necessary to provide the services and content offered by the network. They are accessible by other peers directly, without passing through intermediary entities.

The participants of such a network act as both:

Resource providers (services and content)
Resource requestors (services and content)

This dual nature is known in literature as the servant concept (server + client).

Network Topologies

1. Pure P2P

A network in which the peers themselves are the only entities present.

All nodes are interconnected.
If any individual peer is removed, there is no single point of failure.

2. Hybrid P2P

A central entity or hub nodes exist in the network to coordinate peer discovery or indexing.

If a central hub node goes down, parts of the network become partitioned.
More vulnerable to targeted attacks or single points of failure, but simplifies peer discovery.

The Intuition: Cricket Academy Analogy

Suppose I am in a cricket academy with 50 students, and there is only one bowling machine (the server).

Each student wants to face 60 balls. The machine can only fire one ball at a time.

The 50th student has to wait hours just to get their first ball.
The more students show up, the more frustrated everyone gets.

The Root Cause

The output originates from a single source with fixed speed. If it tries to serve multiple batting nets simultaneously, it has to oscillate back and forth, cutting practice time for all players in half.

What If We Change the Approach?

Instead of relying on one bowling machine:

Distribution: The coach gives a bag of 6 distinct balls to Player A, a different set to Player B, and a third set to Player C.
Exchange:
- Player A bowls their 6 balls to Player B.
- Player B bowls their 6 balls to Player C.
- Player C bowls their 6 balls to Player A.
Result: Everyone is active at the same time. No one is waiting in line for the central machine.

Mapping to BitTorrent

The File: The complete set of balls needed for full practice.
The Server (Coach / Seed): Only needs to distribute the initial pieces once.
The Peers (Players / Leechers): Provide upload capacity by sharing pieces with each other.

GO
type Peer struct {
    ID     string
    Pieces map[int]bool
}

func RequestPiece(from *Peer, to *Peer, piece int) {
    if from.Pieces[piece] {
        to.Pieces[piece] = true
        fmt.Printf("%s received piece %d from %s\n", to.ID, piece, from.ID)
    }
}

BitTorrent Architecture & The Office Analogy

Imagine you are in an office filled with people across dozens of rooms and you need to consult the IT specialist:

The Torrent File (The Blueprint): The map you carry in your hand. It does not contain the IT specialist, but contains cryptographic fingerprints (SHA-1 hashes) of their skills so you can verify anyone you meet.
The Tracker (The Receptionist): Does not solve your problem directly, but maintains an active registry of room numbers for everyone currently working on that task.
The Swarm (The Office Floor): The collective group of peers all sharing and downloading the same torrent.
The Seed (The Senior Specialist): Holds the complete manual (100% of the file).
The Leecher (The Junior Interns): Possess partial chapters. Intern A has Chapter 1; Intern B has Chapter 2. They swap chapters directly while downloading the remainder from the Seed.

GO
type Tracker struct {
    Swarm map[string][]*Peer
}

func (t *Tracker) Announce(torrentID string, p *Peer) {
    t.Swarm[torrentID] = append(t.Swarm[torrentID], p)
}

func (t *Tracker) GetPeers(torrentID string) []*Peer {
    return t.Swarm[torrentID]
}

Pieces and Verification

When creating a torrent file, the original payload is split into fixed-size pieces (typically 256 KB to 1 MB). Each piece is hashed using SHA-1.

Upon downloading a piece, the receiving peer validates its SHA-1 hash against the torrent metainfo. If valid, the peer broadcasts a HAVE message to the swarm, immediately advertising itself as an upload source for that piece.

Piece Selection Policies

A peer needs an intelligent strategy to decide which pieces to request first:

1. Strict Policy

Once a sub-piece (usually 16 KB) of a piece is requested, all remaining sub-pieces for that piece are requested before requesting blocks from any other piece. This ensures pieces are completed quickly for cryptographic verification.

2. Rarest First (Core Engine)

Peers inspect the bitfields of all connected nodes and prioritize downloading the piece held by the fewest peers in the swarm.

Benefits:

Rapidly replicates scarce pieces across the swarm before seeds disconnect.
Maximizes the value of pieces each leecher holds, allowing them to trade effectively.

GO
func RarestFirst(peers []Peer, totalPieces int, myPieces map[int]bool) int {
    count := make([]int, totalPieces)
    for _, p := range peers {
        for piece := range p.Pieces {
            count[piece]++
        }
    }

    rarest, minCount := -1, math.MaxInt
    for piece, c := range count {
        if myPieces[piece] || c == 0 {
            continue
        }
        if c < minCount {
            minCount = c
            rarest = piece
        }
    }
    return rarest
}

3. Random First Piece (Bootstrap)

When a peer first joins the swarm with zero pieces, it cannot participate in trading. Rarest pieces take longer to download because few peers offer them.

To bootstrap quickly, the peer selects its very first piece completely at random. Once that piece is verified, it switches immediately to Rarest First.

4. Endgame Mode

When only a handful of sub-pieces remain (typically at >95% completion), a single sluggish peer could stall the entire transfer.

In Endgame Mode, the client broadcasts requests for all remaining missing blocks to all connected peers simultaneously. As soon as a block arrives, cancel messages are immediately dispatched to the other peers.

Game Theory & The Choke Algorithm

Without enforcement, free-riders would download without uploading. BitTorrent uses a variant of the Tit-For-Tat strategy to achieve Pareto efficiency:

Choking: Temporarily refusing to upload data to a peer while continuing to download from them.
Unchoking: Allocating upload bandwidth to a peer.

Every 10 seconds, each peer ranks connected nodes by their recent download rate and unchokes the top 4 fastest providers.

GO
type PeerStats struct {
    Peer         *Peer
    DownloadRate float64
    UploadRate   float64
}

type Choker struct {
    Peers       []*PeerStats
    MaxUnchoked int
}

func (c *Choker) SelectTopPeers() []*PeerStats {
    sort.Slice(c.Peers, func(i, j int) bool {
        return c.Peers[i].DownloadRate > c.Peers[j].DownloadRate
    })
    if len(c.Peers) < c.MaxUnchoked {
        return c.Peers
    }
    return c.Peers[:c.MaxUnchoked]
}

Optimistic Unchoking

To discover new or faster peers (who otherwise would never get a chance to upload and prove their speed), one random peer is optimistically unchoked every 30 seconds regardless of its current rate.

Decentralized Trackers: Kademlia & DHT

Centralized trackers introduce single points of failure. Modern BitTorrent clients use a Distributed Hash Table (DHT) based on the Kademlia protocol:

Key: The 20-byte infohash of the torrent.
Value: IP addresses and port numbers of active peers.

The XOR Metric

Distance between two 160-bit node IDs $x$ and $y$ is computed as the bitwise exclusive-OR:

$$d(x, y) = x \oplus y$$

The XOR metric is unidirectional, symmetric ($d(x, y) = d(y, x)$), and satisfies the triangle inequality.

GO
func (d *DHT) FindClosest(target NodeID, k int) []*Peer {
    sort.Slice(d.Nodes, func(i, j int) bool {
        di := Distance(d.Nodes[i].NodeID, target)
        dj := Distance(d.Nodes[j].NodeID, target)
        return bytes.Compare(di[:], dj[:]) < 0
    })

    if len(d.Nodes) < k {
        return d.Nodes
    }
    return d.Nodes[:k]
}

Summary

BitTorrent transforms file distribution from a centralized scaling challenge into a cooperative distributed system. By combining cryptographic piece validation, game-theoretic tit-for-tat unchoking, rarest-first piece scheduling, and decentralized Kademlia DHT routing, it creates an exceptionally resilient and scalable peer-to-peer network.