How the Internet Actually Works: A Developer's Guide to DNS, TCP/IP, HTTPS, and the Modern Web Stack

Introduction

Every engineer uses the internet dozens of times a day and most could not fully explain what happens between typing a URL and seeing a page render. That's not a knock on anyone - the internet is intentionally layered so that each piece can be understood, built, and debugged in isolation, without needing to understand everything beneath it. A frontend developer doesn't need to know how TCP handles packet loss to build a React component, and a network engineer configuring BGP routes doesn't need to know how the browser's rendering engine parses CSS.

But that same layering, which makes the internet buildable, also makes it opaque when something breaks. A slow page load could be a DNS misconfiguration, a TLS handshake issue, a backend timeout, or an oversized JavaScript bundle - and diagnosing it well requires at least a working model of every layer involved, even if you only operate directly in one of them.

This article builds that model end to end: the physical and logical layers that move bits from one machine to another, the protocols that structure communication on top of them, the security layer that makes the whole thing usable for anything sensitive, and the application-level concepts - HTML, CSS, JavaScript, frontend and backend architecture, CDNs and cloud infrastructure - that sit on top of all of it. The goal isn't encyclopedic coverage of every protocol detail; it's a mental model precise enough to reason about where a problem lives when something in this enormous stack doesn't behave the way you expect.

The Problem: Why the Internet Needs Layers at All

The internet connects an almost incomprehensible diversity of hardware - fiber optic cables, WiFi radios, satellite links, cellular towers - run by different companies, using different physical technologies, spanning every continent. No single protocol could reasonably span all of that diversity while also handling application-level concerns like "render this webpage" or "deliver this email." The solution, formalized conceptually in models like the OSI model and more concretely in the architecture the actual internet uses (often described as the TCP/IP model or the Internet protocol suite, as documented across the IETF's foundational RFCs), is to split the problem into layers, each responsible for a narrow concern and each built on the guarantees of the layer below it.

This layering means a change at one layer generally doesn't require changes at the others. WiFi replaced Ethernet cables for most home and office connectivity without requiring any change to how IP addressing or HTTP works - because WiFi operates at the link layer, and everything above it simply doesn't need to know or care whether the bits below arrived over radio waves or copper wire. Similarly, HTTP/2 and HTTP/3 introduced meaningful changes to how web traffic is structured without requiring changes to TCP (HTTP/2) or introducing a new transport layer entirely built on UDP (HTTP/3, via QUIC) - and application code written against HTTP semantics largely didn't need to change either way.

Understanding this layered structure isn't academic trivia for working engineers - it's a genuinely practical diagnostic tool. When a request fails, asking "which layer is this actually failing at" narrows the investigation dramatically: a DNS resolution failure, a TCP connection refusal, a TLS certificate error, and an HTTP 500 response are four completely different failure modes that happen to all present to a user as "the website doesn't load," and knowing the layered model is what lets an engineer tell them apart quickly instead of guessing.

The Layers: Link, Internet, Transport, and Application

The Link Layer: Getting Bits Onto the Wire

The link layer is concerned with transmitting raw bits between devices that are directly connected on the same physical or logical network segment - the same LAN (Local Area Network), the same WiFi cell, the same cable run. Ethernet and WiFi (standardized as the IEEE 802.11 family) are the dominant link-layer technologies for wired and wireless local networks respectively. Devices on the same link layer address each other using MAC addresses, hardware identifiers burned into network interfaces, and the Address Resolution Protocol (ARP) maps those MAC addresses to IP addresses within a local network so that link-layer delivery and internet-layer addressing can work together.

This is the layer where "the WiFi is down" or "the cable is unplugged" problems live - genuinely physical or near-physical failures where no amount of correctly configured software above this layer will help, because the bits simply aren't getting from one machine to another. It's also the layer where local network segmentation happens: a home router creating a private local network, or a corporate network isolating devices into VLANs, are link-layer and near-link-layer concerns that shape what a device can even attempt to reach before any higher-layer protocol gets involved.

The Internet Layer: Routing Across Networks

The internet layer, built around the Internet Protocol (IP), solves the problem the link layer can't: getting data from a device on one network to a device on an entirely different network, potentially on the other side of the planet, hopping across many intermediate networks along the way. Every device reachable on the internet has an IP address - either IPv4 (the older, more limited addressing scheme, long since exhausted in terms of available unique addresses) or IPv6 (the newer scheme with a vastly larger address space, designed specifically to outlast IPv4's exhaustion). Routers, the specialized devices that sit at the boundary between networks, use routing protocols to decide which direction to forward each packet, hop by hop, until it reaches its destination network.

Critically, IP itself makes no promises about reliability - a packet might be dropped, duplicated, or arrive out of order, and IP does nothing to detect or correct any of that. This isn't an oversight; it's a deliberate design simplification (part of what's often called the end-to-end principle, articulated in the networking literature going back to the 1980s) that keeps the core network simple and pushes reliability concerns up to the endpoints, where the transport layer picks up the job.

The Transport Layer: Making Communication Reliable (or Not)

The transport layer sits on top of IP and provides the abstraction that applications actually build on. TCP (Transmission Control Protocol) provides a reliable, ordered, connection-oriented stream - it detects dropped packets and retransmits them, reorders packets that arrived out of sequence, and establishes a formal connection (the well-known three-way handshake: SYN, SYN-ACK, ACK) before any application data flows. This reliability comes at the cost of some overhead and latency, which is why TCP isn't universal: UDP (User Datagram Protocol) provides a much simpler, connectionless, unreliable alternative that trades away TCP's guarantees for lower latency and less overhead, making it the right choice for use cases like DNS queries, video streaming, and real-time gaming, where an occasional lost packet is preferable to the delay reliability would introduce.

Most of the protocols developers interact with daily - HTTP, HTTPS, SMTP, WebSocket connections - run on top of TCP specifically because they need its reliability guarantees. The more recent HTTP/3 is a notable exception, running over QUIC, a transport protocol built on UDP that reimplements reliability and ordering at a higher layer while avoiding some of TCP's specific performance limitations, particularly around how it handles packet loss across multiple simultaneous streams.

The Application Layer: Where Meaning Lives

The application layer is where protocols encode actual meaning - a request for a webpage, an email being sent, a domain name being resolved. HTTP (HyperText Transfer Protocol) structures requests and responses for the web. SMTP (Simple Mail Transfer Protocol) governs how mail servers exchange messages. DNS (Domain Name System), covered in depth below, resolves human-readable names into IP addresses. Every one of these protocols assumes the layers below it have already handled getting bytes reliably (or acceptably unreliably, for UDP-based protocols) from one endpoint to another, and concerns itself purely with what those bytes mean.

DNS: Translating Names Into Addresses

DNS deserves its own detailed treatment because it's the piece of infrastructure most invisible when it works and most confusing when it doesn't. Its job is simple to state: translate a human-readable domain name like example.com into the IP address a computer actually needs to open a connection. Its implementation is a distributed, hierarchical system spanning millions of servers worldwide, specifically so that no single server has to know every domain name in existence.

Resolution happens in stages. A client asks a recursive resolver (often run by an ISP, or a public service like Google's 8.8.8.8 or Cloudflare's 1.1.1.1) to resolve a name. If that resolver doesn't already have the answer cached, it queries a root server, which doesn't know the final answer but knows which server is responsible for the domain's top-level domain (.com, .org, and so on). That TLD server, in turn, points to the domain's authoritative name server, which finally returns the actual IP address (or other DNS record types - an A record for IPv4, AAAA for IPv6, CNAME for an alias to another name, MX for mail routing, and others, all standardized across various IETF RFCs). This entire chain typically completes in milliseconds, aggressively cached at every level so that repeated lookups for popular domains rarely need to traverse the full hierarchy again.

# Python: performing a basic DNS lookup and inspecting record types directly
import socket
import dns.resolver  # dnspython - provides more granular record access than socket

def resolve_domain(domain: str):
    # Basic A-record resolution, sufficient for most application code.
    ip_address = socket.gethostbyname(domain)
    print(f"{domain} resolves to {ip_address}")

    # Inspecting other record types requires a dedicated resolver library,
    # since the standard library only exposes basic address resolution.
    for record_type in ("A", "MX", "TXT"):
        try:
            answers = dns.resolver.resolve(domain, record_type)
            for rdata in answers:
                print(f"{record_type} record: {rdata.to_text()}")
        except dns.resolver.NoAnswer:
            print(f"No {record_type} record found for {domain}")

resolve_domain("example.com")

This code illustrates a subtlety worth internalizing: the DNS system isn't just "domain name to IP address." It's a general-purpose distributed key-value lookup for a whole family of record types, and understanding that MX records route mail while A records route web traffic, entirely independently, explains why a misconfigured DNS entry can break email deliverability without affecting website availability at all, or vice versa.

TLS, HTTPS, and Securing the Connection

TLS (Transport Layer Security, the modern successor to the deprecated SSL protocol, though "SSL" remains common shorthand even when TLS is what's actually in use) is what turns plain HTTP into HTTPS - the layer that provides encryption, authentication, and integrity for a connection that would otherwise be sent in plaintext, fully readable by anyone positioned to intercept it along the network path.

The TLS handshake establishes a secure channel before any application data is exchanged. In broad strokes: the client and server agree on a cipher suite, the server presents a certificate (issued by a Certificate Authority the client's system already trusts, per the public key infrastructure that underlies the entire system) proving its identity, and both sides establish a shared symmetric encryption key through a key exchange process - modern TLS versions (TLS 1.3, standardized in RFC 8446, being the current recommended version) use forward-secure key exchange methods so that even if a key is later compromised, previously recorded traffic can't be decrypted retroactively. This is precisely the mechanism that prevents a man-in-the-middle (MITM) attack from succeeding under normal circumstances: an attacker intercepting the traffic sees only encrypted bytes, and cannot present a fraudulent certificate for the target domain without it being rejected by the client, provided the client's certificate validation is intact and the relevant Certificate Authority hasn't been compromised.

// JavaScript (Node.js): inspecting the TLS certificate of a live connection -
// a practical way to verify what a client actually sees during a handshake.
const tls = require("tls");

function inspectCertificate(hostname, port = 443) {
  const socket = tls.connect({ host: hostname, port, servername: hostname }, () => {
    const cert = socket.getPeerCertificate();
    console.log(`Subject: ${cert.subject.CN}`);
    console.log(`Issuer: ${cert.issuer.O}`);
    console.log(`Valid until: ${cert.valid_to}`);
    console.log(`Protocol negotiated: ${socket.getProtocol()}`);
    socket.end();
  });

  socket.on("error", (err) => console.error(`TLS error: ${err.message}`));
}

inspectCertificate("example.com");

MITM attacks are worth understanding precisely because they explain why certificate validation isn't an optional nicety. Without it, an attacker controlling a network segment - a malicious WiFi access point, a compromised router - could intercept a connection and present their own certificate, and if the client accepted any certificate without checking it against a trusted authority, the attacker could decrypt and modify traffic transparently. This is exactly why browsers display prominent warnings for invalid or self-signed certificates, and why disabling certificate validation in application code (a shortcut sometimes taken during development and, alarmingly, sometimes left in production) reintroduces the exact vulnerability the entire TLS system exists to close.

HTTP, WebSocket, and How Applications Actually Talk

HTTP is a request-response protocol: a client sends a request with a method (GET, POST, PUT, DELETE, and others), a target, headers, and optionally a body, and the server sends back a status code and a response body. This model works well for the vast majority of web interactions, where a client asks for something and the server answers, but it's fundamentally a poor fit for scenarios requiring the server to push data to the client without the client asking first - a chat application, a live dashboard, a multiplayer game state.

WebSocket, standardized in RFC 6455, solves this by upgrading an initial HTTP connection into a persistent, full-duplex channel where either side can send messages at any time without the request-response ceremony HTTP requires for every exchange. The upgrade happens through a deliberately designed handshake that starts as a normal HTTP request with an Upgrade: websocket header, so that the connection can pass through existing HTTP infrastructure (proxies, load balancers) before switching protocols.

// TypeScript: a WebSocket client with reconnect logic, reflecting how
// real applications handle the fact that persistent connections do drop.
class ResilientSocket {
  private ws: WebSocket | null = null;
  private reconnectDelay = 1000;
  private readonly maxDelay = 30000;

  constructor(private url: string, private onMessage: (data: string) => void) {
    this.connect();
  }

  private connect(): void {
    this.ws = new WebSocket(this.url);

    this.ws.onopen = () => {
      this.reconnectDelay = 1000; // Reset backoff after a successful connection.
    };

    this.ws.onmessage = (event) => this.onMessage(event.data);

    this.ws.onclose = () => {
      setTimeout(() => this.connect(), this.reconnectDelay);
      this.reconnectDelay = Math.min(this.reconnectDelay * 2, this.maxDelay);
    };

    this.ws.onerror = () => this.ws?.close();
  }

  send(data: string): void {
    if (this.ws?.readyState === WebSocket.OPEN) {
      this.ws.send(data);
    }
  }
}

This example matters because it reflects a genuine production concern: a persistent connection introduces a new failure mode HTTP's request-response model doesn't have - the connection can silently drop, and a naive client that doesn't detect and recover from that will simply stop receiving updates without any obvious error. The exponential backoff on reconnection mirrors the same pattern used for HTTP retries, for the same reason: reconnecting immediately and repeatedly, especially across many clients simultaneously, risks overwhelming a server that's already struggling.

From Backend to Browser: The Frontend, Backend, and Cloud Layers

Everything covered so far explains how bytes get from one machine to another. What those bytes actually contain, once they arrive at a browser, is a separate concern layered on top: HTML structures content, CSS governs its presentation, and JavaScript provides behavior and interactivity - together forming the core of what's generally called the frontend. The backend, running on servers (increasingly, cloud infrastructure operated by providers like AWS, Google Cloud, or Azure rather than physical machines a company owns directly) handles business logic, data persistence, and the API endpoints the frontend calls, typically over HTTP.

The relationship between these two halves has evolved considerably. Early web applications rendered HTML entirely on the server and sent a mostly-complete page to the browser for each request. Modern single-page applications commonly render most of the UI in the browser using JavaScript frameworks, fetching data from backend APIs asynchronously after the initial page load - a shift that moved substantial rendering work from server to client, changing the performance profile of applications considerably (trading server CPU time for client-side JavaScript execution time, and trading full-page reloads for a more granular, ongoing exchange of data over the network). Frameworks and rendering strategies have continued to evolve around this trade-off, with approaches like server-side rendering and static site generation representing attempts to recapture some of the performance and SEO benefits of server-rendered HTML while retaining the interactivity benefits of client-side JavaScript.

A CDN (Content Delivery Network) sits geographically and logically between the origin backend and the end user, caching static assets (and sometimes dynamic content) at edge locations distributed around the world, so that a user in Singapore requesting a website hosted in Virginia gets that content served from a nearby edge server rather than traveling the full physical distance to the origin on every request. This dramatically reduces latency for cacheable content and reduces load on the origin server, which is why CDNs (Cloudflare, Amazon CloudFront, Akamai, Fastly, among others) are near-universal infrastructure for any public-facing website with meaningful traffic. CDNs also frequently terminate TLS at the edge, meaning the CDN itself handles the encryption handshake with the end user, sometimes maintaining a separate encrypted connection back to the origin - an architectural detail worth understanding because it changes where a certificate needs to be valid and where a MITM-style interception would actually need to occur to succeed.

Trade-offs and Pitfalls Across the Stack

Several recurring failure patterns show up across this entire stack, and recognizing them by category speeds up diagnosis considerably.

DNS caching masking or delaying real changes. Because DNS is cached aggressively at multiple layers - the operating system, the browser, intermediate resolvers - a DNS record change (updating where a domain points) doesn't propagate instantly, and the delay depends on the TTL (time-to-live) set on the record and how faithfully every intermediate cache respects it. Engineers unfamiliar with this often assume a DNS update failed when it actually succeeded but hasn't yet propagated everywhere, or conversely assume an old configuration is still live when it's actually just a stale cache entry somewhere in the chain.

Mixed content and certificate mismatches breaking HTTPS silently. A page served over HTTPS that loads even a single resource over plain HTTP triggers browser mixed-content restrictions, sometimes blocking the resource outright and sometimes just warning, depending on the resource type - a frequent source of "why isn't this image loading" confusion on sites migrated from HTTP to HTTPS incompletely. Certificate mismatches between a CDN and an origin server are a related, less obvious failure: if a CDN is configured to validate the origin's certificate strictly and the origin's certificate doesn't match the expected hostname, requests fail at the CDN-to-origin hop, in a way that's invisible from the end user's perspective since their connection to the CDN itself remains healthy.

Treating WebSocket connections as inherently reliable. As shown in the code example earlier, persistent connections drop - due to network changes, server restarts, load balancer idle timeouts, or countless other causes - and an application that doesn't explicitly handle reconnection and message redelivery will silently lose real-time functionality without any error surfaced to the user, since from the browser's perspective, everything "worked," it just isn't receiving anything anymore.

CDN caching serving stale or incorrect content. A CDN caching a response longer than intended, or caching a response that should never have been cached at all (a personalized or authenticated response, for instance), can serve one user's content to another, or serve outdated content long after an origin update - failure modes that are particularly dangerous specifically because CDNs are trusted infrastructure, and a caching misconfiguration doesn't look like an error, it looks like a plausible, if wrong, response.

Underestimating the layers between a user and a server. A request that seems to originate from "a user's browser" typically passes through their local WiFi router, an ISP, potentially a corporate proxy, a CDN edge node, a load balancer, and only then the actual application server - and a MITM attack, a captive portal intercepting traffic, a corporate firewall injecting content, or simply a misconfigured proxy can each interfere with that path in ways that are difficult to distinguish from an application bug without understanding that the path has that many distinct hops in the first place.

Best Practices for Engineers Working Across the Stack

A handful of habits make the difference between confidently diagnosing issues across this stack and guessing.

Learn to use the diagnostic tools that map directly onto each layer: dig or nslookup for DNS resolution, ping and traceroute for basic connectivity and routing, openssl s_client or browser developer tools for inspecting a TLS handshake and certificate chain, and browser network tabs for inspecting the actual HTTP request and response cycle end to end. Each tool corresponds to a specific layer, and knowing which one to reach for first, based on the symptom, dramatically narrows the investigation.

Set explicit, deliberate TTLs on DNS records and cache headers rather than accepting defaults, and understand the propagation delay those settings imply before making a change under time pressure - during an incident is the worst possible moment to discover that a DNS change everyone is waiting on has a 24-hour TTL from a prior configuration.

Validate TLS configuration continuously rather than only at initial setup, since certificates expire, cipher suite recommendations evolve, and misconfigurations (like an origin server with a certificate that no longer matches a CDN's expectations) tend to surface as outages rather than warnings. Automated certificate renewal (via tools like Let's Encrypt's ACME protocol, widely supported across hosting providers) removes an entire category of "the certificate expired and nobody noticed" incidents.

Design for the failure modes each layer actually has, rather than assuming lower layers are reliable because they usually are. This means retry logic with backoff for HTTP calls, explicit reconnection handling for WebSocket connections, and cache invalidation strategies that are tested, not just configured and forgotten - because every one of these layers will fail in the specific ways described above, given enough time and traffic.

Key Takeaways

Diagnose by layer, not by symptom - "the site is down" could be a DNS, TCP, TLS, or application-layer failure, and identifying which one first dramatically narrows the investigation.
Never disable TLS certificate validation, even temporarily in development, since it's the exact mechanism that prevents man-in-the-middle attacks, and shortcuts taken in development have a way of surviving into production.
Treat DNS and cache TTLs as an active configuration decision, not a default to leave alone, since they directly determine how quickly a change actually takes effect for real users.
Build explicit reconnection and retry logic for anything long-lived or over the network - WebSocket connections, HTTP calls to flaky dependencies - since the underlying layers will drop connections as a matter of routine, not exception.
Understand where your CDN terminates TLS and what it caches, since both details determine where a security or staleness problem would actually need to be diagnosed.

Analogies and Mental Models

A useful way to hold the whole stack together: think of sending a letter through a national postal system. The link layer is the truck physically carrying mail between two nearby post offices. The internet layer is the postal routing system that decides which sequence of post offices and trucks gets a letter from any address to any other address in the country, without the sender needing to know the route. The transport layer is the choice between a tracked, signed-for delivery that guarantees arrival and correct order (TCP) versus a cheaper, faster, best-effort delivery with no such guarantee (UDP). The application layer is the actual content and format of the letter - an invoice, a greeting card, a legal notice - which means something specific to sender and recipient but nothing at all to the trucks and post offices moving it along the way.

DNS, in this analogy, is the phone book that turns "my friend Alex" into an actual street address the postal system can route to - and TLS is the equivalent of a tamper-evident, sealed envelope that also proves, via a signature the recipient can verify against a trusted registry, that the letter really did come from who it claims to be from, and wasn't opened and resealed by anyone in between.

The 80/20 Insight

Of everything in this article, two concepts explain a disproportionate share of real-world "why isn't this working" moments: DNS resolution and caching, and the TLS handshake and certificate chain. A remarkable fraction of production incidents that look mysterious at first - intermittent failures, "it works for me but not for them," sudden outages after a routine change - trace back to one of these two areas: a DNS record pointing somewhere unexpected, a cache serving stale data past its intended lifetime, or a certificate that's expired, mismatched, or failing validation for a subtle reason. Engineers who build a genuinely solid mental model of just these two areas, even without deep expertise in routing protocols or transport-layer internals, will diagnose the large majority of real infrastructure issues they encounter faster than engineers who know more protocol trivia but haven't internalized how DNS and TLS actually behave under real-world conditions.

Conclusion

The internet's layered design is what makes it possible for engineers to specialize - to build a frontend without understanding BGP routing, or configure a CDN without understanding how a browser's JavaScript engine works - but that same specialization means most engineers carry an incomplete mental model of the full path a request takes. Filling in that model doesn't require mastering every protocol in depth; it requires knowing what each layer is responsible for, what it assumes about the layers below it, and what actually breaks when that assumption doesn't hold.

That understanding pays off precisely in the moments it matters most: an outage under pressure, a security review that needs to reason about where an attacker could actually intercept traffic, or a performance investigation that needs to distinguish a DNS problem from a TLS problem from an application problem, quickly and correctly, rather than by trial and error.

References

IETF RFC 791. "Internet Protocol." (IPv4 specification.)
IETF RFC 8200. "Internet Protocol, Version 6 (IPv6) Specification."
IETF RFC 9293. "Transmission Control Protocol (TCP)."
IETF RFC 768. "User Datagram Protocol."
IETF RFC 1034 / RFC 1035. "Domain Names - Concepts and Facilities" / "Domain Names - Implementation and Specification." (Foundational DNS specifications.)
IETF RFC 8446. "The Transport Layer Security (TLS) Protocol Version 1.3."
IETF RFC 9110 / RFC 9114. "HTTP Semantics" / "HTTP/3."
IETF RFC 6455. "The WebSocket Protocol."
IETF RFC 5321. "Simple Mail Transfer Protocol (SMTP)."
Mozilla Developer Network (MDN). "An overview of HTTP" and "Transport Layer Security (TLS)." developer.mozilla.org
Cloudflare Learning Center. "What is DNS?" and "What is a CDN?" cloudflare.com/learning/
Kurose, J. F., & Ross, K. W. Computer Networking: A Top-Down Approach. Pearson. (Standard networking textbook covering the layered internet architecture.)