Tom's Tech Corner: On the Matter of Why the Internet Sometimes Buffers

Dear Tom,

My name is Linda and I write to you from Ocean City, Maryland. Almost every evening around 8 o'clock, when my husband and I sit down to watch our shows, the video gets that little spinning circle and freezes for a few seconds before it catches back up. It happens most nights around the same time. Is something wrong with our television?

— Linda, Ocean City, Maryland

Linda,

What a wonderful, elemental question, and one I suspect the two of you have been quietly wondering about for months, perhaps years, without ever putting words to it — which is, of course, the highest compliment a reader can pay a columnist, because it means the itch has finally found its scratch. I want you to know before we begin that there is nothing wrong with your television. Your television is, in all likelihood, an innocent bystander in this drama, much like a courtroom stenographer who is present for the crime but did not commit it. The culprit, as is so often the case in matters of the modern home, lies upstream — quite literally, in fact, if we're being precise about the direction electrons prefer to travel through a coaxial trunk line, which is, of course, downstream toward you, but we'll get to that.

Let us begin, as we must, with first principles. What you are describing — a brief pause around the same hour each evening — is a textbook presentation of what those of us who think about these things for a living call "contention," and I want to be honest with you, Linda: contention is not a small subject. Contention is, in the fullest sense, a study of scarcity, and scarcity, as any economist or thermodynamicist will tell you (and here I should note that the two disciplines are, at a deep enough level, describing the same phenomenon — the irreversible tendency of useful things to become less concentrated over time, a principle first formalized in the study of steam engines and later, rather beautifully, extended to neighborhood cable networks), is simply what happens when many parties want the same finite thing at the same finite moment.

Picture, if you will, the "node" — and I use scare quotes there only because the term has been so badly abused in popular writing that I feel obligated to rescue it — the node being the shared segment of cable infrastructure that serves, typically, several dozen to a few hundred homes in your immediate vicinity, assuming, of course, a standard suburban coaxial topology of the sort common along the Delmarva shoreline, which given your address in Ocean City I think we can take as a safe working assumption, subject to the usual caveats about post-1998 fiber-to-the-node overbuilds, which I won't get into here except to say that they change the arithmetic but not the underlying poetry of the situation. At 8 o'clock, Linda, you are not alone. You are one household among many, all reaching, in that same hour, for the same shared resource, in what network engineers rather charmingly call "prime time" — a term borrowed, interestingly, from broadcast television of the 1950s, back when scarcity was measured in transmitter wattage rather than downstream bandwidth allocation, though the underlying shape of the curve, a single tall peak around the dinner hour, has proven remarkably stable across seventy years of otherwise total technological transformation, which I find, personally, one of the more moving facts in all of consumer electronics.

Now — and this is the part I suspect you've been waiting for — what actually happens, mechanically, during your buffering pause is a small, silent negotiation of extraordinary elegance. Your streaming device does not simply demand video and receive it in one unbroken gush, the way water might flow from a tap. No, it requests the video in small chunks, and it keeps a little reservoir of already-received chunks in something called a buffer, of exactly the kind you'd expect, and it draws down that reservoir while more chunks arrive behind it, and so long as chunks arrive faster than the reservoir empties, everything appears seamless to you, the viewer, blissfully unaware of the machinery. But at 8 o'clock, when the shared node is under maximum load from every household on your segment simultaneously requesting their own chunks, the arrival rate can — just briefly, just for a moment — dip below the drain rate, and the reservoir runs dry, and you see the spinning circle, which is in fact not an error at all but rather your device politely informing you that it has run out of buffer and is waiting, with great patience, for the reservoir to refill. It is, in its way, a rather graceful failure mode, a system degrading softly rather than catastrophically, and I confess I find something almost moving in that design philosophy, though I recognize this may not be much comfort to you and your husband mid-program.

I'd also gently point you toward the matter of "adaptive bitrate switching," which is the mechanism by which your streaming service, sensing this exact contention, will often quietly reduce the picture quality for a few minutes rather than let the buffer empty entirely — a bit like a chef, noticing the kitchen is backed up, plating a slightly smaller portion rather than making the whole table wait, which is, when you think about it, the correct decision, though most diners never notice it was made on their behalf. The full mathematics of how the client estimates available throughput and selects among available quality tiers is a fascinating subject in its own right, involving exponentially weighted moving averages of recent chunk download times, and I direct the technically curious reader to Figure 3, which lays out the decision boundary rather elegantly, though I won't reproduce it here for space.

So what you are witnessing, Linda, every evening at 8, is not a fault in your equipment but a small, perfectly ordinary act of civic life playing out invisibly along the cable running past your home — you and your neighbors, all reaching for the same shared pipe at the same shared hour, in a rhythm as old as prime-time broadcasting itself, mediated now by buffers and reservoirs and adaptive algorithms of real sophistication, all of it working, in the aggregate, rather better than anyone has any right to expect. I find it, honestly, a little bit beautiful — a whole neighborhood's evening leisure balanced on a foundation of queueing theory that not one of you asked for and all of you depend on.

I hope this puts your mind at ease about the television, which really was never the problem at all.

Glad I could clear that up!

— Tom