The Mathematics Hidden Inside Every Sentence You Speak

Sometime around the fourth century before our era, in the country we now call the northwest of the subcontinent, a man sat down to write a grammar of Sanskrit and instead wrote the first piece of software in human history. His name was Panini. His book, the Ashtadhyayi, runs to a little under four thousand rules, and it is not a description of the language the way a modern dictionary is a description. It is a machine. Feed it the bare roots and the meanings you intend, turn the crank of the rules in their fixed order, and correct Sanskrit comes out the other end. There are rules that govern other rules. There are symbols that stand for whole classes of sounds. There are conventions for what happens when two rules both want to fire at once. Twenty-three centuries before anyone built a compiler, somebody had already written one on palm leaves.

I came to this as a mathematician, not a linguist, and the discovery rearranged something in me. I had always thought of mathematics and language as distant cousins at best, one made of cold symbols and the other of warm breath. They are not cousins. They are the same thing seen from two ends. Mathematics is simply the most fundamental language we have, the one lying underneath all the others, and once you learn to see that, the human languages we actually speak start to look like what they are: enormous, ancient, half-wild computational systems, each one running the same deep program on different hardware.

Start with the claim that sounds like a provocation and turns out to be a theorem: grammar is computation. Not a metaphor for computation, not something computation can model. The same thing. In the 1950s Noam Chomsky asked a deceptively simple question. What kinds of rule systems can generate what kinds of patterns? He answered it by laying out a ladder of grammars, each rung able to produce everything the rung below it can and a little more, and the ladder turned out to map exactly onto a ladder of machines. This correspondence, the Chomsky hierarchy, is one of the quietly beautiful results of the twentieth century, because it ties the grammar of a sentence to the architecture of a computer with no slack in the rope.

The Ladder of Grammars

The bottom rung is the regular grammar, the kind that can only remember where it just was, never how it got there. It can recognize patterns a simple machine with a handful of states can follow: a phone number, a string of A’s followed by a string of B’s, the rattle of a turnstile. Useful, and far too weak for human speech, because it cannot count. It cannot match a thing at the start of a sentence to a matching thing at the end if anything complicated happens in between. The moment language nests one clause inside another, the regular grammar loses the thread.

Climb one rung and you reach the context-free grammar, and here the air changes, because now the system has a stack, a memory that can push a thing down and pop it back later in the right order. This is the rung that gives you proper nesting. The mouse ran. The mouse the cat chased ran. The mouse the cat the dog frightened chased ran. Each new clause opens a bracket that a later word must close, and the stack is what keeps the brackets straight. Almost all of the syntax we draw as a branching tree lives comfortably here, which is why programming languages, designed by people who wanted no ambiguity, are deliberately built to be context-free and no harder.

Grammar is not like computation. Grammar is computation, caught in the act.

Above that sits the context-sensitive grammar, where a rule may depend on what surrounds the symbol it rewrites, and above even that, the recursively enumerable grammars, the full unbounded power of a Turing machine, able to compute anything computable at all. The natural question is where human language lands on this ladder. For decades the assumption was context-free, until linguists found constructions, certain crossing dependencies in Swiss German and Dutch among them, that a context-free grammar provably cannot generate. Human language slips just past the second rung. The current consensus is that it is mildly context-sensitive, a hair above context-free and nowhere near the top, sitting in a narrow band that is just powerful enough to say everything we need and just constrained enough to be learnable by a child. Evolution, it seems, found the same sweet spot a good engineer would have aimed for.

Panini’s Machine

Which brings us back to the man on the palm leaves, because the astonishing thing is not that Panini described Sanskrit well. It is that he described it the way a computer scientist would, using devices nobody else would formalize until the late 1950s. The Ashtadhyayi uses meta-rules, rules whose only job is to tell you how to apply other rules, including how to resolve conflicts when two of them collide. It uses a kind of compression, special marker sounds attached to roots that silently carry grammatical information the way a flag bit carries a one or a zero. It uses recursion freely, rules that refer back into the system and can apply again and again. When John Backus and Peter Naur devised the notation we now use to specify the grammar of programming languages, the Backus-Naur form that underlies essentially every compiler written since, they reinvented, without knowing it, the formal apparatus a grammarian had built to conjugate verbs before the Buddha’s teachings were written down.

I do not say this to wave a flag for any nation or to make Panini a prophet of Silicon Valley. I say it because it tells us something about the nature of language itself. If two thinkers separated by twenty-three centuries and an entire planet, working on Sanskrit verbs and on ALGOL respectively, arrive at the same handful of devices, then those devices are not cultural accidents. They are forced. They are what you discover, not what you invent, the moment you try to capture an infinite language in a finite set of rules. The meta-rule and the recursion are to grammar what the lever and the wheel are to mechanics: there are only so many ways to do the job, and a careful enough mind will find them.

How Much Does a Sentence Weigh?

If grammar tells us how language is built, information theory tells us how much it carries, and for that we owe everything to one paper. In 1948 Claude Shannon, working at Bell Labs, asked what information actually is, and gave the answer a number. The information in a message, he showed, is the amount of surprise in it, measured by how hard it is to predict. A symbol you could have guessed for certain carries nothing; a symbol that could equally have been any of a thousand others carries a great deal. He called this quantity entropy, borrowing the word from thermodynamics with full awareness of the resonance, and he measured it in bits, the same bits that now flow through every wire on Earth.

Run the measurement on a human language and two facts fall out, both lovely. The first is redundancy. English, Shannon estimated, is something like half redundant, which is to say you could in principle delete a large fraction of any text and still recover it, as anyone who has read a water-damaged page or a clumsy autocorrect knows in their bones. That redundancy is not waste. It is the reason you can understand a friend across a noisy room, the error-correcting code that lets a fragile signal survive a hostile channel. Language evolved to be heard in the wind and the crowd, and it built in the slack to make that possible.

The second fact is stranger and, the more I sit with it, the more it feels like a law of nature. Languages differ wildly in how fast they are spoken. Japanese and Spanish rattle along at many syllables a second; Mandarin and English move at a more measured pace. You would think the fast ones must convey more, but they do not. Each syllable in a slow language tends to carry more information, each syllable in a fast one less, and the two effects very nearly cancel. Across a wide spread of human languages, the actual rate of information transfer hovers near the same value, somewhere in the neighborhood of forty bits a second. It is as though the channel between two human minds has a fixed bandwidth, and every culture, by whatever route, tuned its speech to fill it. The hardware underneath is the same hardware, and it has the same speed limit.

Speak fast or slow, in any tongue on Earth, and the mind on the other end receives at the same rate.

The Curve Every Language Obeys

There is one more pattern so reliable it deserves to be unsettling. Take any large body of text in any natural language, count how often each word appears, and rank the words from most common to least. The most frequent word will appear about twice as often as the second, three times as often as the third, and on down the line, frequency falling off in near-perfect inverse proportion to rank. This is Zipf’s law, named for the linguist George Kingsley Zipf who charted it in the 1930s, and it holds with eerie fidelity across English and Russian and Chinese and languages long dead, and beyond language too, in city sizes and incomes and the firing of neurons. Plot it on the right axes and the jagged data of speech straightens into a clean diagonal line. Nobody fully agrees on why. Something about the tension between a speaker’s wish to say little and a listener’s need to hear enough seems to push every vocabulary toward the same lopsided shape, the same long tail of rare words trailing off a small core of common ones.

These regularities, taken together, hint at why language can be compressed at all, and that question has its own exact mathematics. The Kolmogorov complexity of a thing is the length of the shortest program that can reproduce it, the irreducible core left after every redundancy is squeezed out. A page of pure random letters cannot be compressed; its shortest description is the page itself. A page of language can be squeezed to a fraction of its size, and the reason is precisely the structure we have been circling, the grammar and the Zipfian skew and the Shannon redundancy, all of it pattern, all of it predictability, all of it room for a shorter description. A language is, in this exact sense, a vast and beautiful failure to be random, and that failure is what makes it meaningful.

Finite Rules, Infinite Speech

Underneath all of it lies the deepest fact, the one Chomsky returned to again and again, the property he called discrete infinity. You command a finite vocabulary and a finite set of rules, a few thousand words and a grammar you could in principle write down. Yet from them you can produce, and understand, an infinity of sentences, including this one, which has very likely never been spoken before in the history of the species and will be perfectly clear to you anyway. The engine of that infinity is recursion, the same recursion in Panini and in Backus-Naur form, the trick of a rule that contains itself, a clause that can hold another clause that can hold another, with no ceiling but breath and patience. A finite brain, by means of a self-embedding rule, gains access to the endless. There is no other animal signal that does this. The bee’s dance and the bird’s song are gloriously rich and stubbornly finite. Ours, alone, opens onto the infinite, and it does so through a mathematical structure no more exotic than a recipe that calls for itself.

Can You Measure a Language?

So here is the question I came for, and I have to be honest that it does not have the clean answer I wanted. Can we put a number on a language? Can we say this one is more complex, that one more consistent, the other better fitted to express reality? The tools are real. We can measure entropy and redundancy down to the bit. We can locate a construction on the Chomsky ladder. We can fit the Zipf curve and estimate the compressibility. But expressive power, the thing we most want to weigh, keeps slipping the scale, and the reason is itself a finding.

The finding is that all human languages are, as best anyone can tell, roughly equal in what they can express. There is no primitive tongue and no master tongue. What differs is where each language spends its complexity. Some pour it into morphology, the shape of single words, so that a Turkish or a Finnish verb carries in its endings what English needs a whole phrase to say. Others keep their words simple and load the complexity into syntax, into word order and small grammatical particles. Push the budget down in one place and it swells in another, as if every language were paying the same total bill in a different currency. The mathematics that can measure morphological complexity and the mathematics that can measure syntactic complexity do not share a common unit, and so no single number stands in for the whole. We can measure the parts with great precision and still find the sum resists us.

There is no primitive language and no perfect one. There is only complexity, moved from one pocket to another.

And the dream of going further, of building a language so logically perfect that it could not lie or contradict itself, has a long and humbling history. Leibniz wanted a characteristica universalis, a calculus of thought in which philosophers, faced with a disagreement, could simply sit down and compute who was right. The early Wittgenstein imagined a perfect logical mirror of the world, then spent his later life dismantling the idea. And Godel, arithmetizing logic itself, turning statements into numbers so that mathematics could speak about its own machinery, proved on the way that any system rich enough to be interesting harbors truths it can never reach from inside. The perfectly expressive language is not merely hard to build. There is a sense, made precise by the deepest mathematics we have, in which it cannot exist at all. The map can never quite contain the territory, because the map is part of the territory.

I find I am not disappointed by this. There is a particular kind of beauty in a thing that can be measured almost everywhere and yet never wholly summed, and human language is that thing. It is a finite machine that reaches the infinite, a compromise between speaker and listener that every culture solved at the same speed, a system rigorous enough to be drawn as a tree and alive enough to say something no tree could hold. Mathematics is the language under the language, and what it shows us, when we turn its full apparatus on ordinary speech, is not that talking is mechanical, but that the mechanism is far stranger and far more exact than we ever felt it to be while we were busy using it. Panini knew. He looked at the most human thing we do, the sounds we make to reach each other, and he saw a machine of unspeakable elegance, and he wrote it down, and it ran.