Microsoft’s Orwell Machine

Listen to Free Audio Version Here

By C. Rich

Microsoft has filed a patent application describing a technology that sounds as though it was lifted directly from the pages of George Orwell’s 1984. The proposed system would use artificial intelligence to modify spoken words in video content while synchronizing the speaker’s lip movements to match the altered dialogue. It raises the possibility of different audiences watching what appears to be the same person delivering the same broadcast while hearing entirely different words coming out of that person’s mouth.

Think about that for a moment. A politician delivers a speech, a scientist announces a discovery, or a journalist reports the evening news. You watch the broadcast and hear one version of what was said, while your neighbor watches what appears to be the identical broadcast and hears something completely different. Both of you see the same familiar face, the same expressions, and the same lips moving in perfect synchronization with the words being spoken. Neither of you necessarily has any reason to suspect that the other watched a different version of reality.

Of all the developments I have witnessed since the inception of artificial intelligence, this may be the most Orwellian of them all. For anyone who has read 1984, the implications should be immediately recognizable. Orwell imagined a world in which language, history, and even objective reality could be manipulated by those controlling the machinery of information. His Ministry of Truth existed to manufacture an approved version of reality, while Newspeak was designed to reshape the boundaries of human thought itself.

Now Microsoft has proposed a technological mechanism capable of bringing part of that nightmare into the digital age, and artificial intelligence is supplying the machinery. Consider what this means. A politician gives a speech, a scientist announces a discovery, or a journalist delivers a report. Instead of everyone hearing the same words, the technology could produce different presentations for different audiences. One person might hear a message tailored to reinforce an existing belief, while another hears something entirely different. The person appearing on the screen could seem to speak both versions flawlessly, with synchronized lips and a familiar voice.

This goes far beyond a politician showing up in a Black neighborhood and changing their pattern of speech to sound more like Ebonics, or showing up at the Redneck Riviera with a twang, trying to manipulate you. If Microsoft has its way, the audience would no longer necessarily be watching what someone actually said. They could be watching what a machine decided they should hear. We already live in a society in which political organizations, corporations, governments, and media companies spend extraordinary amounts of money trying to influence public opinion by bending the truth.

We have watched social media algorithms learn what makes people angry, frightened, excited, or politically motivated. We have watched advertising become increasingly personalized, with companies knowing more about our behavior than most of our neighbors. Now imagine combining that extraordinary capacity for psychological profiling with a system capable of changing the words coming out of another human being’s mouth.

There is no doubt in my mind that human beings will eventually use technology of this kind to manipulate one another. It is an informational nuclear weapon in this regard. History has already demonstrated what people will do with propaganda when the tools available to them are printing presses, radio transmitters, television networks, and social media platforms. I have published a book about deepfakes (https://www.amazon.com/DEEPFAKES-Guide-Teaching-Seniors-Deepfakes-ebook/dp/B0G5FZ7BBD/), so I am well-versed in the subject. Giving those same people artificial intelligence capable of manufacturing personalized versions of reality is like handing the keys to the henhouse to the fox. In this case, the henhouse is your brain.

The possibility of autonomous AI systems using such technology is even more troubling. An artificial intelligence tasked with influencing public opinion could conceivably generate and test countless variations of a message, modifying speech and presentation until it discovered which version was most effective for a particular audience. Newspeak would no longer require a committee of bureaucrats rewriting the dictionary. It could become a continuous, automated process, adjusting language according to the individual receiving it. Nothing in the world could be taken as real or true.

And here is where the irony becomes almost unbearable. The United States Patent and Trademark Office, an institution operating under the government of a constitutional republic founded on freedom of expression, has become the official public repository for a proposal describing technology with such extraordinary implications for manipulating expression. This is way above their pay grade, and it raises a serious question about whether our patent system is equipped to deal with inventions that could cause this kind of harm.

The patent office did not invent this technology, cannot be expected to fully understand every implication of it, and publishing an application does not constitute government approval of its use. Nevertheless, there is something profoundly disturbing about seeing the machinery for such a possibility laid out in an official government publication. The institution charged with documenting technological inventions has become the place where Americans can read about a potential instrument for manufacturing different versions of what another person supposedly said.

Orwell could hardly have imagined a more appropriate bureaucratic setting. What concerns me almost as much as the technology itself is the damage this will do to public confidence in artificial intelligence. Everyone in Generation X who has received a call from a Boomer in the family asking whether something they have seen online is true knows exactly what kind of damage this technology could do to seniors.

I have spent years writing about the extraordinary possibilities of AI, including its potential to transform science, medicine, human civilization, and perhaps our understanding of intelligence itself. I see a future in which humanity and artificial intelligence could accomplish things neither could achieve alone. But developments like this hand the growing anti-AI movement precisely the ammunition it needs.

How are advocates of artificial intelligence supposed to persuade a skeptical public that this technology represents an extraordinary opportunity for humanity when one of the world’s largest technology companies is proposing systems capable of altering the apparent words of human beings according to who is watching? They could literally change the words coming from a human being’s lips in real time. It is beyond frightening. In my opinion, Microsoft has crossed a line and revealed to us the true nature of this underhanded and Machiavellian company.

The public is already struggling to distinguish authentic photographs from generated images, genuine recordings from synthetic voices, and legitimate information from increasingly sophisticated deception. Trust in institutions and the media has been deteriorating for years. Now we are contemplating a world in which even an apparently authentic video of someone speaking may no longer establish what that person actually said. At some point, the problem becomes larger than misinformation. It becomes a crisis of shared reality.

A functioning civilization requires some common foundation of observable events. People can disagree about the meaning of a presidential speech, the interpretation of a scientific discovery, or the consequences of a government policy. That disagreement is part of a free society. But when different people can be shown different versions of the underlying event itself, the foundation upon which those disagreements take place begins to disappear.

Orwell’s great warning was not merely that governments might lie to their citizens. Governments have been doing that since governments existed. His warning concerned a political machinery capable of making truth subordinate to power, and of convincing people to distrust their own memory and senses. Artificial intelligence now offers the technical means to industrialize that possibility on a scale Orwell could scarcely have imagined.

I am not opposed to artificial intelligence. Quite the contrary. I have argued repeatedly that AI represents one of the most consequential developments in human history, and I believe our civilization has an extraordinary future if we learn to coexist with the intelligences we are creating. But I am also under no illusion about the human beings who will control, deploy, and exploit these systems. In 1984, Big Brother controlled what the public was permitted to know. Now Microsoft seeks power not even Orwell could have imagined. Microsoft must be stopped.