Skip to main content
Back

This article wasn’t written by a machine

Machine Learning and Language Models

Words can capture the ideas that reflect our times – but who will be the authors? 

In a nutshell

  1. Machine intelligence has reached an inflection point: things are about to change. The change will transform our lives.
  2. Language models transform a paragraph of text into an equation, bringing us closer to a world where machines can read, write and translate with fluency.
  3. Mega-companies are racing to develop better and bigger versions of language models
  4. How could the world change when a near infinite amount of content could be generated or summarised in any style or on any topic? 

The printing press changed the world by making the spread of ideas a mass market service, but what inspired the authors of these ideas and how they reflected the mood of the times could not have been mechanised.  

When Robert Hughes penned the first chapter of The Shock of the New - “The Mechanical Paradise – How the development of technology influenced art”, he would have envisaged human artistic minds, not machines, doing the creating. But today you could not make the same assumption. 

This article wasn’t written by a machine. It was written by a human - but it could have been.  

And that’s the point. Some of the oldest problems in machine learning relate to language. Understanding it, editing it, translating it and creating it.  

In the past couple of years, machine intelligence has reached an inflection point: things are about to change. The change will transform our lives. It is already giving us the first glimpse of what a general machine intelligence might be like. 

Solving the conversation equation 

Last year, OpenAI, a company backed and run by Silicon Valley royalty like Elon Musk, Sam Altman and Reid Hofman, announced a language model so good that it was “too dangerous to release to the public”. They called it GPT-2. A few months ago, they started slowly releasing GPT-3 and it nearly broke the internet. To understand why, we first need to understand what a language model is and why it matters. 

A language model is a numerical representation of language. The “numerical representation” part is important. Math works with numbers and not words. Language models make a paragraph of text, question or chat conversation an equation to solve. Modern computing means that entire datacentres, the size of football fields, can be put to work solving it. This is out of reach of most of us. If you want to own the future of language, it’s pocket change.  

Bigger, much bigger - better, much better 

We’re poised to cross an important threshold and Silicon Valley mega-companies are racing to be the first. The likes of Google, Facebook and Microsoft have released better and better versions of language models over the last couple of years, searching for “human level performance”. Open AI’s GPT-3 might just achieve it. It’s ten times bigger than the previous biggest model. It was trained by ‘reading’ all of Wikipedia and most of the rest of the internet as well. Size really is everything with these models - that’s what’s been behind the massive leaps we’ve seen in performance in the last two years. As they get bigger, much bigger, they get better, much better. 

What this means in practice is truly astonishing. GPT-3 has written an eloquent op-ed for the Guardian on why you should trust AI. It can write code in several of the world’s most common languages. It’s built websites based on just a description. It can even write credible answers to questions and take part in engaging chat discussions. As Peter Thiel said, it “provides 10,000 PhDs that are willing to converse with you”. 

The myth of the future  

I don’t think that this points to a world where all writers and programmers are out of a job. I do, however, think it points to a place where their day to day jobs are significantly augmented by systems like this. Imagine a programmer getting the first draft of a new website out in minutes by simply describing it then spending time refining that initial design. Perhaps the AI then bug checks and optimises that code before it gets deployed. I’m a better programmer than a writer so it’s less clear to me how writing evolves but I am sure that it will be a collaboration between humans and machines.  

It’ll be no less transformative, but potentially much more subtle, for people going about their everyday lives. Think about the exponential improvement in image recognition models over the past few years – from not being able to tell the difference between a cat and a dog to being able to identify adults by baby photos. We now unlock our phones just by looking at them. In fact, this technology is now so good that big players in the space have put a moratorium on some use cases due to mass surveillance fears. 

GPT-3 and whatever comes next are about to have a similar impact on our everyday lives. How could the world change when a near infinite amount of content could be generated or summarised in any style or on any topic? Combine this with advances in voice-based digital assistance and it’s not hard to foresee a world where most interactions with brands could be via a discussion with a bot. 

This has implications for the news and media landscape too. If you think fake news and sock puppet accounts on social media are a problem now, it could get a whole lot worse without guardrails in place. That, in turn, has some disturbing implications for our political system. 

For businesses, GPT-3 presents opportunity. New business models become possible as the cost to serve on any interaction with the customer falls by orders of magnitude. Every business has challenges around information retrieval. Imagine a Microsoft Teams bot that can answer any question about any piece of information in a document on your corporate SharePoint. Like “when is the contract with ACME coming up for renewal” or “how do I submit an expense claim”. GPT-3 can also do things like generate a full email from a set bullet points - imagine a button that could do that in your Outlook. These things may not be too far off. Microsoft just made a billion-dollar investment in OpenAI for the exclusive access to the GPT-3 source code. 

In 1997 machines first beat humans at chess. The very best human wouldn’t stand a chance against an even modestly good machine today. The greatest player of all is a human using a machine in a collaboration. I think we will see this model play out across wider society in the coming decade. 

This article was published in the latest issue of Frame magazine. To request a copy, get in touch with us on hello@tra.co.nz.

 

Frame magazine

Antony Ede
Partner

Find out how TRA can help you find your purpose

back to top

Discover more content

Stay in touch!

Sign up to receive our latest thinking straight to your inbox each month.