There has been a lot of enthusiasm and hype in the realm of artificial intelligence (AI) surrounding a recently created technology called GPT-3. Simply said, it is an AI that is superior to everything that has come before it at creating content with a language structure – human or machine language.
GPT-3 was developed by OpenAI, a research company co-founded by Elon Musk, and has been dubbed the most significant and usable advancement in artificial intelligence in years.
However, there is some confusion about what it does (and, more importantly, what it does not do), so here I will attempt to simplify it for any non-technical readers interested in understanding the core principles behind it. I’ll also discuss some of the issues it brings, as well as why some believe its importance has been somewhat exaggerated by hype.
Image credits : https://pixabay.com/photos/robot-mech-machine-technology-2301646
What is GPT-3 capable of?
GPT-3 is capable of producing anything with a language structure — this includes answering questions, writing essays, summarising lengthy books, translating languages, taking memos, and even writing computer code.
Indeed, in one online presentation, it is demonstrated how to create an app that looks and performs similarly to the Instagram application by utilising a plugin for the widely known software tool Figma.
This is, of course, quite novel, and if it proves to be usable and beneficial in the long run, it might have profound consequences for the future development of software and applications.
Due to the fact that the code is not yet publicly available (more on that later), access is restricted to a select group of developers via an OpenAI-managed API. Since the API’s release in June of this year, instances of poetry, prose, news reporting, and creative fiction have appeared.
This article is particularly interesting because it shows GPT-3 attempting – pretty persuasively – to convince us humans that it does no harm. Although its robotic honesty forces it to confess that “I am aware that I shall be incapable of avoiding destroying humanity” if wicked people push it to!
How does GPT-3 function?
GPT-3 is a language prediction model in terms of the broad categories of AI applications. This means that it is an algorithmic framework that takes a single piece of language (an input) and transforms it into what the algorithm predicts would be the most beneficial piece of language for the user.
This is possible because of the extensive training analysis performed on the massive amount of material required to “pre-train” it. In comparison to other algorithms that have not been trained, OpenAI has already expended the massive amount of compute resources required for GPT-3 to comprehend how languages work and are constructed. According to OpenAI, the compute time required to accomplish this cost $4.6 million.
To learn how to generate language constructions such as sentences, it uses semantic analytics – not just the words and their meanings, but also how the usage of words varies depending on the other words in the text.
It is also referred to as unsupervised learning because the training data does not include any information about what constitutes a “correct” or “wrong” response, as supervised learning does. All of the information required to compute the probability that the output would satisfy the user’s requirements is acquired directly from the training texts.
This is accomplished by analysing the usage of words and sentences, then dismantling them and attempting to reconstruct them.
For instance, the algorithms may come across the phrase “the home has a red door” during training. It is then given the phrase again with an omission of a word – for example, “the home has a red X.”
It then examines the text in its training data – hundreds of billions of words organised in meaningful language – and chooses which word should be used to reproduce the original phrase.
To begin, it is almost certain to get it wrong — maybe millions of times. However, it will ultimately come up with the correct word. By comparing the result to the original input data, it determines whether the output is right, and a “weight” is awarded to the algorithm step that produced the correct response. This means that it “learns” over time which strategies are most likely to produce the proper response in the future.
The magnitude of this dynamic “weighting” process is what distinguishes GPT-3 as the world’s largest artificial neural network. As has been noted, what it achieves is not novel in some ways, as transformer models of language prediction have existed for many years. However, the system dynamically stores and employs 175 billion weights to process each query — ten times more than its nearest competitor, built by Nvidia.
Two AIs talk about becoming human. (GPT-3)
What are some of the difficulties associated with GPT-3?
GPT-3’s ability to generate language has been lauded as the best yet seen in artificial intelligence; however, there are certain critical points to consider.
Sam Altman, the CEO of OpenAI, stated, “The GPT-3 Hype is excessive.” AI will fundamentally alter the world, but GPT-3 is only a glimpse.”
To begin, it is an extremely expensive tool to use at the moment, owing to the massive amount of compute power required to perform its job. This means that the expense of implementing it would be prohibitively expensive for smaller enterprises.
Furthermore, it is a closed or black-box system. Because OpenAI has not disclosed all of the facts of how its algorithms function, anyone depending on it to answer queries or build valuable products would not be totally certain how they were created.
Thirdly, the system’s output is not yet ideal. While it is capable of producing brief messages and simple software’s, its output becomes less useful (indeed, it is described as “gibberish”) when requested to create something longer or more sophisticated.
These are undoubtedly concerns that will be solved over time — as the cost of compute power continues to fall, standardization around open AI platforms is built, and algorithms are fine-tuned with larger data volumes.
Overall, it’s reasonable to conclude that GPT-3 generates findings that are light years ahead of anything previously observed. Anyone who has seen the outcomes of AI language understands how variable they can be, and GPT-3’s output unquestionably appears to be a step ahead.
When we see it properly placed in the hands of the public and accessible to all, its performance should improve even further.
Also Read:
