Days pass, and even after weeks, news are still talking about ChatGPT.
For those living in a cavern, ChatGPT developed by OpenAI (which gave us earlier Dall-E), is a powerful language model that has been making waves in the field of artificial intelligence (AI). It uses deep learning techniques to generate human-like text. It is trained on a massive dataset of corpus, allowing it to understand and respond to a wide range of topics and styles of writing. Making it a tool for tasks such as language translation, text summarization, writing creative fiction, writing and analyzing code etc…
However, not everyone seems very impressed by its capabilities. French Turing award winner Yann LeCun, pioneer in AI, recently stated about ChatGPT that “In terms of underlying techniques, ChatGPT is not particularly innovative” . Adding as well that “It’s nothing revolutionary, although that’s the way it’s perceived in the public”.
LeCun’s critique is based on the traditional AI approach known as statistical machine learning of ChatGPT, in which OpenAI is far from being the only one working on, nor necessarily ahead of other research labs, nor the breakthroughs maker.
According to LeCun, the technology behind the GPT-3 model comes from different works such as:
- self-supervised learning, a technology advocating for a long time, even before OpenAI existed by LeCun,
- transformers architecture, a deep learning technique introduced by Google researchers in 2017,
- human feedback (RLHF) inaugurated by DeepMind -now owned by Google- in 2017.
As summarized on Twitter : “ChatGPT and other great language models didn’t come out of nowhere, they are the result of decades of contributions from a variety of people” .
So the question is why big companies don’t go public as OpenAI?
They actually kind of did it…
Google made a demonstration of a similar technology about 2 years ago with LaMDA and Meta has provided to the scientific community its own AI named OPT-175B which is similar in size to GPT-3 (use for the current ChatGPT).
But to LeCun, the answer seems obvious. Big companies don’t go public because they have nothing to gain (or on the opposite more to loose, as Meta and Microsoft have experienced with BlenderBot and Tay and the bad press they got) than a small company looking for investors.
For sure, OpenAI gave a big kick in the anthill, and might leave no choice to other actors in the domain to release generative AI services in the future to quickly counter OpenAI’s initiatives.