New Kid on the Block — ChatGPT

Remember those times when even a ball of wool would trigger some curiosity. This series of articles is a mix of child like curiosity and crystal ball gazing, a 360 degree, wide eyed look at the latest and fanciest technology.

The first one is on ChatGPT. So what is it. Let’s lift it to eye level and take a curious look.

You must be living under a rock if the term ChatGPT hasn’t crossed you in the recent times.

Simply put, an interactive chatbot powered by machine learning, ChatGPT is a large-scale language model developed by OpenAI that is capable of generating human-like text. The model is based on the transformer architecture and is trained on a massive dataset of internet text, allowing it to generate highly coherent and contextually appropriate text. Basically, it has consumed and digested all that has ever been written by humanity, learns from it and renders coherent, intelligible text in forms of story, poetry, answer, email, essays or even an article. Though let me hasten to assure that this particular article is written by a human (yours truly) and not artificial intelligence.

One of the key features of ChatGPT is its ability to generate text that is highly contextually appropriate. This is achieved through the use of a large-scale language model that is trained on a massive dataset of internet text. The model is able to understand the context of a given prompt and generate text that is highly coherent and relevant to the given context. In addition to its ability to generate contextually appropriate text, ChatGPT also has several other useful features. For example, the model is able to perform a wide range of natural language processing tasks such as text generation, text completion, and question answering.

One of the major challenges in developing large-scale language models like ChatGPT is the amount of computational resources required to train and run the model. This is due to the large amount of data that is required to train the model, as well as the complexity of the model itself. To overcome this challenge, OpenAI has developed a number of techniques for training and running large-scale language models like training on powerful GPUs, optimised architecture for efficient use of computational resources. It’s also worth noting that OpenAI has released the GPT-3, an even more powerful version of the model with 175 billion parameters, which has been shown to perform several tasks with high accuracy with little or no fine-tuning, and has been used in a wide range of applications. ChatGPT is a variant of the GPT (Generative Pre-training Transformer) language model. It is trained on a large dataset of text and learns to generate text in a similar style. When given a prompt, ChatGPT uses its pre-trained knowledge to generate a response. The model uses a neural network architecture called a transformer, which is able to handle sequential data such as text. It also uses a technique called unsupervised pre-training, which allows the model to learn general language understanding before fine-tuning on specific tasks.

ChatGPT is a variant of the GPT (Generative Pre-training Transformer) language model which uses a transformer neural network architecture. The model is trained on a large dataset of text and learns to generate text in a similar style. The model architecture is composed of an encoder and a decoder. The encoder reads in the input text and generates a set of hidden states. The decoder then uses these hidden states to generate the output text. The transformer architecture allows the model to efficiently handle sequential data such as text by using self-attention mechanisms to weigh the importance of different parts of the input when generating the output.

During the training phase, ChatGPT is pre-trained on a large dataset of text using unsupervised learning. The model is trained to predict the next word in a sentence given the previous words. This pre-training allows the model to learn general language understanding before fine-tuning on specific tasks.

During the inference phase, the model is given a prompt, and it uses its pre-trained knowledge to generate a response. The model uses a technique called beam search to generate the output, which involves maintaining a set of top-scoring output sequences and extending them iteratively to generate the final output.

The model also uses a technique called byte-pair encoding to handle out-of-vocabulary words. This technique uses a set of subwords to represent any word that is not in the predefined vocabulary, allowing the model to handle a large number of words and reduce the size of the model.

Despite its impressive capabilities, ChatGPT also has some limitations. One limitation is that the model is currently only able to generate text in the English language. Additionally, the model’s text generation can be biased or can generate nonsensical or dangerous outputs, especially when given a prompt with a lack of context or a prompt that is designed to generate such outputs.

Overall, ChatGPT is a powerful language model that is capable of generating highly coherent and contextually appropriate text. With continued development and research, it has the potential to become an even more valuable tool for a wide range of natural language processing tasks. However, it is important to be aware of its limitations and use it responsibly.

The world has already lifted ChatGPT to new heights including making it the most searched term on Google. People have also started working on ways to combat its misuse.

There must be alarm bells ringing amongst regulators, academia, poets, authors et al. Are we set to allow and empower nefarious actors to drive public discourse in a harmful direction. Is the education system equipped to distinguish and hence evaluate the human intelligence in face of such easily accessible, human-like intelligence. The academic community is veering around to the opinion that technology should itself be utilised to detect smart substitution of artificial intelligence for getting one’s own intelligence evaluated. One could argue, though, that it takes a special kind of intelligence to even think of this substitution and then further engender it.

Does a creative author stand to lose all credibility and credit since a more accomplished writer is ready at the click of a few keys, with zero probability of writers’ block or grammatical errors. Imagine, having a Shakespeare, GB Shaw or Proust write new material on demand (and command) on topics and situations that did not exist in their times. One could generate a sonnet, I am told, on 5G! Since I am a poet (of a fairly ordinary variety, admittedly) , I am especially shaken up at this premature possibility of being rendered pointless. In the field of writing, AI has been taking over the field bit by bit but ChatGPT is a force and I can visualise many established notions getting blown away in its wake. ChatGPT is strongly indicating that the only intelligence and creativity one needs is that to think of effective prompts.

Another upheaval which could happen is in the field of code writing, given that ChatGPT allows to write codes, flawless and effective in the blink of an eye without having to master any sort of coding language. This portends a mass displacement of the number one aspiration of Indian parents. Simple interpolation of the evident power of this conversational tool is the realisation that members of the hitherto exalted coder community can be quickly rendered superfluous if they are repeating what has already been done.

Communication is a powerful tool and plays a very significant role in defining the social thought, culture and response. Can a Hitler or for that matter, can a Gandhi be recreated on basis of the content of their oratory. Can the society at large , be influenced to follow this recreated version, given that artificial intelligence would assuredly give the best, most effective communication pack in the hands of this recreated, potential leader. This scenario further begets the question, can human intelligence distinguish an artificial influence and guard against it. Is it even artificial, if it has been derived out of the cumulative human communication. Is there a need to guard against such influences. I believe yes we need to. In fact, there is a dire need to guard against all negative influences whether human or artificial but the moot question is how. As history has shown time and again, we the humans, are quite susceptible to intellectual manoeuvring. Hence, the need for a wisdom filter is perennial and paramount. Can this filter now be developed using the very same technology it purports to guard against. Possibly. Hope to see some quick and effective response in this direction, led and guided by policy makers.

Advent of any new technology has the world divided amongst its users, abusers and cynics. ChatGPT is no exception but this time the stakes are huge, complexity is unprecedented and balance harder to achieve.

This article is authored by Alka Selot Asthana, Executive Director – Telecommunications Consultants India Ltd. Views expressed are personal.