ChatGPT is a long-form question-answering AI that has been introduced by OpenAI. In addition to the simple ones, it can answer difficult questions. Due to the revolutionary technology, it can easily understand the question asked by a human. It offers a human-like response naturally. ChatGPT can alter how information is collected through a digital platform. 

What is ChatGPT?

Large language model chatbot ChatGPT can interact with a human using conversations as a medium. It offers surprisingly human responses. The model allows it to predict the next word in the series. RLHF or Reinforcement Learning with Human Feedback provides an additional layer that enables the ChatGPT to learn from the human feedback, follow the directions, and generate satisfactory responses. 

Who Built ChatGPT?

San Francisco-based AI company named OpenAI has developed ChatGPT. OpenAI is known mostly for its DALL-E which is a deep learning model. Using the text instructions ‘prompts’, it can generate images. 

Large Language Models

ChatGPT is a part of the Large Language Model and contains a large number amount of data to predict the word that may come next in a sentence. An increased amount of data enables the language model to deliver more satisfactory answers.  

GPT-3 has about 175 billion parameters and is trained using 570 gigabytes of text. In comparison, GPT 2 is smaller by 100 times with about 1.5 billion parameters. An increase in sales of GPT-3 can change the model behaviour. GPT -3 has not been trained enough. However, it can translate English to French. In the case of GPT-2, this behaviour is missing completely. GPT-3 model can outperform its predecessors and some other models built for solving tasks. 

LLMs can predict the next word as well as the next sentence. It offers automated replies on a huge level. It can write paragraphs and entire page content. However, there is a certain limitation to LLM as it does not always know what a person is searching for.

ChatGPT needs to improve its features using Reinforcement Learning with Human Feedback or RLHF training. 

How ChatGPT is Trained?

GPT-3.5 is trained using massive data with information and code that can be found on various online sources such as Reddit. It enables ChatGPT to learn how to converse. It also helps them to learn a human style of responding.  

ChatGPT receives information from Reinforced Learning with Human Feedback. By looking at the question. AI tries to learn what you expect from it. Training LLM in this way is revolutionary as the process goes beyond the simple training from predicting the next word. The process has a positive impact on the LLM as it teaches what a given human likes to achieve. 

The language model optimizes the prediction of the next word. It is just a proxy to know what the model actually wants to achieve. Techniques used now seem promising. However, it also tells that they have the power to alter everything. Through results, it is proved that the technology used can make the language model more powerful further.

If the language model becomes huge, it may not able to ensure the user’s intent. For example, the output can be toxic and untruthful. In simple words, the model may not align with the users.

To rate the output of both GPT-3 and InstructGPT (ChatGPT sibling model), engineers are hired. By looking at the rating, the researchers have come to this conclusion.

• Labellers prefer InstructGPT in comparison to the GPT-3 output
• InstructGPT shows progress over GPT-3 in truthfulness
• InstructGPT shows relatively lesser progress over GPT-3 in toxicity. However, it is not biased. 

According to the research paper, InstructGPT offers positive results. However, it still needs to improve.

Results have shown that the large language model fine-tunes itself through human interactions. Over time, it has shown significant improvement in behaviour while doing a wide range of tasks. However, a lot of work should be done to improve reliability and safety. 

So, how ChatGPT is different from other simple chatbots? It has been specially trained to understand the intent of humans while they pose a question. In this way, ChatGPT can provide a truthful, helpful and harmless reply.

Due to the training, ChatGPT can challenge a question and eliminate unnecessary parts from it. Other research papers show how AI is trained to predict human preferences.

Limitations of ChatGPT

On paper, ChatGPT is looking very impressive. But it has its limitations. For example, it cannot answer certain questions if it is worded in a specific way. Therefore, questions have to be reworded properly to get a reply. The bigger limitation on the occasion is to deliver a quality answer. Still, some of the experts have explained that ChatGPT can give unique and plagiarism-free answers.

Sometimes, ChatGPT has been accused of giving biased answers. ChatGPT may seem biased towards a certain section of people. Plus, it may have an inclination towards toxicity.