Day 90 in MIT Sloan Fellows Class 2023, Advanced Data Analytics and Machine Learning in Finance 9, Thoughts from ChatGPT

ChatGPT creates a huge phenomenon and shows potential impact of NLP model.

However, many people struggle with what differentiates it from other past model and what would happen next.

This article is a good summary of chat GPT if you are not familiar with the concept.

Anaconda | The Abilities and Limitations of ChatGPT

I am taking this class and we have some guest speakers in the modern NLP world.

So let me introduce some thoughts from MIT CSAIL professionals.

What differentiates ChatGPT from other past AI products?

To be honest, it is just a progress of great UI. It is my opinion, but there is no significan break-through in model itself. The paradigm change already came a couple of years ago. ALl the NLP AIs need to resolve specific tasks such as summarization, QA and translation.

However, nowadays, language model is incredibly simple model.

It is just predicting the next word according to context and previous word.

However, to make this model activated in the various usecase, you just need to expand the model capacity. Now, there is Billion level parameter and TB training data.

Sizes of state-of-the-art NLP models have increased by more than three orders of magnitude from 2018 to 2021.