Studying with ChatGPT: Simplest Code for a MLM

Paul Xiong
Jun 1, 2023

--

can you manipulate a simple dataset, use it to train a MLM, don’t use pre-trained mode, I just want to learn how it will be coded.

Above code has some error, fixed and ran successfully on CoLab. Source code on GitHub.

--

--

Paul Xiong
Paul Xiong

Written by Paul Xiong

Predicting the next word (token) is what powers ChatGPT, while predicting the next photo (embedding) forms the foundation of ImageGPT.

No responses yet