Actually, there is already a language model (LM) named BERT that was very popular in 2018 and onwards. After being open sourced by Google in 2018, it quickly became THE work horse and foundation for many natural language processing (NLP) algorithms and services. Shortly thereafter, similar models based on BERT emerged in the research with a similar naming scheme, such as ALBERT and ERNIE in 2019.
However, even though BERT builds on the same architecture as GPT, BERT differs quite a bit with the currently popularized LLMs because it compresses sentences into a fixed size vector. It does not directly output texts like GPT.
1 Like