B

BERT neural network

BERT is a neural network language model architecture introduced by Google in 2018 (Devlin et al. 2018). When training a BERT model, the network is trained not to predict the next token in a sequence but to predict a masked token as in a cloze test.