Ngram language model LM

Ngram remembers sequences of n tokens.

Unigram model is a special case of n=1

Bigram = 2, Trigram = 3, …

Usage

Predict n-1 word a.k.a. Markov model

E.g. please turn off your hand _ -> please turn off your handphone

Google’s Ngram viewer

Expensive for large N, diminishing returns

Applicable to many classification tasks

2021-01-15