Welcome to Language Model Playground’s documentation!#
Language model playground is a tutorial about “How to implement neural network based language models”. We use Pytorch to implement language models.
We have implemented several language models including:
Elman Net. (See
ElmanNet
.)LSTM and its variations. (See
LSTM1997
,LSTM2000
andLSTM2002
.)Transformer encoder. (See
TransEnc
)And more to come!
You can easily create these models instance using module lmp.model
.
You can also train these models directly using CLI script lmp.script.train_model.
import lmp.model
model = lmp.model.ElmanNet(...) # parameters go in here.
model = lmp.model.LSTM1997(...) # parameters go in here.
model = lmp.model.TransEnc(...) # parameters go in here.
See also
- lmp.model
All available language models.
We have written serveral scripts to demonstrate typical training pipline of language models and demonstrate furthur usage on language models:
Use lmp.script.sample_dset to take a look at available datasets.
Use lmp.script.train_tknzr to train tokenizers.
Use lmp.script.tknz_txt to tokenize text with pre-trained tokenizers.
Use lmp.script.train_model to train language models.
Use lmp.script.eval_txt_ppl to calculate perplexity on the given sample with pre-trained language model checkpoint.
Use lmp.script.eval_dset_ppl to calculate perplexity on dataset over a range of pre-trained language model checkpoints.
Use lmp.script.gen_txt to generate continual text with pre-trained language model checkpoint.
See quick start for typical language model training pipline, or jump directly to contents you are interesting in!