lmp.vars

Variables shared throughout this project.

lmp.vars.BOS_TK

A special token which represents the begining of a sequence.

Type

Final[str]

lmp.vars.BOS_TKID

Token id of lmp.vars.BOS_TK.

Type

Final[int]

lmp.vars.DATA_PATH

Absolute path of all the dataset. Some datasets are hosted on remote servers, thus this variable serve as the dataset download location. See lmp.dset.BaseDset for more information.

Type

Final[str]

lmp.vars.EOS_TK

A special token which represents the end of a sequence.

Type

Final[str]

lmp.vars.EOS_TKID

Token id of lmp.vars.EOS_TK.

Type

Final[int]

lmp.vars.EXP_PATH

Absolute path of all experiments. Experiments are ignored by git. No experiment results (model checkpoints, tokenizer cofiguration, etc.) will be commited.

Type

Final[str]

lmp.vars.LOG_PATH

Absolute path of all experiments’ log. Experiments are ignored by git. No experiment logs will be commited.

Type

Final[str]

lmp.vars.PAD_TK

A special token which represents paddings of a sequence.

Type

Final[str]

lmp.vars.PAD_TKID

Token id of lmp.vars.PAD_TK.

Type

Final[int]

lmp.vars.PROJECT_ROOT

Absolute path of the project root directory.

Type

Final[str]

lmp.vars.SP_TKS

List of special tokens.

Type

Final[list[str]]

lmp.vars.UNK_TK

A special token which represents unknown tokens in a sequence.

Type

Final[str]

lmp.vars.UNK_TKID

Token id of lmp.vars.UNK_TK.

Type

Final[int]

Example

>>> import lmp.vars
>>> assert isinstance(lmp.vars.BOS_TK, str)
>>> assert isinstance(lmp.vars.BOS_TKID, int)
>>> assert isinstance(lmp.vars.DATA_PATH, str)
>>> assert isinstance(lmp.vars.EOS_TK, str)
>>> assert isinstance(lmp.vars.EOS_TKID, int)
>>> assert isinstance(lmp.vars.EXP_PATH, str)
>>> assert isinstance(lmp.vars.LOG_PATH, str)
>>> assert isinstance(lmp.vars.PAD_TK, str)
>>> assert isinstance(lmp.vars.PAD_TKID, int)
>>> assert isinstance(lmp.vars.PROJECT_ROOT, str)
>>> assert isinstance(lmp.vars.SP_TKS, list)
>>> assert lmp.vars.SP_TKS == [lmp.vars.BOS_TK, lmp.vars.EOS_TK, lmp.vars.PAD_TK, lmp.vars.UNK_TK]
>>> assert isinstance(lmp.vars.UNK_TK, str)
>>> assert isinstance(lmp.vars.UNK_TKID, int)