Inference method base class#

class lmp.infer.BaseInfer(*, max_seq_len: int = 32, **kwargs: Any)[source]#

Bases: ABC

Inference method abstract base class.

Implement basic functionalities for language model inference, including text generation and parsing inference hyperparameters.

Parameters

  • max_seq_len (str, default: 32) – Maximum length constraint on generated token list. One can use larger contraint compare to training.

  • kwargs (Any, optional) – Useless parameter. Intently left for subclasses inheritance.

infer_name#

CLI name of the inference method. Only used to parse CLI arguments.

Type

ClassVar[str]

max_seq_len#

Maximum length constraint of generated token list.

Type

str

See also

lmp.infer

All available inference methods.

classmethod add_CLI_args(parser: ArgumentParser) → None[source]#

Add inference method hyperparameters to CLI argument parser.

Parameters

parser (argparse.ArgumentParser) – CLI argument parser.

Return type

None

See also

lmp.script.gen_txt

Use pre-trained language model checkpoint to generate continual text of given text segment.

abstract gen(model: BaseModel, tknzr: BaseTknzr, txt: str) → str[source]#

Generate continual text conditioned on given text segment.

Parameters

  • model (BaseModel) – Pre-trained language model which will be used to generate text.

  • tknzr (BaseTknzr) – Pre-trained tokenizer which performs text encoding and decoding.

  • txt (str) – Text segment which the generation process is conditioned on.

Returns

Generated text.

Return type

str