pythainlp.benchmarks

The pythainlp.benchmarks contains utility functions for benchmarking tasked related to Thai NLP. At the moment, we have only for word tokenization. Other tasks will be added soon.

Modules

Tokenization

Quality

../_images/evaluation.png

Qualitative evaluation of word tokenization.

pythainlp.benchmarks.word_tokenization.compute_stats(ref_sample: str, raw_sample: str) → dict[source]

Compute statistics for tokenization quality

These statistics includes:

Character-Level:

True Positive, False Positive, True Negative, False Negative, Precision, Recall, and f1

Word-Level:

Precision, Recall, and f1

Other:

  • Correct tokenization indicator: {0, 1} sequence indicating the correspoding word is tokenized correctly.

Parameters:

  • ref_sample (str) – ground truth samples

  • samples (str) – samples that we want to evaluate

Returns:

metrics in character and word-level and correctly tokenized word indicators

Return type:

dict[str, float | str]

pythainlp.benchmarks.word_tokenization.benchmark(ref_samples: List[str], samples: List[str]) → DataFrame[source]

Performace benchmark of samples.

Please see pythainlp.benchmarks.word_tokenization.compute_stats() for metrics being computed.

Parameters:

  • ref_samples (list[str]) – ground truth samples

  • samples (list[str]) – samples that we want to evaluate

Returns:

dataframe with row x col = len(samples) x len(metrics)

Return type:

pandas.DataFrame

pythainlp.benchmarks.word_tokenization.preprocessing(txt: str, remove_space: bool = True) → str[source]

Clean up text before performing evaluation.

Parameters:

  • text (str) – text to be preprocessed

  • remove_space (bool) – whether remove white space

Returns:

preprocessed text

Return type:

str