pythainlp.wsd

The pythainlp.wsd contains get word sense function for Thai Word Sense Disambiguation (WSD). The pythainlp.wsd module is designed to assist in Word Sense Disambiguation (WSD) for the Thai language. Word Sense Disambiguation is a crucial task in natural language processing that involves determining the correct sense or meaning of a word within a given context. This module provides a function for achieving precisely that.

Modules

pythainlp.wsd.get_sense(sentence: str, word: str, device: str = 'cpu', custom_dict: dict[str, list[str]] | None=None, custom_tokenizer: Tokenizer = <pythainlp.tokenize.core.Tokenizer object>) → list[tuple[str, float]][source]

Get the sense of a word in a sentence.

This function returns the definitions of the word and their distances from the context in the sentence.

Parameters:

sentence (str) – Thai sentence
word (str) – Thai word
device (str) – device to run the model on
custom_dict (Optional[dict[str, list[str]]]) – Thai dictionary in the form {“word”: [“definition”, …]}
custom_tokenizer (pythainlp.tokenize.Tokenizer) – tokenizer to tokenize the sentence

Returns:

list of definitions and distances (1 - cosine similarity), or an empty list if the word is not in the dictionary

Return type:

list[tuple[str, float]]

This function is based on the ideas in Context-Aware Semantic Similarity Measurement for Unsupervised Word Sense Disambiguation.

It uses the Thai dictionary from Wiktionary. See thai_dict.

It uses the sentence transformers model sentence-transformers/paraphrase-multilingual-mpnet-base-v2 for unsupervised word sense disambiguation.

Example:

>>> from pythainlp.wsd import get_sense
>>> print(get_sense("เขากำลังอบขนมคุกกี้", "คุกกี้"))
[('โปรแกรมคอมพิวเตอร์ใช้ในทางอินเทอร์เน็ตสำหรับเก็บข้อมูลของผู้ใช้งาน',
  0.0974416732788086),
 ('ชื่อขนมชนิดหนึ่งจำพวกขนมเค้ก แต่ทำเป็นชิ้นเล็ก ๆ แบน ๆ แล้วอบให้กรอบ',
  0.09319090843200684)]

>>> print(
...     get_sense("เว็บนี้ต้องการคุกกี้ในการทำงาน", "คุกกี้")
... )
[('โปรแกรมคอมพิวเตอร์ใช้ในทางอินเทอร์เน็ตสำหรับเก็บข้อมูลของผู้ใช้งาน',
  0.1005704402923584),
 ('ชื่อขนมชนิดหนึ่งจำพวกขนมเค้ก แต่ทำเป็นชิ้นเล็ก ๆ แบน ๆ แล้วอบให้กรอบ',
  0.12473666667938232)]

The get_sense function is the primary tool within this module for performing Word Sense Disambiguation in Thai text. Given a word and its context, this function returns the most suitable sense or meaning for that word. This is particularly useful for tasks where word sense ambiguity needs to be resolved, such as text understanding and translation.

By using the pythainlp.wsd module, you can enhance the accuracy of your NLP applications when dealing with Thai text, ensuring that words are interpreted in the correct context.