unigram

The Telegram client planned to be optimized for the Miscrosoft Windows 10 Mobile retro-os one day....

a probabilistic language identification system that identifies the language of a sentence

Python Web Crawler implementing Iterative Deepening Depth Search

Word segmentation to create unigrams in Portuguese (pt-br)

[FR - Duo] 2023 - 2024 Centrale Méditerranée AI Master | NLP project about Markov models

High-Performance Azerbaijani Tokenizers (30% fewer tokens, 40% faster than multilingual alternatives)

Telegram for Windows Modification

Multrenizer is a bilingual English-Turkish Unigram tokenizer built from scratch for Turkish morphology, Turkish-aware casing, and mixed TR-EN text.

COMP472 AI MINI PROJECT 3

200k-vocab SentencePiece (Unigram) tokenizer for German-primary LLMs — German/English/code, low fertility, byte-fallback, chat-template tokens. From the Auralis/Helix project.

All AUT's principles and applications of artificial intelligence course projects.

SentencePiece Tokenizer Wrapper implementation for PLDR-LLM with KV cache and G-cache

[draft] phonological unigrams and bigrams

This is a small program that takes two lists, zips them, and translates a file after making the translation dictionary.

The Windows Universal version of Telegram build by the community

find uni/bi/tri-grams from text files

N gram language model

The projects for the NLP course at the University of Isfahan.

BPE, WordPiece, SentencePiece-BPE e Unigram LM implementados do zero em Rust, com benchmark de tempo e compressão