YouTokenToMe Files

Unsupervised text tokenizer focused on computational efficiency

This is an exact mirror of the YouTokenToMe project, hosted at https://github.com/VKCOM/YouTokenToMe. SourceForge is not affiliated with YouTokenToMe.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
Fix macOS build source code.tar.gz	2019-11-05	89.5 kB	0
Fix macOS build source code.zip	2019-11-05	99.1 kB	0
README.md	2019-11-05	51 Bytes	0
Totals: 3 Items		188.6 kB	0

Fix https://github.com/VKCOM/YouTokenToMe/issues/33

Source: README.md, updated 2019-11-05

Other Useful Business Software

Go from Data Warehouse to Data and AI platform with BigQuery

Build, train, and run ML models with simple SQL. Automate data prep, analysis, and predictions with built-in AI assistance from Gemini.

BigQuery is more than a data warehouse—it's an autonomous data-to-AI platform. Use familiar SQL to train ML models, run time-series forecasts, and generate AI-powered insights with native Gemini integration. Built-in agents handle data engineering and data science workflows automatically. Get $300 in free credit, query 1 TB, and store 10 GB free monthly.

Try BigQuery Free

AI-generated apps that pass security review

Stop waiting on engineering. Build production-ready internal tools with AI—on your company data, in your cloud.

Retool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.

Try Retool free

Recommended Projects

SentencePiece
Unsupervised text tokenizer for Neural Network-based text generation
Hazm
Persian NLP Toolkit
SciSpaCy
A full spaCy pipeline and models for scientific/biomedical documents
UnsupervisedMT
Phrase-Based & Neural Unsupervised Machine Translation
prose NLP
Golang library for text processing

Gen AI apps are built with MongoDB Atlas

Atlas offers built-in vector search and global availability across 125+ regions. Start building AI apps faster, all in one place.

Try Free →

YouTokenToMe Files

Unsupervised text tokenizer focused on computational efficiency

Get an email when there's a new version of YouTokenToMe