← Packages
python-sentencepiece
Unsupervised text tokenizer and detokenizer
- Current
- 0.2.1
- Candidate
- 0.2.3Stale
- License
- Apache-2.0
RuntimeDeps
Optional RuntimeDeps
| Dependency | Observed source | Current requirement | Candidate requirement |
|---|---|---|---|
2.4.6 | Not observed | ||
7.35.1 | Not observed | ||
9.0.3 | |||
testUnmapped | — | Not observed |
Dependency conditions
- Applies when (any)
- extra == "numpy"extra == "test"
- Applies when (any)
- extra == "protobuf"extra == "test"
- Applies when (any)
- extra == "test"
- Applies when (any)
- extra == "testpaths"
Description
SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems. SentencePiece implements subword units (e.g., byte-pair-encoding (BPE) and unigram language model) with the extension of direct training from raw sentences. This package bundles the SentencePiece C++ library.
Changelog
PR · 1
Checks
| Monitor | Observation | Last checked |
|---|---|---|
Stale Repository refresh: git fetch exited 128 (authentication) | ||
Stale Source last confirmed at 2026-11-10T23:09:30+00:00; repository refresh overdue. | ||
Checked | ||
Checked | ||
Not configured No reliable monitor identity/configuration. | — | |
Metadata unavailable Comparable SPDX license expressions are missing; free text and RPM aggregate License are not compared. | Last attempted | |
Checked | Last attempted | |
Checked Current source component only; local patches, build options, bundled dependencies and binary artifacts are not evaluated. | ||
Checked |