TextRank v2.2.0 - Sentence Splitting And Token Fixes

TextRank v2.2.0 landed on 10 August 2026 as a stable tag, not a prerelease. The Go ranking library now keeps abbreviation and decimal dots inside tokens instead of treating every period as a sentence end.

The full release notes and downloads are on the GitHub release page.

TextRank builds a graph from parsed words, phrases, and sentences, then ranks that graph. Parser mistakes become ranking mistakes. If a period always starts a new sentence, strings such as U.S.A and 3.14 split into extra sentences and extra tokens. Occurrence counts, phrase chains, and sentence weights then move.

v2.2.0 is the parser correction for that case. The notes, tracked as issue #14, state that a dot no longer ends a sentence between single letters like U.S.A, or between two digits like 3.14. U.S.A now stays a single token.

That is the change operators will see. Pipelines that extract keywords or summary sentences from news, filings, product pages, or logs will get different graph nodes for the same raw text. Entity names with internal dots no longer split into one letter sentences. Numeric facts no longer split at the decimal.

The tag does not describe a new ranking algorithm, a new finder API, or a language pack change. It describes a tokenizer rule. Compare outputs on a fixture that contains U.S.A and a decimal such as 3.14 before you promote the tag. If a stored gold set was built on the old split, sentence indexes and top N lists will shift.

For batch ETL this is a correctness issue, not a speed issue. Ranked phrases and sentence windows are the product. A broken split on 3.14 or U.S.A changes the product even when wall clock time stays the same.

The same parse path now documents that sentences without words are skipped. Issue #7 is the cited tracker. The notes do not add a config key or a CLI flag for this skip. They document existing behavior.

That still matters on a content ingest path. HTML leftovers, table cells, and OCR noise often produce punctuation only fragments. Those fragments no longer appear in the sentence list that ranking consumes. Finder methods that return the top N sentences then rank a smaller set.

If a downstream job stored sentence position as a stable identifier, the skip can change that position. Recompute stored ranks after the upgrade rather than joining old sentence IDs to new output. The notes do not say the skip is new, only that it is now written down. Treat the documentation as a contract: empty word lists are out of the graph.

Travis is replaced by GitHub Actions. Tests run on Go 1.8 and the latest release. Go 1.8 is the library’s stated minimum. The matrix pins that floor and current tip. It is not a bump of the required language version.

Codecov reporting is on, and the build fails below 95 percent coverage. Test coverage is now 100 percent, including the previously untested root word replacement. Root word replacement sits on the language path. Leaving it untested was a gap for anyone who swaps stop word lists or custom language objects.

The notes also record a data race fix in the multi thread test. TextRank is used concurrently: independent graphs can ingest in parallel. A race in the test is not proof of a production mutex change. The notes do not name a lock, a channel, or a public API that was racy. Treat this as a test harness correction. If you call Populate and Ranking from several goroutines, run the race detector on your own binary after the upgrade.

These CI items do not change ranking math. They change how the project fails a pull request. A 95 percent coverage floor is strict for a small parser. It will reject patches that add untested branches on the tokenize path, which is the path this tag just fixed. The full checklist is in the v2.2.0 release notes.

The rest of the tag is project process. The repo gained a code of conduct, a contributing guide, a security policy, issue templates, and a pull request template. None of those files alter token rules or rank weights.

README now includes a section comparing TextRank to LLMs, and the retired GoDoc badge is replaced with pkg.go.dev. The notes do not ship an LLM client, a prompt path, or an embedding store. TextRank stays extractive ranking on a local graph. The comparison is documentation for people who were about to swap a local ranker for a model call.

If you consume the library as a preprocessor before a model, the tokenizer fix is still the part that changes token counts and therefore cost.