Skip to content

perf: opt-in validated tokenizer for per-atom lines (use_regex=False) - #38

Merged
jameskermode merged 1 commit into
masterfrom
perf/fast-tokenizer
Jun 10, 2026
Merged

perf: opt-in validated tokenizer for per-atom lines (use_regex=False)#38
jameskermode merged 1 commit into
masterfrom
perf/fast-tokenizer

perf: opt-in whitespace tokenizer for per-atom lines (use_regex=False)

9051581
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar