Skip to content
NMSnabbit.

Text Parser Online

Parse text into word/number tokens, one per line.

Text Tools Runs in your browser

What does a text parser do?

Parsing here means tokenising: the input is scanned for runs of Unicode letters and digits, everything else is treated as a separator, and each token lands on its own line. “Order #42 shipped!” yields three lines: Order, 42, shipped. Apostrophes inside words are kept, so “don’t” stays whole. Duplicates are listed every time they occur, in original order.

How to use the Text Parser

  1. 1 Paste a passage of prose or mixed content.
  2. 2 The tokeniser strips punctuation and keeps runs of letters and digits.
  3. 3 Scan the one-token-per-line output for what you need.
  4. 4 Copy the token list into your next step.

What you can use it for

  • Extracting a word list from a passage.
  • Tokenising input for analysis or scripts.
  • Counting or listing distinct words.
  • Preparing text for further processing.

Frequently asked questions

What counts as a token?
An unbroken run of letters and digits, with in-word apostrophes allowed so “it’s” survives as one token. Everything between tokens, from commas to emoji, acts as a separator.
Does it keep duplicate tokens?
Yes, every occurrence stays, in document order. Pipe the output through a duplicate-line remover if you want each word listed once.
Can it handle accented or non-English words?
The tokeniser matches Unicode letter classes, not just ASCII, so café, naïve, Cyrillic and CJK text all come through as whole tokens rather than fragments.

Related tools

Split Text Split text on any delimiter, one piece per line. Text Filter Keep or remove lines that match a word or regex. Word Counter Live counts of words, characters, sentences, lines and paragraphs as you type.

More Text Tools

More tools like this:

All Text Tools