Syntax Highlighter

One semantic tokenizer for browser-native CSS Highlights, HTML/SSR, JSON, and ANSI output.

Why semantic tokens?

Language definitions produce stable types such as keyword, string, function, and comment. Every renderer consumes the same token stream, so a snippet looks identical in a live editor, server-rendered HTML, developer tooling, and the terminal.

import { createHighlighter } from "@opentf/syntax-highlighter";

const highlighter = await createHighlighter({ language: "typescript" });
const tokens = highlighter.highlight(source);
// tokens: [{ type: "keyword", start: 0, end: 5 }, ...]

Tokens use exclusive UTF-16 start/end offsets, so any renderer recovers text with source.slice(start, end) — no text is ever stored inside a token.

The pipeline

1. Tokenize — createHighlighter({ language }) loads the language definition, highlight(source) returns Token[].

2. Render — pick one output per surface: CSSHighlightRenderer in the browser, renderHTML for SSR, renderANSI for terminals, renderJSON for tooling.

3. Theme — shared semantic themes work across the browser and HTML renderers; ANSI themes ship as hex maps.

Next steps

• Installation — add the package with your manager.

• Quick start — highlight code in five minutes.

• Renderers — one token stream, every surface.

• API reference — every export, option, and type.