.sscc v3 Binary Format

.sscc files are compiled representations of .ssc source modules. Version 3 replaces the v2 CBOR+gzip layout with a flat token stream over a patricia-trie string dictionary.

File layout

bytes  0-3   magic        = "sscc"  (0x73 0x73 0x63 0x63)
byte   4     version      = 0x03
byte   5     compression  = 0x00 (no compression) | 0x01 (gzip)
bytes  6-9   crc32        big-endian CRC32 of payload bytes (bytes 10+)
bytes 10+    payload      (decompressed before parsing if compression != 0)

Payload

payload = trieSection streamSection extension*

trieSection  = trieLenBE32(4 bytes)  trieBytes[trieLen]
streamSection= token*   (self-delimited by ModuleEnd sentinel)
extension    = DocumentBlob

All integers inside the stream and trie are unsigned LEB128 varints unless noted otherwise. Big-endian 32-bit integers are used only for trieLen and raw-blob lengths.

Patricia-trie dictionary

Every string in the stream is stored as a varint ID that indexes into the trie dictionary. The trie encodes all unique strings with shared-prefix compression.

Trie encoding

trie = nodeCount:varint  node[nodeCount]

node = edgeLen:varint
       edgeBytes[edgeLen]          UTF-8 bytes of this edge label
       terminal:byte               0x00=non-terminal  0x01=terminal
       [terminalId:varint]         present only when terminal=0x01
       childCount:varint
       child[childCount]:
         firstByte:byte            first byte of child edge (routing key)
         childIdx:varint           DFS pre-order index of child node

Nodes are numbered in DFS pre-order; root = index 0. The decoder reconstructs an Array[String] indexed by terminalId.

Stream token kinds

Token kind values 0-15 cover module structure and the optional content extension. Kind values 16+ are reserved for future sections (YAML events, Phase C).

Import binding = nameRef:varint, hasAlias:byte, [aliasRef if hasAlias=1], hasFrom:byte, [fromRef if hasFrom=1], spanOpt

spanOpt = 0x00 (absent) | 0x01 startLine startCol startOffset endLine endCol endOffset (all varints)

Structural grammar

stream   = ModuleStart spanOpt ManifestBlob? section* ModuleEnd extension*
section  = SectionStart level headRef hSpan sSpan
           content*
           section*
           SectionEnd
content  = Prose textRef spanOpt
         | CodeStart … (CodeBlob | CodeSmTokens) CodeEnd hasError …
         | Import …
         | ListStart … ListItemStart* ListEnd
extension = DocumentBlob len bytes

DocumentBlob is optional and appears only after ModuleEnd. It carries the semantic Markdown-hosted DocumentContent snapshot used by std/content and frontend toolkit lowering. Because the executable stream is self-delimited, older v3 readers stop at ModuleEnd and ignore the trailing blob.

Scalameta token sub-stream (CodeSmTokens)

For parseable code blocks (lang = "scalascript"), the preprocessor (PreprocessorRegistry.applyAll) runs at write-time. The resulting vanilla-Scala 3 source is tokenised with scalameta and stored as a token sub-stream. At read-time the source is reconstructed by concatenating token texts; scalameta re-parses the block without calling the preprocessor.

Each smToken in the sub-stream:

smToken = kind:varint  [dictRef:varint]

dictRef is present only for _variable-text_ kinds (identifiers, string literals, comments, etc.). _Fixed-text_ kinds (keywords, single-character punctuation, whitespace) carry no payload; their text is implied by the kind number.

Sm kind table

Class-map dispatch: the writer uses a HashMap[Class[?], Int] keyed by tok.getClass. This avoids scalameta 4.17's type-pattern ambiguity where abstract supertypes (T.Symbolic, T.HSpace, T.EOL, T.MultiToken) shadow their concrete subtypes. Six private[meta] classes (T.Invalid, T.Unquote, T.MacroSplice, T.LFLF, T.InfixLF, T.Ellipsis) are absent from the map and fall to Sm.Unknown.

Backward compatibility

SsccFormat.read dispatches on the version byte:

• 0x01 / 0x02 → v2 CBOR reader (unchanged)
• 0x03 → v3 token-stream reader (this document)

v3 reader still handles CodeBlob (kind 7) for non-parseable blocks and as a fallback when scalameta tokenisation fails.

The optional trailing DocumentBlob is backward-compatible at the executable module level: old v3 readers ignore it, and new readers accept old files where the blob is absent. A file without the blob loads with Module.document = None.

Versioning policy

Any new structural kind inside the executable stream, sm-token kind, or YAML-event kind (Phase C) is a breaking change to the v3 format. Such additions bump the minor byte planned for the header (reserved; not yet emitted). Optional trailing extensions after ModuleEnd are backward-compatible only when old readers can ignore them without parsing the extension. Readers encountering an unknown minor version return an error rather than silently misinterpreting the stream.

ABI note: preprocessor dependency

The scalameta token sub-stream stores _post-preprocessor_ source. A .sscc v3 file therefore encodes the exact set of preprocessors active at write-time (including any plugin-provided ones). Reading the file with a different plugin set will still reconstruct a syntactically valid Scala 3 source, but the semantics may differ from the original .ssc text. This is intentional: .sscc is an opaque compiled artifact, not a lossless archive of the source.