v0.34.0 Phase 0 Unicode-scalar Design Review
Result: complete with no open design finding (2026-09-04). This review records the design boundary before implementation and does not claim that the candidate syntax or APIs are available.
Findings
The existing UTF-8 implementation already has a package-internal encoder and validator, but the validator owns a private traversal that cannot be reused by scalar iteration. Phase 2 will replace that traversal with a decoder-owned step rather than adding another Chars decoder.
The existing byte literal is less complete: syntax tokenizes and parses b'…', but checking has no ByteLiteral branch and would treat valid documented source as unsupported syntax. Character literal work cannot copy that pipeline. Syntax will publish typed decoded literal values for both forms, and checking will consume them exhaustively.
Using u32 as the runtime primitive identity for char would collide in the runtime primitive table and allow MIR validation to accept integer constants at character sites. The design therefore retains distinct character variants through checked constants, MIR constants, runtime primitives, and machine constants while sharing only the layout-owned 32-bit scalar shape.
Putting Unicode validity inside ARM64 conversion selection would make a target backend a second language authority. Public from_u32 validates in source, while exact package-internal conversion primitives transport already validated bits. Syntax validates literal scalars. No later phase rechecks authored escape text.
Responsibility Test
- syntax can change its lossless token representation without changing checking;
- checking can change constant storage without changing quote parsing;
- the standard UTF-8 decoder can change its algorithm without changing
Charsor validation policy; - machine layout can change target representation without changing semantic scalar validity;
- ARM64 can change instruction selection without learning about Unicode ranges;
- tooling can change presentation without interpreting literal contents.
These substitutions require only the exported contract of the replaced responsibility. No caller- discipline-only invariant or compatibility fallback is part of the accepted design.