String packs, phase 1: data-only packs that replace the wrapped site's own UI strings. Pack and inventory formats, validator (only inventory keys, parameter names, plural and select kinds, no markup), ordered idempotent engine that keeps what a restore needs, coverage and drift report; the per-site catalog hook adapter with a self-check (ok, hook not matched, no catalog seen), a fail-open webpack chunk array hook, a small DOM text net; waybetter strings check
This commit is contained in:
parent
787ebfbb2c
commit
db3fbc8ece
10 changed files with 1857 additions and 2 deletions
|
|
@ -111,6 +111,10 @@ packages/runtime/src/ page runtime core (TypeScript, no dependencies)
|
||||||
siteRuntime.ts the runtime's composition: the IPC bridge, flag-controlled parts with a deck
|
siteRuntime.ts the runtime's composition: the IPC bridge, flag-controlled parts with a deck
|
||||||
column's overrides, the runtime object under its window names, native queues
|
column's overrides, the runtime object under its window names, native queues
|
||||||
pageStrings.ts the page's language: English built in, native deliveries checked as data
|
pageStrings.ts the page's language: English built in, native deliveries checked as data
|
||||||
|
stringPacks.ts string packs (docs/STRING-PACKS.md): pack and inventory formats, validator,
|
||||||
|
ordered engine, coverage, drift report
|
||||||
|
stringPackHook.ts the per-site catalog hook adapter, its self-check, a webpack chunk array hook
|
||||||
|
textNet.ts DOM safety net for labels no catalog hook reaches
|
||||||
panelButton.ts the way into the panel from the page (navigation entry or floating button)
|
panelButton.ts the way into the panel from the page (navigation entry or floating button)
|
||||||
commandReport.ts the palette command report and developer-mode start errors
|
commandReport.ts the palette command report and developer-mode start errors
|
||||||
packages/shell/src/ shell core (TypeScript) and the shell frame (React)
|
packages/shell/src/ shell core (TypeScript) and the shell frame (React)
|
||||||
|
|
@ -474,6 +478,9 @@ The npm package adds the app-side tooling:
|
||||||
machine's account name, and prints hits masked (`m***`). `waybetter release` builds with
|
machine's account name, and prints hits masked (`m***`). `waybetter release` builds with
|
||||||
`build-env` and refuses to release an artifact with a hit or one it could not open. The `.pdb`
|
`build-env` and refuses to release an artifact with a hit or one it could not open. The `.pdb`
|
||||||
next to a Windows binary still holds every path: never publish it.
|
next to a Windows binary still holds every path: never publish it.
|
||||||
|
- `waybetter strings check <pack>`: a string pack (data that replaces the wrapped site's own UI
|
||||||
|
strings) against the site's string inventory: only inventory keys, parameter names, plural and
|
||||||
|
select kinds, no markup; prints the coverage. See docs/STRING-PACKS.md.
|
||||||
- `waybetter i18n check|extract`: an app's locale catalogs against the framework's rules (file
|
- `waybetter i18n check|extract`: an app's locale catalogs against the framework's rules (file
|
||||||
shape, keys English lacks, ICU syntax, placeholder parity, plural forms, no markup), the
|
shape, keys English lacks, ICU syntax, placeholder parity, plural forms, no markup), the
|
||||||
framework's catalogs in the same languages, the generated `languages.json` and native
|
framework's catalogs in the same languages, the generated `languages.json` and native
|
||||||
|
|
|
||||||
147
docs/STRING-PACKS.md
Normal file
147
docs/STRING-PACKS.md
Normal file
|
|
@ -0,0 +1,147 @@
|
||||||
|
# String packs
|
||||||
|
|
||||||
|
A string pack replaces the wrapped site's own UI strings with data: it adds a language the site
|
||||||
|
does not offer, fixes its translation, or rewords it. This is phase 1: the formats, the validator,
|
||||||
|
the application engine, the hook interface and `waybetter strings check`. The Store kind, the
|
||||||
|
install screen and the settings UI are not built yet.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
1. **Idempotent.** The result is a pure function of (the site's own values, the ordered pack list).
|
||||||
|
Applying twice equals applying once. Packs apply in the listed order; a later pack wins per key.
|
||||||
|
2. **The site's own language is the fallback.** A pack patches on top of whatever language the site
|
||||||
|
shows. A key no pack changes keeps exactly the site's value.
|
||||||
|
3. **An inventory of the site's known keys.** A pack can change only inventory keys. An unknown key
|
||||||
|
is reported and never applied.
|
||||||
|
4. **Packs are data.** Plain text in an ICU MessageFormat subset. Never HTML, never code.
|
||||||
|
|
||||||
|
## The pack
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"format": 1,
|
||||||
|
"site": "x",
|
||||||
|
"catalog": "web",
|
||||||
|
"language": "en-x-pirate",
|
||||||
|
"name": "Pirate",
|
||||||
|
"authors": ["Someone"],
|
||||||
|
"base": "en",
|
||||||
|
"strings": {
|
||||||
|
"a0000001": "Home port",
|
||||||
|
"b0000002": "{count, plural, one {# sighting} other {# sightings}}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `site` and `catalog` name the inventory the pack is for. One site can have several catalogs
|
||||||
|
(different builds of the site with different key spaces); each has its own inventory and packs.
|
||||||
|
- `language` is a BCP 47 tag; a private-use tag is fine. `base` is the language whose plural rules
|
||||||
|
the pack follows when the runtime does not know `language`; without either, every number takes
|
||||||
|
the `other` form.
|
||||||
|
- `strings` maps a key id to text. The ids are the inventory's; a pack never carries the site's own
|
||||||
|
text.
|
||||||
|
|
||||||
|
A value may use `{name}`, `{name, plural, one {…} other {…}}` (with `#` for the number and `=0`,
|
||||||
|
`=1` exact forms) and `{name, select, a {…} other {…}}`. A value is refused, and the site's text
|
||||||
|
stays, when:
|
||||||
|
|
||||||
|
- it names a parameter the original does not take;
|
||||||
|
- it pluralizes a parameter the original does not pluralize, or selects on one the original does
|
||||||
|
not select on (the original's plural parameters are the ones known to be numbers);
|
||||||
|
- a plural or select has no `other`;
|
||||||
|
- it uses a number or date format, `selectordinal` or a plural offset;
|
||||||
|
- it contains `<` or `>` (unless the inventory gives the key tag slots; then exactly those tags);
|
||||||
|
- it contains control or bidi-override characters, is empty or is longer than 2000 characters;
|
||||||
|
- the key's kind cannot be replaced by data yet (`rich`, `format`).
|
||||||
|
|
||||||
|
A parameter is inserted the way string concatenation would insert it (`String(value)`).
|
||||||
|
|
||||||
|
## The inventory
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"format": 1, "site": "x", "catalog": "web", "base": "en", "build": "…",
|
||||||
|
"entries": {
|
||||||
|
"a0000001": { "kind": "text", "hash": "0k3j…" },
|
||||||
|
"b0000002": { "kind": "message", "params": ["count"], "plural": ["count"], "hash": "…" },
|
||||||
|
"c0000003": { "kind": "rich", "group": "rich", "hash": "…" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Kind | The site's value | A pack can replace it |
|
||||||
|
|---|---|---|
|
||||||
|
| `text` | a string | yes |
|
||||||
|
| `message` | a function of parameters that returns a string | yes |
|
||||||
|
| `rich` | text with element slots | not yet |
|
||||||
|
| `format` | a compiled number, date or currency formatter | no |
|
||||||
|
|
||||||
|
`hash` is a hash of the base-language value (`hashText`), for the drift report; the inventory holds
|
||||||
|
no text of the site. `group` names the part of the catalog a key arrives with (default `main`).
|
||||||
|
|
||||||
|
`driftReport(inventory, observed, loadedGroups)` compares an inventory with a newer build: `added`,
|
||||||
|
`reworded` (the hash changed), `reshaped` (kind or parameters changed) and `vanished`. A key is
|
||||||
|
called vanished only when its group's base catalog was actually read.
|
||||||
|
|
||||||
|
`packInventory(inventory)` is the form a page runtime ships: ids and shapes, no hashes.
|
||||||
|
|
||||||
|
## The engine
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const engine = createStringEngine(inventory, [compilePack(packA, inventory), compilePack(packB, inventory)]);
|
||||||
|
engine.resolve(id, siteValue); // packB's value, else packA's, else exactly siteValue
|
||||||
|
```
|
||||||
|
|
||||||
|
`resolve` returns a string for a `text` key and a function with the original's call shape for a
|
||||||
|
`message` key (closures over the parsed message; nothing is evaluated). Feeding a value the engine
|
||||||
|
produced back in returns the same value. It keeps the site's own value of every key it changed
|
||||||
|
(`engine.originals()`), which is what a restore needs; `applyPacks` and `restoreCatalog` do both
|
||||||
|
for a whole catalog object without changing it. A live value whose kind differs from the
|
||||||
|
inventory's is left alone and listed in `engine.stats().mismatched`; a key the site has and the
|
||||||
|
inventory lacks is listed in `unlisted`.
|
||||||
|
|
||||||
|
## Hooking a site
|
||||||
|
|
||||||
|
How a site hands its strings to its own code is that site's implementation detail, so the hook is
|
||||||
|
an adapter the app supplies:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const adapter: CatalogHookAdapter = {
|
||||||
|
catalog: 'web',
|
||||||
|
install(win, sink) {
|
||||||
|
// At document start: arrange for the site's catalog to pass through
|
||||||
|
// sink.seen(group) a catalog file is loading
|
||||||
|
// sink.matched(language, group) the hook matched
|
||||||
|
// sink.resolve(id, original) the value to register
|
||||||
|
return () => { /* remove the hook */ };
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const handle = installStringPacks(window, [{ adapter, inventory, packs }], { onStatus });
|
||||||
|
```
|
||||||
|
|
||||||
|
- The adapter must not change the site's objects: hand the site a shadow
|
||||||
|
(`Object.create(module)`) instead.
|
||||||
|
- Everything fails open: an exception in a wrapper leaves the site's behaviour as it was.
|
||||||
|
- A catalog without a usable pack is not hooked at all.
|
||||||
|
- The self-check decides after `settleMs`: `ok`, `not-matched` (a catalog file was seen but the
|
||||||
|
hook did not match: the site changed) or `no-catalog` (no catalog file was seen).
|
||||||
|
|
||||||
|
For a webpack site, `watchChunkArray(win, name, onChunk)` sees every chunk before the site's own
|
||||||
|
`push` gets it and passes it on unchanged; `mapChunkRequires(chunk, map)` lets one identified
|
||||||
|
chunk's modules receive a shadow of what they `require`.
|
||||||
|
|
||||||
|
`installTextNet(win, { map })` is a DOM safety net for labels no catalog hook can reach (the server
|
||||||
|
sends them already worded): exact text in text nodes, `aria-label`, `placeholder` and `title`, the
|
||||||
|
site's value kept per node, restored on `dispose()`. It is secondary: it cannot see parameters or
|
||||||
|
plural forms.
|
||||||
|
|
||||||
|
## Checking a pack
|
||||||
|
|
||||||
|
```sh
|
||||||
|
waybetter strings check pack.strings.json --inventory strings/x.web.inventory.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Prints what was refused and why, the unknown keys, and the coverage per group (translated of
|
||||||
|
translatable, and how many keys are not translatable yet). Exit 1 when anything would not be
|
||||||
|
applied. Without `--inventory`, the inventories come from `waybetter.strings.json`
|
||||||
|
(`{ "inventories": ["…"] }`) in the app's root.
|
||||||
273
packages/runtime/src/stringPackHook.ts
Normal file
273
packages/runtime/src/stringPackHook.ts
Normal file
|
|
@ -0,0 +1,273 @@
|
||||||
|
// String packs in the page (published as `waybetter/runtime/stringPackHook`): the per-site
|
||||||
|
// "catalog hook adapter" interface, the installer that connects adapters to the engine of
|
||||||
|
// stringPacks.ts, a self-check that says when a hook did not match, and the generic half of a
|
||||||
|
// common hook point (a webpack chunk array).
|
||||||
|
//
|
||||||
|
// How a site hands its strings to its own code is that site's implementation detail, so the hook
|
||||||
|
// is an adapter the app supplies. One site can have several catalogs (different builds of the
|
||||||
|
// site, different key spaces); each gets its own adapter, inventory and packs.
|
||||||
|
//
|
||||||
|
// Everything here fails open: an exception inside a wrapper leaves the site's own behaviour as it
|
||||||
|
// was, and with no pack installed nothing is hooked at all.
|
||||||
|
import {
|
||||||
|
compilePack, createStringEngine, type CompiledPack, type EngineStats, type PackReport, type PageInventory, type StringEngine,
|
||||||
|
type StringInventory,
|
||||||
|
} from './stringPacks.ts';
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// The adapter interface
|
||||||
|
|
||||||
|
/** What an adapter reports to and asks from the framework. None of these throw. */
|
||||||
|
export type CatalogSink = {
|
||||||
|
/** A catalog file of this adapter was seen loading (whether or not the hook then matches). */
|
||||||
|
seen(group?: string): void;
|
||||||
|
/** The hook matched: the site is about to register values in `language`. */
|
||||||
|
matched(language: string, group?: string): void;
|
||||||
|
/** The value to register for `id`: a pack's, or exactly `original`. */
|
||||||
|
resolve(id: string, original: unknown): unknown;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type CatalogHookAdapter = {
|
||||||
|
/** The catalog (key space) this adapter hooks; packs and the inventory name the same one. */
|
||||||
|
catalog: string;
|
||||||
|
/** Installs the hook at document start; returns its removal. Must not change the site's objects. */
|
||||||
|
install(win: Window, sink: CatalogSink): () => void;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type CatalogSetup = {
|
||||||
|
adapter: CatalogHookAdapter;
|
||||||
|
inventory: StringInventory | PageInventory;
|
||||||
|
/** Pack documents in application order (a later one wins). Untrusted data: validated here. */
|
||||||
|
packs: readonly unknown[];
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `ok`: the hook matched and keys went through it.
|
||||||
|
* `not-matched`: a catalog file was seen but the hook did not match it (the site changed).
|
||||||
|
* `no-catalog`: no catalog file was seen at all (another build of the site, or not loaded yet).
|
||||||
|
*/
|
||||||
|
export type HookStatus = 'pending' | 'ok' | 'not-matched' | 'no-catalog';
|
||||||
|
|
||||||
|
export type CatalogReport = {
|
||||||
|
catalog: string;
|
||||||
|
status: HookStatus;
|
||||||
|
/** Languages the site registered through the hook. */
|
||||||
|
languages: string[];
|
||||||
|
/** Catalog files seen, per group. */
|
||||||
|
seen: Record<string, number>;
|
||||||
|
packs: { name: string; language: string; report: PackReport }[];
|
||||||
|
/** Pack documents that were refused as a whole. */
|
||||||
|
refused: string[];
|
||||||
|
engine: EngineStats;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type StringPacksHandle = {
|
||||||
|
status(catalog?: string): HookStatus;
|
||||||
|
report(): CatalogReport[];
|
||||||
|
/** The engines, per catalog (the originals they keep are what a restore needs). */
|
||||||
|
engine(catalog: string): StringEngine | undefined;
|
||||||
|
dispose(): void;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type StringPacksOptions = {
|
||||||
|
/** When the self-check decides (default 15 s after install). */
|
||||||
|
settleMs?: number;
|
||||||
|
/** Called once per catalog when its status is decided (`ok` as soon as it is reached). */
|
||||||
|
onStatus?: (report: CatalogReport) => void;
|
||||||
|
};
|
||||||
|
|
||||||
|
type TimerWindow = { setTimeout(handler: () => void, ms: number): unknown; clearTimeout(id: never): void };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Installs the adapters that have at least one usable pack. A catalog without packs is not hooked
|
||||||
|
* (the site runs untouched), so its status stays `pending`.
|
||||||
|
*/
|
||||||
|
export function installStringPacks(win: Window, setups: readonly CatalogSetup[], options: StringPacksOptions = {}): StringPacksHandle {
|
||||||
|
const timers = win as unknown as TimerWindow;
|
||||||
|
const states = setups.map(setup => {
|
||||||
|
const packs: CompiledPack[] = [];
|
||||||
|
const refused: string[] = [];
|
||||||
|
for (const document of setup.packs) {
|
||||||
|
try {
|
||||||
|
packs.push(compilePack(document, setup.inventory));
|
||||||
|
} catch (error) {
|
||||||
|
refused.push(error instanceof Error ? error.message : String(error));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const usable = packs.filter(pack => pack.values.size > 0);
|
||||||
|
return {
|
||||||
|
catalog: setup.adapter.catalog, adapter: setup.adapter, packs, refused, usable,
|
||||||
|
engine: usable.length ? createStringEngine(setup.inventory, usable) : null,
|
||||||
|
seen: {} as Record<string, number>, languages: new Set<string>(), keys: 0, settled: false, told: false,
|
||||||
|
remove: null as null | (() => void),
|
||||||
|
};
|
||||||
|
});
|
||||||
|
type State = (typeof states)[number];
|
||||||
|
|
||||||
|
const status = (state: State): HookStatus => {
|
||||||
|
if (state.languages.size && state.keys > 0) return 'ok';
|
||||||
|
if (!state.settled) return 'pending';
|
||||||
|
return Object.keys(state.seen).length ? 'not-matched' : 'no-catalog';
|
||||||
|
};
|
||||||
|
const reportOf = (state: State): CatalogReport => ({
|
||||||
|
catalog: state.catalog,
|
||||||
|
status: status(state),
|
||||||
|
languages: [...state.languages],
|
||||||
|
seen: { ...state.seen },
|
||||||
|
packs: state.packs.map(pack => ({ name: pack.name, language: pack.language, report: pack.report })),
|
||||||
|
refused: [...state.refused],
|
||||||
|
engine: state.engine?.stats() ?? { seen: 0, applied: 0, unlisted: [], mismatched: [] },
|
||||||
|
});
|
||||||
|
const tell = (state: State) => {
|
||||||
|
if (state.told || status(state) === 'pending') return;
|
||||||
|
state.told = true;
|
||||||
|
try { options.onStatus?.(reportOf(state)); } catch { /* a reporter must not break the page */ }
|
||||||
|
};
|
||||||
|
|
||||||
|
for (const state of states) {
|
||||||
|
const engine = state.engine;
|
||||||
|
if (!engine) continue;
|
||||||
|
const sink: CatalogSink = {
|
||||||
|
seen(group = 'main') { state.seen[group] = (state.seen[group] ?? 0) + 1; },
|
||||||
|
matched(language) { state.languages.add(String(language)); },
|
||||||
|
resolve(id, original) {
|
||||||
|
try {
|
||||||
|
state.keys += 1;
|
||||||
|
const value = engine.resolve(id, original);
|
||||||
|
if (state.keys === 1) tell(state);
|
||||||
|
return value;
|
||||||
|
} catch {
|
||||||
|
return original;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
state.remove = state.adapter.install(win, sink);
|
||||||
|
} catch {
|
||||||
|
state.remove = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const hooked = states.filter(state => state.engine);
|
||||||
|
const timer = hooked.length
|
||||||
|
? timers.setTimeout(() => { for (const state of hooked) { state.settled = true; tell(state); } }, options.settleMs ?? 15000)
|
||||||
|
: null;
|
||||||
|
|
||||||
|
return {
|
||||||
|
status: catalog => {
|
||||||
|
const state = catalog ? states.find(item => item.catalog === catalog) : hooked[0] ?? states[0];
|
||||||
|
return state ? status(state) : 'pending';
|
||||||
|
},
|
||||||
|
report: () => states.map(reportOf),
|
||||||
|
engine: catalog => states.find(item => item.catalog === catalog)?.engine ?? undefined,
|
||||||
|
dispose() {
|
||||||
|
if (timer !== null) timers.clearTimeout(timer as never);
|
||||||
|
for (const state of states) {
|
||||||
|
try { state.remove?.(); } catch { /* the hook is gone either way */ }
|
||||||
|
state.remove = null;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// A webpack chunk array as the hook point
|
||||||
|
//
|
||||||
|
// A webpack site loads its code as chunks pushed onto a global array
|
||||||
|
// (`(globalThis.webpackChunk_x ||= []).push([ids, modules, runtime?])`); webpack's runtime later
|
||||||
|
// replaces the array's `push` with its own and replays what is already there. Watching the array
|
||||||
|
// therefore sees every chunk before its modules are installed, whichever loads first.
|
||||||
|
|
||||||
|
type Chunk = [unknown, Record<string, unknown> | unknown[], ...unknown[]];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Calls `onChunk(chunk, script)` for every chunk pushed onto `win[name]`, before the site's own
|
||||||
|
* `push` gets it; `script` is the URL of the script doing the push (`''` when unknown). Install at
|
||||||
|
* document start. The array and webpack's own `push` replacement keep working unchanged: the
|
||||||
|
* wrapper calls exactly the `push` that was current when it was read, with the same arguments. An
|
||||||
|
* exception in `onChunk` is swallowed. Returns the removal.
|
||||||
|
*/
|
||||||
|
export function watchChunkArray(win: object, name: string, onChunk: (chunk: Chunk, script: string) => void): () => void {
|
||||||
|
const holder = win as Record<string, unknown> & { document?: { currentScript?: { src?: string } | null } };
|
||||||
|
const existing = holder[name];
|
||||||
|
const array: unknown[] = Array.isArray(existing) ? existing : [];
|
||||||
|
if (!Array.isArray(existing)) holder[name] = array;
|
||||||
|
const done = new WeakSet<object>();
|
||||||
|
let current: (...items: unknown[]) => number = array.push;
|
||||||
|
let active = true;
|
||||||
|
const look = (items: ArrayLike<unknown>) => {
|
||||||
|
if (!active) return;
|
||||||
|
for (let index = 0; index < items.length; index += 1) {
|
||||||
|
const chunk = items[index];
|
||||||
|
if (typeof chunk !== 'object' || chunk === null || done.has(chunk)) continue;
|
||||||
|
done.add(chunk);
|
||||||
|
try {
|
||||||
|
onChunk(chunk as Chunk, holder.document?.currentScript?.src ?? '');
|
||||||
|
} catch {
|
||||||
|
// Fail open: the chunk goes on to the site exactly as it came.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
Object.defineProperty(array, 'push', {
|
||||||
|
configurable: true,
|
||||||
|
enumerable: false,
|
||||||
|
get() {
|
||||||
|
const target = current;
|
||||||
|
return function push(this: unknown, ...items: unknown[]) {
|
||||||
|
look(items);
|
||||||
|
return target.apply(this, items);
|
||||||
|
};
|
||||||
|
},
|
||||||
|
set(next: (...items: unknown[]) => number) { current = next; },
|
||||||
|
});
|
||||||
|
} catch {
|
||||||
|
return () => {};
|
||||||
|
}
|
||||||
|
return () => {
|
||||||
|
active = false;
|
||||||
|
try {
|
||||||
|
if (current === Array.prototype.push) delete (array as { push?: unknown }).push;
|
||||||
|
else Object.defineProperty(array, 'push', { configurable: true, enumerable: false, writable: true, value: current });
|
||||||
|
} catch {
|
||||||
|
// Left as the pass-through wrapper.
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Wraps the module factories of one chunk so that what they `require` passes through `map`:
|
||||||
|
* `map(exports)` returns a replacement for that one call, or `undefined` to hand out the module's
|
||||||
|
* own exports. The module itself is never changed (return a shadow, e.g. `Object.create(exports)`).
|
||||||
|
* Only call this for a chunk you have identified; other chunks must stay untouched.
|
||||||
|
*/
|
||||||
|
export function mapChunkRequires(chunk: Chunk, map: (exports: unknown) => unknown): void {
|
||||||
|
const modules = chunk[1];
|
||||||
|
if (typeof modules !== 'object' || modules === null) return;
|
||||||
|
const table = modules as Record<string, unknown>;
|
||||||
|
for (const id of Object.keys(table)) {
|
||||||
|
const factory = table[id];
|
||||||
|
if (typeof factory !== 'function') continue;
|
||||||
|
table[id] = function (this: unknown, module: unknown, exports: unknown, require: unknown, ...rest: unknown[]) {
|
||||||
|
let through = require;
|
||||||
|
if (typeof require === 'function') {
|
||||||
|
try {
|
||||||
|
through = new Proxy(require, {
|
||||||
|
apply(target, self, args) {
|
||||||
|
const result = Reflect.apply(target as (...a: unknown[]) => unknown, self, args);
|
||||||
|
try {
|
||||||
|
const replaced = map(result);
|
||||||
|
return replaced === undefined ? result : replaced;
|
||||||
|
} catch {
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
});
|
||||||
|
} catch {
|
||||||
|
through = require;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return (factory as (...a: unknown[]) => unknown).call(this, module, exports, through, ...rest);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
548
packages/runtime/src/stringPacks.ts
Normal file
548
packages/runtime/src/stringPacks.ts
Normal file
|
|
@ -0,0 +1,548 @@
|
||||||
|
// String packs: data-only packs that replace the wrapped site's own UI strings (published as
|
||||||
|
// `waybetter/runtime/stringPacks`). A pack adds a language the site does not offer, fixes its
|
||||||
|
// translation or rewords it. This file is the data side: the pack and inventory formats, the
|
||||||
|
// validator, the ordered application engine, coverage and the drift report. It touches no DOM and
|
||||||
|
// runs the same in the page, in the shell and in Node (`waybetter strings check`).
|
||||||
|
//
|
||||||
|
// The rules (they are the owner's, and every function here keeps them):
|
||||||
|
// - Idempotent. The result is a pure function of (the site's own values, the ordered pack list):
|
||||||
|
// applying twice equals applying once, and a later pack wins over an earlier one per key.
|
||||||
|
// - The site's own language is the fallback. A key no pack changes keeps exactly the site's value
|
||||||
|
// (the same object, not a copy).
|
||||||
|
// - An inventory of the site's known keys. A pack may change only inventory keys, and only keys
|
||||||
|
// whose kind can be replaced by data; unknown keys are reported, never applied.
|
||||||
|
// - Packs are data: plain text in an ICU MessageFormat subset ({param}, plural, select). Never
|
||||||
|
// HTML, never code. A message is interpreted by closures here; nothing is evaluated.
|
||||||
|
//
|
||||||
|
// One site can have several catalogs (each with its own key space, inventory and hook adapter,
|
||||||
|
// see stringPackHook.ts); a pack names the one it is for.
|
||||||
|
import { parseMessage } from './i18n.ts';
|
||||||
|
|
||||||
|
type Parts = ReturnType<typeof parseMessage>;
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Formats
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What the site's value for a key is:
|
||||||
|
* - `text`: a plain string;
|
||||||
|
* - `message`: a function of a parameter object that returns a string;
|
||||||
|
* - `rich`: text with element slots (not replaceable by data yet);
|
||||||
|
* - `format`: a compiled number/date/currency formatter (not text).
|
||||||
|
* Only `text` and `message` can be replaced by a pack.
|
||||||
|
*/
|
||||||
|
export type StringKind = 'text' | 'message' | 'rich' | 'format';
|
||||||
|
|
||||||
|
export type InventoryEntry = {
|
||||||
|
kind: StringKind;
|
||||||
|
/** Parameter names the site's value takes (`message`). */
|
||||||
|
params?: string[];
|
||||||
|
/** Parameters the site's value pluralizes: known to be numbers, so a pack may pluralize them. */
|
||||||
|
plural?: string[];
|
||||||
|
/** Parameters the site's value selects on. */
|
||||||
|
select?: string[];
|
||||||
|
/** Tag slot names, for catalogs whose values carry `<tag>…</tag>` slots. */
|
||||||
|
tags?: string[];
|
||||||
|
/** Hash of the base-language value (`hashText`): drift detection; never the text itself. */
|
||||||
|
hash: string;
|
||||||
|
/** Part of the catalog the key arrives with (default `main`), e.g. a lazily loaded `emoji` group. */
|
||||||
|
group?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** The known keys of one catalog of one site. It carries ids and shapes, never the site's text. */
|
||||||
|
export type StringInventory = {
|
||||||
|
format: 1;
|
||||||
|
site: string;
|
||||||
|
catalog: string;
|
||||||
|
/** Language the hashes were taken from. */
|
||||||
|
base: string;
|
||||||
|
/** The site build the inventory was read from. */
|
||||||
|
build?: string;
|
||||||
|
entries: Record<string, InventoryEntry>;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** A string pack document: metadata and `key id -> text`. */
|
||||||
|
export type StringPack = {
|
||||||
|
format: 1;
|
||||||
|
site: string;
|
||||||
|
catalog: string;
|
||||||
|
/** BCP 47 tag of the pack's language (private-use tags are fine: `en-x-pirate`). */
|
||||||
|
language: string;
|
||||||
|
name: string;
|
||||||
|
authors: string[];
|
||||||
|
/** A language whose plural rules the pack follows when the runtime does not know `language`. */
|
||||||
|
base?: string;
|
||||||
|
version?: string;
|
||||||
|
description?: string;
|
||||||
|
strings: Record<string, string>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export const STRING_PACK_LIMITS = Object.freeze({ strings: 40000, value: 2000, name: 80, authors: 16 });
|
||||||
|
export const MAIN_GROUP = 'main';
|
||||||
|
|
||||||
|
const TAG = /^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$/;
|
||||||
|
const IDENT = /^[a-z][a-z0-9-]{0,31}$/;
|
||||||
|
const CONTROL = /[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f--]/;
|
||||||
|
const CATEGORIES = new Set(['zero', 'one', 'two', 'few', 'many', 'other']);
|
||||||
|
|
||||||
|
/** Whether a pack can replace a value of this kind. */
|
||||||
|
export function translatable(kind: StringKind): boolean {
|
||||||
|
return kind === 'text' || kind === 'message';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 53-bit hash of a text as 11 base-36 characters. Same result in every runtime. */
|
||||||
|
export function hashText(text: string): string {
|
||||||
|
let h1 = 0xdeadbeef;
|
||||||
|
let h2 = 0x41c6ce57;
|
||||||
|
for (let i = 0; i < text.length; i += 1) {
|
||||||
|
const ch = text.charCodeAt(i);
|
||||||
|
h1 = Math.imul(h1 ^ ch, 2654435761);
|
||||||
|
h2 = Math.imul(h2 ^ ch, 1597334677);
|
||||||
|
}
|
||||||
|
h1 = Math.imul(h1 ^ (h1 >>> 16), 2246822507) ^ Math.imul(h2 ^ (h2 >>> 13), 3266489909);
|
||||||
|
h2 = Math.imul(h2 ^ (h2 >>> 16), 2246822507) ^ Math.imul(h1 ^ (h1 >>> 13), 3266489909);
|
||||||
|
return (4294967296 * (2097151 & h2) + (h1 >>> 0)).toString(36).padStart(11, '0');
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Inventory lookup (the full inventory, or its compact page form)
|
||||||
|
|
||||||
|
/** The facts the validator and the engine need about one key. */
|
||||||
|
export type EntryShape = Pick<InventoryEntry, 'kind' | 'params' | 'plural' | 'select' | 'tags'>;
|
||||||
|
export type InventoryLookup = (id: string) => EntryShape | undefined;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The inventory as it ships inside a page runtime: ids and shapes only, no hashes, a fraction of
|
||||||
|
* the size. `text` and `closed` are space-separated ids; a message is `name name*` (`*` = plural,
|
||||||
|
* `?` = select).
|
||||||
|
*/
|
||||||
|
export type PageInventory = { site: string; catalog: string; text: string; message: Record<string, string>; closed: string };
|
||||||
|
|
||||||
|
export function packInventory(inventory: StringInventory): PageInventory {
|
||||||
|
const text: string[] = [];
|
||||||
|
const closed: string[] = [];
|
||||||
|
const message: Record<string, string> = {};
|
||||||
|
for (const [id, entry] of Object.entries(inventory.entries)) {
|
||||||
|
if (entry.kind === 'text' && !entry.tags?.length) text.push(id);
|
||||||
|
else if (entry.kind === 'message' && !entry.tags?.length) {
|
||||||
|
message[id] = (entry.params ?? []).map(name => `${name}${entry.plural?.includes(name) ? '*' : ''}${entry.select?.includes(name) ? '?' : ''}`).join(' ');
|
||||||
|
} else closed.push(id);
|
||||||
|
}
|
||||||
|
return { site: inventory.site, catalog: inventory.catalog, text: text.join(' '), message, closed: closed.join(' ') };
|
||||||
|
}
|
||||||
|
|
||||||
|
const TEXT_SHAPE: EntryShape = Object.freeze({ kind: 'text' });
|
||||||
|
const CLOSED_SHAPE: EntryShape = Object.freeze({ kind: 'rich' });
|
||||||
|
|
||||||
|
/** A lookup over either form. The page form is decoded on first use. */
|
||||||
|
export function inventoryLookup(inventory: StringInventory | PageInventory): InventoryLookup {
|
||||||
|
if ('entries' in inventory) {
|
||||||
|
const entries = inventory.entries;
|
||||||
|
return id => (Object.prototype.hasOwnProperty.call(entries, id) ? entries[id] : undefined);
|
||||||
|
}
|
||||||
|
let table: Map<string, EntryShape> | null = null;
|
||||||
|
const decode = () => {
|
||||||
|
const map = new Map<string, EntryShape>();
|
||||||
|
if (inventory.text) for (const id of inventory.text.split(' ')) map.set(id, TEXT_SHAPE);
|
||||||
|
if (inventory.closed) for (const id of inventory.closed.split(' ')) map.set(id, CLOSED_SHAPE);
|
||||||
|
for (const [id, spec] of Object.entries(inventory.message)) {
|
||||||
|
const names = spec ? spec.split(' ') : [];
|
||||||
|
map.set(id, {
|
||||||
|
kind: 'message',
|
||||||
|
params: names.map(name => name.replace(/[*?]+$/, '')),
|
||||||
|
plural: names.filter(name => name.includes('*')).map(name => name.replace(/[*?]+$/, '')),
|
||||||
|
select: names.filter(name => name.includes('?')).map(name => name.replace(/[*?]+$/, '')),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return map;
|
||||||
|
};
|
||||||
|
return id => (table ??= decode()).get(id);
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Validator
|
||||||
|
|
||||||
|
export type PackIssue = { level: 'error' | 'warning'; key?: string; message: string };
|
||||||
|
|
||||||
|
const isRecord = (value: unknown): value is Record<string, unknown> => typeof value === 'object' && value !== null && !Array.isArray(value);
|
||||||
|
|
||||||
|
/** Checks the document envelope. Errors mean the pack is not applied at all. */
|
||||||
|
export function checkPackShape(input: unknown): PackIssue[] {
|
||||||
|
const issues: PackIssue[] = [];
|
||||||
|
const error = (message: string) => { issues.push({ level: 'error', message }); };
|
||||||
|
if (!isRecord(input)) return [{ level: 'error', message: 'a string pack is a JSON object' }];
|
||||||
|
if (input.format !== 1) error('"format" must be 1');
|
||||||
|
for (const field of ['site', 'catalog'] as const) {
|
||||||
|
if (typeof input[field] !== 'string' || !IDENT.test(input[field])) error(`"${field}" must be a short lowercase id`);
|
||||||
|
}
|
||||||
|
if (typeof input.language !== 'string' || !TAG.test(input.language)) error('"language" must be a language tag (en, pt-BR, en-x-pirate)');
|
||||||
|
if (input.base !== undefined && (typeof input.base !== 'string' || !TAG.test(input.base))) error('"base" must be a language tag');
|
||||||
|
if (typeof input.name !== 'string' || !input.name.trim() || input.name.length > STRING_PACK_LIMITS.name || CONTROL.test(input.name) || /[<>]/.test(input.name)) {
|
||||||
|
error(`"name" must be 1 to ${STRING_PACK_LIMITS.name} characters of plain text`);
|
||||||
|
}
|
||||||
|
if (!Array.isArray(input.authors) || input.authors.length > STRING_PACK_LIMITS.authors
|
||||||
|
|| input.authors.some(author => typeof author !== 'string' || !author.trim() || author.length > STRING_PACK_LIMITS.name || CONTROL.test(author) || /[<>]/.test(author))) {
|
||||||
|
error(`"authors" must be a list of at most ${STRING_PACK_LIMITS.authors} plain-text names`);
|
||||||
|
}
|
||||||
|
for (const field of ['version', 'description'] as const) {
|
||||||
|
const value = input[field];
|
||||||
|
if (value !== undefined && (typeof value !== 'string' || value.length > 400 || CONTROL.test(value) || /[<>]/.test(value))) error(`"${field}" must be plain text`);
|
||||||
|
}
|
||||||
|
if (!isRecord(input.strings)) error('"strings" must be an object of key id -> text');
|
||||||
|
else if (Object.keys(input.strings).length > STRING_PACK_LIMITS.strings) error(`"strings" has more than ${STRING_PACK_LIMITS.strings} entries`);
|
||||||
|
const known = new Set(['$schema', 'format', 'site', 'catalog', 'language', 'base', 'name', 'authors', 'version', 'description', 'strings']);
|
||||||
|
for (const key of Object.keys(input)) if (!known.has(key)) issues.push({ level: 'warning', message: `unknown field "${key}" is ignored` });
|
||||||
|
return issues;
|
||||||
|
}
|
||||||
|
|
||||||
|
const TAG_TOKEN = /<\/?([A-Za-z][A-Za-z0-9-]*)>/g;
|
||||||
|
|
||||||
|
/** Tag rule: only the original's tag slots, each opened and closed; any other `<` or `>` is refused. */
|
||||||
|
function checkTags(value: string, tags: readonly string[]): string | null {
|
||||||
|
if (!value.includes('<') && !value.includes('>')) {
|
||||||
|
return tags.length ? `the original's tags are missing: ${tags.map(tag => `<${tag}>`).join(' ')}` : null;
|
||||||
|
}
|
||||||
|
if (!tags.length) return 'markup is not allowed: a pack is text, < and > cannot be used';
|
||||||
|
const open: string[] = [];
|
||||||
|
const seen = new Set<string>();
|
||||||
|
for (const match of value.matchAll(TAG_TOKEN)) {
|
||||||
|
const name = match[1]!;
|
||||||
|
if (!tags.includes(name)) return `<${name}> is not a tag of the original`;
|
||||||
|
if (match[0].startsWith('</')) {
|
||||||
|
if (open.pop() !== name) return `</${name}> does not close an open tag`;
|
||||||
|
} else {
|
||||||
|
open.push(name);
|
||||||
|
seen.add(name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (open.length) return `<${open[open.length - 1]}> is not closed`;
|
||||||
|
if (/[<>]/.test(value.replace(TAG_TOKEN, ''))) return '< and > may only be used for the original\'s tags';
|
||||||
|
const missing = tags.filter(tag => !seen.has(tag));
|
||||||
|
return missing.length ? `the original's tags are missing: ${missing.map(tag => `<${tag}>`).join(' ')}` : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Walks a parsed message against the key's shape; returns the first problem. */
|
||||||
|
function checkParts(parts: Parts, shape: EntryShape): string | null {
|
||||||
|
const params = shape.params ?? [];
|
||||||
|
for (const part of parts) {
|
||||||
|
if (typeof part === 'string' || part.kind === 'pound') continue;
|
||||||
|
if (!params.includes(part.name)) {
|
||||||
|
return params.length ? `{${part.name}} is not a parameter of the original (it has: ${params.join(', ')})` : `{${part.name}}: the original takes no parameters`;
|
||||||
|
}
|
||||||
|
if (part.kind === 'arg') {
|
||||||
|
if (part.format) return `{${part.name}, ${part.format}} is not supported: only {name}, plural and select`;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (part.kind === 'plural') {
|
||||||
|
if (part.ordinal) return `{${part.name}, selectordinal} is not supported: only {name}, plural and select`;
|
||||||
|
if (part.offset) return `{${part.name}, plural, offset:…} is not supported`;
|
||||||
|
if (!shape.plural?.includes(part.name)) return `{${part.name}} is not a plural of the original`;
|
||||||
|
for (const selector of Object.keys(part.options)) {
|
||||||
|
if (!CATEGORIES.has(selector) && !/^=\d+$/.test(selector)) return `"${selector}" is not a plural form (zero, one, two, few, many, other, =N)`;
|
||||||
|
}
|
||||||
|
} else if (!shape.select?.includes(part.name)) {
|
||||||
|
return `{${part.name}} is not a select of the original`;
|
||||||
|
}
|
||||||
|
for (const option of Object.values(part.options)) {
|
||||||
|
const nested = checkParts(option, shape);
|
||||||
|
if (nested) return nested;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Validates one pack value against the key's inventory entry. Returns the parsed message, or the
|
||||||
|
* reason it is refused (the original then stays).
|
||||||
|
*/
|
||||||
|
export function validateString(shape: EntryShape, value: unknown): { parts: Parts } | { reason: string } {
|
||||||
|
if (!translatable(shape.kind)) return { reason: `not translatable yet (${shape.kind})` };
|
||||||
|
if (typeof value !== 'string') return { reason: 'the value must be text' };
|
||||||
|
if (!value) return { reason: 'the value is empty' };
|
||||||
|
if (value.length > STRING_PACK_LIMITS.value) return { reason: `longer than ${STRING_PACK_LIMITS.value} characters` };
|
||||||
|
if (CONTROL.test(value)) return { reason: 'control or bidi-override characters are not allowed' };
|
||||||
|
const tags = checkTags(value, shape.tags ?? []);
|
||||||
|
if (tags) return { reason: tags };
|
||||||
|
let parts: Parts;
|
||||||
|
try {
|
||||||
|
parts = parseMessage(value);
|
||||||
|
} catch (error) {
|
||||||
|
return { reason: `invalid message syntax: ${error instanceof Error ? error.message : String(error)}` };
|
||||||
|
}
|
||||||
|
const problem = checkParts(parts, shape.kind === 'message' ? shape : { kind: 'text' });
|
||||||
|
return problem ? { reason: problem } : { parts };
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Compiling a pack (validation against an inventory)
|
||||||
|
|
||||||
|
export type CompiledValue = { text: string } | { parts: Parts };
|
||||||
|
|
||||||
|
export type PackReport = {
|
||||||
|
/** Entries in the pack. */
|
||||||
|
total: number;
|
||||||
|
/** Entries that are valid for an inventory key. */
|
||||||
|
valid: number;
|
||||||
|
/** Ids the inventory does not have: never applied. */
|
||||||
|
unknown: string[];
|
||||||
|
/** Values that failed validation: not applied, the original stays. */
|
||||||
|
rejected: { id: string; reason: string }[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type CompiledPack = {
|
||||||
|
name: string;
|
||||||
|
language: string;
|
||||||
|
base?: string;
|
||||||
|
site: string;
|
||||||
|
catalog: string;
|
||||||
|
values: ReadonlyMap<string, CompiledValue>;
|
||||||
|
report: PackReport;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** A message without arguments as its literal text (quotes resolved). */
|
||||||
|
function literal(parts: Parts): string {
|
||||||
|
return parts.map(part => (typeof part === 'string' ? part : '')).join('');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Validates a pack against an inventory. Throws on a malformed envelope or a pack for another
|
||||||
|
* site/catalog; every other problem is per key and lands in `report`.
|
||||||
|
*/
|
||||||
|
export function compilePack(input: unknown, inventory: StringInventory | PageInventory): CompiledPack {
|
||||||
|
const shapeErrors = checkPackShape(input).filter(issue => issue.level === 'error');
|
||||||
|
if (shapeErrors.length) throw new Error(`invalid string pack: ${shapeErrors.map(issue => issue.message).join('; ')}`);
|
||||||
|
const pack = input as StringPack;
|
||||||
|
if (pack.site !== inventory.site || pack.catalog !== inventory.catalog) {
|
||||||
|
throw new Error(`the pack is for ${pack.site}/${pack.catalog}, the inventory is ${inventory.site}/${inventory.catalog}`);
|
||||||
|
}
|
||||||
|
const lookup = inventoryLookup(inventory);
|
||||||
|
const values = new Map<string, CompiledValue>();
|
||||||
|
const report: PackReport = { total: 0, valid: 0, unknown: [], rejected: [] };
|
||||||
|
for (const [id, value] of Object.entries(pack.strings)) {
|
||||||
|
report.total += 1;
|
||||||
|
const shape = lookup(id);
|
||||||
|
if (!shape) { report.unknown.push(id); continue; }
|
||||||
|
const result = validateString(shape, value);
|
||||||
|
if ('reason' in result) { report.rejected.push({ id, reason: result.reason }); continue; }
|
||||||
|
values.set(id, shape.kind === 'text' ? { text: literal(result.parts) } : { parts: result.parts });
|
||||||
|
report.valid += 1;
|
||||||
|
}
|
||||||
|
return { name: pack.name, language: pack.language, base: pack.base, site: pack.site, catalog: pack.catalog, values, report };
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Interpreting a message (closures over the parsed parts; nothing is evaluated)
|
||||||
|
|
||||||
|
const rulesCache = new Map<string, Intl.PluralRules | null>();
|
||||||
|
|
||||||
|
/** Plural rules of the first language the runtime knows; `null` = every number is `other`. */
|
||||||
|
function rulesFor(languages: readonly (string | undefined)[]): Intl.PluralRules | null {
|
||||||
|
const key = languages.join('|');
|
||||||
|
let rules = rulesCache.get(key);
|
||||||
|
if (rules !== undefined) return rules;
|
||||||
|
rules = null;
|
||||||
|
for (const language of languages) {
|
||||||
|
if (!language) continue;
|
||||||
|
const tag = language.split(/-x-/i)[0]!;
|
||||||
|
try {
|
||||||
|
if (Intl.PluralRules.supportedLocalesOf([tag]).length) { rules = new Intl.PluralRules(tag); break; }
|
||||||
|
} catch {
|
||||||
|
// Not a tag this runtime accepts: try the next.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rulesCache.set(key, rules);
|
||||||
|
return rules;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Renders a validated message. A parameter is inserted the way string concatenation would
|
||||||
|
* (`String(value)`, nothing for a missing one); `#` is the plural's own value.
|
||||||
|
*/
|
||||||
|
export function renderMessage(parts: Parts, params: Readonly<Record<string, unknown>>, languages: readonly (string | undefined)[], pound: unknown = undefined): string {
|
||||||
|
let out = '';
|
||||||
|
for (const part of parts) {
|
||||||
|
if (typeof part === 'string') { out += part; continue; }
|
||||||
|
// The parser only yields `#` inside a plural, so there is always a value it stands for.
|
||||||
|
if (part.kind === 'pound') { out += pound === undefined || pound === null ? '' : String(pound); continue; }
|
||||||
|
const value = params[part.name];
|
||||||
|
if (part.kind === 'arg') { out += value === undefined || value === null ? '' : String(value); continue; }
|
||||||
|
if (part.kind === 'select') {
|
||||||
|
out += renderMessage(part.options[String(value)] ?? part.options.other!, params, languages, pound);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const n = typeof value === 'number' ? value : Number(value);
|
||||||
|
const option = Number.isFinite(n)
|
||||||
|
? part.options[`=${n}`] ?? part.options[rulesFor(languages)?.select(n) ?? 'other'] ?? part.options.other!
|
||||||
|
: part.options.other!;
|
||||||
|
out += renderMessage(option, params, languages, value);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// The ordered application engine
|
||||||
|
|
||||||
|
export type EngineStats = {
|
||||||
|
/** Keys the site registered. */
|
||||||
|
seen: number;
|
||||||
|
/** Keys a pack value was used for. */
|
||||||
|
applied: number;
|
||||||
|
/** Keys the site registered that the inventory does not have (new keys: drift). */
|
||||||
|
unlisted: string[];
|
||||||
|
/** Keys whose live value does not have the inventory's kind: left untouched. */
|
||||||
|
mismatched: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type StringEngine = {
|
||||||
|
/**
|
||||||
|
* The value to use for `id`, given the site's own value: a pack's, or exactly `original`.
|
||||||
|
* Feeding a value this engine produced back in returns the same value (idempotent).
|
||||||
|
*/
|
||||||
|
resolve(id: string, original: unknown): unknown;
|
||||||
|
/** The site's own values of the keys a pack changed: what restoring needs. */
|
||||||
|
originals(): ReadonlyMap<string, unknown>;
|
||||||
|
stats(): EngineStats;
|
||||||
|
};
|
||||||
|
|
||||||
|
const MAX_LISTED = 200;
|
||||||
|
|
||||||
|
/** `packs` in application order: a later pack wins over an earlier one for the same key. */
|
||||||
|
export function createStringEngine(inventory: StringInventory | PageInventory | InventoryLookup, packs: readonly CompiledPack[]): StringEngine {
|
||||||
|
const lookup = typeof inventory === 'function' ? inventory : inventoryLookup(inventory);
|
||||||
|
const originals = new Map<string, unknown>();
|
||||||
|
const applied = new Map<string, unknown>();
|
||||||
|
const produced = new WeakMap<object, unknown>();
|
||||||
|
const seen = new Set<string>();
|
||||||
|
const unlisted = new Set<string>();
|
||||||
|
const mismatched = new Set<string>();
|
||||||
|
|
||||||
|
const winner = (id: string): { pack: CompiledPack; value: CompiledValue } | null => {
|
||||||
|
for (let index = packs.length - 1; index >= 0; index -= 1) {
|
||||||
|
const value = packs[index]!.values.get(id);
|
||||||
|
if (value) return { pack: packs[index]!, value };
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
};
|
||||||
|
|
||||||
|
const resolve = (id: string, incoming: unknown): unknown => {
|
||||||
|
let original = incoming;
|
||||||
|
// A value of ours coming back in (a second application): work from the site's own value again.
|
||||||
|
if (typeof incoming === 'function' && produced.has(incoming)) original = produced.get(incoming);
|
||||||
|
else if (typeof incoming === 'string' && applied.has(id) && applied.get(id) === incoming) original = originals.get(id);
|
||||||
|
seen.add(id);
|
||||||
|
const found = winner(id);
|
||||||
|
if (!found) {
|
||||||
|
if (unlisted.size < MAX_LISTED && !lookup(id)) unlisted.add(id);
|
||||||
|
return original;
|
||||||
|
}
|
||||||
|
const shape = lookup(id);
|
||||||
|
if (!shape || !translatable(shape.kind)) return original;
|
||||||
|
if (shape.kind === 'text' ? typeof original !== 'string' : typeof original !== 'function') {
|
||||||
|
if (mismatched.size < MAX_LISTED) mismatched.add(id);
|
||||||
|
return original;
|
||||||
|
}
|
||||||
|
if (originals.has(id) && originals.get(id) === original) return applied.get(id);
|
||||||
|
let next: unknown;
|
||||||
|
if ('text' in found.value) {
|
||||||
|
next = found.value.text;
|
||||||
|
} else {
|
||||||
|
const parts = found.value.parts;
|
||||||
|
const languages = [found.pack.language, found.pack.base];
|
||||||
|
const message = (params?: unknown): string => renderMessage(parts, isRecord(params) ? params : {}, languages);
|
||||||
|
produced.set(message, original);
|
||||||
|
next = message;
|
||||||
|
}
|
||||||
|
originals.set(id, original);
|
||||||
|
applied.set(id, next);
|
||||||
|
return next;
|
||||||
|
};
|
||||||
|
|
||||||
|
return {
|
||||||
|
resolve,
|
||||||
|
originals: () => originals,
|
||||||
|
stats: () => ({ seen: seen.size, applied: applied.size, unlisted: [...unlisted], mismatched: [...mismatched] }),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The whole catalog at once: a new object with the packs applied; `catalog` itself is not
|
||||||
|
* changed. `applyPacks(applyPacks(c, …), …)` with one engine equals `applyPacks(c, …)`.
|
||||||
|
*/
|
||||||
|
export function applyPacks(catalog: Readonly<Record<string, unknown>>, engine: StringEngine): Record<string, unknown> {
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const id of Object.keys(catalog)) out[id] = engine.resolve(id, catalog[id]);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The inverse: every value a pack changed is the site's own again; `catalog` is not changed. */
|
||||||
|
export function restoreCatalog(catalog: Readonly<Record<string, unknown>>, engine: StringEngine): Record<string, unknown> {
|
||||||
|
const originals = engine.originals();
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const id of Object.keys(catalog)) out[id] = originals.has(id) ? originals.get(id) : catalog[id];
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Coverage
|
||||||
|
|
||||||
|
export type Coverage = {
|
||||||
|
/** Keys a pack can replace (text and messages). */
|
||||||
|
translatable: number;
|
||||||
|
/** Of those, keys the pack has a valid value for. */
|
||||||
|
translated: number;
|
||||||
|
/** Keys that exist but cannot be replaced by data yet (rich text, formatters). */
|
||||||
|
closed: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Coverage of a compiled pack per group of the inventory, plus `all`. */
|
||||||
|
export function coverage(inventory: StringInventory, pack: Pick<CompiledPack, 'values'>): Record<string, Coverage> {
|
||||||
|
const out: Record<string, Coverage> = { all: { translatable: 0, translated: 0, closed: 0 } };
|
||||||
|
for (const [id, entry] of Object.entries(inventory.entries)) {
|
||||||
|
const group = (out[entry.group ?? MAIN_GROUP] ??= { translatable: 0, translated: 0, closed: 0 });
|
||||||
|
for (const bucket of [group, out.all!]) {
|
||||||
|
if (!translatable(entry.kind)) { bucket.closed += 1; continue; }
|
||||||
|
bucket.translatable += 1;
|
||||||
|
if (pack.values.has(id)) bucket.translated += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Drift (the site shipped a new build)
|
||||||
|
|
||||||
|
export type DriftReport = {
|
||||||
|
/** Keys the site has that the inventory lacks. */
|
||||||
|
added: string[];
|
||||||
|
/** Keys whose base-language value changed (same shape). */
|
||||||
|
reworded: string[];
|
||||||
|
/** Keys whose kind or parameters changed: pack values for them have to be checked again. */
|
||||||
|
reshaped: string[];
|
||||||
|
/** Inventory keys the site no longer has. Only for groups whose base catalog was loaded. */
|
||||||
|
vanished: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
const sameList = (a: readonly string[] = [], b: readonly string[] = []) => a.length === b.length && [...a].sort().join('\n') === [...b].sort().join('\n');
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Compares an inventory with what was observed now. `loaded` names the groups whose base catalog
|
||||||
|
* was actually read: a key can only be called vanished when its group was (a lazily loaded group
|
||||||
|
* that never arrived says nothing about its keys).
|
||||||
|
*/
|
||||||
|
export function driftReport(inventory: StringInventory, observed: Readonly<Record<string, InventoryEntry>>, loaded: readonly string[]): DriftReport {
|
||||||
|
const report: DriftReport = { added: [], reworded: [], reshaped: [], vanished: [] };
|
||||||
|
for (const [id, now] of Object.entries(observed)) {
|
||||||
|
const before = Object.prototype.hasOwnProperty.call(inventory.entries, id) ? inventory.entries[id] : undefined;
|
||||||
|
if (!before) { report.added.push(id); continue; }
|
||||||
|
if (before.kind !== now.kind || !sameList(before.params, now.params) || !sameList(before.plural, now.plural) || !sameList(before.select, now.select) || !sameList(before.tags, now.tags)) {
|
||||||
|
report.reshaped.push(id);
|
||||||
|
} else if (before.hash !== now.hash) {
|
||||||
|
report.reworded.push(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const groups = new Set(loaded);
|
||||||
|
for (const [id, entry] of Object.entries(inventory.entries)) {
|
||||||
|
if (!Object.prototype.hasOwnProperty.call(observed, id) && groups.has(entry.group ?? MAIN_GROUP)) report.vanished.push(id);
|
||||||
|
}
|
||||||
|
return report;
|
||||||
|
}
|
||||||
129
packages/runtime/src/textNet.ts
Normal file
129
packages/runtime/src/textNet.ts
Normal file
|
|
@ -0,0 +1,129 @@
|
||||||
|
// A DOM safety net for strings a catalog hook cannot reach (published as
|
||||||
|
// `waybetter/runtime/textNet`): labels the server sends already worded, or text that is painted
|
||||||
|
// before the hook can act. It replaces exact texts in text nodes and in `aria-label`,
|
||||||
|
// `placeholder` and `title`, keeps the site's own value per node, and puts everything back when it
|
||||||
|
// is disposed. It is secondary to a catalog hook: it cannot see parameters or plural forms, and a
|
||||||
|
// rendered text only maps back to one key in a language without homonyms.
|
||||||
|
//
|
||||||
|
// Text is written through `nodeValue` / `setAttribute` (never `textContent` or HTML), so the site's
|
||||||
|
// framework keeps owning its nodes; when the site rewrites a node, the net looks at it again.
|
||||||
|
|
||||||
|
export type TextNetOptions = {
|
||||||
|
/** Exact original text -> replacement. A text that is not a key is left alone. */
|
||||||
|
map: ReadonlyMap<string, string>;
|
||||||
|
/** Attributes to cover (default `aria-label`, `placeholder`, `title`). */
|
||||||
|
attributes?: readonly string[];
|
||||||
|
/** Elements whose content is never touched (default: scripts, styles, editable text). */
|
||||||
|
skip?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type TextNetHandle = {
|
||||||
|
/** Text nodes and attributes currently replaced. */
|
||||||
|
count(): number;
|
||||||
|
/** Restores every node and attribute to the site's own value and stops observing. */
|
||||||
|
dispose(): void;
|
||||||
|
};
|
||||||
|
|
||||||
|
const DEFAULT_ATTRIBUTES = ['aria-label', 'placeholder', 'title'];
|
||||||
|
const DEFAULT_SKIP = 'script, style, noscript, textarea, [contenteditable=""], [contenteditable="true"]';
|
||||||
|
const TEXT_NODE = 3;
|
||||||
|
const ELEMENT_NODE = 1;
|
||||||
|
const SHOW_TEXT = 4;
|
||||||
|
|
||||||
|
export function installTextNet(win: Window, options: TextNetOptions): TextNetHandle {
|
||||||
|
const doc = win.document;
|
||||||
|
const attributes = options.attributes ?? DEFAULT_ATTRIBUTES;
|
||||||
|
const skip = options.skip ?? DEFAULT_SKIP;
|
||||||
|
const map = options.map;
|
||||||
|
/** The site's own value and ours, per text node. */
|
||||||
|
const texts = new WeakMap<Node, { original: string; applied: string }>();
|
||||||
|
/** The same per element and attribute. */
|
||||||
|
const attrs = new WeakMap<Element, Map<string, { original: string; applied: string }>>();
|
||||||
|
let count = 0;
|
||||||
|
let active = true;
|
||||||
|
|
||||||
|
const skipped = (element: Element | null) => !!element && !!skip && !!element.closest(skip);
|
||||||
|
|
||||||
|
const text = (node: Node) => {
|
||||||
|
const value = node.nodeValue ?? '';
|
||||||
|
const kept = texts.get(node);
|
||||||
|
if (kept) {
|
||||||
|
if (value === kept.applied) return;
|
||||||
|
// The site wrote the node again: what it wrote is the new original.
|
||||||
|
texts.delete(node);
|
||||||
|
count -= 1;
|
||||||
|
}
|
||||||
|
const next = map.get(value);
|
||||||
|
if (next === undefined || next === value || skipped(node.parentElement)) return;
|
||||||
|
texts.set(node, { original: value, applied: next });
|
||||||
|
count += 1;
|
||||||
|
node.nodeValue = next;
|
||||||
|
};
|
||||||
|
|
||||||
|
const attribute = (element: Element, name: string) => {
|
||||||
|
const value = element.getAttribute(name);
|
||||||
|
const kept = attrs.get(element)?.get(name);
|
||||||
|
if (kept) {
|
||||||
|
if (value === kept.applied) return;
|
||||||
|
attrs.get(element)!.delete(name);
|
||||||
|
count -= 1;
|
||||||
|
}
|
||||||
|
if (value === null) return;
|
||||||
|
const next = map.get(value);
|
||||||
|
if (next === undefined || next === value || skipped(element)) return;
|
||||||
|
let own = attrs.get(element);
|
||||||
|
if (!own) attrs.set(element, own = new Map());
|
||||||
|
own.set(name, { original: value, applied: next });
|
||||||
|
count += 1;
|
||||||
|
element.setAttribute(name, next);
|
||||||
|
};
|
||||||
|
|
||||||
|
const sweep = (root: Node) => {
|
||||||
|
if (root.nodeType === TEXT_NODE) { text(root); return; }
|
||||||
|
if (root.nodeType !== ELEMENT_NODE) return;
|
||||||
|
const element = root as Element;
|
||||||
|
const walker = doc.createTreeWalker(element, SHOW_TEXT);
|
||||||
|
while (walker.nextNode()) text(walker.currentNode);
|
||||||
|
for (const name of attributes) {
|
||||||
|
if (element.hasAttribute(name)) attribute(element, name);
|
||||||
|
for (const child of element.querySelectorAll(`[${name}]`)) attribute(child, name);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const observer = new (win as unknown as { MutationObserver: typeof MutationObserver }).MutationObserver(records => {
|
||||||
|
if (!active) return;
|
||||||
|
for (const record of records) {
|
||||||
|
if (record.type === 'characterData') text(record.target);
|
||||||
|
else if (record.type === 'attributes') { if (record.attributeName) attribute(record.target as Element, record.attributeName); }
|
||||||
|
else for (const node of record.addedNodes) sweep(node);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if (map.size) {
|
||||||
|
if (doc.documentElement) sweep(doc.documentElement);
|
||||||
|
observer.observe(doc, { subtree: true, childList: true, characterData: true, attributes: true, attributeFilter: [...attributes] });
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
count: () => count,
|
||||||
|
dispose() {
|
||||||
|
if (!active) return;
|
||||||
|
active = false;
|
||||||
|
observer.disconnect();
|
||||||
|
if (!doc.documentElement) return;
|
||||||
|
const walker = doc.createTreeWalker(doc.documentElement, SHOW_TEXT);
|
||||||
|
while (walker.nextNode()) {
|
||||||
|
const node = walker.currentNode;
|
||||||
|
const kept = texts.get(node);
|
||||||
|
if (kept && node.nodeValue === kept.applied) node.nodeValue = kept.original;
|
||||||
|
texts.delete(node);
|
||||||
|
}
|
||||||
|
for (const name of attributes) {
|
||||||
|
for (const element of doc.querySelectorAll(`[${name}]`)) {
|
||||||
|
const kept = attrs.get(element)?.get(name);
|
||||||
|
if (kept && element.getAttribute(name) === kept.applied) element.setAttribute(name, kept.original);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
count = 0;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
545
packages/runtime/tests/stringPacks.test.ts
Normal file
545
packages/runtime/tests/stringPacks.test.ts
Normal file
|
|
@ -0,0 +1,545 @@
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { test } from 'node:test';
|
||||||
|
import {
|
||||||
|
applyPacks, checkPackShape, compilePack, coverage, createStringEngine, driftReport, hashText, inventoryLookup, packInventory,
|
||||||
|
renderMessage, restoreCatalog, validateString, type InventoryEntry, type StringInventory, type StringPack,
|
||||||
|
} from '../src/stringPacks.ts';
|
||||||
|
import { parseMessage } from '../src/i18n.ts';
|
||||||
|
import { installStringPacks, mapChunkRequires, watchChunkArray, type CatalogHookAdapter, type CatalogSink } from '../src/stringPackHook.ts';
|
||||||
|
import { installTextNet } from '../src/textNet.ts';
|
||||||
|
|
||||||
|
// A small site: two texts, two messages (one plural), one rich entry, one formatter, one lazy group.
|
||||||
|
const INVENTORY: StringInventory = {
|
||||||
|
format: 1, site: 'demo', catalog: 'web', base: 'en', build: 'b1',
|
||||||
|
entries: {
|
||||||
|
a0000001: { kind: 'text', hash: hashText('Home') },
|
||||||
|
a0000002: { kind: 'text', hash: hashText('Following') },
|
||||||
|
b0000001: { kind: 'message', params: ['name'], hash: hashText('Hello {name}') },
|
||||||
|
b0000002: { kind: 'message', params: ['count', 'name'], plural: ['count'], hash: hashText('{count} posts by {name}') },
|
||||||
|
c0000001: { kind: 'rich', hash: hashText('rich') },
|
||||||
|
d0000001: { kind: 'format', hash: hashText('fmt') },
|
||||||
|
e0000001: { kind: 'text', group: 'emoji', hash: hashText('Grinning face') },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
/** The site's own values, as its catalog registers them. */
|
||||||
|
const site = () => ({
|
||||||
|
a0000001: 'Home',
|
||||||
|
a0000002: 'Following',
|
||||||
|
b0000001: (e: { name: string }) => `Hello ${e.name}`,
|
||||||
|
b0000002: (e: { count: number; name: string }) => `${e.count} ${e.count === 1 ? 'post' : 'posts'} by ${e.name}`,
|
||||||
|
c0000001: () => ({ element: true }),
|
||||||
|
d0000001: (n: number) => String(n),
|
||||||
|
e0000001: 'Grinning face',
|
||||||
|
});
|
||||||
|
const pack = (strings: Record<string, string>, extra: Partial<StringPack> = {}): StringPack => ({
|
||||||
|
format: 1, site: 'demo', catalog: 'web', language: 'en-x-test', name: 'Test', authors: ['A tester'], base: 'en', strings, ...extra,
|
||||||
|
});
|
||||||
|
|
||||||
|
test('hashText: stable, 11 characters, different for different text', () => {
|
||||||
|
assert.equal(hashText('Home'), hashText('Home'));
|
||||||
|
assert.match(hashText('Home'), /^[0-9a-z]{11}$/);
|
||||||
|
assert.notEqual(hashText('Home'), hashText('home'));
|
||||||
|
assert.equal(hashText(''), hashText(''));
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pack envelope: what is refused as a whole', () => {
|
||||||
|
assert.deepEqual(checkPackShape(pack({ a0000001: 'Hoom' })), []);
|
||||||
|
const errors = (input: unknown) => checkPackShape(input).filter(issue => issue.level === 'error').map(issue => issue.message).join(' | ');
|
||||||
|
assert.match(errors([]), /JSON object/);
|
||||||
|
assert.match(errors({ ...pack({}), format: 2 }), /format/);
|
||||||
|
assert.match(errors({ ...pack({}), language: 'not a tag' }), /language/);
|
||||||
|
assert.match(errors({ ...pack({}), name: '<b>x</b>' }), /name/);
|
||||||
|
assert.match(errors({ ...pack({}), authors: 'me' }), /authors/);
|
||||||
|
assert.match(errors({ ...pack({}), strings: [] }), /strings/);
|
||||||
|
assert.match(errors({ ...pack({}), site: 'Demo Site' }), /site/);
|
||||||
|
assert.deepEqual(checkPackShape({ ...pack({}), extra: 1 }).map(issue => issue.level), ['warning']);
|
||||||
|
assert.throws(() => compilePack({ ...pack({}), format: 2 }, INVENTORY), /invalid string pack/);
|
||||||
|
assert.throws(() => compilePack(pack({}, { catalog: 'other' }), INVENTORY), /the pack is for demo\/other/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('validator: parameters, plural kind, other, markup', () => {
|
||||||
|
const entry = (id: string) => INVENTORY.entries[id]!;
|
||||||
|
const reason = (id: string, value: unknown) => { const result = validateString(entry(id), value); return 'reason' in result ? result.reason : ''; };
|
||||||
|
// Accepted.
|
||||||
|
assert.equal(reason('a0000001', 'Hoom'), '');
|
||||||
|
assert.equal(reason('b0000001', 'Ahoy {name}'), '');
|
||||||
|
assert.equal(reason('b0000001', 'Ahoy'), '', 'a subset of the parameters is fine');
|
||||||
|
assert.equal(reason('b0000002', '{count, plural, one {# postle} other {# postles}} by {name}'), '');
|
||||||
|
assert.equal(reason('b0000002', '{count, plural, =0 {nothing} other {#}}'), '');
|
||||||
|
assert.equal(reason('b0000002', '{count} by {name}'), '', 'a language without plural forms needs none');
|
||||||
|
assert.equal(reason('a0000001', "It's '{'fine'}'"), '', 'quoted braces are text');
|
||||||
|
// Refused.
|
||||||
|
assert.match(reason('a0000001', 'Hi {name}'), /takes no parameters/);
|
||||||
|
assert.match(reason('b0000001', 'Ahoy {user}'), /not a parameter of the original \(it has: name\)/);
|
||||||
|
assert.match(reason('b0000001', '{name, plural, one {a} other {b}}'), /not a plural of the original/);
|
||||||
|
assert.match(reason('b0000002', '{name, select, a {x} other {y}}'), /not a select of the original/);
|
||||||
|
assert.match(reason('b0000002', '{count, plural, one {a}}'), /"other"/);
|
||||||
|
assert.match(reason('b0000002', '{count, plural, lots {a} other {b}}'), /not a plural form/);
|
||||||
|
assert.match(reason('b0000002', '{count, selectordinal, one {a} other {b}}'), /selectordinal/);
|
||||||
|
assert.match(reason('b0000002', '{count, number}'), /not supported/);
|
||||||
|
assert.match(reason('b0000002', '{count, plural, offset:1 other {b}}'), /offset/);
|
||||||
|
assert.match(reason('a0000001', '<b>Home</b>'), /markup is not allowed/);
|
||||||
|
assert.match(reason('a0000001', 'a > b'), /markup is not allowed/);
|
||||||
|
assert.match(reason('a0000001', 'Home {'), /invalid message syntax/);
|
||||||
|
assert.match(reason('a0000001', ''), /empty/);
|
||||||
|
assert.match(reason('a0000001', 42), /must be text/);
|
||||||
|
assert.match(reason('a0000001', 'x'.repeat(2001)), /longer than/);
|
||||||
|
assert.match(reason('a0000001', 'ab'), /control or bidi/);
|
||||||
|
assert.match(reason('c0000001', 'text'), /not translatable yet \(rich\)/);
|
||||||
|
assert.match(reason('d0000001', 'text'), /not translatable yet \(format\)/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('validator: tag slots must be the original\'s', () => {
|
||||||
|
const shape: InventoryEntry = { kind: 'text', tags: ['link'], hash: 'x' };
|
||||||
|
const reason = (value: string) => { const result = validateString(shape, value); return 'reason' in result ? result.reason : ''; };
|
||||||
|
assert.equal(reason('Read <link>the rules</link>'), '');
|
||||||
|
assert.match(reason('Read the rules'), /tags are missing: <link>/);
|
||||||
|
assert.match(reason('Read <b>the</b> <link>rules</link>'), /<b> is not a tag of the original/);
|
||||||
|
assert.match(reason('Read <link>the rules'), /not closed/);
|
||||||
|
assert.match(reason('Read </link>the rules<link>'), /does not close/);
|
||||||
|
assert.match(reason('1 < 2 <link>x</link>'), /may only be used for the original's tags/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('compilePack: unknown keys are reported and never applied; bad values are rejected one by one', () => {
|
||||||
|
const compiled = compilePack(pack({
|
||||||
|
a0000001: 'Hoom', b0000001: 'Ahoy {name}', zzzzzzzz: 'Nope', a0000002: '<i>x</i>', c0000001: 'rich text', b0000002: '{count, plural, one {# postle} other {# postles}}',
|
||||||
|
}), INVENTORY);
|
||||||
|
assert.deepEqual([...compiled.values.keys()].sort(), ['a0000001', 'b0000001', 'b0000002']);
|
||||||
|
assert.deepEqual(compiled.report.unknown, ['zzzzzzzz']);
|
||||||
|
assert.deepEqual(compiled.report.rejected.map(item => item.id).sort(), ['a0000002', 'c0000001']);
|
||||||
|
assert.equal(compiled.report.total, 6);
|
||||||
|
assert.equal(compiled.report.valid, 3);
|
||||||
|
// An unknown key reaches the engine as nothing at all.
|
||||||
|
const engine = createStringEngine(INVENTORY, [compiled]);
|
||||||
|
assert.equal(engine.resolve('zzzzzzzz', 'Site text'), 'Site text');
|
||||||
|
assert.equal(engine.resolve('a0000002', 'Following'), 'Following', 'a rejected value leaves the original');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('renderMessage: concatenation semantics, plural by the pack language, exact forms, select', () => {
|
||||||
|
const render = (source: string, params: Record<string, unknown>, languages: (string | undefined)[] = ['en']) => renderMessage(parseMessage(source), params, languages);
|
||||||
|
assert.equal(render('Ahoy {name}!', { name: 'Ada' }), 'Ahoy Ada!');
|
||||||
|
assert.equal(render('Ahoy {name}!', {}), 'Ahoy !');
|
||||||
|
assert.equal(render('{count} of them', { count: '1,234' }), '1,234 of them', 'a parameter is inserted as it comes');
|
||||||
|
const plural = '{count, plural, =0 {none} one {# postle} other {# postles}}';
|
||||||
|
assert.equal(render(plural, { count: 0 }), 'none');
|
||||||
|
assert.equal(render(plural, { count: 1 }), '1 postle');
|
||||||
|
assert.equal(render(plural, { count: 5 }), '5 postles');
|
||||||
|
assert.equal(render(plural, { count: '7' }), '7 postles');
|
||||||
|
assert.equal(render(plural, { count: 'many' }), 'many postles', 'not a number: other');
|
||||||
|
// A language the runtime does not know falls back to the pack's base, then to `other`.
|
||||||
|
assert.equal(render(plural, { count: 1 }, ['qqq-x-made-up', 'en']), '1 postle');
|
||||||
|
assert.equal(render(plural, { count: 1 }, ['zz-x-test', undefined]), '1 postles');
|
||||||
|
assert.equal(render(plural, { count: 1 }, ['en-x-test']), '1 postle', 'a private-use tag uses its language');
|
||||||
|
assert.equal(render('{count, plural, one {# gönderi} other {# gönderi}}', { count: 2 }, ['tr']), '2 gönderi');
|
||||||
|
assert.equal(render('{kind, select, a {first} other {rest}}', { kind: 'a' }), 'first');
|
||||||
|
assert.equal(render('{kind, select, a {first} other {rest}}', { kind: 'b' }), 'rest');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('engine: a pack value when the key has one, otherwise exactly the site\'s value', () => {
|
||||||
|
const values = site();
|
||||||
|
const engine = createStringEngine(INVENTORY, [compilePack(pack({
|
||||||
|
a0000001: 'Hoom', b0000002: '{count, plural, one {# postle} other {# postles}} by {name}',
|
||||||
|
}), INVENTORY)]);
|
||||||
|
const out = applyPacks(values, engine);
|
||||||
|
assert.equal(out.a0000001, 'Hoom');
|
||||||
|
assert.equal((out.b0000002 as (e: unknown) => string)({ count: 1, name: 'Ada' }), '1 postle by Ada');
|
||||||
|
assert.equal((out.b0000002 as (e: unknown) => string)({ count: 3, name: 'Ada' }), '3 postles by Ada');
|
||||||
|
assert.equal((out.b0000002 as () => string)(), ' postles by ', 'called without parameters: still a string');
|
||||||
|
// Untouched keys are the very same values (same function objects).
|
||||||
|
for (const id of ['a0000002', 'b0000001', 'c0000001', 'd0000001', 'e0000001'] as const) assert.equal(out[id], values[id], id);
|
||||||
|
// The site's catalog object was not changed.
|
||||||
|
assert.deepEqual(Object.keys(values), Object.keys(site()));
|
||||||
|
assert.equal(values.a0000001, 'Home');
|
||||||
|
assert.deepEqual(engine.stats(), { seen: 7, applied: 2, unlisted: [], mismatched: [] });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('engine: applying twice equals applying once', () => {
|
||||||
|
const values = site();
|
||||||
|
const engine = createStringEngine(INVENTORY, [compilePack(pack({ a0000001: 'Hoom', b0000001: 'Ahoy {name}' }), INVENTORY)]);
|
||||||
|
const once = applyPacks(values, engine);
|
||||||
|
const twice = applyPacks(once, engine);
|
||||||
|
const again = applyPacks(values, engine);
|
||||||
|
for (const id of Object.keys(values)) {
|
||||||
|
assert.equal(twice[id], once[id], `${id}: second application changes nothing`);
|
||||||
|
assert.equal(again[id], once[id], `${id}: the same input gives the same value`);
|
||||||
|
}
|
||||||
|
assert.equal(engine.stats().applied, 2);
|
||||||
|
// A fresh engine (a reload) gives the same result for the same input.
|
||||||
|
const reloaded = applyPacks(values, createStringEngine(INVENTORY, [compilePack(pack({ a0000001: 'Hoom', b0000001: 'Ahoy {name}' }), INVENTORY)]));
|
||||||
|
assert.equal(reloaded.a0000001, once.a0000001);
|
||||||
|
assert.equal((reloaded.b0000001 as (e: unknown) => string)({ name: 'Ada' }), (once.b0000001 as (e: unknown) => string)({ name: 'Ada' }));
|
||||||
|
});
|
||||||
|
|
||||||
|
test('engine: packs apply in the listed order, the later one wins per key', () => {
|
||||||
|
const language = compilePack(pack({ a0000001: 'Hoom', a0000002: 'Trailing' }, { name: 'Language' }), INVENTORY);
|
||||||
|
const rewording = compilePack(pack({ a0000001: 'Den' }, { name: 'Rewording' }), INVENTORY);
|
||||||
|
const first = applyPacks(site(), createStringEngine(INVENTORY, [language, rewording]));
|
||||||
|
assert.equal(first.a0000001, 'Den');
|
||||||
|
assert.equal(first.a0000002, 'Trailing', 'a key only the earlier pack has keeps its value');
|
||||||
|
const second = applyPacks(site(), createStringEngine(INVENTORY, [rewording, language]));
|
||||||
|
assert.equal(second.a0000001, 'Hoom');
|
||||||
|
// No packs: everything is the site's.
|
||||||
|
const none = applyPacks(site(), createStringEngine(INVENTORY, []));
|
||||||
|
assert.equal(none.a0000001, 'Home');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('engine: restore gives back exactly the site\'s values', () => {
|
||||||
|
const values = site();
|
||||||
|
const engine = createStringEngine(INVENTORY, [compilePack(pack({ a0000001: 'Hoom', b0000001: 'Ahoy {name}' }), INVENTORY)]);
|
||||||
|
const applied = applyPacks(values, engine);
|
||||||
|
assert.deepEqual([...engine.originals().keys()].sort(), ['a0000001', 'b0000001'], 'only what a pack changed is kept');
|
||||||
|
const restored = restoreCatalog(applied, engine);
|
||||||
|
for (const id of Object.keys(values) as (keyof typeof values)[]) assert.equal(restored[id], values[id], id);
|
||||||
|
assert.equal(applied.a0000001, 'Hoom', 'restoring does not change the applied object');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('engine: a live value of another kind than the inventory says is left alone', () => {
|
||||||
|
const engine = createStringEngine(INVENTORY, [compilePack(pack({ a0000001: 'Hoom', b0000001: 'Ahoy {name}' }), INVENTORY)]);
|
||||||
|
const fn = () => 'now a function';
|
||||||
|
assert.equal(engine.resolve('a0000001', fn), fn);
|
||||||
|
assert.equal(engine.resolve('b0000001', 'now a string'), 'now a string');
|
||||||
|
assert.deepEqual(engine.stats().mismatched.sort(), ['a0000001', 'b0000001']);
|
||||||
|
assert.equal(engine.stats().applied, 0);
|
||||||
|
// A key the site has and the inventory lacks is counted as drift.
|
||||||
|
assert.equal(engine.resolve('f0000009', 'New text'), 'New text');
|
||||||
|
assert.deepEqual(engine.stats().unlisted, ['f0000009']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('page inventory: the compact form answers like the full one', () => {
|
||||||
|
const page = packInventory(INVENTORY);
|
||||||
|
assert.equal(JSON.stringify(page).includes('hash'), false);
|
||||||
|
const full = inventoryLookup(INVENTORY);
|
||||||
|
const compact = inventoryLookup(page);
|
||||||
|
assert.equal(compact('a0000001')?.kind, 'text');
|
||||||
|
assert.equal(compact('e0000001')?.kind, 'text');
|
||||||
|
assert.deepEqual(compact('b0000002'), { kind: 'message', params: ['count', 'name'], plural: ['count'], select: [] });
|
||||||
|
assert.equal(compact('nope'), undefined);
|
||||||
|
assert.equal(full('nope'), undefined);
|
||||||
|
assert.equal(full('constructor'), undefined, 'inherited names are not keys');
|
||||||
|
for (const id of ['c0000001', 'd0000001']) assert.notEqual(compact(id)?.kind, 'text');
|
||||||
|
// The same pack compiles the same against both.
|
||||||
|
const strings = { a0000001: 'Hoom', b0000002: '{count, plural, one {#} other {#}}', c0000001: 'x', zzzzzzzz: 'y' };
|
||||||
|
const a = compilePack(pack(strings), INVENTORY);
|
||||||
|
const b = compilePack(pack(strings), page);
|
||||||
|
assert.deepEqual([...a.values.keys()], [...b.values.keys()]);
|
||||||
|
assert.deepEqual(a.report.unknown, b.report.unknown);
|
||||||
|
assert.deepEqual(a.report.rejected.map(item => item.id), b.report.rejected.map(item => item.id));
|
||||||
|
});
|
||||||
|
|
||||||
|
test('coverage: per group, with what cannot be translated yet counted apart', () => {
|
||||||
|
const compiled = compilePack(pack({ a0000001: 'Hoom', b0000001: 'Ahoy {name}', e0000001: 'Grin' }), INVENTORY);
|
||||||
|
assert.deepEqual(coverage(INVENTORY, compiled), {
|
||||||
|
all: { translatable: 5, translated: 3, closed: 2 },
|
||||||
|
main: { translatable: 4, translated: 2, closed: 2 },
|
||||||
|
emoji: { translatable: 1, translated: 1, closed: 0 },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test('drift report: new, reworded, reshaped; vanished only for a group whose base catalog was loaded', () => {
|
||||||
|
const observed: Record<string, InventoryEntry> = {
|
||||||
|
a0000001: { kind: 'text', hash: hashText('Home') },
|
||||||
|
a0000002: { kind: 'text', hash: hashText('Following now') },
|
||||||
|
b0000001: { kind: 'message', params: ['name', 'title'], hash: hashText('Hello {name}') },
|
||||||
|
c0000001: { kind: 'rich', hash: hashText('rich') },
|
||||||
|
d0000001: { kind: 'format', hash: hashText('fmt') },
|
||||||
|
f0000001: { kind: 'text', hash: hashText('Brand new') },
|
||||||
|
};
|
||||||
|
assert.deepEqual(driftReport(INVENTORY, observed, ['main']), {
|
||||||
|
added: ['f0000001'], reworded: ['a0000002'], reshaped: ['b0000001'], vanished: ['b0000002'],
|
||||||
|
});
|
||||||
|
// The emoji group was never loaded: its key is not called vanished. With it loaded, it is.
|
||||||
|
assert.deepEqual(driftReport(INVENTORY, observed, ['main', 'emoji']).vanished, ['b0000002', 'e0000001']);
|
||||||
|
// Nothing loaded: nothing can be called vanished.
|
||||||
|
assert.deepEqual(driftReport(INVENTORY, observed, []).vanished, []);
|
||||||
|
assert.deepEqual(driftReport(INVENTORY, INVENTORY.entries, ['main', 'emoji']), { added: [], reworded: [], reshaped: [], vanished: [] });
|
||||||
|
});
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// The chunk array hook
|
||||||
|
|
||||||
|
/** A stand-in for webpack's runtime: takes over `push`, replays what is there, installs modules. */
|
||||||
|
function webpackRuntime(array: unknown[]) {
|
||||||
|
const installed: Record<string, unknown> = {};
|
||||||
|
const parentPush = array.push.bind(array);
|
||||||
|
const callback = (parent: ((chunk: unknown) => number) | 0, chunk: unknown) => {
|
||||||
|
const [, modules] = chunk as [unknown, Record<string, unknown>];
|
||||||
|
Object.assign(installed, modules);
|
||||||
|
if (parent) parent(chunk);
|
||||||
|
return 0;
|
||||||
|
};
|
||||||
|
array.forEach(chunk => callback(0, chunk));
|
||||||
|
array.push = ((chunk: unknown) => callback(parentPush, chunk)) as typeof array.push;
|
||||||
|
return installed;
|
||||||
|
}
|
||||||
|
|
||||||
|
test('chunk array hook: every chunk is seen once, before the site gets it, whichever loads first', () => {
|
||||||
|
const win: Record<string, unknown> & { document: { currentScript: { src: string } | null } } = { document: { currentScript: { src: 'https://cdn.test/a.js' } } };
|
||||||
|
const seen: string[] = [];
|
||||||
|
watchChunkArray(win, 'chunks', (chunk, script) => { seen.push(`${String((chunk[0] as unknown[])[0])}@${script}`); });
|
||||||
|
const array = win.chunks as unknown[];
|
||||||
|
const early = [[1], { m1: () => 1 }];
|
||||||
|
array.push(early);
|
||||||
|
assert.equal(array.length, 1, 'before the runtime loads, push is still the array\'s own');
|
||||||
|
const installed = webpackRuntime(array);
|
||||||
|
win.document.currentScript = { src: 'https://cdn.test/b.js' };
|
||||||
|
const late = [[2], { m2: () => 2 }];
|
||||||
|
array.push(late);
|
||||||
|
win.document.currentScript = null;
|
||||||
|
array.push([[3], {}]);
|
||||||
|
assert.deepEqual(seen, ['1@https://cdn.test/a.js', '2@https://cdn.test/b.js', '3@']);
|
||||||
|
assert.deepEqual(Object.keys(installed), ['m1', 'm2']);
|
||||||
|
assert.deepEqual(array, [early, late, [[3], {}]], 'the array holds the same chunks in the same order');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('chunk array hook: a chunk nobody maps passes through byte-identical', () => {
|
||||||
|
const win: Record<string, unknown> = {};
|
||||||
|
watchChunkArray(win, 'chunks', () => { /* looks, touches nothing */ });
|
||||||
|
const array = win.chunks as unknown[];
|
||||||
|
const installed = webpackRuntime(array);
|
||||||
|
const factory = function (module: { exports: unknown }) { module.exports = 42; };
|
||||||
|
const runtime = () => {};
|
||||||
|
const chunk = [[7, 8], { m7: factory }, runtime];
|
||||||
|
const snapshot = { ids: chunk[0], modules: chunk[1], source: factory.toString(), keys: Object.keys(chunk[1] as object) };
|
||||||
|
array.push(chunk);
|
||||||
|
assert.equal(array[0], chunk, 'the same chunk object reaches the array');
|
||||||
|
assert.equal(chunk.length, 3);
|
||||||
|
assert.equal(chunk[0], snapshot.ids);
|
||||||
|
assert.equal(chunk[1], snapshot.modules);
|
||||||
|
assert.equal(chunk[2], runtime);
|
||||||
|
assert.equal((chunk[1] as Record<string, unknown>).m7, factory, 'the same factory function');
|
||||||
|
assert.equal(installed.m7, factory);
|
||||||
|
assert.equal(factory.toString(), snapshot.source);
|
||||||
|
assert.deepEqual(Object.keys(chunk[1] as object), snapshot.keys);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('chunk array hook: fails open, and removal puts the site\'s push back', () => {
|
||||||
|
const win: Record<string, unknown> = {};
|
||||||
|
const remove = watchChunkArray(win, 'chunks', () => { throw new Error('boom'); });
|
||||||
|
const array = win.chunks as unknown[];
|
||||||
|
const installed = webpackRuntime(array);
|
||||||
|
const sitePush = Object.getOwnPropertyDescriptor(array, 'push');
|
||||||
|
assert.ok(sitePush?.get, 'the accessor is ours while installed');
|
||||||
|
assert.doesNotThrow(() => array.push([[1], { m1: 1 }]));
|
||||||
|
assert.deepEqual(Object.keys(installed), ['m1']);
|
||||||
|
remove();
|
||||||
|
assert.equal(typeof Object.getOwnPropertyDescriptor(array, 'push')?.value, 'function', 'a plain property again');
|
||||||
|
array.push([[2], { m2: 2 }]);
|
||||||
|
assert.deepEqual(Object.keys(installed), ['m1', 'm2']);
|
||||||
|
assert.equal(array.length, 2);
|
||||||
|
// Removed before any runtime took over: the array has no own push left.
|
||||||
|
const other: Record<string, unknown> = {};
|
||||||
|
watchChunkArray(other, 'chunks', () => {})();
|
||||||
|
assert.equal(Object.prototype.hasOwnProperty.call(other.chunks, 'push'), false);
|
||||||
|
// An existing array is kept (the site's `||= []` finds it).
|
||||||
|
const existing: unknown[] = [];
|
||||||
|
const third: Record<string, unknown> = { chunks: existing };
|
||||||
|
watchChunkArray(third, 'chunks', () => {});
|
||||||
|
assert.equal(third.chunks, existing);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('mapChunkRequires: a module sees a shadow of what it requires; the required module is unchanged', () => {
|
||||||
|
const registry = { register: (language: string) => `site:${language}`, other: 1 };
|
||||||
|
const require = Object.assign((id: string): any => (id === 'registry' ? registry : { id }), { e: 'kept' });
|
||||||
|
const results: unknown[] = [];
|
||||||
|
const factory = function (this: unknown, module: { exports: unknown }, _exports: unknown, req: typeof require) {
|
||||||
|
results.push(req('registry').register('en'), req('plain'), req.e, this);
|
||||||
|
module.exports = 'done';
|
||||||
|
};
|
||||||
|
const chunk: [unknown, Record<string, unknown>] = [[1], { m: factory }];
|
||||||
|
mapChunkRequires(chunk, exports => {
|
||||||
|
if (exports !== registry) return undefined;
|
||||||
|
const shadow = Object.create(registry) as typeof registry;
|
||||||
|
Object.defineProperty(shadow, 'register', { value: (language: string) => `ours:${language}` });
|
||||||
|
return shadow;
|
||||||
|
});
|
||||||
|
assert.notEqual(chunk[1].m, factory);
|
||||||
|
const module = { exports: {} as unknown };
|
||||||
|
(chunk[1].m as (...args: unknown[]) => void).call('self', module, module.exports, require);
|
||||||
|
assert.deepEqual(results, ['ours:en', { id: 'plain' }, 'kept', 'self']);
|
||||||
|
assert.equal(module.exports, 'done');
|
||||||
|
assert.equal(registry.register('en'), 'site:en', 'the site\'s module is untouched');
|
||||||
|
assert.deepEqual(Object.keys(registry), ['register', 'other']);
|
||||||
|
// A throwing map hands out the module's own exports.
|
||||||
|
const failing: [unknown, Record<string, unknown>] = [[2], { m: factory }];
|
||||||
|
mapChunkRequires(failing, () => { throw new Error('boom'); });
|
||||||
|
results.length = 0;
|
||||||
|
(failing[1].m as (...args: unknown[]) => void).call('self', module, module.exports, require);
|
||||||
|
assert.equal(results[0], 'site:en');
|
||||||
|
});
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// Adapters and the self-check
|
||||||
|
|
||||||
|
function fakeWindow() {
|
||||||
|
const timers: (() => void)[] = [];
|
||||||
|
const win = { setTimeout: (fn: () => void) => { timers.push(fn); return timers.length; }, clearTimeout: () => {} } as unknown as Window;
|
||||||
|
return { win, settle: () => { for (const fn of timers.splice(0)) fn(); } };
|
||||||
|
}
|
||||||
|
/** An adapter driven by the test: `drive` plays the site. */
|
||||||
|
function adapter(catalog = 'web') {
|
||||||
|
const state: { sink: CatalogSink | null; removed: number } = { sink: null, removed: 0 };
|
||||||
|
const hook: CatalogHookAdapter = { catalog, install: (_win, sink) => { state.sink = sink; return () => { state.removed += 1; }; } };
|
||||||
|
return { hook, state };
|
||||||
|
}
|
||||||
|
|
||||||
|
test('self-check: ok, "hook not matched" and "no catalog seen" are distinct', () => {
|
||||||
|
const packs = [pack({ a0000001: 'Hoom' })];
|
||||||
|
// ok: a catalog file was seen, the hook matched, keys went through.
|
||||||
|
{
|
||||||
|
const { win, settle } = fakeWindow();
|
||||||
|
const { hook, state } = adapter();
|
||||||
|
const told: string[] = [];
|
||||||
|
const handle = installStringPacks(win, [{ adapter: hook, inventory: INVENTORY, packs }], { onStatus: report => told.push(report.status) });
|
||||||
|
assert.equal(handle.status(), 'pending');
|
||||||
|
state.sink!.seen();
|
||||||
|
state.sink!.matched('en-GB');
|
||||||
|
assert.equal(state.sink!.resolve('a0000001', 'Home'), 'Hoom');
|
||||||
|
assert.equal(state.sink!.resolve('a0000002', 'Following'), 'Following');
|
||||||
|
assert.equal(handle.status(), 'ok');
|
||||||
|
settle();
|
||||||
|
assert.deepEqual(told, ['ok'], 'told once');
|
||||||
|
const [report] = handle.report();
|
||||||
|
assert.deepEqual({ status: report!.status, languages: report!.languages, seen: report!.seen, applied: report!.engine.applied }, { status: 'ok', languages: ['en-GB'], seen: { main: 1 }, applied: 1 });
|
||||||
|
handle.dispose();
|
||||||
|
assert.equal(state.removed, 1);
|
||||||
|
}
|
||||||
|
// A catalog file was seen but nothing went through the hook.
|
||||||
|
{
|
||||||
|
const { win, settle } = fakeWindow();
|
||||||
|
const { hook, state } = adapter();
|
||||||
|
const told: string[] = [];
|
||||||
|
const handle = installStringPacks(win, [{ adapter: hook, inventory: INVENTORY, packs }], { onStatus: report => told.push(report.status) });
|
||||||
|
state.sink!.seen('main');
|
||||||
|
assert.equal(handle.status(), 'pending');
|
||||||
|
settle();
|
||||||
|
assert.equal(handle.status(), 'not-matched');
|
||||||
|
assert.deepEqual(told, ['not-matched']);
|
||||||
|
}
|
||||||
|
// No catalog file at all.
|
||||||
|
{
|
||||||
|
const { win, settle } = fakeWindow();
|
||||||
|
const { hook } = adapter();
|
||||||
|
const told: string[] = [];
|
||||||
|
const handle = installStringPacks(win, [{ adapter: hook, inventory: INVENTORY, packs }], { onStatus: report => told.push(report.status) });
|
||||||
|
settle();
|
||||||
|
assert.equal(handle.status(), 'no-catalog');
|
||||||
|
assert.deepEqual(told, ['no-catalog']);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('install: no usable pack means no hook; several catalogs keep their own packs', () => {
|
||||||
|
const { win } = fakeWindow();
|
||||||
|
const web = adapter('web');
|
||||||
|
const guest = adapter('guest');
|
||||||
|
const guestInventory: StringInventory = { format: 1, site: 'demo', catalog: 'guest', base: 'en', entries: { 'Sign in': { kind: 'text', hash: hashText('Sign in') } } };
|
||||||
|
const handle = installStringPacks(win, [
|
||||||
|
{ adapter: web.hook, inventory: INVENTORY, packs: [] },
|
||||||
|
{ adapter: guest.hook, inventory: guestInventory, packs: [pack({ 'Sign in': 'Sign inne' }, { catalog: 'guest' }), pack({ a0000001: 'Hoom' }), 'not a pack'] },
|
||||||
|
]);
|
||||||
|
assert.equal(web.state.sink, null, 'a catalog without packs is not hooked');
|
||||||
|
assert.ok(guest.state.sink);
|
||||||
|
assert.equal(guest.state.sink!.resolve('Sign in', 'Sign in'), 'Sign inne');
|
||||||
|
const report = handle.report();
|
||||||
|
assert.equal(report[1]!.packs.length, 1);
|
||||||
|
assert.equal(report[1]!.refused.length, 2, 'a pack for another catalog and a malformed one are refused as a whole');
|
||||||
|
assert.equal(handle.engine('web'), undefined);
|
||||||
|
assert.ok(handle.engine('guest'));
|
||||||
|
// Only unknown keys: nothing usable, nothing hooked.
|
||||||
|
const third = adapter();
|
||||||
|
installStringPacks(win, [{ adapter: third.hook, inventory: INVENTORY, packs: [pack({ zzzzzzzz: 'x' })] }]);
|
||||||
|
assert.equal(third.state.sink, null);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('install: an adapter or engine that throws leaves the site\'s value', () => {
|
||||||
|
const { win } = fakeWindow();
|
||||||
|
const broken: CatalogHookAdapter = { catalog: 'web', install: () => { throw new Error('boom'); } };
|
||||||
|
assert.doesNotThrow(() => installStringPacks(win, [{ adapter: broken, inventory: INVENTORY, packs: [pack({ a0000001: 'Hoom' })] }]).dispose());
|
||||||
|
});
|
||||||
|
|
||||||
|
// -------------------------------------------------------------------------------------------
|
||||||
|
// The DOM safety net (a minimal document: text nodes, elements, attributes)
|
||||||
|
|
||||||
|
type FakeNode = { nodeType: number; nodeValue: string | null; parentElement: FakeElement | null };
|
||||||
|
class FakeElement {
|
||||||
|
nodeType = 1;
|
||||||
|
nodeValue = null;
|
||||||
|
parentElement: FakeElement | null = null;
|
||||||
|
children: (FakeElement | FakeNode)[] = [];
|
||||||
|
attrs = new Map<string, string>();
|
||||||
|
tag: string;
|
||||||
|
constructor(tag: string, attrs: Record<string, string> = {}) { this.tag = tag; for (const [k, v] of Object.entries(attrs)) this.attrs.set(k, v); }
|
||||||
|
add<T extends FakeElement | FakeNode>(child: T): T { child.parentElement = this; this.children.push(child); return child; }
|
||||||
|
text(value: string): FakeNode { return this.add({ nodeType: 3, nodeValue: value, parentElement: null }); }
|
||||||
|
getAttribute(name: string) { return this.attrs.get(name) ?? null; }
|
||||||
|
setAttribute(name: string, value: string) { this.attrs.set(name, value); }
|
||||||
|
hasAttribute(name: string) { return this.attrs.has(name); }
|
||||||
|
closest(selector: string): FakeElement | null {
|
||||||
|
const tags = selector.split(',').map(part => part.trim());
|
||||||
|
for (let el: FakeElement | null = this; el; el = el.parentElement) if (tags.includes(el.tag)) return el;
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
all(): FakeElement[] { return this.children.flatMap(child => (child instanceof FakeElement ? [child, ...child.all()] : [])); }
|
||||||
|
querySelectorAll(selector: string) { const name = selector.slice(1, -1); return this.all().filter(el => el.attrs.has(name)); }
|
||||||
|
textNodes(): FakeNode[] { return this.children.flatMap(child => (child instanceof FakeElement ? child.textNodes() : [child])); }
|
||||||
|
}
|
||||||
|
function fakeDocument() {
|
||||||
|
const root = new FakeElement('html');
|
||||||
|
let callback: (records: unknown[]) => void = () => {};
|
||||||
|
let observing = false;
|
||||||
|
const win = {
|
||||||
|
document: {
|
||||||
|
documentElement: root,
|
||||||
|
createTreeWalker(element: FakeElement) {
|
||||||
|
const nodes = element.textNodes();
|
||||||
|
let index = -1;
|
||||||
|
return { get currentNode() { return nodes[index]; }, nextNode() { index += 1; return index < nodes.length; } };
|
||||||
|
},
|
||||||
|
querySelectorAll: (selector: string) => [...(root.attrs.has(selector.slice(1, -1)) ? [root] : []), ...root.querySelectorAll(selector)],
|
||||||
|
},
|
||||||
|
MutationObserver: class { constructor(fn: (records: unknown[]) => void) { callback = fn; } observe() { observing = true; } disconnect() { observing = false; } },
|
||||||
|
};
|
||||||
|
return { root, win: win as unknown as Window, mutate: (records: unknown[]) => { if (observing) callback(records); } };
|
||||||
|
}
|
||||||
|
|
||||||
|
test('text net: exact texts and attributes are replaced once, rewritten nodes are looked at again, dispose restores', () => {
|
||||||
|
const { root, win, mutate } = fakeDocument();
|
||||||
|
const nav = root.add(new FakeElement('nav', { 'aria-label': 'Trending' }));
|
||||||
|
const label = nav.text('Trending');
|
||||||
|
const other = nav.text('Trending topics today');
|
||||||
|
const script = root.add(new FakeElement('script'));
|
||||||
|
const code = script.text('Trending');
|
||||||
|
const net = installTextNet(win, { map: new Map([['Trending', 'Trendish'], ['News', 'Newsish']]), skip: 'script' });
|
||||||
|
assert.equal(label.nodeValue, 'Trendish');
|
||||||
|
assert.equal(nav.getAttribute('aria-label'), 'Trendish');
|
||||||
|
assert.equal(other.nodeValue, 'Trending topics today', 'only an exact text is a match');
|
||||||
|
assert.equal(code.nodeValue, 'Trending', 'skipped elements are not touched');
|
||||||
|
assert.equal(net.count(), 2);
|
||||||
|
// Our own writes come back as mutations: nothing changes further.
|
||||||
|
mutate([{ type: 'characterData', target: label }, { type: 'attributes', target: nav, attributeName: 'aria-label' }]);
|
||||||
|
assert.equal(label.nodeValue, 'Trendish');
|
||||||
|
assert.equal(net.count(), 2);
|
||||||
|
// A node added later, and the site rewriting a node.
|
||||||
|
const added = new FakeElement('div', { title: 'News' });
|
||||||
|
const news = added.text('News');
|
||||||
|
root.add(added);
|
||||||
|
mutate([{ type: 'childList', addedNodes: [added] }]);
|
||||||
|
assert.equal(news.nodeValue, 'Newsish');
|
||||||
|
assert.equal(added.getAttribute('title'), 'Newsish');
|
||||||
|
label.nodeValue = 'News';
|
||||||
|
mutate([{ type: 'characterData', target: label }]);
|
||||||
|
assert.equal(label.nodeValue, 'Newsish');
|
||||||
|
label.nodeValue = 'Something of the site';
|
||||||
|
mutate([{ type: 'characterData', target: label }]);
|
||||||
|
assert.equal(label.nodeValue, 'Something of the site');
|
||||||
|
assert.equal(net.count(), 3);
|
||||||
|
net.dispose();
|
||||||
|
assert.equal(news.nodeValue, 'News');
|
||||||
|
assert.equal(added.getAttribute('title'), 'News');
|
||||||
|
assert.equal(nav.getAttribute('aria-label'), 'Trending');
|
||||||
|
assert.equal(label.nodeValue, 'Something of the site');
|
||||||
|
assert.equal(net.count(), 0);
|
||||||
|
// Disposed: later mutations do nothing.
|
||||||
|
label.nodeValue = 'Trending';
|
||||||
|
mutate([{ type: 'characterData', target: label }]);
|
||||||
|
assert.equal(label.nodeValue, 'Trending');
|
||||||
|
});
|
||||||
|
|
@ -112,6 +112,7 @@ const Shell = createShell({ ...site, panel: { id: 'app', title: 'wayBetter', com
|
||||||
|
|
||||||
- `waybetter store build --config store.config.mjs`: builds an app's part of the shared store folder (its items only, validated with the apps' own rules).
|
- `waybetter store build --config store.config.mjs`: builds an app's part of the shared store folder (its items only, validated with the apps' own rules).
|
||||||
- `waybetter store sign [--init] --key <private key>` / `waybetter store verify --pub <key file>`: Ed25519 signature over the exact catalog bytes; any listed public key verifies (rotation). The private key is never printed or copied.
|
- `waybetter store sign [--init] --key <private key>` / `waybetter store verify --pub <key file>`: Ed25519 signature over the exact catalog bytes; any listed public key verifies (rotation). The private key is never printed or copied.
|
||||||
|
- `waybetter strings check <pack> [--inventory <file>] [--verbose]`: validates a string pack (data that replaces the wrapped site's own UI strings: `waybetter/runtime/stringPacks`, `waybetter/runtime/stringPackHook`, `waybetter/runtime/textNet`) against the site's string inventory and prints its coverage; the inventories otherwise come from `waybetter.strings.json` in the app's root.
|
||||||
- `waybetter i18n check [--write] [--verbose] [locale…]`: checks an app's catalogs (and the framework's in the same languages) with the `waybetter/i18n/check` rules, bundled plugins' inline English and the generated files; `--write` regenerates `languages.json` and the native localization files (macOS, Linux). `waybetter i18n extract [--missing]`: keys the code looks up that English lacks, and English keys without a literal lookup. Both read `waybetter.i18n.json` in the app's root (`locales`, `sources`, `nativeSources`, `bundledPlugins`, `runtimeKeys`, `supported`, `native`; schema: `i18n-config.schema.json`). `supported` lists the languages in the supported tier; every other catalog gets `tier: "community"` in the generated `languages.json` (`languages.schema.json`), and an app resolves its automatic language choices only to supported ones (`createLanguage({ supported })`).
|
- `waybetter i18n check [--write] [--verbose] [locale…]`: checks an app's catalogs (and the framework's in the same languages) with the `waybetter/i18n/check` rules, bundled plugins' inline English and the generated files; `--write` regenerates `languages.json` and the native localization files (macOS, Linux). `waybetter i18n extract [--missing]`: keys the code looks up that English lacks, and English keys without a literal lookup. Both read `waybetter.i18n.json` in the app's root (`locales`, `sources`, `nativeSources`, `bundledPlugins`, `runtimeKeys`, `supported`, `native`; schema: `i18n-config.schema.json`). `supported` lists the languages in the supported tier; every other catalog gets `tier: "community"` in the generated `languages.json` (`languages.schema.json`), and an app resolves its automatic language choices only to supported ones (`createLanguage({ supported })`).
|
||||||
- `waybetter release windows|macos`: signed native releases and the update manifest (`waybetter.release.json`). The Windows build runs path-free (below) and every artifact must pass the privacy scan before anything is copied or signed.
|
- `waybetter release windows|macos`: signed native releases and the update manifest (`waybetter.release.json`). The Windows build runs path-free (below) and every artifact must pass the privacy scan before anything is copied or signed.
|
||||||
- `waybetter build-env -- <command>`: runs a build (`tauri build`, `cargo build`) so the binary holds none of the machine's paths: the home folder, the workspace, `CARGO_HOME`, `RUSTUP_HOME` and `CARGO_TARGET_DIR` become `/home`, `/src`, `/cargo`, `/rustup` and `/target` (`--remap-path-prefix` appended to the flags already in `RUSTFLAGS`/`CARGO_ENCODED_RUSTFLAGS`, `/PDBALTPATH` on MSVC; for C and C++ a build script compiles, `-ffile-prefix-map` on gcc and clang and, on MSVC, `/d1trimfile:<root>` in `CL` and `/PDBALTPATH` in `LINK`, which reach the cc crate, CMake and MSBuild alike). Without a command it prints that environment (`--json`).
|
- `waybetter build-env -- <command>`: runs a build (`tauri build`, `cargo build`) so the binary holds none of the machine's paths: the home folder, the workspace, `CARGO_HOME`, `RUSTUP_HOME` and `CARGO_TARGET_DIR` become `/home`, `/src`, `/cargo`, `/rustup` and `/target` (`--remap-path-prefix` appended to the flags already in `RUSTFLAGS`/`CARGO_ENCODED_RUSTFLAGS`, `/PDBALTPATH` on MSVC; for C and C++ a build script compiles, `-ffile-prefix-map` on gcc and clang and, on MSVC, `/d1trimfile:<root>` in `CL` and `/PDBALTPATH` in `LINK`, which reach the cc crate, CMake and MSBuild alike). Without a command it prints that environment (`--json`).
|
||||||
|
|
|
||||||
|
|
@ -2,13 +2,15 @@
|
||||||
// The `waybetter` command: `waybetter store build|sign|verify` (cli/store.mjs),
|
// The `waybetter` command: `waybetter store build|sign|verify` (cli/store.mjs),
|
||||||
// `waybetter release windows|macos` (cli/release.mjs), `waybetter i18n check|extract`
|
// `waybetter release windows|macos` (cli/release.mjs), `waybetter i18n check|extract`
|
||||||
// (cli/i18n.mjs), `waybetter privacy-scan` (cli/privacy.mjs) and `waybetter build-env`
|
// (cli/i18n.mjs), `waybetter privacy-scan` (cli/privacy.mjs) and `waybetter build-env`
|
||||||
// (cli/buildenv.mjs), `waybetter dev` (cli/dev.mjs: watchers + tauri dev).
|
// (cli/buildenv.mjs), `waybetter dev` (cli/dev.mjs: watchers + tauri dev), `waybetter strings check`
|
||||||
|
// (cli/strings.mjs).
|
||||||
import { readFileSync } from 'node:fs';
|
import { readFileSync } from 'node:fs';
|
||||||
import { BUILD_ENV_USAGE, buildEnvCommand } from '../cli/buildenv.mjs';
|
import { BUILD_ENV_USAGE, buildEnvCommand } from '../cli/buildenv.mjs';
|
||||||
import { DEV_USAGE, devCommand } from '../cli/dev.mjs';
|
import { DEV_USAGE, devCommand } from '../cli/dev.mjs';
|
||||||
import { I18N_USAGE, i18nCommand } from '../cli/i18n.mjs';
|
import { I18N_USAGE, i18nCommand } from '../cli/i18n.mjs';
|
||||||
import { PRIVACY_USAGE, privacyCommand } from '../cli/privacy.mjs';
|
import { PRIVACY_USAGE, privacyCommand } from '../cli/privacy.mjs';
|
||||||
import { STORE_USAGE, storeCommand } from '../cli/store.mjs';
|
import { STORE_USAGE, storeCommand } from '../cli/store.mjs';
|
||||||
|
import { STRINGS_USAGE, stringsCommand } from '../cli/strings.mjs';
|
||||||
import { RELEASE_USAGE, releaseCommand } from '../cli/release.mjs';
|
import { RELEASE_USAGE, releaseCommand } from '../cli/release.mjs';
|
||||||
|
|
||||||
const [area, ...rest] = process.argv.slice(2);
|
const [area, ...rest] = process.argv.slice(2);
|
||||||
|
|
@ -19,6 +21,8 @@ try {
|
||||||
await releaseCommand(rest);
|
await releaseCommand(rest);
|
||||||
} else if (area === 'i18n') {
|
} else if (area === 'i18n') {
|
||||||
process.exitCode = await i18nCommand(rest);
|
process.exitCode = await i18nCommand(rest);
|
||||||
|
} else if (area === 'strings') {
|
||||||
|
process.exitCode = await stringsCommand(rest);
|
||||||
} else if (area === 'privacy-scan') {
|
} else if (area === 'privacy-scan') {
|
||||||
process.exitCode = privacyCommand(rest);
|
process.exitCode = privacyCommand(rest);
|
||||||
} else if (area === 'build-env') {
|
} else if (area === 'build-env') {
|
||||||
|
|
@ -28,7 +32,7 @@ try {
|
||||||
} else if (area === '--version' || area === '-v') {
|
} else if (area === '--version' || area === '-v') {
|
||||||
console.log(JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version);
|
console.log(JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version);
|
||||||
} else {
|
} else {
|
||||||
console.log(`${STORE_USAGE}\n\n${RELEASE_USAGE}\n\n${I18N_USAGE}\n\n${PRIVACY_USAGE}\n\n${BUILD_ENV_USAGE}
|
console.log(`${STORE_USAGE}\n\n${RELEASE_USAGE}\n\n${I18N_USAGE}\n\n${STRINGS_USAGE}\n\n${PRIVACY_USAGE}\n\n${BUILD_ENV_USAGE}
|
||||||
|
|
||||||
${DEV_USAGE}`);
|
${DEV_USAGE}`);
|
||||||
if (area && area !== '--help' && area !== '-h') process.exitCode = 1;
|
if (area && area !== '--help' && area !== '-h') process.exitCode = 1;
|
||||||
|
|
|
||||||
121
packages/waybetter/cli/strings.mjs
Normal file
121
packages/waybetter/cli/strings.mjs
Normal file
|
|
@ -0,0 +1,121 @@
|
||||||
|
// `waybetter strings check`: a string pack against a site's string inventory, with the
|
||||||
|
// framework's rules (waybetter/runtime/stringPacks: the envelope, only inventory keys, parameter
|
||||||
|
// names, plural and select kinds, no markup) and its coverage.
|
||||||
|
//
|
||||||
|
// waybetter strings check pack.strings.json --inventory strings/x.web.inventory.json
|
||||||
|
// waybetter strings check pack.strings.json the inventory comes from the config
|
||||||
|
// waybetter strings check pack.strings.json --verbose list every key that was refused or unknown
|
||||||
|
//
|
||||||
|
// Config: waybetter.strings.json in the app's root (or --config <file>); paths are relative to it.
|
||||||
|
//
|
||||||
|
// { "inventories": ["strings/x.web.inventory.json"] }
|
||||||
|
//
|
||||||
|
// The pack names its site and catalog; the inventory with the same two is used. Exit 1 when the
|
||||||
|
// envelope is invalid, a value is refused or a key is unknown (none of those is ever applied).
|
||||||
|
import { existsSync, readFileSync } from 'node:fs';
|
||||||
|
import { dirname, isAbsolute, relative, resolve } from 'node:path';
|
||||||
|
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||||
|
|
||||||
|
export const STRINGS_CONFIG_FILE = 'waybetter.strings.json';
|
||||||
|
|
||||||
|
export const STRINGS_USAGE = `Usage:
|
||||||
|
waybetter strings check <pack> [--inventory <file>] [--verbose] [--config <file>]
|
||||||
|
Validates a string pack against the site's string inventory and prints its coverage.
|
||||||
|
|
||||||
|
Config: ${STRINGS_CONFIG_FILE} in the app's root (inventories).`;
|
||||||
|
|
||||||
|
class StringsError extends Error {}
|
||||||
|
|
||||||
|
const LISTED = 12;
|
||||||
|
|
||||||
|
/** The pack rules: the framework's sources inside its repository, else the built package. */
|
||||||
|
async function loadRules() {
|
||||||
|
const source = fileURLToPath(new URL('../../runtime/src/stringPacks.ts', import.meta.url));
|
||||||
|
const built = fileURLToPath(new URL('../dist/runtime/stringPacks.js', import.meta.url));
|
||||||
|
const file = existsSync(source) ? source : built;
|
||||||
|
if (!existsSync(file)) throw new StringsError('the string pack rules are not built (run the package build)');
|
||||||
|
return import(pathToFileURL(file).href);
|
||||||
|
}
|
||||||
|
|
||||||
|
function readJson(file, what) {
|
||||||
|
if (!existsSync(file)) throw new StringsError(`${what} not found: ${file}`);
|
||||||
|
try {
|
||||||
|
return JSON.parse(readFileSync(file, 'utf8'));
|
||||||
|
} catch (error) {
|
||||||
|
throw new StringsError(`${what} is not JSON: ${file} (${error instanceof Error ? error.message : error})`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The inventories a config lists, as absolute paths. */
|
||||||
|
export function loadStringsConfig(cwd, file) {
|
||||||
|
const path = resolve(cwd, file ?? STRINGS_CONFIG_FILE);
|
||||||
|
if (!existsSync(path)) throw new StringsError(`${relative(cwd, path) || path} not found: pass --inventory <file> (see \`waybetter strings --help\`)`);
|
||||||
|
const config = readJson(path, 'the config');
|
||||||
|
if (!Array.isArray(config.inventories) || config.inventories.some(item => typeof item !== 'string')) throw new StringsError(`${relative(cwd, path)}: "inventories" must be a list of files`);
|
||||||
|
return { inventories: config.inventories.map(item => resolve(dirname(path), item)) };
|
||||||
|
}
|
||||||
|
|
||||||
|
const percent = (part, whole) => (whole ? `${(part / whole * 100).toFixed(part && part < whole / 1000 ? 2 : 1)}%` : 'n/a');
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks one pack. `inventories` are inventory files; the one for the pack's site and catalog is
|
||||||
|
* used. Returns the lines to print, the number of errors and the compiled report.
|
||||||
|
*/
|
||||||
|
export async function checkPack(packFile, inventories, { verbose = false } = {}) {
|
||||||
|
const rules = await loadRules();
|
||||||
|
const pack = readJson(packFile, 'the pack');
|
||||||
|
const lines = [];
|
||||||
|
const issues = rules.checkPackShape(pack);
|
||||||
|
for (const issue of issues) lines.push(`${issue.level}: ${issue.message}`);
|
||||||
|
if (issues.some(issue => issue.level === 'error')) return { lines, errors: issues.filter(issue => issue.level === 'error').length, report: null, coverage: null };
|
||||||
|
const loaded = inventories.map(file => ({ file, inventory: readJson(file, 'the inventory') }));
|
||||||
|
const match = loaded.find(item => item.inventory.site === pack.site && item.inventory.catalog === pack.catalog);
|
||||||
|
if (!match) throw new StringsError(`no inventory for ${pack.site}/${pack.catalog} (have: ${loaded.map(item => `${item.inventory.site}/${item.inventory.catalog}`).join(', ') || 'none'})`);
|
||||||
|
if (match.inventory.format !== 1 || typeof match.inventory.entries !== 'object') throw new StringsError(`${match.file} is not a string inventory`);
|
||||||
|
const compiled = rules.compilePack(pack, match.inventory);
|
||||||
|
const { report } = compiled;
|
||||||
|
const coverage = rules.coverage(match.inventory, compiled);
|
||||||
|
lines.push(`${pack.name} (${pack.language}) for ${pack.site}/${pack.catalog}, inventory build ${match.inventory.build ?? 'unknown'}`);
|
||||||
|
lines.push(`${report.total} strings: ${report.valid} valid, ${report.rejected.length} refused, ${report.unknown.length} unknown`);
|
||||||
|
const list = (items, render) => {
|
||||||
|
for (const item of verbose ? items : items.slice(0, LISTED)) lines.push(` ${render(item)}`);
|
||||||
|
if (!verbose && items.length > LISTED) lines.push(` … ${items.length - LISTED} more (--verbose lists all)`);
|
||||||
|
};
|
||||||
|
if (report.rejected.length) {
|
||||||
|
lines.push('Refused (not applied, the site\'s text stays):');
|
||||||
|
list(report.rejected, item => `${item.id}: ${item.reason}`);
|
||||||
|
}
|
||||||
|
if (report.unknown.length) {
|
||||||
|
lines.push('Unknown keys (not in the inventory, never applied):');
|
||||||
|
list(report.unknown, id => id);
|
||||||
|
}
|
||||||
|
lines.push('Coverage:');
|
||||||
|
for (const [group, numbers] of Object.entries(coverage)) {
|
||||||
|
lines.push(` ${group.padEnd(8)} ${numbers.translated} of ${numbers.translatable} translatable strings (${percent(numbers.translated, numbers.translatable)})${numbers.closed ? `; ${numbers.closed} not translatable yet (rich text, formatters)` : ''}`);
|
||||||
|
}
|
||||||
|
return { lines, errors: report.rejected.length + report.unknown.length, report, coverage };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function stringsCommand(args, { cwd = process.cwd(), log = console.log } = {}) {
|
||||||
|
const [action, ...rest] = args;
|
||||||
|
const value = flag => {
|
||||||
|
const at = rest.indexOf(flag);
|
||||||
|
if (at < 0) return undefined;
|
||||||
|
if (!rest[at + 1] || rest[at + 1].startsWith('--')) throw new StringsError(`${flag} needs a file`);
|
||||||
|
return rest[at + 1];
|
||||||
|
};
|
||||||
|
if (action === 'check') {
|
||||||
|
const inventory = value('--inventory');
|
||||||
|
const config = value('--config');
|
||||||
|
const taken = new Set([inventory, config].filter(Boolean));
|
||||||
|
const [pack, ...extra] = rest.filter(arg => !arg.startsWith('--') && !taken.has(arg));
|
||||||
|
if (!pack || extra.length) throw new StringsError('check needs one pack file');
|
||||||
|
const at = file => (isAbsolute(file) ? file : resolve(cwd, file));
|
||||||
|
const inventories = inventory ? [at(inventory)] : loadStringsConfig(cwd, config).inventories;
|
||||||
|
const result = await checkPack(at(pack), inventories, { verbose: rest.includes('--verbose') });
|
||||||
|
for (const line of result.lines) log(line);
|
||||||
|
return result.errors ? 1 : 0;
|
||||||
|
}
|
||||||
|
log(STRINGS_USAGE);
|
||||||
|
return action && action !== '--help' && action !== '-h' ? 1 : 0;
|
||||||
|
}
|
||||||
80
packages/waybetter/tests/strings-cli.test.mjs
Normal file
80
packages/waybetter/tests/strings-cli.test.mjs
Normal file
|
|
@ -0,0 +1,80 @@
|
||||||
|
// `waybetter strings check` on a small site inventory.
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { execFileSync, spawnSync } from 'node:child_process';
|
||||||
|
import { mkdtempSync, writeFileSync } from 'node:fs';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import test from 'node:test';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import { checkPack, loadStringsConfig, stringsCommand } from '../cli/strings.mjs';
|
||||||
|
|
||||||
|
const BIN = fileURLToPath(new URL('../bin/waybetter.mjs', import.meta.url));
|
||||||
|
const json = (file, value) => writeFileSync(file, `${JSON.stringify(value, null, 2)}\n`);
|
||||||
|
|
||||||
|
/** An app folder: an inventory (2 texts, 1 plural message, 1 rich entry, 1 emoji name), a config, a pack. */
|
||||||
|
function app(strings, extra = {}) {
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), 'wb-strings-'));
|
||||||
|
json(join(dir, 'demo.web.inventory.json'), { format: 1, site: 'demo', catalog: 'web', base: 'en', build: 'b7', entries: {
|
||||||
|
a0000001: { kind: 'text', hash: '00000000001' },
|
||||||
|
a0000002: { kind: 'text', hash: '00000000002' },
|
||||||
|
b0000001: { kind: 'message', params: ['count'], plural: ['count'], hash: '00000000003' },
|
||||||
|
c0000001: { kind: 'rich', hash: '00000000004' },
|
||||||
|
e0000001: { kind: 'text', group: 'emoji', hash: '00000000005' },
|
||||||
|
} });
|
||||||
|
json(join(dir, 'waybetter.strings.json'), { inventories: ['demo.web.inventory.json'] });
|
||||||
|
json(join(dir, 'pack.json'), { format: 1, site: 'demo', catalog: 'web', language: 'en-x-test', name: 'Testish', authors: ['A tester'], strings, ...extra });
|
||||||
|
return dir;
|
||||||
|
}
|
||||||
|
|
||||||
|
test('check: a valid pack passes and prints its coverage per group', async () => {
|
||||||
|
const dir = app({ a0000001: 'Hoom', b0000001: '{count, plural, one {# postle} other {# postles}}' });
|
||||||
|
const result = await checkPack(join(dir, 'pack.json'), [join(dir, 'demo.web.inventory.json')]);
|
||||||
|
assert.equal(result.errors, 0);
|
||||||
|
const text = result.lines.join('\n');
|
||||||
|
assert.match(text, /Testish \(en-x-test\) for demo\/web, inventory build b7/);
|
||||||
|
assert.match(text, /2 strings: 2 valid, 0 refused, 0 unknown/);
|
||||||
|
assert.match(text, /all\s+2 of 4 translatable strings \(50\.0%\); 1 not translatable yet/);
|
||||||
|
assert.match(text, /main\s+2 of 3 translatable strings/);
|
||||||
|
assert.match(text, /emoji\s+0 of 1 translatable strings \(0\.0%\)/);
|
||||||
|
assert.deepEqual(result.coverage.all, { translatable: 4, translated: 2, closed: 1 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('check: unknown keys and refused values are errors, listed with the reason', async () => {
|
||||||
|
const dir = app({ a0000001: '<b>Hoom</b>', b0000001: '{total} postles', c0000001: 'Rich', zzzzzzzz: 'Nope', a0000002: 'Fine' });
|
||||||
|
const lines = [];
|
||||||
|
const code = await stringsCommand(['check', 'pack.json'], { cwd: dir, log: line => lines.push(line) });
|
||||||
|
assert.equal(code, 1);
|
||||||
|
const text = lines.join('\n');
|
||||||
|
assert.match(text, /5 strings: 1 valid, 3 refused, 1 unknown/);
|
||||||
|
assert.match(text, /a0000001: markup is not allowed/);
|
||||||
|
assert.match(text, /b0000001: \{total\} is not a parameter of the original \(it has: count\)/);
|
||||||
|
assert.match(text, /c0000001: not translatable yet \(rich\)/);
|
||||||
|
assert.match(text, /Unknown keys[^\n]*\n\s+zzzzzzzz/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('check: an invalid envelope stops before the inventory; a pack for another catalog has no inventory', async () => {
|
||||||
|
const dir = app({}, { language: 'not a tag', format: 3 });
|
||||||
|
const result = await checkPack(join(dir, 'pack.json'), [join(dir, 'demo.web.inventory.json')]);
|
||||||
|
assert.equal(result.errors, 2);
|
||||||
|
assert.equal(result.report, null);
|
||||||
|
const other = app({ a0000001: 'x' }, { catalog: 'guest' });
|
||||||
|
await assert.rejects(stringsCommand(['check', 'pack.json'], { cwd: other, log() {} }), /no inventory for demo\/guest \(have: demo\/web\)/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('command: the config, --inventory, usage and the binary', async () => {
|
||||||
|
const dir = app({ a0000001: 'Hoom' });
|
||||||
|
assert.deepEqual(loadStringsConfig(dir).inventories, [join(dir, 'demo.web.inventory.json')]);
|
||||||
|
assert.throws(() => loadStringsConfig(tmpdir(), 'nope.json'), /pass --inventory/);
|
||||||
|
const lines = [];
|
||||||
|
assert.equal(await stringsCommand(['check', 'pack.json', '--inventory', 'demo.web.inventory.json'], { cwd: dir, log: line => lines.push(line) }), 0);
|
||||||
|
assert.equal(await stringsCommand([], { log: line => lines.push(line) }), 0);
|
||||||
|
assert.equal(await stringsCommand(['nope'], { log() {} }), 1);
|
||||||
|
await assert.rejects(stringsCommand(['check'], { cwd: dir, log() {} }), /one pack file/);
|
||||||
|
await assert.rejects(stringsCommand(['check', 'pack.json', '--inventory'], { cwd: dir, log() {} }), /--inventory needs a file/);
|
||||||
|
const out = execFileSync(process.execPath, [BIN, 'strings', 'check', 'pack.json'], { cwd: dir, encoding: 'utf8' });
|
||||||
|
assert.match(out, /1 strings: 1 valid/);
|
||||||
|
const bad = app({ zzzzzzzz: 'x' });
|
||||||
|
const run = spawnSync(process.execPath, [BIN, 'strings', 'check', 'pack.json'], { cwd: bad, encoding: 'utf8' });
|
||||||
|
assert.equal(run.status, 1);
|
||||||
|
assert.match(execFileSync(process.execPath, [BIN, '--help'], { encoding: 'utf8' }), /waybetter strings check/);
|
||||||
|
});
|
||||||
Loading…
Add table
Add a link
Reference in a new issue