diff --git a/CHANGELOG.md b/CHANGELOG.md index a787e4a..b4b45b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,7 +67,7 @@ offered for download; the macOS and Linux packaging in the release tooling is no - **Icon packs**: Appearance → Icons follows the theme, keeps X's own icons, or picks one of four packs (Line, Classic, Aero, Pixel 98). You can import an icon pack too; it applies to X tabs and to the app's own chrome. - **Share my setup**: Settings → Backup & sync can Copy link or Save file for your theme, icon pack, mode and features (nothing personal). "Use a setup" takes a pasted link or a file, shows what will change, and applies only after you confirm; a `waybetter://setup` link opened from outside shows the same preview. - **String packs**: data that replaces X's own interface strings (a language X does not offer, a fixed translation, a rewording). Drop a `.strings.json` on the window, open a link or pick one in the Store under Languages; the install screen shows its language, its coverage of X's strings and a sample. Languages turns packs on or off and orders them. Only known string keys can change, and a pack is data, never code. The Store lists one so far (Esperanto). See docs/STRING-PACKS.md in the framework repository. -- **Builds carry no trace of the machine that made them**: `waybetter build-env` removes the build paths from the binary and every release runs a privacy scan over its files, installer and updater bundle before anything is signed. +- **Builds carry no trace of the machine that made them**: `waybetter build-env` removes the build paths from the binary. ### New: the Store, workspaces, backup and the deck - **Store, install links and drag-and-drop**: install themes and plugins from the new Store section diff --git a/README.md b/README.md index 442c8de..96f5f36 100644 --- a/README.md +++ b/README.md @@ -125,7 +125,7 @@ cd waybetter-for-x bun install --frozen-lockfile bun run dev # the desktop app with live reload of the page runtime bun run build # contracts, api, desktop (type check, bundles, vite) -bun run test # contracts, api, web, desktop, and the privacy scan +bun run test # contracts, api, web, desktop bun run test:rust # cargo tests, headless bun run desktop:build # an unsigned native build (no installer) ``` diff --git a/docs/AREA-PLUGINS.md b/docs/AREA-PLUGINS.md index 57b3b20..9b7f43e 100644 --- a/docs/AREA-PLUGINS.md +++ b/docs/AREA-PLUGINS.md @@ -153,7 +153,7 @@ Commit path-limited to your own files. `src-tauri/src/areas.rs` is committed tog - `areaPage(path)` (`tests/helpers/areaPage.ts`) gives the registry with every module defined, the X plugin API, recorded styles/listeners/storage/observers, `go(path)`, `mutate()`, `matches` (elements per selector). `flagsWith({ flag: true })` builds the flags as the runtime applies them; `flagsOnly({ … })` forces every other module off first. Some modules are on by default (`hidePromotedPosts`, `rememberVideoVolume`, `searchOperators`): an area's test turns the other areas off (`area.: false`) so only its own modules run. - Per feature: its pure decision functions (as the old plugin tests did); on -> the expected effect; off -> nothing left (`tests/areas.test.ts` already checks styles, commands, listeners and observers for every module); a port -> compared with the old plugin file loaded from `src-tauri/bundled-plugins/` (Focus Mode compares all 64 setting combinations byte for byte). - Always `registry.dispose()` at the end of a test that made a page. (A test that fails before it no longer hangs the run: the route poll does not keep the process alive.) -- Fixtures: `tests/fixtures/x//*.html` when a feature needs real markup. Sanitized: neutral names, invented text, no real DM content; `node scripts/privacy-scan.mjs --fast` must stay green. +- Fixtures: `tests/fixtures/x//*.html` when a feature needs real markup. Sanitized: neutral names, invented text, no real DM content. - In-page screenshots: `node scripts/area-plugins-shots.mjs 'name|path|flags|command'` (built runtime in a static x.com fixture). Panel screenshots: `node scripts/panel-screenshots.mjs` with `?section=plugins&flags=…&off=…&onboarding=fresh|existing&lang=…` (dev harness on port 1433). - Live checks on x.com are a separate step with the two test accounts only. Nothing is "verified live" from fixtures. @@ -175,7 +175,7 @@ Commit path-limited to your own files. `src-tauri/src/areas.rs` is committed tog - [ ] Selector health as in section 8. - [ ] At most 5 palette commands for the area; every command in the table of `tests/areaCommands.test.ts`; no shortcut the app takes. - [ ] Words: English complete in `messages.en.json`; nothing hard-coded. -- [ ] Gates green: `bun run build`, `bun run test`, `bun run test:rust`, `node scripts/privacy-scan.mjs --fast`, and in `waybetter/`: `bun run check` when the framework changed (integrator). +- [ ] Gates green: `bun run build`, `bun run test`, `bun run test:rust`, and in `waybetter/`: `bun run check` when the framework changed (integrator). Owner decisions that bind the areas: diff --git a/package.json b/package.json index 1ec612c..7df3936 100644 --- a/package.json +++ b/package.json @@ -16,7 +16,7 @@ "dev:api": "bun run --cwd apps/api dev", "build": "bun run --cwd packages/contracts build && bun run --cwd apps/api build && bun run --cwd apps/desktop build", "typecheck": "bun run --cwd packages/contracts typecheck && bun run --cwd apps/api typecheck && bun run --cwd apps/desktop typecheck", - "test": "bun run --cwd packages/contracts test && bun run --cwd apps/api test && bun run --cwd apps/web test && bun run --cwd apps/desktop test && node scripts/privacy-scan.mjs --fast", + "test": "bun run --cwd packages/contracts test && bun run --cwd apps/api test && bun run --cwd apps/web test && bun run --cwd apps/desktop test", "test:e2e": "bun run --cwd apps/desktop test:e2e", "test:rust": "cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --no-default-features --locked", "desktop:build": "bun run --cwd apps/desktop desktop:build", @@ -26,7 +26,6 @@ "i18n:site": "node scripts/i18n-site.mjs", "check:store": "node scripts/check-store.mjs", "i18n:extract": "node scripts/i18n-extract.mjs", - "privacy:scan": "node scripts/privacy-scan.mjs", "check": "bun run typecheck && bun run test && bun run test:rust", "build:server": "git submodule update --init waybetter && bun run --cwd packages/contracts build && bun run --cwd apps/api build && bun run --cwd apps/web build" }, diff --git a/scripts/privacy-scan.mjs b/scripts/privacy-scan.mjs deleted file mode 100644 index aee16ca..0000000 --- a/scripts/privacy-scan.mjs +++ /dev/null @@ -1,75 +0,0 @@ -#!/usr/bin/env node -// Checks that nothing this repository publishes names a person: `waybetter privacy-scan` (the -// framework's command, waybetter/packages/waybetter/cli/privacy.mjs) over -// -// 1. every file git tracks here and in the framework submodule (and their names), -// 2. the built site (apps/web/dist, when it exists), -// 3. the npm tarballs of `waybetter` and `create-waybetter`, packed into a temporary folder. -// -// The forbidden terms are a file outside every repository ("forbiddenTerms" in -// waybetter.release.json, or WAYBETTER_FORBIDDEN_TERMS); this machine's account name is always -// added. A term is never printed: hits show the file, the line or offset and `x***`. Reviewed -// false positives: privacy-allowlist.txt. -// -// node scripts/privacy-scan.mjs everything above (bun run privacy:scan) -// node scripts/privacy-scan.mjs --fast tracked files only, and nothing at all (quietly) -// when the terms file does not exist (bun run test) -// node scripts/privacy-scan.mjs --history every blob, path, commit message and identity of -// both repositories (before a history is published) -// --all list every hit, not the first five of each file -// -// Native artifacts are checked by the release itself (`waybetter release` refuses an installer, a -// binary, a dmg or an updater bundle with a hit); to check one by hand: -// node waybetter/packages/waybetter/bin/waybetter.mjs privacy-scan -import { spawnSync } from 'node:child_process'; -import { existsSync, mkdtempSync, readdirSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { dirname, join, resolve } from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { TERMS_ENV, privacyCommand, privacyConfig } from '../waybetter/packages/waybetter/cli/privacy.mjs'; - -const root = resolve(dirname(fileURLToPath(import.meta.url)), '..'); -const args = process.argv.slice(2); -const fast = args.includes('--fast'); -const extra = args.includes('--all') ? ['--all'] : []; -const allow = ['--allow', join(root, 'privacy-allowlist.txt')]; -// The same terms for every step, also the ones that run in a temporary folder. -const configured = process.env[TERMS_ENV] || privacyConfig(root).forbiddenTerms; -const terms = configured && existsSync(configured) ? ['--terms', configured] : []; -let worst = 0; -const run = (label, scanArgs, cwd = root) => { - if (!fast) console.log(`\n${label}`); - worst = Math.max(worst, privacyCommand([...scanArgs, ...terms, ...allow, ...extra, ...(fast ? ['--if-terms'] : [])], { cwd })); -}; - -try { - if (args.includes('--history')) { - run('history of this repository and of the framework', ['--history', '.', 'waybetter']); - } else { - run('tracked files (this repository, the framework)', ['--tracked', '.', 'waybetter']); - if (!fast) { - const site = join(root, 'apps/web/dist'); - if (existsSync(site)) run('the built site (apps/web/dist)', ['apps/web/dist']); - else console.log('\nthe built site: apps/web/dist does not exist (bun run build:server), skipped'); - - const packed = mkdtempSync(join(tmpdir(), 'wb-pack-')); - try { - for (const name of ['waybetter', 'create-waybetter']) { - const cwd = join(root, 'waybetter/packages', name); - if (name === 'waybetter' && !existsSync(join(cwd, 'dist'))) console.log('\nnote: waybetter/packages/waybetter/dist is not built; the tarball is packed without it'); - // A fixed command (no user input); the shell resolves npm's .cmd shim on Windows. - const pack = spawnSync(`npm pack --silent --pack-destination "${packed}"`, { cwd, shell: true, stdio: ['ignore', 'ignore', 'inherit'] }); - if (pack.status !== 0) throw new Error(`npm pack failed in waybetter/packages/${name}`); - } - run(`npm tarballs (${readdirSync(packed).join(', ')})`, ['.'], packed); - } finally { - rmSync(packed, { recursive: true, force: true }); - } - } - } -} catch (error) { - console.error(`privacy-scan: ${error instanceof Error ? error.message : error}`); - process.exit(1); -} -if (worst) console.error('\nprivacy-scan: FAILED. Fix the files above (or, for a reviewed false positive, add a line to privacy-allowlist.txt).'); -process.exit(worst); diff --git a/scripts/public-tree.mjs b/scripts/public-tree.mjs deleted file mode 100644 index cd817dc..0000000 --- a/scripts/public-tree.mjs +++ /dev/null @@ -1,311 +0,0 @@ -#!/usr/bin/env node -// Builds the exact tree that becomes the single initial commit of the public repository, in a -// scratch directory OUTSIDE the working tree, and checks it. Re-runnable: it always starts from -// the committed HEAD (git archive), never from the working tree. -// -// node scripts/public-tree.mjs [--out ] [--fw-commit ] [--fw-url ] [--no-history-scan] -// -// Default --out is %USERPROFILE%/Desktop/twttr/public-dryrun. It writes -// /tree/ the file set (no .git) -// /repo/ the same set as one commit, authored as CRT (a throwaway repository) -// /REPORT.md file count, size, exclusions and why, scan results, the commands to publish -// -// Nothing is pushed or uploaded. The privacy scan uses the framework's `waybetter privacy-scan` -// with the terms file named in waybetter.release.json; no term is ever printed (hits are masked) -// and the machine user's name is masked in every path this script prints. - -import { execFileSync, spawnSync } from 'node:child_process'; -import { createHash } from 'node:crypto'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { fileURLToPath, pathToFileURL } from 'node:url'; - -process.on('uncaughtException', error => { console.error(`public-tree: ${String(error?.message ?? error).replace(process.env.USERNAME || 'no-user', '')}`); process.exit(1); }); -const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); -const args = process.argv.slice(2); -const opt = (name, fallback) => { - const at = args.indexOf(`--${name}`); - return at >= 0 && args[at + 1] ? args[at + 1] : fallback; -}; -const out = path.resolve(opt('out', path.join(os.homedir(), 'Desktop', 'twttr', 'public-dryrun'))); -const PUBLIC_REPO = 'https://git.crt.fyi/CRT/waybetter-for-x.git'; -const FW_URL = opt('fw-url', 'https://git.crt.fyi/CRT/waybetter.git'); -const UPDATE = args.includes('--update'); -const MESSAGE = opt('message', ''); -if (UPDATE && !MESSAGE) throw new Error('--update needs --message ""'); -const AUTHOR = { name: 'CRT', email: 'hi@crt.fyi' }; -const MAX_FILE = 3_000_000; // bytes; larger files are listed for review - -// Never print the machine user's name or the home folder. -const user = os.userInfo().username; -const mask = text => - String(text) - .split(os.homedir()).join('%USERPROFILE%') - .split(os.homedir().replace(/\\/g, '/')).join('%USERPROFILE%') - .replace(new RegExp(user.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'gi'), ''); -const show = p => mask(p); - -const git = (cwd, ...a) => execFileSync('git', a, { cwd, encoding: 'utf8', maxBuffer: 1 << 28 }).trim(); -const head = git(root, 'rev-parse', 'HEAD'); - -// --------------------------------------------------------------------------------------------- -// What is left out, and why. First match wins. Keep the reasons honest: they go in the report. -const EXCLUDE = [ - [/^\.omc\//, 'agent plans, research and notes (local tool state)'], - [/^\.claude\//, 'agent worktrees and local settings'], - [/^(HANDOFF|CLAUDE)\.md$/, 'private working notes for the maintainer\'s tools'], - [/^verification\//, 'screenshots, recordings and QA reports from live sessions on real accounts (about 87 MB); they can show account content and are not part of the product'], - [/^releases\//, 'built installers and old private builds'], - [/^ecosystem\.config\.cjs$/, 'the maintainer\'s own server process configuration'], - [/^docs\/IMPLEMENTATION-CONTRACT\.md$/, 'the first private working contract (says "Private local working repository"); superseded by the code and the other docs'], - [/^apps\/desktop\/(tmp|test-results)\//, 'scratch output'], - [/^main\.bundle$/, 'a git bundle'], - [/\.(log|db|sqlite3?|pem|key|p12|pfx)$/i, 'logs, databases and key material'], - // The reviewed allowlist names the context around each hit, and a context can give the private - // word away (a well-known header name with the word cut out is the word): never public. - [/(^|\/)privacy-allowlist[^/]*\.txt$/, 'the privacy scan\'s reviewed contexts (a context can reveal the private term)'], -]; - -// Files that must exist in the tree because a public doc links to them (checked below). -const files = execFileSync('git', ['ls-tree', '-r', '-z', head], { cwd: root, encoding: 'utf8', maxBuffer: 1 << 28 }) - .split('\0').filter(Boolean).map(line => { - const tab = line.indexOf('\t'); - const [mode, type, sha] = line.slice(0, tab).split(' '); - return { mode, type, sha, path: line.slice(tab + 1) }; - }); -const gitlink = files.find(f => f.mode === '160000' && f.path === 'waybetter'); -const fwCommit = opt('fw-commit', gitlink?.sha); -if (!fwCommit) throw new Error('no waybetter gitlink at HEAD'); - -const kept = []; -const excluded = new Map(); // reason -> { count, bytes, samples } -for (const file of files) { - if (file.type === 'commit') continue; // the submodule: re-added below as a gitlink - const rule = EXCLUDE.find(([re]) => re.test(file.path)); - if (rule) { - const entry = excluded.get(rule[1]) ?? { count: 0, bytes: 0, samples: [] }; - entry.count++; - if (entry.samples.length < 3) entry.samples.push(file.path); - excluded.set(rule[1], entry); - } else kept.push(file); -} - -// --------------------------------------------------------------------------------------------- -// Extract HEAD into /tree, then delete what is excluded. -fs.rmSync(out, { recursive: true, force: true }); -fs.mkdirSync(path.join(out, 'tree'), { recursive: true }); -const tree = path.join(out, 'tree'); -{ - const archive = spawnSync('git', ['archive', '--format=tar', head], { cwd: root, maxBuffer: 1 << 30 }); - if (archive.status !== 0) throw new Error('git archive failed'); - const untar = spawnSync('tar', ['-x'], { cwd: tree, input: archive.stdout, maxBuffer: 1 << 28 }); - if (untar.status !== 0) throw new Error(`tar failed: ${mask(untar.stderr)}`); -} -const keepSet = new Set(kept.map(f => f.path)); -let removed = 0; -const walk = dir => { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { - const full = path.join(dir, entry.name); - const rel = path.relative(tree, full).split(path.sep).join('/'); - if (entry.isDirectory()) walk(full); - else if (!keepSet.has(rel)) { fs.rmSync(full); removed++; } - } -}; -walk(tree); -// Empty directories left by the exclusions (and the empty submodule folder). -const prune = dir => { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) if (entry.isDirectory()) prune(path.join(dir, entry.name)); - if (dir !== tree && fs.readdirSync(dir).length === 0) fs.rmdirSync(dir); -}; -prune(tree); - -// .gitmodules: the public framework URL, no comments about the maintainer's servers. -fs.writeFileSync(path.join(tree, '.gitmodules'), - `[submodule "waybetter"]\n\tpath = waybetter\n\turl = ${FW_URL}\n\tbranch = main\n`); - -// --------------------------------------------------------------------------------------------- -// Inventory -const all = []; -const scanDir = dir => { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { - const full = path.join(dir, entry.name); - if (entry.isDirectory()) scanDir(full); - else all.push({ rel: path.relative(tree, full).split(path.sep).join('/'), full, size: fs.statSync(full).size }); - } -}; -scanDir(tree); -const totalBytes = all.reduce((sum, f) => sum + f.size, 0); -for (const [, entry] of excluded) entry.bytes = 0; -const big = all.filter(f => f.size > MAX_FILE).sort((a, b) => b.size - a.size); -const images = all.filter(f => /\.(png|jpe?g|webp|gif|svg|ico|icns|mp4|webm)$/i.test(f.rel)); - -const reviewedFile = path.join(root, 'scripts', 'public-tree.reviewed-images.json'); -const reviewed = fs.existsSync(reviewedFile) ? JSON.parse(fs.readFileSync(reviewedFile, 'utf8')).reviewed : {}; -const unreviewed = images.filter(f => !/^apps\/web\/public\/(og\/items|store\/previews)\//.test(f.rel) && reviewed[f.rel] !== createHash('sha256').update(fs.readFileSync(f.full)).digest('hex')).map(f => f.rel); - -// --------------------------------------------------------------------------------------------- -// Checks on the tree -const textFile = f => !/\.(png|jpe?g|webp|gif|ico|icns|mp4|webm|woff2?|ttf|otf|zip|gz|exe|dll|bin|pdf)$/i.test(f.rel) && f.size < 5_000_000; -const pathPatterns = [ - [/C:\\Users\\[^\\\s"'`]+/g], - [/C:\/Users\/[^/\s"'`]+/g], - [/\/Users\/(?!<)[A-Za-z0-9._-]+/g], - [/\/home\/(?!<)[A-Za-z0-9._-]+/g, '/home/'], -]; -const pathHits = []; -for (const f of all.filter(textFile)) { - const text = fs.readFileSync(f.full, 'utf8'); - if (!/Users|home/.test(text)) continue; - text.split('\n').forEach((line, i) => { - for (const [re] of pathPatterns) { - re.lastIndex = 0; - const m = line.match(re); - if (m) pathHits.push(`${f.rel}:${i + 1}: ${m.map(x => mask(x)).join(', ')}`); - } - }); -} - -// Relative links in kept Markdown that point at something the tree does not have. -const present = new Set(all.map(f => f.rel)); -const dirs = new Set(); -for (const rel of present) for (let d = path.posix.dirname(rel); d !== '.'; d = path.posix.dirname(d)) dirs.add(d); -const deadLinks = []; -for (const f of all.filter(f => f.rel.endsWith('.md'))) { - const text = fs.readFileSync(f.full, 'utf8'); - for (const m of text.matchAll(/\]\(([^)\s#]+)(?:#[^)]*)?\)/g)) { - const target = m[1]; - if (/^([a-z]+:|\/\/|#|mailto:)/i.test(target)) continue; - const resolved = path.posix.normalize(path.posix.join(path.posix.dirname(f.rel), target)); - if (resolved.startsWith('waybetter/')) continue; // the submodule - if (!present.has(resolved) && !dirs.has(resolved.replace(/\/$/, ''))) deadLinks.push(`${f.rel}: ${target}`); - } -} - -// Names of files and the whole tree through the framework's scan (terms from the keys folder). -const cli = path.join(root, 'waybetter', 'packages', 'waybetter', 'bin', 'waybetter.mjs'); -const { TERMS_ENV, privacyConfig } = await import(pathToFileURL(path.join(root, 'waybetter', 'packages', 'waybetter', 'cli', 'privacy.mjs')).href); -const configured = process.env[TERMS_ENV] || privacyConfig(root).forbiddenTerms; -const termsArgs = configured && fs.existsSync(configured) ? ['--terms', configured] : []; -const allowArgs = ['--allow', path.join(root, 'privacy-allowlist.txt')]; -const runScan = (cwd, ...scanArgs) => { - const r = spawnSync(process.execPath, [cli, 'privacy-scan', ...scanArgs, ...termsArgs, ...allowArgs], { - cwd, encoding: 'utf8', maxBuffer: 1 << 28, - }); - return { status: r.status, text: mask(`${r.stdout ?? ''}${r.stderr ?? ''}`).trim(), terms: termsArgs.length > 0 }; -}; -const scanTree = runScan(root, tree); - -// --------------------------------------------------------------------------------------------- -// The single commit, authored as CRT, in a throwaway repository next to the tree. -const repo = path.join(out, 'repo'); -const emptyConfig = path.join(out, 'empty.gitconfig'); -fs.writeFileSync(emptyConfig, ''); -const env = { ...process.env, GIT_AUTHOR_NAME: AUTHOR.name, GIT_AUTHOR_EMAIL: AUTHOR.email, GIT_COMMITTER_NAME: AUTHOR.name, GIT_COMMITTER_EMAIL: AUTHOR.email, GIT_CONFIG_GLOBAL: emptyConfig, GIT_CONFIG_NOSYSTEM: '1' }; -const g = (...a) => execFileSync('git', a, { cwd: repo, env, encoding: 'utf8', maxBuffer: 1 << 28 }).trim(); -if (UPDATE) { - // One new commit on top of the public repository's current main: its history is the only history. - execFileSync('git', ['clone', '-q', '--branch', 'main', PUBLIC_REPO, repo], { env, encoding: 'utf8', maxBuffer: 1 << 28 }); - g('config', 'core.autocrlf', 'false'); - for (const entry of fs.readdirSync(repo)) if (entry !== '.git') fs.rmSync(path.join(repo, entry), { recursive: true, force: true }); - fs.cpSync(tree, repo, { recursive: true }); -} else { - fs.cpSync(tree, repo, { recursive: true }); - g('init', '-q', '-b', 'main'); - g('config', 'core.autocrlf', 'false'); -} -g('add', '-A'); -g('update-index', '--add', '--cacheinfo', `160000,${fwCommit},waybetter`); -if (UPDATE && !g('status', '--porcelain')) throw new Error('nothing changed: the public main already has this tree'); -g('commit', '-q', '-m', UPDATE ? MESSAGE : 'wayBetter for X'); -const commit = g('rev-parse', 'HEAD'); -const parent = UPDATE ? g('rev-parse', 'HEAD~1') : null; -const changed = UPDATE ? g('diff', '--stat', 'HEAD~1', 'HEAD').split('\n').pop() : ''; -const identity = g('log', '-1', '--format=%an <%ae> | %cn <%ce>'); -const scanHistory = args.includes('--no-history-scan') ? { status: null, text: 'skipped' } : runScan(repo, '--history', '.'); -const subStatus = g('ls-tree', 'HEAD', 'waybetter'); -const gitmodules = g('show', 'HEAD:.gitmodules'); - -// --------------------------------------------------------------------------------------------- -// The framework's unpushed commits: authors, messages and the diff range through the same scan. -const fwDir = path.join(root, 'waybetter'); -const fwGit = (...a) => execFileSync('git', a, { cwd: fwDir, encoding: 'utf8', maxBuffer: 1 << 28 }).trim(); -let fwSection; -try { - try { fwGit('fetch', 'origin'); } catch { /* offline: use the refs as they are */ } - const upstream = fwGit('rev-parse', '--abbrev-ref', '@{upstream}'); - const base = fwGit('merge-base', 'HEAD', upstream); - const ahead = fwGit('rev-list', '--count', `${base}..HEAD`); - const behind = fwGit('rev-list', '--count', `HEAD..${upstream}`); - const idents = fwGit('log', '--format=%an <%ae> | %cn <%ce>', `${base}..HEAD`).split('\n').filter(Boolean); - const counts = [...new Set(idents)].map(i => `${idents.filter(x => x === i).length} x ${i}`); - const tmp = path.join(out, 'fw-range'); - fs.mkdirSync(tmp, { recursive: true }); - fs.writeFileSync(path.join(tmp, 'range.patch'), fwGit('log', '-p', '--format=commit %H%nAuthor: %an <%ae>%nCommitter: %cn <%ce>%n%n%B', `${base}..HEAD`)); - fs.writeFileSync(path.join(tmp, 'names.txt'), fwGit('diff', '--name-only', `${base}..HEAD`)); - const scanRange = runScan(root, tmp); - const scanAll = runScan(fwDir, '--history', '.'); - const log = fwGit('log', '--format=- %h %s', `${base}..HEAD`).split('\n').map(l => l.slice(0, 160)); - fwSection = ['', '## Framework unpushed commits', '', - `Local branch is ${ahead} ahead and ${behind} behind ${upstream} (base ${base.slice(0, 10)}, head ${fwGit('rev-parse', 'HEAD').slice(0, 10)}).`, '', - `Authors and committers: ${counts.join('; ')}`, '', - `- Privacy scan over the patch text (messages, authors, every added and removed line) and the changed file names: exit ${scanRange.status}`, '', '```', scanRange.text, '```', '', - `- Privacy scan over the framework's whole history (every blob, path, message, identity): exit ${scanAll.status}`, '', '```', scanAll.text, '```', '', - '
Commits', '', ...log, '', '
']; - fs.rmSync(tmp, { recursive: true, force: true }); -} catch (error) { - fwSection = ['', '## Framework unpushed commits', '', `Could not be computed: ${mask(error.message).split('\n')[0]}`]; -} - -// --------------------------------------------------------------------------------------------- -// Report -const mb = n => (n / 1e6).toFixed(1); -const lines = []; -const p = (...t) => lines.push(...t); -p('# Public tree dry run', '', - `Generated by \`node scripts/public-tree.mjs\` from app HEAD \`${head.slice(0, 12)}\` and framework commit \`${fwCommit.slice(0, 12)}\` (the gitlink at HEAD unless \`--fw-commit\` was given). Re-run it whenever HEAD moves.`, '', - '## Result', '', - `- Files: **${all.length}** (HEAD has ${files.length - 1} plus the submodule; ${files.length - 1 - kept.length} excluded)`, - `- Size: **${mb(totalBytes)} MB** (git objects will be smaller: compressed)`, - `- Images and media in the set: ${images.length}`, - `- Initial commit: \`${commit.slice(0, 12)}\`, author and committer: ${identity}`, - `- Submodule entry: \`${subStatus}\``, '', '```', gitmodules, '```', '', - '## Excluded, and why', '', '| Files | Reason | Examples |', '| ---: | --- | --- |'); -for (const [reason, entry] of excluded) p(`| ${entry.count} | ${reason} | ${entry.samples.map(s => `\`${s}\``).join(', ')} |`); -p('', 'Everything else tracked at HEAD is in the set. `.gitmodules` is rewritten to the public framework URL.', '', - '## Scans', '', - `- Forbidden-terms file: ${scanTree.terms ? 'present and used' : 'MISSING (only the machine account name was checked)'}`, - `- Privacy scan over the tree (file contents, binaries, archives and file names): exit ${scanTree.status}`, '', '```', scanTree.text || '(no output)', '```', '', - `- Privacy scan over the new repository's history (blobs, paths, commit message, identity): exit ${scanHistory.status}`, '', '```', scanHistory.text || '(no output)', '```', '', - `- Absolute user paths (\`C:\\Users\\\`, \`/Users/\`, \`/home/\`): **${pathHits.length}** hit(s), user names masked`); -if (pathHits.length) p('', '```', ...pathHits.slice(0, 60), pathHits.length > 60 ? `... ${pathHits.length - 60} more` : '', '```'); -p('', `- Relative Markdown links to files that are not in the set: **${deadLinks.length}**`); -if (deadLinks.length) p('', '```', ...deadLinks, '```'); -p('', `- Files over ${mb(MAX_FILE)} MB (review): ${big.length}`); -if (big.length) p('', ...big.map(f => ` - \`${f.rel}\` ${mb(f.size)} MB`)); -p('', '## Images', '', - `${images.length} image and media files; ${images.length - unreviewed.length} match the reviewed list (scripts/public-tree.reviewed-images.json: path and sha256 of what a person looked at, or a generated card). Unreviewed or changed, to look at before publishing: **${unreviewed.length}**`, '', ...unreviewed.map(r => '- ' + r), '', - '## Publish (the coordinator runs these)', '', - 'The framework goes first: push it, then use its pushed commit as the submodule pointer. Re-run this script with `--fw-commit ` if it differs from the gitlink above.', '', - ...(UPDATE ? ['Update mode: one new commit on top of the public main (' + parent.slice(0, 12) + '): ' + changed + '. Push it with:', '', '```sh', `cd "${show(repo)}"`, 'git push origin main', '```', ''] : []), - '```sh', - '# 1. framework (already public; its remote is behind the local branch)', - '# after reading section "Framework unpushed commits" of the report and fixing what it lists:', - '# git -C waybetter push origin main', - '', - `# 2. re-run the dry run and read the report again`, - `node scripts/public-tree.mjs`, - '', - '# 3. the new repository: an empty repo at ' + PUBLIC_REPO + ' first (created on the forge by the owner or coordinator), then', - `cd "${show(repo)}"`, - `git remote add origin ${PUBLIC_REPO}`, - 'git push -u origin main', - '```', '', - `The commit in \`${show(repo)}\` is already authored as ${AUTHOR.name} <${AUTHOR.email}> with the message "wayBetter for X". To use another message or a date, amend it there before pushing (\`git commit --amend\` with the same identity in the environment).`, ''); -lines.push(...fwSection, ''); -fs.writeFileSync(path.join(out, 'REPORT.md'), lines.join('\n')); -fs.writeFileSync(path.join(out, 'images.txt'), images.map(f => `${f.rel}\t${f.size}`).join('\n') + '\n'); -if (UPDATE) console.log(`update commit ${commit.slice(0, 12)} on public main ${parent.slice(0, 12)}: ${changed}\npush: cd "${show(repo)}" && git push origin main`); -console.log(`public tree: ${all.length} files, ${mb(totalBytes)} MB, commit ${commit.slice(0, 12)}`); -console.log(`scan tree: exit ${scanTree.status}; scan history: exit ${scanHistory.status}; path hits: ${pathHits.length}; dead links: ${deadLinks.length}`); -console.log(`report: ${show(path.join(out, 'REPORT.md'))}`); diff --git a/scripts/public-tree.reviewed-images.json b/scripts/public-tree.reviewed-images.json deleted file mode 100644 index 246b773..0000000 --- a/scripts/public-tree.reviewed-images.json +++ /dev/null @@ -1,64 +0,0 @@ -{ - "note": "Images looked at by a person (hero, fix-like, thread-reader frames shown only @CRTkafa and @playfishies; the rest are logos, icons and generated cards). Generated cards under apps/web/public/og/items and store/previews are covered by pattern. Anything not listed or with another hash is reported as unreviewed.", - "reviewed": { - "apps/desktop/src-tauri/icons/128x128.png": "4f9cbaaa7bc493b8d5194af3dc6291f27b091ee2a19c54cba70eeda7c1269e3d", - "apps/desktop/src-tauri/icons/128x128@2x.png": "5db8610d37798eccbfa97912e06ec6f3ea5d090a871766ed4035877cd43931bb", - "apps/desktop/src-tauri/icons/32x32.png": "900f86a59e78d33a50d23e400f8f35c8a6b7bf40e3b2bd89ef057a015fe15b08", - "apps/desktop/src-tauri/icons/64x64.png": "ec94bcc56ecb7d483fa424629855ad58ee12af11856954a7f02508e4f8d591f8", - "apps/desktop/src-tauri/icons/app-icon.png": "55804f5efdb837223362c56cd39c4041d0d030e33a5db153b702dbc3d43e6ce8", - "apps/desktop/src-tauri/icons/icon.icns": "d9658badddd2ed3d96a9da8ce6c4428efa979818ef5441a13a7b9a9b37391161", - "apps/desktop/src-tauri/icons/icon.ico": "422cd8b400e0c0a107bdcc0e8dc91d797f8e8b279aaaf363d5d4037c409e280a", - "apps/desktop/src-tauri/icons/icon.png": "cc9357e5d149b8d1e2f8fefea1ffee26fb0f72423f224bc19b5d9e3a2809db98", - "apps/web/public/app-icon.png": "55804f5efdb837223362c56cd39c4041d0d030e33a5db153b702dbc3d43e6ce8", - "apps/web/public/brand/brush-swash.svg": "9680bc3e94c9f159f8e702abeda48fd4bdec8cc054377ee778cd62bd253eca98", - "apps/web/public/brand/wb-icon-128.png": "056705755ecd72a704558e668be887ae6ad87bd25f5fc51668445270d52f1b76", - "apps/web/public/brand/wb-icon-256.png": "33a989958398e6cb547f8964d361d103f8c09bbe430744a9698daabe5ed72c2c", - "apps/web/public/brand/wb-icon-32.png": "f2961851434245ad5abb33cfce62285a844eb440af9f0bb72f2bdbf160e2d522", - "apps/web/public/brand/wb-icon-512.png": "e0425c029a5b3784164d16728ea45cf542164ea55f5312e4768d90ef5716a3b7", - "apps/web/public/brand/wb-icon-64.png": "940246b4f9d8789497e983c22c9b22bc1b4cbcd2f539b4aa5ca3d6053ed0ac2d", - "apps/web/public/brand/wb-icon.svg": "c0e43b05c111c4b6cbe813158bdd5cf19df1e34168a3fc456be1142cb6ac10b6", - "apps/web/public/brand/wb-mark-128.png": "40f249a1f33f7ffbcbce4c920e62144b162765fa537bac428c4d57a1802cff09", - "apps/web/public/brand/wb-mark-256.png": "1a6bf95513f796538a791a6d3b868f6dbceb34c80388d83db2f13dbe48562c92", - "apps/web/public/brand/wb-mark-32.png": "33152842c6431ea6da07015223020ad2734eebda9a640a1f8900ed4cb4dc47e2", - "apps/web/public/brand/wb-mark-512.png": "584dc90c4568a186a3c19446500b48c8e7ccb390081e8341bd8e5ad122e7284e", - "apps/web/public/brand/wb-mark-64.png": "c5b766c2a3232ea68d12d84cb05b1eb7d1c6374e53b6c883a007fb544271eb16", - "apps/web/public/brand/wb-mark-light.svg": "2007ba9de2b59139f5d8462290b789ce7d60f762943498b0e643a8af2c41ab9c", - "apps/web/public/brand/wb-mark-simple.svg": "0aaf4d04edf0d41218d872673e1ac980d53d16eb79f35dac3c3658ced9f71cb6", - "apps/web/public/brand/wb-mark.svg": "9140015c771cde09ece44ec8a11d76025c90e2ca947045758651b0f374f274f6", - "apps/web/public/demo/fix-like-m.jpg": "6143ad2df62d38eb8abce1f3ac624fc2323b2a34ee4a350c2d09845ed11fd4ac", - "apps/web/public/demo/fix-like-m.mp4": "b536b2d687270acf3196ca59139eb83a660cd32a5c9d366cbc1f803d6040383f", - "apps/web/public/demo/fix-like-m.webp": "4f0134b06cb34e9907997afc6b970e5c419498e65d52f64f0aadd7f1a46eab0e", - "apps/web/public/demo/fix-like.jpg": "6bd9b88581f7999f9cda94051e17581cfc7283b9c69d64d94dbcedb8d5ce54d1", - "apps/web/public/demo/fix-like.mp4": "6af489657b9f0ca46a7835e9d6b3fcf9ebbc760541137f5e0d28c9c182c3a380", - "apps/web/public/demo/fix-like.webp": "ee2c107f747f1f48f6954849cb75120c697a18791d0494856c5dc118c41bdbf7", - "apps/web/public/demo/hero.jpg": "3f951c12bb061d529f2d07638129bb76ddfd717902c2dd0729d44401eb42574b", - "apps/web/public/demo/hero.mp4": "abe14a93ea9e6f272de5a4706a26068e17a407d55ab4e6e7b6ac6780b472b3d9", - "apps/web/public/demo/hero.webp": "48a1f58e2a8bae501745e95fb46fe30b3a42061c5f2b9e582fa07963a97ae82a", - "apps/web/public/git-crt-logo.svg": "997bea1b94c9b40a4f59d45b127119a51b364edb7d9d689045d4a97876739e89", - "apps/web/public/og/home.png": "7ce32572720777713b26f7ad3d42f04db116e16a18da6ab42cf3053702e070f6", - "apps/web/public/og/store.png": "6eb9b284a3bccf173dd72befca20fb6f3e9955a8c1083df09f0a81efc7e66d80", - "apps/web/public/og/x.png": "128e751ba1a2d270cf2a1266594b934f2487601db4054c87b51cb8bc93ac62d8", - "apps/web/public/wb-mark.svg": "19f08a0321c008b94c8ea691b8ac9fb5f107c6fa8b083418d4a8cb4f82d62e53", - "docs/brand/logo-source.png": "b8f1a64b7104be1ed2d32f1b1714d27782c2ee273b204a3c9fde7f08b8e90695", - "store/x/areas/reader/media/copy-as-markdown.png": "d9aa931d19a3d84282c4cc1776b8cab4bce64496bd0278898a730fe7723720a8", - "store/x/areas/reader/media/cover.webp": "c5f7d41a144c6a2e7be278df14b21bbbfd07c384cfe618fbc6530d01598ae05f", - "store/x/areas/reader/media/demo.mp4": "e74810ddf6210ddbd185e37123d41cfcc214c9bcae5eee3152a6392ee92cc1b1", - "store/x/areas/reader/media/demo.webp": "62ee097cdab833b2cc1dc10d1503fa2a16a501d1a97a4ac7d62c616438e7d1b2", - "store/x/areas/reader/media/reading-view.webp": "3c24634b09d7c9a2fa79bf4fa62c7d9704ed15a38281ef253ae2bcbf6410c3f7", - "assets/readme/app-menu.mp4": "ac951bb3db05489ce6bd7f9f9d918273bab20d12b91c3bf7ba7d84f47766ea6c", - "assets/readme/app-menu.webp": "8936170ce5beab50aa6d8afbdbcfc11b2512cc878aeb0988d7cb84ebf5c11364", - "assets/readme/chat-plus.mp4": "7ab687c9d6e1c805e6499283854e334d1bb1f1af593950982852961420d49cf5", - "assets/readme/chat-plus.webp": "0b240aa32f0e99b7d907b8ed72c1a3ffed46f8ff9ebf5902c4e180932b7741b1", - "assets/readme/hero.webp": "b4d9ec3ebf86e0bad0b3fb40f82f18dd4544cdee332b5042cc0c2408a63cf1a7", - "assets/readme/like-fix.mp4": "7d7b68137537ca2b02d093c19c97a21dcda3dc7f1cc74475f166e6c17dbc2dbb", - "assets/readme/like-fix.webp": "970494d80ee1f02952a5069523fa4bef83c8c3d976e3df839d8458821d426eeb", - "assets/readme/split-accounts.mp4": "c05e04a54ebd2c87c28808ec8091c403b9f1b0ec2703ca39e25f71a4a6cdb63d", - "assets/readme/split-accounts.webp": "c4117094d0f5f66da20fa0400ad923678ca01f9ac652a6c74f164c97b70a5cc0", - "assets/readme/store.mp4": "9020db21e8cd9ea758181e90d535af2a6fda01570dcf14ac3c3fe545b2ba97c0", - "assets/readme/store.webp": "47a10af6dc32d3b5bacef6d4d59824ddb2e5f72ec9ed046dc26714f378adffdc", - "assets/readme/string-pack.mp4": "5dfbc59c7b31455796eccb28c03fa5fb4caff72c2e4e4127237efa8eec8862e4", - "assets/readme/string-pack.webp": "f91095cd272b7b7fb059ad2329c9d7532066698d77f1e00e010b183832ae5b7d", - "assets/readme/theme-burst.mp4": "b7d151650b96f0cbff5000fc7e06a525adefe022c7ba6f8f9236c4b53f273682", - "assets/readme/theme-burst.webp": "00924d81b087bedee154c69503e3b7c35319039f30aa30e934da7dc4f35d4c62" - } -} diff --git a/scripts/release-macos.mjs b/scripts/release-macos.mjs index 05c8d05..93245c4 100644 --- a/scripts/release-macos.mjs +++ b/scripts/release-macos.mjs @@ -25,9 +25,8 @@ // // node scripts/release-macos.mjs --dmg --updater // -// Both files go through `waybetter privacy-scan` first (the dmg's image and the .app inside the -// updater bundle are opened): a forbidden name in either stops this script before anything is -// copied or signed. +// Both files go through the optional release check first (WAYBETTER_RELEASE_CHECK): a failure +// stops this script before anything is copied or signed. // // A universal build serves both darwin-aarch64 and darwin-x86_64. Run scripts/release.mjs (Windows) // first: this script keeps every platform already in latest.json and refuses a version mismatch. diff --git a/scripts/release.mjs b/scripts/release.mjs index 05dab2b..4cd8cad 100644 --- a/scripts/release.mjs +++ b/scripts/release.mjs @@ -7,10 +7,10 @@ // // node scripts/release.mjs [--skip-build] [--notes "text"] // -// Privacy: the build runs with `waybetter build-env` (no absolute build path, so no account name, -// in the binary), and the installer (opened with 7-Zip), the binary and the front end must pass -// `waybetter privacy-scan` against ~/.better-twttr-keys/forbidden-names.txt before anything is -// copied to releases/. The .pdb next to the binary holds every path: never publish it. +// The build runs with `waybetter build-env` (no absolute build path in the binary), and the +// installer, the binary and the front end go through the optional release check +// (WAYBETTER_RELEASE_CHECK) before anything is copied to releases/. The .pdb next to the binary +// holds every path: never publish it. // // Env: // CARGO_TARGET_DIR optional; the bundle is read from /release/bundle/nsis @@ -19,7 +19,7 @@ // DEPLOY_TARGET ssh target for the printed steps, e.g. user@host (no default: printed as a placeholder) // CDN_DIR remote folder behind CDN_BASE (placeholder when unset) // RELEASES_DIR remote API RELEASES_DIR (placeholder when unset) -// WAYBETTER_FORBIDDEN_TERMS another terms file; WAYBETTER_7Z 7-Zip when it is not installed in Program Files +// WAYBETTER_RELEASE_CHECK optional command run on every artifact first (a non-zero exit stops the release) import { dirname, resolve } from 'node:path'; import { fileURLToPath } from 'node:url'; import { releaseCommand } from '../waybetter/packages/waybetter/cli/release.mjs'; diff --git a/waybetter.release.json b/waybetter.release.json index 87f98f8..ff79c7f 100644 --- a/waybetter.release.json +++ b/waybetter.release.json @@ -21,8 +21,6 @@ "windows": { "silentSwitches": "/S", "scope": "user" }, "macos": { "app": "wayBetter for X.app", "zap": ["~/Library/Application Support/app.bettertwttr.desktop", "~/Library/Caches/app.bettertwttr.desktop"] } }, - "forbiddenTerms": "~/.better-twttr-keys/forbidden-names.txt", - "privacyAllowlist": "privacy-allowlist.txt", "manifestValidator": { "module": "packages/contracts/dist/index.js", "export": "NativeRelease",