usefulunpack

Imported from znso4pa/usefulunpack (AGENTS.md). Install upstream with npx skills add znso4pa/usefulunpack. Copyright stays with the author.

AGENTS.md

Instructions for AI coding agents working in this repository.

UsefulUnpack = Android app (Kotlin) + Rust cdylib per archive format, glued by JNI. Human-facing docs: README · CONTRIBUTING (中文: CONTRIBUTING-zh) · TODO.md (changelog + roadmap, source of truth).

Commands

bash build.sh                      # full build: Rust cross-compile (3 ABIs) -> jniLibs -> assembleRelease -> ./UsefulUnpack.apk (needs NDK)
./gradlew :app:assembleRelease     # Kotlin-only changes (output lands in app/build/outputs/apk/release/ — NOT the root APK)
cargo test --workspace             # Rust suite — must stay green
./gradlew lintDebug                # baseline: 0 errors / 271 warnings — NEVER introduce new errors

build.sh is the only script that refreshes the root UsefulUnpack.apk. A bare :app:assembleRelease does not, so adb install UsefulUnpack.apk after one silently ships the PREVIOUS build.

Non-negotiable architecture invariants

1. One global progress/CANCEL slot per native library on the Rust side (progress_store! in crates/common). Therefore EVERY archive operation must go through:

   val opH = tryStartOperation(activity, fmtKey)   // never blocks/refuses; queues instead
   thread {
       if (!opH.await()) return@thread             // cancelled while queued → abort silently
       try { /* work */ } finally { opH.release() }
   }

   Same fmtKey serializes; different keys share 3 slots. Composite two-format flows use pseudo-key "merge"; zip entry edits use "zip". Keys that share a .so MUST also share a slot — go through schedulerKeyOf() in archive/ArchiveExtractor.kt, never the raw key. Three such families exist: pf6→pfs; the five tar variants (tar/tgz/tbz2/txz/tzst)→tar; and all eight rgss*/rpgmv*→rgss. A per-format .so with one compress_progress static, polled by several keys, means the second op's reset(total) wipes the first one's total and the second op's cancel aborts the first. This already happened twice (pf6, then the tar family) — the schedulerKeyOf comment lists the families; keep it accurate when adding a format.
2. Password prompts BEFORE tryStartOperation — modals block ~30 s and must never hold a slot/format lock.
3. Every new JNI operation entry calls clear_cancel() first, or the previous op's cancel poisons it.
4. Cancel isolation: queued-cancel must NOT set the format-global CANCEL flag (it would kill an unrelated running same-format op). Use PollingProgressDialog.cancelQueuedThenNotify() semantics.
5. Every runOnUiThread block that touches UI (Toast, AlertDialog, Progress.dismiss, etc.) MUST start with if (isFinishing || isDestroyed) return@runOnUiThread. When a prog.dismiss() appears in the same block, place it AFTER the guard to avoid dismissing on a destroyed Activity.
6. Deliberate legacy exceptions — do NOT migrate without revisiting rationale: delete/recycle flows and signature scan stay on old OperationLock.
7. viewPager.offscreenPageLimit = MAX_TABS - 1 keeps all fragments alive; don't lower it.
8. Derived UI state, never stored: anything a tab shows about its own state must be a function of that state. TabState.displayPath() exists because tvPath was written in only one of the three places a preview gets rendered — a rebuilt fragment left the bar at the layout default /. Same rule for syncMergeButton() and the merge target list, which share mergeTargetGroups() so the button and the picker can never disagree.
9. Semi-transparent drawing needs its own layer. Painting segments straight onto a persistent bitmap blends each one into the previous (SRC_OVER), so a 50% stroke gets darker the longer it is drawn and blobs at the joints. ImageEditorView renders each stroke at FULL alpha into activeLayer and applies that stroke's alpha once at composite time; the stroke records its own colour/alpha/width so re-rendering history (undo) cannot restyle it.
10. One source of truth per dispatch table. Duplicated when (format) blocks drift. The archive listing dispatch is listEntriesJson(); merge availability is mergeTargetGroups(). When you add a format, add it there and let the others forward.

Known traps

• ⚠️ Package ≠ directory: archive/ArchiveExtractor.kt and archive/ExtractProgress.kt sit under archive/ but declare root package com.usefulunpacker. Import their symbols explicitly from other packages (com.usefulunpacker.archive.OpScheduler style). Same class of quirk may exist elsewhere — trust package lines, not folders.
• Batch-bar buttons: visibility matches semantic tags ("extract"/"preview"/"compress") from buildBatchBar, checked in syncMultiBar. NEVER match display text; emojis exist only in string resources, never concatenated in Kotlin.
• Cross-tab multi-select: selections persist per-tab (TabState.multiSelected); batch ops aggregate via allSelectedFiles() / totalSelectedCount(). Cancel clears ALL tabs via exitAllMultiSelect(). Tab badges auto-refresh via syncAllTabAdapters() + tabAdapter.notifyDataSetChanged().
• Async completion dialogs: any runOnUiThread { ... AlertDialog ... } after background work requires isFinishing || isDestroyed guard (BadTokenException class — exterminated twice).
• Honor/EMUI ROM: custom ScrollView/TextView/EditText must not enable native scrollbars (ROM NPE in onDrawScrollBars).
• ⚠️ An opaque background kills the user's wallpaper: the wallpaper is drawn on R.id.root, so any opaque view sitting on top of it hides the wallpaper completely. 4.x was single-window, where clearing panel was enough; the multi-window layout then added viewPager / folderRoot / previewRoot / tabBar, and applyBackgroundImage was never updated — the wallpaper stopped showing anywhere, and every opaque layer added later silently re-broke it again. Rules: (1) the yield tables BACKDROP_TRANSLUCENT / BACKDROP_TRANSPARENT / BACKDROP_ORIGINAL live in ui/AppSettings.kt; any view that introduces an opaque background must be registered in them; (2) panel / pathBar / bottomBar / folderRoot / previewRoot exist once PER TAB, so they must be handled by walking the tree (refreshBackdrop()) — findViewById only ever reaches the first one; (3) FolderFragment must call applyBackdropInTree() on its own root at bind time, NOT refreshBackdrop() — in onCreateView the fragment's view is not yet attached to the ViewPager, so a walk from R.id.root cannot reach it (shipped that way once; only the tab strip showed through). Also, applyBackgroundImage must not bail silently when root.width == 0 on the first frame — retry one frame later instead. After touching any of this, set a real wallpaper on-device and confirm all four areas show through: tab strip / toolbar / file list / preview.
• New strings go to ALL FOUR locales: values/, -zh-rCN/, -zh-rTW/, -ja/. Check with a set comparison, not eyeballing — values-zh-rTW had drifted 7 <string> and 1 help_tutorials item behind for a long time. In XML, ' must be \' (AAPT2 fails obscurely otherwise) and a help entry MUST carry Title\nBody or parseItem renders the whole thing as a bold title with no body.
• Android string resources also carry a \\n (literal backslash-n) style in older entries; both appear in strings.xml. Match the style of the array you are editing instead of normalizing the file.

The lesson that keeps costing the most

Self-made fixtures fail together with the assumption they encode. Every one of these shipped green locally and was caught only by real data:

• An M4A fixture whose ftyp minor version was the same wrong guess as the code (0.0.2.0; real files use 0.0.0.0).
• A 24-byte probe window that stopped one byte short of the moov box, so a real .rpgmvm was rejected as invalid.
• A hand-typed obfuscated fixture whose 16-byte header did not match RPGM_HEADER (which carries 03 01), so the reader passed it through as "not obfuscated" and the test still went green.
• An M4A validation that searched for a box type at the position implied by the value it had just derived — unfalsifiable by construction.

When adding a format, get real samples and check them against an independent oracle, and prefer byte-identity over "it extracted without error": re-packing must reproduce the original file exactly. See the RPG Maker corpus note below.

Real-file regression corpora

Download the RPG Maker files with curl from raw.githubusercontent.com/uuksu/RPGMakerDecrypter/master/RPGMakerDecrypter.Tests/. A harness that exercises the parser from OUTSIDE the workspace is preferable — crates/rgss-core is an rlib as well as a cdylib, so a scratch crate can path-depend on it and leave the repo untouched. The env-gated examples/probe.rs is the on-device version of the same idea.

Conventions

• Commits: conventional prefixes; releases are single narrative commits feat(vX.Y.Z): … + dedicated TODO.md section + versionCode/versionName bump in app/build.gradle.
• TODO.md is the changelog AND roadmap — append a bullet under the current dev section for behavioral changes.
• Format listing JSON contract: [{"n":name,"s":size,"d":isDir,"e":encrypted}]; selective extraction takes newline-separated paths with exact + directory-prefix matching. A forwarded selection argument must be forwarded on EVERY branch: dropping it silently turns a selective extract into a full one.
• Rust JNI style: compact single-line functions wrapped in guarded(); reuse archive_common::{s, json_escape, safe_join}.
• Progress reporting: feed extract_progress::reset/set_file/add_bytes (packing: compress_progress::*). Whole-member buffered decodes feed the top bar from the WRITE side (ProgressWriter::extract) so totals stay exact.
• Pick ONE progress caliber per format and state it in a comment. Two exist: WRITE side (total = uncompressed size, bytes = bytes written — exact, but the total is unknowable whenever the header doesn't store it) and READ side (total = archive size, bytes = archive bytes consumed — always has a denominator). Since .lzma headers carry uncompressed_size = -1 in real files and zstd/brotli/xz frames don't store a size at all, the five single-file streams (brotli, bzip2, xz, zstd, lzma) use the READ side; gzip keeps the WRITE side for single members (exact and cheap) and uses READ for concatenated members. A format that does reset(0) and only feeds output bytes shows a spinner for the whole operation — ExtractProgress.kt treats total <= 0 as indeterminate.
• ProgressReader implements BOTH Read and BufRead — never wrap it in a BufReader. BufReader satisfies reads from its own 8 KB buffer, whose fill_buf does not count and whose consume only fires on consumption, so the counter drifts both short of and (via readahead) past the real size. Correct nesting is BufReader::with_capacity(64 KiB, ProgressReader::extract(file)) — ProgressReader wraps the raw file.
• A decoder that can fail must delete its partial output. File::create(&dest) runs before any content is validated, so a corrupt stream leaves a truncated file behind that looks like a successful extraction to both the user and any later "did it work?" check. Wrap every error path in a fail() closure that calls remove_file.
• Check third-party decoder limits against real data. ruzstd's DEFAULT_MAX_WINDOW_SIZE is 100 MB, but zstd level 22 writes window_log = 27 (128 MB) — every .zst-22 sample was rejected while the system zstd decoded it fine. Call new_with_max_window_size explicitly; keep it bounded, since the window ring buffer is allocated from the header before any content is validated.
• A Result<(u32, u32)> where the second number is an error count is not a success signal. zip/7z/rar return Ok even when every entry failed: a wrong password takes the per-entry fail += 1 branch and the archive-level call still returns Ok. Check err > 0 || total == 0.
• RPG Maker specifics worth remembering: RGSS v1 and v2 share one layout (VX is written as v1, keyed off the extension); a packed archive the engine will open must be named Game.<ext>, and there is no usable fallback name if that slot is taken. MV/MZ obfuscation XORs only the first 16 asset bytes behind a 16-byte RPGMV header, so without the game's key the header is reconstructed per container — and the M4A minor version is a reconstruction, not a recovery.

Map of non-obvious places

Verification before finishing a task

1. cargo test --workspace green (if Rust touched)
2. ./gradlew lintDebug — no new errors (if Kotlin/resources touched)
3. bash build.sh completes (if JNI surface changed)
4. If a format or codec was touched, run the real-file corpora above — synthetic round-trips are necessary, not sufficient
5. Behavioral checklist for UI-sensitive changes: rotation survival, cancel mid-op cleanup, parallel smoke (different formats overlap; same format queues; queued-cancel isolates), Honor scrollbar avoidance
6. Four-locale string set comparison (same key set in all four strings.xml, and every help_tutorials item has a Title\nBody split)