Verified copy into managed storage with collision handling #24

Open
opened 2026-10-10 22:21:47 +00:00 by keith · 1 comment
Owner

Context

Implement this MVP work package under M3 - Import & Indexing Engine, following the managed-storage decision in #1.

Acceptance criteria

  • Preserve relative hierarchy; copy to same-filesystem temporary files and verify hashes before atomic publication.
  • Never overwrite or alter originals; deterministic collision suffixes preserve existing destinations.
  • Coordinate database and filesystem publication with compensating cleanup/recovery; reject path and symlink escapes.
  • Test interruption, checksum mismatch, collision, disk-full and unchanged source trees; no partial files appear as completed imports.

Blocked by: #21, #22, #23

## Context Implement this MVP work package under M3 - Import & Indexing Engine, following the managed-storage decision in #1. ## Acceptance criteria - [ ] Preserve relative hierarchy; copy to same-filesystem temporary files and verify hashes before atomic publication. - [ ] Never overwrite or alter originals; deterministic collision suffixes preserve existing destinations. - [ ] Coordinate database and filesystem publication with compensating cleanup/recovery; reject path and symlink escapes. - [ ] Test interruption, checksum mismatch, collision, disk-full and unchanged source trees; no partial files appear as completed imports. Blocked by: #21, #22, #23
keith added this to the M3 - Import & Indexing Engine milestone 2026-10-10 22:21:47 +00:00
keith added the area:indexertype:featurepriority:medium labels 2026-10-10 22:21:47 +00:00
Author
Owner

Acceptance reconciliation at c1ef2ee: keeping open. Worker flattens nested hierarchy. Source-derived crash window: collision suffix is saved after publication; death before commit can cause another suffix on replay. Catalog failure after publication can leave an unmanaged file. Add regression tests before claiming recovery. This is a source/test review, not a newly executed crash or restore experiment. Fresh Windows suite: 291 tests, 1801 assertions, 22 skips; lint passes. PostgreSQL fixture success does not waive these feature criteria.

Acceptance reconciliation at c1ef2ee: keeping open. Worker flattens nested hierarchy. Source-derived crash window: collision suffix is saved after publication; death before commit can cause another suffix on replay. Catalog failure after publication can leave an unmanaged file. Add regression tests before claiming recovery. This is a source/test review, not a newly executed crash or restore experiment. Fresh Windows suite: 291 tests, 1801 assertions, 22 skips; lint passes. PostgreSQL fixture success does not waive these feature criteria.
Sign in to join this conversation.