# Bounded #28 Catalog Contract — Explicit Grouping & Stable Identity

**Status:** Design phase — not implemented. Extends Tasks 1-4 contracts without breaking them.
**Baseline:** `main` at `9dd8c07` (Tasks 1-4 integrated, #28 catalog core closed).

## 1. Problem Statement

The current `LegacyGuessedImportGroupingAdapter` implements **guessed** grouping:
- First directory component under source root → model name
- Files in source root → model named after source directory
- Stateless, fresh lookup on every call via `ensureModel()`
- **No user confirmation**, no persistence of grouping decisions

Gaps from reconciliation table:
- **#18**: Confirmed explicit grouping (user confirmation) missing
- **#20**: Explicit user-confirmed grouping (not guessed); stable model identity across imports; grouping persistence
- **#24**: Crash between publish and suffix save → duplicate suffixed copies; orphaned published files on catalog failure; replay idempotency

## 2. Design Principles

1. **Retain PHP/PDO/Phinx and existing repositories** — no schema rewrites, new migrations only
2. **Extend, don't replace** — `CatalogServiceInterface`/`CatalogRepositoryInterface` unchanged
3. **Explicit grouping is opt-in** — legacy guessed adapter remains for backward compatibility
4. **Bounded scope** — no UI, no scheduler, no async workers; pure service/contract layer
5. **Lock order preserved** — root → folder → model → file (same as Task 4)
6. **Savepoint semantics** — caller transaction owns commit/rollback; catalog operations use savepoints

## 3. New Contracts (Interfaces)

### 3.1 Explicit Grouping Adapter Interface

```php
// src/Indexer/Contracts/ExplicitImportGroupingAdapterInterface.php
namespace MeshVault\Indexer\Contracts;

use MeshVault\Indexer\ScannedFile;

interface ExplicitImportGroupingAdapterInterface
{
    public function modelFor(string $sourcePath, ScannedFile $file, array $context = []): int;
    public function persistDecisions(int $jobId, array $decisions): void;
    public function recoverDecisions(int $jobId): array;
}
```

### 3.2 Explicit Grouping Adapter Implementation

```php
// src/Indexer/ExplicitImportGroupingAdapter.php
namespace MeshVault\Indexer;

use MeshVault\Indexer\Contracts\ExplicitImportGroupingAdapterInterface;
use MeshVault\Indexer\ScannedFile;
use MeshVault\Repository\CatalogRepositoryInterface;
use MeshVault\Repository\ImportJobRepositoryInterface;
use DomainException;
use InvalidArgumentException;

final class ExplicitImportGroupingAdapter implements ExplicitImportGroupingAdapterInterface
{
    public function __construct(
        private readonly CatalogRepositoryInterface $catalog,
        private readonly ImportJobRepositoryInterface $jobs,
    ) {}

    public function modelFor(string $sourcePath, ScannedFile $file, array $context = []): int
    {
        if (isset($context['model_id'])) {
            $modelId = (int) $context['model_id'];
            $aggregate = $this->catalog->aggregate($modelId);
            if ($aggregate === null) {
                throw new DomainException('Explicit model not found.');
            }
            if (isset($context['model_name']) && $aggregate['model']['name'] !== $context['model_name']) {
                throw new DomainException('Model title mismatch.');
            }
            if (isset($context['folder_path'])) {
                $expectedFolder = '/' . ltrim($context['folder_path'], '/');
                if ($aggregate['folder']['path'] !== $expectedFolder) {
                    throw new DomainException('Model folder mismatch.');
                }
            }
            return $modelId;
        }

        if (isset($context['folder_path']) && isset($context['model_name'])) {
            $folderPath = '/' . ltrim($context['folder_path'], '/');
            $modelName = $context['model_name'];
            if ($context['create_new'] ?? false) {
                return $this->catalog->atomic(fn () => $this->catalog->createModel($folderPath, $modelName));
            }
            return $this->catalog->atomic(fn () => $this->catalog->ensureModel($folderPath, $modelName));
        }

        $parent = trim(dirname(str_replace('\\', '/', $file->relativePath)), '/');
        $group = $parent === '' || $parent === '.'
            ? basename(rtrim(str_replace('\\', '/', $sourcePath), '/'))
            : explode('/', $parent)[0];
        return $this->catalog->atomic(fn () => $this->catalog->ensureModel('/' . $group, $group));
    }

    public function persistDecisions(int $jobId, array $decisions): void
    {
        $this->jobs->atomic(function () use ($jobId, $decisions) {
            foreach ($decisions as $decision) {
                $this->jobs->recordGroupingDecision(
                    $jobId,
                    $decision['source_path'],
                    $decision['model_id'],
                    $decision['context'] ?? []
                );
            }
        });
    }

    public function recoverDecisions(int $jobId): array
    {
        return $this->jobs->getGroupingDecisions($jobId);
    }
}
```

### 3.3 Import Job Repository Extensions

```php
// src/Repository/ImportJobRepositoryInterface.php (additions)
namespace MeshVault\Repository;

interface ImportJobRepositoryInterface
{
    // ... existing methods ...
    public function recordGroupingDecision(int $jobId, string $sourcePath, int $modelId, array $context): void;
    public function getGroupingDecisions(int $jobId): array;
}
```

### 3.4 Catalog Service Extensions (for stable identity)

```php
// src/Catalog/CatalogServiceInterface.php (additions)
namespace MeshVault\Catalog;

interface CatalogServiceInterface
{
    // ... existing methods ...
    public function findModel(string $folderPath, string $modelName): ?array;
    public function listModelsInFolder(string $folderPath): array;
    public function getOrCreateModel(string $folderPath, string $modelName): array;
}
```

## 4. Database Migrations

### 4.1 Migration: `20261014120001_import_grouping_decisions.php`

```php
<?php
declare(strict_types=1);

use Phinx\Migration\AbstractMigration;

final class ImportGroupingDecisions extends AbstractMigration
{
    public function up(): void
    {
        $this->table('import_jobs_decisions', ['id' => false, 'primary_key' => ['job_id', 'source_path']])
            ->addColumn('job_id', 'integer', ['signed' => false])
            ->addColumn('source_path', 'text')
            ->addColumn('model_id', 'integer', ['signed' => false])
            ->addColumn('context', 'jsonb', ['default' => '{}'])
            ->addColumn('created_at', 'timestamp', ['default' => 'CURRENT_TIMESTAMP'])
            ->addForeignKey('job_id', 'import_jobs', 'id', ['delete' => 'CASCADE', 'update' => 'NO_ACTION'])
            ->addIndex(['model_id'], ['name' => 'idx_import_decisions_model'])
            ->create();
    }

    public function down(): void
    {
        $this->table('import_jobs_decisions')->drop()->save();
    }
}
```

### 4.2 Migration: `20261014120002_model_identity_stability.php`

```php
<?php
declare(strict_types=1);

use Phinx\Migration\AbstractMigration;

final class ModelIdentityStability extends AbstractMigration
{
    public function up(): void
    {
        // No schema changes — this migration documents the stability contract
        // Models are now immutable in identity (folder_id + name unique)
        // Files reference models via nullable FK (from Task 4)
        // Explicit grouping decisions recorded in import_jobs_decisions
    }

    public function down(): void
    {
        // No-op: identity stability is a contract, not a schema change
    }
}
```

## 5. Import Service Integration

```php
// src/Indexer/ImportService.php (modified)
namespace MeshVault\Indexer;

use MeshVault\Indexer\Contracts\ExplicitImportGroupingAdapterInterface;
use MeshVault\Indexer\Contracts\FileEnumeratorInterface;
use MeshVault\Repository\ImportJobRepositoryInterface;
use Throwable;

final readonly class ImportService
{
    public function __construct(
        private FileEnumeratorInterface $enumerator,
        private ImportJobRepositoryInterface $jobs,
        private ExplicitImportGroupingAdapterInterface $groupingAdapter, // Changed from LegacyGuessed
        private SpaceChecker $space,
        private string $libraryRoot,
    ) {}

    /** @param array<string, mixed> $groupingContext Optional explicit grouping context per file */
    public function import(string $sourcePath, array $extensions = [], array $groupingContext = []): int
    {
        $scan = $this->enumerator->enumerate($sourcePath, $extensions);
        if ($scan->files === []) {
            $detail = $scan->errors === [] ? 'No importable files were found.' : $scan->errors[0]->message;
            throw new ImportException('SCAN_EMPTY', $detail);
        }
        $this->space->check($this->libraryRoot, $scan->totalBytes);

        try {
            return $this->jobs->atomic(
                fn (): int => $this->createJobWithGrouping($sourcePath, $scan, $groupingContext)
            );
        } catch (ImportException $error) {
            throw $error;
        } catch (Throwable $error) {
            throw new ImportException('DB_ERROR', 'Could not persist the import job.', $error);
        }
    }

    /** @return list<array<string, mixed>> */
    private function createJobWithGrouping(string $sourcePath, ScanResult $scan, array $groupingContext): int
    {
        $items = [];
        $decisions = [];

        foreach ($scan->files as $file) {
            $fileContext = $groupingContext[$file->relativePath] ?? $groupingContext['*'] ?? [];
            $modelId = $this->groupingAdapter->modelFor($sourcePath, $file, $fileContext);

            $items[] = [
                'source_path' => $file->sourcePath,
                'size' => $file->size,
                'model_id' => $modelId,
            ];

            $decisions[] = [
                'source_path' => $file->sourcePath,
                'model_id' => $modelId,
                'context' => $fileContext,
            ];
        }

        foreach ($scan->errors as $error) {
            $items[] = [
                'source_path' => $error->sourcePath,
                'size' => 0,
                'model_id' => null,
                'status' => 'Failed',
                'failure_reason' => $error->code . ': ' . $error->message,
            ];
        }

        $jobId = $this->jobs->create($sourcePath, $items);

        if ($decisions !== []) {
            $this->groupingAdapter->persistDecisions($jobId, $decisions);
        }

        return $jobId;
    }
}
```

## 6. Worker Recovery & Idempotency (Addresses #24)

### 6.1 Current Gap
- Worker persists planned name **before** copy
- Saves collision suffix **after** publication
- Crash between → duplicate suffixed copies on replay
- Catalog failure after publish → orphaned file on disk

### 6.2 Solution: Atomic Publish-Then-Register

```php
// src/Indexer/ImportWorker.php (modified publish method)
private function publish(array $plan, string $destination): string
{
    $directory = dirname($destination);
    if (!is_dir($directory) && !@mkdir($directory, 0775, true) && !is_dir($directory)) {
        throw new ImportException('COPY_FAILED', 'Cannot create the managed destination directory.');
    }

    $destination = $this->pathSafety->ensureDestSafe($destination);
    clearstatcache(true, $destination);

    if (is_file($destination)) {
        if (hash_file('sha256', $destination) === $plan['hash']) {
            return $destination;
        }
    } else {
        $this->spaceChecker->check($directory, $plan['size']);
    }

    // CRITICAL CHANGE: Copy to TEMP, then atomic rename + catalog register in ONE transaction
    $tempPath = $destination . '.tmp.' . bin2hex(random_bytes(8));
    try {
        $this->copier->copy($plan['source'], $tempPath, $plan['size'], $plan['hash']);

        $managedPath = $this->relative($tempPath);
        $this->catalog->atomic(function () use ($tempPath, $destination, $plan, &$managedPath) {
            if (!@rename($tempPath, $destination)) {
                throw new ImportException('COPY_FAILED', 'Cannot finalize the published file.');
            }

            $managedPath = $this->relative($destination);

            $fileId = $this->catalog->registerFile(
                $plan['model_id'],
                $managedPath,
                $plan['hash'],
                $plan['size'],
                $plan['mime'] ?? null,
                'Present'
            );

            $this->jobs->updateItemManagedPath($plan['item_id'], $managedPath, $fileId);
        });

        return $destination;
    } catch (Throwable $error) {
        if (is_file($tempPath)) {
            @unlink($tempPath);
        }
        throw $error;
    }
}
```

### 6.3 New Job Repository Method

```php
// src/Repository/ImportJobRepositoryInterface.php (addition)
public function updateItemManagedPath(int $itemId, string $managedPath, int $fileId): void;
```

## 7. Lock Order & Transaction Boundaries

| Operation | Locks Acquired (Order) | Savepoint |
|-----------|------------------------|-----------|
| `createModel` | Root → Folder → Model | Yes |
| `ensureModel` | Root → Folder → Model (SELECT FOR SHARE) | Yes |
| `registerFile` | Root → Folder → File | Yes |
| `changeFileOwner` | Root → File (model FK key-share) | Yes |
| `updateModelMetadata` | Model (FOR NO KEY UPDATE) | Yes |
| `changeFileStatus` | File (FOR UPDATE) | Yes |
| **Explicit grouping decide** | Root → Folder → Model (same as ensureModel) | Yes |
| **Worker publish+register** | Root → Folder → File (same as registerFile) | Yes |

**Invariant:** Never hold job/item locks while entering catalog mutations. The worker's file lock (`.meshvault-import.lock`) is **outside** database transactions.

## 8. Test Plan (RED → GREEN)

### 8.1 New Test Cases

| Test | Scenario | Expected |
|------|----------|----------|
| `testExplicitGroupingWithModelId` | Provide existing model_id in context | Reuses model, validates folder/title match |
| `testExplicitGroupingNewModel` | Provide folder_path + model_name + create_new | Creates new model at exact path |
| `testExplicitGroupingFindOrCreate` | Provide folder_path + model_name (no create_new) | Finds existing or creates |
| `testExplicitGroupingAmbiguityRejected` | Two models match folder/title (legacy data) | DomainException |
| `testGroupingDecisionsPersisted` | Import with explicit context | Decisions recoverable after job creation |
| `testGroupingDecisionsRecovered` | Resume interrupted job | Reuses recorded model_ids |
| `testWorkerPublishAtomic` | Crash simulation between copy/register | No orphan temp, no duplicate suffix |
| `testWorkerReplayIdempotent` | Re-run worker on completed item | Adopts existing file, no re-copy |
| `testWorkerCatalogFailureRollback` | Catalog error after file copy | Temp cleaned, no orphan on disk |
| `testStableModelIdentity` | Multiple imports of same source tree | Same model IDs reused (not recreated) |

### 8.2 Mutation Tests (Negative Controls)

| Mutation | Expected Failure |
|----------|------------------|
| Remove FOR SHARE in `ensureModel` | Ambiguous title not detected |
| Remove temp-file atomic rename | Orphan temp files on crash |
| Skip decision persistence | Recovered job uses guessed grouping |

## 9. Backward Compatibility

- `LegacyGuessedImportGroupingAdapter` **unchanged** — remains default in container
- New `ExplicitImportGroupingAdapter` is opt-in via container configuration
- No schema changes to existing tables (new table only)
- Existing `ImportService::import()` signature extended with optional `$groupingContext`

## 10. Container Wiring (config/container.php)

```php
// Legacy (default)
$container->set(ImportService::class, fn ($c) => new ImportService(
    $c->get(FileEnumeratorInterface::class),
    $c->get(ImportJobRepositoryInterface::class),
    $c->get(LegacyGuessedImportGroupingAdapter::class),  // Existing
    $c->get(SpaceChecker::class),
    $c->get(string::class, 'library_root'),
));

// Explicit (new, opt-in)
$container->set(ExplicitImportGroupingAdapter::class, fn ($c) => new ExplicitImportGroupingAdapter(
    $c->get(CatalogRepositoryInterface::class),
    $c->get(ImportJobRepositoryInterface::class),
));

$container->set(ImportService::class, fn ($c) => new ImportService(
    $c->get(FileEnumeratorInterface::class),
    $c->get(ImportJobRepositoryInterface::class),
    $c->get(ExplicitImportGroupingAdapter::class),  // New
    $c->get(SpaceChecker::class),
    $c->get(string::class, 'library_root'),
));
```

## 11. Acceptance Criteria (Per #28 Bullets)

| Bullet | Evidence |
|--------|----------|
| **1. Transactional multipart aggregate** | Existing `CatalogService::atomic` + `aggregate()` — Tasks 1-4 |
| **2. Shared service contracts consumed by import/metadata/status** | `ImportService` uses `CatalogServiceInterface`; `ImportWorker` uses same; metadata/status via `CatalogService` — Tasks 1-4 |
| **3. Explicit reassignment/rollback/FK evidence** | `reassignFiles`, `detachFiles`, `changeFileOwner` with FK KEY SHARE — Task 4 |
| **NEW: Explicit confirmed grouping** | `ExplicitImportGroupingAdapter` with persisted decisions — This design |
| **NEW: Stable model identity** | `findModel`, `getOrCreateModel`, decision recovery — This design |
| **NEW: Crash-safe publish** | Atomic temp-rename + catalog register in savepoint — This design |

## 12. Out of Scope (Future Milestones)

- UI for grouping confirmation (#14)
- Scanner incremental/resumable (#15)
- Missing-file replace/remove UI (#25)
- Progress/ETA/pause/resume contracts (#26)
- Retry/backoff/dead-letter contracts (#27)
- Full-text search/facets (#29)
- Duplicate detection/review queue (#30)

## 13. Implementation Sequence

1. **Migration 1**: `import_jobs_decisions` table
2. **Interface**: `ExplicitImportGroupingAdapterInterface`
3. **Implementation**: `ExplicitImportGroupingAdapter`
4. **Repository extension**: `recordGroupingDecision`, `getGroupingDecisions`, `updateItemManagedPath`
5. **Catalog extension**: `findModel`, `listModelsInFolder`, `getOrCreateModel`
6. **ImportService integration**: Optional `$groupingContext` parameter
7. **Worker atomic publish**: Temp copy → atomic rename + register in savepoint
8. **Tests**: RED → GREEN for all cases in §8
9. **Mutation verification**: Two negative controls
10. **Documentation**: Update ledger and project-status

## 14. Risk Assessment

| Risk | Mitigation |
|------|------------|
| Lock order violation | All new paths follow root→folder→model→file; code review + mutation tests |
| Savepoint leakage | All catalog ops use existing `atomic()` wrapper; no raw transactions |
| Decision table growth | One row per file per job; bounded by job size; cleaned with job |
| Temp file cleanup on crash | `finally` block in worker; temp files in library root (same FS) |
| Backward compatibility break | Legacy adapter unchanged; explicit adapter opt-in only |
| Migration failure on existing data | Preflight checks (Task 1 pattern); reject if unassigned/ambiguous |

---

**Next Steps:** Implement RED tests first, then GREEN implementation per sequence in §13. No commits to `main` until fresh whole-branch review.