# Torn API v1 → v2 Migration Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Rewrite Torn Vault Tracker's log-ingestion code to consume the new v2 Torn API JSON shape (array-of-entries with `_metadata.links.next` pagination and `Authorization: ApiKey` header), with a single shared page-fetch helper and a golden-file unit test.

**Architecture:** All v2-aware behavior lives in `includes/utilities.php` (`executeApiCall`, `validateApiResponse`, `processLogEntries`, plus a new `fetchAndStoreLogPage` helper). `functions.php` exposes two thin entry points — `backfillUserLogs` (full-history pagination on empty DB) and `syncUserLogs` (incremental fetch from last seen timestamp). `index.php` swaps its call sites. The only new behavior is in the helper module; everything else is either renamed or a one-line swap.

**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies. No framework. Pure CLI test runner (`php tests/process_log_entries_test.php`) using the existing `backup/api-sample-new.json` fixture.

## Global Constraints

- PHP 8.1+ syntax only.
- Database schema (`vault` table with `id TEXT PRIMARY KEY`) is **not** changed.
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries — operation must be idempotent across re-runs.
- API authentication: `Authorization: ApiKey <key>` header; **never** pass the key as a `?key=` query parameter.
- Pagination: follow `_metadata.links.next` URL until null. Never compute timestamps manually for pagination; v2 provides the cursor.
- Use the fixture at `backup/api-sample-new.json` (100 entries) for the golden-file test.
- All existing helpers outside the rewritten ones stay byte-identical (no drive-by refactors).
- Commit after each task. Do not push.

---

### Task 1: Add `LogEntryIncompleteException`

**Files:**
- Modify: `includes/exceptions.php` (append new class after the existing four)
- No test file — exercised via Task 5's golden test.

**Interfaces:**
- Consumes: none.
- Produces: a new exception class `LogEntryIncompleteException extends Exception`, with the same constructor signature as the existing four exception classes in the file.

- [ ] **Step 1: Append the new class to `includes/exceptions.php`**

Append after the existing `ApiValidationException` class, following the same doc-comment + constructor pattern used by the other four classes in the file:

```php
/**
 * Exception thrown when a log entry is missing required data fields.
 *
 * @category Exception
 * @package  TornVaultTracker
 * @author   Keith Solomon <ksolomon@gmail.com>
 * @license  Unlicense https://unlicense.org/
 * @link     https://github.com/ksolomon/Torn-Vault-Tracker
 */
class LogEntryIncompleteException extends Exception {
    /**
     * Constructor.
     *
     * @param string    $message  The exception message.
     * @param int       $code     The exception code.
     * @param Throwable $previous The previous throwable.
     */
    public function __construct($message, $code = 0, Throwable $previous = null) {
        parent::__construct($message, $code, $previous);
    }
}
```

- [ ] **Step 2: Verify file still parses**

Run: `php -l includes/exceptions.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add includes/exceptions.php
git commit -m "Add LogEntryIncompleteException for malformed v2 entries"
```

---

### Task 2: Rewrite `executeApiCall` for v2 authentication

**Files:**
- Modify: `includes/utilities.php` (replace the existing `executeApiCall` function body)
- No new test file for this task alone — exercised by Task 5's golden test and by the integration in Task 4.

**Interfaces:**
- Consumes: `$apiEndpoint` (full URL), `$apiKey` (string). Uses `\Exception` (existing) for `CurlErrorException` and `JsonDataException` from `includes/exceptions.php`.
- Produces: `array` — the decoded JSON response, or throws `CurlErrorException` / `JsonDataException`. Header is `Authorization: ApiKey <apiKey>` only (no `Content-Type`).

- [ ] **Step 1: Replace `executeApiCall` in `includes/utilities.php`**

Replace the existing function (currently takes only `$apiEndpoint`) with this v2-aware version:

```php
function executeApiCall($apiEndpoint, $apiKey) {
    if (empty($apiKey)) {
        throw new ApiKeyMissingException('API key is required for executeApiCall.');
    }

    $headers = ["Authorization: ApiKey $apiKey"];
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

    $response = curl_exec($ch);

    if (curl_errno($ch)) {
        throw new CurlErrorException('cURL error: ' . curl_error($ch));
    }

    curl_close($ch);

    $responseData = json_decode($response, true);

    if (json_last_error() !== JSON_ERROR_NONE) {
        throw new JsonDataException('Failed to decode JSON response: ' . json_last_error_msg());
    }

    return $responseData;
}
```

- [ ] **Step 2: Verify file still parses**

Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add includes/utilities.php
git commit -m "Rewrite executeApiCall for v2 API (Authorization header)"
```

---

### Task 3: Tighten `validateApiResponse` for v2 shape

**Files:**
- Modify: `includes/utilities.php` (replace the existing `validateApiResponse` function body)
- No test file for this task alone — exercised by Task 5's golden test.

**Interfaces:**
- Consumes: `$responseData` (array). Throws `ApiValidationException` (already defined).
- Produces: void. `log` must exist and be an array.

- [ ] **Step 1: Replace `validateApiResponse` in `includes/utilities.php`**

Replace the existing function with:

```php
function validateApiResponse($responseData) {
    if (!isset($responseData['log']) || !is_array($responseData['log'])) {
        throw new ApiValidationException('Invalid log data received from the API.');
    }
}
```

(The change from the old version is tightening `is_array($responseData['log'])` — v2 always returns an array; v1 returned an object-keyed map, which `is_array()` rejects.)

- [ ] **Step 2: Verify file still parses**

Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add includes/utilities.php
git commit -m "Tighten validateApiResponse for v2 array shape"
```

---

### Task 4: Add `fetchAndStoreLogPage` helper

**Files:**
- Modify: `includes/utilities.php` (append new function after `processLogEntries`)
- No test file for this task alone — exercised by Task 5 and the integration tests in Task 7.

**Interfaces:**
- Consumes: `$pdo` (PDO), `$user` (string — key into `USER_KEYS`), `$url` (string — full v2 API URL, possibly a `_metadata.links.next` URL). Throws `ApiKeyMissingException`, `CurlErrorException`, `JsonDataException`, `ApiValidationException` (all already defined).
- Produces: `?string` — the next page URL from `_metadata.links.next`, or `null` if no more pages.

- [ ] **Step 1: Add the new function to `includes/utilities.php`**

Append the following at the end of the file (after `consoleLog()`):

```php
/**
 * Fetch a single page of log entries from the Torn v2 API and store them.
 *
 * Performs one HTTP request, validates the response, inserts each vault
 * log entry (idempotently via INSERT ... ON CONFLICT), and returns the
 * pagination cursor (`_metadata.links.next`) if more pages remain.
 *
 * @param PDO    $pdo  Database connection used to insert vault entries.
 * @param string $user The user whose log entries are being fetched.
 * @param string $url  Full URL for the v2 API request.
 *
 * @throws ApiKeyMissingException If no API key is configured for the user.
 * @throws CurlErrorException     If the HTTP request fails.
 * @throws JsonDataException      If the response body is not valid JSON.
 * @throws ApiValidationException If the response is missing the `log` array.
 *
 * @return string|null The `_metadata.links.next` URL, or null when there are no more pages.
 */
function fetchAndStoreLogPage($pdo, $user, $url) {
    if (!array_key_exists($user, USER_KEYS)) {
        throw new ApiKeyMissingException("User does not have an API key configured.");
    }

    $responseData = executeApiCall($url, USER_KEYS[$user]);
    validateApiResponse($responseData);

    $insertStmt = $pdo->prepare(
        'INSERT INTO vault (id, user, timestamp, description, amount) '
        . 'VALUES (:id, :user, :timestamp, :description, :amount) '
        . 'ON CONFLICT(id) DO NOTHING'
    );

    processLogEntries($responseData['log'], $user, $insertStmt);

    return $responseData['_metadata']['links']['next'] ?? null;
}
```

- [ ] **Step 2: Verify file still parses**

Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add includes/utilities.php
git commit -m "Add fetchAndStoreLogPage helper for v2 pagination"
```

---

### Task 5: Rewrite `processLogEntries` for v2 shape (with golden-file test)

**Files:**
- Create: `tests/process_log_entries_test.php`
- Modify: `includes/utilities.php` (replace the existing `processLogEntries` function body)

**Interfaces:**
- Consumes: `$logEntries` (array of v2 entry objects), `$user` (string), `$insertStmt` (PDOStatement prepared with the `INSERT … ON CONFLICT(id) DO NOTHING` query). Throws `LogEntryIncompleteException` (new in Task 1).
- Produces: void. For each entry: extracts `id`, `timestamp`, `details.title`, `data.deposited` (deposit) or `data.withdrawn` (withdraw) and binds to `$insertStmt`, then `execute()`. Skips entries missing both `data.deposited` and `data.withdrawn` after logging via `consoleLog`.

- [ ] **Step 1: Write the failing test**

Create `tests/process_log_entries_test.php`:

```php
<?php
/**
 * Golden-file test for processLogEntries() v2 behavior.
 *
 * Loads backup/api-sample-new.json, runs each entry through the v2
 * processor against an in-memory SQLite database, and asserts the
 * resulting rows match the expected (id, user, timestamp, description,
 * amount) tuples derived from the fixture.
 *
 * Run: php tests/process_log_entries_test.php
 * Exit code 0 = pass.
 */

declare(strict_types=1);

// Make USER_KEYS available without including config.php (it would force a
// real DB connection).
if (!defined('USER_KEYS')) {
    define('USER_KEYS', [
        'zarathos' => 'test-key-not-used',
        'symos'    => 'test-key-not-used',
    ]);
}

require_once __DIR__ . '/../includes/exceptions.php';
require_once __DIR__ . '/../includes/utilities.php';

function fail(string $message): void {
    fwrite(STDERR, "FAIL: $message\n");
    exit(1);
}

function assertSame($expected, $actual, string $label): void {
    if ($expected !== $actual) {
        $exp = var_export($expected, true);
        $act = var_export($actual, true);
        fail("$label: expected $exp, got $act");
    }
}

// In-memory DB so we don't touch the user's vault.db.
$pdo = new PDO('sqlite::memory:');
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->exec(
    'CREATE TABLE vault (
        id          TEXT PRIMARY KEY,
        user        TEXT NOT NULL,
        timestamp   INTEGER NOT NULL,
        description TEXT NOT NULL,
        amount      REAL NOT NULL
    )'
);

$insertStmt = $pdo->prepare(
    'INSERT INTO vault (id, user, timestamp, description, amount) '
    . 'VALUES (:id, :user, :timestamp, :description, :amount) '
    . 'ON CONFLICT(id) DO NOTHING'
);

// Load the golden file.
$fixturePath = __DIR__ . '/../backup/api-sample-new.json';
if (!is_readable($fixturePath)) {
    fail("Fixture not readable at $fixturePath");
}
$fixture = json_decode(file_get_contents($fixturePath), true);
if (!is_array($fixture) || !isset($fixture['log'])) {
    fail('Fixture is missing the log array.');
}

processLogEntries($fixture['log'], 'zarathos', $insertStmt);

// 100 entries in the fixture.
$count = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count, 'row count');

// Spot-check first entry (newest, withdraw) and last (oldest, deposit).
$rows = $pdo->query('SELECT id, user, timestamp, description, amount FROM vault ORDER BY timestamp DESC')->fetchAll(PDO::FETCH_ASSOC);

$first = $rows[0];
assertSame('c1pEDQ7jl3kuBHV4NqDI', $first['id'], 'first.id');
assertSame('zarathos', $first['user'], 'first.user');
assertSame(1785325489, (int)$first['timestamp'], 'first.timestamp');
assertSame('Vault withdraw', $first['description'], 'first.description');
assertSame(-498870964, (int)$first['amount'], 'first.amount');

$last = $rows[count($rows) - 1];
assertSame('8m5jrUKTiKk0SB4MxzIy', $last['id'], 'last.id');
assertSame(1714350415, (int)$last['timestamp'], 'last.timestamp');
assertSame('Vault deposit', $last['description'], 'last.description');
assertSame(231492, (int)$last['amount'], 'last.amount');

// Re-running with the same fixture inserts zero new rows (idempotency).
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
$count2 = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count2, 'row count after re-run');

echo "OK: processLogEntries v2 golden-file test passed (100 entries).\n";
```

- [ ] **Step 2: Run the test and verify it fails**

Run: `php tests/process_log_entries_test.php`
Expected: PHP fatal error: `processLogEntries()` exists but reads `$entry['log']` (v1) and crashes when `log` is not a key — or returns zero rows. Either way the test will not print `OK: …`.

- [ ] **Step 3: Rewrite `processLogEntries` in `includes/utilities.php`**

Replace the existing `processLogEntries` function with:

```php
function processLogEntries($logEntries, $user, $insertStmt) {
    foreach ($logEntries as $entry) {
        $id = $entry['id'] ?? null;

        if (!$id) {
            consoleLog('Skipping entry with no id: ' . print_r($entry, true));
            continue;
        }

        $timestamp   = $entry['timestamp'] ?? null;
        $description = $entry['details']['title'] ?? null;
        $hasDeposit  = isset($entry['data']['deposited']);
        $hasWithdraw = isset($entry['data']['withdrawn']);

        if ($hasDeposit) {
            $amount = (int)$entry['data']['deposited'];
        } elseif ($hasWithdraw) {
            $amount = -((int)$entry['data']['withdrawn']);
        } else {
            consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
            continue;
        }

        if ($timestamp === null || $description === null) {
            throw new LogEntryIncompleteException(
                "Entry $id missing timestamp or details.title."
            );
        }

        $insertStmt->bindValue(':id', $id);
        $insertStmt->bindValue(':user', $user);
        $insertStmt->bindValue(':timestamp', $timestamp);
        $insertStmt->bindValue(':description', $description);
        $insertStmt->bindValue(':amount', $amount);
        $insertStmt->execute();
    }
}
```

Note: signature changed from `(array, string, PDOStatement, PDOStatement, bool)` to `(array, string, PDOStatement)`. The two statements (check/insert) collapsed into one ON CONFLICT insert, the `debug` flag is gone (debug logging was unused), and the `$debug` boolean parameter is dropped.

- [ ] **Step 4: Run the test and verify it passes**

Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`

- [ ] **Step 5: Commit**

```bash
git add tests/process_log_entries_test.php includes/utilities.php
git commit -m "Rewrite processLogEntries for v2 shape; add golden-file test"
```

---

### Task 6: Replace `firstRun`/`getLog` with `backfillUserLogs`/`syncUserLogs`

**Files:**
- Modify: `functions.php` (remove `firstRun`, remove `getLog`, add `backfillUserLogs` and `syncUserLogs`)
- No new test file — verified end-to-end in Task 8.

**Interfaces:**
- Consumes: `$user` (string — key into `USER_KEYS`). Uses `getDatabaseConnection()`, `dbNew()`, `fetchAndStoreLogPage()` (Task 4).
- Produces: `void`. Both functions are idempotent.

- [ ] **Step 1: Remove `firstRun` from `functions.php`**

Delete the entire `firstRun()` function (lines 72-119 in the current file, including the doc-comment block above it).

- [ ] **Step 2: Remove `getLog` from `functions.php`**

Delete the entire `getLog()` function (lines 132-153 in the current file, including the doc-comment block above it).

- [ ] **Step 3: Add `backfillUserLogs` and `syncUserLogs`**

Insert the following in place of the deleted functions:

```php
/**
 * Pulls and stores the full vault transaction log history for a user.
 *
 * Pages through the v2 Torn API using `_metadata.links.next` until the
 * API reports no further pages. Intended for first-run use when the
 * local database is empty. Idempotent: re-running on a partially
 * populated database inserts only new entries (ON CONFLICT DO NOTHING).
 *
 * @param string $user The user whose logs should be fetched.
 *
 * @return void
 */
function backfillUserLogs($user) {
    $url  = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
    $pdo  = getDatabaseConnection();

    do {
        $next = fetchAndStoreLogPage($pdo, $user, $url);
        $url  = $next;
    } while ($next !== null);
}

/**
 * Synchronizes recent vault transactions for a user.
 *
 * On a non-empty database, fetches only entries newer than the user's
 * most recent row. Falls back to full backfill when the database is
 * empty.
 *
 * @param string $user The user whose logs should be synced.
 *
 * @return void
 */
function syncUserLogs($user) {
    if (dbNew()) {
        backfillUserLogs($user);
        return;
    }

    $pdo = getDatabaseConnection();

    $stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
    $stmt->bindValue(':user', $user);
    $stmt->execute();
    $lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];

    $url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);

    do {
        $next = fetchAndStoreLogPage($pdo, $user, $url);
        $url  = $next;
    } while ($next !== null);
}
```

- [ ] **Step 4: Verify the file still parses**

Run: `php -l functions.php`
Expected: `No syntax errors detected`.

- [ ] **Step 5: Commit**

```bash
git add functions.php
git commit -m "Replace firstRun/getLog with backfillUserLogs/syncUserLogs"
```

---

### Task 7: Update `index.php` call sites

**Files:**
- Modify: `index.php` (rename two function calls in the `dbNew()` branches)

**Interfaces:**
- Consumes: existing `index.php` control flow (calls into `firstRun` / `getLog`).
- Produces: same control flow, but calling `backfillUserLogs` / `syncUserLogs` instead.

- [ ] **Step 1: Replace `firstRun` with `backfillUserLogs`**

In `index.php`, replace `firstRun($key)` with `backfillUserLogs($key)`.

The result should read:

```php
if (dbNew()) {
    foreach (USER_KEYS as $key => $value) {
        backfillUserLogs($key);
    }
    header('Location: /');
} else {
    foreach (USER_KEYS as $key => $value) {
        syncUserLogs($key);
    }
}
```

- [ ] **Step 2: Replace `getLog` with `syncUserLogs`**

In the same file, in the `else` branch, replace `getLog($key)` with `syncUserLogs($key)`.

- [ ] **Step 3: Verify the file still parses**

Run: `php -l index.php`
Expected: `No syntax errors detected`.

- [ ] **Step 4: Commit**

```bash
git add index.php
git commit -m "Update index.php to call backfillUserLogs/syncUserLogs"
```

---

### Task 8: End-to-end smoke check

**Files:**
- No code changes. Just verification steps.

- [ ] **Step 1: Run the unit test once more**

Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`

- [ ] **Step 2: Verify no remaining references to the v1 functions**

Run:

```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -rn "firstRun\|?key=\|api.torn.com/user/" --include="*.php" .
```

Expected: no matches (except possibly `backup/` reference files, which are read-only fixtures). The grep should report nothing under `functions.php`, `index.php`, `includes/`, or `tests/`.

- [ ] **Step 3: (Optional) Run `phpcs` against the touched files**

Run: `phpcs functions.php index.php includes/utilities.php includes/exceptions.php tests/process_log_entries_test.php`

If `phpcs` is installed and clean: skip Step 4. If it reports issues, fix them and commit. (The project's existing `phpcs.xml` is the source of style rules; there is no CI check, so this is a manual sweep.)

---

## Self-Review Notes

**Spec coverage:**

| Spec requirement | Task |
|---|---|
| Single v2 path for both bootstrap and steady-state | Task 6 (single `fetchAndStoreLogPage` consumed by both) |
| `Authorization: ApiKey` header, no `?key=` | Task 2 |
| `_metadata.links.next` pagination | Task 4 (helper returns it) + Task 6 (loop in both entry points) |
| Entry id as `TEXT PRIMARY KEY` with `ON CONFLICT DO NOTHING` | Task 4 (insert stmt) + Task 5 (test asserts idempotency) |
| LogEntryIncompleteException for malformed entries | Task 1 (class) + Task 5 (throw site) |
| Golden-file test against `backup/api-sample-new.json` | Task 5 |
| `firstRun` → `backfillUserLogs`, `getLog` → `syncUserLogs` | Task 6 (definitions) + Task 7 (call sites) |
| DB schema unchanged, UI helpers unchanged | Tasks 6 & 7 only swap the log-fetching entry points; `buildTable`, `generateBalance`, `getSpace`, `vaultLoop`, `fetchVaultRecords`, `dbNew`, `getDatabaseConnection` all untouched. |
| Tests verified manually via `php tests/...` | Task 5 (initial run) + Task 8 (final run) |
| `config.php`, `style.css`, `script.js`, `backup/`, README untouched | No task touches them. (Spec said README update was optional; we skip it.) |

**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions in the plan.

**Type consistency:** The `executeApiCall(string $url, string $apiKey)` signature in Task 2 matches the call site `executeApiCall($url, USER_KEYS[$user])` in Task 4. The `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): ?string` signature in Task 4 matches the loop body in both `backfillUserLogs` and `syncUserLogs` in Task 6. The `processLogEntries(array, string, PDOStatement)` signature in Task 5 matches the call site `processLogEntries($responseData['log'], $user, $insertStmt)` in Task 4. The `dbNew()` function called in Task 6 is unchanged and already exists in `includes/utilities.php`.