Torn-Vault-Tracker/docs/superpowers/plans/2026-08-03-torn-api-v2-migration.md

Torn API v1 → v2 Migration Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Rewrite Torn Vault Tracker's log-ingestion code to consume the new v2 Torn API JSON shape (array-of-entries with _metadata.links.next pagination and Authorization: ApiKey header), with a single shared page-fetch helper and a golden-file unit test.

Architecture: All v2-aware behavior lives in includes/utilities.php (executeApiCall, validateApiResponse, processLogEntries, plus a new fetchAndStoreLogPage helper). functions.php exposes two thin entry points — backfillUserLogs (full-history pagination on empty DB) and syncUserLogs (incremental fetch from last seen timestamp). index.php swaps its call sites. The only new behavior is in the helper module; everything else is either renamed or a one-line swap.

Tech Stack: PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies. No framework. Pure CLI test runner (php tests/process_log_entries_test.php) using the existing backup/api-sample-new.json fixture.

Global Constraints

• PHP 8.1+ syntax only.
• Database schema (vault table with id TEXT PRIMARY KEY) is not changed.
• INSERT … ON CONFLICT(id) DO NOTHING is the only INSERT form used for log entries — operation must be idempotent across re-runs.
• API authentication: Authorization: ApiKey <key> header; never pass the key as a ?key= query parameter.
• Pagination: follow _metadata.links.next URL until null. Never compute timestamps manually for pagination; v2 provides the cursor.
• Use the fixture at backup/api-sample-new.json (100 entries) for the golden-file test.
• All existing helpers outside the rewritten ones stay byte-identical (no drive-by refactors).
• Commit after each task. Do not push.

---

Task 1: Add LogEntryIncompleteException

Files:

• Modify: includes/exceptions.php (append new class after the existing four)
• No test file — exercised via Task 5's golden test.

Interfaces:

• Consumes: none.

• Produces: a new exception class LogEntryIncompleteException extends Exception, with the same constructor signature as the existing four exception classes in the file.

• Step 1: Append the new class to includes/exceptions.php

Append after the existing ApiValidationException class, following the same doc-comment + constructor pattern used by the other four classes in the file:

/**
 * Exception thrown when a log entry is missing required data fields.
 *
 * @category Exception
 * @package  TornVaultTracker
 * @author   Keith Solomon <ksolomon@gmail.com>
 * @license  Unlicense https://unlicense.org/
 * @link     https://github.com/ksolomon/Torn-Vault-Tracker
 */
class LogEntryIncompleteException extends Exception {
    /**
     * Constructor.
     *
     * @param string    $message  The exception message.
     * @param int       $code     The exception code.
     * @param Throwable $previous The previous throwable.
     */
    public function __construct($message, $code = 0, Throwable $previous = null) {
        parent::__construct($message, $code, $previous);
    }
}

• Step 2: Verify file still parses

Run: php -l includes/exceptions.php Expected: No syntax errors detected.

• Step 3: Commit

git add includes/exceptions.php
git commit -m "Add LogEntryIncompleteException for malformed v2 entries"

---

Task 2: Rewrite executeApiCall for v2 authentication

Files:

• Modify: includes/utilities.php (replace the existing executeApiCall function body)
• No new test file for this task alone — exercised by Task 5's golden test and by the integration in Task 4.

Interfaces:

• Consumes: $apiEndpoint (full URL), $apiKey (string). Uses \Exception (existing) for CurlErrorException and JsonDataException from includes/exceptions.php.

• Produces: array — the decoded JSON response, or throws CurlErrorException / JsonDataException. Header is Authorization: ApiKey <apiKey> only (no Content-Type).

• Step 1: Replace executeApiCall in includes/utilities.php

Replace the existing function (currently takes only $apiEndpoint) with this v2-aware version:

function executeApiCall($apiEndpoint, $apiKey) {
    if (empty($apiKey)) {
        throw new ApiKeyMissingException('API key is required for executeApiCall.');
    }

    $headers = ["Authorization: ApiKey $apiKey"];
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

    $response = curl_exec($ch);

    if (curl_errno($ch)) {
        throw new CurlErrorException('cURL error: ' . curl_error($ch));
    }

    curl_close($ch);

    $responseData = json_decode($response, true);

    if (json_last_error() !== JSON_ERROR_NONE) {
        throw new JsonDataException('Failed to decode JSON response: ' . json_last_error_msg());
    }

    return $responseData;
}

• Step 2: Verify file still parses

Run: php -l includes/utilities.php Expected: No syntax errors detected.

• Step 3: Commit

git add includes/utilities.php
git commit -m "Rewrite executeApiCall for v2 API (Authorization header)"

---

Task 3: Tighten validateApiResponse for v2 shape

Files:

• Modify: includes/utilities.php (replace the existing validateApiResponse function body)
• No test file for this task alone — exercised by Task 5's golden test.

Interfaces:

• Consumes: $responseData (array). Throws ApiValidationException (already defined).

• Produces: void. log must exist and be an array.

• Step 1: Replace validateApiResponse in includes/utilities.php

Replace the existing function with:

function validateApiResponse($responseData) {
    if (!isset($responseData['log']) || !is_array($responseData['log'])) {
        throw new ApiValidationException('Invalid log data received from the API.');
    }
}

(The change from the old version is tightening is_array($responseData['log']) — v2 always returns an array; v1 returned an object-keyed map, which is_array() rejects.)

• Step 2: Verify file still parses

Run: php -l includes/utilities.php Expected: No syntax errors detected.

• Step 3: Commit

git add includes/utilities.php
git commit -m "Tighten validateApiResponse for v2 array shape"

---

Task 4: Add fetchAndStoreLogPage helper

Files:

• Modify: includes/utilities.php (append new function after processLogEntries)
• No test file for this task alone — exercised by Task 5 and the integration tests in Task 7.

Interfaces:

• Consumes: $pdo (PDO), $user (string — key into USER_KEYS), $url (string — full v2 API URL, possibly a _metadata.links.next URL). Throws ApiKeyMissingException, CurlErrorException, JsonDataException, ApiValidationException (all already defined).

• Produces: ?string — the next page URL from _metadata.links.next, or null if no more pages.

• Step 1: Add the new function to includes/utilities.php

Append the following at the end of the file (after consoleLog()):

/**
 * Fetch a single page of log entries from the Torn v2 API and store them.
 *
 * Performs one HTTP request, validates the response, inserts each vault
 * log entry (idempotently via INSERT ... ON CONFLICT), and returns the
 * pagination cursor (`_metadata.links.next`) if more pages remain.
 *
 * @param PDO    $pdo  Database connection used to insert vault entries.
 * @param string $user The user whose log entries are being fetched.
 * @param string $url  Full URL for the v2 API request.
 *
 * @throws ApiKeyMissingException If no API key is configured for the user.
 * @throws CurlErrorException     If the HTTP request fails.
 * @throws JsonDataException      If the response body is not valid JSON.
 * @throws ApiValidationException If the response is missing the `log` array.
 *
 * @return string|null The `_metadata.links.next` URL, or null when there are no more pages.
 */
function fetchAndStoreLogPage($pdo, $user, $url) {
    if (!array_key_exists($user, USER_KEYS)) {
        throw new ApiKeyMissingException("User does not have an API key configured.");
    }

    $responseData = executeApiCall($url, USER_KEYS[$user]);
    validateApiResponse($responseData);

    $insertStmt = $pdo->prepare(
        'INSERT INTO vault (id, user, timestamp, description, amount) '
        . 'VALUES (:id, :user, :timestamp, :description, :amount) '
        . 'ON CONFLICT(id) DO NOTHING'
    );

    processLogEntries($responseData['log'], $user, $insertStmt);

    return $responseData['_metadata']['links']['next'] ?? null;
}

• Step 2: Verify file still parses

Run: php -l includes/utilities.php Expected: No syntax errors detected.

• Step 3: Commit

git add includes/utilities.php
git commit -m "Add fetchAndStoreLogPage helper for v2 pagination"

---

Task 5: Rewrite processLogEntries for v2 shape (with golden-file test)

Files:

• Create: tests/process_log_entries_test.php
• Modify: includes/utilities.php (replace the existing processLogEntries function body)

Interfaces:

• Consumes: $logEntries (array of v2 entry objects), $user (string), $insertStmt (PDOStatement prepared with the INSERT … ON CONFLICT(id) DO NOTHING query). Throws LogEntryIncompleteException (new in Task 1).

• Produces: void. For each entry: extracts id, timestamp, details.title, data.deposited (deposit) or data.withdrawn (withdraw) and binds to $insertStmt, then execute(). Skips entries missing both data.deposited and data.withdrawn after logging via consoleLog.

• Step 1: Write the failing test

Create tests/process_log_entries_test.php:

<?php
/**
 * Golden-file test for processLogEntries() v2 behavior.
 *
 * Loads backup/api-sample-new.json, runs each entry through the v2
 * processor against an in-memory SQLite database, and asserts the
 * resulting rows match the expected (id, user, timestamp, description,
 * amount) tuples derived from the fixture.
 *
 * Run: php tests/process_log_entries_test.php
 * Exit code 0 = pass.
 */

declare(strict_types=1);

// Make USER_KEYS available without including config.php (it would force a
// real DB connection).
if (!defined('USER_KEYS')) {
    define('USER_KEYS', [
        'zarathos' => 'test-key-not-used',
        'symos'    => 'test-key-not-used',
    ]);
}

require_once __DIR__ . '/../includes/exceptions.php';
require_once __DIR__ . '/../includes/utilities.php';

function fail(string $message): void {
    fwrite(STDERR, "FAIL: $message\n");
    exit(1);
}

function assertSame($expected, $actual, string $label): void {
    if ($expected !== $actual) {
        $exp = var_export($expected, true);
        $act = var_export($actual, true);
        fail("$label: expected $exp, got $act");
    }
}

// In-memory DB so we don't touch the user's vault.db.
$pdo = new PDO('sqlite::memory:');
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->exec(
    'CREATE TABLE vault (
        id          TEXT PRIMARY KEY,
        user        TEXT NOT NULL,
        timestamp   INTEGER NOT NULL,
        description TEXT NOT NULL,
        amount      REAL NOT NULL
    )'
);

$insertStmt = $pdo->prepare(
    'INSERT INTO vault (id, user, timestamp, description, amount) '
    . 'VALUES (:id, :user, :timestamp, :description, :amount) '
    . 'ON CONFLICT(id) DO NOTHING'
);

// Load the golden file.
$fixturePath = __DIR__ . '/../backup/api-sample-new.json';
if (!is_readable($fixturePath)) {
    fail("Fixture not readable at $fixturePath");
}
$fixture = json_decode(file_get_contents($fixturePath), true);
if (!is_array($fixture) || !isset($fixture['log'])) {
    fail('Fixture is missing the log array.');
}

processLogEntries($fixture['log'], 'zarathos', $insertStmt);

// 100 entries in the fixture.
$count = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count, 'row count');

// Spot-check first entry (newest, withdraw) and last (oldest, deposit).
$rows = $pdo->query('SELECT id, user, timestamp, description, amount FROM vault ORDER BY timestamp DESC')->fetchAll(PDO::FETCH_ASSOC);

$first = $rows[0];
assertSame('c1pEDQ7jl3kuBHV4NqDI', $first['id'], 'first.id');
assertSame('zarathos', $first['user'], 'first.user');
assertSame(1785325489, (int)$first['timestamp'], 'first.timestamp');
assertSame('Vault withdraw', $first['description'], 'first.description');
assertSame(-498870964, (int)$first['amount'], 'first.amount');

$last = $rows[count($rows) - 1];
assertSame('8m5jrUKTiKk0SB4MxzIy', $last['id'], 'last.id');
assertSame(1714350415, (int)$last['timestamp'], 'last.timestamp');
assertSame('Vault deposit', $last['description'], 'last.description');
assertSame(231492, (int)$last['amount'], 'last.amount');

// Re-running with the same fixture inserts zero new rows (idempotency).
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
$count2 = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count2, 'row count after re-run');

echo "OK: processLogEntries v2 golden-file test passed (100 entries).\n";

• Step 2: Run the test and verify it fails

Run: php tests/process_log_entries_test.php Expected: PHP fatal error: processLogEntries() exists but reads $entry['log'] (v1) and crashes when log is not a key — or returns zero rows. Either way the test will not print OK: ….

• Step 3: Rewrite processLogEntries in includes/utilities.php

Replace the existing processLogEntries function with:

function processLogEntries($logEntries, $user, $insertStmt) {
    foreach ($logEntries as $entry) {
        $id = $entry['id'] ?? null;

        if (!$id) {
            consoleLog('Skipping entry with no id: ' . print_r($entry, true));
            continue;
        }

        $timestamp   = $entry['timestamp'] ?? null;
        $description = $entry['details']['title'] ?? null;
        $hasDeposit  = isset($entry['data']['deposited']);
        $hasWithdraw = isset($entry['data']['withdrawn']);

        if ($hasDeposit) {
            $amount = (int)$entry['data']['deposited'];
        } elseif ($hasWithdraw) {
            $amount = -((int)$entry['data']['withdrawn']);
        } else {
            consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
            continue;
        }

        if ($timestamp === null || $description === null) {
            throw new LogEntryIncompleteException(
                "Entry $id missing timestamp or details.title."
            );
        }

        $insertStmt->bindValue(':id', $id);
        $insertStmt->bindValue(':user', $user);
        $insertStmt->bindValue(':timestamp', $timestamp);
        $insertStmt->bindValue(':description', $description);
        $insertStmt->bindValue(':amount', $amount);
        $insertStmt->execute();
    }
}

Note: signature changed from (array, string, PDOStatement, PDOStatement, bool) to (array, string, PDOStatement). The two statements (check/insert) collapsed into one ON CONFLICT insert, the debug flag is gone (debug logging was unused), and the $debug boolean parameter is dropped.

• Step 4: Run the test and verify it passes

Run: php tests/process_log_entries_test.php Expected: OK: processLogEntries v2 golden-file test passed (100 entries).

• Step 5: Commit

git add tests/process_log_entries_test.php includes/utilities.php
git commit -m "Rewrite processLogEntries for v2 shape; add golden-file test"

---

Task 6: Replace firstRun/getLog with backfillUserLogs/syncUserLogs

Files:

• Modify: functions.php (remove firstRun, remove getLog, add backfillUserLogs and syncUserLogs)
• No new test file — verified end-to-end in Task 8.

Interfaces:

• Consumes: $user (string — key into USER_KEYS). Uses getDatabaseConnection(), dbNew(), fetchAndStoreLogPage() (Task 4).

• Produces: void. Both functions are idempotent.

• Step 1: Remove firstRun from functions.php

Delete the entire firstRun() function (lines 72-119 in the current file, including the doc-comment block above it).

• Step 2: Remove getLog from functions.php

Delete the entire getLog() function (lines 132-153 in the current file, including the doc-comment block above it).

• Step 3: Add backfillUserLogs and syncUserLogs

Insert the following in place of the deleted functions:

/**
 * Pulls and stores the full vault transaction log history for a user.
 *
 * Pages through the v2 Torn API using `_metadata.links.next` until the
 * API reports no further pages. Intended for first-run use when the
 * local database is empty. Idempotent: re-running on a partially
 * populated database inserts only new entries (ON CONFLICT DO NOTHING).
 *
 * @param string $user The user whose logs should be fetched.
 *
 * @return void
 */
function backfillUserLogs($user) {
    $url  = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
    $pdo  = getDatabaseConnection();

    do {
        $next = fetchAndStoreLogPage($pdo, $user, $url);
        $url  = $next;
    } while ($next !== null);
}

/**
 * Synchronizes recent vault transactions for a user.
 *
 * On a non-empty database, fetches only entries newer than the user's
 * most recent row. Falls back to full backfill when the database is
 * empty.
 *
 * @param string $user The user whose logs should be synced.
 *
 * @return void
 */
function syncUserLogs($user) {
    if (dbNew()) {
        backfillUserLogs($user);
        return;
    }

    $pdo = getDatabaseConnection();

    $stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
    $stmt->bindValue(':user', $user);
    $stmt->execute();
    $lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];

    $url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);

    do {
        $next = fetchAndStoreLogPage($pdo, $user, $url);
        $url  = $next;
    } while ($next !== null);
}

• Step 4: Verify the file still parses

Run: php -l functions.php Expected: No syntax errors detected.

• Step 5: Commit

git add functions.php
git commit -m "Replace firstRun/getLog with backfillUserLogs/syncUserLogs"

---

Task 7: Update index.php call sites

Files:

• Modify: index.php (rename two function calls in the dbNew() branches)

Interfaces:

• Consumes: existing index.php control flow (calls into firstRun / getLog).

• Produces: same control flow, but calling backfillUserLogs / syncUserLogs instead.

• Step 1: Replace firstRun with backfillUserLogs

In index.php, replace firstRun($key) with backfillUserLogs($key).

The result should read:

if (dbNew()) {
    foreach (USER_KEYS as $key => $value) {
        backfillUserLogs($key);
    }
    header('Location: /');
} else {
    foreach (USER_KEYS as $key => $value) {
        syncUserLogs($key);
    }
}

• Step 2: Replace getLog with syncUserLogs

In the same file, in the else branch, replace getLog($key) with syncUserLogs($key).

• Step 3: Verify the file still parses

Run: php -l index.php Expected: No syntax errors detected.

• Step 4: Commit

git add index.php
git commit -m "Update index.php to call backfillUserLogs/syncUserLogs"

---

Task 8: End-to-end smoke check

Files:

• No code changes. Just verification steps.

• Step 1: Run the unit test once more

Run: php tests/process_log_entries_test.php Expected: OK: processLogEntries v2 golden-file test passed (100 entries).

• Step 2: Verify no remaining references to the v1 functions

Run:

cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -rn "firstRun\|?key=\|api.torn.com/user/" --include="*.php" .

Expected: no matches (except possibly backup/ reference files, which are read-only fixtures). The grep should report nothing under functions.php, index.php, includes/, or tests/.

• Step 3: (Optional) Run phpcs against the touched files

Run: phpcs functions.php index.php includes/utilities.php includes/exceptions.php tests/process_log_entries_test.php

If phpcs is installed and clean: skip Step 4. If it reports issues, fix them and commit. (The project's existing phpcs.xml is the source of style rules; there is no CI check, so this is a manual sweep.)

---

Self-Review Notes

Spec coverage:

Spec requirement	Task
Single v2 path for both bootstrap and steady-state	Task 6 (single fetchAndStoreLogPage consumed by both)
Authorization: ApiKey header, no ?key=	Task 2
_metadata.links.next pagination	Task 4 (helper returns it) + Task 6 (loop in both entry points)
Entry id as TEXT PRIMARY KEY with ON CONFLICT DO NOTHING	Task 4 (insert stmt) + Task 5 (test asserts idempotency)
LogEntryIncompleteException for malformed entries	Task 1 (class) + Task 5 (throw site)
Golden-file test against backup/api-sample-new.json	Task 5
firstRun → backfillUserLogs, getLog → syncUserLogs	Task 6 (definitions) + Task 7 (call sites)
DB schema unchanged, UI helpers unchanged	Tasks 6 & 7 only swap the log-fetching entry points; buildTable, generateBalance, getSpace, vaultLoop, fetchVaultRecords, dbNew, getDatabaseConnection all untouched.
Tests verified manually via php tests/...	Task 5 (initial run) + Task 8 (final run)
config.php, style.css, script.js, backup/, README untouched	No task touches them. (Spec said README update was optional; we skip it.)

Placeholder scan: No "TODO", "TBD", "implement later", or vague instructions in the plan.

Type consistency: The executeApiCall(string $url, string $apiKey) signature in Task 2 matches the call site executeApiCall($url, USER_KEYS[$user]) in Task 4. The fetchAndStoreLogPage(PDO $pdo, string $user, string $url): ?string signature in Task 4 matches the loop body in both backfillUserLogs and syncUserLogs in Task 6. The processLogEntries(array, string, PDOStatement) signature in Task 5 matches the call site processLogEntries($responseData['log'], $user, $insertStmt) in Task 4. The dbNew() function called in Task 6 is unchanged and already exists in includes/utilities.php.