# Vault Balance From Per-Entry API Value — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Fix the vault balance display so it shows the live vault total (per the Torn v2 API's `data.balance` field) instead of `SUM(amount)` of stored entries. The v2 API only returns the most recent ~100 entries per user, so summing deltas cannot yield the true current balance — only the per-entry running balance can.

**Architecture:** Add a `running_balance INTEGER` column to the `vault` table. Capture it from `$entry['data']['balance']` in `processLogEntries`. Replace balance SQL from `SUM(amount)` to `MAX(running_balance)` per user. Add a one-shot live-read backfill that broadcasts the latest entry's `data.balance` to all NULL rows of a user. Add a "Balance After" column to the rendered table.

**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.

## Global Constraints

- PHP 8.1+ syntax only.
- Database schema gains one column (`running_balance INTEGER`). Existing columns and their semantics are unchanged.
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries.
- API authentication: `Authorization: ApiKey <key>` header.
- Pagination: follow `_metadata.links.next` URL until null.
- Idempotency: every change must be safe to re-run (especially the schema migration and the backfill).
- Commits per task. Do not push.
- All existing helpers outside the rewritten ones stay byte-identical unless the task explicitly says otherwise.

---

### Task 1: Schema — add `running_balance` column (idempotent on existing DBs)

**Files:**
- Modify: `functions.php` (the `getDatabaseConnection` function only)

**Interfaces:**
- Consumes: PDO from `PDO(DB_DSN, …)`. Reads `PRAGMA table_info(vault)` to detect an existing column.
- Produces: a `vault` table whose schema includes `running_balance INTEGER`. For existing DBs, the column is added via `ALTER TABLE`.

- [ ] **Step 1: Update the CREATE TABLE statement**

In `functions.php`, replace the `CREATE TABLE IF NOT EXISTS vault` (currently at lines 32-38) with the version that includes `running_balance`:

```php
$createTableSQL = "CREATE TABLE IF NOT EXISTS vault (
    id              TEXT PRIMARY KEY,
    user            TEXT NOT NULL,
    timestamp       INTEGER NOT NULL,
    description     TEXT NOT NULL,
    amount          REAL NOT NULL,
    running_balance INTEGER
);";
```

- [ ] **Step 2: Add the idempotent ALTER TABLE for existing DBs**

Replace the `if ($pdo === null)` block's `$pdo->exec($createTableSQL);` call with the following code (keep the rest of the block intact):

```php
$pdo->exec($createTableSQL);

// Add the running_balance column for databases created before the
// column was added. Idempotent: skip if it already exists.
$columns = $pdo->query("PRAGMA table_info(vault)")->fetchAll(PDO::FETCH_COLUMN, 1);
if (!in_array('running_balance', $columns, true)) {
    $pdo->exec('ALTER TABLE vault ADD COLUMN running_balance INTEGER');
}
```

- [ ] **Step 3: Verify the file parses**

Run: `php -l functions.php`
Expected: `No syntax errors detected in functions.php`.

- [ ] **Step 4: Verify the migration is idempotent on the live DB**

Run once:

```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php -r "
\$db = new PDO('sqlite:data/vault.db');
foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; }
"
```

Expected: `running_balance INTEGER` appears in the column list.

Run a second time (Page-load will do this; or re-run the script above twice in a row). Expected: no error (the `ALTER TABLE` fires only when the column is missing).

- [ ] **Step 5: Commit**

```bash
git add functions.php
git commit -m "Schema: add running_balance column to vault table"
```

---

### Task 2: Rewrite `fetchAndStoreLogPage` and `processLogEntries` to capture `data.balance` (with failing test)

**Files:**
- Modify: `includes/utilities.php` (`fetchAndStoreLogPage` and `processLogEntries`)
- Modify: `tests/process_log_entries_test.php` (extend to assert `running_balance` round-trips)

**Interfaces:**
- `processLogEntries($logEntries, $user, $insertStmt)`: reads `$entry['data']['balance']` (int) and binds it as `:running_balance`. If `data.balance` is missing, binds NULL.
- `fetchAndStoreLogPage($pdo, $user, $url)`: INSERT statement now includes `running_balance`.

- [ ] **Step 1: Extend the failing test**

In `tests/process_log_entries_test.php`, after the existing "row count after re-run" assertion, add these new assertions for the `running_balance` column. The fixture has 100 entries; the most recent entry's `data.balance` is `69235831` (newest, withdraw) and the oldest entry's `data.balance` is `509622821` (oldest, deposit) — both per the fixture verified during task planning.

Append after the existing "row count after re-run" block:

```php
// round-trip the running_balance column from $entry['data']['balance']
$firstRunning = (int)$first['running_balance'];
$lastRunning  = (int)$last['running_balance'];
assertSame(69235831, $firstRunning, 'first.running_balance');
assertSame(509622821, $lastRunning, 'last.running_balance');
```

- [ ] **Step 2: Run the test and verify it fails**

Run: `php tests/process_log_entries_test.php`
Expected: failure with `first.running_balance: expected 69235831, got ` (likely 0 or NULL because the column doesn't get inserted yet).

- [ ] **Step 3: Update `processLogEntries`**

In `includes/utilities.php`, replace the existing `processLogEntries` function (currently at lines 122-157) with:

```php
function processLogEntries($logEntries, $user, $insertStmt) {
    foreach ($logEntries as $entry) {
        $id = $entry['id'] ?? null;

        if (!$id) {
            consoleLog('Skipping entry with no id: ' . print_r($entry, true));
            continue;
        }

        $timestamp   = $entry['timestamp'] ?? null;
        $description = $entry['details']['title'] ?? null;
        $hasDeposit  = isset($entry['data']['deposited']);
        $hasWithdraw = isset($entry['data']['withdrawn']);

        if ($hasDeposit) {
            $amount = (int)$entry['data']['deposited'];
        } elseif ($hasWithdraw) {
            $amount = -((int)$entry['data']['withdrawn']);
        } else {
            consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
            continue;
        }

        if ($timestamp === null || $description === null) {
            consoleLog('Skipping entry ' . $id . ' missing timestamp or details.title: ' . print_r($entry, true));
            continue;
        }

        $runningBalance = isset($entry['data']['balance']) ? (int)$entry['data']['balance'] : null;

        $insertStmt->bindValue(':id', $id);
        $insertStmt->bindValue(':user', $user);
        $insertStmt->bindValue(':timestamp', $timestamp);
        $insertStmt->bindValue(':description', $description);
        $insertStmt->bindValue(':amount', $amount);
        $insertStmt->bindValue(':running_balance', $runningBalance);
        $insertStmt->execute();
    }
}
```

- [ ] **Step 4: Update `fetchAndStoreLogPage`'s INSERT statement**

In `includes/utilities.php`, replace the `$insertStmt` block inside `fetchAndStoreLogPage` (currently at lines 252-256) with:

```php
$insertStmt = $pdo->prepare(
    'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
    . 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
    . 'ON CONFLICT(id) DO NOTHING'
);
```

- [ ] **Step 5: Run the test and verify it passes**

Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`

- [ ] **Step 6: Verify the test file still parses**

Run: `php -l tests/process_log_entries_test.php`
Expected: `No syntax errors detected`.

- [ ] **Step 7: Commit**

```bash
git add includes/utilities.php tests/process_log_entries_test.php
git commit -m "Capture running_balance from v2 API data.balance"
```

---

### Task 3: Update `fetchVaultRecords` to include `running_balance`

**Files:**
- Modify: `includes/utilities.php` (`fetchVaultRecords` only)

**Interfaces:**
- Consumes: existing signature unchanged.
- Produces: rows that include the `running_balance` column (existing callers that ignore unknown keys continue to work).

- [ ] **Step 1: Replace `fetchVaultRecords` with explicit column list**

In `includes/utilities.php`, replace the body of `fetchVaultRecords` (currently at lines 168-189) with:

```php
function fetchVaultRecords($user = null) {
    $pdo = getDatabaseConnection();

    if ($user) {
        $query  = "SELECT id, user, timestamp, description, amount, running_balance FROM vault WHERE user = :user ORDER BY timestamp DESC";
        $params = [':user' => $user];
    } else {
        $query  = "SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC";
        $params = [];
    }

    try {
        $stmt = $pdo->prepare($query);
        $stmt->execute($params);

        return $stmt->fetchAll(PDO::FETCH_ASSOC);
    } catch (PDOException $e) {
        echo "Error fetching records: " . $e->getMessage();

        return [];
    }
}
```

- [ ] **Step 2: Verify the file parses**

Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add includes/utilities.php
git commit -m "fetchVaultRecords: explicit column list including running_balance"
```

---

### Task 4: Switch `vaultLoop` / `generateBalance` / `getSpace` to MAX(running_balance)

**Files:**
- Modify: `functions.php` (the three functions)

**Interfaces:**
- `vaultLoop($name=null)`: returns the per-user vault balance (the most recent entry's `running_balance`) for the given user, or the sum of those across all users when `$name` is null. Goes directly to PDO instead of fetching records.
- `generateBalance($name=null)`: continues to wrap `vaultLoop`, returns the formatted value.
- `getSpace($name=null)`: continues to compute `limit - balance` against the same value.

- [ ] **Step 1: Replace `vaultLoop` with a SQL-based implementation**

In `functions.php`, replace `vaultLoop` (currently at lines 199-208) with:

```php
function vaultLoop ($name=null) {
    $pdo = getDatabaseConnection();

    if ($name === null) {
        $stmt = $pdo->query(
            'SELECT COALESCE(SUM(max_balance), 0) FROM '
            . '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
        );

        return (int)$stmt->fetchColumn();
    }

    $stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
    $stmt->bindValue(':user', $name);
    $stmt->execute();

    return (int)$stmt->fetchColumn();
}
```

- [ ] **Step 2: Verify the file parses**

Run: `php -l functions.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add functions.php
git commit -m "vaultLoop: use MAX(running_balance) per user instead of SUM(amount)"
```

---

### Task 5: Add `backfillRunningBalances` helper

**Files:**
- Modify: `includes/utilities.php` (append new function after `consoleLog`)

**Interfaces:**
- `backfillRunningBalances(PDO $pdo, string $user): void`
- Counts rows where `running_balance IS NULL` for the user. If zero, returns. Otherwise hits the API with `selections=log&log=5850,5851&limit=1&sort=DESC` and uses the first entry's `data.balance` to UPDATE all NULL rows for that user.

- [ ] **Step 1: Append the new function**

In `includes/utilities.php`, append this function at the end of the file (after `consoleLog`):

```php
/**
 * Backfill the running_balance column for any user rows that have NULL.
 *
 * The v2 API's `data.balance` field is the vault balance after each entry.
 * If the column is missing for a row, treating it as NULL means
 * MAX(running_balance) ignores it. This routine fetches the latest entry's
 * balance from the API and broadcasts it to all NULL rows of the user,
 * so the displayed balance becomes accurate after the first sync following
 * the schema upgrade.
 *
 * @param PDO    $pdo  Database connection.
 * @param string $user The user whose NULL rows should be backfilled.
 *
 * @return void
 */
function backfillRunningBalances($pdo, $user) {
    if (!array_key_exists($user, USER_KEYS)) {
        throw new ApiKeyMissingException("User does not have an API key configured.");
    }

    $countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
    $countStmt->bindValue(':user', $user);
    $countStmt->execute();

    if ((int)$countStmt->fetchColumn() === 0) {
        return;
    }

    $url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851&limit=1&sort=DESC';
    $responseData = executeApiCall($url, USER_KEYS[$user]);
    validateApiResponse($responseData);

    if (empty($responseData['log'])) {
        return;
    }

    $latest = $responseData['log'][0];
    if (!isset($latest['data']['balance'])) {
        return;
    }

    $balance = (int)$latest['data']['balance'];

    $updateStmt = $pdo->prepare('UPDATE vault SET running_balance = :balance WHERE user = :user AND running_balance IS NULL');
    $updateStmt->bindValue(':balance', $balance);
    $updateStmt->bindValue(':user', $user);
    $updateStmt->execute();
}
```

- [ ] **Step 2: Verify the file parses**

Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add includes/utilities.php
git commit -m "Add backfillRunningBalances helper for NULL running_balance rows"
```

---

### Task 6: Wire `backfillRunningBalances` into the entry points

**Files:**
- Modify: `functions.php` (`backfillUserLogs` and `syncUserLogs`)

**Interfaces:**
- `backfillUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`.
- `syncUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`.

- [ ] **Step 1: Update `backfillUserLogs`**

In `functions.php`, replace the body of `backfillUserLogs` (currently at lines 66-74) with:

```php
function backfillUserLogs($user) {
    $url  = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
    $pdo  = getDatabaseConnection();

    do {
        $next = fetchAndStoreLogPage($pdo, $user, $url);
        $url  = $next;
    } while ($next !== null);

    backfillRunningBalances($pdo, $user);
}
```

- [ ] **Step 2: Update `syncUserLogs`**

In `functions.php`, replace the body of `syncUserLogs` (currently at lines 87-106) with:

```php
function syncUserLogs($user) {
    if (dbNew()) {
        backfillUserLogs($user);
        return;
    }

    $pdo = getDatabaseConnection();

    $stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
    $stmt->bindValue(':user', $user);
    $stmt->execute();
    $lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];

    $url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);

    do {
        $next = fetchAndStoreLogPage($pdo, $user, $url);
        $url  = $next;
    } while ($next !== null);

    backfillRunningBalances($pdo, $user);
}
```

- [ ] **Step 3: Verify the file parses**

Run: `php -l functions.php`
Expected: `No syntax errors detected`.

- [ ] **Step 4: Commit**

```bash
git add functions.php
git commit -m "Invoke backfillRunningBalances after sync/backfill"
```

---

### Task 7: Add "Balance After" column to `buildTable`

**Files:**
- Modify: `functions.php` (`buildTable` only)

**Interfaces:**
- `buildTable()`: emits a 5th `<td>` with `number_format($entry['running_balance'], 0)` per row. The `running_balance` field may be NULL on freshly-backfilled rows; render as `—` (em-dash) so the table doesn't show "0" by default.

- [ ] **Step 1: Update `buildTable`**

In `functions.php`, replace the `buildTable` function (currently at lines 155-181) with:

```php
function buildTable () {
    $records = fetchVaultRecords();

    foreach ($records as $entry) {
        $user        = $entry['user'];
        $timestamp   = $entry['timestamp'];
        $description = $entry['description'];
        $amount      = $entry['amount'];
        $sign        = getSign($amount);

        if ($description == 'Vault withdraw') {
            $class  = 'debit';
            $amount = substr($amount, 1); // Remove the negative sign
        } else {
            $class  = 'credit';
        }

        $amount = number_format($amount, 0); // Format the amount as an integer

        $runningBalance = isset($entry['running_balance']) && $entry['running_balance'] !== null
            ? '$' . number_format((int)$entry['running_balance'], 0)
            : '—';

        echo '<tr class="'.$class.'">';
        echo '<td>'.$user.'</td>';
        echo '<td>'.date("F j, Y / H:i", $timestamp).'</td>';
        echo '<td>'.$description.'</td>';
        echo '<td>'.$sign.$amount.'</td>';
        echo '<td>'.$runningBalance.'</td>';
        echo '</tr>';
    }
}
```

- [ ] **Step 2: Verify the file parses**

Run: `php -l functions.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add functions.php
git commit -m "buildTable: add Balance After column showing per-entry running_balance"
```

---

### Task 8: Add the column header in `index.php`

**Files:**
- Modify: `index.php` (one `<th>` element)

**Interfaces:**
- Just the HTML table header.

- [ ] **Step 1: Add the `<th>Balance After</th>` element**

In `index.php`, after the existing `<th><h4>Amount</h4></th>` (currently at line 69), add:

```php
<th><h4>Balance After</h4></th>
```

The result should read:

```php
<th><h4>User</h4></th>
<th><h4>Date / Time (TCT)</h4></th>
<th><h4>Operation</h4></th>
<th><h4>Amount</h4></th>
<th><h4>Balance After</h4></th>
```

- [ ] **Step 2: Verify the file parses**

Run: `php -l index.php`
Expected: `No syntax errors detected`.

- [ ] **Step 3: Commit**

```bash
git add index.php
git commit -m "index.php: add Balance After column header"
```

---

### Task 9: End-to-end smoke check

**Files:** No code changes - just verification.

- [ ] **Step 1: Run the test suite**

Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`

- [ ] **Step 2: Verify the schema migration survived on the live DB**

Run:

```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php -r "
\$db = new PDO('sqlite:data/vault.db');
foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; }
echo 'Total rows: ' . \$db->query('SELECT COUNT(*) FROM vault')->fetchColumn() . PHP_EOL;
echo 'Rows with running_balance after backfill: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NOT NULL')->fetchColumn() . PHP_EOL;
echo 'Rows with NULL running_balance: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NULL')->fetchColumn() . PHP_EOL;
"
```

Expected: column list shows `running_balance INTEGER`. After Task 6's backfill has run on the live DB (manually once via the test instance), the NULL count should be 0.

- [ ] **Step 3: Probe the test instance**

Open the test instance in the browser (or `curl http://127.0.0.1:8001/`). The right-side "Balances" panel should show a positive total (`$615,423,760` = $69M Zarathos + $546M Symos), not a negative sum. The transactions table should show a 5th "Balance After" column with `$X` values.

- [ ] **Step 4: Verify no v1 references remain**

Run:

```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php
```

Expected: no matches.

- [ ] **Step 5: Commit if any incidental fixes were needed**

If Step 3 or 4 produced any required fixes, commit them. Otherwise, no commit.

---

## Self-Review Notes

**Spec coverage:**

| Requirement | Task |
|---|---|
| Add `running_balance` column to vault table | Task 1 |
| Idempotent schema migration for existing DBs | Task 1 (PRAGMA check + ALTER TABLE) |
| Capture `data.balance` from v2 API in `processLogEntries` | Task 2 |
| Update INSERT statement in `fetchAndStoreLogPage` | Task 2 |
| Update `fetchVaultRecords` to include new column | Task 3 |
| Replace `SUM(amount)` with `MAX(running_balance)` per user | Task 4 |
| Backfill NULL rows from latest API balance | Task 5 |
| Wire backfill into entry points | Task 6 |
| Add "Balance After" column to `buildTable` | Task 7 |
| Add header `<th>` in `index.php` | Task 8 |
| Test asserts `running_balance` round-trips | Task 2 |
| End-to-end smoke check | Task 9 |

**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions.

**Type consistency:** The `backfillRunningBalances(PDO $pdo, string $user): void` signature in Task 5 matches the call sites in Task 6 (`backfillRunningBalances($pdo, $user)` in both `backfillUserLogs` and `syncUserLogs`). The `vaultLoop` SQL in Task 4 uses `MAX(running_balance)` for the per-user case and `SUM(max_balance)` over a per-user MAX subquery for the all-users case — both return int, matching the existing `(int)` cast in `generateBalance`/`getSpace`. The `fetchAndStoreLogPage` INSERT in Task 2 includes `:running_balance` matching the new bind in `processLogEntries`.

**Risks addressed:**
- The schema migration is idempotent (PRAGMA check) and safe for existing DBs.
- The backfill is a no-op when no NULL rows exist for a user.
- The `running_balance` column is nullable, so entries missing `data.balance` from the API don't break the insert.
- All existing SQL (purported to depend on the table schema) keeps working because the new column is `INTEGER` with null default.