From 908e2c50ab3a248ea2271ae26ef5b432f35c769e Mon Sep 17 00:00:00 2001 From: Keith Solomon Date: Mon, 3 Aug 2026 21:41:33 -0500 Subject: [PATCH] Add plan: vault balance from per-entry API value - 9 tasks: schema migration, capture data.balance, fetchVaultRecords column list, vaultLoop switch to MAX, backfill helper, wire backfill, buildTable column, index.php header, end-to-end smoke check - TDD: failing test in Task 2 before rewriting processLogEntries - Idempotent schema migration via PRAGMA table_info check - One-shot live-read backfill broadcasts latest balance to NULL rows --- ...026-08-03-vault-balance-from-api-design.md | 623 ++++++++++++++++++ 1 file changed, 623 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-03-vault-balance-from-api-design.md diff --git a/docs/superpowers/plans/2026-08-03-vault-balance-from-api-design.md b/docs/superpowers/plans/2026-08-03-vault-balance-from-api-design.md new file mode 100644 index 0000000..36aab2e --- /dev/null +++ b/docs/superpowers/plans/2026-08-03-vault-balance-from-api-design.md @@ -0,0 +1,623 @@ +# Vault Balance From Per-Entry API Value — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Fix the vault balance display so it shows the live vault total (per the Torn v2 API's `data.balance` field) instead of `SUM(amount)` of stored entries. The v2 API only returns the most recent ~100 entries per user, so summing deltas cannot yield the true current balance — only the per-entry running balance can. + +**Architecture:** Add a `running_balance INTEGER` column to the `vault` table. Capture it from `$entry['data']['balance']` in `processLogEntries`. Replace balance SQL from `SUM(amount)` to `MAX(running_balance)` per user. Add a one-shot live-read backfill that broadcasts the latest entry's `data.balance` to all NULL rows of a user. Add a "Balance After" column to the rendered table. + +**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies. + +## Global Constraints + +- PHP 8.1+ syntax only. +- Database schema gains one column (`running_balance INTEGER`). Existing columns and their semantics are unchanged. +- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries. +- API authentication: `Authorization: ApiKey ` header. +- Pagination: follow `_metadata.links.next` URL until null. +- Idempotency: every change must be safe to re-run (especially the schema migration and the backfill). +- Commits per task. Do not push. +- All existing helpers outside the rewritten ones stay byte-identical unless the task explicitly says otherwise. + +--- + +### Task 1: Schema — add `running_balance` column (idempotent on existing DBs) + +**Files:** +- Modify: `functions.php` (the `getDatabaseConnection` function only) + +**Interfaces:** +- Consumes: PDO from `PDO(DB_DSN, …)`. Reads `PRAGMA table_info(vault)` to detect an existing column. +- Produces: a `vault` table whose schema includes `running_balance INTEGER`. For existing DBs, the column is added via `ALTER TABLE`. + +- [ ] **Step 1: Update the CREATE TABLE statement** + +In `functions.php`, replace the `CREATE TABLE IF NOT EXISTS vault` (currently at lines 32-38) with the version that includes `running_balance`: + +```php +$createTableSQL = "CREATE TABLE IF NOT EXISTS vault ( + id TEXT PRIMARY KEY, + user TEXT NOT NULL, + timestamp INTEGER NOT NULL, + description TEXT NOT NULL, + amount REAL NOT NULL, + running_balance INTEGER +);"; +``` + +- [ ] **Step 2: Add the idempotent ALTER TABLE for existing DBs** + +Replace the `if ($pdo === null)` block's `$pdo->exec($createTableSQL);` call with the following code (keep the rest of the block intact): + +```php +$pdo->exec($createTableSQL); + +// Add the running_balance column for databases created before the +// column was added. Idempotent: skip if it already exists. +$columns = $pdo->query("PRAGMA table_info(vault)")->fetchAll(PDO::FETCH_COLUMN, 1); +if (!in_array('running_balance', $columns, true)) { + $pdo->exec('ALTER TABLE vault ADD COLUMN running_balance INTEGER'); +} +``` + +- [ ] **Step 3: Verify the file parses** + +Run: `php -l functions.php` +Expected: `No syntax errors detected in functions.php`. + +- [ ] **Step 4: Verify the migration is idempotent on the live DB** + +Run once: + +```bash +cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker" +php -r " +\$db = new PDO('sqlite:data/vault.db'); +foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; } +" +``` + +Expected: `running_balance INTEGER` appears in the column list. + +Run a second time (Page-load will do this; or re-run the script above twice in a row). Expected: no error (the `ALTER TABLE` fires only when the column is missing). + +- [ ] **Step 5: Commit** + +```bash +git add functions.php +git commit -m "Schema: add running_balance column to vault table" +``` + +--- + +### Task 2: Rewrite `fetchAndStoreLogPage` and `processLogEntries` to capture `data.balance` (with failing test) + +**Files:** +- Modify: `includes/utilities.php` (`fetchAndStoreLogPage` and `processLogEntries`) +- Modify: `tests/process_log_entries_test.php` (extend to assert `running_balance` round-trips) + +**Interfaces:** +- `processLogEntries($logEntries, $user, $insertStmt)`: reads `$entry['data']['balance']` (int) and binds it as `:running_balance`. If `data.balance` is missing, binds NULL. +- `fetchAndStoreLogPage($pdo, $user, $url)`: INSERT statement now includes `running_balance`. + +- [ ] **Step 1: Extend the failing test** + +In `tests/process_log_entries_test.php`, after the existing "row count after re-run" assertion, add these new assertions for the `running_balance` column. The fixture has 100 entries; the most recent entry's `data.balance` is `69235831` (newest, withdraw) and the oldest entry's `data.balance` is `509622821` (oldest, deposit) — both per the fixture verified during task planning. + +Append after the existing "row count after re-run" block: + +```php +// round-trip the running_balance column from $entry['data']['balance'] +$firstRunning = (int)$first['running_balance']; +$lastRunning = (int)$last['running_balance']; +assertSame(69235831, $firstRunning, 'first.running_balance'); +assertSame(509622821, $lastRunning, 'last.running_balance'); +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `php tests/process_log_entries_test.php` +Expected: failure with `first.running_balance: expected 69235831, got ` (likely 0 or NULL because the column doesn't get inserted yet). + +- [ ] **Step 3: Update `processLogEntries`** + +In `includes/utilities.php`, replace the existing `processLogEntries` function (currently at lines 122-157) with: + +```php +function processLogEntries($logEntries, $user, $insertStmt) { + foreach ($logEntries as $entry) { + $id = $entry['id'] ?? null; + + if (!$id) { + consoleLog('Skipping entry with no id: ' . print_r($entry, true)); + continue; + } + + $timestamp = $entry['timestamp'] ?? null; + $description = $entry['details']['title'] ?? null; + $hasDeposit = isset($entry['data']['deposited']); + $hasWithdraw = isset($entry['data']['withdrawn']); + + if ($hasDeposit) { + $amount = (int)$entry['data']['deposited']; + } elseif ($hasWithdraw) { + $amount = -((int)$entry['data']['withdrawn']); + } else { + consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true)); + continue; + } + + if ($timestamp === null || $description === null) { + consoleLog('Skipping entry ' . $id . ' missing timestamp or details.title: ' . print_r($entry, true)); + continue; + } + + $runningBalance = isset($entry['data']['balance']) ? (int)$entry['data']['balance'] : null; + + $insertStmt->bindValue(':id', $id); + $insertStmt->bindValue(':user', $user); + $insertStmt->bindValue(':timestamp', $timestamp); + $insertStmt->bindValue(':description', $description); + $insertStmt->bindValue(':amount', $amount); + $insertStmt->bindValue(':running_balance', $runningBalance); + $insertStmt->execute(); + } +} +``` + +- [ ] **Step 4: Update `fetchAndStoreLogPage`'s INSERT statement** + +In `includes/utilities.php`, replace the `$insertStmt` block inside `fetchAndStoreLogPage` (currently at lines 252-256) with: + +```php +$insertStmt = $pdo->prepare( + 'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) ' + . 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) ' + . 'ON CONFLICT(id) DO NOTHING' +); +``` + +- [ ] **Step 5: Run the test and verify it passes** + +Run: `php tests/process_log_entries_test.php` +Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).` + +- [ ] **Step 6: Verify the test file still parses** + +Run: `php -l tests/process_log_entries_test.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 7: Commit** + +```bash +git add includes/utilities.php tests/process_log_entries_test.php +git commit -m "Capture running_balance from v2 API data.balance" +``` + +--- + +### Task 3: Update `fetchVaultRecords` to include `running_balance` + +**Files:** +- Modify: `includes/utilities.php` (`fetchVaultRecords` only) + +**Interfaces:** +- Consumes: existing signature unchanged. +- Produces: rows that include the `running_balance` column (existing callers that ignore unknown keys continue to work). + +- [ ] **Step 1: Replace `fetchVaultRecords` with explicit column list** + +In `includes/utilities.php`, replace the body of `fetchVaultRecords` (currently at lines 168-189) with: + +```php +function fetchVaultRecords($user = null) { + $pdo = getDatabaseConnection(); + + if ($user) { + $query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault WHERE user = :user ORDER BY timestamp DESC"; + $params = [':user' => $user]; + } else { + $query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC"; + $params = []; + } + + try { + $stmt = $pdo->prepare($query); + $stmt->execute($params); + + return $stmt->fetchAll(PDO::FETCH_ASSOC); + } catch (PDOException $e) { + echo "Error fetching records: " . $e->getMessage(); + + return []; + } +} +``` + +- [ ] **Step 2: Verify the file parses** + +Run: `php -l includes/utilities.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 3: Commit** + +```bash +git add includes/utilities.php +git commit -m "fetchVaultRecords: explicit column list including running_balance" +``` + +--- + +### Task 4: Switch `vaultLoop` / `generateBalance` / `getSpace` to MAX(running_balance) + +**Files:** +- Modify: `functions.php` (the three functions) + +**Interfaces:** +- `vaultLoop($name=null)`: returns the per-user vault balance (the most recent entry's `running_balance`) for the given user, or the sum of those across all users when `$name` is null. Goes directly to PDO instead of fetching records. +- `generateBalance($name=null)`: continues to wrap `vaultLoop`, returns the formatted value. +- `getSpace($name=null)`: continues to compute `limit - balance` against the same value. + +- [ ] **Step 1: Replace `vaultLoop` with a SQL-based implementation** + +In `functions.php`, replace `vaultLoop` (currently at lines 199-208) with: + +```php +function vaultLoop ($name=null) { + $pdo = getDatabaseConnection(); + + if ($name === null) { + $stmt = $pdo->query( + 'SELECT COALESCE(SUM(max_balance), 0) FROM ' + . '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)' + ); + + return (int)$stmt->fetchColumn(); + } + + $stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user'); + $stmt->bindValue(':user', $name); + $stmt->execute(); + + return (int)$stmt->fetchColumn(); +} +``` + +- [ ] **Step 2: Verify the file parses** + +Run: `php -l functions.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 3: Commit** + +```bash +git add functions.php +git commit -m "vaultLoop: use MAX(running_balance) per user instead of SUM(amount)" +``` + +--- + +### Task 5: Add `backfillRunningBalances` helper + +**Files:** +- Modify: `includes/utilities.php` (append new function after `consoleLog`) + +**Interfaces:** +- `backfillRunningBalances(PDO $pdo, string $user): void` +- Counts rows where `running_balance IS NULL` for the user. If zero, returns. Otherwise hits the API with `selections=log&log=5850,5851&limit=1&sort=DESC` and uses the first entry's `data.balance` to UPDATE all NULL rows for that user. + +- [ ] **Step 1: Append the new function** + +In `includes/utilities.php`, append this function at the end of the file (after `consoleLog`): + +```php +/** + * Backfill the running_balance column for any user rows that have NULL. + * + * The v2 API's `data.balance` field is the vault balance after each entry. + * If the column is missing for a row, treating it as NULL means + * MAX(running_balance) ignores it. This routine fetches the latest entry's + * balance from the API and broadcasts it to all NULL rows of the user, + * so the displayed balance becomes accurate after the first sync following + * the schema upgrade. + * + * @param PDO $pdo Database connection. + * @param string $user The user whose NULL rows should be backfilled. + * + * @return void + */ +function backfillRunningBalances($pdo, $user) { + if (!array_key_exists($user, USER_KEYS)) { + throw new ApiKeyMissingException("User does not have an API key configured."); + } + + $countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL'); + $countStmt->bindValue(':user', $user); + $countStmt->execute(); + + if ((int)$countStmt->fetchColumn() === 0) { + return; + } + + $url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851&limit=1&sort=DESC'; + $responseData = executeApiCall($url, USER_KEYS[$user]); + validateApiResponse($responseData); + + if (empty($responseData['log'])) { + return; + } + + $latest = $responseData['log'][0]; + if (!isset($latest['data']['balance'])) { + return; + } + + $balance = (int)$latest['data']['balance']; + + $updateStmt = $pdo->prepare('UPDATE vault SET running_balance = :balance WHERE user = :user AND running_balance IS NULL'); + $updateStmt->bindValue(':balance', $balance); + $updateStmt->bindValue(':user', $user); + $updateStmt->execute(); +} +``` + +- [ ] **Step 2: Verify the file parses** + +Run: `php -l includes/utilities.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 3: Commit** + +```bash +git add includes/utilities.php +git commit -m "Add backfillRunningBalances helper for NULL running_balance rows" +``` + +--- + +### Task 6: Wire `backfillRunningBalances` into the entry points + +**Files:** +- Modify: `functions.php` (`backfillUserLogs` and `syncUserLogs`) + +**Interfaces:** +- `backfillUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`. +- `syncUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`. + +- [ ] **Step 1: Update `backfillUserLogs`** + +In `functions.php`, replace the body of `backfillUserLogs` (currently at lines 66-74) with: + +```php +function backfillUserLogs($user) { + $url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851'; + $pdo = getDatabaseConnection(); + + do { + $next = fetchAndStoreLogPage($pdo, $user, $url); + $url = $next; + } while ($next !== null); + + backfillRunningBalances($pdo, $user); +} +``` + +- [ ] **Step 2: Update `syncUserLogs`** + +In `functions.php`, replace the body of `syncUserLogs` (currently at lines 87-106) with: + +```php +function syncUserLogs($user) { + if (dbNew()) { + backfillUserLogs($user); + return; + } + + $pdo = getDatabaseConnection(); + + $stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user'); + $stmt->bindValue(':user', $user); + $stmt->execute(); + $lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts']; + + $url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1); + + do { + $next = fetchAndStoreLogPage($pdo, $user, $url); + $url = $next; + } while ($next !== null); + + backfillRunningBalances($pdo, $user); +} +``` + +- [ ] **Step 3: Verify the file parses** + +Run: `php -l functions.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 4: Commit** + +```bash +git add functions.php +git commit -m "Invoke backfillRunningBalances after sync/backfill" +``` + +--- + +### Task 7: Add "Balance After" column to `buildTable` + +**Files:** +- Modify: `functions.php` (`buildTable` only) + +**Interfaces:** +- `buildTable()`: emits a 5th `` with `number_format($entry['running_balance'], 0)` per row. The `running_balance` field may be NULL on freshly-backfilled rows; render as `—` (em-dash) so the table doesn't show "0" by default. + +- [ ] **Step 1: Update `buildTable`** + +In `functions.php`, replace the `buildTable` function (currently at lines 155-181) with: + +```php +function buildTable () { + $records = fetchVaultRecords(); + + foreach ($records as $entry) { + $user = $entry['user']; + $timestamp = $entry['timestamp']; + $description = $entry['description']; + $amount = $entry['amount']; + $sign = getSign($amount); + + if ($description == 'Vault withdraw') { + $class = 'debit'; + $amount = substr($amount, 1); // Remove the negative sign + } else { + $class = 'credit'; + } + + $amount = number_format($amount, 0); // Format the amount as an integer + + $runningBalance = isset($entry['running_balance']) && $entry['running_balance'] !== null + ? '$' . number_format((int)$entry['running_balance'], 0) + : '—'; + + echo ''; + echo ''.$user.''; + echo ''.date("F j, Y / H:i", $timestamp).''; + echo ''.$description.''; + echo ''.$sign.$amount.''; + echo ''.$runningBalance.''; + echo ''; + } +} +``` + +- [ ] **Step 2: Verify the file parses** + +Run: `php -l functions.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 3: Commit** + +```bash +git add functions.php +git commit -m "buildTable: add Balance After column showing per-entry running_balance" +``` + +--- + +### Task 8: Add the column header in `index.php` + +**Files:** +- Modify: `index.php` (one `` element) + +**Interfaces:** +- Just the HTML table header. + +- [ ] **Step 1: Add the `Balance After` element** + +In `index.php`, after the existing `

Amount

` (currently at line 69), add: + +```php +

Balance After

+``` + +The result should read: + +```php +

User

+

Date / Time (TCT)

+

Operation

+

Amount

+

Balance After

+``` + +- [ ] **Step 2: Verify the file parses** + +Run: `php -l index.php` +Expected: `No syntax errors detected`. + +- [ ] **Step 3: Commit** + +```bash +git add index.php +git commit -m "index.php: add Balance After column header" +``` + +--- + +### Task 9: End-to-end smoke check + +**Files:** No code changes - just verification. + +- [ ] **Step 1: Run the test suite** + +Run: `php tests/process_log_entries_test.php` +Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).` + +- [ ] **Step 2: Verify the schema migration survived on the live DB** + +Run: + +```bash +cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker" +php -r " +\$db = new PDO('sqlite:data/vault.db'); +foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; } +echo 'Total rows: ' . \$db->query('SELECT COUNT(*) FROM vault')->fetchColumn() . PHP_EOL; +echo 'Rows with running_balance after backfill: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NOT NULL')->fetchColumn() . PHP_EOL; +echo 'Rows with NULL running_balance: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NULL')->fetchColumn() . PHP_EOL; +" +``` + +Expected: column list shows `running_balance INTEGER`. After Task 6's backfill has run on the live DB (manually once via the test instance), the NULL count should be 0. + +- [ ] **Step 3: Probe the test instance** + +Open the test instance in the browser (or `curl http://127.0.0.1:8001/`). The right-side "Balances" panel should show a positive total (`$615,423,760` = $69M Zarathos + $546M Symos), not a negative sum. The transactions table should show a 5th "Balance After" column with `$X` values. + +- [ ] **Step 4: Verify no v1 references remain** + +Run: + +```bash +cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker" +grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php +``` + +Expected: no matches. + +- [ ] **Step 5: Commit if any incidental fixes were needed** + +If Step 3 or 4 produced any required fixes, commit them. Otherwise, no commit. + +--- + +## Self-Review Notes + +**Spec coverage:** + +| Requirement | Task | +|---|---| +| Add `running_balance` column to vault table | Task 1 | +| Idempotent schema migration for existing DBs | Task 1 (PRAGMA check + ALTER TABLE) | +| Capture `data.balance` from v2 API in `processLogEntries` | Task 2 | +| Update INSERT statement in `fetchAndStoreLogPage` | Task 2 | +| Update `fetchVaultRecords` to include new column | Task 3 | +| Replace `SUM(amount)` with `MAX(running_balance)` per user | Task 4 | +| Backfill NULL rows from latest API balance | Task 5 | +| Wire backfill into entry points | Task 6 | +| Add "Balance After" column to `buildTable` | Task 7 | +| Add header `` in `index.php` | Task 8 | +| Test asserts `running_balance` round-trips | Task 2 | +| End-to-end smoke check | Task 9 | + +**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions. + +**Type consistency:** The `backfillRunningBalances(PDO $pdo, string $user): void` signature in Task 5 matches the call sites in Task 6 (`backfillRunningBalances($pdo, $user)` in both `backfillUserLogs` and `syncUserLogs`). The `vaultLoop` SQL in Task 4 uses `MAX(running_balance)` for the per-user case and `SUM(max_balance)` over a per-user MAX subquery for the all-users case — both return int, matching the existing `(int)` cast in `generateBalance`/`getSpace`. The `fetchAndStoreLogPage` INSERT in Task 2 includes `:running_balance` matching the new bind in `processLogEntries`. + +**Risks addressed:** +- The schema migration is idempotent (PRAGMA check) and safe for existing DBs. +- The backfill is a no-op when no NULL rows exist for a user. +- The `running_balance` column is nullable, so entries missing `data.balance` from the API don't break the insert. +- All existing SQL (purported to depend on the table schema) keeps working because the new column is `INTEGER` with null default.