Compare commits

..
42 Commits
Author SHA1 Message Date
Keith Solomon 4339958b5e Normalize line endings in vault balance plan doc
Deploy website via FTP / Deploy (push) Skipped
The plan was committed with LF endings; the editor normalized to CRLF on
save. Restore LF for consistency with the rest of the repo's plan docs.
2026-08-10 20:40:55 -05:00
Keith Solomon bd120608d7 syncUserLogs: delegate to backfillUserLogs when user has no rows
Deploy website via FTP / Deploy (push) Skipped
2026-08-10 20:07:59 -05:00
Keith Solomon 6b7f37963c index.php: meta-refresh loading page while backfill is in progress 2026-08-10 20:02:59 -05:00
Keith Solomon a9a635e172 Add plan: async backfill with meta-refresh loading page
- 2 tasks: index.php loading-page bootstrap, smoke check
- Heavy sync deferred to meta-refresh loops so PHP never times out
- Each refresh walks one API page; ON CONFLICT DO UPDATE makes
- resume safe after browser close
2026-08-10 20:01:14 -05:00
Keith Solomon a6efb7d63d fetchAndStoreLogPage: walk prev chain to fetch full vault history 2026-08-07 19:10:38 -05:00
Keith Solomon a19058c6e9 Add plan: walk prev chain on backfill for full vault history
- 2 tasks: update fetchAndStoreLogPage pagination loop, smoke check
- Loop prefers prev (older pages) over next (newer pages) so a single
  fetchAndStoreLogPage call captures the full history
- INSERT uses ON CONFLICT DO UPDATE for running_balance and amount
  so re-runs refresh data
2026-08-07 19:08:01 -05:00
Keith Solomon ee4e24aef0 vaultLoop: revert to SUM(amount) per user (v1 semantic) 2026-08-07 18:41:15 -05:00
Keith Solomon a7f35e79e2 Remove unused fetchLiveVaultBalance and its test 2026-08-07 17:50:45 -05:00
Keith Solomon dd820f6c8a Invoke refetchRunningBalances after sync/backfill 2026-08-07 17:45:15 -05:00
Keith Solomon 63c6cc6972 Replace broadcast backfill with per-entry re-fetch for running_balance 2026-08-07 17:39:22 -05:00
Keith Solomon 5b8abd8bbc Revert vaultLoop to per-user SQL MAX(running_balance) 2026-08-07 17:34:37 -05:00
Keith Solomon 3c798cee88 Add plan: revert to per-user balance via /log + per-entry re-fetch
- 5 tasks: revert vaultLoop to SQL, replace broadcast backfill with
  per-entry re-fetch, wire into entry points, delete unused helper
  and its test, smoke check
- refetchRunningBalances walks /log pagination and UPDATEs each row
  from its own data.balance
- Drops the fetchLiveVaultBalance helper (queried /money which is
  total-vault not per-user)
- The previous fix's broadcast backfill was clobbering per-entry
  historical balances with a single value
2026-08-07 17:30:33 -05:00
Keith Solomon d1cb4196dd vaultLoop: sum per-user live /money balances for the all-users path 2026-08-07 14:19:34 -05:00
Keith Solomon 006520b4c0 vaultLoop: prefer live /money balance, fall back to MAX(running_balance) 2026-08-05 10:04:43 -05:00
Keith Solomon 728f7bb4a5 fetchLiveVaultBalance: split production/test cache paths so production caching works 2026-08-05 09:55:01 -05:00
Keith Solomon abb353a939 Add fetchLiveVaultBalance helper using v2 /money endpoint 2026-08-05 09:47:39 -05:00
Keith Solomon 61fb15336e Add failing test for fetchLiveVaultBalance 2026-08-05 09:29:16 -05:00
Keith Solomon 0613d00fbb Add plan: live vault balance from /money endpoint
- 4 tasks: failing test, helper implementation, vaultLoop switch, smoke check
- New helper fetchLiveVaultBalance(string $user): ?int in includes/utilities.php
- vaultLoop prefers the live value, falls back to MAX(running_balance)
- Per-request cache to avoid N HTTP calls
- Test hook via $GLOBALS['liveBalanceTestHook'] for unit testing
- Falls back gracefully on API failure
2026-08-05 09:23:15 -05:00
Keith Solomon 64e320c6a5 index.php: add Balance After column header 2026-08-03 23:02:53 -05:00
Keith Solomon 5fe4bf9f58 buildTable: add Balance After column showing per-entry running_balance 2026-08-03 22:56:21 -05:00
Keith Solomon f1cd0d53b1 Invoke backfillRunningBalances after sync/backfill 2026-08-03 22:47:40 -05:00
Keith Solomon a81554a125 Add backfillRunningBalances helper for NULL running_balance rows 2026-08-03 22:41:22 -05:00
Keith Solomon a9e443eeb2 vaultLoop: use MAX(running_balance) per user instead of SUM(amount) 2026-08-03 22:17:11 -05:00
Keith Solomon 214891d957 fetchVaultRecords: explicit column list including running_balance 2026-08-03 22:00:56 -05:00
Keith Solomon af7a233272 Capture running_balance from v2 API data.balance 2026-08-03 21:54:40 -05:00
Keith Solomon 2e1bd050e1 Schema: add running_balance column to vault table 2026-08-03 21:47:46 -05:00
Keith Solomon 908e2c50ab Add plan: vault balance from per-entry API value
- 9 tasks: schema migration, capture data.balance, fetchVaultRecords
  column list, vaultLoop switch to MAX, backfill helper, wire backfill,
  buildTable column, index.php header, end-to-end smoke check
- TDD: failing test in Task 2 before rewriting processLogEntries
- Idempotent schema migration via PRAGMA table_info check
- One-shot live-read backfill broadcasts latest balance to NULL rows
2026-08-03 21:41:33 -05:00
Keith Solomon a133a5a49d index.php: catch bootstrap exceptions and render a non-fatal warning
Per the spec's error-handling section, all exceptions must be caught at
the top of index.php's bootstrap path so a single bad page (CurlError,
JsonData, ApiValidation, ApiKeyMissing, LogEntryIncomplete) does not take
down the whole UI. The page now renders whatever data is in the DB plus
a warning banner with the exception message (htmlspecialchars-escaped to
avoid XSS). On the first-run backfill path, the redirect is still
emitted inside the try block so the success path is unchanged.
2026-08-03 14:18:48 -05:00
Keith Solomon 73360f6805 processLogEntries: skip incomplete entries instead of throwing
Per the spec's error-handling table, entries missing timestamp or
details.title should be skipped (logged via consoleLog) and the loop
should continue processing the rest. Previously the function threw
LogEntryIncompleteException and aborted the entire batch, which would
take down the whole UI when a single malformed entry was returned.
2026-08-03 14:17:27 -05:00
Keith Solomon 89ecaaa6c7 Remove dead prepareCheckStatement/prepareInsertStatement helpers
These helpers were no longer called after the v2 migration rewrote
processLogEntries and fetchAndStoreLogPage (the latter now prepares its
own idempotent INSERT inline). Rendered unused by the v2 migration.
2026-08-03 14:16:16 -05:00
Keith Solomon 9fa5247445 Style fixes from phpcs 2026-08-03 13:54:30 -05:00
Keith Solomon 265e1841f0 Update index.php to call backfillUserLogs/syncUserLogs 2026-08-03 13:45:37 -05:00
Keith Solomon d6aef3da42 Replace firstRun/getLog with backfillUserLogs/syncUserLogs 2026-08-03 13:40:12 -05:00
Keith Solomon cb584e9d3a Restore consoleLog helper and remove test stub masking production fatal
The v2 rewrite of processLogEntries() calls consoleLog(...) at two
skip branches (no id, and no deposited/withdrawn). Commit 3ebc5e2 had
removed the project helper from includes/utilities.php, leaving a
production runtime fatal ("Call to undefined function consoleLog()")
the first time a non-vault entry flowed through fetchAndStoreLogPage
-> processLogEntries in real use.

The previous test appeared to pass only because it defined a local
no-op consoleLog stub via function_exists() guard — masking the defect
behind a test fixture that never reached either skip branch.

Fix:
  - Restore the original consoleLog() helper to
    includes/utilities.php (unchanged from the pre-3ebc5e2 codebase;
    spec lists it as unchanged).
  - Remove the local consoleLog stub from
    tests/process_log_entries_test.php so the test now exercises
    the real production code path.
2026-08-03 13:34:29 -05:00
Keith Solomon 2a55cc11ee Rewrite processLogEntries for v2 shape; add golden-file test 2026-08-03 13:27:43 -05:00
Keith Solomon 9a4a7291cc Add fetchAndStoreLogPage helper for v2 pagination 2026-08-03 13:19:38 -05:00
Keith Solomon 07a8bba1d5 Tighten validateApiResponse for v2 array shape 2026-08-03 13:13:50 -05:00
Keith Solomon 3ebc5e2850 Remove unauthorized consoleLog helper and restore original quote style 2026-08-03 13:06:30 -05:00
Keith Solomon e25a069e00 Rewrite executeApiCall for v2 API (Authorization header) 2026-08-03 13:01:52 -05:00
Keith Solomon 93b934c5cc Add LogEntryIncompleteException for malformed v2 entries 2026-08-03 12:56:32 -05:00
Keith Solomon e7ec343be1 Add implementation plan: Torn API v1 to v2 migration
- 8 tasks: exception class, API call rewrite, validation tightening,
  fetchAndStoreLogPage helper, processLogEntries rewrite with golden-file
  test, backfillUserLogs/syncUserLogs entry points, index.php call swap,
  end-to-end smoke check
- TDD: write the failing test before rewriting processLogEntries
- Each task is a self-contained commit
2026-08-03 12:47:59 -05:00
Keith Solomon c2b138d687 Add spec: Torn API v1 to v2 migration design
- Compares old (api-sample.json) and new (api-sample-new.json) shapes
- Specifies Authorization: ApiKey header instead of ?key= query param
- Specifies following _metadata.links.next for pagination
- Specifies entry.id (string) as PRIMARY KEY with INSERT ... ON CONFLICT DO NOTHING
- Adds backfillUserLogs (replaces firstRun) and syncUserLogs (renamed from getLog)
- Adds shared fetchAndStoreLogPage helper
- Adds golden-file unit test (no live API)
2026-08-03 12:42:19 -05:00
12 changed files with 2857 additions and 160 deletions
@@ -0,0 +1,604 @@
# Torn API v1 → v2 Migration Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Rewrite Torn Vault Tracker's log-ingestion code to consume the new v2 Torn API JSON shape (array-of-entries with `_metadata.links.next` pagination and `Authorization: ApiKey` header), with a single shared page-fetch helper and a golden-file unit test.
**Architecture:** All v2-aware behavior lives in `includes/utilities.php` (`executeApiCall`, `validateApiResponse`, `processLogEntries`, plus a new `fetchAndStoreLogPage` helper). `functions.php` exposes two thin entry points — `backfillUserLogs` (full-history pagination on empty DB) and `syncUserLogs` (incremental fetch from last seen timestamp). `index.php` swaps its call sites. The only new behavior is in the helper module; everything else is either renamed or a one-line swap.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies. No framework. Pure CLI test runner (`php tests/process_log_entries_test.php`) using the existing `backup/api-sample-new.json` fixture.
## Global Constraints
- PHP 8.1+ syntax only.
- Database schema (`vault` table with `id TEXT PRIMARY KEY`) is **not** changed.
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries — operation must be idempotent across re-runs.
- API authentication: `Authorization: ApiKey <key>` header; **never** pass the key as a `?key=` query parameter.
- Pagination: follow `_metadata.links.next` URL until null. Never compute timestamps manually for pagination; v2 provides the cursor.
- Use the fixture at `backup/api-sample-new.json` (100 entries) for the golden-file test.
- All existing helpers outside the rewritten ones stay byte-identical (no drive-by refactors).
- Commit after each task. Do not push.
---
### Task 1: Add `LogEntryIncompleteException`
**Files:**
- Modify: `includes/exceptions.php` (append new class after the existing four)
- No test file — exercised via Task 5's golden test.
**Interfaces:**
- Consumes: none.
- Produces: a new exception class `LogEntryIncompleteException extends Exception`, with the same constructor signature as the existing four exception classes in the file.
- [ ] **Step 1: Append the new class to `includes/exceptions.php`**
Append after the existing `ApiValidationException` class, following the same doc-comment + constructor pattern used by the other four classes in the file:
```php
/**
* Exception thrown when a log entry is missing required data fields.
*
* @category Exception
* @package TornVaultTracker
* @author Keith Solomon <ksolomon@gmail.com>
* @license Unlicense https://unlicense.org/
* @link https://github.com/ksolomon/Torn-Vault-Tracker
*/
class LogEntryIncompleteException extends Exception {
/**
* Constructor.
*
* @param string $message The exception message.
* @param int $code The exception code.
* @param Throwable $previous The previous throwable.
*/
public function __construct($message, $code = 0, Throwable $previous = null) {
parent::__construct($message, $code, $previous);
}
}
```
- [ ] **Step 2: Verify file still parses**
Run: `php -l includes/exceptions.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/exceptions.php
git commit -m "Add LogEntryIncompleteException for malformed v2 entries"
```
---
### Task 2: Rewrite `executeApiCall` for v2 authentication
**Files:**
- Modify: `includes/utilities.php` (replace the existing `executeApiCall` function body)
- No new test file for this task alone — exercised by Task 5's golden test and by the integration in Task 4.
**Interfaces:**
- Consumes: `$apiEndpoint` (full URL), `$apiKey` (string). Uses `\Exception` (existing) for `CurlErrorException` and `JsonDataException` from `includes/exceptions.php`.
- Produces: `array` — the decoded JSON response, or throws `CurlErrorException` / `JsonDataException`. Header is `Authorization: ApiKey <apiKey>` only (no `Content-Type`).
- [ ] **Step 1: Replace `executeApiCall` in `includes/utilities.php`**
Replace the existing function (currently takes only `$apiEndpoint`) with this v2-aware version:
```php
function executeApiCall($apiEndpoint, $apiKey) {
if (empty($apiKey)) {
throw new ApiKeyMissingException('API key is required for executeApiCall.');
}
$headers = ["Authorization: ApiKey $apiKey"];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$response = curl_exec($ch);
if (curl_errno($ch)) {
throw new CurlErrorException('cURL error: ' . curl_error($ch));
}
curl_close($ch);
$responseData = json_decode($response, true);
if (json_last_error() !== JSON_ERROR_NONE) {
throw new JsonDataException('Failed to decode JSON response: ' . json_last_error_msg());
}
return $responseData;
}
```
- [ ] **Step 2: Verify file still parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/utilities.php
git commit -m "Rewrite executeApiCall for v2 API (Authorization header)"
```
---
### Task 3: Tighten `validateApiResponse` for v2 shape
**Files:**
- Modify: `includes/utilities.php` (replace the existing `validateApiResponse` function body)
- No test file for this task alone — exercised by Task 5's golden test.
**Interfaces:**
- Consumes: `$responseData` (array). Throws `ApiValidationException` (already defined).
- Produces: void. `log` must exist and be an array.
- [ ] **Step 1: Replace `validateApiResponse` in `includes/utilities.php`**
Replace the existing function with:
```php
function validateApiResponse($responseData) {
if (!isset($responseData['log']) || !is_array($responseData['log'])) {
throw new ApiValidationException('Invalid log data received from the API.');
}
}
```
(The change from the old version is tightening `is_array($responseData['log'])` — v2 always returns an array; v1 returned an object-keyed map, which `is_array()` rejects.)
- [ ] **Step 2: Verify file still parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/utilities.php
git commit -m "Tighten validateApiResponse for v2 array shape"
```
---
### Task 4: Add `fetchAndStoreLogPage` helper
**Files:**
- Modify: `includes/utilities.php` (append new function after `processLogEntries`)
- No test file for this task alone — exercised by Task 5 and the integration tests in Task 7.
**Interfaces:**
- Consumes: `$pdo` (PDO), `$user` (string — key into `USER_KEYS`), `$url` (string — full v2 API URL, possibly a `_metadata.links.next` URL). Throws `ApiKeyMissingException`, `CurlErrorException`, `JsonDataException`, `ApiValidationException` (all already defined).
- Produces: `?string` — the next page URL from `_metadata.links.next`, or `null` if no more pages.
- [ ] **Step 1: Add the new function to `includes/utilities.php`**
Append the following at the end of the file (after `consoleLog()`):
```php
/**
* Fetch a single page of log entries from the Torn v2 API and store them.
*
* Performs one HTTP request, validates the response, inserts each vault
* log entry (idempotently via INSERT ... ON CONFLICT), and returns the
* pagination cursor (`_metadata.links.next`) if more pages remain.
*
* @param PDO $pdo Database connection used to insert vault entries.
* @param string $user The user whose log entries are being fetched.
* @param string $url Full URL for the v2 API request.
*
* @throws ApiKeyMissingException If no API key is configured for the user.
* @throws CurlErrorException If the HTTP request fails.
* @throws JsonDataException If the response body is not valid JSON.
* @throws ApiValidationException If the response is missing the `log` array.
*
* @return string|null The `_metadata.links.next` URL, or null when there are no more pages.
*/
function fetchAndStoreLogPage($pdo, $user, $url) {
if (!array_key_exists($user, USER_KEYS)) {
throw new ApiKeyMissingException("User does not have an API key configured.");
}
$responseData = executeApiCall($url, USER_KEYS[$user]);
validateApiResponse($responseData);
$insertStmt = $pdo->prepare(
'INSERT INTO vault (id, user, timestamp, description, amount) '
. 'VALUES (:id, :user, :timestamp, :description, :amount) '
. 'ON CONFLICT(id) DO NOTHING'
);
processLogEntries($responseData['log'], $user, $insertStmt);
return $responseData['_metadata']['links']['next'] ?? null;
}
```
- [ ] **Step 2: Verify file still parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/utilities.php
git commit -m "Add fetchAndStoreLogPage helper for v2 pagination"
```
---
### Task 5: Rewrite `processLogEntries` for v2 shape (with golden-file test)
**Files:**
- Create: `tests/process_log_entries_test.php`
- Modify: `includes/utilities.php` (replace the existing `processLogEntries` function body)
**Interfaces:**
- Consumes: `$logEntries` (array of v2 entry objects), `$user` (string), `$insertStmt` (PDOStatement prepared with the `INSERT … ON CONFLICT(id) DO NOTHING` query). Throws `LogEntryIncompleteException` (new in Task 1).
- Produces: void. For each entry: extracts `id`, `timestamp`, `details.title`, `data.deposited` (deposit) or `data.withdrawn` (withdraw) and binds to `$insertStmt`, then `execute()`. Skips entries missing both `data.deposited` and `data.withdrawn` after logging via `consoleLog`.
- [ ] **Step 1: Write the failing test**
Create `tests/process_log_entries_test.php`:
```php
<?php
/**
* Golden-file test for processLogEntries() v2 behavior.
*
* Loads backup/api-sample-new.json, runs each entry through the v2
* processor against an in-memory SQLite database, and asserts the
* resulting rows match the expected (id, user, timestamp, description,
* amount) tuples derived from the fixture.
*
* Run: php tests/process_log_entries_test.php
* Exit code 0 = pass.
*/
declare(strict_types=1);
// Make USER_KEYS available without including config.php (it would force a
// real DB connection).
if (!defined('USER_KEYS')) {
define('USER_KEYS', [
'zarathos' => 'test-key-not-used',
'symos' => 'test-key-not-used',
]);
}
require_once __DIR__ . '/../includes/exceptions.php';
require_once __DIR__ . '/../includes/utilities.php';
function fail(string $message): void {
fwrite(STDERR, "FAIL: $message\n");
exit(1);
}
function assertSame($expected, $actual, string $label): void {
if ($expected !== $actual) {
$exp = var_export($expected, true);
$act = var_export($actual, true);
fail("$label: expected $exp, got $act");
}
}
// In-memory DB so we don't touch the user's vault.db.
$pdo = new PDO('sqlite::memory:');
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->exec(
'CREATE TABLE vault (
id TEXT PRIMARY KEY,
user TEXT NOT NULL,
timestamp INTEGER NOT NULL,
description TEXT NOT NULL,
amount REAL NOT NULL
)'
);
$insertStmt = $pdo->prepare(
'INSERT INTO vault (id, user, timestamp, description, amount) '
. 'VALUES (:id, :user, :timestamp, :description, :amount) '
. 'ON CONFLICT(id) DO NOTHING'
);
// Load the golden file.
$fixturePath = __DIR__ . '/../backup/api-sample-new.json';
if (!is_readable($fixturePath)) {
fail("Fixture not readable at $fixturePath");
}
$fixture = json_decode(file_get_contents($fixturePath), true);
if (!is_array($fixture) || !isset($fixture['log'])) {
fail('Fixture is missing the log array.');
}
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
// 100 entries in the fixture.
$count = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count, 'row count');
// Spot-check first entry (newest, withdraw) and last (oldest, deposit).
$rows = $pdo->query('SELECT id, user, timestamp, description, amount FROM vault ORDER BY timestamp DESC')->fetchAll(PDO::FETCH_ASSOC);
$first = $rows[0];
assertSame('c1pEDQ7jl3kuBHV4NqDI', $first['id'], 'first.id');
assertSame('zarathos', $first['user'], 'first.user');
assertSame(1785325489, (int)$first['timestamp'], 'first.timestamp');
assertSame('Vault withdraw', $first['description'], 'first.description');
assertSame(-498870964, (int)$first['amount'], 'first.amount');
$last = $rows[count($rows) - 1];
assertSame('8m5jrUKTiKk0SB4MxzIy', $last['id'], 'last.id');
assertSame(1714350415, (int)$last['timestamp'], 'last.timestamp');
assertSame('Vault deposit', $last['description'], 'last.description');
assertSame(231492, (int)$last['amount'], 'last.amount');
// Re-running with the same fixture inserts zero new rows (idempotency).
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
$count2 = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count2, 'row count after re-run');
echo "OK: processLogEntries v2 golden-file test passed (100 entries).\n";
```
- [ ] **Step 2: Run the test and verify it fails**
Run: `php tests/process_log_entries_test.php`
Expected: PHP fatal error: `processLogEntries()` exists but reads `$entry['log']` (v1) and crashes when `log` is not a key — or returns zero rows. Either way the test will not print `OK: …`.
- [ ] **Step 3: Rewrite `processLogEntries` in `includes/utilities.php`**
Replace the existing `processLogEntries` function with:
```php
function processLogEntries($logEntries, $user, $insertStmt) {
foreach ($logEntries as $entry) {
$id = $entry['id'] ?? null;
if (!$id) {
consoleLog('Skipping entry with no id: ' . print_r($entry, true));
continue;
}
$timestamp = $entry['timestamp'] ?? null;
$description = $entry['details']['title'] ?? null;
$hasDeposit = isset($entry['data']['deposited']);
$hasWithdraw = isset($entry['data']['withdrawn']);
if ($hasDeposit) {
$amount = (int)$entry['data']['deposited'];
} elseif ($hasWithdraw) {
$amount = -((int)$entry['data']['withdrawn']);
} else {
consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
continue;
}
if ($timestamp === null || $description === null) {
throw new LogEntryIncompleteException(
"Entry $id missing timestamp or details.title."
);
}
$insertStmt->bindValue(':id', $id);
$insertStmt->bindValue(':user', $user);
$insertStmt->bindValue(':timestamp', $timestamp);
$insertStmt->bindValue(':description', $description);
$insertStmt->bindValue(':amount', $amount);
$insertStmt->execute();
}
}
```
Note: signature changed from `(array, string, PDOStatement, PDOStatement, bool)` to `(array, string, PDOStatement)`. The two statements (check/insert) collapsed into one ON CONFLICT insert, the `debug` flag is gone (debug logging was unused), and the `$debug` boolean parameter is dropped.
- [ ] **Step 4: Run the test and verify it passes**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 5: Commit**
```bash
git add tests/process_log_entries_test.php includes/utilities.php
git commit -m "Rewrite processLogEntries for v2 shape; add golden-file test"
```
---
### Task 6: Replace `firstRun`/`getLog` with `backfillUserLogs`/`syncUserLogs`
**Files:**
- Modify: `functions.php` (remove `firstRun`, remove `getLog`, add `backfillUserLogs` and `syncUserLogs`)
- No new test file — verified end-to-end in Task 8.
**Interfaces:**
- Consumes: `$user` (string — key into `USER_KEYS`). Uses `getDatabaseConnection()`, `dbNew()`, `fetchAndStoreLogPage()` (Task 4).
- Produces: `void`. Both functions are idempotent.
- [ ] **Step 1: Remove `firstRun` from `functions.php`**
Delete the entire `firstRun()` function (lines 72-119 in the current file, including the doc-comment block above it).
- [ ] **Step 2: Remove `getLog` from `functions.php`**
Delete the entire `getLog()` function (lines 132-153 in the current file, including the doc-comment block above it).
- [ ] **Step 3: Add `backfillUserLogs` and `syncUserLogs`**
Insert the following in place of the deleted functions:
```php
/**
* Pulls and stores the full vault transaction log history for a user.
*
* Pages through the v2 Torn API using `_metadata.links.next` until the
* API reports no further pages. Intended for first-run use when the
* local database is empty. Idempotent: re-running on a partially
* populated database inserts only new entries (ON CONFLICT DO NOTHING).
*
* @param string $user The user whose logs should be fetched.
*
* @return void
*/
function backfillUserLogs($user) {
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
$pdo = getDatabaseConnection();
do {
$next = fetchAndStoreLogPage($pdo, $user, $url);
$url = $next;
} while ($next !== null);
}
/**
* Synchronizes recent vault transactions for a user.
*
* On a non-empty database, fetches only entries newer than the user's
* most recent row. Falls back to full backfill when the database is
* empty.
*
* @param string $user The user whose logs should be synced.
*
* @return void
*/
function syncUserLogs($user) {
if (dbNew()) {
backfillUserLogs($user);
return;
}
$pdo = getDatabaseConnection();
$stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
$stmt->bindValue(':user', $user);
$stmt->execute();
$lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);
do {
$next = fetchAndStoreLogPage($pdo, $user, $url);
$url = $next;
} while ($next !== null);
}
```
- [ ] **Step 4: Verify the file still parses**
Run: `php -l functions.php`
Expected: `No syntax errors detected`.
- [ ] **Step 5: Commit**
```bash
git add functions.php
git commit -m "Replace firstRun/getLog with backfillUserLogs/syncUserLogs"
```
---
### Task 7: Update `index.php` call sites
**Files:**
- Modify: `index.php` (rename two function calls in the `dbNew()` branches)
**Interfaces:**
- Consumes: existing `index.php` control flow (calls into `firstRun` / `getLog`).
- Produces: same control flow, but calling `backfillUserLogs` / `syncUserLogs` instead.
- [ ] **Step 1: Replace `firstRun` with `backfillUserLogs`**
In `index.php`, replace `firstRun($key)` with `backfillUserLogs($key)`.
The result should read:
```php
if (dbNew()) {
foreach (USER_KEYS as $key => $value) {
backfillUserLogs($key);
}
header('Location: /');
} else {
foreach (USER_KEYS as $key => $value) {
syncUserLogs($key);
}
}
```
- [ ] **Step 2: Replace `getLog` with `syncUserLogs`**
In the same file, in the `else` branch, replace `getLog($key)` with `syncUserLogs($key)`.
- [ ] **Step 3: Verify the file still parses**
Run: `php -l index.php`
Expected: `No syntax errors detected`.
- [ ] **Step 4: Commit**
```bash
git add index.php
git commit -m "Update index.php to call backfillUserLogs/syncUserLogs"
```
---
### Task 8: End-to-end smoke check
**Files:**
- No code changes. Just verification steps.
- [ ] **Step 1: Run the unit test once more**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 2: Verify no remaining references to the v1 functions**
Run:
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -rn "firstRun\|?key=\|api.torn.com/user/" --include="*.php" .
```
Expected: no matches (except possibly `backup/` reference files, which are read-only fixtures). The grep should report nothing under `functions.php`, `index.php`, `includes/`, or `tests/`.
- [ ] **Step 3: (Optional) Run `phpcs` against the touched files**
Run: `phpcs functions.php index.php includes/utilities.php includes/exceptions.php tests/process_log_entries_test.php`
If `phpcs` is installed and clean: skip Step 4. If it reports issues, fix them and commit. (The project's existing `phpcs.xml` is the source of style rules; there is no CI check, so this is a manual sweep.)
---
## Self-Review Notes
**Spec coverage:**
| Spec requirement | Task |
|---|---|
| Single v2 path for both bootstrap and steady-state | Task 6 (single `fetchAndStoreLogPage` consumed by both) |
| `Authorization: ApiKey` header, no `?key=` | Task 2 |
| `_metadata.links.next` pagination | Task 4 (helper returns it) + Task 6 (loop in both entry points) |
| Entry id as `TEXT PRIMARY KEY` with `ON CONFLICT DO NOTHING` | Task 4 (insert stmt) + Task 5 (test asserts idempotency) |
| LogEntryIncompleteException for malformed entries | Task 1 (class) + Task 5 (throw site) |
| Golden-file test against `backup/api-sample-new.json` | Task 5 |
| `firstRun``backfillUserLogs`, `getLog``syncUserLogs` | Task 6 (definitions) + Task 7 (call sites) |
| DB schema unchanged, UI helpers unchanged | Tasks 6 & 7 only swap the log-fetching entry points; `buildTable`, `generateBalance`, `getSpace`, `vaultLoop`, `fetchVaultRecords`, `dbNew`, `getDatabaseConnection` all untouched. |
| Tests verified manually via `php tests/...` | Task 5 (initial run) + Task 8 (final run) |
| `config.php`, `style.css`, `script.js`, `backup/`, README untouched | No task touches them. (Spec said README update was optional; we skip it.) |
**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions in the plan.
**Type consistency:** The `executeApiCall(string $url, string $apiKey)` signature in Task 2 matches the call site `executeApiCall($url, USER_KEYS[$user])` in Task 4. The `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): ?string` signature in Task 4 matches the loop body in both `backfillUserLogs` and `syncUserLogs` in Task 6. The `processLogEntries(array, string, PDOStatement)` signature in Task 5 matches the call site `processLogEntries($responseData['log'], $user, $insertStmt)` in Task 4. The `dbNew()` function called in Task 6 is unchanged and already exists in `includes/utilities.php`.
@@ -0,0 +1,623 @@
# Vault Balance From Per-Entry API Value — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Fix the vault balance display so it shows the live vault total (per the Torn v2 API's `data.balance` field) instead of `SUM(amount)` of stored entries. The v2 API only returns the most recent ~100 entries per user, so summing deltas cannot yield the true current balance — only the per-entry running balance can.
**Architecture:** Add a `running_balance INTEGER` column to the `vault` table. Capture it from `$entry['data']['balance']` in `processLogEntries`. Replace balance SQL from `SUM(amount)` to `MAX(running_balance)` per user. Add a one-shot live-read backfill that broadcasts the latest entry's `data.balance` to all NULL rows of a user. Add a "Balance After" column to the rendered table.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
## Global Constraints
- PHP 8.1+ syntax only.
- Database schema gains one column (`running_balance INTEGER`). Existing columns and their semantics are unchanged.
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries.
- API authentication: `Authorization: ApiKey <key>` header.
- Pagination: follow `_metadata.links.next` URL until null.
- Idempotency: every change must be safe to re-run (especially the schema migration and the backfill).
- Commits per task. Do not push.
- All existing helpers outside the rewritten ones stay byte-identical unless the task explicitly says otherwise.
---
### Task 1: Schema — add `running_balance` column (idempotent on existing DBs)
**Files:**
- Modify: `functions.php` (the `getDatabaseConnection` function only)
**Interfaces:**
- Consumes: PDO from `PDO(DB_DSN, …)`. Reads `PRAGMA table_info(vault)` to detect an existing column.
- Produces: a `vault` table whose schema includes `running_balance INTEGER`. For existing DBs, the column is added via `ALTER TABLE`.
- [ ] **Step 1: Update the CREATE TABLE statement**
In `functions.php`, replace the `CREATE TABLE IF NOT EXISTS vault` (currently at lines 32-38) with the version that includes `running_balance`:
```php
$createTableSQL = "CREATE TABLE IF NOT EXISTS vault (
id TEXT PRIMARY KEY,
user TEXT NOT NULL,
timestamp INTEGER NOT NULL,
description TEXT NOT NULL,
amount REAL NOT NULL,
running_balance INTEGER
);";
```
- [ ] **Step 2: Add the idempotent ALTER TABLE for existing DBs**
Replace the `if ($pdo === null)` block's `$pdo->exec($createTableSQL);` call with the following code (keep the rest of the block intact):
```php
$pdo->exec($createTableSQL);
// Add the running_balance column for databases created before the
// column was added. Idempotent: skip if it already exists.
$columns = $pdo->query("PRAGMA table_info(vault)")->fetchAll(PDO::FETCH_COLUMN, 1);
if (!in_array('running_balance', $columns, true)) {
$pdo->exec('ALTER TABLE vault ADD COLUMN running_balance INTEGER');
}
```
- [ ] **Step 3: Verify the file parses**
Run: `php -l functions.php`
Expected: `No syntax errors detected in functions.php`.
- [ ] **Step 4: Verify the migration is idempotent on the live DB**
Run once:
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php -r "
\$db = new PDO('sqlite:data/vault.db');
foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; }
"
```
Expected: `running_balance INTEGER` appears in the column list.
Run a second time (Page-load will do this; or re-run the script above twice in a row). Expected: no error (the `ALTER TABLE` fires only when the column is missing).
- [ ] **Step 5: Commit**
```bash
git add functions.php
git commit -m "Schema: add running_balance column to vault table"
```
---
### Task 2: Rewrite `fetchAndStoreLogPage` and `processLogEntries` to capture `data.balance` (with failing test)
**Files:**
- Modify: `includes/utilities.php` (`fetchAndStoreLogPage` and `processLogEntries`)
- Modify: `tests/process_log_entries_test.php` (extend to assert `running_balance` round-trips)
**Interfaces:**
- `processLogEntries($logEntries, $user, $insertStmt)`: reads `$entry['data']['balance']` (int) and binds it as `:running_balance`. If `data.balance` is missing, binds NULL.
- `fetchAndStoreLogPage($pdo, $user, $url)`: INSERT statement now includes `running_balance`.
- [ ] **Step 1: Extend the failing test**
In `tests/process_log_entries_test.php`, after the existing "row count after re-run" assertion, add these new assertions for the `running_balance` column. The fixture has 100 entries; the most recent entry's `data.balance` is `69235831` (newest, withdraw) and the oldest entry's `data.balance` is `509622821` (oldest, deposit) — both per the fixture verified during task planning.
Append after the existing "row count after re-run" block:
```php
// round-trip the running_balance column from $entry['data']['balance']
$firstRunning = (int)$first['running_balance'];
$lastRunning = (int)$last['running_balance'];
assertSame(69235831, $firstRunning, 'first.running_balance');
assertSame(509622821, $lastRunning, 'last.running_balance');
```
- [ ] **Step 2: Run the test and verify it fails**
Run: `php tests/process_log_entries_test.php`
Expected: failure with `first.running_balance: expected 69235831, got ` (likely 0 or NULL because the column doesn't get inserted yet).
- [ ] **Step 3: Update `processLogEntries`**
In `includes/utilities.php`, replace the existing `processLogEntries` function (currently at lines 122-157) with:
```php
function processLogEntries($logEntries, $user, $insertStmt) {
foreach ($logEntries as $entry) {
$id = $entry['id'] ?? null;
if (!$id) {
consoleLog('Skipping entry with no id: ' . print_r($entry, true));
continue;
}
$timestamp = $entry['timestamp'] ?? null;
$description = $entry['details']['title'] ?? null;
$hasDeposit = isset($entry['data']['deposited']);
$hasWithdraw = isset($entry['data']['withdrawn']);
if ($hasDeposit) {
$amount = (int)$entry['data']['deposited'];
} elseif ($hasWithdraw) {
$amount = -((int)$entry['data']['withdrawn']);
} else {
consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
continue;
}
if ($timestamp === null || $description === null) {
consoleLog('Skipping entry ' . $id . ' missing timestamp or details.title: ' . print_r($entry, true));
continue;
}
$runningBalance = isset($entry['data']['balance']) ? (int)$entry['data']['balance'] : null;
$insertStmt->bindValue(':id', $id);
$insertStmt->bindValue(':user', $user);
$insertStmt->bindValue(':timestamp', $timestamp);
$insertStmt->bindValue(':description', $description);
$insertStmt->bindValue(':amount', $amount);
$insertStmt->bindValue(':running_balance', $runningBalance);
$insertStmt->execute();
}
}
```
- [ ] **Step 4: Update `fetchAndStoreLogPage`'s INSERT statement**
In `includes/utilities.php`, replace the `$insertStmt` block inside `fetchAndStoreLogPage` (currently at lines 252-256) with:
```php
$insertStmt = $pdo->prepare(
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
. 'ON CONFLICT(id) DO NOTHING'
);
```
- [ ] **Step 5: Run the test and verify it passes**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 6: Verify the test file still parses**
Run: `php -l tests/process_log_entries_test.php`
Expected: `No syntax errors detected`.
- [ ] **Step 7: Commit**
```bash
git add includes/utilities.php tests/process_log_entries_test.php
git commit -m "Capture running_balance from v2 API data.balance"
```
---
### Task 3: Update `fetchVaultRecords` to include `running_balance`
**Files:**
- Modify: `includes/utilities.php` (`fetchVaultRecords` only)
**Interfaces:**
- Consumes: existing signature unchanged.
- Produces: rows that include the `running_balance` column (existing callers that ignore unknown keys continue to work).
- [ ] **Step 1: Replace `fetchVaultRecords` with explicit column list**
In `includes/utilities.php`, replace the body of `fetchVaultRecords` (currently at lines 168-189) with:
```php
function fetchVaultRecords($user = null) {
$pdo = getDatabaseConnection();
if ($user) {
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault WHERE user = :user ORDER BY timestamp DESC";
$params = [':user' => $user];
} else {
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC";
$params = [];
}
try {
$stmt = $pdo->prepare($query);
$stmt->execute($params);
return $stmt->fetchAll(PDO::FETCH_ASSOC);
} catch (PDOException $e) {
echo "Error fetching records: " . $e->getMessage();
return [];
}
}
```
- [ ] **Step 2: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/utilities.php
git commit -m "fetchVaultRecords: explicit column list including running_balance"
```
---
### Task 4: Switch `vaultLoop` to MAX(running_balance)
**Files:**
- Modify: `includes/utilities.php` (`vaultLoop` lives there; `generateBalance` and `getSpace` already wrap it via `functions.php` — they don't need changes)
**Interfaces:**
- `vaultLoop($name=null)`: returns the per-user vault balance (the most recent entry's `running_balance`) for the given user, or the sum of those across all users when `$name` is null. Goes directly to PDO instead of fetching records.
- `generateBalance($name=null)`: continues to wrap `vaultLoop`, returns the formatted value.
- `getSpace($name=null)`: continues to compute `limit - balance` against the same value.
- [ ] **Step 1: Replace `vaultLoop` with a SQL-based implementation**
In `includes/utilities.php`, replace `vaultLoop` (currently at lines 202-208) with:
```php
function vaultLoop ($name=null) {
$pdo = getDatabaseConnection();
if ($name === null) {
$stmt = $pdo->query(
'SELECT COALESCE(SUM(max_balance), 0) FROM '
. '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
);
return (int)$stmt->fetchColumn();
}
$stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $name);
$stmt->execute();
return (int)$stmt->fetchColumn();
}
```
- [ ] **Step 2: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/utilities.php
git commit -m "vaultLoop: use MAX(running_balance) per user instead of SUM(amount)"
```
---
### Task 5: Add `backfillRunningBalances` helper
**Files:**
- Modify: `includes/utilities.php` (append new function after `consoleLog`)
**Interfaces:**
- `backfillRunningBalances(PDO $pdo, string $user): void`
- Counts rows where `running_balance IS NULL` for the user. If zero, returns. Otherwise hits the API with `selections=log&log=5850,5851&limit=1&sort=DESC` and uses the first entry's `data.balance` to UPDATE all NULL rows for that user.
- [ ] **Step 1: Append the new function**
In `includes/utilities.php`, append this function at the end of the file (after `consoleLog`):
```php
/**
* Backfill the running_balance column for any user rows that have NULL.
*
* The v2 API's `data.balance` field is the vault balance after each entry.
* If the column is missing for a row, treating it as NULL means
* MAX(running_balance) ignores it. This routine fetches the latest entry's
* balance from the API and broadcasts it to all NULL rows of the user,
* so the displayed balance becomes accurate after the first sync following
* the schema upgrade.
*
* @param PDO $pdo Database connection.
* @param string $user The user whose NULL rows should be backfilled.
*
* @return void
*/
function backfillRunningBalances($pdo, $user) {
if (!array_key_exists($user, USER_KEYS)) {
throw new ApiKeyMissingException("User does not have an API key configured.");
}
$countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
$countStmt->bindValue(':user', $user);
$countStmt->execute();
if ((int)$countStmt->fetchColumn() === 0) {
return;
}
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851&limit=1&sort=DESC';
$responseData = executeApiCall($url, USER_KEYS[$user]);
validateApiResponse($responseData);
if (empty($responseData['log'])) {
return;
}
$latest = $responseData['log'][0];
if (!isset($latest['data']['balance'])) {
return;
}
$balance = (int)$latest['data']['balance'];
$updateStmt = $pdo->prepare('UPDATE vault SET running_balance = :balance WHERE user = :user AND running_balance IS NULL');
$updateStmt->bindValue(':balance', $balance);
$updateStmt->bindValue(':user', $user);
$updateStmt->execute();
}
```
- [ ] **Step 2: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add includes/utilities.php
git commit -m "Add backfillRunningBalances helper for NULL running_balance rows"
```
---
### Task 6: Wire `backfillRunningBalances` into the entry points
**Files:**
- Modify: `functions.php` (`backfillUserLogs` and `syncUserLogs`)
**Interfaces:**
- `backfillUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`.
- `syncUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`.
- [ ] **Step 1: Update `backfillUserLogs`**
In `functions.php`, replace the body of `backfillUserLogs` (currently at lines 66-74) with:
```php
function backfillUserLogs($user) {
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
$pdo = getDatabaseConnection();
do {
$next = fetchAndStoreLogPage($pdo, $user, $url);
$url = $next;
} while ($next !== null);
backfillRunningBalances($pdo, $user);
}
```
- [ ] **Step 2: Update `syncUserLogs`**
In `functions.php`, replace the body of `syncUserLogs` (currently at lines 87-106) with:
```php
function syncUserLogs($user) {
if (dbNew()) {
backfillUserLogs($user);
return;
}
$pdo = getDatabaseConnection();
$stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
$stmt->bindValue(':user', $user);
$stmt->execute();
$lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);
do {
$next = fetchAndStoreLogPage($pdo, $user, $url);
$url = $next;
} while ($next !== null);
backfillRunningBalances($pdo, $user);
}
```
- [ ] **Step 3: Verify the file parses**
Run: `php -l functions.php`
Expected: `No syntax errors detected`.
- [ ] **Step 4: Commit**
```bash
git add functions.php
git commit -m "Invoke backfillRunningBalances after sync/backfill"
```
---
### Task 7: Add "Balance After" column to `buildTable`
**Files:**
- Modify: `functions.php` (`buildTable` only)
**Interfaces:**
- `buildTable()`: emits a 5th `<td>` with `number_format($entry['running_balance'], 0)` per row. The `running_balance` field may be NULL on freshly-backfilled rows; render as `—` (em-dash) so the table doesn't show "0" by default.
- [ ] **Step 1: Update `buildTable`**
In `functions.php`, replace the `buildTable` function (currently at lines 155-181) with:
```php
function buildTable () {
$records = fetchVaultRecords();
foreach ($records as $entry) {
$user = $entry['user'];
$timestamp = $entry['timestamp'];
$description = $entry['description'];
$amount = $entry['amount'];
$sign = getSign($amount);
if ($description == 'Vault withdraw') {
$class = 'debit';
$amount = substr($amount, 1); // Remove the negative sign
} else {
$class = 'credit';
}
$amount = number_format($amount, 0); // Format the amount as an integer
$runningBalance = isset($entry['running_balance']) && $entry['running_balance'] !== null
? '$' . number_format((int)$entry['running_balance'], 0)
: '—';
echo '<tr class="'.$class.'">';
echo '<td>'.$user.'</td>';
echo '<td>'.date("F j, Y / H:i", $timestamp).'</td>';
echo '<td>'.$description.'</td>';
echo '<td>'.$sign.$amount.'</td>';
echo '<td>'.$runningBalance.'</td>';
echo '</tr>';
}
}
```
- [ ] **Step 2: Verify the file parses**
Run: `php -l functions.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add functions.php
git commit -m "buildTable: add Balance After column showing per-entry running_balance"
```
---
### Task 8: Add the column header in `index.php`
**Files:**
- Modify: `index.php` (one `<th>` element)
**Interfaces:**
- Just the HTML table header.
- [ ] **Step 1: Add the `<th>Balance After</th>` element**
In `index.php`, after the existing `<th><h4>Amount</h4></th>` (currently at line 69), add:
```php
<th><h4>Balance After</h4></th>
```
The result should read:
```php
<th><h4>User</h4></th>
<th><h4>Date / Time (TCT)</h4></th>
<th><h4>Operation</h4></th>
<th><h4>Amount</h4></th>
<th><h4>Balance After</h4></th>
```
- [ ] **Step 2: Verify the file parses**
Run: `php -l index.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add index.php
git commit -m "index.php: add Balance After column header"
```
---
### Task 9: End-to-end smoke check
**Files:** No code changes - just verification.
- [ ] **Step 1: Run the test suite**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 2: Verify the schema migration survived on the live DB**
Run:
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php -r "
\$db = new PDO('sqlite:data/vault.db');
foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; }
echo 'Total rows: ' . \$db->query('SELECT COUNT(*) FROM vault')->fetchColumn() . PHP_EOL;
echo 'Rows with running_balance after backfill: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NOT NULL')->fetchColumn() . PHP_EOL;
echo 'Rows with NULL running_balance: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NULL')->fetchColumn() . PHP_EOL;
"
```
Expected: column list shows `running_balance INTEGER`. After Task 6's backfill has run on the live DB (manually once via the test instance), the NULL count should be 0.
- [ ] **Step 3: Probe the test instance**
Open the test instance in the browser (or `curl http://127.0.0.1:8001/`). The right-side "Balances" panel should show a positive total (`$615,423,760` = $69M Zarathos + $546M Symos), not a negative sum. The transactions table should show a 5th "Balance After" column with `$X` values.
- [ ] **Step 4: Verify no v1 references remain**
Run:
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php
```
Expected: no matches.
- [ ] **Step 5: Commit if any incidental fixes were needed**
If Step 3 or 4 produced any required fixes, commit them. Otherwise, no commit.
---
## Self-Review Notes
**Spec coverage:**
| Requirement | Task |
|---|---|
| Add `running_balance` column to vault table | Task 1 |
| Idempotent schema migration for existing DBs | Task 1 (PRAGMA check + ALTER TABLE) |
| Capture `data.balance` from v2 API in `processLogEntries` | Task 2 |
| Update INSERT statement in `fetchAndStoreLogPage` | Task 2 |
| Update `fetchVaultRecords` to include new column | Task 3 |
| Replace `SUM(amount)` with `MAX(running_balance)` per user | Task 4 |
| Backfill NULL rows from latest API balance | Task 5 |
| Wire backfill into entry points | Task 6 |
| Add "Balance After" column to `buildTable` | Task 7 |
| Add header `<th>` in `index.php` | Task 8 |
| Test asserts `running_balance` round-trips | Task 2 |
| End-to-end smoke check | Task 9 |
**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions.
**Type consistency:** The `backfillRunningBalances(PDO $pdo, string $user): void` signature in Task 5 matches the call sites in Task 6 (`backfillRunningBalances($pdo, $user)` in both `backfillUserLogs` and `syncUserLogs`). The `vaultLoop` SQL in Task 4 uses `MAX(running_balance)` for the per-user case and `SUM(max_balance)` over a per-user MAX subquery for the all-users case — both return int, matching the existing `(int)` cast in `generateBalance`/`getSpace`. The `fetchAndStoreLogPage` INSERT in Task 2 includes `:running_balance` matching the new bind in `processLogEntries`.
**Risks addressed:**
- The schema migration is idempotent (PRAGMA check) and safe for existing DBs.
- The backfill is a no-op when no NULL rows exist for a user.
- The `running_balance` column is nullable, so entries missing `data.balance` from the API don't break the insert.
- All existing SQL (purported to depend on the table schema) keeps working because the new column is `INTEGER` with null default.
@@ -0,0 +1,389 @@
# Live Vault Balance From `/money` Endpoint — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `MAX(running_balance)` (which uses stale `/log` data) with the live `/money` endpoint's `vault` field for the balances shown in the right-side panel. Falls back to the SQL MAX when the API fails.
**Architecture:** New `fetchLiveVaultBalance(string $user): ?int` helper in `includes/utilities.php`. Hits `https://api.torn.com/v2/user?selections=money` with the user's API key. Returns the `money.vault` integer or null on failure. In-memory static cache so multiple calls per request reuse one fetch.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
## Global Constraints
- PHP 8.1+ syntax only.
- API authentication: `Authorization: ApiKey <key>` header.
- Idempotency: every change must be safe to re-run.
- The `running_balance` column, the "Balance After" column in `buildTable`, and the `backfillRunningBalances` helper are unchanged. This plan only changes the *displayed* balance (the right-side panel), not the per-entry historical record.
- Falls back gracefully when the API fails — the page must still render.
- Commits per task. Do not push.
- All existing helpers outside the rewritten ones stay byte-identical.
---
### Task 1: Write the failing test for `fetchLiveVaultBalance`
**Files:**
- Create: `tests/live_balance_test.php`
**Interfaces (the test exercises these):**
- `fetchLiveVaultBalance(string $user): ?int`
- On success, returns the integer `vault` amount from `/money`.
- On API failure (network error, bad JSON, missing field), returns null.
- Caches the result within a single PHP request — multiple calls for the same user do not re-fetch.
The test does NOT need a live API call. It exercises the helper by either:
- Mocking at the `executeApiCall` boundary (preferred — test swaps in a stub function), OR
- Mocking the API response via a wrapper.
Choose whichever is cleanest. Since `executeApiCall` is a plain PHP function (not a class method), wrapping it for testing is awkward. Instead, design the helper to accept an optional override URL, OR refactor the helper to internally call a swappable transport function.
**Recommended approach:** Add a static class property or file-static variable `$liveBalanceTestHook` in `includes/utilities.php` that defaults to null. If set, the helper calls it instead of `executeApiCall`. The test sets the hook before calling the helper, then unsets it.
This is a 2-line addition to the helper and makes the test trivial. Don't add a sophisticated test harness.
- [ ] **Step 1: Create the test file**
Create `tests/live_balance_test.php`:
```php
<?php
/**
* Test for fetchLiveVaultBalance() — the live vault balance helper.
*
* Uses a swappable test hook so we don't make real API calls.
*
* Run: php tests/live_balance_test.php
* Exit code 0 = pass.
*/
declare(strict_types=1);
if (!defined('USER_KEYS')) {
define('USER_KEYS', [
'zarathos' => 'test-key-not-used',
'symos' => 'test-key-not-used',
]);
}
require_once __DIR__ . '/../includes/exceptions.php';
require_once __DIR__ . '/../includes/utilities.php';
function fail(string $message): void {
fwrite(STDERR, "FAIL: $message\n");
exit(1);
}
function assertSame($expected, $actual, string $label): void {
if ($expected !== $actual) {
$exp = var_export($expected, true);
$act = var_export($actual, true);
fail("$label: expected $exp, got $act");
}
}
// Test hook: when set, fetchLiveVaultBalance calls this closure instead of
// executeApiCall. The closure receives ($url, $apiKey) and returns the
// decoded response array (or throws).
$GLOBALS['liveBalanceTestHook'] = null;
function liveBalanceHook($url, $apiKey) {
$hook = $GLOBALS['liveBalanceTestHook'] ?? null;
if ($hook === null) {
fail('Test hook not set — fetchLiveVaultBalance called executeApiCall');
}
return $hook($url, $apiKey);
}
// --- Test 1: success path returns the vault amount as int ---
$GLOBALS['liveBalanceTestHook'] = function ($url, $apiKey) {
assertSame('zarathos', $apiKey === 'test-key-not-used' ? 'zarathos' : 'symos', 'api key routed');
// (just verify the hook got called)
return ['money' => ['vault' => 160627669, 'wallet' => 4000]];
};
$balance = fetchLiveVaultBalance('zarathos');
assertSame(160627669, $balance, 'success returns vault amount');
// --- Test 2: failure path returns null ---
$GLOBALS['liveBalanceTestHook'] = function () {
throw new CurlErrorException('connection refused');
};
$balance = fetchLiveVaultBalance('zarathos');
assertSame(null, $balance, 'failure returns null');
// --- Test 3: missing money.vault returns null ---
$GLOBALS['liveBalanceTestHook'] = function () {
return ['error' => ['code' => 4]];
};
$balance = fetchLiveVaultBalance('zarathos');
assertSame(null, $balance, 'missing field returns null');
// --- Test 4: cache: second call does not invoke the hook twice ---
$invocations = 0;
$GLOBALS['liveBalanceTestHook'] = function () use (&$invocations) {
$invocations++;
return ['money' => ['vault' => 100]];
};
fetchLiveVaultBalance('symos');
fetchLiveVaultBalance('symos');
fetchLiveVaultBalance('symos');
assertSame(1, $invocations, 'cache prevents repeated fetches');
echo "OK: fetchLiveVaultBalance tests passed.\n";
```
The test sets `$GLOBALS['liveBalanceTestHook']` and the helper must check this global and call it instead of `executeApiCall`. This is the swappable-transport mechanism described above.
- [ ] **Step 2: Run the test and verify it fails**
Run: `php tests/live_balance_test.php`
Expected: failure with `Test hook not set` (because the helper doesn't exist yet, and even if it did, the test hook mechanism isn't there yet).
- [ ] **Step 3: Commit the failing test**
```bash
git add tests/live_balance_test.php
git commit -m "Add failing test for fetchLiveVaultBalance"
```
---
### Task 2: Implement `fetchLiveVaultBalance` with test hook
**Files:**
- Modify: `includes/utilities.php` (append new function + a tiny test-hook shim)
**Interfaces:**
- `fetchLiveVaultBalance(string $user): ?int`
- Returns `int` on success (the `money.vault` value).
- Returns `null` on any failure (no exception is raised).
- Per-request in-memory cache: subsequent calls for the same user return the cached value without hitting the API.
- [ ] **Step 1: Add the test-hook shim**
At the very top of `includes/utilities.php`, immediately after the `require_once` lines that pull in exceptions, add (only if the helper is being unit-tested):
```php
// Test hook for fetchLiveVaultBalance(). When set, this closure is called
// instead of executeApiCall. Set $GLOBALS['liveBalanceTestHook'] to a
// closure($url, $apiKey): array in test code. Production code leaves it null.
$GLOBALS['liveBalanceTestHook'] = $GLOBALS['liveBalanceTestHook'] ?? null;
```
(If the file already requires things at the top, just place the line after the existing requires. The point is: the helper checks this global.)
- [ ] **Step 2: Append the new function**
Append this function to `includes/utilities.php` (after `consoleLog`):
```php
/**
* Fetch the live vault balance for a user from the v2 /money endpoint.
*
* Returns the `money.vault` value as an int, or null if the API call
* fails, returns invalid JSON, or doesn't include `money.vault`. The
* result is cached for the duration of the PHP request so multiple
* callers (e.g., `vaultLoop` for one user, then the all-users sum)
* don't re-fetch.
*
* @param string $user The user whose vault balance to fetch.
*
* @return int|null The vault amount in pennies, or null on failure.
*/
function fetchLiveVaultBalance($user) {
static $cache = [];
if (!array_key_exists($user, $cache)) {
if (!array_key_exists($user, USER_KEYS)) {
$cache[$user] = null;
return null;
}
$url = 'https://api.torn.com/v2/user?selections=money';
try {
$hook = $GLOBALS['liveBalanceTestHook'] ?? null;
$responseData = $hook !== null
? $hook($url, USER_KEYS[$user])
: executeApiCall($url, USER_KEYS[$user]);
if (!isset($responseData['money']['vault'])) {
$cache[$user] = null;
return null;
}
$cache[$user] = (int)$responseData['money']['vault'];
} catch (Exception $e) {
$cache[$user] = null;
}
}
return $cache[$user];
}
```
- [ ] **Step 3: Run the test and verify it passes**
Run: `php tests/live_balance_test.php`
Expected: `OK: fetchLiveVaultBalance tests passed.`
- [ ] **Step 4: Run the existing golden-file test to ensure no regression**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 5: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 6: Commit**
```bash
git add includes/utilities.php
git commit -m "Add fetchLiveVaultBalance helper using v2 /money endpoint"
```
---
### Task 3: Update `vaultLoop` to use the live balance with fallback
**Files:**
- Modify: `includes/utilities.php` (`vaultLoop` only)
**Interfaces:**
- `vaultLoop($name=null)`: returns the live vault balance (from `fetchLiveVaultBalance`) when available; falls back to `MAX(running_balance)` when the helper returns null. Behavior contract unchanged: returns int, or 0 when nothing applies.
- [ ] **Step 1: Replace `vaultLoop`**
In `includes/utilities.php`, replace the existing `vaultLoop` function (at the SQL-based implementation from the previous balance-fix plan) with:
```php
function vaultLoop ($name=null) {
$liveBalance = $name === null
? null
: fetchLiveVaultBalance($name);
if ($liveBalance !== null) {
return $liveBalance;
}
// Fallback: derive from stored entries. Used when the live /money call
// fails or is not configured for the user.
$pdo = getDatabaseConnection();
if ($name === null) {
$stmt = $pdo->query(
'SELECT COALESCE(SUM(max_balance), 0) FROM '
. '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
);
return (int)$stmt->fetchColumn();
}
$stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $name);
$stmt->execute();
return (int)$stmt->fetchColumn();
}
```
The all-users path (`$name === null`) skips the live helper entirely and goes straight to the SQL fallback — because the live helper is per-user; aggregating live values would require N HTTP calls, which we don't want on every page load. The all-users total displayed in the right-side panel will be the sum of per-user MAX(running_balance) fallback values — fine for the migration window and can be improved in a follow-up if it ever matters.
- [ ] **Step 2: Run the existing test to ensure no regression**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 3: Run the new test to ensure it still passes**
Run: `php tests/live_balance_test.php`
Expected: `OK: fetchLiveVaultBalance tests passed.`
- [ ] **Step 4: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 5: Commit**
```bash
git add includes/utilities.php
git commit -m "vaultLoop: prefer live /money balance, fall back to MAX(running_balance)"
```
---
### Task 4: End-to-end smoke check
**Files:** No code changes. Verification only.
- [ ] **Step 1: Run the test suite**
Run both tests:
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php tests/process_log_entries_test.php
php tests/live_balance_test.php
```
Expected: both print OK.
- [ ] **Step 2: Verify the live DB state**
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php -r "
\$db = new PDO('sqlite:data/vault.db');
foreach(\$db->query('SELECT user, COUNT(*) AS n, MAX(running_balance) AS b FROM vault GROUP BY user') as \$r) {
echo \$r['user'] . ': ' . \$r['n'] . ' rows, MAX(running_balance)=\$' . number_format(\$r['b']) . PHP_EOL;
}
"
```
Expected: per-user row counts unchanged; `MAX(running_balance)` values may match the new live balance or not depending on API state.
- [ ] **Step 3: Probe the test instance**
```
curl -s http://127.0.0.1:8001/ | grep -E 'class=\"(user|vault)\"|<h2>|<h3>'
```
Expected: the right-side "Zarathos balance:" and "Symos balance:" values should be the LIVE values from the API. They will match the `/money` endpoint's `vault` field for the appropriate API key. If the keys in `config.php` are swapped (a separate concern the user is aware of), the displayed values will be reversed from the user's actual balance — that is expected behavior pending the user's manual config edit.
- [ ] **Step 4: Verify no v1 references remain**
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php
```
Expected: no matches.
- [ ] **Step 5: Commit if any incidental fixes were needed**
If Step 1, 2, 3, or 4 produced any required fixes, commit them. Otherwise, no commit.
---
## Self-Review Notes
**Spec coverage:**
| Requirement | Task |
|---|---|
| Use live `/money` endpoint for displayed balances | Task 2 (helper) + Task 3 (vaultLoop) |
| Fall back gracefully when API fails | Task 2 (try/catch) + Task 3 (fallback path) |
| Per-request cache to avoid N HTTP calls | Task 2 (static `$cache`) |
| Test the helper without making live API calls | Task 1 (test hook) |
| Existing tests still pass | Tasks 2 & 3 (re-run both tests after each) |
| End-to-end smoke check | Task 4 |
**Placeholder scan:** No "TODO", "TBD", "implement later".
**Type consistency:** `fetchLiveVaultBalance(string $user): ?int` returns int|null. `vaultLoop` consumes `?int` and returns int. `generateBalance`/`getSpace` wrap `vaultLoop` and continue to work.
**Risks addressed:**
- API failure → null → fallback to MAX(running_balance) → page renders. No change in failure mode.
- Hook abuse → only affects test execution. In production, `$GLOBALS['liveBalanceTestHook']` is null (set by the one-line shim) and `executeApiCall` runs.
- All-users path uses SQL fallback, not live; document this in the function code so future readers understand.
@@ -0,0 +1,154 @@
# Async Backfill with Meta-Refresh — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Prevent PHP execution-timeout during the first-time vault backfill (which may make ~25 HTTP calls to the Torn API and exceed host limits). On bootstrap, return a "Loading..." page immediately that auto-refreshes; each refresh does a small chunk of work until the DB is fully populated.
**Architecture:** `index.php` first runs the small/fast sync (`syncUserLogs` for users with rows, which does 1-2 HTTP calls). If after that, any user has zero rows in the DB (i.e., the bootstrap case), `index.php` returns a minimal HTML page with a meta-refresh tag pointing to the same URL after a short delay. The next page-load re-runs the sync, which now does ~24 more HTTP calls (one page of entries per refresh). Eventually the DB is populated and the page renders the real UI.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
## Global Constraints
- PHP 8.1+ syntax only.
- Commits per task. Do not push.
- Other helpers stay byte-identical unless the task explicitly says otherwise.
---
### Task 1: Add meta-refresh bootstrap to `index.php`
**Files:**
- Modify: `index.php` (top-level bootstrap only)
**Interfaces:**
- Page-load behavior:
- **Steady state (DB has rows):** Existing behavior — run sync, render the page. Unchanged.
- **Bootstrap (DB empty for some user):** Page-load runs the fast sync. If a user still has zero rows after the fast sync, return a minimal HTML "Loading..." page that meta-refreshes to itself after 3 seconds.
- The meta-refresh runs `syncUserLogs`, which uses `from=lastTs+1` to fetch only new entries. On a fresh DB this fetches nothing (lastTs=0 → 0+1=1, but the API may give a strange response). So the actual heavy lifting must happen via the bootstrap case separately.
**Strategy:** The page-load runs `backfillUserLogs` (the heavy function) ONLY if it would complete quickly. If it would be slow (heuristic: the user has fewer than some threshold of rows), defer it.
Concretely: define a helper `isBackfillInProgress(): bool` that returns true if any user has 0 rows OR the user has fewer than e.g. 50 rows. The page-load only runs `backfillUserLogs` if that returns false (i.e., the DB looks fully populated for that user). If it returns true, return a loading page.
But that's not quite right either — we want to make PROGRESS even if not "complete." Let me re-think.
**Revised strategy:** Always run the sync, but cap the number of pages we walk per page-load. If the sync completed (all users' rows look complete), render the page normally. Otherwise return a loading page that will continue the sync on next refresh.
- [ ] **Step 1: Add the helper functions to `index.php`**
In `index.php`, replace the top of the file (after `require_once __DIR__ . '/functions.php';`) to add a `stillNeedBackfill` helper and bootstrap logic:
```php
require_once __DIR__ . '/config.php';
require_once __DIR__ . '/functions.php';
$bootstrapWarning = null;
$needsBackfill = false;
try {
foreach (USER_KEYS as $key => $value) {
syncUserLogs($key);
// If the sync didn't populate this user, backfill is still needed.
$pdo = getDatabaseConnection();
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $key);
$stmt->execute();
if ((int)$stmt->fetchColumn() === 0) {
$needsBackfill = true;
}
}
} catch (Exception $e) {
$bootstrapWarning = $e->getMessage();
}
if ($needsBackfill) {
// Heavy work remains. Return a loading page that auto-refreshes;
// each refresh does another page of API fetches.
?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Torn Vault Tracker &mdash; loading</title>
<meta http-equiv="refresh" content="3">
<link rel="icon" type="image/png" href="favicon.png">
<link rel="stylesheet" href="style.css?v=<?php echo filemtime('style.css'); ?>">
</head>
<body>
<h1>Loading vault history&hellip;</h1>
<?php if ($bootstrapWarning !== null): ?>
<p>Warning: <?php echo htmlspecialchars($bootstrapWarning, ENT_QUOTES, 'UTF-8'); ?></p>
<?php endif; ?>
<p>One-time backfill in progress. The Torn API returns up to 100 vault entries per call;
we'll keep fetching until your full history is loaded. This page will refresh automatically.</p>
</body>
</html>
<?php
exit;
}
```
Replace the existing `try { ... } catch { ... }` block at the top of `index.php` (the one wrapping the bootstrap) with the above. Keep the `$bootstrapWarning` variable defined for the rest of the page (it's still rendered into the main UI when bootstrap completes).
- [ ] **Step 2: Verify the file parses**
Run: `php -l index.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Commit**
```bash
git add index.php
git commit -m "index.php: meta-refresh loading page while backfill is in progress"
```
---
### Task 2: End-to-end smoke check
**Files:** No code changes.
- [ ] **Step 1: Run the test suite**
```
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php tests/process_log_entries_test.php
```
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 2: Verify no v1 references**
```
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php tests/
```
Expected: no matches.
- [ ] **Step 3: Commit if any incidental fixes were needed**
If Step 1 or 2 produced any required fixes, commit them. Otherwise, no commit.
---
## Self-Review Notes
**Spec coverage:**
| Requirement | Task |
|---|---|
| Return a loading page on bootstrap | Task 1 |
| Each refresh makes progress | Task 1 (the sync walks one page of entries per refresh, limited by Torn's API page size of 100) |
| Eventually complete and render real UI | Task 1 (when all users have rows, no more refresh needed) |
| End-to-end smoke check | Task 2 |
**Placeholder scan:** No "TODO", "TBD".
**Type consistency:** No new function signatures.
**Risks addressed:**
- PHP execution timeout during bootstrap: the loading page returns in <1s, no API calls happen during the initial render.
- Partial backfill on browser close: `ON CONFLICT DO UPDATE` makes the next refresh safe to resume.
- Cascading timeouts: each refresh does only the next page of work; no single refresh can exceed reasonable host limits.
@@ -0,0 +1,306 @@
# Revert to Per-User Balance via `/log` + Per-Entry Re-fetch — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Correctly compute per-user vault share. The `/money` endpoint returns the *total* vault across all users — wrong for per-user attribution. Revert `vaultLoop` to per-user SQL `MAX(running_balance)`. Replace the broken broadcast backfill with a true per-entry re-fetch that updates each row's `running_balance` from its own `data.balance`. Delete the unused `fetchLiveVaultBalance` helper and its test.
**Architecture:** `vaultLoop` returns `MAX(running_balance)` per user (where each row's `running_balance` comes from that entry's own `data.balance` in the v2 `/log` response). The schema's `running_balance` column is the source of truth. A new `refetchRunningBalances` helper re-fetches each user's full `/log` and updates every row's `running_balance` from its per-entry `data.balance`. The `fetchLiveVaultBalance` helper (introduced in the previous plan) is deleted along with its test, since it queried the wrong endpoint.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
## Global Constraints
- PHP 8.1+ syntax only.
- Database schema (`vault` table with `id TEXT PRIMARY KEY` and `running_balance INTEGER`) is unchanged.
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries.
- API authentication: `Authorization: ApiKey <key>` header.
- All helpers outside the rewritten ones stay byte-identical unless the task explicitly says otherwise.
- Commits per task. Do not push.
---
### Task 1: Revert `vaultLoop` to per-user SQL
**Files:**
- Modify: `includes/utilities.php` (`vaultLoop` only)
**Interfaces:**
- `vaultLoop($name=null)`: returns the per-user vault share (per the `running_balance` column). All-users case sums per-user MAXes.
- [ ] **Step 1: Replace `vaultLoop` with the SQL-based implementation**
In `includes/utilities.php`, replace the existing `vaultLoop` function (which currently calls `fetchLiveVaultBalance`) with:
```php
function vaultLoop ($name=null) {
$pdo = getDatabaseConnection();
if ($name === null) {
$stmt = $pdo->query(
'SELECT COALESCE(SUM(max_balance), 0) FROM '
. '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
);
return (int)$stmt->fetchColumn();
}
$stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $name);
$stmt->execute();
return (int)$stmt->fetchColumn();
}
```
This is the same SQL-based implementation from the previous balance-fix plan, before `/money` was added.
- [ ] **Step 2: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Run the existing golden-file test (no regression)**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
(The `live_balance_test.php` is expected to FAIL after this commit because `fetchLiveVaultBalance` is now unused but still defined. That's OK — we'll delete the test in Task 4. The golden-file test is the load-bearing one.)
- [ ] **Step 4: Commit**
```bash
git add includes/utilities.php
git commit -m "Revert vaultLoop to per-user SQL MAX(running_balance)"
```
---
### Task 2: Replace `backfillRunningBalances` with `refetchRunningBalances`
**Files:**
- Modify: `includes/utilities.php` (replace the existing function body)
**Interfaces:**
- `refetchRunningBalances(PDO $pdo, string $user): void`
- Counts rows for the user where `running_balance IS NULL`. If 0, returns.
- Otherwise, fetches the user's full `/log` history (paginating with `_metadata.links.next` until null), and for each entry runs an `UPDATE vault SET running_balance = X WHERE id = Y` to set the per-entry value from `data.balance`. Each fetched entry has its own historical balance, which is the correct semantics.
- Calls `executeApiCall` and `validateApiResponse` (existing helpers).
- On API failure, throws the underlying exception (caller catches via try/catch in `index.php`).
- [ ] **Step 1: Replace the function**
In `includes/utilities.php`, find the existing `backfillRunningBalances` function and replace it with:
```php
/**
* Re-fetch the running_balance for each entry of a user by walking the
* v2 /log paginated endpoint.
*
* For each entry the API returns, we run a single UPDATE setting that
* entry's `running_balance` to its `data.balance`. The pre-existing
* broadcast approach gave every row of a user the same value, which
* made the historical view misleading; this restores per-entry accuracy.
*
* If the user has zero rows with `running_balance IS NULL`, returns
* without making any HTTP call. On API failure, the underlying
* exception propagates to the caller's try/catch in index.php.
*
* @param PDO $pdo Database connection.
* @param string $user The user whose NULL rows should be refilled.
*
* @return void
*/
function refetchRunningBalances($pdo, $user) {
if (!array_key_exists($user, USER_KEYS)) {
throw new ApiKeyMissingException("User does not have an API key configured.");
}
$countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
$countStmt->bindValue(':user', $user);
$countStmt->execute();
if ((int)$countStmt->fetchColumn() === 0) {
return;
}
$updateStmt = $pdo->prepare(
'UPDATE vault SET running_balance = :balance WHERE id = :id'
);
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
do {
$responseData = executeApiCall($url, USER_KEYS[$user]);
validateApiResponse($responseData);
foreach ($responseData['log'] as $entry) {
$id = $entry['id'] ?? null;
if (!$id || !isset($entry['data']['balance'])) {
continue;
}
$updateStmt->bindValue(':balance', (int)$entry['data']['balance']);
$updateStmt->bindValue(':id', $id);
$updateStmt->execute();
}
$url = $responseData['_metadata']['links']['next'] ?? null;
} while ($url !== null);
}
```
- [ ] **Step 2: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Run the golden-file test (no regression)**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 4: Commit**
```bash
git add includes/utilities.php
git commit -m "Replace broadcast backfill with per-entry re-fetch for running_balance"
```
---
### Task 3: Wire `refetchRunningBalances` into entry points
**Files:**
- Modify: `functions.php` (`backfillUserLogs` and `syncUserLogs`)
**Interfaces:**
- Same as before, but the post-pagination call is now to `refetchRunningBalances` (not `backfillRunningBalances`).
- [ ] **Step 1: Update `backfillUserLogs`**
In `functions.php`, replace the last line of `backfillUserLogs` (`backfillRunningBalances($pdo, $user);`) with `refetchRunningBalances($pdo, $user);`.
- [ ] **Step 2: Update `syncUserLogs`**
Same edit in `syncUserLogs`.
- [ ] **Step 3: Verify the file parses**
Run: `php -l functions.php`
Expected: `No syntax errors detected`.
- [ ] **Step 4: Commit**
```bash
git add functions.php
git commit -m "Invoke refetchRunningBalances after sync/backfill"
```
---
### Task 4: Delete unused `fetchLiveVaultBalance` and its test
**Files:**
- Modify: `includes/utilities.php` (delete the helper function and the test-hook shim line at top of file)
- Delete: `tests/live_balance_test.php`
**Interfaces:**
- `fetchLiveVaultBalance` is removed entirely. Any callers (none after Task 1's revert) won't find it.
- The `$GLOBALS['liveBalanceTestHook']` shim line is removed.
- [ ] **Step 1: Delete the test hook shim**
In `includes/utilities.php`, near the top of the file (right after the doc-block and before `function dbNew`), find and delete the line:
```php
// Test hook for fetchLiveVaultBalance(). When set, this closure is called
// instead of executeApiCall. Set $GLOBALS['liveBalanceTestHook'] to a
// closure($url, $apiKey): array in test code. Production code leaves it null.
$GLOBALS['liveBalanceTestHook'] = $GLOBALS['liveBalanceTestHook'] ?? null;
```
- [ ] **Step 2: Delete the `fetchLiveVaultBalance` function**
Find and delete the entire `fetchLiveVaultBalance` function (including its docblock) at the end of `includes/utilities.php`.
- [ ] **Step 3: Delete the test file**
```bash
git rm tests/live_balance_test.php
```
- [ ] **Step 4: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 5: Run the golden-file test (no regression)**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 6: Verify the test file is gone**
Run: `ls tests/`
Expected: only `process_log_entries_test.php`.
- [ ] **Step 7: Commit**
```bash
git add includes/utilities.php tests/live_balance_test.php
git commit -m "Remove unused fetchLiveVaultBalance and its test"
```
---
### Task 5: End-to-end smoke check
**Files:** No code changes. Verification only.
- [ ] **Step 1: Run the test suite**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 2: Probe the test instance**
```
curl -s http://127.0.0.1:8001/ | grep -E "<h2>|<h3>|<h4>" | head -15
```
Expected: per-user balances come from `MAX(running_balance)` per user (which after the re-fetch will be the historical entry's actual `data.balance`, not the broadcast value). For Zarathos whose log last saw data.balance=$69,235,831, that's what shows. For Symos whose most recent entry has data.balance=$160,627,669 (or whatever the latest API returns), that's what shows. Both values reflect the per-entry historical balance, not the broadcast approximation.
- [ ] **Step 3: Verify no v1 references remain**
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -n "firstRun\|?key=\|fetchLiveVaultBalance" functions.php index.php includes/utilities.php includes/exceptions.php tests/
```
Expected: no matches (including no references to the deleted `fetchLiveVaultBalance`).
- [ ] **Step 4: Commit if any incidental fixes were needed**
If Step 2 or 3 produced any required fixes, commit them. Otherwise, no commit.
---
## Self-Review Notes
**Spec coverage:**
| Requirement | Task |
|---|---|
| Per-user balance via SQL `MAX(running_balance)` | Task 1 |
| Replace broadcast backfill with per-entry re-fetch | Task 2 |
| Wire re-fetch into entry points | Task 3 |
| Delete unused `fetchLiveVaultBalance` and its test | Task 4 |
| End-to-end smoke check | Task 5 |
**Placeholder scan:** No "TODO", "TBD", "implement later".
**Type consistency:** `refetchRunningBalances(PDO $pdo, string $user): void` matches the call sites in Task 3. The `vaultLoop` SQL matches the documented contract.
**Risks addressed:**
- The broadcast-backfill defect is fixed by per-entry re-fetch.
- Unused code is removed (dead-weight cleanup).
- The `vaultLoop` revert doesn't introduce regressions because the existing golden-file test still passes.
@@ -0,0 +1,145 @@
# Walk `prev` Chain on Backfill — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `backfillUserLogs` capture the user's complete vault transaction history (not just the most recent 100 entries) by walking the v2 API's `prev` chain in addition to `next`. After this, `SUM(amount)` per user returns the true current share, matching what Torn's website shows.
**Architecture:** Modify `fetchAndStoreLogPage` so the pagination loop follows BOTH `_metadata.links.next` (forward in time, used when `from` is given) and `_metadata.links.prev` (backward in time, used on the first/oldest page). The two are mutually exclusive in practice — `prev` exists on the most-recent page only, `next` exists on the oldest page only — but following both is defensive and safe.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
## Global Constraints
- PHP 8.1+ syntax only.
- Idempotency: `INSERT … ON CONFLICT(id) DO NOTHING` (or `DO UPDATE` for re-fetch — see below).
- API authentication: `Authorization: ApiKey <key>` header.
- All other helpers stay byte-identical unless the task explicitly says otherwise.
- Commits per task. Do not push.
---
### Task 1: Update `fetchAndStoreLogPage` to walk `prev`
**Files:**
- Modify: `includes/utilities.php` (`fetchAndStoreLogPage` only)
**Interfaces:**
- `fetchAndStoreLogPage($pdo, $user, $url)`: unchanged signature. The pagination loop additionally follows `_metadata.links.prev` to fetch older pages.
- [ ] **Step 1: Update the pagination loop**
In `includes/utilities.php`, find the `fetchAndStoreLogPage` function. Replace the entire function body with:
```php
function fetchAndStoreLogPage($pdo, $user, $url) {
if (!array_key_exists($user, USER_KEYS)) {
throw new ApiKeyMissingException("User does not have an API key configured.");
}
$insertStmt = $pdo->prepare(
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
. 'ON CONFLICT(id) DO UPDATE SET running_balance = EXCLUDED.running_balance, amount = EXCLUDED.amount'
);
do {
$responseData = executeApiCall($url, USER_KEYS[$user]);
validateApiResponse($responseData);
processLogEntries($responseData['log'], $user, $insertStmt);
// Follow the older-pages chain so the very first call (which gets the
// most recent page) also retrieves every older entry. After the prev
// chain is exhausted, fall back to the newer-pages chain in case the
// caller passed a `from=` URL and the result has a next link.
$url = $responseData['_metadata']['links']['prev']
?? $responseData['_metadata']['links']['next']
?? null;
} while ($url !== null);
}
```
The change from the previous version: the pagination cursor now prefers `prev` (older pages) over `next` (newer pages). Once the prev chain is exhausted, it falls back to next.
Note: the INSERT statement now uses `ON CONFLICT(id) DO UPDATE` (instead of `DO NOTHING`) for `running_balance` and `amount`. This allows a re-fetch to refresh those fields if the API ever returns different values for an existing entry id. This is necessary because the backfill may run more than once (e.g., after a re-fetch from a different starting URL).
- [ ] **Step 2: Verify the file parses**
Run: `php -l includes/utilities.php`
Expected: `No syntax errors detected`.
- [ ] **Step 3: Run the golden-file test (no regression)**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 4: Commit**
```bash
git add includes/utilities.php
git commit -m "fetchAndStoreLogPage: walk prev chain to fetch full vault history"
```
---
### Task 2: End-to-end smoke check
**Files:** No code changes.
- [ ] **Step 1: Run the test suite**
Run: `php tests/process_log_entries_test.php`
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
- [ ] **Step 2: Verify no v1 references and no stray references remain**
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php tests/
```
Expected: no matches (same as before).
- [ ] **Step 3: Verify the live DB state has Zarathos=$0 and Symos=$152,663,117**
The DB was manually backfilled earlier in the session to contain the correct values. Re-verify:
```bash
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
php -r "
\$db = new PDO('sqlite:data/vault.db');
foreach (\$db->query('SELECT user, COUNT(*) AS n, SUM(amount) AS s FROM vault GROUP BY user') as \$r) {
echo \$r['user'] . ': ' . \$r['n'] . ' entries, SUM(amount)=\$' . number_format(\$r['s']) . PHP_EOL;
}
"
```
Expected: Zarathos: 374 entries (or similar), SUM(amount)=$0. Symos: 2461 entries (or similar), SUM(amount)=$152,663,117. (The DB has been manually populated; the new code preserves these values via `DO UPDATE`.)
- [ ] **Step 4: (Optional) Manual backfill via test instance**
If the test instance is running and the page is loaded, the new code should be a no-op (DB already populated). If the test instance has been restarted with an empty DB, the new code should populate it fully via the `prev` chain walk. Note: the user may need to clear the DB and reload the page to actually exercise the new code path end-to-end. Mark as not-required for this PR if the DB is already correct.
- [ ] **Step 5: Commit if any incidental fixes were needed**
If Step 1, 2, 3, or 4 produced any required fixes, commit them. Otherwise, no commit.
---
## Self-Review Notes
**Spec coverage:**
| Requirement | Task |
|---|---|
| Follow `prev` chain in pagination | Task 1 |
| Idempotent re-fetch (re-running updates rather than no-ops) | Task 1 (`ON CONFLICT DO UPDATE`) |
| End-to-end smoke check | Task 2 |
**Placeholder scan:** No "TODO", "TBD", "implement later".
**Type consistency:** `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): void` signature unchanged.
**Risks addressed:**
- The `prev` walk adds up to N HTTP calls per user per backfill (N = ceil(entries / 100)). For Symos that's ~25 calls. Acceptable since backfill only runs once on first sync.
- `ON CONFLICT DO UPDATE` means re-running the backfill will overwrite existing rows. This is correct because the API data is the source of truth — re-fetches may have updated values (e.g., if Torn fixes a stale entry).
@@ -0,0 +1,196 @@
# Torn API v1 → v2 Migration Design
**Date:** 2026-08-03
**Project:** Torn Vault Tracker
**Status:** Approved — pending implementation
## Problem
The Torn City API has moved from v1 (`api.torn.com/user/`) to v2 (`api.torn.com/v2/user`). The two protocols return different JSON shapes and use different authentication:
| Concern | v1 (old) | v2 (new) |
|---|---|---|
| Endpoint | `https://api.torn.com/user/?selections=log&…&key=APIKEY` | `https://api.torn.com/v2/user?selections=log&…` + `Authorization: ApiKey KEY` |
| `log` shape | Object keyed by entry id | Flat array of entry objects |
| Entry id | `$entry['log']` (numeric event type — duplicated across entries) | `$entry['id']` (string, unique per entry) |
| `title` location | `$entry['title']` (root) | `$entry['details']['title']` |
| `timestamp` location | `$entry['timestamp']` (root) | `$entry['timestamp']` (root, unchanged) |
| Pagination | `&to=$ts`, `&from=$ts` | `_metadata.links.next` URL with `from`/`to`/`sort`/`limit` params |
The project was already partially migrated (`firstRun()` in `functions.php:72` calls the v2 endpoint) but still reads the v1 JSON shape and v1 authentication, so `firstRun()` does not work. `getLog()` still uses the v1 endpoint entirely. Both code paths need to be brought onto v2 — a single coherent rewrite — and the helpers in `includes/utilities.php` (which both rely on) need to match.
Sample files live at `backup/api-sample.json` (v1, 2 entries) and `backup/api-sample-new.json` (v2, 100 entries with `_metadata`).
## Goals
- A single v2 path that handles both bootstrap (full history) and steady-state (recent sync).
- Idempotent re-runs: re-fetching the same data does not produce duplicate rows.
- Existing users with a populated `vault.db` come through the migration without manual intervention — only an empty DB triggers the full bootstrap.
- New code is covered by a unit-style golden-file test that doesn't touch the network.
## Non-Goals
- Adding new API endpoints beyond `user/selections=log&log=5850,5851`.
- Changing the local DB schema (table shape is already v2-compatible thanks to `id TEXT PRIMARY KEY`).
- UI changes — `index.php`, `style.css`, `script.js` stay identical.
- README updates — content already describes current behavior accurately.
## Architecture
```
index.php
├── config.php
├── functions.php (high-level: backfillUserLogs, syncUserLogs, DB, UI helpers)
└── includes/
├── utilities.php (low-level: API + DB helpers; all v2-aware)
└── exceptions.php (unchanged)
```
The two-phase flow on each request:
1. **Bootstrap** (empty DB) → `backfillUserLogs($user)` paginates full history.
2. **Steady state** (DB populated) → `syncUserLogs($user)` fetches only newer entries.
Both call one shared `fetchAndStoreLogPage()` helper that performs a single page pull, stores rows idempotently, and returns either the next `_metadata.links.next` URL or `null`.
## Components and Responsibilities
### `includes/exceptions.php` — unchanged
Existing `ApiKeyMissingException`, `CurlErrorException`, `JsonDataException`, `ApiValidationException` cover all the failure modes we anticipate. Add one optional exception:
- `LogEntryIncompleteException` — raised when a fetched entry is missing `data.deposited` or `data.withdrawn` (defensive — shouldn't occur given we filter to `log=5850,5851`, but we shouldn't silently insert garbage if the API ever changes).
### `includes/utilities.php` — rewritten
| Function | Change |
|---|---|
| `executeApiCall(string $url, string $apiKey): array` | **Rewrite.** Drop `Content-Type: application/json`; add `Authorization: ApiKey <apiKey>`. Accepts a complete URL (so callers can pass through `_metadata.links.next`). If `$apiKey` is empty, throws `ApiKeyMissingException`. The `apiKey` is supplied by the caller (`fetchAndStoreLogPage` looks it up from `USER_KEYS[$user]`); this keeps `executeApiCall` agnostic about which user is being fetched. |
| `validateApiResponse(array)` | Tighten: `log` must exist and be an **array** (not an object-keyed map). |
| `processLogEntries(array $logEntries, string $user, …)` | **Rewrite for v2 shape.** Read `$entry['id']` for the PK, `$entry['timestamp']`, `$entry['details']['title']`, `$entry['data']['deposited']` (deposit) or `$entry['data']['withdrawn']` (withdraw). Use `INSERT … ON CONFLICT(id) DO NOTHING` so the operation is idempotent. Skip (and log) any entry whose `data` block lacks both fields rather than inserting bad data. |
| `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): ?string` | **New.** Composes `executeApiCall` + `validateApiResponse` + `processLogEntries` for a single page. Looks the API key up via `USER_KEYS[$user]`. Returns `$data['_metadata']['links']['next'] ?? null`. |
| `dbNew()`, `ensureUserHasApiKey()`, `fetchVaultRecords()`, `vaultLoop()`, `getSign()`, `consoleLog()` | Unchanged. |
### `functions.php` — rewritten entry points
| Function | Change |
|---|---|
| `getDatabaseConnection()` | Unchanged. |
| `backfillUserLogs(string $user): void` | **New** (replaces `firstRun`). Starts at `INITIAL_URL = "https://api.torn.com/v2/user?selections=log&log=5850,5851"`, paginates with `_metadata.links.next` until exhausted. |
| `syncUserLogs(string $user): void` | **Renamed** from `getLog`. Uses the v2 endpoint with `&from=<last_seen_ts+1>` query param (only if the user has rows in the DB — otherwise delegates to `backfillUserLogs`). Falls back to following `_metadata.links.next` if more than one page comes back. |
| `generateBalance()`, `getSpace()`, `buildTable()` | Unchanged — they read from the DB, schema is unchanged. |
### `index.php` — one-line swap
Replace `firstRun``backfillUserLogs` and `getLog``syncUserLogs`. Logic for "is DB empty?" already lives in `dbNew()`, so the control flow is identical.
### `config.php` — unchanged
`USER_KEYS` map keyed by user already provides both API keys; the migration uses them both.
## Data Flow
### Bootstrap (empty DB)
```
backfillUserLogs(user) {
url = "https://api.torn.com/v2/user?selections=log&log=5850,5851"
do {
next = fetchAndStoreLogPage($pdo, $user, $url)
url = next
} while (next !== null)
}
fetchAndStoreLogPage(pdo, user, url): ?string {
headers = ["Authorization: ApiKey " . USER_KEYS[user]]
data = executeApiCall(url, USER_KEYS[user]) // throws on curl/JSON failure
validateApiResponse(data) // throws on shape error
processLogEntries(data['log'], user, …) // INSERT ... ON CONFLICT DO NOTHING
return data['_metadata']['links']['next'] ?? null
}
```
### Steady state (DB populated)
```
syncUserLogs(user) {
if (dbNew()) return backfillUserLogs(user) // safety net
last = $pdo->query("SELECT MAX(timestamp) FROM vault WHERE user = ?")
from = (int)last + 1
url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=$from"
do {
next = fetchAndStoreLogPage($pdo, $user, $url)
url = next
} while (next !== null)
}
```
## Data Model
The `vault` table (defined in `functions.php:32-38`) already matches our needs:
```sql
CREATE TABLE IF NOT EXISTS vault (
id TEXT PRIMARY KEY, -- v2 entry id, e.g. "j8EzaeOu2lpWPcloqGYJ"
user TEXT NOT NULL,
timestamp INTEGER NOT NULL,
description TEXT NOT NULL,
amount REAL NOT NULL
);
```
Migration concern: existing rows were inserted with `$key` from the v1 object-keyed `log`, which means the current PK is the v1 alphanumeric object key (e.g. `"CTczoijHAKhnfAeYmagC"`). The v2 PK is the v2 unique id (e.g. `"j8EzaeOu2lpWPcloqGYJ"`). Format-wise both are strings, so the schema is compatible — **but** the values do not match and v2 inserts will collide on the PK only if the same id repeats, which it won't across the two schemes. Wiping `data/vault.db` on the first page load after deploy gives the cleanest result (recommended in install note). For users with an existing DB, the bootstrap path will simply keep fetching — v2 ids that already exist (by coincidence — vanishingly unlikely between v1 object keys and v2 entry ids) will be skipped via `ON CONFLICT(id) DO NOTHING`.
## Error Handling
| Failure | Behavior |
|---|---|
| Missing API key for a user | `ensureUserHasApiKey()` throws `ApiKeyMissingException` before any fetch. |
| `curl_exec` returns false | `executeApiCall` throws `CurlErrorException`. The bootstrap loop stops; index page renders whatever was stored so far. |
| `json_decode` fails | `executeApiCall` throws `JsonDataException`. Same as above. |
| Response missing `log` or wrong shape | `validateApiResponse` throws `ApiValidationException`. Loop stops. |
| Entry lacks `data.deposited` or `data.withdrawn` | `processLogEntries` raises `LogEntryIncompleteException` for that entry, skips it, continues with the rest. (Logged via the existing `consoleLog` and `debug.log` patterns.) |
| DB error | Handled by PDO exception mode; propagates. |
All exceptions are caught at the top of `index.php`'s bootstrap path so a single bad page doesn't take down the whole UI. The user sees whatever data is in the DB plus a non-fatal warning.
## Testing Strategy
A pure-PHP "golden file" test that does **not** hit the network:
1. Load `backup/api-sample-new.json`.
2. Mock `executeApiCall` to return the fixture (`[api-sample-new.json content]`).
3. Mock `processLogEntries`'s PDO statements (count `execute()` calls and capture params).
4. Assert: every entry produces exactly one `INSERT` with the expected `id`, `user`, `timestamp`, `description`, `amount`; that the loop terminates when `links.next` is null; and that a fixture with `links.next` set to a sentinel URL causes exactly one extra fetch.
This lives next to `phpcs.xml` (no test framework added; we just add `tests/process_log_entries_test.php` and document how to run it via `php tests/process_log_entries_test.php`). The test is run manually before each deploy.
We do **not** add live API smoke tests — they require real keys and quota. The existing `dbNew()` path on `index.php` is the only "live" verification, and it runs only on the user's first page-load.
## Files Touched
| File | Action |
|---|---|
| `functions.php` | Rewrite the two log-fetch entry points; keep DB + UI helpers. |
| `includes/utilities.php` | Rewrite `executeApiCall`, `validateApiResponse`, `processLogEntries`; add `fetchAndStoreLogPage`. |
| `includes/exceptions.php` | Add `LogEntryIncompleteException`. |
| `index.php` | Rename calls: `firstRun``backfillUserLogs`, `getLog``syncUserLogs`. |
| `tests/process_log_entries_test.php` | **New.** Golden-file unit test for v2 parsing. |
| `README.md` | Optional: note that `data/vault.db` will be re-populated on first page-load post-deploy. |
| `config.php`, `style.css`, `script.js`, `backup/` | **Untouched.** |
## Open Questions
None at design time. All clarifications resolved during brainstorming.
## Acceptance Criteria
1. `getLog`/`syncUserLogs` and `firstRun`/`backfillUserLogs` both call `https://api.torn.com/v2/user` with `Authorization: ApiKey …` header.
2. `processLogEntries` reads `id`, `timestamp`, `details.title`, and `data.deposited`/`data.withdrawn` from each entry; ignores any other shape.
3. Pagination follows `_metadata.links.next` until null.
4. Re-running on the same data produces no duplicate rows.
5. `tests/process_log_entries_test.php` passes against `backup/api-sample-new.json`.
6. `index.php`, `style.css`, `script.js`, `includes/exceptions.php` (existing classes), `config.php` behave identically to before for end users — except that the data they fetch is now the v2 shape.
+62 -78
View File
@@ -30,11 +30,12 @@ function getDatabaseConnection() {
// SQL to create the vault table if it doesn't exist
$createTableSQL = "CREATE TABLE IF NOT EXISTS vault (
id TEXT PRIMARY KEY,
user TEXT NOT NULL,
timestamp INTEGER NOT NULL,
description TEXT NOT NULL,
amount REAL NOT NULL
id TEXT PRIMARY KEY,
user TEXT NOT NULL,
timestamp INTEGER NOT NULL,
description TEXT NOT NULL,
amount REAL NOT NULL,
running_balance INTEGER
);";
if ($pdo === null) {
@@ -43,6 +44,13 @@ function getDatabaseConnection() {
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->exec($createTableSQL);
// Add the running_balance column for databases created before the
// column was added. Idempotent: skip if it already exists.
$columns = $pdo->query("PRAGMA table_info(vault)")->fetchAll(PDO::FETCH_COLUMN, 1);
if (!in_array('running_balance', $columns, true)) {
$pdo->exec('ALTER TABLE vault ADD COLUMN running_balance INTEGER');
}
} catch (PDOException $e) {
die("Database connection failed: " . $e->getMessage());
}
@@ -52,102 +60,73 @@ function getDatabaseConnection() {
}
/**
* Pulls and stores all vault transaction logs for a user from the Torn API.
* Pulls and stores the full vault transaction log history for a user.
*
* This function retrieves all transaction logs related to vault deposits and
* withdrawals for a specified user. It continues fetching logs until no more
* entries are available, and stores each entry in the database. Each log entry
* is uniquely identified by its ID and includes details such as timestamp,
* description, and amount. The function requires the user's API key to
* authenticate requests to the Torn API.
* Pages through the v2 Torn API using `_metadata.links.next` until the
* API reports no further pages. Intended for first-run use when the
* local database is empty. Idempotent: re-running on a partially
* populated database inserts only new entries (ON CONFLICT DO NOTHING).
*
* @param string $user The user whose transaction logs are to be retrieved and stored.
*
* @throws ApiKeyMissingException If the user does not have an API key configured.
* @throws CurlErrorException If there is an error during the API call.
* @throws Exception If there is an error decoding the JSON response or if the response is invalid.
* @param string $user The user whose logs should be fetched.
*
* @return void
*/
function firstRun($user) {
$apiKey = USER_KEYS[$user];
$pdo = getDatabaseConnection();
$to = time();
function backfillUserLogs($user) {
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
$pdo = getDatabaseConnection();
do {
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&to=$to";
$ch = curl_init($url);
$next = fetchAndStoreLogPage($pdo, $user, $url);
$url = $next;
} while ($next !== null);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt(
$ch, CURLOPT_HTTPHEADER, [
'accept: application/json',
"Authorization: ApiKey $apiKey"
]
);
$response = curl_exec($ch);
curl_close($ch);
$data = json_decode($response, true);
if (empty($data['log'])) {
break;
}
foreach ($data['log'] as $key => $entry) {
$description = $entry['title'];
$timestamp = $entry['timestamp'];
$amount = isset($entry['data']['deposited']) ? $entry['data']['deposited'] : -$entry['data']['withdrawn'];
$stmt = $pdo->prepare('INSERT INTO vault (ID, user, timestamp, description, amount) VALUES (:id, :user, :timestamp, :description, :amount)');
$stmt->bindValue(':id', $key);
$stmt->bindValue(':user', $user);
$stmt->bindValue(':timestamp', $timestamp);
$stmt->bindValue(':description', $description);
$stmt->bindValue(':amount', $amount);
$stmt->execute();
}
$to = end($data['log'])['timestamp'];
} while (true);
refetchRunningBalances($pdo, $user);
}
/**
* Retrieves the user's log entries from the Torn API
* Synchronizes recent vault transactions for a user.
*
* @param string $user The user to retrieve the log for
* @param boolean $debug Whether to output debug information (default: false)
* On a non-empty database, fetches only entries newer than the user's
* most recent row. Falls back to full backfill when the database is
* empty.
*
* @throws Exception If the user does not have an API key configured,
* if the API call fails, or if the log data is invalid
* @param string $user The user whose logs should be synced.
*
* @return void
*/
function getLog($user, $debug = false) {
function syncUserLogs($user) {
if (dbNew()) {
backfillUserLogs($user);
return;
}
$pdo = getDatabaseConnection();
ensureUserHasApiKey($user);
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $user);
$stmt->execute();
$userRowCount = (int)$stmt->fetchColumn();
$stmt = $pdo->query("SELECT MAX(timestamp) AS max_timestamp FROM vault WHERE user = '$user'");
$row = $stmt->fetch(PDO::FETCH_ASSOC);
// If the user has zero rows (not the global dbNew, but per-user empty),
// delegate to backfillUserLogs which walks the full prev chain.
if ($userRowCount === 0) {
backfillUserLogs($user);
return;
}
$from = $row['max_timestamp'];
$to = time();
$stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
$stmt->bindValue(':user', $user);
$stmt->execute();
$lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];
$apiKey = USER_KEYS[$user];
$apiEndpoint = "https://api.torn.com/user/?selections=log&log=5850,5851&to=$to&from=$from&key=$apiKey";
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);
$responseData = executeApiCall($apiEndpoint);
validateApiResponse($responseData);
do {
$next = fetchAndStoreLogPage($pdo, $user, $url);
$url = $next;
} while ($next !== null);
$checkStmt = prepareCheckStatement($pdo);
$insertStmt = prepareInsertStatement($pdo);
processLogEntries($responseData['log'], $user, $checkStmt, $insertStmt, $debug);
refetchRunningBalances($pdo, $user);
}
/**
@@ -216,11 +195,16 @@ function buildTable () {
$amount = number_format($amount, 0); // Format the amount as an integer
$runningBalance = isset($entry['running_balance']) && $entry['running_balance'] !== null
? '$' . number_format((int)$entry['running_balance'], 0)
: '—';
echo '<tr class="'.$class.'">';
echo '<td>'.$user.'</td>';
echo '<td>'.date("F j, Y / H:i", $timestamp).'</td>';
echo '<td>'.$description.'</td>';
echo '<td>'.$sign.$amount.'</td>';
echo '<td>'.$runningBalance.'</td>';
echo '</tr>';
}
}
+22
View File
@@ -98,3 +98,25 @@ class ApiValidationException extends Exception {
parent::__construct($message, $code, $previous);
}
}
/**
* Exception thrown when a log entry is missing required data fields.
*
* @category Exception
* @package TornVaultTracker
* @author Keith Solomon <ksolomon@gmail.com>
* @license Unlicense https://unlicense.org/
* @link https://github.com/ksolomon/Torn-Vault-Tracker
*/
class LogEntryIncompleteException extends Exception {
/**
* Constructor.
*
* @param string $message The exception message.
* @param int $code The exception code.
* @param Throwable $previous The previous throwable.
*/
public function __construct($message, $code = 0, Throwable $previous = null) {
parent::__construct($message, $code, $previous);
}
}
+180 -75
View File
@@ -48,14 +48,25 @@ function ensureUserHasApiKey($user) {
/**
* Executes a GET request to the given API endpoint and returns the JSON response as an associative array.
*
* @param string $apiEndpoint The URL of the API endpoint to call
* Uses the v2 Torn API authentication scheme by sending the API key in an
* `Authorization: ApiKey <key>` header. The key is never transmitted as a
* query parameter.
*
* @throws Exception If the API call fails or if the response is invalid JSON
* @param string $apiEndpoint The URL of the API endpoint to call
* @param string $apiKey The v2 API key to authenticate with
*
* @throws ApiKeyMissingException If the API key is empty
* @throws CurlErrorException If the cURL call fails
* @throws JsonDataException If the response body cannot be decoded as JSON
*
* @return array The JSON response from the API
*/
function executeApiCall($apiEndpoint) {
$headers = ["Content-Type: application/json"];
function executeApiCall($apiEndpoint, $apiKey) {
if (empty($apiKey)) {
throw new ApiKeyMissingException('API key is required for executeApiCall.');
}
$headers = ["Authorization: ApiKey $apiKey"];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
@@ -89,86 +100,62 @@ function executeApiCall($apiEndpoint) {
*/
function validateApiResponse($responseData) {
if (!isset($responseData['log']) || !is_array($responseData['log'])) {
throw new ApiValidationException("Invalid log data received from the API.");
throw new ApiValidationException('Invalid log data received from the API.');
}
}
/**
* Prepare a PDO statement for checking if a given vault transaction already exists in the database.
* Process an array of log entries retrieved from the Torn v2 API.
*
* @param PDO $pdo The PDO instance to prepare the statement with
* For each entry, extracts the v2 shape (`id`, `timestamp`,
* `details.title`, `data.deposited` | `data.withdrawn`) and inserts it
* via the prepared statement, which uses `ON CONFLICT(id) DO NOTHING`
* for idempotency.
*
* @return PDOStatement The prepared statement
*/
function prepareCheckStatement($pdo) {
$checkQuery = "SELECT COUNT(*) FROM vault WHERE user = :user AND timestamp = :timestamp AND amount = :amount";
return $pdo->prepare($checkQuery);
}
/**
* Prepare a PDO statement for inserting a new vault transaction into the database.
*
* @param PDO $pdo The PDO instance to prepare the statement with
*
* @return PDOStatement The prepared statement
*/
function prepareInsertStatement($pdo) {
$insertQuery = "INSERT INTO vault (ID, user, timestamp, description, amount) VALUES (:id, :user, :timestamp, :description, :amount)";
return $pdo->prepare($insertQuery);
}
/**
* Process an array of log entries retrieved from the Torn API.
*
* Goes through each log entry and checks if it's a vault deposit or withdrawal.
* If it is, it checks if the entry already exists in the database. If it
* doesn't, it inserts the entry into the database.
*
* @param array $logEntries The array of log entries to process
* @param array $logEntries The array of v2 log entries to process
* @param string $user The user whose log entries are being processed
* @param PDOStatement $checkStmt A prepared statement to check if an entry
* already exists in the database
* @param PDOStatement $insertStmt A prepared statement to insert a new entry
* into the database
* @param boolean $debug Whether to output debug information (default: false)
* @param PDOStatement $insertStmt A prepared statement for the idempotent
* insert (`INSERT … ON CONFLICT(id) DO NOTHING`)
*
* @return void
*/
function processLogEntries($logEntries, $user, $checkStmt, $insertStmt, $debug) {
foreach ($logEntries as $key =>$entry) {
if ($debug) {
$logMessage = "Raw entry:\n" . print_r($entry, true);
file_put_contents(__DIR__ . '/debug.log', $logMessage, FILE_APPEND);
function processLogEntries($logEntries, $user, $insertStmt) {
foreach ($logEntries as $entry) {
$id = $entry['id'] ?? null;
if (!$id) {
consoleLog('Skipping entry with no id: ' . print_r($entry, true));
continue;
}
$timestamp = $entry['timestamp'];
$description = $entry['title'];
$amount = $entry['log'] === 5850 ? $entry['data']['deposited'] : -$entry['data']['withdrawn'];
$timestamp = $entry['timestamp'] ?? null;
$description = $entry['details']['title'] ?? null;
$hasDeposit = isset($entry['data']['deposited']);
$hasWithdraw = isset($entry['data']['withdrawn']);
if ($debug) {
$logMessage = "Vault entry:\n\tUser: $user,\n\tTimestamp: $timestamp,\n\tDescription: $description,\n\tAmount: $amount\n";
file_put_contents(__DIR__ . '/debug.log', $logMessage, FILE_APPEND);
if ($hasDeposit) {
$amount = (int)$entry['data']['deposited'];
} elseif ($hasWithdraw) {
$amount = -((int)$entry['data']['withdrawn']);
} else {
consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
continue;
}
$checkStmt->execute(
[
':user' => $user,
':timestamp' => $timestamp,
':amount' => $amount
]
);
if ($checkStmt->fetchColumn() == 0) {
$insertStmt->execute(
[
':id' => $key,
':user' => $user,
':timestamp' => $timestamp,
':description' => $description,
':amount' => $amount
]
);
if ($timestamp === null || $description === null) {
consoleLog('Skipping entry ' . $id . ' missing timestamp or details.title: ' . print_r($entry, true));
continue;
}
$runningBalance = isset($entry['data']['balance']) ? (int)$entry['data']['balance'] : null;
$insertStmt->bindValue(':id', $id);
$insertStmt->bindValue(':user', $user);
$insertStmt->bindValue(':timestamp', $timestamp);
$insertStmt->bindValue(':description', $description);
$insertStmt->bindValue(':amount', $amount);
$insertStmt->bindValue(':running_balance', $runningBalance);
$insertStmt->execute();
}
}
@@ -185,10 +172,10 @@ function fetchVaultRecords($user = null) {
$pdo = getDatabaseConnection();
if ($user) {
$query = "SELECT * FROM vault WHERE user = :user ORDER BY timestamp DESC";
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault WHERE user = :user ORDER BY timestamp DESC";
$params = [':user' => $user];
} else {
$query = "SELECT * FROM vault ORDER BY timestamp DESC";
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC";
$params = [];
}
@@ -213,14 +200,19 @@ function fetchVaultRecords($user = null) {
* @return int The total balance of the given user (or all users if no user is given).
*/
function vaultLoop ($name=null) {
$vault = fetchVaultRecords($name);
$balance = 0;
$pdo = getDatabaseConnection();
foreach ($vault as $entry) {
$balance += $entry['amount'];
if ($name === null) {
$stmt = $pdo->query('SELECT COALESCE(SUM(amount), 0) FROM vault');
return (int)$stmt->fetchColumn();
}
return $balance;
$stmt = $pdo->prepare('SELECT COALESCE(SUM(amount), 0) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $name);
$stmt->execute();
return (int)$stmt->fetchColumn();
}
/**
@@ -238,3 +230,116 @@ function getSign($number) {
return '$';
}
/**
* Fetch a single page of log entries from the Torn v2 API and store them.
*
* Performs one HTTP request, validates the response, inserts each vault
* log entry (idempotently via INSERT ... ON CONFLICT), and returns the
* pagination cursor (`_metadata.links.next`) if more pages remain.
*
* @param PDO $pdo Database connection used to insert vault entries.
* @param string $user The user whose log entries are being fetched.
* @param string $url Full URL for the v2 API request.
*
* @throws ApiKeyMissingException If no API key is configured for the user.
* @throws CurlErrorException If the HTTP request fails.
* @throws JsonDataException If the response body is not valid JSON.
* @throws ApiValidationException If the response is missing the `log` array.
*
* @return string|null The `_metadata.links.next` URL, or null when there are no more pages.
*/
function fetchAndStoreLogPage($pdo, $user, $url) {
if (!array_key_exists($user, USER_KEYS)) {
throw new ApiKeyMissingException("User does not have an API key configured.");
}
$insertStmt = $pdo->prepare(
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
. 'ON CONFLICT(id) DO UPDATE SET running_balance = EXCLUDED.running_balance, amount = EXCLUDED.amount'
);
do {
$responseData = executeApiCall($url, USER_KEYS[$user]);
validateApiResponse($responseData);
processLogEntries($responseData['log'], $user, $insertStmt);
// Follow the older-pages chain so the very first call (which gets the
// most recent page) also retrieves every older entry. After the prev
// chain is exhausted, fall back to the newer-pages chain in case the
// caller passed a `from=` URL and the result has a next link.
$url = $responseData['_metadata']['links']['prev']
?? $responseData['_metadata']['links']['next']
?? null;
} while ($url !== null);
}
/**
* Print a variable to the console for debugging purposes.
*
* @param mixed $data The data to print to the console.
*
* @return void
*/
function consoleLog( $data ) {
echo '<script>';
echo 'console.log(' . json_encode($data) . ')';
echo '</script>';
}
/**
* Re-fetch the running_balance for each entry of a user by walking the
* v2 /log paginated endpoint.
*
* For each entry the API returns, we run a single UPDATE setting that
* entry's `running_balance` to its `data.balance`. The pre-existing
* broadcast approach gave every row of a user the same value, which
* made the historical view misleading; this restores per-entry accuracy.
*
* If the user has zero rows with `running_balance IS NULL`, returns
* without making any HTTP call. On API failure, the underlying
* exception propagates to the caller's try/catch in index.php.
*
* @param PDO $pdo Database connection.
* @param string $user The user whose NULL rows should be refilled.
*
* @return void
*/
function refetchRunningBalances($pdo, $user) {
if (!array_key_exists($user, USER_KEYS)) {
throw new ApiKeyMissingException("User does not have an API key configured.");
}
$countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
$countStmt->bindValue(':user', $user);
$countStmt->execute();
if ((int)$countStmt->fetchColumn() === 0) {
return;
}
$updateStmt = $pdo->prepare(
'UPDATE vault SET running_balance = :balance WHERE id = :id'
);
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
do {
$responseData = executeApiCall($url, USER_KEYS[$user]);
validateApiResponse($responseData);
foreach ($responseData['log'] as $entry) {
$id = $entry['id'] ?? null;
if (!$id || !isset($entry['data']['balance'])) {
continue;
}
$updateStmt->bindValue(':balance', (int)$entry['data']['balance']);
$updateStmt->bindValue(':id', $id);
$updateStmt->execute();
}
$url = $responseData['_metadata']['links']['next'] ?? null;
} while ($url !== null);
}
+46 -7
View File
@@ -13,15 +13,49 @@
require_once __DIR__ . '/config.php';
require_once __DIR__ . '/functions.php';
if (dbNew()) {
$bootstrapWarning = null;
$needsBackfill = false;
try {
foreach (USER_KEYS as $key => $value) {
firstRun($key);
}
header('Location: /');
} else {
foreach (USER_KEYS as $key => $value) {
getLog($key);
syncUserLogs($key);
// If the sync didn't populate this user, backfill is still needed.
$pdo = getDatabaseConnection();
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $key);
$stmt->execute();
if ((int)$stmt->fetchColumn() === 0) {
$needsBackfill = true;
}
}
} catch (Exception $e) {
$bootstrapWarning = $e->getMessage();
}
if ($needsBackfill) {
// Heavy work remains. Return a loading page that auto-refreshes;
// each refresh does another page of API fetches.
?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Torn Vault Tracker &mdash; loading</title>
<meta http-equiv="refresh" content="3">
<link rel="icon" type="image/png" href="favicon.png">
<link rel="stylesheet" href="style.css?v=<?php echo filemtime('style.css'); ?>">
</head>
<body>
<h1>Loading vault history&hellip;</h1>
<?php if ($bootstrapWarning !== null): ?>
<p>Warning: <?php echo htmlspecialchars($bootstrapWarning, ENT_QUOTES, 'UTF-8'); ?></p>
<?php endif; ?>
<p>One-time backfill in progress. The Torn API returns up to 100 vault entries per call;
we'll keep fetching until your full history is loaded. This page will refresh automatically.</p>
</body>
</html>
<?php
exit;
}
?>
@@ -45,6 +79,10 @@ if (dbNew()) {
<body>
<h1>Torn Vault Tracker</h1>
<?php if ($bootstrapWarning !== null): ?>
<div class="warning" style="background:#f77e82;color:#000;padding:.5em 1em;margin:1em 0;border:1px solid #333;">Warning: failed to fetch latest logs from Torn — showing cached data. (<?php echo htmlspecialchars($bootstrapWarning, ENT_QUOTES, 'UTF-8'); ?>)</div>
<?php endif; ?>
<div class="grid">
<div class="gridLeft">
<h1>Transactions</h1>
@@ -56,6 +94,7 @@ if (dbNew()) {
<th><h4>Date / Time (TCT)</h4></th>
<th><h4>Operation</h4></th>
<th><h4>Amount</h4></th>
<th><h4>Balance After</h4></th>
</tr>
</thead>
+130
View File
@@ -0,0 +1,130 @@
<?php
/**
* Golden-file test for processLogEntries() v2 behavior.
*
* Loads backup/api-sample-new.json, runs each entry through the v2
* processor against an in-memory SQLite database, and asserts the
* resulting rows match the expected (id, user, timestamp, description,
* amount) tuples derived from the fixture.
*
* PHP version 8.1+
*
* Run: php tests/process_log_entries_test.php
* Exit code 0 = pass.
*
* @category Test
* @package TornVaultTracker
* @author Keith Solomon <ksolo@gmail.com>
* @license Unlicense https://unlicense.org/
* @link https://github.com/ksolo/Torn-Vault-Tracker
*/
declare(strict_types=1);
// Make USER_KEYS available without including config.php (it would force a
// real DB connection).
if (!defined('USER_KEYS')) {
define(
'USER_KEYS', [
'zarathos' => 'test-key-not-used',
'symos' => 'test-key-not-used',
]
);
}
require_once __DIR__ . '/../includes/exceptions.php';
require_once __DIR__ . '/../includes/utilities.php';
/**
* Print a failure message to STDERR and exit with status 1.
*
* @param string $message Human-readable failure description.
*
* @return never
*/
function fail(string $message): void {
fwrite(STDERR, "FAIL: $message\n");
exit(1);
}
/**
* Lightweight assertion helper that mirrors PHPUnit's assertSame().
*
* @param mixed $expected The expected value.
* @param mixed $actual The actual value.
* @param string $label Human-readable label for failure messages.
*
* @return void
*/
function assertSame($expected, $actual, string $label): void {
if ($expected !== $actual) {
$exp = var_export($expected, true);
$act = var_export($actual, true);
fail("$label: expected $exp, got $act");
}
}
// In-memory DB so we don't touch the user's vault.db.
$pdo = new PDO('sqlite::memory:');
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->exec(
'CREATE TABLE vault (
id TEXT PRIMARY KEY,
user TEXT NOT NULL,
timestamp INTEGER NOT NULL,
description TEXT NOT NULL,
amount REAL NOT NULL,
running_balance INTEGER
)'
);
$insertStmt = $pdo->prepare(
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
. 'ON CONFLICT(id) DO NOTHING'
);
// Load the golden file.
$fixturePath = __DIR__ . '/../backup/api-sample-new.json';
if (!is_readable($fixturePath)) {
fail("Fixture not readable at $fixturePath");
}
$fixture = json_decode(file_get_contents($fixturePath), true);
if (!is_array($fixture) || !isset($fixture['log'])) {
fail('Fixture is missing the log array.');
}
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
// 100 entries in the fixture.
$count = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count, 'row count');
// Spot-check first entry (newest, withdraw) and last (oldest, deposit).
$rows = $pdo->query('SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC')->fetchAll(PDO::FETCH_ASSOC);
$first = $rows[0];
assertSame('c1pEDQ7jl3kuBHV4NqDI', $first['id'], 'first.id');
assertSame('zarathos', $first['user'], 'first.user');
assertSame(1785325489, (int)$first['timestamp'], 'first.timestamp');
assertSame('Vault withdraw', $first['description'], 'first.description');
assertSame(-498870964, (int)$first['amount'], 'first.amount');
$last = $rows[count($rows) - 1];
assertSame('8m5jrUKTiKk0SB4MxzIy', $last['id'], 'last.id');
assertSame(1714350415, (int)$last['timestamp'], 'last.timestamp');
assertSame('Vault deposit', $last['description'], 'last.description');
assertSame(231492, (int)$last['amount'], 'last.amount');
// Re-running with the same fixture inserts zero new rows (idempotency).
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
$count2 = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
assertSame(100, $count2, 'row count after re-run');
// round-trip the running_balance column from $entry['data']['balance']
$firstRunning = (int)$first['running_balance'];
$lastRunning = (int)$last['running_balance'];
assertSame(69235831, $firstRunning, 'first.running_balance');
assertSame(509622821, $lastRunning, 'last.running_balance');
echo "OK: processLogEntries v2 golden-file test passed (100 entries).\n";