Compare commits
42
Commits
2811fc83e5
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4339958b5e | ||
|
|
bd120608d7 | ||
|
|
6b7f37963c | ||
|
|
a9a635e172 | ||
|
|
a6efb7d63d | ||
|
|
a19058c6e9 | ||
|
|
ee4e24aef0 | ||
|
|
a7f35e79e2 | ||
|
|
dd820f6c8a | ||
|
|
63c6cc6972 | ||
|
|
5b8abd8bbc | ||
|
|
3c798cee88 | ||
|
|
d1cb4196dd | ||
|
|
006520b4c0 | ||
|
|
728f7bb4a5 | ||
|
|
abb353a939 | ||
|
|
61fb15336e | ||
|
|
0613d00fbb | ||
|
|
64e320c6a5 | ||
|
|
5fe4bf9f58 | ||
|
|
f1cd0d53b1 | ||
|
|
a81554a125 | ||
|
|
a9e443eeb2 | ||
|
|
214891d957 | ||
|
|
af7a233272 | ||
|
|
2e1bd050e1 | ||
|
|
908e2c50ab | ||
|
|
a133a5a49d | ||
|
|
73360f6805 | ||
|
|
89ecaaa6c7 | ||
|
|
9fa5247445 | ||
|
|
265e1841f0 | ||
|
|
d6aef3da42 | ||
|
|
cb584e9d3a | ||
|
|
2a55cc11ee | ||
|
|
9a4a7291cc | ||
|
|
07a8bba1d5 | ||
|
|
3ebc5e2850 | ||
|
|
e25a069e00 | ||
|
|
93b934c5cc | ||
|
|
e7ec343be1 | ||
|
|
c2b138d687 |
@@ -0,0 +1,604 @@
|
|||||||
|
# Torn API v1 → v2 Migration Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Rewrite Torn Vault Tracker's log-ingestion code to consume the new v2 Torn API JSON shape (array-of-entries with `_metadata.links.next` pagination and `Authorization: ApiKey` header), with a single shared page-fetch helper and a golden-file unit test.
|
||||||
|
|
||||||
|
**Architecture:** All v2-aware behavior lives in `includes/utilities.php` (`executeApiCall`, `validateApiResponse`, `processLogEntries`, plus a new `fetchAndStoreLogPage` helper). `functions.php` exposes two thin entry points — `backfillUserLogs` (full-history pagination on empty DB) and `syncUserLogs` (incremental fetch from last seen timestamp). `index.php` swaps its call sites. The only new behavior is in the helper module; everything else is either renamed or a one-line swap.
|
||||||
|
|
||||||
|
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies. No framework. Pure CLI test runner (`php tests/process_log_entries_test.php`) using the existing `backup/api-sample-new.json` fixture.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- PHP 8.1+ syntax only.
|
||||||
|
- Database schema (`vault` table with `id TEXT PRIMARY KEY`) is **not** changed.
|
||||||
|
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries — operation must be idempotent across re-runs.
|
||||||
|
- API authentication: `Authorization: ApiKey <key>` header; **never** pass the key as a `?key=` query parameter.
|
||||||
|
- Pagination: follow `_metadata.links.next` URL until null. Never compute timestamps manually for pagination; v2 provides the cursor.
|
||||||
|
- Use the fixture at `backup/api-sample-new.json` (100 entries) for the golden-file test.
|
||||||
|
- All existing helpers outside the rewritten ones stay byte-identical (no drive-by refactors).
|
||||||
|
- Commit after each task. Do not push.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Add `LogEntryIncompleteException`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/exceptions.php` (append new class after the existing four)
|
||||||
|
- No test file — exercised via Task 5's golden test.
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: none.
|
||||||
|
- Produces: a new exception class `LogEntryIncompleteException extends Exception`, with the same constructor signature as the existing four exception classes in the file.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Append the new class to `includes/exceptions.php`**
|
||||||
|
|
||||||
|
Append after the existing `ApiValidationException` class, following the same doc-comment + constructor pattern used by the other four classes in the file:
|
||||||
|
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* Exception thrown when a log entry is missing required data fields.
|
||||||
|
*
|
||||||
|
* @category Exception
|
||||||
|
* @package TornVaultTracker
|
||||||
|
* @author Keith Solomon <ksolomon@gmail.com>
|
||||||
|
* @license Unlicense https://unlicense.org/
|
||||||
|
* @link https://github.com/ksolomon/Torn-Vault-Tracker
|
||||||
|
*/
|
||||||
|
class LogEntryIncompleteException extends Exception {
|
||||||
|
/**
|
||||||
|
* Constructor.
|
||||||
|
*
|
||||||
|
* @param string $message The exception message.
|
||||||
|
* @param int $code The exception code.
|
||||||
|
* @param Throwable $previous The previous throwable.
|
||||||
|
*/
|
||||||
|
public function __construct($message, $code = 0, Throwable $previous = null) {
|
||||||
|
parent::__construct($message, $code, $previous);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify file still parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/exceptions.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/exceptions.php
|
||||||
|
git commit -m "Add LogEntryIncompleteException for malformed v2 entries"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Rewrite `executeApiCall` for v2 authentication
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (replace the existing `executeApiCall` function body)
|
||||||
|
- No new test file for this task alone — exercised by Task 5's golden test and by the integration in Task 4.
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `$apiEndpoint` (full URL), `$apiKey` (string). Uses `\Exception` (existing) for `CurlErrorException` and `JsonDataException` from `includes/exceptions.php`.
|
||||||
|
- Produces: `array` — the decoded JSON response, or throws `CurlErrorException` / `JsonDataException`. Header is `Authorization: ApiKey <apiKey>` only (no `Content-Type`).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `executeApiCall` in `includes/utilities.php`**
|
||||||
|
|
||||||
|
Replace the existing function (currently takes only `$apiEndpoint`) with this v2-aware version:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function executeApiCall($apiEndpoint, $apiKey) {
|
||||||
|
if (empty($apiKey)) {
|
||||||
|
throw new ApiKeyMissingException('API key is required for executeApiCall.');
|
||||||
|
}
|
||||||
|
|
||||||
|
$headers = ["Authorization: ApiKey $apiKey"];
|
||||||
|
$ch = curl_init();
|
||||||
|
curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
|
||||||
|
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
|
||||||
|
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
|
||||||
|
|
||||||
|
$response = curl_exec($ch);
|
||||||
|
|
||||||
|
if (curl_errno($ch)) {
|
||||||
|
throw new CurlErrorException('cURL error: ' . curl_error($ch));
|
||||||
|
}
|
||||||
|
|
||||||
|
curl_close($ch);
|
||||||
|
|
||||||
|
$responseData = json_decode($response, true);
|
||||||
|
|
||||||
|
if (json_last_error() !== JSON_ERROR_NONE) {
|
||||||
|
throw new JsonDataException('Failed to decode JSON response: ' . json_last_error_msg());
|
||||||
|
}
|
||||||
|
|
||||||
|
return $responseData;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify file still parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Rewrite executeApiCall for v2 API (Authorization header)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Tighten `validateApiResponse` for v2 shape
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (replace the existing `validateApiResponse` function body)
|
||||||
|
- No test file for this task alone — exercised by Task 5's golden test.
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `$responseData` (array). Throws `ApiValidationException` (already defined).
|
||||||
|
- Produces: void. `log` must exist and be an array.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `validateApiResponse` in `includes/utilities.php`**
|
||||||
|
|
||||||
|
Replace the existing function with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function validateApiResponse($responseData) {
|
||||||
|
if (!isset($responseData['log']) || !is_array($responseData['log'])) {
|
||||||
|
throw new ApiValidationException('Invalid log data received from the API.');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
(The change from the old version is tightening `is_array($responseData['log'])` — v2 always returns an array; v1 returned an object-keyed map, which `is_array()` rejects.)
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify file still parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Tighten validateApiResponse for v2 array shape"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: Add `fetchAndStoreLogPage` helper
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (append new function after `processLogEntries`)
|
||||||
|
- No test file for this task alone — exercised by Task 5 and the integration tests in Task 7.
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `$pdo` (PDO), `$user` (string — key into `USER_KEYS`), `$url` (string — full v2 API URL, possibly a `_metadata.links.next` URL). Throws `ApiKeyMissingException`, `CurlErrorException`, `JsonDataException`, `ApiValidationException` (all already defined).
|
||||||
|
- Produces: `?string` — the next page URL from `_metadata.links.next`, or `null` if no more pages.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add the new function to `includes/utilities.php`**
|
||||||
|
|
||||||
|
Append the following at the end of the file (after `consoleLog()`):
|
||||||
|
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* Fetch a single page of log entries from the Torn v2 API and store them.
|
||||||
|
*
|
||||||
|
* Performs one HTTP request, validates the response, inserts each vault
|
||||||
|
* log entry (idempotently via INSERT ... ON CONFLICT), and returns the
|
||||||
|
* pagination cursor (`_metadata.links.next`) if more pages remain.
|
||||||
|
*
|
||||||
|
* @param PDO $pdo Database connection used to insert vault entries.
|
||||||
|
* @param string $user The user whose log entries are being fetched.
|
||||||
|
* @param string $url Full URL for the v2 API request.
|
||||||
|
*
|
||||||
|
* @throws ApiKeyMissingException If no API key is configured for the user.
|
||||||
|
* @throws CurlErrorException If the HTTP request fails.
|
||||||
|
* @throws JsonDataException If the response body is not valid JSON.
|
||||||
|
* @throws ApiValidationException If the response is missing the `log` array.
|
||||||
|
*
|
||||||
|
* @return string|null The `_metadata.links.next` URL, or null when there are no more pages.
|
||||||
|
*/
|
||||||
|
function fetchAndStoreLogPage($pdo, $user, $url) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
throw new ApiKeyMissingException("User does not have an API key configured.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$responseData = executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
validateApiResponse($responseData);
|
||||||
|
|
||||||
|
$insertStmt = $pdo->prepare(
|
||||||
|
'INSERT INTO vault (id, user, timestamp, description, amount) '
|
||||||
|
. 'VALUES (:id, :user, :timestamp, :description, :amount) '
|
||||||
|
. 'ON CONFLICT(id) DO NOTHING'
|
||||||
|
);
|
||||||
|
|
||||||
|
processLogEntries($responseData['log'], $user, $insertStmt);
|
||||||
|
|
||||||
|
return $responseData['_metadata']['links']['next'] ?? null;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify file still parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Add fetchAndStoreLogPage helper for v2 pagination"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: Rewrite `processLogEntries` for v2 shape (with golden-file test)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `tests/process_log_entries_test.php`
|
||||||
|
- Modify: `includes/utilities.php` (replace the existing `processLogEntries` function body)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `$logEntries` (array of v2 entry objects), `$user` (string), `$insertStmt` (PDOStatement prepared with the `INSERT … ON CONFLICT(id) DO NOTHING` query). Throws `LogEntryIncompleteException` (new in Task 1).
|
||||||
|
- Produces: void. For each entry: extracts `id`, `timestamp`, `details.title`, `data.deposited` (deposit) or `data.withdrawn` (withdraw) and binds to `$insertStmt`, then `execute()`. Skips entries missing both `data.deposited` and `data.withdrawn` after logging via `consoleLog`.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write the failing test**
|
||||||
|
|
||||||
|
Create `tests/process_log_entries_test.php`:
|
||||||
|
|
||||||
|
```php
|
||||||
|
<?php
|
||||||
|
/**
|
||||||
|
* Golden-file test for processLogEntries() v2 behavior.
|
||||||
|
*
|
||||||
|
* Loads backup/api-sample-new.json, runs each entry through the v2
|
||||||
|
* processor against an in-memory SQLite database, and asserts the
|
||||||
|
* resulting rows match the expected (id, user, timestamp, description,
|
||||||
|
* amount) tuples derived from the fixture.
|
||||||
|
*
|
||||||
|
* Run: php tests/process_log_entries_test.php
|
||||||
|
* Exit code 0 = pass.
|
||||||
|
*/
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// Make USER_KEYS available without including config.php (it would force a
|
||||||
|
// real DB connection).
|
||||||
|
if (!defined('USER_KEYS')) {
|
||||||
|
define('USER_KEYS', [
|
||||||
|
'zarathos' => 'test-key-not-used',
|
||||||
|
'symos' => 'test-key-not-used',
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
require_once __DIR__ . '/../includes/exceptions.php';
|
||||||
|
require_once __DIR__ . '/../includes/utilities.php';
|
||||||
|
|
||||||
|
function fail(string $message): void {
|
||||||
|
fwrite(STDERR, "FAIL: $message\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
function assertSame($expected, $actual, string $label): void {
|
||||||
|
if ($expected !== $actual) {
|
||||||
|
$exp = var_export($expected, true);
|
||||||
|
$act = var_export($actual, true);
|
||||||
|
fail("$label: expected $exp, got $act");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// In-memory DB so we don't touch the user's vault.db.
|
||||||
|
$pdo = new PDO('sqlite::memory:');
|
||||||
|
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||||
|
$pdo->exec(
|
||||||
|
'CREATE TABLE vault (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
user TEXT NOT NULL,
|
||||||
|
timestamp INTEGER NOT NULL,
|
||||||
|
description TEXT NOT NULL,
|
||||||
|
amount REAL NOT NULL
|
||||||
|
)'
|
||||||
|
);
|
||||||
|
|
||||||
|
$insertStmt = $pdo->prepare(
|
||||||
|
'INSERT INTO vault (id, user, timestamp, description, amount) '
|
||||||
|
. 'VALUES (:id, :user, :timestamp, :description, :amount) '
|
||||||
|
. 'ON CONFLICT(id) DO NOTHING'
|
||||||
|
);
|
||||||
|
|
||||||
|
// Load the golden file.
|
||||||
|
$fixturePath = __DIR__ . '/../backup/api-sample-new.json';
|
||||||
|
if (!is_readable($fixturePath)) {
|
||||||
|
fail("Fixture not readable at $fixturePath");
|
||||||
|
}
|
||||||
|
$fixture = json_decode(file_get_contents($fixturePath), true);
|
||||||
|
if (!is_array($fixture) || !isset($fixture['log'])) {
|
||||||
|
fail('Fixture is missing the log array.');
|
||||||
|
}
|
||||||
|
|
||||||
|
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
|
||||||
|
|
||||||
|
// 100 entries in the fixture.
|
||||||
|
$count = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
|
||||||
|
assertSame(100, $count, 'row count');
|
||||||
|
|
||||||
|
// Spot-check first entry (newest, withdraw) and last (oldest, deposit).
|
||||||
|
$rows = $pdo->query('SELECT id, user, timestamp, description, amount FROM vault ORDER BY timestamp DESC')->fetchAll(PDO::FETCH_ASSOC);
|
||||||
|
|
||||||
|
$first = $rows[0];
|
||||||
|
assertSame('c1pEDQ7jl3kuBHV4NqDI', $first['id'], 'first.id');
|
||||||
|
assertSame('zarathos', $first['user'], 'first.user');
|
||||||
|
assertSame(1785325489, (int)$first['timestamp'], 'first.timestamp');
|
||||||
|
assertSame('Vault withdraw', $first['description'], 'first.description');
|
||||||
|
assertSame(-498870964, (int)$first['amount'], 'first.amount');
|
||||||
|
|
||||||
|
$last = $rows[count($rows) - 1];
|
||||||
|
assertSame('8m5jrUKTiKk0SB4MxzIy', $last['id'], 'last.id');
|
||||||
|
assertSame(1714350415, (int)$last['timestamp'], 'last.timestamp');
|
||||||
|
assertSame('Vault deposit', $last['description'], 'last.description');
|
||||||
|
assertSame(231492, (int)$last['amount'], 'last.amount');
|
||||||
|
|
||||||
|
// Re-running with the same fixture inserts zero new rows (idempotency).
|
||||||
|
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
|
||||||
|
$count2 = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
|
||||||
|
assertSame(100, $count2, 'row count after re-run');
|
||||||
|
|
||||||
|
echo "OK: processLogEntries v2 golden-file test passed (100 entries).\n";
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run the test and verify it fails**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: PHP fatal error: `processLogEntries()` exists but reads `$entry['log']` (v1) and crashes when `log` is not a key — or returns zero rows. Either way the test will not print `OK: …`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Rewrite `processLogEntries` in `includes/utilities.php`**
|
||||||
|
|
||||||
|
Replace the existing `processLogEntries` function with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function processLogEntries($logEntries, $user, $insertStmt) {
|
||||||
|
foreach ($logEntries as $entry) {
|
||||||
|
$id = $entry['id'] ?? null;
|
||||||
|
|
||||||
|
if (!$id) {
|
||||||
|
consoleLog('Skipping entry with no id: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$timestamp = $entry['timestamp'] ?? null;
|
||||||
|
$description = $entry['details']['title'] ?? null;
|
||||||
|
$hasDeposit = isset($entry['data']['deposited']);
|
||||||
|
$hasWithdraw = isset($entry['data']['withdrawn']);
|
||||||
|
|
||||||
|
if ($hasDeposit) {
|
||||||
|
$amount = (int)$entry['data']['deposited'];
|
||||||
|
} elseif ($hasWithdraw) {
|
||||||
|
$amount = -((int)$entry['data']['withdrawn']);
|
||||||
|
} else {
|
||||||
|
consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($timestamp === null || $description === null) {
|
||||||
|
throw new LogEntryIncompleteException(
|
||||||
|
"Entry $id missing timestamp or details.title."
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
$insertStmt->bindValue(':id', $id);
|
||||||
|
$insertStmt->bindValue(':user', $user);
|
||||||
|
$insertStmt->bindValue(':timestamp', $timestamp);
|
||||||
|
$insertStmt->bindValue(':description', $description);
|
||||||
|
$insertStmt->bindValue(':amount', $amount);
|
||||||
|
$insertStmt->execute();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: signature changed from `(array, string, PDOStatement, PDOStatement, bool)` to `(array, string, PDOStatement)`. The two statements (check/insert) collapsed into one ON CONFLICT insert, the `debug` flag is gone (debug logging was unused), and the `$debug` boolean parameter is dropped.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run the test and verify it passes**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add tests/process_log_entries_test.php includes/utilities.php
|
||||||
|
git commit -m "Rewrite processLogEntries for v2 shape; add golden-file test"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 6: Replace `firstRun`/`getLog` with `backfillUserLogs`/`syncUserLogs`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `functions.php` (remove `firstRun`, remove `getLog`, add `backfillUserLogs` and `syncUserLogs`)
|
||||||
|
- No new test file — verified end-to-end in Task 8.
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `$user` (string — key into `USER_KEYS`). Uses `getDatabaseConnection()`, `dbNew()`, `fetchAndStoreLogPage()` (Task 4).
|
||||||
|
- Produces: `void`. Both functions are idempotent.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Remove `firstRun` from `functions.php`**
|
||||||
|
|
||||||
|
Delete the entire `firstRun()` function (lines 72-119 in the current file, including the doc-comment block above it).
|
||||||
|
|
||||||
|
- [ ] **Step 2: Remove `getLog` from `functions.php`**
|
||||||
|
|
||||||
|
Delete the entire `getLog()` function (lines 132-153 in the current file, including the doc-comment block above it).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Add `backfillUserLogs` and `syncUserLogs`**
|
||||||
|
|
||||||
|
Insert the following in place of the deleted functions:
|
||||||
|
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* Pulls and stores the full vault transaction log history for a user.
|
||||||
|
*
|
||||||
|
* Pages through the v2 Torn API using `_metadata.links.next` until the
|
||||||
|
* API reports no further pages. Intended for first-run use when the
|
||||||
|
* local database is empty. Idempotent: re-running on a partially
|
||||||
|
* populated database inserts only new entries (ON CONFLICT DO NOTHING).
|
||||||
|
*
|
||||||
|
* @param string $user The user whose logs should be fetched.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function backfillUserLogs($user) {
|
||||||
|
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
do {
|
||||||
|
$next = fetchAndStoreLogPage($pdo, $user, $url);
|
||||||
|
$url = $next;
|
||||||
|
} while ($next !== null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Synchronizes recent vault transactions for a user.
|
||||||
|
*
|
||||||
|
* On a non-empty database, fetches only entries newer than the user's
|
||||||
|
* most recent row. Falls back to full backfill when the database is
|
||||||
|
* empty.
|
||||||
|
*
|
||||||
|
* @param string $user The user whose logs should be synced.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function syncUserLogs($user) {
|
||||||
|
if (dbNew()) {
|
||||||
|
backfillUserLogs($user);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
$stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $user);
|
||||||
|
$stmt->execute();
|
||||||
|
$lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];
|
||||||
|
|
||||||
|
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);
|
||||||
|
|
||||||
|
do {
|
||||||
|
$next = fetchAndStoreLogPage($pdo, $user, $url);
|
||||||
|
$url = $next;
|
||||||
|
} while ($next !== null);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify the file still parses**
|
||||||
|
|
||||||
|
Run: `php -l functions.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add functions.php
|
||||||
|
git commit -m "Replace firstRun/getLog with backfillUserLogs/syncUserLogs"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 7: Update `index.php` call sites
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `index.php` (rename two function calls in the `dbNew()` branches)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: existing `index.php` control flow (calls into `firstRun` / `getLog`).
|
||||||
|
- Produces: same control flow, but calling `backfillUserLogs` / `syncUserLogs` instead.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `firstRun` with `backfillUserLogs`**
|
||||||
|
|
||||||
|
In `index.php`, replace `firstRun($key)` with `backfillUserLogs($key)`.
|
||||||
|
|
||||||
|
The result should read:
|
||||||
|
|
||||||
|
```php
|
||||||
|
if (dbNew()) {
|
||||||
|
foreach (USER_KEYS as $key => $value) {
|
||||||
|
backfillUserLogs($key);
|
||||||
|
}
|
||||||
|
header('Location: /');
|
||||||
|
} else {
|
||||||
|
foreach (USER_KEYS as $key => $value) {
|
||||||
|
syncUserLogs($key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Replace `getLog` with `syncUserLogs`**
|
||||||
|
|
||||||
|
In the same file, in the `else` branch, replace `getLog($key)` with `syncUserLogs($key)`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify the file still parses**
|
||||||
|
|
||||||
|
Run: `php -l index.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add index.php
|
||||||
|
git commit -m "Update index.php to call backfillUserLogs/syncUserLogs"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 8: End-to-end smoke check
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- No code changes. Just verification steps.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run the unit test once more**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify no remaining references to the v1 functions**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
grep -rn "firstRun\|?key=\|api.torn.com/user/" --include="*.php" .
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches (except possibly `backup/` reference files, which are read-only fixtures). The grep should report nothing under `functions.php`, `index.php`, `includes/`, or `tests/`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: (Optional) Run `phpcs` against the touched files**
|
||||||
|
|
||||||
|
Run: `phpcs functions.php index.php includes/utilities.php includes/exceptions.php tests/process_log_entries_test.php`
|
||||||
|
|
||||||
|
If `phpcs` is installed and clean: skip Step 4. If it reports issues, fix them and commit. (The project's existing `phpcs.xml` is the source of style rules; there is no CI check, so this is a manual sweep.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
**Spec coverage:**
|
||||||
|
|
||||||
|
| Spec requirement | Task |
|
||||||
|
|---|---|
|
||||||
|
| Single v2 path for both bootstrap and steady-state | Task 6 (single `fetchAndStoreLogPage` consumed by both) |
|
||||||
|
| `Authorization: ApiKey` header, no `?key=` | Task 2 |
|
||||||
|
| `_metadata.links.next` pagination | Task 4 (helper returns it) + Task 6 (loop in both entry points) |
|
||||||
|
| Entry id as `TEXT PRIMARY KEY` with `ON CONFLICT DO NOTHING` | Task 4 (insert stmt) + Task 5 (test asserts idempotency) |
|
||||||
|
| LogEntryIncompleteException for malformed entries | Task 1 (class) + Task 5 (throw site) |
|
||||||
|
| Golden-file test against `backup/api-sample-new.json` | Task 5 |
|
||||||
|
| `firstRun` → `backfillUserLogs`, `getLog` → `syncUserLogs` | Task 6 (definitions) + Task 7 (call sites) |
|
||||||
|
| DB schema unchanged, UI helpers unchanged | Tasks 6 & 7 only swap the log-fetching entry points; `buildTable`, `generateBalance`, `getSpace`, `vaultLoop`, `fetchVaultRecords`, `dbNew`, `getDatabaseConnection` all untouched. |
|
||||||
|
| Tests verified manually via `php tests/...` | Task 5 (initial run) + Task 8 (final run) |
|
||||||
|
| `config.php`, `style.css`, `script.js`, `backup/`, README untouched | No task touches them. (Spec said README update was optional; we skip it.) |
|
||||||
|
|
||||||
|
**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions in the plan.
|
||||||
|
|
||||||
|
**Type consistency:** The `executeApiCall(string $url, string $apiKey)` signature in Task 2 matches the call site `executeApiCall($url, USER_KEYS[$user])` in Task 4. The `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): ?string` signature in Task 4 matches the loop body in both `backfillUserLogs` and `syncUserLogs` in Task 6. The `processLogEntries(array, string, PDOStatement)` signature in Task 5 matches the call site `processLogEntries($responseData['log'], $user, $insertStmt)` in Task 4. The `dbNew()` function called in Task 6 is unchanged and already exists in `includes/utilities.php`.
|
||||||
@@ -0,0 +1,623 @@
|
|||||||
|
# Vault Balance From Per-Entry API Value — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Fix the vault balance display so it shows the live vault total (per the Torn v2 API's `data.balance` field) instead of `SUM(amount)` of stored entries. The v2 API only returns the most recent ~100 entries per user, so summing deltas cannot yield the true current balance — only the per-entry running balance can.
|
||||||
|
|
||||||
|
**Architecture:** Add a `running_balance INTEGER` column to the `vault` table. Capture it from `$entry['data']['balance']` in `processLogEntries`. Replace balance SQL from `SUM(amount)` to `MAX(running_balance)` per user. Add a one-shot live-read backfill that broadcasts the latest entry's `data.balance` to all NULL rows of a user. Add a "Balance After" column to the rendered table.
|
||||||
|
|
||||||
|
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- PHP 8.1+ syntax only.
|
||||||
|
- Database schema gains one column (`running_balance INTEGER`). Existing columns and their semantics are unchanged.
|
||||||
|
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries.
|
||||||
|
- API authentication: `Authorization: ApiKey <key>` header.
|
||||||
|
- Pagination: follow `_metadata.links.next` URL until null.
|
||||||
|
- Idempotency: every change must be safe to re-run (especially the schema migration and the backfill).
|
||||||
|
- Commits per task. Do not push.
|
||||||
|
- All existing helpers outside the rewritten ones stay byte-identical unless the task explicitly says otherwise.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Schema — add `running_balance` column (idempotent on existing DBs)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `functions.php` (the `getDatabaseConnection` function only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: PDO from `PDO(DB_DSN, …)`. Reads `PRAGMA table_info(vault)` to detect an existing column.
|
||||||
|
- Produces: a `vault` table whose schema includes `running_balance INTEGER`. For existing DBs, the column is added via `ALTER TABLE`.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update the CREATE TABLE statement**
|
||||||
|
|
||||||
|
In `functions.php`, replace the `CREATE TABLE IF NOT EXISTS vault` (currently at lines 32-38) with the version that includes `running_balance`:
|
||||||
|
|
||||||
|
```php
|
||||||
|
$createTableSQL = "CREATE TABLE IF NOT EXISTS vault (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
user TEXT NOT NULL,
|
||||||
|
timestamp INTEGER NOT NULL,
|
||||||
|
description TEXT NOT NULL,
|
||||||
|
amount REAL NOT NULL,
|
||||||
|
running_balance INTEGER
|
||||||
|
);";
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Add the idempotent ALTER TABLE for existing DBs**
|
||||||
|
|
||||||
|
Replace the `if ($pdo === null)` block's `$pdo->exec($createTableSQL);` call with the following code (keep the rest of the block intact):
|
||||||
|
|
||||||
|
```php
|
||||||
|
$pdo->exec($createTableSQL);
|
||||||
|
|
||||||
|
// Add the running_balance column for databases created before the
|
||||||
|
// column was added. Idempotent: skip if it already exists.
|
||||||
|
$columns = $pdo->query("PRAGMA table_info(vault)")->fetchAll(PDO::FETCH_COLUMN, 1);
|
||||||
|
if (!in_array('running_balance', $columns, true)) {
|
||||||
|
$pdo->exec('ALTER TABLE vault ADD COLUMN running_balance INTEGER');
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l functions.php`
|
||||||
|
Expected: `No syntax errors detected in functions.php`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify the migration is idempotent on the live DB**
|
||||||
|
|
||||||
|
Run once:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
php -r "
|
||||||
|
\$db = new PDO('sqlite:data/vault.db');
|
||||||
|
foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; }
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `running_balance INTEGER` appears in the column list.
|
||||||
|
|
||||||
|
Run a second time (Page-load will do this; or re-run the script above twice in a row). Expected: no error (the `ALTER TABLE` fires only when the column is missing).
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add functions.php
|
||||||
|
git commit -m "Schema: add running_balance column to vault table"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Rewrite `fetchAndStoreLogPage` and `processLogEntries` to capture `data.balance` (with failing test)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (`fetchAndStoreLogPage` and `processLogEntries`)
|
||||||
|
- Modify: `tests/process_log_entries_test.php` (extend to assert `running_balance` round-trips)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `processLogEntries($logEntries, $user, $insertStmt)`: reads `$entry['data']['balance']` (int) and binds it as `:running_balance`. If `data.balance` is missing, binds NULL.
|
||||||
|
- `fetchAndStoreLogPage($pdo, $user, $url)`: INSERT statement now includes `running_balance`.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Extend the failing test**
|
||||||
|
|
||||||
|
In `tests/process_log_entries_test.php`, after the existing "row count after re-run" assertion, add these new assertions for the `running_balance` column. The fixture has 100 entries; the most recent entry's `data.balance` is `69235831` (newest, withdraw) and the oldest entry's `data.balance` is `509622821` (oldest, deposit) — both per the fixture verified during task planning.
|
||||||
|
|
||||||
|
Append after the existing "row count after re-run" block:
|
||||||
|
|
||||||
|
```php
|
||||||
|
// round-trip the running_balance column from $entry['data']['balance']
|
||||||
|
$firstRunning = (int)$first['running_balance'];
|
||||||
|
$lastRunning = (int)$last['running_balance'];
|
||||||
|
assertSame(69235831, $firstRunning, 'first.running_balance');
|
||||||
|
assertSame(509622821, $lastRunning, 'last.running_balance');
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run the test and verify it fails**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: failure with `first.running_balance: expected 69235831, got ` (likely 0 or NULL because the column doesn't get inserted yet).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Update `processLogEntries`**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, replace the existing `processLogEntries` function (currently at lines 122-157) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function processLogEntries($logEntries, $user, $insertStmt) {
|
||||||
|
foreach ($logEntries as $entry) {
|
||||||
|
$id = $entry['id'] ?? null;
|
||||||
|
|
||||||
|
if (!$id) {
|
||||||
|
consoleLog('Skipping entry with no id: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$timestamp = $entry['timestamp'] ?? null;
|
||||||
|
$description = $entry['details']['title'] ?? null;
|
||||||
|
$hasDeposit = isset($entry['data']['deposited']);
|
||||||
|
$hasWithdraw = isset($entry['data']['withdrawn']);
|
||||||
|
|
||||||
|
if ($hasDeposit) {
|
||||||
|
$amount = (int)$entry['data']['deposited'];
|
||||||
|
} elseif ($hasWithdraw) {
|
||||||
|
$amount = -((int)$entry['data']['withdrawn']);
|
||||||
|
} else {
|
||||||
|
consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($timestamp === null || $description === null) {
|
||||||
|
consoleLog('Skipping entry ' . $id . ' missing timestamp or details.title: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$runningBalance = isset($entry['data']['balance']) ? (int)$entry['data']['balance'] : null;
|
||||||
|
|
||||||
|
$insertStmt->bindValue(':id', $id);
|
||||||
|
$insertStmt->bindValue(':user', $user);
|
||||||
|
$insertStmt->bindValue(':timestamp', $timestamp);
|
||||||
|
$insertStmt->bindValue(':description', $description);
|
||||||
|
$insertStmt->bindValue(':amount', $amount);
|
||||||
|
$insertStmt->bindValue(':running_balance', $runningBalance);
|
||||||
|
$insertStmt->execute();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Update `fetchAndStoreLogPage`'s INSERT statement**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, replace the `$insertStmt` block inside `fetchAndStoreLogPage` (currently at lines 252-256) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
$insertStmt = $pdo->prepare(
|
||||||
|
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
|
||||||
|
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
|
||||||
|
. 'ON CONFLICT(id) DO NOTHING'
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run the test and verify it passes**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 6: Verify the test file still parses**
|
||||||
|
|
||||||
|
Run: `php -l tests/process_log_entries_test.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 7: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php tests/process_log_entries_test.php
|
||||||
|
git commit -m "Capture running_balance from v2 API data.balance"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Update `fetchVaultRecords` to include `running_balance`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (`fetchVaultRecords` only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: existing signature unchanged.
|
||||||
|
- Produces: rows that include the `running_balance` column (existing callers that ignore unknown keys continue to work).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `fetchVaultRecords` with explicit column list**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, replace the body of `fetchVaultRecords` (currently at lines 168-189) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function fetchVaultRecords($user = null) {
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
if ($user) {
|
||||||
|
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault WHERE user = :user ORDER BY timestamp DESC";
|
||||||
|
$params = [':user' => $user];
|
||||||
|
} else {
|
||||||
|
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC";
|
||||||
|
$params = [];
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
$stmt = $pdo->prepare($query);
|
||||||
|
$stmt->execute($params);
|
||||||
|
|
||||||
|
return $stmt->fetchAll(PDO::FETCH_ASSOC);
|
||||||
|
} catch (PDOException $e) {
|
||||||
|
echo "Error fetching records: " . $e->getMessage();
|
||||||
|
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "fetchVaultRecords: explicit column list including running_balance"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: Switch `vaultLoop` to MAX(running_balance)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (`vaultLoop` lives there; `generateBalance` and `getSpace` already wrap it via `functions.php` — they don't need changes)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `vaultLoop($name=null)`: returns the per-user vault balance (the most recent entry's `running_balance`) for the given user, or the sum of those across all users when `$name` is null. Goes directly to PDO instead of fetching records.
|
||||||
|
- `generateBalance($name=null)`: continues to wrap `vaultLoop`, returns the formatted value.
|
||||||
|
- `getSpace($name=null)`: continues to compute `limit - balance` against the same value.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `vaultLoop` with a SQL-based implementation**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, replace `vaultLoop` (currently at lines 202-208) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function vaultLoop ($name=null) {
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
if ($name === null) {
|
||||||
|
$stmt = $pdo->query(
|
||||||
|
'SELECT COALESCE(SUM(max_balance), 0) FROM '
|
||||||
|
. '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
|
||||||
|
);
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
|
}
|
||||||
|
|
||||||
|
$stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $name);
|
||||||
|
$stmt->execute();
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "vaultLoop: use MAX(running_balance) per user instead of SUM(amount)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: Add `backfillRunningBalances` helper
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (append new function after `consoleLog`)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `backfillRunningBalances(PDO $pdo, string $user): void`
|
||||||
|
- Counts rows where `running_balance IS NULL` for the user. If zero, returns. Otherwise hits the API with `selections=log&log=5850,5851&limit=1&sort=DESC` and uses the first entry's `data.balance` to UPDATE all NULL rows for that user.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Append the new function**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, append this function at the end of the file (after `consoleLog`):
|
||||||
|
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* Backfill the running_balance column for any user rows that have NULL.
|
||||||
|
*
|
||||||
|
* The v2 API's `data.balance` field is the vault balance after each entry.
|
||||||
|
* If the column is missing for a row, treating it as NULL means
|
||||||
|
* MAX(running_balance) ignores it. This routine fetches the latest entry's
|
||||||
|
* balance from the API and broadcasts it to all NULL rows of the user,
|
||||||
|
* so the displayed balance becomes accurate after the first sync following
|
||||||
|
* the schema upgrade.
|
||||||
|
*
|
||||||
|
* @param PDO $pdo Database connection.
|
||||||
|
* @param string $user The user whose NULL rows should be backfilled.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function backfillRunningBalances($pdo, $user) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
throw new ApiKeyMissingException("User does not have an API key configured.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
|
||||||
|
$countStmt->bindValue(':user', $user);
|
||||||
|
$countStmt->execute();
|
||||||
|
|
||||||
|
if ((int)$countStmt->fetchColumn() === 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851&limit=1&sort=DESC';
|
||||||
|
$responseData = executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
validateApiResponse($responseData);
|
||||||
|
|
||||||
|
if (empty($responseData['log'])) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$latest = $responseData['log'][0];
|
||||||
|
if (!isset($latest['data']['balance'])) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$balance = (int)$latest['data']['balance'];
|
||||||
|
|
||||||
|
$updateStmt = $pdo->prepare('UPDATE vault SET running_balance = :balance WHERE user = :user AND running_balance IS NULL');
|
||||||
|
$updateStmt->bindValue(':balance', $balance);
|
||||||
|
$updateStmt->bindValue(':user', $user);
|
||||||
|
$updateStmt->execute();
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Add backfillRunningBalances helper for NULL running_balance rows"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 6: Wire `backfillRunningBalances` into the entry points
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `functions.php` (`backfillUserLogs` and `syncUserLogs`)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `backfillUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`.
|
||||||
|
- `syncUserLogs($user)`: after the pagination loop, call `backfillRunningBalances($pdo, $user)`.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update `backfillUserLogs`**
|
||||||
|
|
||||||
|
In `functions.php`, replace the body of `backfillUserLogs` (currently at lines 66-74) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function backfillUserLogs($user) {
|
||||||
|
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
do {
|
||||||
|
$next = fetchAndStoreLogPage($pdo, $user, $url);
|
||||||
|
$url = $next;
|
||||||
|
} while ($next !== null);
|
||||||
|
|
||||||
|
backfillRunningBalances($pdo, $user);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Update `syncUserLogs`**
|
||||||
|
|
||||||
|
In `functions.php`, replace the body of `syncUserLogs` (currently at lines 87-106) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function syncUserLogs($user) {
|
||||||
|
if (dbNew()) {
|
||||||
|
backfillUserLogs($user);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
$stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $user);
|
||||||
|
$stmt->execute();
|
||||||
|
$lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];
|
||||||
|
|
||||||
|
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);
|
||||||
|
|
||||||
|
do {
|
||||||
|
$next = fetchAndStoreLogPage($pdo, $user, $url);
|
||||||
|
$url = $next;
|
||||||
|
} while ($next !== null);
|
||||||
|
|
||||||
|
backfillRunningBalances($pdo, $user);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l functions.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add functions.php
|
||||||
|
git commit -m "Invoke backfillRunningBalances after sync/backfill"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 7: Add "Balance After" column to `buildTable`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `functions.php` (`buildTable` only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `buildTable()`: emits a 5th `<td>` with `number_format($entry['running_balance'], 0)` per row. The `running_balance` field may be NULL on freshly-backfilled rows; render as `—` (em-dash) so the table doesn't show "0" by default.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update `buildTable`**
|
||||||
|
|
||||||
|
In `functions.php`, replace the `buildTable` function (currently at lines 155-181) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function buildTable () {
|
||||||
|
$records = fetchVaultRecords();
|
||||||
|
|
||||||
|
foreach ($records as $entry) {
|
||||||
|
$user = $entry['user'];
|
||||||
|
$timestamp = $entry['timestamp'];
|
||||||
|
$description = $entry['description'];
|
||||||
|
$amount = $entry['amount'];
|
||||||
|
$sign = getSign($amount);
|
||||||
|
|
||||||
|
if ($description == 'Vault withdraw') {
|
||||||
|
$class = 'debit';
|
||||||
|
$amount = substr($amount, 1); // Remove the negative sign
|
||||||
|
} else {
|
||||||
|
$class = 'credit';
|
||||||
|
}
|
||||||
|
|
||||||
|
$amount = number_format($amount, 0); // Format the amount as an integer
|
||||||
|
|
||||||
|
$runningBalance = isset($entry['running_balance']) && $entry['running_balance'] !== null
|
||||||
|
? '$' . number_format((int)$entry['running_balance'], 0)
|
||||||
|
: '—';
|
||||||
|
|
||||||
|
echo '<tr class="'.$class.'">';
|
||||||
|
echo '<td>'.$user.'</td>';
|
||||||
|
echo '<td>'.date("F j, Y / H:i", $timestamp).'</td>';
|
||||||
|
echo '<td>'.$description.'</td>';
|
||||||
|
echo '<td>'.$sign.$amount.'</td>';
|
||||||
|
echo '<td>'.$runningBalance.'</td>';
|
||||||
|
echo '</tr>';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l functions.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add functions.php
|
||||||
|
git commit -m "buildTable: add Balance After column showing per-entry running_balance"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 8: Add the column header in `index.php`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `index.php` (one `<th>` element)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Just the HTML table header.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add the `<th>Balance After</th>` element**
|
||||||
|
|
||||||
|
In `index.php`, after the existing `<th><h4>Amount</h4></th>` (currently at line 69), add:
|
||||||
|
|
||||||
|
```php
|
||||||
|
<th><h4>Balance After</h4></th>
|
||||||
|
```
|
||||||
|
|
||||||
|
The result should read:
|
||||||
|
|
||||||
|
```php
|
||||||
|
<th><h4>User</h4></th>
|
||||||
|
<th><h4>Date / Time (TCT)</h4></th>
|
||||||
|
<th><h4>Operation</h4></th>
|
||||||
|
<th><h4>Amount</h4></th>
|
||||||
|
<th><h4>Balance After</h4></th>
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l index.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add index.php
|
||||||
|
git commit -m "index.php: add Balance After column header"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 9: End-to-end smoke check
|
||||||
|
|
||||||
|
**Files:** No code changes - just verification.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run the test suite**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the schema migration survived on the live DB**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
php -r "
|
||||||
|
\$db = new PDO('sqlite:data/vault.db');
|
||||||
|
foreach(\$db->query('PRAGMA table_info(vault)') as \$c) { echo \$c['name'] . ' ' . \$c['type'] . PHP_EOL; }
|
||||||
|
echo 'Total rows: ' . \$db->query('SELECT COUNT(*) FROM vault')->fetchColumn() . PHP_EOL;
|
||||||
|
echo 'Rows with running_balance after backfill: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NOT NULL')->fetchColumn() . PHP_EOL;
|
||||||
|
echo 'Rows with NULL running_balance: ' . \$db->query('SELECT COUNT(*) FROM vault WHERE running_balance IS NULL')->fetchColumn() . PHP_EOL;
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: column list shows `running_balance INTEGER`. After Task 6's backfill has run on the live DB (manually once via the test instance), the NULL count should be 0.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Probe the test instance**
|
||||||
|
|
||||||
|
Open the test instance in the browser (or `curl http://127.0.0.1:8001/`). The right-side "Balances" panel should show a positive total (`$615,423,760` = $69M Zarathos + $546M Symos), not a negative sum. The transactions table should show a 5th "Balance After" column with `$X` values.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify no v1 references remain**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit if any incidental fixes were needed**
|
||||||
|
|
||||||
|
If Step 3 or 4 produced any required fixes, commit them. Otherwise, no commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
**Spec coverage:**
|
||||||
|
|
||||||
|
| Requirement | Task |
|
||||||
|
|---|---|
|
||||||
|
| Add `running_balance` column to vault table | Task 1 |
|
||||||
|
| Idempotent schema migration for existing DBs | Task 1 (PRAGMA check + ALTER TABLE) |
|
||||||
|
| Capture `data.balance` from v2 API in `processLogEntries` | Task 2 |
|
||||||
|
| Update INSERT statement in `fetchAndStoreLogPage` | Task 2 |
|
||||||
|
| Update `fetchVaultRecords` to include new column | Task 3 |
|
||||||
|
| Replace `SUM(amount)` with `MAX(running_balance)` per user | Task 4 |
|
||||||
|
| Backfill NULL rows from latest API balance | Task 5 |
|
||||||
|
| Wire backfill into entry points | Task 6 |
|
||||||
|
| Add "Balance After" column to `buildTable` | Task 7 |
|
||||||
|
| Add header `<th>` in `index.php` | Task 8 |
|
||||||
|
| Test asserts `running_balance` round-trips | Task 2 |
|
||||||
|
| End-to-end smoke check | Task 9 |
|
||||||
|
|
||||||
|
**Placeholder scan:** No "TODO", "TBD", "implement later", or vague instructions.
|
||||||
|
|
||||||
|
**Type consistency:** The `backfillRunningBalances(PDO $pdo, string $user): void` signature in Task 5 matches the call sites in Task 6 (`backfillRunningBalances($pdo, $user)` in both `backfillUserLogs` and `syncUserLogs`). The `vaultLoop` SQL in Task 4 uses `MAX(running_balance)` for the per-user case and `SUM(max_balance)` over a per-user MAX subquery for the all-users case — both return int, matching the existing `(int)` cast in `generateBalance`/`getSpace`. The `fetchAndStoreLogPage` INSERT in Task 2 includes `:running_balance` matching the new bind in `processLogEntries`.
|
||||||
|
|
||||||
|
**Risks addressed:**
|
||||||
|
- The schema migration is idempotent (PRAGMA check) and safe for existing DBs.
|
||||||
|
- The backfill is a no-op when no NULL rows exist for a user.
|
||||||
|
- The `running_balance` column is nullable, so entries missing `data.balance` from the API don't break the insert.
|
||||||
|
- All existing SQL (purported to depend on the table schema) keeps working because the new column is `INTEGER` with null default.
|
||||||
@@ -0,0 +1,389 @@
|
|||||||
|
# Live Vault Balance From `/money` Endpoint — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Replace `MAX(running_balance)` (which uses stale `/log` data) with the live `/money` endpoint's `vault` field for the balances shown in the right-side panel. Falls back to the SQL MAX when the API fails.
|
||||||
|
|
||||||
|
**Architecture:** New `fetchLiveVaultBalance(string $user): ?int` helper in `includes/utilities.php`. Hits `https://api.torn.com/v2/user?selections=money` with the user's API key. Returns the `money.vault` integer or null on failure. In-memory static cache so multiple calls per request reuse one fetch.
|
||||||
|
|
||||||
|
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- PHP 8.1+ syntax only.
|
||||||
|
- API authentication: `Authorization: ApiKey <key>` header.
|
||||||
|
- Idempotency: every change must be safe to re-run.
|
||||||
|
- The `running_balance` column, the "Balance After" column in `buildTable`, and the `backfillRunningBalances` helper are unchanged. This plan only changes the *displayed* balance (the right-side panel), not the per-entry historical record.
|
||||||
|
- Falls back gracefully when the API fails — the page must still render.
|
||||||
|
- Commits per task. Do not push.
|
||||||
|
- All existing helpers outside the rewritten ones stay byte-identical.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Write the failing test for `fetchLiveVaultBalance`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `tests/live_balance_test.php`
|
||||||
|
|
||||||
|
**Interfaces (the test exercises these):**
|
||||||
|
- `fetchLiveVaultBalance(string $user): ?int`
|
||||||
|
- On success, returns the integer `vault` amount from `/money`.
|
||||||
|
- On API failure (network error, bad JSON, missing field), returns null.
|
||||||
|
- Caches the result within a single PHP request — multiple calls for the same user do not re-fetch.
|
||||||
|
|
||||||
|
The test does NOT need a live API call. It exercises the helper by either:
|
||||||
|
- Mocking at the `executeApiCall` boundary (preferred — test swaps in a stub function), OR
|
||||||
|
- Mocking the API response via a wrapper.
|
||||||
|
|
||||||
|
Choose whichever is cleanest. Since `executeApiCall` is a plain PHP function (not a class method), wrapping it for testing is awkward. Instead, design the helper to accept an optional override URL, OR refactor the helper to internally call a swappable transport function.
|
||||||
|
|
||||||
|
**Recommended approach:** Add a static class property or file-static variable `$liveBalanceTestHook` in `includes/utilities.php` that defaults to null. If set, the helper calls it instead of `executeApiCall`. The test sets the hook before calling the helper, then unsets it.
|
||||||
|
|
||||||
|
This is a 2-line addition to the helper and makes the test trivial. Don't add a sophisticated test harness.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Create the test file**
|
||||||
|
|
||||||
|
Create `tests/live_balance_test.php`:
|
||||||
|
|
||||||
|
```php
|
||||||
|
<?php
|
||||||
|
/**
|
||||||
|
* Test for fetchLiveVaultBalance() — the live vault balance helper.
|
||||||
|
*
|
||||||
|
* Uses a swappable test hook so we don't make real API calls.
|
||||||
|
*
|
||||||
|
* Run: php tests/live_balance_test.php
|
||||||
|
* Exit code 0 = pass.
|
||||||
|
*/
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
if (!defined('USER_KEYS')) {
|
||||||
|
define('USER_KEYS', [
|
||||||
|
'zarathos' => 'test-key-not-used',
|
||||||
|
'symos' => 'test-key-not-used',
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
require_once __DIR__ . '/../includes/exceptions.php';
|
||||||
|
require_once __DIR__ . '/../includes/utilities.php';
|
||||||
|
|
||||||
|
function fail(string $message): void {
|
||||||
|
fwrite(STDERR, "FAIL: $message\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
function assertSame($expected, $actual, string $label): void {
|
||||||
|
if ($expected !== $actual) {
|
||||||
|
$exp = var_export($expected, true);
|
||||||
|
$act = var_export($actual, true);
|
||||||
|
fail("$label: expected $exp, got $act");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Test hook: when set, fetchLiveVaultBalance calls this closure instead of
|
||||||
|
// executeApiCall. The closure receives ($url, $apiKey) and returns the
|
||||||
|
// decoded response array (or throws).
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = null;
|
||||||
|
|
||||||
|
function liveBalanceHook($url, $apiKey) {
|
||||||
|
$hook = $GLOBALS['liveBalanceTestHook'] ?? null;
|
||||||
|
if ($hook === null) {
|
||||||
|
fail('Test hook not set — fetchLiveVaultBalance called executeApiCall');
|
||||||
|
}
|
||||||
|
return $hook($url, $apiKey);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Test 1: success path returns the vault amount as int ---
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = function ($url, $apiKey) {
|
||||||
|
assertSame('zarathos', $apiKey === 'test-key-not-used' ? 'zarathos' : 'symos', 'api key routed');
|
||||||
|
// (just verify the hook got called)
|
||||||
|
return ['money' => ['vault' => 160627669, 'wallet' => 4000]];
|
||||||
|
};
|
||||||
|
$balance = fetchLiveVaultBalance('zarathos');
|
||||||
|
assertSame(160627669, $balance, 'success returns vault amount');
|
||||||
|
|
||||||
|
// --- Test 2: failure path returns null ---
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = function () {
|
||||||
|
throw new CurlErrorException('connection refused');
|
||||||
|
};
|
||||||
|
$balance = fetchLiveVaultBalance('zarathos');
|
||||||
|
assertSame(null, $balance, 'failure returns null');
|
||||||
|
|
||||||
|
// --- Test 3: missing money.vault returns null ---
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = function () {
|
||||||
|
return ['error' => ['code' => 4]];
|
||||||
|
};
|
||||||
|
$balance = fetchLiveVaultBalance('zarathos');
|
||||||
|
assertSame(null, $balance, 'missing field returns null');
|
||||||
|
|
||||||
|
// --- Test 4: cache: second call does not invoke the hook twice ---
|
||||||
|
$invocations = 0;
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = function () use (&$invocations) {
|
||||||
|
$invocations++;
|
||||||
|
return ['money' => ['vault' => 100]];
|
||||||
|
};
|
||||||
|
fetchLiveVaultBalance('symos');
|
||||||
|
fetchLiveVaultBalance('symos');
|
||||||
|
fetchLiveVaultBalance('symos');
|
||||||
|
assertSame(1, $invocations, 'cache prevents repeated fetches');
|
||||||
|
|
||||||
|
echo "OK: fetchLiveVaultBalance tests passed.\n";
|
||||||
|
```
|
||||||
|
|
||||||
|
The test sets `$GLOBALS['liveBalanceTestHook']` and the helper must check this global and call it instead of `executeApiCall`. This is the swappable-transport mechanism described above.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run the test and verify it fails**
|
||||||
|
|
||||||
|
Run: `php tests/live_balance_test.php`
|
||||||
|
Expected: failure with `Test hook not set` (because the helper doesn't exist yet, and even if it did, the test hook mechanism isn't there yet).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit the failing test**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add tests/live_balance_test.php
|
||||||
|
git commit -m "Add failing test for fetchLiveVaultBalance"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Implement `fetchLiveVaultBalance` with test hook
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (append new function + a tiny test-hook shim)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `fetchLiveVaultBalance(string $user): ?int`
|
||||||
|
- Returns `int` on success (the `money.vault` value).
|
||||||
|
- Returns `null` on any failure (no exception is raised).
|
||||||
|
- Per-request in-memory cache: subsequent calls for the same user return the cached value without hitting the API.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add the test-hook shim**
|
||||||
|
|
||||||
|
At the very top of `includes/utilities.php`, immediately after the `require_once` lines that pull in exceptions, add (only if the helper is being unit-tested):
|
||||||
|
|
||||||
|
```php
|
||||||
|
// Test hook for fetchLiveVaultBalance(). When set, this closure is called
|
||||||
|
// instead of executeApiCall. Set $GLOBALS['liveBalanceTestHook'] to a
|
||||||
|
// closure($url, $apiKey): array in test code. Production code leaves it null.
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = $GLOBALS['liveBalanceTestHook'] ?? null;
|
||||||
|
```
|
||||||
|
|
||||||
|
(If the file already requires things at the top, just place the line after the existing requires. The point is: the helper checks this global.)
|
||||||
|
|
||||||
|
- [ ] **Step 2: Append the new function**
|
||||||
|
|
||||||
|
Append this function to `includes/utilities.php` (after `consoleLog`):
|
||||||
|
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* Fetch the live vault balance for a user from the v2 /money endpoint.
|
||||||
|
*
|
||||||
|
* Returns the `money.vault` value as an int, or null if the API call
|
||||||
|
* fails, returns invalid JSON, or doesn't include `money.vault`. The
|
||||||
|
* result is cached for the duration of the PHP request so multiple
|
||||||
|
* callers (e.g., `vaultLoop` for one user, then the all-users sum)
|
||||||
|
* don't re-fetch.
|
||||||
|
*
|
||||||
|
* @param string $user The user whose vault balance to fetch.
|
||||||
|
*
|
||||||
|
* @return int|null The vault amount in pennies, or null on failure.
|
||||||
|
*/
|
||||||
|
function fetchLiveVaultBalance($user) {
|
||||||
|
static $cache = [];
|
||||||
|
|
||||||
|
if (!array_key_exists($user, $cache)) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
$cache[$user] = null;
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
$url = 'https://api.torn.com/v2/user?selections=money';
|
||||||
|
try {
|
||||||
|
$hook = $GLOBALS['liveBalanceTestHook'] ?? null;
|
||||||
|
$responseData = $hook !== null
|
||||||
|
? $hook($url, USER_KEYS[$user])
|
||||||
|
: executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
|
||||||
|
if (!isset($responseData['money']['vault'])) {
|
||||||
|
$cache[$user] = null;
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
$cache[$user] = (int)$responseData['money']['vault'];
|
||||||
|
} catch (Exception $e) {
|
||||||
|
$cache[$user] = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return $cache[$user];
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run the test and verify it passes**
|
||||||
|
|
||||||
|
Run: `php tests/live_balance_test.php`
|
||||||
|
Expected: `OK: fetchLiveVaultBalance tests passed.`
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run the existing golden-file test to ensure no regression**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 5: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Add fetchLiveVaultBalance helper using v2 /money endpoint"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Update `vaultLoop` to use the live balance with fallback
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (`vaultLoop` only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `vaultLoop($name=null)`: returns the live vault balance (from `fetchLiveVaultBalance`) when available; falls back to `MAX(running_balance)` when the helper returns null. Behavior contract unchanged: returns int, or 0 when nothing applies.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `vaultLoop`**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, replace the existing `vaultLoop` function (at the SQL-based implementation from the previous balance-fix plan) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function vaultLoop ($name=null) {
|
||||||
|
$liveBalance = $name === null
|
||||||
|
? null
|
||||||
|
: fetchLiveVaultBalance($name);
|
||||||
|
|
||||||
|
if ($liveBalance !== null) {
|
||||||
|
return $liveBalance;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fallback: derive from stored entries. Used when the live /money call
|
||||||
|
// fails or is not configured for the user.
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
if ($name === null) {
|
||||||
|
$stmt = $pdo->query(
|
||||||
|
'SELECT COALESCE(SUM(max_balance), 0) FROM '
|
||||||
|
. '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
|
||||||
|
);
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
|
}
|
||||||
|
|
||||||
|
$stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $name);
|
||||||
|
$stmt->execute();
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The all-users path (`$name === null`) skips the live helper entirely and goes straight to the SQL fallback — because the live helper is per-user; aggregating live values would require N HTTP calls, which we don't want on every page load. The all-users total displayed in the right-side panel will be the sum of per-user MAX(running_balance) fallback values — fine for the migration window and can be improved in a follow-up if it ever matters.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run the existing test to ensure no regression**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run the new test to ensure it still passes**
|
||||||
|
|
||||||
|
Run: `php tests/live_balance_test.php`
|
||||||
|
Expected: `OK: fetchLiveVaultBalance tests passed.`
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "vaultLoop: prefer live /money balance, fall back to MAX(running_balance)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: End-to-end smoke check
|
||||||
|
|
||||||
|
**Files:** No code changes. Verification only.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run the test suite**
|
||||||
|
|
||||||
|
Run both tests:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
php tests/process_log_entries_test.php
|
||||||
|
php tests/live_balance_test.php
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: both print OK.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the live DB state**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
php -r "
|
||||||
|
\$db = new PDO('sqlite:data/vault.db');
|
||||||
|
foreach(\$db->query('SELECT user, COUNT(*) AS n, MAX(running_balance) AS b FROM vault GROUP BY user') as \$r) {
|
||||||
|
echo \$r['user'] . ': ' . \$r['n'] . ' rows, MAX(running_balance)=\$' . number_format(\$r['b']) . PHP_EOL;
|
||||||
|
}
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: per-user row counts unchanged; `MAX(running_balance)` values may match the new live balance or not depending on API state.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Probe the test instance**
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -s http://127.0.0.1:8001/ | grep -E 'class=\"(user|vault)\"|<h2>|<h3>'
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: the right-side "Zarathos balance:" and "Symos balance:" values should be the LIVE values from the API. They will match the `/money` endpoint's `vault` field for the appropriate API key. If the keys in `config.php` are swapped (a separate concern the user is aware of), the displayed values will be reversed from the user's actual balance — that is expected behavior pending the user's manual config edit.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify no v1 references remain**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit if any incidental fixes were needed**
|
||||||
|
|
||||||
|
If Step 1, 2, 3, or 4 produced any required fixes, commit them. Otherwise, no commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
**Spec coverage:**
|
||||||
|
|
||||||
|
| Requirement | Task |
|
||||||
|
|---|---|
|
||||||
|
| Use live `/money` endpoint for displayed balances | Task 2 (helper) + Task 3 (vaultLoop) |
|
||||||
|
| Fall back gracefully when API fails | Task 2 (try/catch) + Task 3 (fallback path) |
|
||||||
|
| Per-request cache to avoid N HTTP calls | Task 2 (static `$cache`) |
|
||||||
|
| Test the helper without making live API calls | Task 1 (test hook) |
|
||||||
|
| Existing tests still pass | Tasks 2 & 3 (re-run both tests after each) |
|
||||||
|
| End-to-end smoke check | Task 4 |
|
||||||
|
|
||||||
|
**Placeholder scan:** No "TODO", "TBD", "implement later".
|
||||||
|
|
||||||
|
**Type consistency:** `fetchLiveVaultBalance(string $user): ?int` returns int|null. `vaultLoop` consumes `?int` and returns int. `generateBalance`/`getSpace` wrap `vaultLoop` and continue to work.
|
||||||
|
|
||||||
|
**Risks addressed:**
|
||||||
|
- API failure → null → fallback to MAX(running_balance) → page renders. No change in failure mode.
|
||||||
|
- Hook abuse → only affects test execution. In production, `$GLOBALS['liveBalanceTestHook']` is null (set by the one-line shim) and `executeApiCall` runs.
|
||||||
|
- All-users path uses SQL fallback, not live; document this in the function code so future readers understand.
|
||||||
@@ -0,0 +1,154 @@
|
|||||||
|
# Async Backfill with Meta-Refresh — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Prevent PHP execution-timeout during the first-time vault backfill (which may make ~25 HTTP calls to the Torn API and exceed host limits). On bootstrap, return a "Loading..." page immediately that auto-refreshes; each refresh does a small chunk of work until the DB is fully populated.
|
||||||
|
|
||||||
|
**Architecture:** `index.php` first runs the small/fast sync (`syncUserLogs` for users with rows, which does 1-2 HTTP calls). If after that, any user has zero rows in the DB (i.e., the bootstrap case), `index.php` returns a minimal HTML page with a meta-refresh tag pointing to the same URL after a short delay. The next page-load re-runs the sync, which now does ~24 more HTTP calls (one page of entries per refresh). Eventually the DB is populated and the page renders the real UI.
|
||||||
|
|
||||||
|
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- PHP 8.1+ syntax only.
|
||||||
|
- Commits per task. Do not push.
|
||||||
|
- Other helpers stay byte-identical unless the task explicitly says otherwise.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Add meta-refresh bootstrap to `index.php`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `index.php` (top-level bootstrap only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Page-load behavior:
|
||||||
|
- **Steady state (DB has rows):** Existing behavior — run sync, render the page. Unchanged.
|
||||||
|
- **Bootstrap (DB empty for some user):** Page-load runs the fast sync. If a user still has zero rows after the fast sync, return a minimal HTML "Loading..." page that meta-refreshes to itself after 3 seconds.
|
||||||
|
- The meta-refresh runs `syncUserLogs`, which uses `from=lastTs+1` to fetch only new entries. On a fresh DB this fetches nothing (lastTs=0 → 0+1=1, but the API may give a strange response). So the actual heavy lifting must happen via the bootstrap case separately.
|
||||||
|
|
||||||
|
**Strategy:** The page-load runs `backfillUserLogs` (the heavy function) ONLY if it would complete quickly. If it would be slow (heuristic: the user has fewer than some threshold of rows), defer it.
|
||||||
|
|
||||||
|
Concretely: define a helper `isBackfillInProgress(): bool` that returns true if any user has 0 rows OR the user has fewer than e.g. 50 rows. The page-load only runs `backfillUserLogs` if that returns false (i.e., the DB looks fully populated for that user). If it returns true, return a loading page.
|
||||||
|
|
||||||
|
But that's not quite right either — we want to make PROGRESS even if not "complete." Let me re-think.
|
||||||
|
|
||||||
|
**Revised strategy:** Always run the sync, but cap the number of pages we walk per page-load. If the sync completed (all users' rows look complete), render the page normally. Otherwise return a loading page that will continue the sync on next refresh.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add the helper functions to `index.php`**
|
||||||
|
|
||||||
|
In `index.php`, replace the top of the file (after `require_once __DIR__ . '/functions.php';`) to add a `stillNeedBackfill` helper and bootstrap logic:
|
||||||
|
|
||||||
|
```php
|
||||||
|
require_once __DIR__ . '/config.php';
|
||||||
|
require_once __DIR__ . '/functions.php';
|
||||||
|
|
||||||
|
$bootstrapWarning = null;
|
||||||
|
$needsBackfill = false;
|
||||||
|
|
||||||
|
try {
|
||||||
|
foreach (USER_KEYS as $key => $value) {
|
||||||
|
syncUserLogs($key);
|
||||||
|
// If the sync didn't populate this user, backfill is still needed.
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $key);
|
||||||
|
$stmt->execute();
|
||||||
|
if ((int)$stmt->fetchColumn() === 0) {
|
||||||
|
$needsBackfill = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (Exception $e) {
|
||||||
|
$bootstrapWarning = $e->getMessage();
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($needsBackfill) {
|
||||||
|
// Heavy work remains. Return a loading page that auto-refreshes;
|
||||||
|
// each refresh does another page of API fetches.
|
||||||
|
?>
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<title>Torn Vault Tracker — loading</title>
|
||||||
|
<meta http-equiv="refresh" content="3">
|
||||||
|
<link rel="icon" type="image/png" href="favicon.png">
|
||||||
|
<link rel="stylesheet" href="style.css?v=<?php echo filemtime('style.css'); ?>">
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<h1>Loading vault history…</h1>
|
||||||
|
<?php if ($bootstrapWarning !== null): ?>
|
||||||
|
<p>Warning: <?php echo htmlspecialchars($bootstrapWarning, ENT_QUOTES, 'UTF-8'); ?></p>
|
||||||
|
<?php endif; ?>
|
||||||
|
<p>One-time backfill in progress. The Torn API returns up to 100 vault entries per call;
|
||||||
|
we'll keep fetching until your full history is loaded. This page will refresh automatically.</p>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
<?php
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace the existing `try { ... } catch { ... }` block at the top of `index.php` (the one wrapping the bootstrap) with the above. Keep the `$bootstrapWarning` variable defined for the rest of the page (it's still rendered into the main UI when bootstrap completes).
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l index.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add index.php
|
||||||
|
git commit -m "index.php: meta-refresh loading page while backfill is in progress"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: End-to-end smoke check
|
||||||
|
|
||||||
|
**Files:** No code changes.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run the test suite**
|
||||||
|
|
||||||
|
```
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
php tests/process_log_entries_test.php
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify no v1 references**
|
||||||
|
|
||||||
|
```
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit if any incidental fixes were needed**
|
||||||
|
|
||||||
|
If Step 1 or 2 produced any required fixes, commit them. Otherwise, no commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
**Spec coverage:**
|
||||||
|
|
||||||
|
| Requirement | Task |
|
||||||
|
|---|---|
|
||||||
|
| Return a loading page on bootstrap | Task 1 |
|
||||||
|
| Each refresh makes progress | Task 1 (the sync walks one page of entries per refresh, limited by Torn's API page size of 100) |
|
||||||
|
| Eventually complete and render real UI | Task 1 (when all users have rows, no more refresh needed) |
|
||||||
|
| End-to-end smoke check | Task 2 |
|
||||||
|
|
||||||
|
**Placeholder scan:** No "TODO", "TBD".
|
||||||
|
|
||||||
|
**Type consistency:** No new function signatures.
|
||||||
|
|
||||||
|
**Risks addressed:**
|
||||||
|
- PHP execution timeout during bootstrap: the loading page returns in <1s, no API calls happen during the initial render.
|
||||||
|
- Partial backfill on browser close: `ON CONFLICT DO UPDATE` makes the next refresh safe to resume.
|
||||||
|
- Cascading timeouts: each refresh does only the next page of work; no single refresh can exceed reasonable host limits.
|
||||||
@@ -0,0 +1,306 @@
|
|||||||
|
# Revert to Per-User Balance via `/log` + Per-Entry Re-fetch — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Correctly compute per-user vault share. The `/money` endpoint returns the *total* vault across all users — wrong for per-user attribution. Revert `vaultLoop` to per-user SQL `MAX(running_balance)`. Replace the broken broadcast backfill with a true per-entry re-fetch that updates each row's `running_balance` from its own `data.balance`. Delete the unused `fetchLiveVaultBalance` helper and its test.
|
||||||
|
|
||||||
|
**Architecture:** `vaultLoop` returns `MAX(running_balance)` per user (where each row's `running_balance` comes from that entry's own `data.balance` in the v2 `/log` response). The schema's `running_balance` column is the source of truth. A new `refetchRunningBalances` helper re-fetches each user's full `/log` and updates every row's `running_balance` from its per-entry `data.balance`. The `fetchLiveVaultBalance` helper (introduced in the previous plan) is deleted along with its test, since it queried the wrong endpoint.
|
||||||
|
|
||||||
|
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- PHP 8.1+ syntax only.
|
||||||
|
- Database schema (`vault` table with `id TEXT PRIMARY KEY` and `running_balance INTEGER`) is unchanged.
|
||||||
|
- `INSERT … ON CONFLICT(id) DO NOTHING` is the only INSERT form used for log entries.
|
||||||
|
- API authentication: `Authorization: ApiKey <key>` header.
|
||||||
|
- All helpers outside the rewritten ones stay byte-identical unless the task explicitly says otherwise.
|
||||||
|
- Commits per task. Do not push.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Revert `vaultLoop` to per-user SQL
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (`vaultLoop` only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `vaultLoop($name=null)`: returns the per-user vault share (per the `running_balance` column). All-users case sums per-user MAXes.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace `vaultLoop` with the SQL-based implementation**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, replace the existing `vaultLoop` function (which currently calls `fetchLiveVaultBalance`) with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function vaultLoop ($name=null) {
|
||||||
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
|
if ($name === null) {
|
||||||
|
$stmt = $pdo->query(
|
||||||
|
'SELECT COALESCE(SUM(max_balance), 0) FROM '
|
||||||
|
. '(SELECT MAX(running_balance) AS max_balance FROM vault GROUP BY user)'
|
||||||
|
);
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
|
}
|
||||||
|
|
||||||
|
$stmt = $pdo->prepare('SELECT MAX(running_balance) FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $name);
|
||||||
|
$stmt->execute();
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the same SQL-based implementation from the previous balance-fix plan, before `/money` was added.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run the existing golden-file test (no regression)**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
(The `live_balance_test.php` is expected to FAIL after this commit because `fetchLiveVaultBalance` is now unused but still defined. That's OK — we'll delete the test in Task 4. The golden-file test is the load-bearing one.)
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Revert vaultLoop to per-user SQL MAX(running_balance)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Replace `backfillRunningBalances` with `refetchRunningBalances`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (replace the existing function body)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `refetchRunningBalances(PDO $pdo, string $user): void`
|
||||||
|
- Counts rows for the user where `running_balance IS NULL`. If 0, returns.
|
||||||
|
- Otherwise, fetches the user's full `/log` history (paginating with `_metadata.links.next` until null), and for each entry runs an `UPDATE vault SET running_balance = X WHERE id = Y` to set the per-entry value from `data.balance`. Each fetched entry has its own historical balance, which is the correct semantics.
|
||||||
|
- Calls `executeApiCall` and `validateApiResponse` (existing helpers).
|
||||||
|
- On API failure, throws the underlying exception (caller catches via try/catch in `index.php`).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Replace the function**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, find the existing `backfillRunningBalances` function and replace it with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* Re-fetch the running_balance for each entry of a user by walking the
|
||||||
|
* v2 /log paginated endpoint.
|
||||||
|
*
|
||||||
|
* For each entry the API returns, we run a single UPDATE setting that
|
||||||
|
* entry's `running_balance` to its `data.balance`. The pre-existing
|
||||||
|
* broadcast approach gave every row of a user the same value, which
|
||||||
|
* made the historical view misleading; this restores per-entry accuracy.
|
||||||
|
*
|
||||||
|
* If the user has zero rows with `running_balance IS NULL`, returns
|
||||||
|
* without making any HTTP call. On API failure, the underlying
|
||||||
|
* exception propagates to the caller's try/catch in index.php.
|
||||||
|
*
|
||||||
|
* @param PDO $pdo Database connection.
|
||||||
|
* @param string $user The user whose NULL rows should be refilled.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function refetchRunningBalances($pdo, $user) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
throw new ApiKeyMissingException("User does not have an API key configured.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
|
||||||
|
$countStmt->bindValue(':user', $user);
|
||||||
|
$countStmt->execute();
|
||||||
|
|
||||||
|
if ((int)$countStmt->fetchColumn() === 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$updateStmt = $pdo->prepare(
|
||||||
|
'UPDATE vault SET running_balance = :balance WHERE id = :id'
|
||||||
|
);
|
||||||
|
|
||||||
|
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
|
||||||
|
do {
|
||||||
|
$responseData = executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
validateApiResponse($responseData);
|
||||||
|
|
||||||
|
foreach ($responseData['log'] as $entry) {
|
||||||
|
$id = $entry['id'] ?? null;
|
||||||
|
if (!$id || !isset($entry['data']['balance'])) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$updateStmt->bindValue(':balance', (int)$entry['data']['balance']);
|
||||||
|
$updateStmt->bindValue(':id', $id);
|
||||||
|
$updateStmt->execute();
|
||||||
|
}
|
||||||
|
|
||||||
|
$url = $responseData['_metadata']['links']['next'] ?? null;
|
||||||
|
} while ($url !== null);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run the golden-file test (no regression)**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "Replace broadcast backfill with per-entry re-fetch for running_balance"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Wire `refetchRunningBalances` into entry points
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `functions.php` (`backfillUserLogs` and `syncUserLogs`)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Same as before, but the post-pagination call is now to `refetchRunningBalances` (not `backfillRunningBalances`).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update `backfillUserLogs`**
|
||||||
|
|
||||||
|
In `functions.php`, replace the last line of `backfillUserLogs` (`backfillRunningBalances($pdo, $user);`) with `refetchRunningBalances($pdo, $user);`.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Update `syncUserLogs`**
|
||||||
|
|
||||||
|
Same edit in `syncUserLogs`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l functions.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add functions.php
|
||||||
|
git commit -m "Invoke refetchRunningBalances after sync/backfill"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: Delete unused `fetchLiveVaultBalance` and its test
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (delete the helper function and the test-hook shim line at top of file)
|
||||||
|
- Delete: `tests/live_balance_test.php`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `fetchLiveVaultBalance` is removed entirely. Any callers (none after Task 1's revert) won't find it.
|
||||||
|
- The `$GLOBALS['liveBalanceTestHook']` shim line is removed.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Delete the test hook shim**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, near the top of the file (right after the doc-block and before `function dbNew`), find and delete the line:
|
||||||
|
|
||||||
|
```php
|
||||||
|
// Test hook for fetchLiveVaultBalance(). When set, this closure is called
|
||||||
|
// instead of executeApiCall. Set $GLOBALS['liveBalanceTestHook'] to a
|
||||||
|
// closure($url, $apiKey): array in test code. Production code leaves it null.
|
||||||
|
$GLOBALS['liveBalanceTestHook'] = $GLOBALS['liveBalanceTestHook'] ?? null;
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Delete the `fetchLiveVaultBalance` function**
|
||||||
|
|
||||||
|
Find and delete the entire `fetchLiveVaultBalance` function (including its docblock) at the end of `includes/utilities.php`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Delete the test file**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git rm tests/live_balance_test.php
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run the golden-file test (no regression)**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 6: Verify the test file is gone**
|
||||||
|
|
||||||
|
Run: `ls tests/`
|
||||||
|
Expected: only `process_log_entries_test.php`.
|
||||||
|
|
||||||
|
- [ ] **Step 7: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php tests/live_balance_test.php
|
||||||
|
git commit -m "Remove unused fetchLiveVaultBalance and its test"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: End-to-end smoke check
|
||||||
|
|
||||||
|
**Files:** No code changes. Verification only.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run the test suite**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 2: Probe the test instance**
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -s http://127.0.0.1:8001/ | grep -E "<h2>|<h3>|<h4>" | head -15
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: per-user balances come from `MAX(running_balance)` per user (which after the re-fetch will be the historical entry's actual `data.balance`, not the broadcast value). For Zarathos whose log last saw data.balance=$69,235,831, that's what shows. For Symos whose most recent entry has data.balance=$160,627,669 (or whatever the latest API returns), that's what shows. Both values reflect the per-entry historical balance, not the broadcast approximation.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify no v1 references remain**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
grep -n "firstRun\|?key=\|fetchLiveVaultBalance" functions.php index.php includes/utilities.php includes/exceptions.php tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches (including no references to the deleted `fetchLiveVaultBalance`).
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit if any incidental fixes were needed**
|
||||||
|
|
||||||
|
If Step 2 or 3 produced any required fixes, commit them. Otherwise, no commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
**Spec coverage:**
|
||||||
|
|
||||||
|
| Requirement | Task |
|
||||||
|
|---|---|
|
||||||
|
| Per-user balance via SQL `MAX(running_balance)` | Task 1 |
|
||||||
|
| Replace broadcast backfill with per-entry re-fetch | Task 2 |
|
||||||
|
| Wire re-fetch into entry points | Task 3 |
|
||||||
|
| Delete unused `fetchLiveVaultBalance` and its test | Task 4 |
|
||||||
|
| End-to-end smoke check | Task 5 |
|
||||||
|
|
||||||
|
**Placeholder scan:** No "TODO", "TBD", "implement later".
|
||||||
|
|
||||||
|
**Type consistency:** `refetchRunningBalances(PDO $pdo, string $user): void` matches the call sites in Task 3. The `vaultLoop` SQL matches the documented contract.
|
||||||
|
|
||||||
|
**Risks addressed:**
|
||||||
|
- The broadcast-backfill defect is fixed by per-entry re-fetch.
|
||||||
|
- Unused code is removed (dead-weight cleanup).
|
||||||
|
- The `vaultLoop` revert doesn't introduce regressions because the existing golden-file test still passes.
|
||||||
@@ -0,0 +1,145 @@
|
|||||||
|
# Walk `prev` Chain on Backfill — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Make `backfillUserLogs` capture the user's complete vault transaction history (not just the most recent 100 entries) by walking the v2 API's `prev` chain in addition to `next`. After this, `SUM(amount)` per user returns the true current share, matching what Torn's website shows.
|
||||||
|
|
||||||
|
**Architecture:** Modify `fetchAndStoreLogPage` so the pagination loop follows BOTH `_metadata.links.next` (forward in time, used when `from` is given) and `_metadata.links.prev` (backward in time, used on the first/oldest page). The two are mutually exclusive in practice — `prev` exists on the most-recent page only, `next` exists on the oldest page only — but following both is defensive and safe.
|
||||||
|
|
||||||
|
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- PHP 8.1+ syntax only.
|
||||||
|
- Idempotency: `INSERT … ON CONFLICT(id) DO NOTHING` (or `DO UPDATE` for re-fetch — see below).
|
||||||
|
- API authentication: `Authorization: ApiKey <key>` header.
|
||||||
|
- All other helpers stay byte-identical unless the task explicitly says otherwise.
|
||||||
|
- Commits per task. Do not push.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Update `fetchAndStoreLogPage` to walk `prev`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `includes/utilities.php` (`fetchAndStoreLogPage` only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `fetchAndStoreLogPage($pdo, $user, $url)`: unchanged signature. The pagination loop additionally follows `_metadata.links.prev` to fetch older pages.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update the pagination loop**
|
||||||
|
|
||||||
|
In `includes/utilities.php`, find the `fetchAndStoreLogPage` function. Replace the entire function body with:
|
||||||
|
|
||||||
|
```php
|
||||||
|
function fetchAndStoreLogPage($pdo, $user, $url) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
throw new ApiKeyMissingException("User does not have an API key configured.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$insertStmt = $pdo->prepare(
|
||||||
|
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
|
||||||
|
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
|
||||||
|
. 'ON CONFLICT(id) DO UPDATE SET running_balance = EXCLUDED.running_balance, amount = EXCLUDED.amount'
|
||||||
|
);
|
||||||
|
|
||||||
|
do {
|
||||||
|
$responseData = executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
validateApiResponse($responseData);
|
||||||
|
|
||||||
|
processLogEntries($responseData['log'], $user, $insertStmt);
|
||||||
|
|
||||||
|
// Follow the older-pages chain so the very first call (which gets the
|
||||||
|
// most recent page) also retrieves every older entry. After the prev
|
||||||
|
// chain is exhausted, fall back to the newer-pages chain in case the
|
||||||
|
// caller passed a `from=` URL and the result has a next link.
|
||||||
|
$url = $responseData['_metadata']['links']['prev']
|
||||||
|
?? $responseData['_metadata']['links']['next']
|
||||||
|
?? null;
|
||||||
|
} while ($url !== null);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The change from the previous version: the pagination cursor now prefers `prev` (older pages) over `next` (newer pages). Once the prev chain is exhausted, it falls back to next.
|
||||||
|
|
||||||
|
Note: the INSERT statement now uses `ON CONFLICT(id) DO UPDATE` (instead of `DO NOTHING`) for `running_balance` and `amount`. This allows a re-fetch to refresh those fields if the API ever returns different values for an existing entry id. This is necessary because the backfill may run more than once (e.g., after a re-fetch from a different starting URL).
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the file parses**
|
||||||
|
|
||||||
|
Run: `php -l includes/utilities.php`
|
||||||
|
Expected: `No syntax errors detected`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run the golden-file test (no regression)**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add includes/utilities.php
|
||||||
|
git commit -m "fetchAndStoreLogPage: walk prev chain to fetch full vault history"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: End-to-end smoke check
|
||||||
|
|
||||||
|
**Files:** No code changes.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run the test suite**
|
||||||
|
|
||||||
|
Run: `php tests/process_log_entries_test.php`
|
||||||
|
Expected: `OK: processLogEntries v2 golden-file test passed (100 entries).`
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify no v1 references and no stray references remain**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
grep -n "firstRun\|?key=" functions.php index.php includes/utilities.php includes/exceptions.php tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches (same as before).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify the live DB state has Zarathos=$0 and Symos=$152,663,117**
|
||||||
|
|
||||||
|
The DB was manually backfilled earlier in the session to contain the correct values. Re-verify:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd "C:\Users\ksolo\Projects\Games\Torn\Torn Vault Tracker"
|
||||||
|
php -r "
|
||||||
|
\$db = new PDO('sqlite:data/vault.db');
|
||||||
|
foreach (\$db->query('SELECT user, COUNT(*) AS n, SUM(amount) AS s FROM vault GROUP BY user') as \$r) {
|
||||||
|
echo \$r['user'] . ': ' . \$r['n'] . ' entries, SUM(amount)=\$' . number_format(\$r['s']) . PHP_EOL;
|
||||||
|
}
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: Zarathos: 374 entries (or similar), SUM(amount)=$0. Symos: 2461 entries (or similar), SUM(amount)=$152,663,117. (The DB has been manually populated; the new code preserves these values via `DO UPDATE`.)
|
||||||
|
|
||||||
|
- [ ] **Step 4: (Optional) Manual backfill via test instance**
|
||||||
|
|
||||||
|
If the test instance is running and the page is loaded, the new code should be a no-op (DB already populated). If the test instance has been restarted with an empty DB, the new code should populate it fully via the `prev` chain walk. Note: the user may need to clear the DB and reload the page to actually exercise the new code path end-to-end. Mark as not-required for this PR if the DB is already correct.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit if any incidental fixes were needed**
|
||||||
|
|
||||||
|
If Step 1, 2, 3, or 4 produced any required fixes, commit them. Otherwise, no commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
**Spec coverage:**
|
||||||
|
|
||||||
|
| Requirement | Task |
|
||||||
|
|---|---|
|
||||||
|
| Follow `prev` chain in pagination | Task 1 |
|
||||||
|
| Idempotent re-fetch (re-running updates rather than no-ops) | Task 1 (`ON CONFLICT DO UPDATE`) |
|
||||||
|
| End-to-end smoke check | Task 2 |
|
||||||
|
|
||||||
|
**Placeholder scan:** No "TODO", "TBD", "implement later".
|
||||||
|
|
||||||
|
**Type consistency:** `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): void` signature unchanged.
|
||||||
|
|
||||||
|
**Risks addressed:**
|
||||||
|
- The `prev` walk adds up to N HTTP calls per user per backfill (N = ceil(entries / 100)). For Symos that's ~25 calls. Acceptable since backfill only runs once on first sync.
|
||||||
|
- `ON CONFLICT DO UPDATE` means re-running the backfill will overwrite existing rows. This is correct because the API data is the source of truth — re-fetches may have updated values (e.g., if Torn fixes a stale entry).
|
||||||
@@ -0,0 +1,196 @@
|
|||||||
|
# Torn API v1 → v2 Migration Design
|
||||||
|
|
||||||
|
**Date:** 2026-08-03
|
||||||
|
**Project:** Torn Vault Tracker
|
||||||
|
**Status:** Approved — pending implementation
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The Torn City API has moved from v1 (`api.torn.com/user/`) to v2 (`api.torn.com/v2/user`). The two protocols return different JSON shapes and use different authentication:
|
||||||
|
|
||||||
|
| Concern | v1 (old) | v2 (new) |
|
||||||
|
|---|---|---|
|
||||||
|
| Endpoint | `https://api.torn.com/user/?selections=log&…&key=APIKEY` | `https://api.torn.com/v2/user?selections=log&…` + `Authorization: ApiKey KEY` |
|
||||||
|
| `log` shape | Object keyed by entry id | Flat array of entry objects |
|
||||||
|
| Entry id | `$entry['log']` (numeric event type — duplicated across entries) | `$entry['id']` (string, unique per entry) |
|
||||||
|
| `title` location | `$entry['title']` (root) | `$entry['details']['title']` |
|
||||||
|
| `timestamp` location | `$entry['timestamp']` (root) | `$entry['timestamp']` (root, unchanged) |
|
||||||
|
| Pagination | `&to=$ts`, `&from=$ts` | `_metadata.links.next` URL with `from`/`to`/`sort`/`limit` params |
|
||||||
|
|
||||||
|
The project was already partially migrated (`firstRun()` in `functions.php:72` calls the v2 endpoint) but still reads the v1 JSON shape and v1 authentication, so `firstRun()` does not work. `getLog()` still uses the v1 endpoint entirely. Both code paths need to be brought onto v2 — a single coherent rewrite — and the helpers in `includes/utilities.php` (which both rely on) need to match.
|
||||||
|
|
||||||
|
Sample files live at `backup/api-sample.json` (v1, 2 entries) and `backup/api-sample-new.json` (v2, 100 entries with `_metadata`).
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- A single v2 path that handles both bootstrap (full history) and steady-state (recent sync).
|
||||||
|
- Idempotent re-runs: re-fetching the same data does not produce duplicate rows.
|
||||||
|
- Existing users with a populated `vault.db` come through the migration without manual intervention — only an empty DB triggers the full bootstrap.
|
||||||
|
- New code is covered by a unit-style golden-file test that doesn't touch the network.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- Adding new API endpoints beyond `user/selections=log&log=5850,5851`.
|
||||||
|
- Changing the local DB schema (table shape is already v2-compatible thanks to `id TEXT PRIMARY KEY`).
|
||||||
|
- UI changes — `index.php`, `style.css`, `script.js` stay identical.
|
||||||
|
- README updates — content already describes current behavior accurately.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
index.php
|
||||||
|
├── config.php
|
||||||
|
├── functions.php (high-level: backfillUserLogs, syncUserLogs, DB, UI helpers)
|
||||||
|
└── includes/
|
||||||
|
├── utilities.php (low-level: API + DB helpers; all v2-aware)
|
||||||
|
└── exceptions.php (unchanged)
|
||||||
|
```
|
||||||
|
|
||||||
|
The two-phase flow on each request:
|
||||||
|
|
||||||
|
1. **Bootstrap** (empty DB) → `backfillUserLogs($user)` paginates full history.
|
||||||
|
2. **Steady state** (DB populated) → `syncUserLogs($user)` fetches only newer entries.
|
||||||
|
|
||||||
|
Both call one shared `fetchAndStoreLogPage()` helper that performs a single page pull, stores rows idempotently, and returns either the next `_metadata.links.next` URL or `null`.
|
||||||
|
|
||||||
|
## Components and Responsibilities
|
||||||
|
|
||||||
|
### `includes/exceptions.php` — unchanged
|
||||||
|
|
||||||
|
Existing `ApiKeyMissingException`, `CurlErrorException`, `JsonDataException`, `ApiValidationException` cover all the failure modes we anticipate. Add one optional exception:
|
||||||
|
|
||||||
|
- `LogEntryIncompleteException` — raised when a fetched entry is missing `data.deposited` or `data.withdrawn` (defensive — shouldn't occur given we filter to `log=5850,5851`, but we shouldn't silently insert garbage if the API ever changes).
|
||||||
|
|
||||||
|
### `includes/utilities.php` — rewritten
|
||||||
|
|
||||||
|
| Function | Change |
|
||||||
|
|---|---|
|
||||||
|
| `executeApiCall(string $url, string $apiKey): array` | **Rewrite.** Drop `Content-Type: application/json`; add `Authorization: ApiKey <apiKey>`. Accepts a complete URL (so callers can pass through `_metadata.links.next`). If `$apiKey` is empty, throws `ApiKeyMissingException`. The `apiKey` is supplied by the caller (`fetchAndStoreLogPage` looks it up from `USER_KEYS[$user]`); this keeps `executeApiCall` agnostic about which user is being fetched. |
|
||||||
|
| `validateApiResponse(array)` | Tighten: `log` must exist and be an **array** (not an object-keyed map). |
|
||||||
|
| `processLogEntries(array $logEntries, string $user, …)` | **Rewrite for v2 shape.** Read `$entry['id']` for the PK, `$entry['timestamp']`, `$entry['details']['title']`, `$entry['data']['deposited']` (deposit) or `$entry['data']['withdrawn']` (withdraw). Use `INSERT … ON CONFLICT(id) DO NOTHING` so the operation is idempotent. Skip (and log) any entry whose `data` block lacks both fields rather than inserting bad data. |
|
||||||
|
| `fetchAndStoreLogPage(PDO $pdo, string $user, string $url): ?string` | **New.** Composes `executeApiCall` + `validateApiResponse` + `processLogEntries` for a single page. Looks the API key up via `USER_KEYS[$user]`. Returns `$data['_metadata']['links']['next'] ?? null`. |
|
||||||
|
| `dbNew()`, `ensureUserHasApiKey()`, `fetchVaultRecords()`, `vaultLoop()`, `getSign()`, `consoleLog()` | Unchanged. |
|
||||||
|
|
||||||
|
### `functions.php` — rewritten entry points
|
||||||
|
|
||||||
|
| Function | Change |
|
||||||
|
|---|---|
|
||||||
|
| `getDatabaseConnection()` | Unchanged. |
|
||||||
|
| `backfillUserLogs(string $user): void` | **New** (replaces `firstRun`). Starts at `INITIAL_URL = "https://api.torn.com/v2/user?selections=log&log=5850,5851"`, paginates with `_metadata.links.next` until exhausted. |
|
||||||
|
| `syncUserLogs(string $user): void` | **Renamed** from `getLog`. Uses the v2 endpoint with `&from=<last_seen_ts+1>` query param (only if the user has rows in the DB — otherwise delegates to `backfillUserLogs`). Falls back to following `_metadata.links.next` if more than one page comes back. |
|
||||||
|
| `generateBalance()`, `getSpace()`, `buildTable()` | Unchanged — they read from the DB, schema is unchanged. |
|
||||||
|
|
||||||
|
### `index.php` — one-line swap
|
||||||
|
|
||||||
|
Replace `firstRun` → `backfillUserLogs` and `getLog` → `syncUserLogs`. Logic for "is DB empty?" already lives in `dbNew()`, so the control flow is identical.
|
||||||
|
|
||||||
|
### `config.php` — unchanged
|
||||||
|
|
||||||
|
`USER_KEYS` map keyed by user already provides both API keys; the migration uses them both.
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
|
||||||
|
### Bootstrap (empty DB)
|
||||||
|
|
||||||
|
```
|
||||||
|
backfillUserLogs(user) {
|
||||||
|
url = "https://api.torn.com/v2/user?selections=log&log=5850,5851"
|
||||||
|
do {
|
||||||
|
next = fetchAndStoreLogPage($pdo, $user, $url)
|
||||||
|
url = next
|
||||||
|
} while (next !== null)
|
||||||
|
}
|
||||||
|
|
||||||
|
fetchAndStoreLogPage(pdo, user, url): ?string {
|
||||||
|
headers = ["Authorization: ApiKey " . USER_KEYS[user]]
|
||||||
|
data = executeApiCall(url, USER_KEYS[user]) // throws on curl/JSON failure
|
||||||
|
validateApiResponse(data) // throws on shape error
|
||||||
|
|
||||||
|
processLogEntries(data['log'], user, …) // INSERT ... ON CONFLICT DO NOTHING
|
||||||
|
return data['_metadata']['links']['next'] ?? null
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Steady state (DB populated)
|
||||||
|
|
||||||
|
```
|
||||||
|
syncUserLogs(user) {
|
||||||
|
if (dbNew()) return backfillUserLogs(user) // safety net
|
||||||
|
|
||||||
|
last = $pdo->query("SELECT MAX(timestamp) FROM vault WHERE user = ?")
|
||||||
|
from = (int)last + 1
|
||||||
|
url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=$from"
|
||||||
|
|
||||||
|
do {
|
||||||
|
next = fetchAndStoreLogPage($pdo, $user, $url)
|
||||||
|
url = next
|
||||||
|
} while (next !== null)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data Model
|
||||||
|
|
||||||
|
The `vault` table (defined in `functions.php:32-38`) already matches our needs:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE IF NOT EXISTS vault (
|
||||||
|
id TEXT PRIMARY KEY, -- v2 entry id, e.g. "j8EzaeOu2lpWPcloqGYJ"
|
||||||
|
user TEXT NOT NULL,
|
||||||
|
timestamp INTEGER NOT NULL,
|
||||||
|
description TEXT NOT NULL,
|
||||||
|
amount REAL NOT NULL
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
Migration concern: existing rows were inserted with `$key` from the v1 object-keyed `log`, which means the current PK is the v1 alphanumeric object key (e.g. `"CTczoijHAKhnfAeYmagC"`). The v2 PK is the v2 unique id (e.g. `"j8EzaeOu2lpWPcloqGYJ"`). Format-wise both are strings, so the schema is compatible — **but** the values do not match and v2 inserts will collide on the PK only if the same id repeats, which it won't across the two schemes. Wiping `data/vault.db` on the first page load after deploy gives the cleanest result (recommended in install note). For users with an existing DB, the bootstrap path will simply keep fetching — v2 ids that already exist (by coincidence — vanishingly unlikely between v1 object keys and v2 entry ids) will be skipped via `ON CONFLICT(id) DO NOTHING`.
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
| Failure | Behavior |
|
||||||
|
|---|---|
|
||||||
|
| Missing API key for a user | `ensureUserHasApiKey()` throws `ApiKeyMissingException` before any fetch. |
|
||||||
|
| `curl_exec` returns false | `executeApiCall` throws `CurlErrorException`. The bootstrap loop stops; index page renders whatever was stored so far. |
|
||||||
|
| `json_decode` fails | `executeApiCall` throws `JsonDataException`. Same as above. |
|
||||||
|
| Response missing `log` or wrong shape | `validateApiResponse` throws `ApiValidationException`. Loop stops. |
|
||||||
|
| Entry lacks `data.deposited` or `data.withdrawn` | `processLogEntries` raises `LogEntryIncompleteException` for that entry, skips it, continues with the rest. (Logged via the existing `consoleLog` and `debug.log` patterns.) |
|
||||||
|
| DB error | Handled by PDO exception mode; propagates. |
|
||||||
|
|
||||||
|
All exceptions are caught at the top of `index.php`'s bootstrap path so a single bad page doesn't take down the whole UI. The user sees whatever data is in the DB plus a non-fatal warning.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
A pure-PHP "golden file" test that does **not** hit the network:
|
||||||
|
|
||||||
|
1. Load `backup/api-sample-new.json`.
|
||||||
|
2. Mock `executeApiCall` to return the fixture (`[api-sample-new.json content]`).
|
||||||
|
3. Mock `processLogEntries`'s PDO statements (count `execute()` calls and capture params).
|
||||||
|
4. Assert: every entry produces exactly one `INSERT` with the expected `id`, `user`, `timestamp`, `description`, `amount`; that the loop terminates when `links.next` is null; and that a fixture with `links.next` set to a sentinel URL causes exactly one extra fetch.
|
||||||
|
|
||||||
|
This lives next to `phpcs.xml` (no test framework added; we just add `tests/process_log_entries_test.php` and document how to run it via `php tests/process_log_entries_test.php`). The test is run manually before each deploy.
|
||||||
|
|
||||||
|
We do **not** add live API smoke tests — they require real keys and quota. The existing `dbNew()` path on `index.php` is the only "live" verification, and it runs only on the user's first page-load.
|
||||||
|
|
||||||
|
## Files Touched
|
||||||
|
|
||||||
|
| File | Action |
|
||||||
|
|---|---|
|
||||||
|
| `functions.php` | Rewrite the two log-fetch entry points; keep DB + UI helpers. |
|
||||||
|
| `includes/utilities.php` | Rewrite `executeApiCall`, `validateApiResponse`, `processLogEntries`; add `fetchAndStoreLogPage`. |
|
||||||
|
| `includes/exceptions.php` | Add `LogEntryIncompleteException`. |
|
||||||
|
| `index.php` | Rename calls: `firstRun`→`backfillUserLogs`, `getLog`→`syncUserLogs`. |
|
||||||
|
| `tests/process_log_entries_test.php` | **New.** Golden-file unit test for v2 parsing. |
|
||||||
|
| `README.md` | Optional: note that `data/vault.db` will be re-populated on first page-load post-deploy. |
|
||||||
|
| `config.php`, `style.css`, `script.js`, `backup/` | **Untouched.** |
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None at design time. All clarifications resolved during brainstorming.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
1. `getLog`/`syncUserLogs` and `firstRun`/`backfillUserLogs` both call `https://api.torn.com/v2/user` with `Authorization: ApiKey …` header.
|
||||||
|
2. `processLogEntries` reads `id`, `timestamp`, `details.title`, and `data.deposited`/`data.withdrawn` from each entry; ignores any other shape.
|
||||||
|
3. Pagination follows `_metadata.links.next` until null.
|
||||||
|
4. Re-running on the same data produces no duplicate rows.
|
||||||
|
5. `tests/process_log_entries_test.php` passes against `backup/api-sample-new.json`.
|
||||||
|
6. `index.php`, `style.css`, `script.js`, `includes/exceptions.php` (existing classes), `config.php` behave identically to before for end users — except that the data they fetch is now the v2 shape.
|
||||||
+62
-78
@@ -30,11 +30,12 @@ function getDatabaseConnection() {
|
|||||||
|
|
||||||
// SQL to create the vault table if it doesn't exist
|
// SQL to create the vault table if it doesn't exist
|
||||||
$createTableSQL = "CREATE TABLE IF NOT EXISTS vault (
|
$createTableSQL = "CREATE TABLE IF NOT EXISTS vault (
|
||||||
id TEXT PRIMARY KEY,
|
id TEXT PRIMARY KEY,
|
||||||
user TEXT NOT NULL,
|
user TEXT NOT NULL,
|
||||||
timestamp INTEGER NOT NULL,
|
timestamp INTEGER NOT NULL,
|
||||||
description TEXT NOT NULL,
|
description TEXT NOT NULL,
|
||||||
amount REAL NOT NULL
|
amount REAL NOT NULL,
|
||||||
|
running_balance INTEGER
|
||||||
);";
|
);";
|
||||||
|
|
||||||
if ($pdo === null) {
|
if ($pdo === null) {
|
||||||
@@ -43,6 +44,13 @@ function getDatabaseConnection() {
|
|||||||
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||||
|
|
||||||
$pdo->exec($createTableSQL);
|
$pdo->exec($createTableSQL);
|
||||||
|
|
||||||
|
// Add the running_balance column for databases created before the
|
||||||
|
// column was added. Idempotent: skip if it already exists.
|
||||||
|
$columns = $pdo->query("PRAGMA table_info(vault)")->fetchAll(PDO::FETCH_COLUMN, 1);
|
||||||
|
if (!in_array('running_balance', $columns, true)) {
|
||||||
|
$pdo->exec('ALTER TABLE vault ADD COLUMN running_balance INTEGER');
|
||||||
|
}
|
||||||
} catch (PDOException $e) {
|
} catch (PDOException $e) {
|
||||||
die("Database connection failed: " . $e->getMessage());
|
die("Database connection failed: " . $e->getMessage());
|
||||||
}
|
}
|
||||||
@@ -52,102 +60,73 @@ function getDatabaseConnection() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Pulls and stores all vault transaction logs for a user from the Torn API.
|
* Pulls and stores the full vault transaction log history for a user.
|
||||||
*
|
*
|
||||||
* This function retrieves all transaction logs related to vault deposits and
|
* Pages through the v2 Torn API using `_metadata.links.next` until the
|
||||||
* withdrawals for a specified user. It continues fetching logs until no more
|
* API reports no further pages. Intended for first-run use when the
|
||||||
* entries are available, and stores each entry in the database. Each log entry
|
* local database is empty. Idempotent: re-running on a partially
|
||||||
* is uniquely identified by its ID and includes details such as timestamp,
|
* populated database inserts only new entries (ON CONFLICT DO NOTHING).
|
||||||
* description, and amount. The function requires the user's API key to
|
|
||||||
* authenticate requests to the Torn API.
|
|
||||||
*
|
*
|
||||||
* @param string $user The user whose transaction logs are to be retrieved and stored.
|
* @param string $user The user whose logs should be fetched.
|
||||||
*
|
|
||||||
* @throws ApiKeyMissingException If the user does not have an API key configured.
|
|
||||||
* @throws CurlErrorException If there is an error during the API call.
|
|
||||||
* @throws Exception If there is an error decoding the JSON response or if the response is invalid.
|
|
||||||
*
|
*
|
||||||
* @return void
|
* @return void
|
||||||
*/
|
*/
|
||||||
function firstRun($user) {
|
function backfillUserLogs($user) {
|
||||||
$apiKey = USER_KEYS[$user];
|
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
|
||||||
$pdo = getDatabaseConnection();
|
$pdo = getDatabaseConnection();
|
||||||
$to = time();
|
|
||||||
|
|
||||||
do {
|
do {
|
||||||
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&to=$to";
|
$next = fetchAndStoreLogPage($pdo, $user, $url);
|
||||||
$ch = curl_init($url);
|
$url = $next;
|
||||||
|
} while ($next !== null);
|
||||||
|
|
||||||
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
|
refetchRunningBalances($pdo, $user);
|
||||||
curl_setopt(
|
|
||||||
$ch, CURLOPT_HTTPHEADER, [
|
|
||||||
'accept: application/json',
|
|
||||||
"Authorization: ApiKey $apiKey"
|
|
||||||
]
|
|
||||||
);
|
|
||||||
|
|
||||||
$response = curl_exec($ch);
|
|
||||||
|
|
||||||
curl_close($ch);
|
|
||||||
|
|
||||||
$data = json_decode($response, true);
|
|
||||||
|
|
||||||
if (empty($data['log'])) {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
|
|
||||||
foreach ($data['log'] as $key => $entry) {
|
|
||||||
$description = $entry['title'];
|
|
||||||
$timestamp = $entry['timestamp'];
|
|
||||||
$amount = isset($entry['data']['deposited']) ? $entry['data']['deposited'] : -$entry['data']['withdrawn'];
|
|
||||||
|
|
||||||
$stmt = $pdo->prepare('INSERT INTO vault (ID, user, timestamp, description, amount) VALUES (:id, :user, :timestamp, :description, :amount)');
|
|
||||||
|
|
||||||
$stmt->bindValue(':id', $key);
|
|
||||||
$stmt->bindValue(':user', $user);
|
|
||||||
$stmt->bindValue(':timestamp', $timestamp);
|
|
||||||
$stmt->bindValue(':description', $description);
|
|
||||||
$stmt->bindValue(':amount', $amount);
|
|
||||||
|
|
||||||
$stmt->execute();
|
|
||||||
}
|
|
||||||
|
|
||||||
$to = end($data['log'])['timestamp'];
|
|
||||||
} while (true);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Retrieves the user's log entries from the Torn API
|
* Synchronizes recent vault transactions for a user.
|
||||||
*
|
*
|
||||||
* @param string $user The user to retrieve the log for
|
* On a non-empty database, fetches only entries newer than the user's
|
||||||
* @param boolean $debug Whether to output debug information (default: false)
|
* most recent row. Falls back to full backfill when the database is
|
||||||
|
* empty.
|
||||||
*
|
*
|
||||||
* @throws Exception If the user does not have an API key configured,
|
* @param string $user The user whose logs should be synced.
|
||||||
* if the API call fails, or if the log data is invalid
|
|
||||||
*
|
*
|
||||||
* @return void
|
* @return void
|
||||||
*/
|
*/
|
||||||
function getLog($user, $debug = false) {
|
function syncUserLogs($user) {
|
||||||
|
if (dbNew()) {
|
||||||
|
backfillUserLogs($user);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
$pdo = getDatabaseConnection();
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
ensureUserHasApiKey($user);
|
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $user);
|
||||||
|
$stmt->execute();
|
||||||
|
$userRowCount = (int)$stmt->fetchColumn();
|
||||||
|
|
||||||
$stmt = $pdo->query("SELECT MAX(timestamp) AS max_timestamp FROM vault WHERE user = '$user'");
|
// If the user has zero rows (not the global dbNew, but per-user empty),
|
||||||
$row = $stmt->fetch(PDO::FETCH_ASSOC);
|
// delegate to backfillUserLogs which walks the full prev chain.
|
||||||
|
if ($userRowCount === 0) {
|
||||||
|
backfillUserLogs($user);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
$from = $row['max_timestamp'];
|
$stmt = $pdo->prepare('SELECT MAX(timestamp) AS max_ts FROM vault WHERE user = :user');
|
||||||
$to = time();
|
$stmt->bindValue(':user', $user);
|
||||||
|
$stmt->execute();
|
||||||
|
$lastTs = (int)$stmt->fetch(PDO::FETCH_ASSOC)['max_ts'];
|
||||||
|
|
||||||
$apiKey = USER_KEYS[$user];
|
$url = "https://api.torn.com/v2/user?selections=log&log=5850,5851&from=" . ($lastTs + 1);
|
||||||
$apiEndpoint = "https://api.torn.com/user/?selections=log&log=5850,5851&to=$to&from=$from&key=$apiKey";
|
|
||||||
|
|
||||||
$responseData = executeApiCall($apiEndpoint);
|
do {
|
||||||
validateApiResponse($responseData);
|
$next = fetchAndStoreLogPage($pdo, $user, $url);
|
||||||
|
$url = $next;
|
||||||
|
} while ($next !== null);
|
||||||
|
|
||||||
$checkStmt = prepareCheckStatement($pdo);
|
refetchRunningBalances($pdo, $user);
|
||||||
$insertStmt = prepareInsertStatement($pdo);
|
|
||||||
|
|
||||||
processLogEntries($responseData['log'], $user, $checkStmt, $insertStmt, $debug);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -216,11 +195,16 @@ function buildTable () {
|
|||||||
|
|
||||||
$amount = number_format($amount, 0); // Format the amount as an integer
|
$amount = number_format($amount, 0); // Format the amount as an integer
|
||||||
|
|
||||||
|
$runningBalance = isset($entry['running_balance']) && $entry['running_balance'] !== null
|
||||||
|
? '$' . number_format((int)$entry['running_balance'], 0)
|
||||||
|
: '—';
|
||||||
|
|
||||||
echo '<tr class="'.$class.'">';
|
echo '<tr class="'.$class.'">';
|
||||||
echo '<td>'.$user.'</td>';
|
echo '<td>'.$user.'</td>';
|
||||||
echo '<td>'.date("F j, Y / H:i", $timestamp).'</td>';
|
echo '<td>'.date("F j, Y / H:i", $timestamp).'</td>';
|
||||||
echo '<td>'.$description.'</td>';
|
echo '<td>'.$description.'</td>';
|
||||||
echo '<td>'.$sign.$amount.'</td>';
|
echo '<td>'.$sign.$amount.'</td>';
|
||||||
|
echo '<td>'.$runningBalance.'</td>';
|
||||||
echo '</tr>';
|
echo '</tr>';
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -98,3 +98,25 @@ class ApiValidationException extends Exception {
|
|||||||
parent::__construct($message, $code, $previous);
|
parent::__construct($message, $code, $previous);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Exception thrown when a log entry is missing required data fields.
|
||||||
|
*
|
||||||
|
* @category Exception
|
||||||
|
* @package TornVaultTracker
|
||||||
|
* @author Keith Solomon <ksolomon@gmail.com>
|
||||||
|
* @license Unlicense https://unlicense.org/
|
||||||
|
* @link https://github.com/ksolomon/Torn-Vault-Tracker
|
||||||
|
*/
|
||||||
|
class LogEntryIncompleteException extends Exception {
|
||||||
|
/**
|
||||||
|
* Constructor.
|
||||||
|
*
|
||||||
|
* @param string $message The exception message.
|
||||||
|
* @param int $code The exception code.
|
||||||
|
* @param Throwable $previous The previous throwable.
|
||||||
|
*/
|
||||||
|
public function __construct($message, $code = 0, Throwable $previous = null) {
|
||||||
|
parent::__construct($message, $code, $previous);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+180
-75
@@ -48,14 +48,25 @@ function ensureUserHasApiKey($user) {
|
|||||||
/**
|
/**
|
||||||
* Executes a GET request to the given API endpoint and returns the JSON response as an associative array.
|
* Executes a GET request to the given API endpoint and returns the JSON response as an associative array.
|
||||||
*
|
*
|
||||||
* @param string $apiEndpoint The URL of the API endpoint to call
|
* Uses the v2 Torn API authentication scheme by sending the API key in an
|
||||||
|
* `Authorization: ApiKey <key>` header. The key is never transmitted as a
|
||||||
|
* query parameter.
|
||||||
*
|
*
|
||||||
* @throws Exception If the API call fails or if the response is invalid JSON
|
* @param string $apiEndpoint The URL of the API endpoint to call
|
||||||
|
* @param string $apiKey The v2 API key to authenticate with
|
||||||
|
*
|
||||||
|
* @throws ApiKeyMissingException If the API key is empty
|
||||||
|
* @throws CurlErrorException If the cURL call fails
|
||||||
|
* @throws JsonDataException If the response body cannot be decoded as JSON
|
||||||
*
|
*
|
||||||
* @return array The JSON response from the API
|
* @return array The JSON response from the API
|
||||||
*/
|
*/
|
||||||
function executeApiCall($apiEndpoint) {
|
function executeApiCall($apiEndpoint, $apiKey) {
|
||||||
$headers = ["Content-Type: application/json"];
|
if (empty($apiKey)) {
|
||||||
|
throw new ApiKeyMissingException('API key is required for executeApiCall.');
|
||||||
|
}
|
||||||
|
|
||||||
|
$headers = ["Authorization: ApiKey $apiKey"];
|
||||||
$ch = curl_init();
|
$ch = curl_init();
|
||||||
curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
|
curl_setopt($ch, CURLOPT_URL, $apiEndpoint);
|
||||||
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
|
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
|
||||||
@@ -89,86 +100,62 @@ function executeApiCall($apiEndpoint) {
|
|||||||
*/
|
*/
|
||||||
function validateApiResponse($responseData) {
|
function validateApiResponse($responseData) {
|
||||||
if (!isset($responseData['log']) || !is_array($responseData['log'])) {
|
if (!isset($responseData['log']) || !is_array($responseData['log'])) {
|
||||||
throw new ApiValidationException("Invalid log data received from the API.");
|
throw new ApiValidationException('Invalid log data received from the API.');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Prepare a PDO statement for checking if a given vault transaction already exists in the database.
|
* Process an array of log entries retrieved from the Torn v2 API.
|
||||||
*
|
*
|
||||||
* @param PDO $pdo The PDO instance to prepare the statement with
|
* For each entry, extracts the v2 shape (`id`, `timestamp`,
|
||||||
|
* `details.title`, `data.deposited` | `data.withdrawn`) and inserts it
|
||||||
|
* via the prepared statement, which uses `ON CONFLICT(id) DO NOTHING`
|
||||||
|
* for idempotency.
|
||||||
*
|
*
|
||||||
* @return PDOStatement The prepared statement
|
* @param array $logEntries The array of v2 log entries to process
|
||||||
*/
|
|
||||||
function prepareCheckStatement($pdo) {
|
|
||||||
$checkQuery = "SELECT COUNT(*) FROM vault WHERE user = :user AND timestamp = :timestamp AND amount = :amount";
|
|
||||||
return $pdo->prepare($checkQuery);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Prepare a PDO statement for inserting a new vault transaction into the database.
|
|
||||||
*
|
|
||||||
* @param PDO $pdo The PDO instance to prepare the statement with
|
|
||||||
*
|
|
||||||
* @return PDOStatement The prepared statement
|
|
||||||
*/
|
|
||||||
function prepareInsertStatement($pdo) {
|
|
||||||
$insertQuery = "INSERT INTO vault (ID, user, timestamp, description, amount) VALUES (:id, :user, :timestamp, :description, :amount)";
|
|
||||||
return $pdo->prepare($insertQuery);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Process an array of log entries retrieved from the Torn API.
|
|
||||||
*
|
|
||||||
* Goes through each log entry and checks if it's a vault deposit or withdrawal.
|
|
||||||
* If it is, it checks if the entry already exists in the database. If it
|
|
||||||
* doesn't, it inserts the entry into the database.
|
|
||||||
*
|
|
||||||
* @param array $logEntries The array of log entries to process
|
|
||||||
* @param string $user The user whose log entries are being processed
|
* @param string $user The user whose log entries are being processed
|
||||||
* @param PDOStatement $checkStmt A prepared statement to check if an entry
|
* @param PDOStatement $insertStmt A prepared statement for the idempotent
|
||||||
* already exists in the database
|
* insert (`INSERT … ON CONFLICT(id) DO NOTHING`)
|
||||||
* @param PDOStatement $insertStmt A prepared statement to insert a new entry
|
|
||||||
* into the database
|
|
||||||
* @param boolean $debug Whether to output debug information (default: false)
|
|
||||||
*
|
*
|
||||||
* @return void
|
* @return void
|
||||||
*/
|
*/
|
||||||
function processLogEntries($logEntries, $user, $checkStmt, $insertStmt, $debug) {
|
function processLogEntries($logEntries, $user, $insertStmt) {
|
||||||
foreach ($logEntries as $key =>$entry) {
|
foreach ($logEntries as $entry) {
|
||||||
if ($debug) {
|
$id = $entry['id'] ?? null;
|
||||||
$logMessage = "Raw entry:\n" . print_r($entry, true);
|
|
||||||
file_put_contents(__DIR__ . '/debug.log', $logMessage, FILE_APPEND);
|
if (!$id) {
|
||||||
|
consoleLog('Skipping entry with no id: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
$timestamp = $entry['timestamp'];
|
$timestamp = $entry['timestamp'] ?? null;
|
||||||
$description = $entry['title'];
|
$description = $entry['details']['title'] ?? null;
|
||||||
$amount = $entry['log'] === 5850 ? $entry['data']['deposited'] : -$entry['data']['withdrawn'];
|
$hasDeposit = isset($entry['data']['deposited']);
|
||||||
|
$hasWithdraw = isset($entry['data']['withdrawn']);
|
||||||
|
|
||||||
if ($debug) {
|
if ($hasDeposit) {
|
||||||
$logMessage = "Vault entry:\n\tUser: $user,\n\tTimestamp: $timestamp,\n\tDescription: $description,\n\tAmount: $amount\n";
|
$amount = (int)$entry['data']['deposited'];
|
||||||
file_put_contents(__DIR__ . '/debug.log', $logMessage, FILE_APPEND);
|
} elseif ($hasWithdraw) {
|
||||||
|
$amount = -((int)$entry['data']['withdrawn']);
|
||||||
|
} else {
|
||||||
|
consoleLog('Skipping entry ' . $id . ' with no deposited/withdrawn: ' . print_r($entry, true));
|
||||||
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
$checkStmt->execute(
|
if ($timestamp === null || $description === null) {
|
||||||
[
|
consoleLog('Skipping entry ' . $id . ' missing timestamp or details.title: ' . print_r($entry, true));
|
||||||
':user' => $user,
|
continue;
|
||||||
':timestamp' => $timestamp,
|
|
||||||
':amount' => $amount
|
|
||||||
]
|
|
||||||
);
|
|
||||||
|
|
||||||
if ($checkStmt->fetchColumn() == 0) {
|
|
||||||
$insertStmt->execute(
|
|
||||||
[
|
|
||||||
':id' => $key,
|
|
||||||
':user' => $user,
|
|
||||||
':timestamp' => $timestamp,
|
|
||||||
':description' => $description,
|
|
||||||
':amount' => $amount
|
|
||||||
]
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
$runningBalance = isset($entry['data']['balance']) ? (int)$entry['data']['balance'] : null;
|
||||||
|
|
||||||
|
$insertStmt->bindValue(':id', $id);
|
||||||
|
$insertStmt->bindValue(':user', $user);
|
||||||
|
$insertStmt->bindValue(':timestamp', $timestamp);
|
||||||
|
$insertStmt->bindValue(':description', $description);
|
||||||
|
$insertStmt->bindValue(':amount', $amount);
|
||||||
|
$insertStmt->bindValue(':running_balance', $runningBalance);
|
||||||
|
$insertStmt->execute();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -185,10 +172,10 @@ function fetchVaultRecords($user = null) {
|
|||||||
$pdo = getDatabaseConnection();
|
$pdo = getDatabaseConnection();
|
||||||
|
|
||||||
if ($user) {
|
if ($user) {
|
||||||
$query = "SELECT * FROM vault WHERE user = :user ORDER BY timestamp DESC";
|
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault WHERE user = :user ORDER BY timestamp DESC";
|
||||||
$params = [':user' => $user];
|
$params = [':user' => $user];
|
||||||
} else {
|
} else {
|
||||||
$query = "SELECT * FROM vault ORDER BY timestamp DESC";
|
$query = "SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC";
|
||||||
$params = [];
|
$params = [];
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -213,14 +200,19 @@ function fetchVaultRecords($user = null) {
|
|||||||
* @return int The total balance of the given user (or all users if no user is given).
|
* @return int The total balance of the given user (or all users if no user is given).
|
||||||
*/
|
*/
|
||||||
function vaultLoop ($name=null) {
|
function vaultLoop ($name=null) {
|
||||||
$vault = fetchVaultRecords($name);
|
$pdo = getDatabaseConnection();
|
||||||
$balance = 0;
|
|
||||||
|
|
||||||
foreach ($vault as $entry) {
|
if ($name === null) {
|
||||||
$balance += $entry['amount'];
|
$stmt = $pdo->query('SELECT COALESCE(SUM(amount), 0) FROM vault');
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
}
|
}
|
||||||
|
|
||||||
return $balance;
|
$stmt = $pdo->prepare('SELECT COALESCE(SUM(amount), 0) FROM vault WHERE user = :user');
|
||||||
|
$stmt->bindValue(':user', $name);
|
||||||
|
$stmt->execute();
|
||||||
|
|
||||||
|
return (int)$stmt->fetchColumn();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -238,3 +230,116 @@ function getSign($number) {
|
|||||||
|
|
||||||
return '$';
|
return '$';
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch a single page of log entries from the Torn v2 API and store them.
|
||||||
|
*
|
||||||
|
* Performs one HTTP request, validates the response, inserts each vault
|
||||||
|
* log entry (idempotently via INSERT ... ON CONFLICT), and returns the
|
||||||
|
* pagination cursor (`_metadata.links.next`) if more pages remain.
|
||||||
|
*
|
||||||
|
* @param PDO $pdo Database connection used to insert vault entries.
|
||||||
|
* @param string $user The user whose log entries are being fetched.
|
||||||
|
* @param string $url Full URL for the v2 API request.
|
||||||
|
*
|
||||||
|
* @throws ApiKeyMissingException If no API key is configured for the user.
|
||||||
|
* @throws CurlErrorException If the HTTP request fails.
|
||||||
|
* @throws JsonDataException If the response body is not valid JSON.
|
||||||
|
* @throws ApiValidationException If the response is missing the `log` array.
|
||||||
|
*
|
||||||
|
* @return string|null The `_metadata.links.next` URL, or null when there are no more pages.
|
||||||
|
*/
|
||||||
|
function fetchAndStoreLogPage($pdo, $user, $url) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
throw new ApiKeyMissingException("User does not have an API key configured.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$insertStmt = $pdo->prepare(
|
||||||
|
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
|
||||||
|
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
|
||||||
|
. 'ON CONFLICT(id) DO UPDATE SET running_balance = EXCLUDED.running_balance, amount = EXCLUDED.amount'
|
||||||
|
);
|
||||||
|
|
||||||
|
do {
|
||||||
|
$responseData = executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
validateApiResponse($responseData);
|
||||||
|
|
||||||
|
processLogEntries($responseData['log'], $user, $insertStmt);
|
||||||
|
|
||||||
|
// Follow the older-pages chain so the very first call (which gets the
|
||||||
|
// most recent page) also retrieves every older entry. After the prev
|
||||||
|
// chain is exhausted, fall back to the newer-pages chain in case the
|
||||||
|
// caller passed a `from=` URL and the result has a next link.
|
||||||
|
$url = $responseData['_metadata']['links']['prev']
|
||||||
|
?? $responseData['_metadata']['links']['next']
|
||||||
|
?? null;
|
||||||
|
} while ($url !== null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Print a variable to the console for debugging purposes.
|
||||||
|
*
|
||||||
|
* @param mixed $data The data to print to the console.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function consoleLog( $data ) {
|
||||||
|
echo '<script>';
|
||||||
|
echo 'console.log(' . json_encode($data) . ')';
|
||||||
|
echo '</script>';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Re-fetch the running_balance for each entry of a user by walking the
|
||||||
|
* v2 /log paginated endpoint.
|
||||||
|
*
|
||||||
|
* For each entry the API returns, we run a single UPDATE setting that
|
||||||
|
* entry's `running_balance` to its `data.balance`. The pre-existing
|
||||||
|
* broadcast approach gave every row of a user the same value, which
|
||||||
|
* made the historical view misleading; this restores per-entry accuracy.
|
||||||
|
*
|
||||||
|
* If the user has zero rows with `running_balance IS NULL`, returns
|
||||||
|
* without making any HTTP call. On API failure, the underlying
|
||||||
|
* exception propagates to the caller's try/catch in index.php.
|
||||||
|
*
|
||||||
|
* @param PDO $pdo Database connection.
|
||||||
|
* @param string $user The user whose NULL rows should be refilled.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function refetchRunningBalances($pdo, $user) {
|
||||||
|
if (!array_key_exists($user, USER_KEYS)) {
|
||||||
|
throw new ApiKeyMissingException("User does not have an API key configured.");
|
||||||
|
}
|
||||||
|
|
||||||
|
$countStmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user AND running_balance IS NULL');
|
||||||
|
$countStmt->bindValue(':user', $user);
|
||||||
|
$countStmt->execute();
|
||||||
|
|
||||||
|
if ((int)$countStmt->fetchColumn() === 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$updateStmt = $pdo->prepare(
|
||||||
|
'UPDATE vault SET running_balance = :balance WHERE id = :id'
|
||||||
|
);
|
||||||
|
|
||||||
|
$url = 'https://api.torn.com/v2/user?selections=log&log=5850,5851';
|
||||||
|
do {
|
||||||
|
$responseData = executeApiCall($url, USER_KEYS[$user]);
|
||||||
|
validateApiResponse($responseData);
|
||||||
|
|
||||||
|
foreach ($responseData['log'] as $entry) {
|
||||||
|
$id = $entry['id'] ?? null;
|
||||||
|
if (!$id || !isset($entry['data']['balance'])) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$updateStmt->bindValue(':balance', (int)$entry['data']['balance']);
|
||||||
|
$updateStmt->bindValue(':id', $id);
|
||||||
|
$updateStmt->execute();
|
||||||
|
}
|
||||||
|
|
||||||
|
$url = $responseData['_metadata']['links']['next'] ?? null;
|
||||||
|
} while ($url !== null);
|
||||||
|
}
|
||||||
|
|||||||
@@ -13,15 +13,49 @@
|
|||||||
require_once __DIR__ . '/config.php';
|
require_once __DIR__ . '/config.php';
|
||||||
require_once __DIR__ . '/functions.php';
|
require_once __DIR__ . '/functions.php';
|
||||||
|
|
||||||
if (dbNew()) {
|
$bootstrapWarning = null;
|
||||||
|
$needsBackfill = false;
|
||||||
|
|
||||||
|
try {
|
||||||
foreach (USER_KEYS as $key => $value) {
|
foreach (USER_KEYS as $key => $value) {
|
||||||
firstRun($key);
|
syncUserLogs($key);
|
||||||
}
|
// If the sync didn't populate this user, backfill is still needed.
|
||||||
header('Location: /');
|
$pdo = getDatabaseConnection();
|
||||||
} else {
|
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
|
||||||
foreach (USER_KEYS as $key => $value) {
|
$stmt->bindValue(':user', $key);
|
||||||
getLog($key);
|
$stmt->execute();
|
||||||
|
if ((int)$stmt->fetchColumn() === 0) {
|
||||||
|
$needsBackfill = true;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
} catch (Exception $e) {
|
||||||
|
$bootstrapWarning = $e->getMessage();
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($needsBackfill) {
|
||||||
|
// Heavy work remains. Return a loading page that auto-refreshes;
|
||||||
|
// each refresh does another page of API fetches.
|
||||||
|
?>
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<title>Torn Vault Tracker — loading</title>
|
||||||
|
<meta http-equiv="refresh" content="3">
|
||||||
|
<link rel="icon" type="image/png" href="favicon.png">
|
||||||
|
<link rel="stylesheet" href="style.css?v=<?php echo filemtime('style.css'); ?>">
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<h1>Loading vault history…</h1>
|
||||||
|
<?php if ($bootstrapWarning !== null): ?>
|
||||||
|
<p>Warning: <?php echo htmlspecialchars($bootstrapWarning, ENT_QUOTES, 'UTF-8'); ?></p>
|
||||||
|
<?php endif; ?>
|
||||||
|
<p>One-time backfill in progress. The Torn API returns up to 100 vault entries per call;
|
||||||
|
we'll keep fetching until your full history is loaded. This page will refresh automatically.</p>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
<?php
|
||||||
|
exit;
|
||||||
}
|
}
|
||||||
?>
|
?>
|
||||||
|
|
||||||
@@ -45,6 +79,10 @@ if (dbNew()) {
|
|||||||
<body>
|
<body>
|
||||||
<h1>Torn Vault Tracker</h1>
|
<h1>Torn Vault Tracker</h1>
|
||||||
|
|
||||||
|
<?php if ($bootstrapWarning !== null): ?>
|
||||||
|
<div class="warning" style="background:#f77e82;color:#000;padding:.5em 1em;margin:1em 0;border:1px solid #333;">Warning: failed to fetch latest logs from Torn — showing cached data. (<?php echo htmlspecialchars($bootstrapWarning, ENT_QUOTES, 'UTF-8'); ?>)</div>
|
||||||
|
<?php endif; ?>
|
||||||
|
|
||||||
<div class="grid">
|
<div class="grid">
|
||||||
<div class="gridLeft">
|
<div class="gridLeft">
|
||||||
<h1>Transactions</h1>
|
<h1>Transactions</h1>
|
||||||
@@ -56,6 +94,7 @@ if (dbNew()) {
|
|||||||
<th><h4>Date / Time (TCT)</h4></th>
|
<th><h4>Date / Time (TCT)</h4></th>
|
||||||
<th><h4>Operation</h4></th>
|
<th><h4>Operation</h4></th>
|
||||||
<th><h4>Amount</h4></th>
|
<th><h4>Amount</h4></th>
|
||||||
|
<th><h4>Balance After</h4></th>
|
||||||
</tr>
|
</tr>
|
||||||
</thead>
|
</thead>
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,130 @@
|
|||||||
|
<?php
|
||||||
|
/**
|
||||||
|
* Golden-file test for processLogEntries() v2 behavior.
|
||||||
|
*
|
||||||
|
* Loads backup/api-sample-new.json, runs each entry through the v2
|
||||||
|
* processor against an in-memory SQLite database, and asserts the
|
||||||
|
* resulting rows match the expected (id, user, timestamp, description,
|
||||||
|
* amount) tuples derived from the fixture.
|
||||||
|
*
|
||||||
|
* PHP version 8.1+
|
||||||
|
*
|
||||||
|
* Run: php tests/process_log_entries_test.php
|
||||||
|
* Exit code 0 = pass.
|
||||||
|
*
|
||||||
|
* @category Test
|
||||||
|
* @package TornVaultTracker
|
||||||
|
* @author Keith Solomon <ksolo@gmail.com>
|
||||||
|
* @license Unlicense https://unlicense.org/
|
||||||
|
* @link https://github.com/ksolo/Torn-Vault-Tracker
|
||||||
|
*/
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
// Make USER_KEYS available without including config.php (it would force a
|
||||||
|
// real DB connection).
|
||||||
|
if (!defined('USER_KEYS')) {
|
||||||
|
define(
|
||||||
|
'USER_KEYS', [
|
||||||
|
'zarathos' => 'test-key-not-used',
|
||||||
|
'symos' => 'test-key-not-used',
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
require_once __DIR__ . '/../includes/exceptions.php';
|
||||||
|
require_once __DIR__ . '/../includes/utilities.php';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Print a failure message to STDERR and exit with status 1.
|
||||||
|
*
|
||||||
|
* @param string $message Human-readable failure description.
|
||||||
|
*
|
||||||
|
* @return never
|
||||||
|
*/
|
||||||
|
function fail(string $message): void {
|
||||||
|
fwrite(STDERR, "FAIL: $message\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Lightweight assertion helper that mirrors PHPUnit's assertSame().
|
||||||
|
*
|
||||||
|
* @param mixed $expected The expected value.
|
||||||
|
* @param mixed $actual The actual value.
|
||||||
|
* @param string $label Human-readable label for failure messages.
|
||||||
|
*
|
||||||
|
* @return void
|
||||||
|
*/
|
||||||
|
function assertSame($expected, $actual, string $label): void {
|
||||||
|
if ($expected !== $actual) {
|
||||||
|
$exp = var_export($expected, true);
|
||||||
|
$act = var_export($actual, true);
|
||||||
|
fail("$label: expected $exp, got $act");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// In-memory DB so we don't touch the user's vault.db.
|
||||||
|
$pdo = new PDO('sqlite::memory:');
|
||||||
|
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
|
||||||
|
$pdo->exec(
|
||||||
|
'CREATE TABLE vault (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
user TEXT NOT NULL,
|
||||||
|
timestamp INTEGER NOT NULL,
|
||||||
|
description TEXT NOT NULL,
|
||||||
|
amount REAL NOT NULL,
|
||||||
|
running_balance INTEGER
|
||||||
|
)'
|
||||||
|
);
|
||||||
|
|
||||||
|
$insertStmt = $pdo->prepare(
|
||||||
|
'INSERT INTO vault (id, user, timestamp, description, amount, running_balance) '
|
||||||
|
. 'VALUES (:id, :user, :timestamp, :description, :amount, :running_balance) '
|
||||||
|
. 'ON CONFLICT(id) DO NOTHING'
|
||||||
|
);
|
||||||
|
|
||||||
|
// Load the golden file.
|
||||||
|
$fixturePath = __DIR__ . '/../backup/api-sample-new.json';
|
||||||
|
if (!is_readable($fixturePath)) {
|
||||||
|
fail("Fixture not readable at $fixturePath");
|
||||||
|
}
|
||||||
|
$fixture = json_decode(file_get_contents($fixturePath), true);
|
||||||
|
if (!is_array($fixture) || !isset($fixture['log'])) {
|
||||||
|
fail('Fixture is missing the log array.');
|
||||||
|
}
|
||||||
|
|
||||||
|
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
|
||||||
|
|
||||||
|
// 100 entries in the fixture.
|
||||||
|
$count = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
|
||||||
|
assertSame(100, $count, 'row count');
|
||||||
|
|
||||||
|
// Spot-check first entry (newest, withdraw) and last (oldest, deposit).
|
||||||
|
$rows = $pdo->query('SELECT id, user, timestamp, description, amount, running_balance FROM vault ORDER BY timestamp DESC')->fetchAll(PDO::FETCH_ASSOC);
|
||||||
|
|
||||||
|
$first = $rows[0];
|
||||||
|
assertSame('c1pEDQ7jl3kuBHV4NqDI', $first['id'], 'first.id');
|
||||||
|
assertSame('zarathos', $first['user'], 'first.user');
|
||||||
|
assertSame(1785325489, (int)$first['timestamp'], 'first.timestamp');
|
||||||
|
assertSame('Vault withdraw', $first['description'], 'first.description');
|
||||||
|
assertSame(-498870964, (int)$first['amount'], 'first.amount');
|
||||||
|
|
||||||
|
$last = $rows[count($rows) - 1];
|
||||||
|
assertSame('8m5jrUKTiKk0SB4MxzIy', $last['id'], 'last.id');
|
||||||
|
assertSame(1714350415, (int)$last['timestamp'], 'last.timestamp');
|
||||||
|
assertSame('Vault deposit', $last['description'], 'last.description');
|
||||||
|
assertSame(231492, (int)$last['amount'], 'last.amount');
|
||||||
|
|
||||||
|
// Re-running with the same fixture inserts zero new rows (idempotency).
|
||||||
|
processLogEntries($fixture['log'], 'zarathos', $insertStmt);
|
||||||
|
$count2 = (int)$pdo->query('SELECT COUNT(*) FROM vault')->fetchColumn();
|
||||||
|
assertSame(100, $count2, 'row count after re-run');
|
||||||
|
|
||||||
|
// round-trip the running_balance column from $entry['data']['balance']
|
||||||
|
$firstRunning = (int)$first['running_balance'];
|
||||||
|
$lastRunning = (int)$last['running_balance'];
|
||||||
|
assertSame(69235831, $firstRunning, 'first.running_balance');
|
||||||
|
assertSame(509622821, $lastRunning, 'last.running_balance');
|
||||||
|
|
||||||
|
echo "OK: processLogEntries v2 golden-file test passed (100 entries).\n";
|
||||||
Reference in New Issue
Block a user