# Async Backfill with Meta-Refresh — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Prevent PHP execution-timeout during the first-time vault backfill (which may make ~25 HTTP calls to the Torn API and exceed host limits). On bootstrap, return a "Loading..." page immediately that auto-refreshes; each refresh does a small chunk of work until the DB is fully populated.
**Architecture:** `index.php` first runs the small/fast sync (`syncUserLogs` for users with rows, which does 1-2 HTTP calls). If after that, any user has zero rows in the DB (i.e., the bootstrap case), `index.php` returns a minimal HTML page with a meta-refresh tag pointing to the same URL after a short delay. The next page-load re-runs the sync, which now does ~24 more HTTP calls (one page of entries per refresh). Eventually the DB is populated and the page renders the real UI.
**Tech Stack:** PHP 8.1+, SQLite3 via PDO, cURL. No new dependencies.
## Global Constraints
- PHP 8.1+ syntax only.
- Commits per task. Do not push.
- Other helpers stay byte-identical unless the task explicitly says otherwise.
---
### Task 1: Add meta-refresh bootstrap to `index.php`
**Files:**
- Modify: `index.php` (top-level bootstrap only)
**Interfaces:**
- Page-load behavior:
- **Steady state (DB has rows):** Existing behavior — run sync, render the page. Unchanged.
- **Bootstrap (DB empty for some user):** Page-load runs the fast sync. If a user still has zero rows after the fast sync, return a minimal HTML "Loading..." page that meta-refreshes to itself after 3 seconds.
- The meta-refresh runs `syncUserLogs`, which uses `from=lastTs+1` to fetch only new entries. On a fresh DB this fetches nothing (lastTs=0 → 0+1=1, but the API may give a strange response). So the actual heavy lifting must happen via the bootstrap case separately.
**Strategy:** The page-load runs `backfillUserLogs` (the heavy function) ONLY if it would complete quickly. If it would be slow (heuristic: the user has fewer than some threshold of rows), defer it.
Concretely: define a helper `isBackfillInProgress(): bool` that returns true if any user has 0 rows OR the user has fewer than e.g. 50 rows. The page-load only runs `backfillUserLogs` if that returns false (i.e., the DB looks fully populated for that user). If it returns true, return a loading page.
But that's not quite right either — we want to make PROGRESS even if not "complete." Let me re-think.
**Revised strategy:** Always run the sync, but cap the number of pages we walk per page-load. If the sync completed (all users' rows look complete), render the page normally. Otherwise return a loading page that will continue the sync on next refresh.
- [ ] **Step 1: Add the helper functions to `index.php`**
In `index.php`, replace the top of the file (after `require_once __DIR__ . '/functions.php';`) to add a `stillNeedBackfill` helper and bootstrap logic:
```php
require_once __DIR__ . '/config.php';
require_once __DIR__ . '/functions.php';
$bootstrapWarning = null;
$needsBackfill = false;
try {
foreach (USER_KEYS as $key => $value) {
syncUserLogs($key);
// If the sync didn't populate this user, backfill is still needed.
$pdo = getDatabaseConnection();
$stmt = $pdo->prepare('SELECT COUNT(*) FROM vault WHERE user = :user');
$stmt->bindValue(':user', $key);
$stmt->execute();
if ((int)$stmt->fetchColumn() === 0) {
$needsBackfill = true;
}
}
} catch (Exception $e) {
$bootstrapWarning = $e->getMessage();
}
if ($needsBackfill) {
// Heavy work remains. Return a loading page that auto-refreshes;
// each refresh does another page of API fetches.
?>
Torn Vault Tracker — loading
Loading vault history…
Warning:
One-time backfill in progress. The Torn API returns up to 100 vault entries per call;
we'll keep fetching until your full history is loaded. This page will refresh automatically.