feat: implement newsletter browser UI and enhance link filtering functionality

Reviewed-on: #1

.gitignore

@@ -2,3 +2,4 @@ node_modules
 dist
 config.yaml
 output
+data

README.md

@@ -11,6 +11,7 @@ nlc run
 nlc run --from 2026-05-01 --to 2026-05-16
 nlc run --last 30d
 nlc run --enrich-only
+nlc serve
 ```
 
 ## Setup
@@ -22,6 +23,8 @@ nlc run --enrich-only
    - `~/.nlc/gmail-credentials.json`
    - `~/.nlc/sheets-credentials.json`
 5. Run `node dist/index.js run --dry-run` before live writes.
+6. Run `node dist/index.js run` to import into SQLite.
+7. Run `node dist/index.js serve` and open <http://127.0.0.1:3000>.
 
 Tokens are persisted locally under `~/.nlc` and must not be committed.
 
@@ -81,9 +84,26 @@ Start from [config.example.yaml](config.example.yaml). The important choices are
 - `gmail.folder`: the single Gmail label/folder to process.
 - `output.excel.enabled`: writes a local `.xlsx` file.
 - `output.sheets_api.enabled`: enables Google Sheets integration when credentials and spreadsheet ID are configured.
+- `database.enabled`: writes SQLite catalog data during `nlc run`; defaults to `true`.
+- `database.path`: SQLite database path; defaults to `./data/newsletter-catalog.sqlite`.
 - `links.tracking_params`: query parameters stripped during URL normalization.
 - `categories.llm`: optional BYOK categorization provider.
 
+## SQLite and Web App
+
+SQLite is the default catalog store. `nlc run` writes imported links, sponsors, dead links, and run history to the configured database even when spreadsheet outputs are disabled.
+
+Start the local read-only web app with:
+
+```bash
+nlc serve --host 127.0.0.1 --port 3000
+```
+
+The web UI is intentionally functional and read-only. It includes dashboard totals, a two-pane
+newsletter browser, all links, sponsored links, dead links, and run history. The newsletter browser
+opens on the most recent issue to keep the default view focused; switch to **All Issues** when you
+want the full link history for the selected newsletter.
+
 ## Build and Distribution
 
 The build uses `tsup` for the JavaScript bundle and `@yao-pkg/pkg` for the standalone executable:

config.example.yaml

@@ -14,6 +14,10 @@ output:
     enabled: true
     path: './output/newsletter-catalog.xlsx'
 
+database:
+  enabled: true
+  path: './data/newsletter-catalog.sqlite'
+
 newsletters:
   'sender@example.com':
     display_name: 'Example Newsletter'

docs/superpowers/plans/2026-05-17-newsletter-ui.md

@@ -0,0 +1,70 @@
+# Newsletter UI Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Replace the rough web views with a functional two-pane newsletter browser that defaults to the latest issue and can switch to all issues.
+
+**Architecture:** Keep the current Express/server-rendered UI. Add focused SQLite read methods to `CatalogDatabase`, render reusable layout/navigation/table helpers in `src/web/views.ts`, and wire new routes in `src/web/app.ts` for dashboard, newsletter browser, all links, sponsors, dead links, and runs. Use query parameters for selected newsletter, issue scope, link search, and category filters.
+
+**Tech Stack:** TypeScript, Express, Node `node:sqlite`, server-rendered HTML/CSS, Vitest.
+
+---
+
+### Task 1: Database Read Models
+
+**Files:**
+
+- Modify: `src/database/store.ts`
+- Test: `tests/web.test.ts`
+
+- [ ] Write failing tests that a fixture database can return newsletter summaries with issue/link/sponsor counts.
+- [ ] Write failing tests that selected newsletter links default to the latest issue and can include all issues.
+- [ ] Add `newsletterSummaries()`, `newsletterById()`, `newsletterLinks()`, `categoriesForNewsletter()`, and `allLinks()` read methods.
+- [ ] Run `npm test -- tests/web.test.ts`.
+
+### Task 2: Two-Pane Newsletter Page
+
+**Files:**
+
+- Modify: `src/web/views.ts`
+- Modify: `src/web/app.ts`
+- Test: `tests/web.test.ts`
+
+- [ ] Write failing tests that `/newsletters` renders a two-pane layout, newsletter search input, selected state, latest/all issue controls, category filter, and external link attributes.
+- [ ] Implement reusable page layout with dark default styling, light mode toggle, global nav, responsive two-pane layout, and empty states.
+- [ ] Implement `/newsletters` with query params: `newsletter`, `scope`, `q`, `category`.
+- [ ] Run `npm test -- tests/web.test.ts`.
+
+### Task 3: Global Views and Smoke
+
+**Files:**
+
+- Modify: `src/web/app.ts`
+- Modify: `src/web/views.ts`
+- Modify: `scripts/smoke.mjs`
+- Test: `tests/web.test.ts`
+
+- [ ] Write failing tests that dashboard, all links, sponsored links, dead links, and runs use the new layout.
+- [ ] Update `/`, `/links`, `/sponsors`, `/dead-links`, and `/runs` to use the shared layout and table helpers.
+- [ ] Keep `nlc serve --help` in smoke validation.
+- [ ] Run `npm test -- tests/web.test.ts`.
+
+### Task 4: Docs and Verification
+
+**Files:**
+
+- Add: `notes/UI Spec.md`
+- Modify: `README.md`
+
+- [ ] Track the approved UI spec.
+- [ ] Document `nlc serve` and the newsletter browser behavior.
+- [ ] Run `npm run lint`.
+- [ ] Run `npm run format:check`.
+- [ ] Run `npm run typecheck`.
+- [ ] Run `npm test`.
+- [ ] Run `npm run build`.
+- [ ] Run `npm run smoke`.
+
+## Self-Review
+
+The plan covers the approved UI spec: two-pane layout, newsletter navigation, latest issue default, all-issues option, global navigation, search/filter controls, dark default, light option, mobile-friendly responsive behavior, and safe external links. It avoids a frontend build stack and keeps the first implementation functional.

docs/superpowers/plans/2026-05-17-sqlite-web-app.md

@@ -0,0 +1,86 @@
+# SQLite Web App Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Make SQLite the default catalog store and add a functional local web app for browsing imported newsletter data.
+
+**Architecture:** Use Node's built-in `node:sqlite` `DatabaseSync` API for a packaged-executable-friendly SQLite layer. Keep the existing Gmail ingestion pipeline and spreadsheet writers, add a `DatabaseWriter` as the default writer, and add an Express server that renders read-only catalog pages from SQLite.
+
+**Tech Stack:** TypeScript, Node.js `node:sqlite`, Express, Commander, Vitest, existing parser/output modules.
+
+---
+
+### Task 1: Config Defaults
+
+**Files:**
+
+- Modify: `src/config/config.ts`
+- Modify: `config.example.yaml`
+- Test: `tests/config.test.ts`
+
+- [ ] Write a failing test that `database.enabled` defaults to `true` and `database.path` defaults to `./data/newsletter-catalog.sqlite`.
+- [ ] Add the database schema to config validation.
+- [ ] Add `database` to `config.example.yaml`.
+- [ ] Run `npm test -- tests/config.test.ts`.
+
+### Task 2: SQLite Schema and Store
+
+**Files:**
+
+- Create: `src/database/schema.ts`
+- Create: `src/database/store.ts`
+- Test: `tests/database.test.ts`
+
+- [ ] Write failing tests that migrations create `newsletters`, `issues`, `links`, `link_occurrences`, `sponsors`, `dead_links`, and `runs`.
+- [ ] Write failing tests that saving a catalog payload deduplicates canonical URLs and stores occurrences per issue.
+- [ ] Implement migration and transactional persistence with `node:sqlite`.
+- [ ] Run `npm test -- tests/database.test.ts`.
+
+### Task 3: Default Database Writer
+
+**Files:**
+
+- Create: `src/output/databaseWriter.ts`
+- Modify: `src/run/runCatalog.ts`
+- Modify: `src/cli/program.ts`
+- Test: `tests/run.test.ts`
+
+- [ ] Write a failing test that `runCatalog` can write rows to a database writer and still preserves spreadsheet writer behavior.
+- [ ] Add a `DatabaseWriter` implementing the existing `OutputWriter` interface.
+- [ ] Make `nlc run` add `DatabaseWriter` by default when `database.enabled` is true.
+- [ ] Run `npm test -- tests/run.test.ts tests/database.test.ts`.
+
+### Task 4: Web App
+
+**Files:**
+
+- Create: `src/web/app.ts`
+- Create: `src/web/views.ts`
+- Modify: `src/cli/program.ts`
+- Test: `tests/web.test.ts`
+
+- [ ] Write failing tests that dashboard, links, sponsored links, dead links, and runs routes render data from a fixture database.
+- [ ] Add `nlc serve --config --host --port`.
+- [ ] Implement read-only Express routes and simple server-rendered HTML.
+- [ ] Run `npm test -- tests/web.test.ts`.
+
+### Task 5: Docs and Verification
+
+**Files:**
+
+- Modify: `README.md`
+- Modify: `package.json`
+
+- [ ] Document SQLite default storage and `nlc serve`.
+- [ ] Add Express dependencies and update lockfile.
+- [ ] Run `npm install`.
+- [ ] Run `npm run lint`.
+- [ ] Run `npm run format:check`.
+- [ ] Run `npm run typecheck`.
+- [ ] Run `npm test`.
+- [ ] Run `npm run build`.
+- [ ] Run `npm run smoke`.
+
+## Self-Review
+
+This plan covers all accepted spec requirements: SQLite defaults, schema, default import persistence, optional spreadsheet output, `nlc serve`, functional catalog views, docs, and automated tests. No unresolved placeholders remain. Types and paths align with the existing project structure.

docs/superpowers/specs/2026-05-17-sqlite-web-app-design.md

@@ -0,0 +1,135 @@
+# SQLite Store and Web App Design
+
+## Goal
+
+Move Newsletter Link Catalog from spreadsheet-first output to a SQLite-first local catalog, while preserving spreadsheet output as an optional export/sync path. Add a functional local web app for browsing the catalog.
+
+## Scope
+
+This first pass keeps the working CLI ingestion flow. `nlc run` continues to fetch Gmail newsletters, parse links, categorize, enrich when enabled, and update incremental state. The main behavior change is that extracted data is persisted to SQLite by default.
+
+The web app is intentionally plain and functional. It should make the catalog useful before investing in visual polish.
+
+## Architecture
+
+Add a persistence layer alongside the existing output writers:
+
+- `DatabaseWriter` persists run results into SQLite.
+- Spreadsheet writers remain available through the existing config-driven write path during `nlc run`.
+- Web routes read from SQLite and do not re-run Gmail ingestion.
+- Existing parsing, filtering, categorization, enrichment, state, and OAuth modules stay in place.
+
+The CLI remains the control surface for ingestion:
+
+- `nlc run` writes to SQLite by default.
+- `nlc run` may also write Excel and/or Google Sheets if configured.
+- `nlc serve` starts a local web server against the configured SQLite database.
+
+## SQLite Database
+
+Default database path: `./data/newsletter-catalog.sqlite`, configurable as `database.path`.
+
+Tables:
+
+- `newsletters`: newsletter identity and display name.
+- `issues`: one row per processed email issue, linked to a newsletter.
+- `links`: canonical URL-level records.
+- `link_occurrences`: per-issue link appearances, including category, title, description, page metadata, and also-in text.
+- `sponsors`: sponsor/ad appearances, linked to issue and URL where possible.
+- `dead_links`: dead or unreachable link records.
+- `runs`: import run history with counts, timestamps, mode, and error counts.
+
+Constraints:
+
+- URLs are deduplicated in `links` by normalized URL.
+- Link occurrences are deduplicated per issue by normalized URL.
+- Processed Gmail message IDs remain tracked by the existing state file for now, avoiding a risky migration in this first pass.
+
+## Configuration
+
+Extend `config.yaml`:
+
+```yaml
+database:
+  enabled: true
+  path: './data/newsletter-catalog.sqlite'
+```
+
+Behavior:
+
+- If `database.enabled` is omitted, it defaults to `true`.
+- Spreadsheet output remains controlled by the existing `output.excel.enabled` and `output.sheets_api.enabled` settings.
+- If all outputs are disabled, `nlc run` still writes to SQLite.
+
+## Web App
+
+Add `nlc serve [flags]`.
+
+Flags:
+
+- `--config PATH`, default `./config.yaml`
+- `--host HOST`, default `127.0.0.1`
+- `--port PORT`, default `3000`
+
+Views:
+
+- Dashboard: counts for newsletters, issues, links, sponsors, dead links, and recent runs.
+- Links: searchable table of content links with filters for newsletter, category, and date range.
+- Newsletter detail: issues and links for one newsletter.
+- Sponsored links: global sponsor table.
+- Dead links: global dead/unreachable table.
+- Runs: recent import run summaries.
+
+The first UI can use server-rendered HTML with minimal CSS. No SPA framework is required.
+
+## Data Flow
+
+Import:
+
+1. Gmail fetches configured label messages.
+2. Existing parser extracts links and sponsor entries.
+3. Existing cleanup/categorization/enrichment pipeline prepares rows.
+4. `DatabaseWriter` upserts newsletters, issues, links, occurrences, sponsors, dead links, and run summary.
+5. Optional spreadsheet writers run after database persistence when configured.
+
+Web:
+
+1. `nlc serve` opens the SQLite database.
+2. Request handlers query read-only catalog views.
+3. HTML pages render tables and simple filters.
+
+## Error Handling
+
+- Database open/migration errors are critical and stop `nlc run` or `nlc serve`.
+- A single failed row insert during import increments the run error count and stops the database transaction for that run.
+- SQLite writes use transactions so partial import data is not committed on failure.
+- Web route errors return a simple 500 page and log the error.
+
+## Testing
+
+Add tests for:
+
+- Config defaults for `database.enabled` and `database.path`.
+- Schema migration creates expected tables.
+- Database writer inserts newsletters, issues, links, sponsors, and dead links.
+- Duplicate normalized URLs are deduplicated while occurrences remain per issue.
+- `nlc run` writes to SQLite by default even when spreadsheet outputs are disabled.
+- Web routes render dashboard, links, sponsored links, dead links, and run history using a fixture database.
+- Existing spreadsheet writer tests continue to pass.
+
+## Out of Scope for This Pass
+
+- Browser-based Gmail OAuth setup.
+- Running Gmail imports from a web button.
+- User accounts or remote hosting.
+- Full visual design polish.
+- Migrating old JSON state into SQLite.
+- Removing Excel or Google Sheets support.
+
+## Acceptance Criteria
+
+- `nlc run` creates/updates the SQLite database by default.
+- `nlc serve` starts a local web app and displays imported catalog data.
+- Spreadsheet output still works when enabled.
+- Existing CLI validations remain green.
+- Tests cover database persistence and web read paths without live Gmail or Google credentials.

notes/UI Spec.md

@@ -0,0 +1,88 @@
+# Newsletter Link Catalog UI Spec
+
+## Goal
+
+Create a clean, functional web UI for browsing the SQLite-backed newsletter catalog. The UI should feel like a focused database browser: fast to scan, easy to filter, and useful before visual polish.
+
+## Layout
+
+Use a two-pane catalog layout.
+
+### Left Pane: Newsletter Navigation
+
+- Narrow column on desktop.
+- Contains a newsletter search field at the top.
+- Shows a scrollable list of newsletters below the search.
+- Each newsletter item should show:
+  - Newsletter name.
+  - Optional short description or source email when available.
+  - Compact counts such as issues, links, and sponsored links.
+- The currently selected newsletter should have a clear selected state.
+
+### Right Pane: Newsletter Detail
+
+- Shows details for the selected newsletter.
+- Defaults to the most recent issue only to reduce cognitive load.
+- Header should show:
+  - Newsletter name.
+  - Optional description/source email.
+  - Latest issue date.
+  - Counts for the displayed issue.
+- Links should be displayed in a scannable table or grouped list.
+- Link rows should include:
+  - Title.
+  - Category.
+  - Description.
+  - URL.
+  - Also In, when available.
+- Links should be clickable and open in a new tab using `target="_blank"` and `rel="noopener noreferrer"`.
+
+## Issue Scope Controls
+
+The right pane defaults to **Latest Issue**.
+
+Provide a clear control to switch to **All Issues** for the selected newsletter. In All Issues mode:
+
+- Show links from every issue for the selected newsletter.
+- Include issue date on each row.
+- Keep the same search/filter controls.
+- Make it easy to return to Latest Issue mode.
+
+## Global Navigation
+
+Provide lightweight navigation for:
+
+- Dashboard.
+- Newsletters.
+- All Links.
+- Sponsored Links.
+- Dead Links.
+- Runs.
+
+The Newsletter view should be the primary working screen.
+
+## Search and Filters
+
+- Newsletter search filters the left pane immediately.
+- Link search filters the right pane within the selected newsletter and current issue scope.
+- Category filtering should be available in the right pane.
+- Date filtering is only needed in All Issues mode.
+- Empty states should explain what is missing:
+  - No newsletters imported yet.
+  - No links for the selected newsletter.
+  - No matches for the current search/filter.
+
+## Theme and Responsiveness
+
+- Dark mode should be the default visual direction.
+- Respect system preference where practical.
+- Provide a light mode option.
+- Mobile layout should collapse the left pane into a drawer, dropdown, or stacked selector.
+- The UI should remain usable on small screens without horizontal scrolling for core actions.
+
+## Visual Style
+
+- Clean, restrained, and operational.
+- Avoid marketing-style hero sections.
+- Prioritize readable tables/lists, clear selected states, and compact metadata.
+- UI polish can remain minimal for the first implementation, but spacing, contrast, and text wrapping should be professional.

package-lock.json

@@ -12,6 +12,7 @@
         "@commander-js/extra-typings": "^12.1.0",
         "cheerio": "^1.0.0",
         "commander": "^12.1.0",
+        "express": "^5.2.1",
         "googleapis": "^140.0.1",
         "ora": "^8.1.1",
         "xlsx": "^0.18.5",
@@ -22,6 +23,7 @@
         "nlc": "dist/index.js"
       },
       "devDependencies": {
+        "@types/express": "^5.0.6",
         "@types/node": "^22.9.0",
         "@typescript-eslint/eslint-plugin": "^8.11.0",
         "@typescript-eslint/parser": "^8.11.0",
@@ -1244,6 +1246,27 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/@types/body-parser": {
+      "version": "1.19.6",
+      "resolved": "https://registry.npmjs.org/@types/body-parser/-/body-parser-1.19.6.tgz",
+      "integrity": "sha512-HLFeCYgz89uk22N5Qg3dvGvsv46B8GLvKKo1zKG4NybA8U2DiEO3w9lqGg29t/tfLRJpJ6iQxnVw4OnB7MoM9g==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/connect": "*",
+        "@types/node": "*"
+      }
+    },
+    "node_modules/@types/connect": {
+      "version": "3.4.38",
+      "resolved": "https://registry.npmjs.org/@types/connect/-/connect-3.4.38.tgz",
+      "integrity": "sha512-K6uROf1LD88uDQqJCktA4yzL1YYAK6NgfsI0v/mTgyPKWsX1CnJ0XPSDhViejru1GcRkLWb8RlzFYJRqGUbaug==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/node": "*"
+      }
+    },
     "node_modules/@types/estree": {
       "version": "1.0.8",
       "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz",
@@ -1251,6 +1274,38 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/@types/express": {
+      "version": "5.0.6",
+      "resolved": "https://registry.npmjs.org/@types/express/-/express-5.0.6.tgz",
+      "integrity": "sha512-sKYVuV7Sv9fbPIt/442koC7+IIwK5olP1KWeD88e/idgoJqDm3JV/YUiPwkoKK92ylff2MGxSz1CSjsXelx0YA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/body-parser": "*",
+        "@types/express-serve-static-core": "^5.0.0",
+        "@types/serve-static": "^2"
+      }
+    },
+    "node_modules/@types/express-serve-static-core": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/@types/express-serve-static-core/-/express-serve-static-core-5.1.1.tgz",
+      "integrity": "sha512-v4zIMr/cX7/d2BpAEX3KNKL/JrT1s43s96lLvvdTmza1oEvDudCqK9aF/djc/SWgy8Yh0h30TZx5VpzqFCxk5A==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/node": "*",
+        "@types/qs": "*",
+        "@types/range-parser": "*",
+        "@types/send": "*"
+      }
+    },
+    "node_modules/@types/http-errors": {
+      "version": "2.0.5",
+      "resolved": "https://registry.npmjs.org/@types/http-errors/-/http-errors-2.0.5.tgz",
+      "integrity": "sha512-r8Tayk8HJnX0FztbZN7oVqGccWgw98T/0neJphO91KkmOzug1KkofZURD4UaD5uH8AqcFLfdPErnBod0u71/qg==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/@types/json5": {
       "version": "0.0.29",
       "resolved": "https://registry.npmjs.org/@types/json5/-/json5-0.0.29.tgz",
@@ -1268,6 +1323,41 @@
         "undici-types": "~6.21.0"
       }
     },
+    "node_modules/@types/qs": {
+      "version": "6.15.1",
+      "resolved": "https://registry.npmjs.org/@types/qs/-/qs-6.15.1.tgz",
+      "integrity": "sha512-GZHUBZR9hckSUhrxmp1nG6NwdpM9fCunJwyThLW1X3AyHgd9IlHb6VANpQQqDr2o/qQp6McZ3y/IA2rVzKzSbw==",
+      "dev": true,
+      "license": "MIT"
+    },
+    "node_modules/@types/range-parser": {
+      "version": "1.2.7",
+      "resolved": "https://registry.npmjs.org/@types/range-parser/-/range-parser-1.2.7.tgz",
+      "integrity": "sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ==",
+      "dev": true,
+      "license": "MIT"
+    },
+    "node_modules/@types/send": {
+      "version": "1.2.1",
+      "resolved": "https://registry.npmjs.org/@types/send/-/send-1.2.1.tgz",
+      "integrity": "sha512-arsCikDvlU99zl1g69TcAB3mzZPpxgw0UQnaHeC1Nwb015xp8bknZv5rIfri9xTOcMuaVgvabfIRA7PSZVuZIQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/node": "*"
+      }
+    },
+    "node_modules/@types/serve-static": {
+      "version": "2.2.0",
+      "resolved": "https://registry.npmjs.org/@types/serve-static/-/serve-static-2.2.0.tgz",
+      "integrity": "sha512-8mam4H1NHLtu7nmtalF7eyBH14QyOASmcxHhSfEoRyr0nP/YdoesEtU+uSRvMe96TW/HPTtkoKqQLl53N7UXMQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/http-errors": "*",
+        "@types/node": "*"
+      }
+    },
     "node_modules/@typescript-eslint/eslint-plugin": {
       "version": "8.59.3",
       "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.59.3.tgz",
@@ -1737,6 +1827,19 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
+    "node_modules/accepts": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/accepts/-/accepts-2.0.0.tgz",
+      "integrity": "sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==",
+      "license": "MIT",
+      "dependencies": {
+        "mime-types": "^3.0.0",
+        "negotiator": "^1.0.0"
+      },
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
     "node_modules/acorn": {
       "version": "8.16.0",
       "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.16.0.tgz",
@@ -2163,6 +2266,46 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/body-parser": {
+      "version": "2.2.2",
+      "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.2.2.tgz",
+      "integrity": "sha512-oP5VkATKlNwcgvxi0vM0p/D3n2C3EReYVX+DNYs5TjZFn/oQt2j+4sVJtSMr18pdRr8wjTcBl6LoV+FUwzPmNA==",
+      "license": "MIT",
+      "dependencies": {
+        "bytes": "^3.1.2",
+        "content-type": "^1.0.5",
+        "debug": "^4.4.3",
+        "http-errors": "^2.0.0",
+        "iconv-lite": "^0.7.0",
+        "on-finished": "^2.4.1",
+        "qs": "^6.14.1",
+        "raw-body": "^3.0.1",
+        "type-is": "^2.0.1"
+      },
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
+    "node_modules/body-parser/node_modules/iconv-lite": {
+      "version": "0.7.2",
+      "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.2.tgz",
+      "integrity": "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw==",
+      "license": "MIT",
+      "dependencies": {
+        "safer-buffer": ">= 2.1.2 < 3.0.0"
+      },
+      "engines": {
+        "node": ">=0.10.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/boolbase": {
       "version": "1.0.0",
       "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
@@ -2229,6 +2372,15 @@
         "esbuild": ">=0.18"
       }
     },
+    "node_modules/bytes": {
+      "version": "3.1.2",
+      "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz",
+      "integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/cac": {
       "version": "6.7.14",
       "resolved": "https://registry.npmjs.org/cac/-/cac-6.7.14.tgz",
@@ -2552,6 +2704,46 @@
         "node": "^14.18.0 || >=16.10.0"
       }
     },
+    "node_modules/content-disposition": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-1.1.0.tgz",
+      "integrity": "sha512-5jRCH9Z/+DRP7rkvY83B+yGIGX96OYdJmzngqnw2SBSxqCFPd0w2km3s5iawpGX8krnwSGmF0FW5Nhr0Hfai3g==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
+    "node_modules/content-type": {
+      "version": "1.0.5",
+      "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.5.tgz",
+      "integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
+    "node_modules/cookie": {
+      "version": "0.7.2",
+      "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.2.tgz",
+      "integrity": "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
+    "node_modules/cookie-signature": {
+      "version": "1.2.2",
+      "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.2.2.tgz",
+      "integrity": "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=6.6.0"
+      }
+    },
     "node_modules/core-util-is": {
       "version": "1.0.3",
       "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.3.tgz",
@@ -2764,6 +2956,15 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
+    "node_modules/depd": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
+      "integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/detect-libc": {
       "version": "2.1.2",
       "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
@@ -2915,12 +3116,27 @@
         "safe-buffer": "^5.0.1"
       }
     },
+    "node_modules/ee-first": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
+      "integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==",
+      "license": "MIT"
+    },
     "node_modules/emoji-regex": {
       "version": "10.6.0",
       "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-10.6.0.tgz",
       "integrity": "sha512-toUI84YS5YmxW219erniWD0CIVOo46xGKColeNQRgOzDorgBi1v4D71/OFzgD9GO2UGKIv1C3Sp8DAn0+j5w7A==",
       "license": "MIT"
     },
+    "node_modules/encodeurl": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-2.0.0.tgz",
+      "integrity": "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/encoding-sniffer": {
       "version": "0.2.1",
       "resolved": "https://registry.npmjs.org/encoding-sniffer/-/encoding-sniffer-0.2.1.tgz",
@@ -3161,6 +3377,12 @@
         "node": ">=6"
       }
     },
+    "node_modules/escape-html": {
+      "version": "1.0.3",
+      "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
+      "integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==",
+      "license": "MIT"
+    },
     "node_modules/escape-string-regexp": {
       "version": "4.0.0",
       "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz",
@@ -3567,6 +3789,15 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/etag": {
+      "version": "1.8.1",
+      "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
+      "integrity": "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
     "node_modules/events-universal": {
       "version": "1.0.1",
       "resolved": "https://registry.npmjs.org/events-universal/-/events-universal-1.0.1.tgz",
@@ -3597,6 +3828,49 @@
         "node": ">=12.0.0"
       }
     },
+    "node_modules/express": {
+      "version": "5.2.1",
+      "resolved": "https://registry.npmjs.org/express/-/express-5.2.1.tgz",
+      "integrity": "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==",
+      "license": "MIT",
+      "dependencies": {
+        "accepts": "^2.0.0",
+        "body-parser": "^2.2.1",
+        "content-disposition": "^1.0.0",
+        "content-type": "^1.0.5",
+        "cookie": "^0.7.1",
+        "cookie-signature": "^1.2.1",
+        "debug": "^4.4.0",
+        "depd": "^2.0.0",
+        "encodeurl": "^2.0.0",
+        "escape-html": "^1.0.3",
+        "etag": "^1.8.1",
+        "finalhandler": "^2.1.0",
+        "fresh": "^2.0.0",
+        "http-errors": "^2.0.0",
+        "merge-descriptors": "^2.0.0",
+        "mime-types": "^3.0.0",
+        "on-finished": "^2.4.1",
+        "once": "^1.4.0",
+        "parseurl": "^1.3.3",
+        "proxy-addr": "^2.0.7",
+        "qs": "^6.14.0",
+        "range-parser": "^1.2.1",
+        "router": "^2.2.0",
+        "send": "^1.1.0",
+        "serve-static": "^2.2.0",
+        "statuses": "^2.0.1",
+        "type-is": "^2.0.1",
+        "vary": "^1.1.2"
+      },
+      "engines": {
+        "node": ">= 18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/extend": {
       "version": "3.0.2",
       "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
@@ -3672,6 +3946,27 @@
         "node": "^10.12.0 || >=12.0.0"
       }
     },
+    "node_modules/finalhandler": {
+      "version": "2.1.1",
+      "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-2.1.1.tgz",
+      "integrity": "sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA==",
+      "license": "MIT",
+      "dependencies": {
+        "debug": "^4.4.0",
+        "encodeurl": "^2.0.0",
+        "escape-html": "^1.0.3",
+        "on-finished": "^2.4.1",
+        "parseurl": "^1.3.3",
+        "statuses": "^2.0.1"
+      },
+      "engines": {
+        "node": ">= 18.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/find-up": {
       "version": "5.0.0",
       "resolved": "https://registry.npmjs.org/find-up/-/find-up-5.0.0.tgz",
@@ -3739,6 +4034,15 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
+    "node_modules/forwarded": {
+      "version": "0.2.0",
+      "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz",
+      "integrity": "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
     "node_modules/frac": {
       "version": "1.1.2",
       "resolved": "https://registry.npmjs.org/frac/-/frac-1.1.2.tgz",
@@ -3748,6 +4052,15 @@
         "node": ">=0.8"
       }
     },
+    "node_modules/fresh": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/fresh/-/fresh-2.0.0.tgz",
+      "integrity": "sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/fs-constants": {
       "version": "1.0.0",
       "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
@@ -4273,6 +4586,26 @@
         "url": "https://github.com/fb55/entities?sponsor=1"
       }
     },
+    "node_modules/http-errors": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz",
+      "integrity": "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==",
+      "license": "MIT",
+      "dependencies": {
+        "depd": "~2.0.0",
+        "inherits": "~2.0.4",
+        "setprototypeof": "~1.2.0",
+        "statuses": "~2.0.2",
+        "toidentifier": "~1.0.1"
+      },
+      "engines": {
+        "node": ">= 0.8"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/https-proxy-agent": {
       "version": "7.0.6",
       "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz",
@@ -4372,7 +4705,6 @@
       "version": "2.0.4",
       "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
       "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
-      "dev": true,
       "license": "ISC"
     },
     "node_modules/ini": {
@@ -4410,6 +4742,15 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/ipaddr.js": {
+      "version": "1.9.1",
+      "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
+      "integrity": "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.10"
+      }
+    },
     "node_modules/is-array-buffer": {
       "version": "3.0.5",
       "resolved": "https://registry.npmjs.org/is-array-buffer/-/is-array-buffer-3.0.5.tgz",
@@ -4679,6 +5020,12 @@
         "node": ">=8"
       }
     },
+    "node_modules/is-promise": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/is-promise/-/is-promise-4.0.0.tgz",
+      "integrity": "sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==",
+      "license": "MIT"
+    },
     "node_modules/is-regex": {
       "version": "1.2.1",
       "resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.2.1.tgz",
@@ -5125,6 +5472,52 @@
         "node": ">= 0.4"
       }
     },
+    "node_modules/media-typer": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz",
+      "integrity": "sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
+    "node_modules/merge-descriptors": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-2.0.0.tgz",
+      "integrity": "sha512-Snk314V5ayFLhp3fkUREub6WtjBfPdCPY1Ln8/8munuLuiYhsABgBVWsozAG+MWMbVEvcdcpbi9R7ww22l9Q3g==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/sindresorhus"
+      }
+    },
+    "node_modules/mime-db": {
+      "version": "1.54.0",
+      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz",
+      "integrity": "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
+    "node_modules/mime-types": {
+      "version": "3.0.2",
+      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-3.0.2.tgz",
+      "integrity": "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==",
+      "license": "MIT",
+      "dependencies": {
+        "mime-db": "^1.54.0"
+      },
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/mimic-function": {
       "version": "5.0.1",
       "resolved": "https://registry.npmjs.org/mimic-function/-/mimic-function-5.0.1.tgz",
@@ -5295,6 +5688,15 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/negotiator": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-1.0.0.tgz",
+      "integrity": "sha512-8Ofs/AUQh8MaEcrlq5xOX0CQ9ypTF5dl78mjlMNfOK08fzpgTHQRQPBxcPlEtIw0yRpws+Zo/3r+5WRby7u3Gg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
     "node_modules/node-abi": {
       "version": "3.92.0",
       "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.92.0.tgz",
@@ -5498,11 +5900,22 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
+    "node_modules/on-finished": {
+      "version": "2.4.1",
+      "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz",
+      "integrity": "sha512-oVlzkg3ENAhCk2zdv7IJwd/QUD4z2RxRwpkcGY8psCVcCYZNq4wYnVWALHM+brtuJjePWiYF/ClmuDr8Ch5+kg==",
+      "license": "MIT",
+      "dependencies": {
+        "ee-first": "1.1.1"
+      },
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/once": {
       "version": "1.4.0",
       "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
       "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
-      "dev": true,
       "license": "ISC",
       "dependencies": {
         "wrappy": "1"
@@ -5715,6 +6128,15 @@
         "url": "https://github.com/fb55/entities?sponsor=1"
       }
     },
+    "node_modules/parseurl": {
+      "version": "1.3.3",
+      "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
+      "integrity": "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/path-exists": {
       "version": "4.0.0",
       "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz",
@@ -5752,6 +6174,16 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/path-to-regexp": {
+      "version": "8.4.2",
+      "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-8.4.2.tgz",
+      "integrity": "sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==",
+      "license": "MIT",
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/pathe": {
       "version": "2.0.3",
       "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz",
@@ -6027,6 +6459,19 @@
         "node": ">=0.4.0"
       }
     },
+    "node_modules/proxy-addr": {
+      "version": "2.0.7",
+      "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
+      "integrity": "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==",
+      "license": "MIT",
+      "dependencies": {
+        "forwarded": "0.2.0",
+        "ipaddr.js": "1.9.1"
+      },
+      "engines": {
+        "node": ">= 0.10"
+      }
+    },
     "node_modules/pump": {
       "version": "3.0.4",
       "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
@@ -6084,6 +6529,46 @@
       ],
       "license": "MIT"
     },
+    "node_modules/range-parser": {
+      "version": "1.2.1",
+      "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz",
+      "integrity": "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
+    "node_modules/raw-body": {
+      "version": "3.0.2",
+      "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-3.0.2.tgz",
+      "integrity": "sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA==",
+      "license": "MIT",
+      "dependencies": {
+        "bytes": "~3.1.2",
+        "http-errors": "~2.0.1",
+        "iconv-lite": "~0.7.0",
+        "unpipe": "~1.0.0"
+      },
+      "engines": {
+        "node": ">= 0.10"
+      }
+    },
+    "node_modules/raw-body/node_modules/iconv-lite": {
+      "version": "0.7.2",
+      "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.2.tgz",
+      "integrity": "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw==",
+      "license": "MIT",
+      "dependencies": {
+        "safer-buffer": ">= 2.1.2 < 3.0.0"
+      },
+      "engines": {
+        "node": ">=0.10.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/rc": {
       "version": "1.2.8",
       "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
@@ -6326,6 +6811,22 @@
         "fsevents": "~2.3.2"
       }
     },
+    "node_modules/router": {
+      "version": "2.2.0",
+      "resolved": "https://registry.npmjs.org/router/-/router-2.2.0.tgz",
+      "integrity": "sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ==",
+      "license": "MIT",
+      "dependencies": {
+        "debug": "^4.4.0",
+        "depd": "^2.0.0",
+        "is-promise": "^4.0.0",
+        "parseurl": "^1.3.3",
+        "path-to-regexp": "^8.0.0"
+      },
+      "engines": {
+        "node": ">= 18"
+      }
+    },
     "node_modules/run-parallel": {
       "version": "1.2.0",
       "resolved": "https://registry.npmjs.org/run-parallel/-/run-parallel-1.2.0.tgz",
@@ -6444,6 +6945,51 @@
         "node": ">=10"
       }
     },
+    "node_modules/send": {
+      "version": "1.2.1",
+      "resolved": "https://registry.npmjs.org/send/-/send-1.2.1.tgz",
+      "integrity": "sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==",
+      "license": "MIT",
+      "dependencies": {
+        "debug": "^4.4.3",
+        "encodeurl": "^2.0.0",
+        "escape-html": "^1.0.3",
+        "etag": "^1.8.1",
+        "fresh": "^2.0.0",
+        "http-errors": "^2.0.1",
+        "mime-types": "^3.0.2",
+        "ms": "^2.1.3",
+        "on-finished": "^2.4.1",
+        "range-parser": "^1.2.1",
+        "statuses": "^2.0.2"
+      },
+      "engines": {
+        "node": ">= 18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
+    "node_modules/serve-static": {
+      "version": "2.2.1",
+      "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-2.2.1.tgz",
+      "integrity": "sha512-xRXBn0pPqQTVQiC8wyQrKs2MOlX24zQ0POGaj0kultvoOCstBQM5yvOhAVSUwOMjQtTvsPWoNCHfPGwaaQJhTw==",
+      "license": "MIT",
+      "dependencies": {
+        "encodeurl": "^2.0.0",
+        "escape-html": "^1.0.3",
+        "parseurl": "^1.3.3",
+        "send": "^1.2.0"
+      },
+      "engines": {
+        "node": ">= 18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/set-function-length": {
       "version": "1.2.2",
       "resolved": "https://registry.npmjs.org/set-function-length/-/set-function-length-1.2.2.tgz",
@@ -6493,6 +7039,12 @@
         "node": ">= 0.4"
       }
     },
+    "node_modules/setprototypeof": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz",
+      "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==",
+      "license": "ISC"
+    },
     "node_modules/shebang-command": {
       "version": "2.0.0",
       "resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
@@ -6693,6 +7245,15 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/statuses": {
+      "version": "2.0.2",
+      "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
+      "integrity": "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/std-env": {
       "version": "3.10.0",
       "resolved": "https://registry.npmjs.org/std-env/-/std-env-3.10.0.tgz",
@@ -7152,6 +7713,15 @@
         "node": ">=14.0.0"
       }
     },
+    "node_modules/toidentifier": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
+      "integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=0.6"
+      }
+    },
     "node_modules/tr46": {
       "version": "0.0.3",
       "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
@@ -7806,6 +8376,37 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/type-is": {
+      "version": "2.1.0",
+      "resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz",
+      "integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==",
+      "license": "MIT",
+      "dependencies": {
+        "content-type": "^2.0.0",
+        "media-typer": "^1.1.0",
+        "mime-types": "^3.0.0"
+      },
+      "engines": {
+        "node": ">= 18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
+    "node_modules/type-is/node_modules/content-type": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
+      "integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=18"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/express"
+      }
+    },
     "node_modules/typed-array-buffer": {
       "version": "1.0.3",
       "resolved": "https://registry.npmjs.org/typed-array-buffer/-/typed-array-buffer-1.0.3.tgz",
@@ -7950,6 +8551,15 @@
         "node": ">= 10.0.0"
       }
     },
+    "node_modules/unpipe": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
+      "integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/unzipper": {
       "version": "0.12.3",
       "resolved": "https://registry.npmjs.org/unzipper/-/unzipper-0.12.3.tgz",
@@ -8001,6 +8611,15 @@
         "uuid": "dist/bin/uuid"
       }
     },
+    "node_modules/vary": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
+      "integrity": "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.8"
+      }
+    },
     "node_modules/vite": {
       "version": "5.4.21",
       "resolved": "https://registry.npmjs.org/vite/-/vite-5.4.21.tgz",
@@ -8826,7 +9445,6 @@
       "version": "1.0.2",
       "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
       "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
-      "dev": true,
       "license": "ISC"
     },
     "node_modules/xlsx": {

package.json

@@ -28,6 +28,7 @@
     "@commander-js/extra-typings": "^12.1.0",
     "cheerio": "^1.0.0",
     "commander": "^12.1.0",
+    "express": "^5.2.1",
     "googleapis": "^140.0.1",
     "ora": "^8.1.1",
     "xlsx": "^0.18.5",
@@ -35,6 +36,7 @@
     "zod": "^3.23.8"
   },
   "devDependencies": {
+    "@types/express": "^5.0.6",
     "@types/node": "^22.9.0",
     "@typescript-eslint/eslint-plugin": "^8.11.0",
     "@typescript-eslint/parser": "^8.11.0",

scripts/smoke.mjs

@@ -28,6 +28,7 @@ state_file: ${JSON.stringify(join(dir, 'state.json'))}
   await exec(binary, ['--help']);
   await exec('node', [cli, 'init', '--help']);
   await exec('node', [cli, 'run', '--help']);
+  await exec('node', [cli, 'serve', '--help']);
   await exec('node', [cli, 'run', '--config', config, '--dry-run'], {
     env: { ...process.env, NLC_FIXTURE: '1' }
   });

src/cli/program.ts

@@ -2,10 +2,12 @@ import { Command, Option } from 'commander';
 import { writeFile } from 'node:fs/promises';
 import { loadConfig } from '../config/config.js';
 import { createGmailClient } from '../gmail/client.js';
+import { DatabaseWriter } from '../output/databaseWriter.js';
 import { ExcelWriter } from '../output/excel.js';
 import { createGoogleSheetsWriter } from '../output/googleSheets.js';
 import { OutputWriter } from '../output/sheets.js';
 import { runCatalog } from '../run/runCatalog.js';
+import { createWebApp } from '../web/app.js';
 import { validateDateFilters } from './flags.js';
 
 const sampleConfig = `gmail:
@@ -79,6 +81,20 @@ export function createProgram(): Command {
       console.log(JSON.stringify(summary, null, 2));
     });
 
+  program
+    .command('serve')
+    .description('Start a local web app for browsing the SQLite catalog')
+    .option('--config <path>', 'Config path', './config.yaml')
+    .option('--host <host>', 'Host to bind', '127.0.0.1')
+    .option('--port <port>', 'Port to bind', (value) => Number(value), 3000)
+    .action(async (options) => {
+      const config = await loadConfig(options.config);
+      const app = createWebApp(config.database.path);
+      app.listen(options.port, options.host, () => {
+        console.log(`Newsletter Link Catalog listening at http://${options.host}:${options.port}`);
+      });
+    });
+
   return program;
 }
 
@@ -86,6 +102,9 @@ async function createWriters(
   config: Awaited<ReturnType<typeof loadConfig>>
 ): Promise<OutputWriter[]> {
   const writers: OutputWriter[] = [];
+  if (config.database.enabled) {
+    writers.push(new DatabaseWriter(config.database.path));
+  }
   if (config.output.excel.enabled) {
     writers.push(new ExcelWriter(config.output.excel.path));
   }

src/config/config.ts

@@ -81,6 +81,12 @@ const configSchema = z
         linkConcurrency: z.number().int().positive().default(3)
       })
       .default({}),
+    database: z
+      .object({
+        enabled: z.boolean().default(true),
+        path: z.string().default('./data/newsletter-catalog.sqlite')
+      })
+      .default({}),
     stateFile: z.string().default('~/.nlc/state.json'),
     plugins: z.record(z.string(), z.any()).default({})
   })

src/database/schema.ts

@@ -0,0 +1,63 @@
+export const catalogSchema = `
+PRAGMA foreign_keys = ON;
+
+CREATE TABLE IF NOT EXISTS newsletters (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  name TEXT NOT NULL UNIQUE
+);
+
+CREATE TABLE IF NOT EXISTS issues (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  newsletter_id INTEGER NOT NULL REFERENCES newsletters(id) ON DELETE CASCADE,
+  issue_date TEXT NOT NULL,
+  title TEXT NOT NULL DEFAULT '',
+  message_id TEXT,
+  UNIQUE(newsletter_id, issue_date, title)
+);
+
+CREATE TABLE IF NOT EXISTS links (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  url TEXT NOT NULL UNIQUE
+);
+
+CREATE TABLE IF NOT EXISTS link_occurrences (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  issue_id INTEGER NOT NULL REFERENCES issues(id) ON DELETE CASCADE,
+  link_id INTEGER NOT NULL REFERENCES links(id) ON DELETE CASCADE,
+  category TEXT NOT NULL DEFAULT '',
+  title TEXT NOT NULL DEFAULT '',
+  description TEXT NOT NULL DEFAULT '',
+  page_title_meta TEXT NOT NULL DEFAULT '',
+  also_in TEXT NOT NULL DEFAULT '',
+  UNIQUE(issue_id, link_id)
+);
+
+CREATE TABLE IF NOT EXISTS sponsors (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  issue_id INTEGER REFERENCES issues(id) ON DELETE SET NULL,
+  link_id INTEGER REFERENCES links(id) ON DELETE SET NULL,
+  newsletter TEXT NOT NULL DEFAULT '',
+  sponsor TEXT NOT NULL DEFAULT '',
+  description TEXT NOT NULL DEFAULT ''
+);
+
+CREATE TABLE IF NOT EXISTS dead_links (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  link_id INTEGER REFERENCES links(id) ON DELETE SET NULL,
+  url TEXT NOT NULL,
+  status TEXT NOT NULL DEFAULT '',
+  source TEXT NOT NULL DEFAULT '',
+  date TEXT NOT NULL DEFAULT ''
+);
+
+CREATE TABLE IF NOT EXISTS runs (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  started_at TEXT NOT NULL,
+  mode TEXT NOT NULL DEFAULT '',
+  newsletters_processed INTEGER NOT NULL DEFAULT 0,
+  links_extracted INTEGER NOT NULL DEFAULT 0,
+  sponsors INTEGER NOT NULL DEFAULT 0,
+  dead_links INTEGER NOT NULL DEFAULT 0,
+  errors INTEGER NOT NULL DEFAULT 0
+);
+`;

src/database/store.ts

@@ -0,0 +1,302 @@
+import { mkdirSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { dirname } from 'node:path';
+import { CatalogPayload } from '../output/sheets.js';
+import { catalogSchema } from './schema.js';
+
+const require = createRequire(import.meta.url);
+const sqlite = require('node:sqlite') as typeof import('node:sqlite');
+
+export interface CatalogRunPayload extends CatalogPayload {
+  mode: string;
+  newslettersProcessed: number;
+  linksExtracted: number;
+  sponsorCount: number;
+  deadLinkCount: number;
+  errors: number;
+}
+
+export interface NewsletterSummary {
+  id: number;
+  name: string;
+  latestIssueDate: string;
+  issueCount: number;
+  linkCount: number;
+  sponsorCount: number;
+}
+
+export interface NewsletterLinkQuery {
+  scope?: 'latest' | 'all';
+  search?: string;
+  category?: string;
+}
+
+export class CatalogDatabase {
+  private readonly db: import('node:sqlite').DatabaseSync;
+
+  public constructor(private readonly path: string) {
+    if (path !== ':memory:') {
+      mkdirSync(dirname(path), { recursive: true });
+    }
+    this.db = new sqlite.DatabaseSync(path);
+    this.db.exec('PRAGMA foreign_keys = ON');
+  }
+
+  public migrate(): void {
+    this.db.exec(catalogSchema);
+  }
+
+  public close(): void {
+    this.db.close();
+  }
+
+  public tableNames(): string[] {
+    return this.db
+      .prepare(
+        "SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name"
+      )
+      .all()
+      .map((row: any) => row.name);
+  }
+
+  public count(table: string): number {
+    const safeTable = table.replace(/[^a-z_]/g, '');
+    const row = this.db.prepare(`SELECT COUNT(*) AS count FROM ${safeTable}`).get() as {
+      count: number;
+    };
+    return row.count;
+  }
+
+  public saveCatalogRun(payload: CatalogRunPayload): void {
+    this.migrate();
+    this.db.exec('BEGIN');
+    try {
+      this.insertRun(payload);
+      for (const row of payload.rows) {
+        this.insertOccurrence(row);
+      }
+      for (const sponsor of payload.sponsors) {
+        this.insertSponsor(sponsor);
+      }
+      for (const deadLink of payload.deadLinks) {
+        this.insertDeadLink(deadLink);
+      }
+      this.db.exec('COMMIT');
+    } catch (error) {
+      this.db.exec('ROLLBACK');
+      throw error;
+    }
+  }
+
+  public dashboardCounts(): Record<string, number> {
+    return {
+      newsletters: this.count('newsletters'),
+      issues: this.count('issues'),
+      links: this.count('links'),
+      sponsors: this.count('sponsors'),
+      deadLinks: this.count('dead_links'),
+      runs: this.count('runs')
+    };
+  }
+
+  public contentLinks(): any[] {
+    return this.db
+      .prepare(
+        `SELECT n.name AS newsletter, i.issue_date AS issueDate, o.category, o.title, l.url, o.description
+         FROM link_occurrences o
+         JOIN issues i ON i.id = o.issue_id
+         JOIN newsletters n ON n.id = i.newsletter_id
+         JOIN links l ON l.id = o.link_id
+         ORDER BY i.issue_date DESC, n.name, o.title`
+      )
+      .all();
+  }
+
+  public newsletterSummaries(search = ''): NewsletterSummary[] {
+    const searchPattern = `%${search}%`;
+    return this.db
+      .prepare(
+        `SELECT
+           n.id,
+           n.name,
+           COALESCE(MAX(i.issue_date), '') AS latestIssueDate,
+           COUNT(DISTINCT i.id) AS issueCount,
+           COUNT(DISTINCT o.id) AS linkCount,
+           COUNT(DISTINCT s.id) AS sponsorCount
+         FROM newsletters n
+         LEFT JOIN issues i ON i.newsletter_id = n.id
+         LEFT JOIN link_occurrences o ON o.issue_id = i.id
+         LEFT JOIN sponsors s ON s.newsletter = n.name
+         WHERE n.name LIKE ?
+         GROUP BY n.id, n.name
+         ORDER BY n.name`
+      )
+      .all(searchPattern) as unknown as NewsletterSummary[];
+  }
+
+  public newsletterById(id: number): NewsletterSummary | undefined {
+    return this.newsletterSummaries().find((newsletter) => newsletter.id === id);
+  }
+
+  public categoriesForNewsletter(newsletterId: number): string[] {
+    return this.db
+      .prepare(
+        `SELECT DISTINCT o.category
+         FROM link_occurrences o
+         JOIN issues i ON i.id = o.issue_id
+         WHERE i.newsletter_id = ? AND o.category <> ''
+         ORDER BY o.category`
+      )
+      .all(newsletterId)
+      .map((row: any) => row.category);
+  }
+
+  public newsletterLinks(newsletterId: number, query: NewsletterLinkQuery = {}): any[] {
+    const scope = query.scope ?? 'latest';
+    const filters = ['i.newsletter_id = ?'];
+    const params: Array<string | number> = [newsletterId];
+
+    if (scope === 'latest') {
+      filters.push('i.issue_date = (SELECT MAX(issue_date) FROM issues WHERE newsletter_id = ?)');
+      params.push(newsletterId);
+    }
+    if (query.search) {
+      filters.push('(o.title LIKE ? OR o.description LIKE ? OR l.url LIKE ?)');
+      const search = `%${query.search}%`;
+      params.push(search, search, search);
+    }
+    if (query.category) {
+      filters.push('o.category = ?');
+      params.push(query.category);
+    }
+
+    return this.db
+      .prepare(
+        `SELECT i.issue_date AS issueDate, o.category, o.title, o.description, l.url, o.also_in AS alsoIn
+         FROM link_occurrences o
+         JOIN issues i ON i.id = o.issue_id
+         JOIN links l ON l.id = o.link_id
+         WHERE ${filters.join(' AND ')}
+         ORDER BY i.issue_date DESC, o.title`
+      )
+      .all(...params);
+  }
+
+  public allLinks(search = ''): any[] {
+    const searchPattern = `%${search}%`;
+    return this.db
+      .prepare(
+        `SELECT n.name AS newsletter, i.issue_date AS issueDate, o.category, o.title, l.url, o.description
+         FROM link_occurrences o
+         JOIN issues i ON i.id = o.issue_id
+         JOIN newsletters n ON n.id = i.newsletter_id
+         JOIN links l ON l.id = o.link_id
+         WHERE o.title LIKE ? OR o.description LIKE ? OR l.url LIKE ? OR n.name LIKE ?
+         ORDER BY i.issue_date DESC, n.name, o.title`
+      )
+      .all(searchPattern, searchPattern, searchPattern, searchPattern);
+  }
+
+  public sponsoredLinks(): any[] {
+    return this.db
+      .prepare('SELECT newsletter, sponsor, description FROM sponsors ORDER BY newsletter, sponsor')
+      .all();
+  }
+
+  public deadLinks(): any[] {
+    return this.db
+      .prepare('SELECT url, status, source, date FROM dead_links ORDER BY date DESC, url')
+      .all();
+  }
+
+  public runs(): any[] {
+    return this.db.prepare('SELECT * FROM runs ORDER BY started_at DESC, id DESC').all();
+  }
+
+  private insertRun(payload: CatalogRunPayload): void {
+    this.db
+      .prepare(
+        `INSERT INTO runs
+         (started_at, mode, newsletters_processed, links_extracted, sponsors, dead_links, errors)
+         VALUES (?, ?, ?, ?, ?, ?, ?)`
+      )
+      .run(
+        new Date().toISOString(),
+        payload.mode,
+        payload.newslettersProcessed,
+        payload.linksExtracted,
+        payload.sponsorCount,
+        payload.deadLinkCount,
+        payload.errors
+      );
+  }
+
+  private insertOccurrence(row: Record<string, unknown>): void {
+    const newsletterId = this.upsertNewsletter(String(row['Source Newsletter'] ?? 'Newsletter'));
+    const issueId = this.upsertIssue(newsletterId, String(row['Issue Date'] ?? ''), '');
+    const linkId = this.upsertLink(String(row['Link URL'] ?? ''));
+    this.db
+      .prepare(
+        `INSERT INTO link_occurrences
+         (issue_id, link_id, category, title, description, page_title_meta, also_in)
+         VALUES (?, ?, ?, ?, ?, ?, ?)
+         ON CONFLICT(issue_id, link_id) DO UPDATE SET
+           category = excluded.category,
+           title = excluded.title,
+           description = excluded.description,
+           page_title_meta = excluded.page_title_meta,
+           also_in = excluded.also_in`
+      )
+      .run(
+        issueId,
+        linkId,
+        String(row.Category ?? ''),
+        String(row.Title ?? ''),
+        String(row.Description ?? ''),
+        String(row['Page Title + Meta'] ?? ''),
+        String(row['Also In'] ?? '')
+      );
+  }
+
+  private insertSponsor(row: Record<string, unknown>): void {
+    const newsletter = String(row.Newsletter ?? '');
+    const linkId = this.upsertLink(String(row.Link ?? ''));
+    this.db
+      .prepare(
+        `INSERT INTO sponsors (issue_id, link_id, newsletter, sponsor, description)
+         VALUES (?, ?, ?, ?, ?)`
+      )
+      .run(null, linkId, newsletter, String(row.Sponsor ?? ''), String(row.Description ?? ''));
+  }
+
+  private insertDeadLink(row: Record<string, unknown>): void {
+    const url = String(row.URL ?? '');
+    const linkId = this.upsertLink(url);
+    this.db
+      .prepare('INSERT INTO dead_links (link_id, url, status, source, date) VALUES (?, ?, ?, ?, ?)')
+      .run(linkId, url, String(row.Status ?? ''), String(row.Source ?? ''), String(row.Date ?? ''));
+  }
+
+  private upsertNewsletter(name: string): number {
+    this.db.prepare('INSERT OR IGNORE INTO newsletters (name) VALUES (?)').run(name);
+    return (
+      this.db.prepare('SELECT id FROM newsletters WHERE name = ?').get(name) as { id: number }
+    ).id;
+  }
+
+  private upsertIssue(newsletterId: number, issueDate: string, title: string): number {
+    this.db
+      .prepare('INSERT OR IGNORE INTO issues (newsletter_id, issue_date, title) VALUES (?, ?, ?)')
+      .run(newsletterId, issueDate, title);
+    return (
+      this.db
+        .prepare('SELECT id FROM issues WHERE newsletter_id = ? AND issue_date = ? AND title = ?')
+        .get(newsletterId, issueDate, title) as { id: number }
+    ).id;
+  }
+
+  private upsertLink(url: string): number {
+    this.db.prepare('INSERT OR IGNORE INTO links (url) VALUES (?)').run(url);
+    return (this.db.prepare('SELECT id FROM links WHERE url = ?').get(url) as { id: number }).id;
+  }
+}

src/output/databaseWriter.ts

@@ -0,0 +1,23 @@
+import { CatalogDatabase } from '../database/store.js';
+import { CatalogPayload, OutputWriter } from './sheets.js';
+
+export class DatabaseWriter implements OutputWriter {
+  public constructor(private readonly path: string) {}
+
+  public async write(payload: CatalogPayload, summary: Record<string, unknown> = {}): Promise<void> {
+    const db = new CatalogDatabase(this.path);
+    try {
+      db.saveCatalogRun({
+        mode: String(summary.mode ?? 'run'),
+        newslettersProcessed: Number(summary.newslettersProcessed ?? 0),
+        linksExtracted: Number(summary.linksExtracted ?? payload.rows.length),
+        sponsorCount: Number(summary.sponsors ?? payload.sponsors.length),
+        deadLinkCount: Number(summary.deadLinks ?? payload.deadLinks.length),
+        errors: Number(summary.errors ?? 0),
+        ...payload
+      });
+    } finally {
+      db.close();
+    }
+  }
+}

src/output/sheets.ts

@@ -19,5 +19,5 @@ export interface CatalogPayload {
 }
 
 export interface OutputWriter {
-  write(payload: CatalogPayload): Promise<unknown>;
+  write(payload: CatalogPayload, summary?: Record<string, unknown>): Promise<unknown>;
 }

src/parsing/generic.ts

@@ -32,6 +32,10 @@ function sponsorMarkerText(value: string): string | undefined {
 }
 
 function blockTokens($: cheerio.CheerioAPI, node: any): Token[] {
+  if (!node) {
+    return [];
+  }
+
   if (node.type === 'text') {
     const text = compactText(node.data ?? '');
     return text ? [{ type: 'text', text }] : [];
@@ -49,6 +53,10 @@ function blockTokens($: cheerio.CheerioAPI, node: any): Token[] {
 
 function localContext($: cheerio.CheerioAPI, element: any, title: string): string {
   const block = $(element).closest('p,li,td,div').first();
+  if (block.length === 0) {
+    return title;
+  }
+
   const tokens = blockTokens($, block.get(0));
   const anchorIndex = tokens.findIndex(
     (token) => token.type === 'anchor' && token.element === element

src/run/runCatalog.ts

@@ -47,6 +47,16 @@ function escapeRegExp(value: string): string {
   return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
 }
 
+function runMode(options: RunOptions): string {
+  if (options.full) {
+    return 'full';
+  }
+  if (options.dryRun) {
+    return 'dry-run';
+  }
+  return 'incremental';
+}
+
 export async function runCatalog(options: RunOptions): Promise<RunSummary> {
   const config = normalizeConfig(options.config);
   const state = new StateStore(config.stateFile);
@@ -113,17 +123,20 @@ export async function runCatalog(options: RunOptions): Promise<RunSummary> {
     }
   }
 
-  if (!options.dryRun) {
-    for (const writer of options.writers) {
-      await writer.write({ rows, sponsors, deadLinks: [] });
-    }
-  }
-
-  return {
+  const summary = {
+    mode: runMode(options),
     newslettersProcessed: messages.length,
     linksExtracted: rows.length,
     sponsors: sponsors.length,
     deadLinks: 0,
     errors
   };
+
+  if (!options.dryRun) {
+    for (const writer of options.writers) {
+      await writer.write({ rows, sponsors, deadLinks: [] }, summary);
+    }
+  }
+
+  return summary;
 }

src/web/app.ts

@@ -0,0 +1,158 @@
+import express from 'express';
+import { CatalogDatabase } from '../database/store.js';
+import { dashboard, newsletterBrowser, page, table } from './views.js';
+
+export function createWebApp(databasePath: string) {
+  const app = express();
+
+  app.get('/', (_req, res, next) => {
+    withDatabase(databasePath, (db) => res.send(dashboard(db.dashboardCounts()))).catch(next);
+  });
+
+  app.get('/links', (req, res, next) => {
+    const search = stringQuery(req.query.q);
+    withDatabase(databasePath, (db) =>
+      res.send(
+        page(
+          'Links',
+          `<h1>All Links</h1>
+          <form class="toolbar" method="get" action="/links">
+            <label>Search links
+              <input type="search" name="q" value="${escapeHtml(search)}">
+            </label>
+            <button class="button" type="submit">Apply</button>
+          </form>
+          ${table(db.allLinks(search), [
+            ['newsletter', 'Newsletter'],
+            ['issueDate', 'Issue Date'],
+            ['category', 'Category'],
+            ['title', 'Title'],
+            ['url', 'URL'],
+            ['description', 'Description']
+          ])}`,
+          'links'
+        )
+      )
+    ).catch(next);
+  });
+
+  app.get('/newsletters', (req, res, next) => {
+    withDatabase(databasePath, (db) => {
+      const scope = req.query.scope === 'all' ? 'all' : 'latest';
+      const linkSearch = stringQuery(req.query.q);
+      const category = stringQuery(req.query.category);
+      const newsletters = db.newsletterSummaries();
+      const requestedNewsletterId = Number(req.query.newsletter);
+      const selected =
+        newsletters.find((newsletter) => newsletter.id === requestedNewsletterId) ?? newsletters[0];
+
+      res.send(
+        newsletterBrowser({
+          newsletters,
+          selected,
+          links: selected
+            ? db.newsletterLinks(selected.id, {
+                category,
+                scope,
+                search: linkSearch
+              })
+            : [],
+          categories: selected ? db.categoriesForNewsletter(selected.id) : [],
+          filters: {
+            category,
+            linkSearch,
+            scope
+          }
+        })
+      );
+    }).catch(next);
+  });
+
+  app.get('/sponsors', (_req, res, next) => {
+    withDatabase(databasePath, (db) =>
+      res.send(
+        page(
+          'Sponsored Links',
+          `<h1>Sponsored Links</h1>${table(db.sponsoredLinks(), [
+            ['newsletter', 'Newsletter'],
+            ['sponsor', 'Sponsor'],
+            ['description', 'Description']
+          ])}`,
+          'sponsors'
+        )
+      )
+    ).catch(next);
+  });
+
+  app.get('/dead-links', (_req, res, next) => {
+    withDatabase(databasePath, (db) =>
+      res.send(
+        page(
+          'Dead Links',
+          `<h1>Dead Links</h1>${table(db.deadLinks(), [
+            ['url', 'URL'],
+            ['status', 'Status'],
+            ['source', 'Source'],
+            ['date', 'Date']
+          ])}`,
+          'dead-links'
+        )
+      )
+    ).catch(next);
+  });
+
+  app.get('/runs', (_req, res, next) => {
+    withDatabase(databasePath, (db) =>
+      res.send(
+        page(
+          'Runs',
+          `<h1>Runs</h1>${table(db.runs(), [
+            ['started_at', 'Started'],
+            ['mode', 'Mode'],
+            ['newsletters_processed', 'Newsletters'],
+            ['links_extracted', 'Links'],
+            ['sponsors', 'Sponsors'],
+            ['dead_links', 'Dead Links'],
+            ['errors', 'Errors']
+          ])}`,
+          'runs'
+        )
+      )
+    ).catch(next);
+  });
+
+  app.use(
+    (error: Error, _req: express.Request, res: express.Response, _next: express.NextFunction) => {
+      console.error(error);
+      res.status(500).send(page('Error', '<h1>Error</h1><p>Something went wrong.</p>'));
+    }
+  );
+
+  return app;
+}
+
+function escapeHtml(value: unknown): string {
+  return String(value ?? '')
+    .replaceAll('&', '&amp;')
+    .replaceAll('<', '&lt;')
+    .replaceAll('>', '&gt;')
+    .replaceAll('"', '&quot;')
+    .replaceAll("'", '&#39;');
+}
+
+function stringQuery(value: unknown): string {
+  return typeof value === 'string' ? value : '';
+}
+
+async function withDatabase(
+  databasePath: string,
+  callback: (database: CatalogDatabase) => void
+): Promise<void> {
+  const db = new CatalogDatabase(databasePath);
+  try {
+    db.migrate();
+    callback(db);
+  } finally {
+    db.close();
+  }
+}

src/web/views.ts

@@ -0,0 +1,456 @@
+import { NewsletterSummary } from '../database/store.js';
+
+type ActivePage = 'dashboard' | 'newsletters' | 'links' | 'sponsors' | 'dead-links' | 'runs';
+
+interface NewsletterBrowserModel {
+  newsletters: NewsletterSummary[];
+  selected?: NewsletterSummary;
+  links: Record<string, unknown>[];
+  categories: string[];
+  filters: {
+    category: string;
+    linkSearch: string;
+    scope: 'latest' | 'all';
+  };
+}
+
+function escapeHtml(value: unknown): string {
+  return String(value ?? '')
+    .replaceAll('&', '&amp;')
+    .replaceAll('<', '&lt;')
+    .replaceAll('>', '&gt;')
+    .replaceAll('"', '&quot;')
+    .replaceAll("'", '&#39;');
+}
+
+function escapeAttr(value: unknown): string {
+  return escapeHtml(value);
+}
+
+function pageUrl(path: string, params: Record<string, string | number | undefined>): string {
+  const query = new URLSearchParams();
+  Object.entries(params).forEach(([key, value]) => {
+    if (value !== undefined && value !== '') {
+      query.set(key, String(value));
+    }
+  });
+  const serialized = query.toString();
+  return serialized ? `${path}?${serialized}` : path;
+}
+
+function navLink(href: string, label: string, pageName: ActivePage, active: ActivePage): string {
+  const current = pageName === active ? ' aria-current="page"' : '';
+  return `<a href="${href}"${current}>${escapeHtml(label)}</a>`;
+}
+
+export function page(title: string, body: string, active: ActivePage = 'dashboard'): string {
+  return `<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>${escapeHtml(title)}</title>
+  <style>
+    :root {
+      color-scheme: dark;
+      --bg: #101214;
+      --panel: #171a1f;
+      --panel-strong: #20242b;
+      --text: #eef1f4;
+      --muted: #a5adb7;
+      --line: #303640;
+      --accent: #62b3ff;
+      --accent-soft: rgba(98, 179, 255, 0.14);
+      --danger: #ff7d7d;
+    }
+    :root[data-theme="light"] {
+      color-scheme: light;
+      --bg: #f6f7f9;
+      --panel: #ffffff;
+      --panel-strong: #eef1f5;
+      --text: #1d232b;
+      --muted: #5f6b78;
+      --line: #d9dee5;
+      --accent: #0369a1;
+      --accent-soft: rgba(3, 105, 161, 0.1);
+      --danger: #b42318;
+    }
+    * { box-sizing: border-box; }
+    body {
+      margin: 0;
+      min-height: 100vh;
+      background: var(--bg);
+      color: var(--text);
+      font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
+      line-height: 1.45;
+    }
+    a { color: var(--accent); text-decoration: none; }
+    a:hover { text-decoration: underline; }
+    .topbar {
+      display: flex;
+      align-items: center;
+      justify-content: space-between;
+      gap: 1rem;
+      min-height: 3.75rem;
+      padding: 0.75rem 1.25rem;
+      border-bottom: 1px solid var(--line);
+      background: var(--panel);
+    }
+    .brand { color: var(--text); font-size: 1rem; font-weight: 700; }
+    .global-nav { display: flex; flex-wrap: wrap; gap: 0.35rem; align-items: center; }
+    .global-nav a,
+    .theme-toggle {
+      min-height: 2rem;
+      border: 1px solid transparent;
+      border-radius: 6px;
+      padding: 0.35rem 0.6rem;
+      color: var(--muted);
+      background: transparent;
+      font: inherit;
+    }
+    .global-nav a[aria-current="page"] {
+      color: var(--text);
+      border-color: var(--line);
+      background: var(--panel-strong);
+    }
+    .theme-toggle { cursor: pointer; border-color: var(--line); }
+    main { padding: 1.25rem; }
+    h1, h2, h3 { margin: 0; line-height: 1.2; letter-spacing: 0; }
+    h1 { font-size: 1.6rem; }
+    h2 { font-size: 1.05rem; }
+    p { margin: 0; }
+    table { border-collapse: collapse; width: 100%; margin-top: 1rem; }
+    th, td {
+      border-bottom: 1px solid var(--line);
+      padding: 0.65rem 0.5rem;
+      text-align: left;
+      vertical-align: top;
+    }
+    th { color: var(--muted); font-size: 0.78rem; font-weight: 700; text-transform: uppercase; }
+    input, select {
+      width: 100%;
+      min-height: 2.25rem;
+      border: 1px solid var(--line);
+      border-radius: 6px;
+      padding: 0.4rem 0.55rem;
+      color: var(--text);
+      background: var(--panel);
+      font: inherit;
+    }
+    .cards { display: grid; grid-template-columns: repeat(auto-fit, minmax(9rem, 1fr)); gap: 0.8rem; }
+    .card {
+      border: 1px solid var(--line);
+      border-radius: 8px;
+      padding: 1rem;
+      background: var(--panel);
+    }
+    .metric { display: block; font-size: 1.45rem; font-weight: 750; }
+    .muted { color: var(--muted); }
+    .toolbar { display: flex; flex-wrap: wrap; gap: 0.75rem; align-items: end; margin: 1rem 0; }
+    .toolbar label { display: grid; gap: 0.25rem; min-width: 12rem; color: var(--muted); font-size: 0.83rem; }
+    .button {
+      display: inline-flex;
+      align-items: center;
+      justify-content: center;
+      min-height: 2.25rem;
+      border: 1px solid var(--line);
+      border-radius: 6px;
+      padding: 0.35rem 0.75rem;
+      color: var(--text);
+      background: var(--panel-strong);
+      font: inherit;
+      cursor: pointer;
+    }
+    .app-shell {
+      display: grid;
+      grid-template-columns: minmax(15rem, 21rem) minmax(0, 1fr);
+      gap: 1rem;
+      min-height: calc(100vh - 6.25rem);
+    }
+    .sidebar,
+    .detail {
+      min-width: 0;
+      border: 1px solid var(--line);
+      border-radius: 8px;
+      background: var(--panel);
+    }
+    .sidebar { padding: 0.85rem; }
+    .sidebar-header { display: grid; gap: 0.75rem; margin-bottom: 0.75rem; }
+    .newsletter-list { display: grid; gap: 0.45rem; }
+    .newsletter-item {
+      display: grid;
+      gap: 0.35rem;
+      border: 1px solid transparent;
+      border-radius: 8px;
+      padding: 0.65rem;
+      color: var(--text);
+      background: transparent;
+    }
+    .newsletter-item[aria-current="true"] {
+      border-color: var(--accent);
+      background: var(--accent-soft);
+    }
+    .newsletter-item:hover { border-color: var(--line); text-decoration: none; }
+    .newsletter-title { font-weight: 700; }
+    .newsletter-meta { display: flex; flex-wrap: wrap; gap: 0.45rem; color: var(--muted); font-size: 0.82rem; }
+    .detail { overflow: hidden; }
+    .detail-header {
+      display: grid;
+      gap: 0.75rem;
+      padding: 1rem;
+      border-bottom: 1px solid var(--line);
+      background: var(--panel-strong);
+    }
+    .detail-title-row {
+      display: flex;
+      flex-wrap: wrap;
+      align-items: start;
+      justify-content: space-between;
+      gap: 0.75rem;
+    }
+    .counts { display: flex; flex-wrap: wrap; gap: 0.5rem; }
+    .count-pill {
+      border: 1px solid var(--line);
+      border-radius: 999px;
+      padding: 0.25rem 0.55rem;
+      color: var(--muted);
+      font-size: 0.82rem;
+      white-space: nowrap;
+    }
+    .segmented {
+      display: inline-flex;
+      overflow: hidden;
+      border: 1px solid var(--line);
+      border-radius: 7px;
+      width: fit-content;
+    }
+    .segmented a {
+      min-width: 6.5rem;
+      padding: 0.45rem 0.65rem;
+      color: var(--muted);
+      text-align: center;
+    }
+    .segmented a[aria-current="true"] {
+      color: var(--text);
+      background: var(--accent-soft);
+    }
+    .detail-body { padding: 1rem; overflow-x: auto; }
+    .link-title { font-weight: 700; }
+    .description { max-width: 36rem; color: var(--muted); }
+    .url-cell { max-width: 20rem; overflow-wrap: anywhere; }
+    .empty {
+      border: 1px dashed var(--line);
+      border-radius: 8px;
+      padding: 1.25rem;
+      color: var(--muted);
+      text-align: center;
+    }
+    @media (max-width: 800px) {
+      .topbar { align-items: flex-start; flex-direction: column; }
+      main { padding: 0.75rem; }
+      .app-shell { grid-template-columns: 1fr; }
+      .sidebar { max-height: 18rem; overflow: auto; }
+      .toolbar label { min-width: 100%; }
+    }
+  </style>
+</head>
+<body>
+  <header class="topbar">
+    <a class="brand" href="/">Newsletter Link Catalog</a>
+    <nav class="global-nav" aria-label="Primary">
+      ${navLink('/', 'Dashboard', 'dashboard', active)}
+      ${navLink('/newsletters', 'Newsletters', 'newsletters', active)}
+      ${navLink('/links', 'All Links', 'links', active)}
+      ${navLink('/sponsors', 'Sponsored Links', 'sponsors', active)}
+      ${navLink('/dead-links', 'Dead Links', 'dead-links', active)}
+      ${navLink('/runs', 'Runs', 'runs', active)}
+    </nav>
+    <button class="theme-toggle" type="button" data-theme-toggle>Theme</button>
+  </header>
+  <main>${body}</main>
+  <script>
+    (() => {
+      const root = document.documentElement;
+      const savedTheme = localStorage.getItem('nlc-theme');
+      if (savedTheme) root.dataset.theme = savedTheme;
+      document.querySelector('[data-theme-toggle]')?.addEventListener('click', () => {
+        const next = root.dataset.theme === 'light' ? '' : 'light';
+        if (next) {
+          root.dataset.theme = next;
+          localStorage.setItem('nlc-theme', next);
+        } else {
+          root.removeAttribute('data-theme');
+          localStorage.removeItem('nlc-theme');
+        }
+      });
+      const search = document.querySelector('[data-newsletter-search]');
+      search?.addEventListener('input', () => {
+        const term = search.value.toLowerCase();
+        document.querySelectorAll('[data-newsletter-name]').forEach((item) => {
+          item.hidden = !item.dataset.newsletterName.includes(term);
+        });
+      });
+    })();
+  </script>
+</body>
+</html>`;
+}
+
+export function table(rows: Record<string, unknown>[], columns: Array<[string, string]>): string {
+  if (rows.length === 0) {
+    return '<p class="muted">No rows yet.</p>';
+  }
+  return `<table><thead><tr>${columns.map(([, label]) => `<th>${escapeHtml(label)}</th>`).join('')}</tr></thead><tbody>${rows
+    .map((row) => `<tr>${columns.map(([key]) => `<td>${escapeHtml(row[key])}</td>`).join('')}</tr>`)
+    .join('')}</tbody></table>`;
+}
+
+export function dashboard(counts: Record<string, number>): string {
+  return page(
+    'Newsletter Link Catalog',
+    `<h1>Newsletter Link Catalog</h1>
+    <section class="cards" aria-label="Catalog totals">
+      ${Object.entries(counts)
+        .map(
+          ([key, value]) =>
+            `<div class="card"><span class="metric">${escapeHtml(value)}</span><span class="muted">${escapeHtml(key)}</span></div>`
+        )
+        .join('')}
+    </section>`,
+    'dashboard'
+  );
+}
+
+export function newsletterBrowser(model: NewsletterBrowserModel): string {
+  const { selected } = model;
+  const selectedId = selected?.id ?? '';
+  return page(
+    'Newsletters',
+    `<div class="app-shell">
+      <aside class="sidebar" aria-label="Newsletters">
+        <div class="sidebar-header">
+          <h1>Newsletters</h1>
+          <input type="search" placeholder="Search newsletters" aria-label="Search newsletters" data-newsletter-search>
+        </div>
+        <div class="newsletter-list">
+          ${model.newsletters.length === 0 ? '<p class="muted">No newsletters yet.</p>' : ''}
+          ${model.newsletters
+            .map((newsletter) =>
+              newsletterListItem(newsletter, model.filters, newsletter.id === selected?.id)
+            )
+            .join('')}
+        </div>
+      </aside>
+      <section class="detail" aria-label="Selected newsletter links">
+        ${selected ? selectedNewsletterView(model, selectedId) : '<div class="detail-body"><p class="empty">No newsletter selected.</p></div>'}
+      </section>
+    </div>`,
+    'newsletters'
+  );
+}
+
+function newsletterListItem(
+  newsletter: NewsletterSummary,
+  filters: NewsletterBrowserModel['filters'],
+  isSelected: boolean
+): string {
+  const href = pageUrl('/newsletters', {
+    newsletter: newsletter.id,
+    scope: filters.scope,
+    q: filters.linkSearch,
+    category: filters.category
+  });
+  return `<a class="newsletter-item" href="${escapeAttr(href)}" data-newsletter-name="${escapeAttr(newsletter.name.toLowerCase())}" aria-current="${isSelected}">
+    <span class="newsletter-title">${escapeHtml(newsletter.name)}</span>
+    <span class="newsletter-meta">
+      <span>${escapeHtml(newsletter.latestIssueDate || 'No issues')}</span>
+      <span>${escapeHtml(newsletter.issueCount)} issues</span>
+      <span>${escapeHtml(newsletter.linkCount)} links</span>
+    </span>
+  </a>`;
+}
+
+function selectedNewsletterView(
+  model: NewsletterBrowserModel,
+  selectedId: number | string
+): string {
+  const { filters, selected } = model;
+  const latestHref = pageUrl('/newsletters', {
+    newsletter: selectedId,
+    scope: 'latest',
+    q: filters.linkSearch,
+    category: filters.category
+  });
+  const allHref = pageUrl('/newsletters', {
+    newsletter: selectedId,
+    scope: 'all',
+    q: filters.linkSearch,
+    category: filters.category
+  });
+  return `<div class="detail-header">
+    <div class="detail-title-row">
+      <div>
+        <h1>${escapeHtml(selected?.name)}</h1>
+        <p class="muted">Latest issue ${escapeHtml(selected?.latestIssueDate || 'not available')}</p>
+      </div>
+      <div class="counts" aria-label="Newsletter counts">
+        <span class="count-pill">${escapeHtml(selected?.issueCount)} issues</span>
+        <span class="count-pill">${escapeHtml(selected?.linkCount)} links</span>
+        <span class="count-pill">${escapeHtml(selected?.sponsorCount)} sponsored</span>
+      </div>
+    </div>
+    <div class="segmented" aria-label="Issue scope">
+      <a href="${escapeAttr(latestHref)}" aria-current="${filters.scope === 'latest'}">Latest Issue</a>
+      <a href="${escapeAttr(allHref)}" aria-current="${filters.scope === 'all'}">All Issues</a>
+    </div>
+    <form class="toolbar" method="get" action="/newsletters">
+      <input type="hidden" name="newsletter" value="${escapeAttr(selectedId)}">
+      <input type="hidden" name="scope" value="${escapeAttr(filters.scope)}">
+      <label>Search links
+        <input type="search" name="q" value="${escapeAttr(filters.linkSearch)}">
+      </label>
+      <label>Category
+        <select name="category">
+          <option value="">All categories</option>
+          ${model.categories
+            .map(
+              (category) =>
+                `<option value="${escapeAttr(category)}"${category === filters.category ? ' selected' : ''}>${escapeHtml(category)}</option>`
+            )
+            .join('')}
+        </select>
+      </label>
+      <button class="button" type="submit">Apply</button>
+    </form>
+  </div>
+  <div class="detail-body">
+    ${linksTable(model.links, filters.scope)}
+  </div>`;
+}
+
+function linksTable(rows: Record<string, unknown>[], scope: 'latest' | 'all'): string {
+  if (rows.length === 0) {
+    return '<p class="empty">No links match the current filters.</p>';
+  }
+  const issueHeader = scope === 'all' ? '<th>Issue Date</th>' : '';
+  return `<table>
+    <thead>
+      <tr>${issueHeader}<th>Title</th><th>Category</th><th>Description</th><th>URL</th><th>Also In</th></tr>
+    </thead>
+    <tbody>
+      ${rows.map((row) => linkRow(row, scope)).join('')}
+    </tbody>
+  </table>`;
+}
+
+function linkRow(row: Record<string, unknown>, scope: 'latest' | 'all'): string {
+  const issueDate = scope === 'all' ? `<td>${escapeHtml(row.issueDate)}</td>` : '';
+  return `<tr>
+    ${issueDate}
+    <td><a class="link-title" href="${escapeAttr(row.url)}" target="_blank" rel="noopener noreferrer">${escapeHtml(row.title || row.url)}</a></td>
+    <td>${escapeHtml(row.category)}</td>
+    <td class="description">${escapeHtml(row.description)}</td>
+    <td class="url-cell"><a href="${escapeAttr(row.url)}" target="_blank" rel="noopener noreferrer">${escapeHtml(row.url)}</a></td>
+    <td>${escapeHtml(row.alsoIn)}</td>
+  </tr>`;
+}

tests/config.test.ts

@@ -17,6 +17,8 @@ output:
     expect(config.gmail.folder).toBe('Newsletters');
     expect(config.links.trackingParams).toContain('utm_*');
     expect(config.enrichment.concurrency).toBe(3);
+    expect(config.database.enabled).toBe(true);
+    expect(config.database.path).toBe('./data/newsletter-catalog.sqlite');
   });
 
   it('treats comment-only YAML maps as empty objects', () => {

tests/database.test.ts

@@ -0,0 +1,90 @@
+import { mkdtemp, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { afterEach, beforeEach, describe, expect, it } from 'vitest';
+import { CatalogDatabase } from '../src/database/store.js';
+
+let dir = '';
+
+beforeEach(async () => {
+  dir = await mkdtemp(join(tmpdir(), 'nlc-db-'));
+});
+
+afterEach(async () => {
+  await rm(dir, { force: true, recursive: true });
+});
+
+describe('CatalogDatabase', () => {
+  it('creates the catalog schema', () => {
+    const db = new CatalogDatabase(join(dir, 'catalog.sqlite'));
+    db.migrate();
+
+    expect(db.tableNames()).toEqual([
+      'dead_links',
+      'issues',
+      'link_occurrences',
+      'links',
+      'newsletters',
+      'runs',
+      'sponsors'
+    ]);
+
+    db.close();
+  });
+
+  it('persists catalog payloads and deduplicates canonical URLs', () => {
+    const db = new CatalogDatabase(join(dir, 'catalog.sqlite'));
+    db.migrate();
+    db.saveCatalogRun({
+      mode: 'test',
+      newslettersProcessed: 2,
+      linksExtracted: 2,
+      sponsorCount: 1,
+      deadLinkCount: 1,
+      errors: 0,
+      rows: [
+        {
+          'Issue Date': '2026-05-17',
+          Category: 'JavaScript',
+          'Link URL': 'https://example.com/post',
+          Title: 'First mention',
+          Description: 'One',
+          'Page Title + Meta': '',
+          'Source Newsletter': 'Alpha Weekly',
+          'Also In': ''
+        },
+        {
+          'Issue Date': '2026-05-18',
+          Category: 'DevOps',
+          'Link URL': 'https://example.com/post',
+          Title: 'Second mention',
+          Description: 'Two',
+          'Page Title + Meta': '',
+          'Source Newsletter': 'Beta Weekly',
+          'Also In': ''
+        }
+      ],
+      sponsors: [
+        {
+          Newsletter: 'Alpha Weekly',
+          Sponsor: 'Acme',
+          Link: 'https://sponsor.example',
+          Description: 'Sponsor blurb'
+        }
+      ],
+      deadLinks: [
+        { URL: 'https://dead.example', Status: '404', Source: 'Alpha Weekly', Date: '2026-05-17' }
+      ]
+    });
+
+    expect(db.count('links')).toBe(3);
+    expect(db.count('link_occurrences')).toBe(2);
+    expect(db.count('newsletters')).toBe(2);
+    expect(db.count('issues')).toBe(2);
+    expect(db.count('sponsors')).toBe(1);
+    expect(db.count('dead_links')).toBe(1);
+    expect(db.count('runs')).toBe(1);
+
+    db.close();
+  });
+});

tests/run.test.ts

@@ -2,6 +2,7 @@ import { mkdtemp, rm } from 'node:fs/promises';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { afterEach, beforeEach, describe, expect, it } from 'vitest';
+import { CatalogDatabase } from '../src/database/store.js';
 import { runCatalog } from '../src/run/runCatalog.js';
 
 let dir = '';
@@ -86,4 +87,45 @@ describe('run orchestration', () => {
       'Typescale AI'
     ]);
   });
+
+  it('can persist run output to SQLite through a database writer', async () => {
+    const path = join(dir, 'catalog.sqlite');
+    const db = new CatalogDatabase(path);
+    db.migrate();
+
+    const result = await runCatalog({
+      config: {
+        gmail: { folder: 'Newsletters' },
+        output: { name: 'Catalog', excel: { enabled: false } },
+        stateFile: join(dir, 'state.json')
+      },
+      messages: [
+        {
+          id: 'msg-1',
+          messageId: '<msg-1>',
+          from: 'SQLite Weekly <sqlite@example.com>',
+          date: '2026-05-17T00:00:00.000Z',
+          html: '<h2>Databases</h2><a href="https://sqlite.example">SQLite Post</a> - Local data.'
+        }
+      ],
+      writers: [
+        {
+          write: async (payload) =>
+            db.saveCatalogRun({
+              mode: 'test',
+              newslettersProcessed: 1,
+              linksExtracted: payload.rows.length,
+              sponsorCount: payload.sponsors.length,
+              deadLinkCount: payload.deadLinks.length,
+              errors: 0,
+              ...payload
+            })
+        }
+      ]
+    });
+
+    expect(result.linksExtracted).toBe(1);
+    expect(db.count('link_occurrences')).toBe(1);
+    db.close();
+  });
 });

tests/web.test.ts

@@ -0,0 +1,165 @@
+import { mkdtemp, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { AddressInfo } from 'node:net';
+import { afterEach, beforeEach, describe, expect, it } from 'vitest';
+import { CatalogDatabase } from '../src/database/store.js';
+import { createWebApp } from '../src/web/app.js';
+
+let dir = '';
+
+beforeEach(async () => {
+  dir = await mkdtemp(join(tmpdir(), 'nlc-web-'));
+});
+
+afterEach(async () => {
+  await rm(dir, { force: true, recursive: true });
+});
+
+async function withServer(path: string, pathname: string): Promise<string> {
+  const app = createWebApp(path);
+  const server = app.listen(0);
+  try {
+    const { port } = server.address() as AddressInfo;
+    const response = await fetch(`http://127.0.0.1:${port}${pathname}`);
+    return response.text();
+  } finally {
+    server.close();
+  }
+}
+
+function fixtureDatabase(): string {
+  const path = join(dir, 'catalog.sqlite');
+  const db = new CatalogDatabase(path);
+  db.saveCatalogRun({
+    mode: 'test',
+    newslettersProcessed: 2,
+    linksExtracted: 3,
+    sponsorCount: 1,
+    deadLinkCount: 1,
+    errors: 0,
+    rows: [
+      {
+        'Issue Date': '2026-05-10',
+        Category: 'SQLite',
+        'Link URL': 'https://sqlite.example/old',
+        Title: 'Older SQLite Post',
+        Description: 'An older database post',
+        'Page Title + Meta': '',
+        'Source Newsletter': 'DB Weekly',
+        'Also In': ''
+      },
+      {
+        'Issue Date': '2026-05-17',
+        Category: 'SQLite',
+        'Link URL': 'https://sqlite.example',
+        Title: 'SQLite Post',
+        Description: 'A local database post',
+        'Page Title + Meta': '',
+        'Source Newsletter': 'DB Weekly',
+        'Also In': ''
+      },
+      {
+        'Issue Date': '2026-05-16',
+        Category: 'JavaScript',
+        'Link URL': 'https://js.example',
+        Title: 'JS Post',
+        Description: 'A JavaScript post',
+        'Page Title + Meta': '',
+        'Source Newsletter': 'JS Weekly',
+        'Also In': ''
+      }
+    ],
+    sponsors: [
+      {
+        Newsletter: 'DB Weekly',
+        Sponsor: 'Acme',
+        Link: 'https://sponsor.example',
+        Description: 'Blurb'
+      }
+    ],
+    deadLinks: [
+      { URL: 'https://dead.example', Status: '404', Source: 'DB Weekly', Date: '2026-05-17' }
+    ]
+  });
+  db.close();
+  return path;
+}
+
+describe('web app', () => {
+  it('provides newsletter summaries and latest/all issue link scopes', () => {
+    const path = fixtureDatabase();
+    const db = new CatalogDatabase(path);
+    try {
+      const summaries = db.newsletterSummaries();
+      expect(
+        summaries.map(({ name, latestIssueDate, issueCount, linkCount, sponsorCount }) => ({
+          name,
+          latestIssueDate,
+          issueCount,
+          linkCount,
+          sponsorCount
+        }))
+      ).toEqual([
+        {
+          name: 'DB Weekly',
+          latestIssueDate: '2026-05-17',
+          issueCount: 2,
+          linkCount: 2,
+          sponsorCount: 1
+        },
+        {
+          name: 'JS Weekly',
+          latestIssueDate: '2026-05-16',
+          issueCount: 1,
+          linkCount: 1,
+          sponsorCount: 0
+        }
+      ]);
+
+      const dbWeekly = summaries.find((newsletter) => newsletter.name === 'DB Weekly');
+      expect(
+        db.newsletterLinks(dbWeekly?.id ?? 0, { scope: 'latest' }).map((link) => link.title)
+      ).toEqual(['SQLite Post']);
+      expect(
+        db.newsletterLinks(dbWeekly?.id ?? 0, { scope: 'all' }).map((link) => link.title)
+      ).toEqual(['SQLite Post', 'Older SQLite Post']);
+    } finally {
+      db.close();
+    }
+  });
+
+  it('renders dashboard and catalog pages from SQLite', async () => {
+    const path = fixtureDatabase();
+
+    await expect(withServer(path, '/')).resolves.toContain('Newsletter Link Catalog');
+    await expect(withServer(path, '/links')).resolves.toContain('SQLite Post');
+    await expect(withServer(path, '/links?q=JavaScript')).resolves.toContain('JS Post');
+    await expect(withServer(path, '/links?q=JavaScript')).resolves.not.toContain('SQLite Post');
+    await expect(withServer(path, '/sponsors')).resolves.toContain('Acme');
+    await expect(withServer(path, '/dead-links')).resolves.toContain('https://dead.example');
+    await expect(withServer(path, '/runs')).resolves.toContain('test');
+  });
+
+  it('renders the two-pane newsletter browser with latest issue by default and all issue mode', async () => {
+    const path = fixtureDatabase();
+
+    const latest = await withServer(path, '/newsletters');
+    expect(latest).toContain('class="app-shell"');
+    expect(latest).toContain('Search newsletters');
+    expect(latest).toContain('aria-current="true"');
+    expect(latest).toContain('Latest Issue');
+    expect(latest).toContain('All Issues');
+    expect(latest).toContain('SQLite Post');
+    expect(latest).not.toContain('Older SQLite Post');
+    expect(latest).toContain('target="_blank"');
+    expect(latest).toContain('rel="noopener noreferrer"');
+
+    const allIssues = await withServer(path, '/newsletters?scope=all');
+    expect(allIssues).toContain('SQLite Post');
+    expect(allIssues).toContain('Older SQLite Post');
+    expect(allIssues).toContain('2026-05-10');
+    expect(allIssues).toContain('name="category"');
+    expect(allIssues).toContain('name="q"');
+  });
+});