Keith Solomon
87 lines
3.4 KiB
Markdown
87 lines
3.4 KiB
Markdown
# SQLite Web App Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Make SQLite the default catalog store and add a functional local web app for browsing imported newsletter data.
|
|
|
|
**Architecture:** Use Node's built-in `node:sqlite` `DatabaseSync` API for a packaged-executable-friendly SQLite layer. Keep the existing Gmail ingestion pipeline and spreadsheet writers, add a `DatabaseWriter` as the default writer, and add an Express server that renders read-only catalog pages from SQLite.
|
|
|
|
**Tech Stack:** TypeScript, Node.js `node:sqlite`, Express, Commander, Vitest, existing parser/output modules.
|
|
|
|
---
|
|
|
|
### Task 1: Config Defaults
|
|
|
|
**Files:**
|
|
|
|
- Modify: `src/config/config.ts`
|
|
- Modify: `config.example.yaml`
|
|
- Test: `tests/config.test.ts`
|
|
|
|
- [ ] Write a failing test that `database.enabled` defaults to `true` and `database.path` defaults to `./data/newsletter-catalog.sqlite`.
|
|
- [ ] Add the database schema to config validation.
|
|
- [ ] Add `database` to `config.example.yaml`.
|
|
- [ ] Run `npm test -- tests/config.test.ts`.
|
|
|
|
### Task 2: SQLite Schema and Store
|
|
|
|
**Files:**
|
|
|
|
- Create: `src/database/schema.ts`
|
|
- Create: `src/database/store.ts`
|
|
- Test: `tests/database.test.ts`
|
|
|
|
- [ ] Write failing tests that migrations create `newsletters`, `issues`, `links`, `link_occurrences`, `sponsors`, `dead_links`, and `runs`.
|
|
- [ ] Write failing tests that saving a catalog payload deduplicates canonical URLs and stores occurrences per issue.
|
|
- [ ] Implement migration and transactional persistence with `node:sqlite`.
|
|
- [ ] Run `npm test -- tests/database.test.ts`.
|
|
|
|
### Task 3: Default Database Writer
|
|
|
|
**Files:**
|
|
|
|
- Create: `src/output/databaseWriter.ts`
|
|
- Modify: `src/run/runCatalog.ts`
|
|
- Modify: `src/cli/program.ts`
|
|
- Test: `tests/run.test.ts`
|
|
|
|
- [ ] Write a failing test that `runCatalog` can write rows to a database writer and still preserves spreadsheet writer behavior.
|
|
- [ ] Add a `DatabaseWriter` implementing the existing `OutputWriter` interface.
|
|
- [ ] Make `nlc run` add `DatabaseWriter` by default when `database.enabled` is true.
|
|
- [ ] Run `npm test -- tests/run.test.ts tests/database.test.ts`.
|
|
|
|
### Task 4: Web App
|
|
|
|
**Files:**
|
|
|
|
- Create: `src/web/app.ts`
|
|
- Create: `src/web/views.ts`
|
|
- Modify: `src/cli/program.ts`
|
|
- Test: `tests/web.test.ts`
|
|
|
|
- [ ] Write failing tests that dashboard, links, sponsored links, dead links, and runs routes render data from a fixture database.
|
|
- [ ] Add `nlc serve --config --host --port`.
|
|
- [ ] Implement read-only Express routes and simple server-rendered HTML.
|
|
- [ ] Run `npm test -- tests/web.test.ts`.
|
|
|
|
### Task 5: Docs and Verification
|
|
|
|
**Files:**
|
|
|
|
- Modify: `README.md`
|
|
- Modify: `package.json`
|
|
|
|
- [ ] Document SQLite default storage and `nlc serve`.
|
|
- [ ] Add Express dependencies and update lockfile.
|
|
- [ ] Run `npm install`.
|
|
- [ ] Run `npm run lint`.
|
|
- [ ] Run `npm run format:check`.
|
|
- [ ] Run `npm run typecheck`.
|
|
- [ ] Run `npm test`.
|
|
- [ ] Run `npm run build`.
|
|
- [ ] Run `npm run smoke`.
|
|
|
|
## Self-Review
|
|
|
|
This plan covers all accepted spec requirements: SQLite defaults, schema, default import persistence, optional spreadsheet output, `nlc serve`, functional catalog views, docs, and automated tests. No unresolved placeholders remain. Types and paths align with the existing project structure.
|