273 lines
7.5 KiB
Markdown
273 lines
7.5 KiB
Markdown
---
|
||
title: SoloForge Documentation
|
||
section: docs
|
||
summary: SoloForge Infrastructure Documentation for Self-Hosted Gitea with Actions Runner on Hetzner
|
||
tags: [servers, infrasctructure, gitea, ci/cd, hetzner, docker]
|
||
nav: 1
|
||
---
|
||
|
||
## Self-Hosted Gitea + Actions Runner (Hetzner Deployment)
|
||
|
||
This document describes the current SoloForge setup — Gitea, Traefik routing, the Gitea Actions runner, backup routines, directory layout, and operational notes.
|
||
|
||
This repo exists so future-me (and actual-me) don’t need to reverse-engineer anything when something eventually explodes.
|
||
|
||
## 1. Overview
|
||
|
||
SoloForge is a self-hosted Gitea instance running on a Hetzner VM.
|
||
|
||
It provides:
|
||
|
||
- Private (but login-protected) Git hosting
|
||
- Public-visible repositories (instance auth is required anyway)
|
||
- Gitea Actions, backed by a Docker-based self-hosted runner
|
||
- Automatic CI for TODO-to-issue sync and other workflows
|
||
- Reverse-proxy via Traefik
|
||
- Automated Gitea backups
|
||
|
||
The system replaces the original Proxmox-hosted Gitea instance lost due to disk failure.
|
||
|
||
---
|
||
|
||
## 2. System Layout
|
||
|
||
### Hetzner VM
|
||
|
||
- Debian 13
|
||
- Docker + Docker Compose installed
|
||
- Traefik reverse proxy (existing before SoloForge migration)
|
||
- HTTPS termination handled by Traefik via Let’s Encrypt
|
||
|
||
### Directory structure
|
||
|
||
```bash
|
||
/gitea/
|
||
├── backups/ # Gitea nightly backups
|
||
├── gitea/ # Gitea app + persistent data
|
||
├── postgres/ # PostgreSQL data directory (if using Postgres)
|
||
└── docker-compose.yml # Main Gitea stack
|
||
```
|
||
|
||
### Runner lives separately
|
||
|
||
```bash
|
||
/gitea/gitea-runner/
|
||
├── docker-compose.yml # Actions runner stack
|
||
└── data/ # Contains .runner registration + job cache
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Gitea Deployment
|
||
|
||
### 3.1 Gitea Compose Service
|
||
|
||
Gitea is launched via Docker Compose and reverse-proxied through Traefik.
|
||
Data lives under `/gitea/gitea` to ensure persistence.
|
||
|
||
### 3.2 Traefik Routing
|
||
|
||
Traefik handles:
|
||
|
||
- HTTPS certificate generation
|
||
- Routing git.keithsolomon.net → Gitea web UI
|
||
- Exposing SSH port (222) for git-over-SSH
|
||
|
||
No YAML generator required anymore — everything is stable and hand-maintained.
|
||
|
||
---
|
||
|
||
## 4. Gitea Actions Runner
|
||
|
||
SoloForge uses a self-hosted Gitea Actions runner, running via Docker and capable of executing JavaScript (Node-based) GitHub-style actions.
|
||
|
||
### 4.1 Runner compose file
|
||
|
||
Located at:
|
||
|
||
`/gitea/gitea-runner/docker-compose.yml`
|
||
|
||
Core configuration:
|
||
|
||
```yaml
|
||
environment:
|
||
GITEA_INSTANCE_URL: "https://git.keithsolomon.net"
|
||
GITEA_RUNNER_REGISTRATION_TOKEN: "<token>"
|
||
GITEA_RUNNER_NAME: "hetzner-runner-1"
|
||
GITEA_RUNNER_LABELS: "ubuntu-latest:docker://node:20-bullseye,self-hosted,linux,x86_64,docker"
|
||
```
|
||
|
||
By default, GitHub runners provide Node.js preinstalled.
|
||
Self-hosted runners do NOT.
|
||
|
||
Mapping:
|
||
|
||
`ubuntu-latest:docker://node:20-bullseye`
|
||
|
||
ensures any workflow using:
|
||
|
||
```yaml
|
||
runs-on: ubuntu-latest
|
||
```
|
||
|
||
runs inside a Node-enabled container, fixing "node: command not found" errors.
|
||
|
||
### 4.3 Re-registering the runner (important!)
|
||
|
||
If labels change or the runner breaks:
|
||
|
||
```bash
|
||
cd /gitea/gitea-runner
|
||
docker compose down
|
||
rm -f data/.runner # Forces new registration
|
||
docker compose up -d # Registers with current labels
|
||
```
|
||
|
||
Check runner status in Gitea:
|
||
|
||
**Site Admin → Actions → Runners**
|
||
|
||
---
|
||
|
||
## 5. Workflows
|
||
|
||
### 5.1 TODO-to-Issue Sync
|
||
|
||
Certain repos use a custom JavaScript action to:
|
||
|
||
- Parse TODO comments
|
||
- Generate/close GitHub-style issues inside Gitea
|
||
|
||
These workflows run cleanly now because:
|
||
|
||
- The runner supports Node (ubuntu-latest → node:20 container)
|
||
- Repository permissions allow issue writing
|
||
|
||
### 5.2 Secret tokens
|
||
|
||
Unlike GitHub, Gitea does not auto-inject GITHUB_TOKEN.
|
||
Workflows requiring an auth token need one defined manually in:
|
||
|
||
**Repo → Settings → Secrets**
|
||
|
||
Example:
|
||
|
||
`GITHUB_TOKEN = <personal access token>`
|
||
|
||
(Or rename to something more Gitea-themed.)
|
||
|
||
---
|
||
|
||
## 6. Repository Management
|
||
|
||
### 6.1 Bulk import
|
||
|
||
All repos were migrated using a [custom bulk-mirror script](/assets/files/gitea/git-bulk) that:
|
||
|
||
- Created missing repos via the Gitea API
|
||
- Pushed full history via git push --all and --tags
|
||
|
||
### 6.2 Public visibility
|
||
|
||
All repos are public (since Gitea login protects everything).
|
||
A [bulk-update script](/assets/files/gitea/git-flip) is available to flip visibility via API if needed.
|
||
|
||
---
|
||
|
||
## 7. Backups
|
||
|
||
Gitea supports built-in dumps via:
|
||
|
||
`gitea dump`
|
||
|
||
A cronjob is installed to dump nightly at 3am:
|
||
|
||
```bash
|
||
/gitea/backups/
|
||
└── gitea-dump-YYYYMMDD.zip
|
||
```
|
||
|
||
Recommended: sync this folder offsite or back to home lab.
|
||
|
||
---
|
||
|
||
## 8. Restore Notes
|
||
|
||
If Gitea must be restored from dump:
|
||
|
||
```bash
|
||
docker compose down
|
||
rm -rf gitea/* postgres/*
|
||
unzip gitea-dump.zip into /gitea/gitea
|
||
docker compose up -d
|
||
```
|
||
|
||
If the runner needs re-registration, follow section 4.3.
|
||
|
||
---
|
||
|
||
## 9. Future Improvements (Optional)
|
||
|
||
- Mirror “source of truth” repos between GitHub ↔ Gitea
|
||
- Add automated org-level secrets
|
||
- Configure multiple runners (home lab, Hetzner, etc.)
|
||
- Add Prometheus metrics + Grafana board for CI activity
|
||
- Set up Gitea’s dependency listing or vulnerability scanning
|
||
|
||
---
|
||
|
||
## 10. CLI Toolkit
|
||
|
||
### [`forge`](/assets/files/gitea/forge.sh) Tool
|
||
|
||
A custom CLI tool to help manage common tasks:
|
||
|
||
| Command | Description |
|
||
| --------------------------- | ---------------------------------------------------------- |
|
||
| `forge status` | Show status of Gitea and runner containers |
|
||
| `forge ps` | Alias for status |
|
||
| `forge gitea-logs` | Tail logs from Gitea container |
|
||
| `forge runner-logs` | Tail logs from Actions runner container |
|
||
| `forge backup` | Run a Gitea dump and move it into BACKUP_DIR |
|
||
| `forge restore-test` | Run backup restore sanity checks (unzip + extract) |
|
||
| `forge restart-gitea` | Restart Gitea stack |
|
||
| `forge restart-runner` | Restart Actions runner stack |
|
||
| `forge runner-reset` | Re-register runner with current labels (destroys .runner) |
|
||
| `forge diag` | Quick diagnostic summary |
|
||
|
||
### [`forge-alert`](/assets/files/gitea/forge-alert.sh) Tool
|
||
|
||
A companion script that serves as a basic reusable alert sender, capable of logging to syslog and sending notifications via Telegram. Telegram bot token and chat ID must be set in environment variables by editing `/etc/environment` in a root/`sudo` shell.
|
||
|
||
### [`forge-b2-backup`](/assets/files/gitea/forge-b2-backup.sh) Tool
|
||
|
||
A backup script that uploads Gitea dumps to Backblaze B2 cloud storage. Utilizes `rclone`, so make sure to configure an appropriate remote (`B2` by default) before use. Alerts via `forge-alert.sh` for backup status.
|
||
|
||
### [`forge-restore-test`](/assets/files/gitea/forge-restore-test.sh) Tool
|
||
|
||
A restore test script that performs basic integrity checks on the latest Gitea dump file, including unzip testing and extraction smoke test. Alerts via `forge-alert.sh` if any issues are detected.
|
||
|
||
---
|
||
|
||
## TL;DR Cheat Sheet
|
||
|
||
### Runner broke?
|
||
|
||
`→ delete data/.runner, docker compose up -d`
|
||
|
||
### `node` not found?
|
||
|
||
`→ ensure ubuntu-latest label is mapped to node:20-bullseye`
|
||
|
||
### Release workflows failing?
|
||
|
||
`→ they're GitHub-only; they run on GitHub mirrors`
|
||
|
||
### Backup?
|
||
|
||
`→ see /gitea/backups, nightly gitea dump`
|
||
|
||
### Repo not found?
|
||
|
||
`→ bulk import script: auto-create + push mirror`
|