Initial commit
This commit is contained in:
347
README.md
Normal file
347
README.md
Normal file
@@ -0,0 +1,347 @@
|
||||
# Skipper MVP
|
||||
|
||||
Skipper is a file-backed control plane and host agent pair for lightweight hosting orchestration.
|
||||
|
||||
## Structure
|
||||
|
||||
- `skipper-api/`: Express `/v1` control plane API
|
||||
- `skippy-agent/`: polling agent that executes declarative work orders
|
||||
- `shared/`: storage, tracing, auth, logging, events, snapshots
|
||||
- `data/`: JSON-backed resources, work orders, logs, events, and auth records
|
||||
|
||||
## Local Run
|
||||
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
## Skipper CLI
|
||||
|
||||
A minimal CLI is now available for common operator tasks.
|
||||
|
||||
Run it from the repo with:
|
||||
|
||||
```bash
|
||||
npm run cli -- --help
|
||||
```
|
||||
|
||||
Available commands:
|
||||
|
||||
```bash
|
||||
npm run cli -- health --url http://127.0.0.1:3000
|
||||
npm run cli -- deploy apply --url http://127.0.0.1:3000 --admin-token YOUR_ADMIN_TOKEN --tenant-id example-tenant
|
||||
DATA_DIR=/opt/skipper/data npm run cli -- node register --node-id host-1 --role worker --region local
|
||||
npm run cli -- node show --url http://127.0.0.1:3000 --admin-token YOUR_ADMIN_TOKEN --node-id host-1
|
||||
```
|
||||
|
||||
If you install the repo as an executable package, the binary name is `skipper`.
|
||||
|
||||
## Deploy Skipper
|
||||
|
||||
The controller can be deployed on its own host from the internal registry.
|
||||
|
||||
First-start behavior:
|
||||
|
||||
- Skipper now bootstraps its own on-disk data layout on startup.
|
||||
- An empty bind-mounted data directory is valid.
|
||||
- You still need to supply an `ADMIN_TOKEN`.
|
||||
|
||||
Files:
|
||||
|
||||
- [`deploy/skipper/docker-compose.yml`](/home/sundown/Projekter/nodeJS/Skipper/deploy/skipper/docker-compose.yml#L1)
|
||||
- [`deploy/skipper/.env.example`](/home/sundown/Projekter/nodeJS/Skipper/deploy/skipper/.env.example#L1)
|
||||
|
||||
Basic flow on the target server:
|
||||
|
||||
```bash
|
||||
mkdir -p /opt/skipper
|
||||
cd /opt/skipper
|
||||
cp /path/to/repo/deploy/skipper/docker-compose.yml .
|
||||
cp /path/to/repo/deploy/skipper/.env.example .env
|
||||
```
|
||||
|
||||
Edit `.env`:
|
||||
|
||||
- set `SKIPPER_IMAGE` to the tag you released
|
||||
- set a strong `ADMIN_TOKEN`
|
||||
- optionally change `SKIPPER_PORT`
|
||||
|
||||
Start Skipper:
|
||||
|
||||
```bash
|
||||
docker login registry.internal.budgethost.io
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
One-line first boot with `docker run`:
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name skipper-api \
|
||||
--restart unless-stopped \
|
||||
-p 3000:3000 \
|
||||
-e HOST=0.0.0.0 \
|
||||
-e PORT=3000 \
|
||||
-e DATA_DIR=/app/data \
|
||||
-e ADMIN_TOKEN='replace-with-a-long-random-admin-token' \
|
||||
-v /opt/skipper/data:/app/data \
|
||||
registry.internal.budgethost.io/skipper/skipper-api:latest
|
||||
```
|
||||
|
||||
That command is enough for first boot. The container will create the required `/app/data` subdirectories automatically.
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
curl http://127.0.0.1:3000/v1/health
|
||||
```
|
||||
|
||||
Persisted controller state will live in:
|
||||
|
||||
```bash
|
||||
/opt/skipper/data
|
||||
```
|
||||
|
||||
The next step after this is deploying one or more `Skippy` agents pointed at the controller URL.
|
||||
|
||||
## Attach Skippy
|
||||
|
||||
Attaching a `Skippy` agent has two parts:
|
||||
|
||||
1. register the node on the Skipper host
|
||||
2. start the agent container on the target host
|
||||
|
||||
### 1. Register The Node On Skipper
|
||||
|
||||
Run this on the Skipper host, in the same repo or deployment workspace that owns `/opt/skipper/data`:
|
||||
|
||||
```bash
|
||||
DATA_DIR=/opt/skipper/data npm run cli -- node register --node-id host-1 --role worker --region se-sto-1
|
||||
```
|
||||
|
||||
That writes:
|
||||
|
||||
- `/opt/skipper/data/resources/nodes/host-1.json`
|
||||
- `/opt/skipper/data/auth/nodes/host-1.json`
|
||||
|
||||
The command prints the generated token. Save it and use it as `AGENT_TOKEN` on the target host.
|
||||
|
||||
You can also provide your own token:
|
||||
|
||||
```bash
|
||||
DATA_DIR=/opt/skipper/data npm run cli -- node register --node-id host-1 --token 'your-long-random-token'
|
||||
```
|
||||
|
||||
### 2. Start The Agent On The Target Host
|
||||
|
||||
Files:
|
||||
|
||||
- [`deploy/skippy/docker-compose.yml`](/home/sundown/Projekter/nodeJS/Skipper/deploy/skippy/docker-compose.yml#L1)
|
||||
- [`deploy/skippy/.env.example`](/home/sundown/Projekter/nodeJS/Skipper/deploy/skippy/.env.example#L1)
|
||||
|
||||
Basic flow on the target host:
|
||||
|
||||
```bash
|
||||
mkdir -p /opt/skippy
|
||||
cd /opt/skippy
|
||||
cp /path/to/repo/deploy/skippy/docker-compose.yml .
|
||||
cp /path/to/repo/deploy/skippy/.env.example .env
|
||||
```
|
||||
|
||||
Edit `.env`:
|
||||
|
||||
- set `SKIPPY_IMAGE`
|
||||
- set `SKIPPER_URL`
|
||||
- set `AGENT_ID`
|
||||
- set `AGENT_TOKEN`
|
||||
|
||||
Start the agent:
|
||||
|
||||
```bash
|
||||
docker login registry.internal.budgethost.io
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
One-line first boot with `docker run`:
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name skippy-agent \
|
||||
--restart unless-stopped \
|
||||
-e DATA_DIR=/app/data \
|
||||
-e SKIPPER_URL='http://your-skipper-host:3000' \
|
||||
-e AGENT_ID='host-1' \
|
||||
-e AGENT_TOKEN='replace-with-generated-token' \
|
||||
-e POLL_INTERVAL_MS=5000 \
|
||||
-e HEARTBEAT_INTERVAL_MS=15000 \
|
||||
-e SKIPPY_COMPOSE_BASE_DIR=/opt/skipper/tenants \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-v /opt/skippy/data:/app/data \
|
||||
-v /opt/skippy/tenants:/opt/skipper/tenants \
|
||||
registry.internal.budgethost.io/skipper/skippy-agent:latest
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
docker logs skippy-agent --tail 50
|
||||
curl -H "x-admin-token: <your-admin-token>" http://your-skipper-host:3000/v1/resources/node/host-1
|
||||
```
|
||||
|
||||
## Image Build And Push
|
||||
|
||||
Default image targets:
|
||||
|
||||
- `registry.internal.budgethost.io/skipper/skipper-api`
|
||||
- `registry.internal.budgethost.io/skipper/skippy-agent`
|
||||
|
||||
Prerequisites:
|
||||
|
||||
- Docker daemon access must work from your shell
|
||||
- You must be logged into `registry.internal.budgethost.io`
|
||||
|
||||
Sanity checks:
|
||||
|
||||
```bash
|
||||
docker info
|
||||
docker login registry.internal.budgethost.io
|
||||
```
|
||||
|
||||
Preview tags:
|
||||
|
||||
```bash
|
||||
npm run images:print
|
||||
```
|
||||
|
||||
Build locally:
|
||||
|
||||
```bash
|
||||
IMAGE_TAG=0.1.0 npm run images:build
|
||||
```
|
||||
|
||||
Build and push to the internal registry:
|
||||
|
||||
```bash
|
||||
IMAGE_TAG=0.1.0 npm run images:release
|
||||
```
|
||||
|
||||
Push prebuilt tags only:
|
||||
|
||||
```bash
|
||||
IMAGE_TAG=0.1.0 npm run images:push
|
||||
```
|
||||
|
||||
Optional overrides:
|
||||
|
||||
- `IMAGE_REGISTRY`
|
||||
- `IMAGE_NAMESPACE`
|
||||
- `IMAGE_TAG`
|
||||
- `IMAGE_PLATFORM`
|
||||
|
||||
Run compose from registry-pushed images instead of local builds:
|
||||
|
||||
```bash
|
||||
IMAGE_TAG=0.1.0 docker compose -f docker-compose.yml -f docker-compose.registry.yml up -d
|
||||
```
|
||||
|
||||
First release sequence:
|
||||
|
||||
```bash
|
||||
npm run smoke:test
|
||||
docker login registry.internal.budgethost.io
|
||||
IMAGE_TAG=0.1.0 npm run images:release
|
||||
IMAGE_TAG=0.1.0 docker compose -f docker-compose.yml -f docker-compose.registry.yml up -d
|
||||
```
|
||||
|
||||
Verification:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.registry.yml ps
|
||||
curl http://localhost:3000/v1/health
|
||||
```
|
||||
|
||||
Current note:
|
||||
|
||||
- The application smoke test passed in this repository.
|
||||
- The image release flow was prepared and invoked, but this environment could not access `/var/run/docker.sock`, so the actual build and push must be run from a host session with Docker daemon permissions.
|
||||
|
||||
## Release Manifest
|
||||
|
||||
A release-oriented wrapper is available to generate git-aware image tags and write a deployment manifest.
|
||||
|
||||
Print the next release plan:
|
||||
|
||||
```bash
|
||||
npm run release:print
|
||||
```
|
||||
|
||||
Write manifest files without building:
|
||||
|
||||
```bash
|
||||
npm run release:plan
|
||||
```
|
||||
|
||||
Build release-tagged images and write manifest:
|
||||
|
||||
```bash
|
||||
RUN_SMOKE_TEST=1 npm run release:build
|
||||
```
|
||||
|
||||
Build, push, and write manifest:
|
||||
|
||||
```bash
|
||||
RUN_SMOKE_TEST=1 npm run release:publish
|
||||
```
|
||||
|
||||
Generated manifest files:
|
||||
|
||||
- `artifacts/releases/latest.json`
|
||||
- `artifacts/releases/<version_tag>.json`
|
||||
|
||||
Release environment overrides:
|
||||
|
||||
- `RELEASE_CHANNEL`
|
||||
- `RELEASE_VERSION_TAG`
|
||||
- `RELEASE_GIT_SHA`
|
||||
- `IMAGE_REGISTRY`
|
||||
- `IMAGE_NAMESPACE`
|
||||
|
||||
Tag behavior:
|
||||
|
||||
- If git metadata is available, the release tag includes the short commit SHA.
|
||||
- If git metadata is not available, the release tag falls back to `nogit`.
|
||||
- Unless overridden, the generated tag format is `<channel>-<git_sha>-<timestamp>`.
|
||||
|
||||
## Smoke Test
|
||||
|
||||
```bash
|
||||
npm run smoke:test
|
||||
```
|
||||
|
||||
This starts the API and agent as local subprocesses, uses a mocked `docker` binary, triggers one deploy, and verifies the completed job plus written compose artifacts.
|
||||
|
||||
Create a deployment work order:
|
||||
|
||||
```bash
|
||||
curl -X POST \
|
||||
-H 'x-admin-token: dev-admin-token' \
|
||||
-H 'x-idempotency-key: local-run-1' \
|
||||
-H 'x-request-id: 11111111-1111-1111-1111-111111111111' \
|
||||
-H 'x-correlation-id: 22222222-2222-2222-2222-222222222222' \
|
||||
http://localhost:3000/v1/deployments/example-tenant/apply
|
||||
```
|
||||
|
||||
Inspect work orders:
|
||||
|
||||
```bash
|
||||
find data/work-orders -type f | sort
|
||||
```
|
||||
|
||||
## Example Flow
|
||||
|
||||
1. `POST /v1/deployments/example-tenant/apply` loads [`data/resources/tenants/example-tenant.json`](/home/sundown/Projekter/nodeJS/Skipper/data/resources/tenants/example-tenant.json#L1) and creates a declarative `deploy_service` work order in `data/work-orders/pending/`.
|
||||
2. `skippy-agent` polls `GET /v1/nodes/host-1/work-orders/next` every 5 seconds and receives the next work order for `host-1`.
|
||||
3. The agent writes the compose file to `/opt/skipper/tenants/example-tenant/docker-compose.yml`, writes `.env`, and runs `docker compose up -d`.
|
||||
4. The agent sends `POST /v1/work-orders/:id/result` with a structured result object plus state updates.
|
||||
5. `skipper-api` moves the finished work order into `data/work-orders/finished/`, updates resource state, writes structured logs, and emits events.
|
||||
Reference in New Issue
Block a user