Internal, not published

The Agent-Native Manifesto

Engineering principles for building software with and for agents in 2026, and a minimal, portable infrastructure for deploying it. April 2026, revised September 2026 following review.

Part One
Software Engineering

Part One restates established software engineering principles for a setting in which agents read, write and review code alongside people.

Preamble

Our software is written, reviewed and maintained by people and agents together. An agent may draft a feature, a person may approve it, and another agent may later be asked to change it. Whoever opens the code next, person or agent, has to be able to understand it. The practices that make code readable to people also make it readable to agents.

The principles below are established practice. Agents change code frequently, so the cost of ignoring these principles appears on every change, where previously it built up over the life of a project. We therefore treat them as requirements.

One principle is listed here first because the others depend on it: every agent run leaves a record of the task it was given, the files it touched, the checks that were run, and the commits that captured its work. Without that record the work cannot be verified and nobody can be held accountable for the result. Principle 9 sets it out in detail, including what the record can and cannot prove.

Engineering Principles

1. Make the System Visible

All dependencies, side effects and decision points in a system must be visible from reading the code. When a part of the system needs something from outside itself, it should declare that need explicitly instead of looking it up itself. When running a piece of code causes something to happen elsewhere (a record changed, a message sent, a file written), that consequence should be discoverable without running the code. Hidden behaviour is a common source of defects and is hard to inspect afterwards.

2. Verify What Crosses a Boundary

Wherever data moves between parts of a system, its shape and meaning must be described and checked at that point. This applies equally to the boundary between a browser and a server, between two services, and between one piece of code and the data returned to it. An unchecked boundary relies on assumptions about the data, and a wrong assumption often causes a failure some distance from where the data entered.

3. Build From Small, Self-Contained Units

A system should be composed of units that each have a single responsibility, a clearly described input, and a clearly described output. In the browser, this unit is the component. On the server, it is the module or service. Each unit should be understandable without reading the rest of the system. Units are combined into larger features, and features into the application. Maintenance is easier when a change can be made to one unit without reading the rest.

4. Separate Independent Concerns

Business rules, data storage and presentation are kept in separate parts of the system. A change to the way data is stored should not require changes to business rules. A change to the appearance of a screen should not require understanding how users are authenticated. Separated this way, each part can be inspected, tested and replaced independently.

5. Let Tests Document the Intent

Tests confirm that the system behaves as intended and record that intention for whoever reads them next. Well-written tests describe what the code is supposed to do in language close to the problem itself. When a person or an agent opens the code for the first time, the tests should answer the question what is this supposed to do before the implementation is ever read.

If an agent writes both the code and the tests for it, the tests show that the code does what the agent understood the task to be. They do not show that the agent understood the task correctly. The tests are therefore split by author:

The engineer writes the schema for each boundary and the acceptance tests for each feature before an agent starts on the implementation. They define what counts as correct. An agent may not change them; if a change is needed, the engineer makes it and records why.
The agent writes the unit tests for its own implementation. These are useful for catching regressions and for documenting how the code works, and they are held to the same standards as the code. They are not evidence that the feature is correct.

Acceptance tests can only check what they can observe at a boundary. Some rules live inside it and produce no visible failure when they are wrong: a retention period set to the wrong number of days, an authorisation check that admits one tenant too many, a currency conversion applied the wrong way round. No test written by the same person or agent that wrote the rule can prove the rule is right. For each feature the engineer keeps a short written list of these rules, and checks each one by hand against the source it came from (the contract, the policy, the regulation) before the feature is released. On a one-person project that check is done by the same engineer on a different day from the implementation, with the list in front of them.

6. Capture Reasoning Alongside the Code

Commit messages should explain why a change was made as well as what changed. Architectural decisions should be recorded with the alternatives considered and the trade-offs accepted. When agents are used to produce code, the instructions they were given should be stored and versioned as carefully as the code itself. Both are handed over to future maintainers. The agent-specific extension of this principle, logging the agent’s reasoning, prompts, file accesses, and commit references in a structured form, is set out in principle 9.

7. Define Every Interface with a Schema

Any exchange between independent parts of a system (between services, between agents, between a client and a server) must pass through a schema. A schema describes what the data looks like, what values are allowed, and how changes to the shape will be handled over time. Every exchange is validated against its schema at the point of crossing. A validation failure at the boundary is easier to diagnose than a failure further into the system.

8. Record Events Where History Matters

In the parts of a system where history has value (financial transactions, regulated decisions, actions taken on behalf of a user by an agent), every change is stored as a recorded event instead of overwriting the previous value. This gives a complete, ordered record of what happened and why, at the cost of extra complexity, so the pattern is used only where the record justifies that cost.

9. Keep a Record of Every Agent Run

Where an agent contributes to a system, each run leaves a record that a person can read later to find out what was asked, what was done, and how it was checked. The record is a file committed to the repository in the same change as the code it describes, so it is versioned, reviewed, and handed over with the source. The commit message names the record.

Each record holds, verbatim:

The task the agent was given and the instruction files in force at the time, by path and version.
Each tool call the agent made, with its arguments and result.
The files the agent read, by path and version, and the files it created, modified, or deleted.
The diff, and the commit hash or hashes that carried it.
The test output, type-check output, structure gate results, and any other verification evidence captured at the time.
The review record: findings, verdicts, and citations from the reviewer and critic.
Work that was tried and abandoned, and the stated reason.

Where the model vendor exposes the model’s reasoning text, it is kept in the record as well. It is labelled as the model’s own account of its work. Vendors summarise or withhold this text at their discretion, and research on these traces has found that they can leave out the factors that actually determined the output. The record proves what was done and what was checked. It does not prove why the model chose what it chose, and it is not presented to a client or a regulator as if it did.

The record must not contain anything that cannot stay in the repository for its whole life, because a committed file cannot be reliably removed from history. The following rules therefore apply:

Secrets are never placed in a task or an instruction. Agents read credentials from the same credential store as the running service, and the record is scanned for secret patterns before it is committed.
Personal data is not placed in tasks or instructions where the work can be done without it. Where it cannot, the record is redacted before commit, and the redaction is noted in the record.
Client data in the record is limited to what the run needed. The client’s contract names the retention period for the repository, and that period applies to the records in it. Where personal data remains in a record, the lawful basis for holding it is named in the same contract.

The run record, with its references to commits, files and verification output, allows the history of the system to be reconstructed for audit.

10. Enforce Structural Limits

Code written by agents tends to get longer and more tangled with each change. Functions grow, branches are copied instead of being extracted into shared code, and the type checker and the tests both pass throughout. Review of intent and architecture does not detect this, so it is measured.

Every repository enforces the following in continuous integration as merge blockers (a failure blocks the merge):

A cyclomatic complexity ceiling per function.
A size ceiling per file and per module.
Duplication detection, with a ceiling on the proportion of duplicated code.

The ceilings are set per repository, recorded in version control, and lowered over time. A change that breaks a ceiling is fixed before it merges. Raising a ceiling is a decision that is recorded with its reasons.

The same measurements are recorded on every merge and kept as a time series per repository. The trend is reviewed at a fixed interval. If complexity, size, or duplication is rising, refactoring work is scheduled before further feature work. A fixed share of each iteration is reserved for refactoring whether or not the trend calls for it, because better prompting has been found not to stop the decay.

Applying the Principles to Browser-Based Systems

Most of the software we deliver reaches its user through a web browser. The following is how the principles above translate into a working architecture for that environment.

The Browser

The interface is built from small components, each with clearly defined inputs and outputs
Shared state is held in stores that can be read and inspected without running the application
A consistent set of interface patterns is used across the whole product, so that the same ideas always look and behave the same way
Every component has tests that describe the behaviour it is meant to produce

The Interface Between Browser and Server

The shape of every request and every response is described by a schema held in version control
The schema is written before the implementation, and the implementation is required to conform to it
Data entering and leaving the server is validated against the schema at the boundary

The Server

The business rules of the application live in their own layer, separate from the code that stores data and the code that handles network traffic
Each piece of the server receives the things it depends on from outside itself instead of creating them internally
Where the history of changes has business value, changes are recorded as events instead of overwriting earlier values
The server keeps a structured log of what happened and why, in a form that can be read and analysed later

The Workflow

Agents produce code against defined schemas and described interfaces
The engineer writes the schema and the acceptance tests first; the agent implements against them and writes its own unit tests; neither may alter the acceptance tests
Every change includes the tests that demonstrate it works and the reasoning that explains why it was made
Rules that tests cannot observe are listed per feature and checked by hand before release
Every change passes the structure gates (complexity, size, duplication) before it can merge; a failing gate blocks the merge
Human review covers intent, architecture, and the meaning of the change; structural decay is caught by the gates, not by reading every line
The structure measurements are reviewed as a trend at a fixed interval, and refactoring is scheduled from that review
Every change is reviewed by a reviewer agent and a critic agent under the written protocol below, and the person reads the review record before merging
The instructions given to agents, including the review protocol, are versioned alongside the code they produce

Review

Every change is reviewed by agents before a person sees it. The protocol is a file in each repository, versioned with the code and the other agent instructions, so that the review a change received can be reproduced later. The protocol has three roles:

Implementer. The agent that made the change. It takes no part in the review.
Reviewer. A second agent, run on a different model family from the implementer where one is available. It reads the change against the task, the schema, and the acceptance tests, and writes a review.
Critic. A third agent that audits the review. It looks for defects the reviewer missed and for findings the reviewer raised and then dropped.

The change is frozen while review runs. Nothing is edited until the review is complete and the person has read it.

Agents reviewing each other’s work tend to agree with each other. The rules below are intended to prevent that:

Every finding raised by the critic ends with one of three verdicts: confirmed, citing the file and lines that show the defect; refuted, citing the file and lines that show why it is not a defect; or out of scope, naming the task boundary it falls outside.
The reviewer may not withdraw a finding in response to argument. A finding is withdrawn only by citing code that refutes it.
Anything the reviewer or critic wants changed that is outside the task is recorded as a ticket. It is not added to the change. A review that adds changes beyond the task is treated as failed.

The engineer reads the review record: the findings, the verdicts, and the citations. They read the diff itself where a finding points them to it, and wherever the change touches a boundary, an authorisation decision, or a rule on the hand-checked list. They do not read every line, and the process does not depend on them doing so.

The review record is retained with the run record described in principle 9.

Part Two
Infrastructure Architecture

A minimal, portable server architecture for running agent-native web applications. The same engineering principles apply: visibility, separation, verification at boundaries, and a preference for the smallest number of parts that solve the problem.

Guiding Principles

Fewest Components

Each component in the stack needs a stated reason to be there. If a capability can be met by something already present, nothing new is introduced. Each additional piece of software has to be secured, patched and monitored, and can fail in unexpected ways. Components are not added because they are popular.

Actively Maintained Projects Only

Every piece of software in the stack must have an active maintainer community, regular security releases, and a clear line of succession if the primary maintainer steps back. Projects that are stagnant or abandoned are replaced early, so that the replacement can be chosen before it is needed urgently.

Portable Across Providers

No component may depend on a specific hosting provider to function. Everything in the stack must be able to run on any infrastructure capable of hosting a Debian virtual machine. If a tool requires a proprietary service or a provider-specific interface to operate, it is not considered for use.

One Consistent Stack Across All Servers

Every customer server runs the same base stack, installed by the same automation. Differences between servers are documented, version-controlled, and justified. Configuration drift, the slow divergence of servers over time, is treated as a defect and corrected.

Secure by Default

Every server is hardened when it is first built. Security patches for the operating system are applied automatically. Services run under accounts with only the privileges they need. Secrets are encrypted in storage and provided to services only at the point of use. Remote access is by key only; password login is disabled.

Infrastructure Overview

Each customer application runs on its own dedicated Debian virtual machine. We arrange hosting for every application: some run on our own infrastructure, some with a hosting partner, and some on the client’s own servers. The default deployment target is a Proxmox cluster at Krystal Hosting. The same machine images and configuration deploy unchanged to any of these, so the architecture is the same in each case and the client can move the application to another host. A separate machine per customer means that a failure, compromise or misbehaviour on one machine does not reach another, unless the hypervisor itself is compromised.

A customer server runs the application and nothing else. The things that define and operate it (source code, deployment automation, secrets, backups, log collection, alerting) live on a separate set of machines that we operate, called the control plane. The reason for the split is that a compromised application should not hand the attacker the repository, the history, the inventory, and the means to rebuild every other server. It also means a customer server can be rebuilt from the control plane when the server itself is gone.

The Control Plane

The control plane is small and is built by the same automation as everything else. It consists of:

One Forgejo instance, with an organisation per customer, holding the application source, the infrastructure definitions, and the run records
One control node from which Terraform and Ansible are run, and on which application artefacts are built
One backup host, which pulls backups from customer servers and holds them
One collection point for logs and alerts

Customer servers hold no credential that reaches the control plane or any other customer server. The control plane holds the credentials that reach customer servers, each limited to the one job it does (deploy, pull backups, receive logs). The control plane is backed up to a second location and to an offline copy at a fixed interval, and its own rebuild is tested on the same schedule as customer restores. The control plane is a single point of control for every customer. That risk is accepted so that customer servers hold nothing except the application, and the control plane is defended and backed up accordingly.

Where a client hosts themselves, they either run their own copy of the control plane from the same automation, or receive the repository by push mirror and the deployment automation as part of handover. The customer server is the same in either case.

Layer	Technology	Purpose
Hosting	Proxmox cluster	Creates and runs virtual machines
Operating System	Debian (stable release)	Base system on every machine
Machine provisioning	Terraform	Describes and builds virtual machines. Licensed under the Business Source Licence since 2023; OpenTofu is the open fork and the fallback if the licence tightens further
Configuration	Ansible	Installs software, deploys, and recovers
Secrets	ansible-vault	Encrypted storage of credentials and keys
Source and records	Forgejo (control plane)	Holds code, automation, and run records for every customer

Application Stack

The application stack uses a single programming language from the browser to the server. A shared language reduces the number of things that have to be understood to work on any part of the application, and it allows the description of data to be written once and used in both places. TypeScript is used because it is widely supported, statically type-checked, and runs in both the browser and on the server.

Web Traffic and Transport Security

NGINX handles all incoming traffic and terminates encrypted connections
Let’s Encrypt provides the certificates, renewed automatically
NGINX forwards plain traffic to the application on an internal port
The application does not handle encryption directly

NGINX accepts encrypted traffic on the standard secure port, decrypts it, and passes it to the application running on a local port. Separating transport security from application logic means that the application has fewer concerns and the security-sensitive part is handled by software specialised for it.

Application Runtime

Node.js runs the server-side application, installed from the Debian package so that it is patched by the same mechanism as the rest of the operating system
The application is written in TypeScript and compiled before it runs
The server uses a minimal web framework (Express or Fastify)
The browser interface is built in TypeScript using React or Vue
The shape of every exchange between browser and server is described in a shared schema

Dependencies and Supply Chain

The npm registry is the largest attack surface in this stack. Packages have been hijacked, malicious packages have been published under names close to popular ones, and install scripts have been used to run code on developer machines. The following rules apply to every application.

The lockfile is committed. Installs use the lockfile exactly and fail if it does not match the manifest.
Package install scripts are disabled. A package that needs one to work is not used.
A newly published version of any package is not adopted until it has been public for a set number of days, so that a hijacked release is likely to have been reported and pulled before it reaches us.
Dependencies are updated on a weekly schedule by an agent under the normal review process. Advisories rated high or critical are fixed and deployed within fourteen days, which is the window Cyber Essentials expects.
A software bill of materials is produced for every release and kept with the run record for that release.
The application is built on the control node. The customer server receives a built artefact containing only production dependencies. npm is not installed on a customer server and no build runs there.

Node.js itself comes from the Debian package. Debian’s version lags upstream, and its upstream support period may have ended, but the Debian security team maintains the package for the life of the release and unattended-upgrades applies its fixes without further work. That is preferred over a newer version from a third-party repository that no automatic mechanism would patch. If an application needs a feature the Debian version lacks, that is recorded as an exception with its own patching arrangement.

Remote Access

OpenSSH is the only means of remote access to any server
Authentication is by key only; passwords are disabled entirely
The same secure channel is used for development, deployment, and backup retrieval

Data Storage

Data storage follows the same rule as the rest of the stack. Two database engines (PostgreSQL and SQLite) and Redis for queues and caching cover the expected needs, and no further storage systems are added unless a use case cannot be met by these.

Engine	Use	Notes
PostgreSQL	Main relational storage	One database per customer. Also handles searchable embeddings through the pgvector extension.
SQLite	Smaller applications that do not need a separate database server	File-based and requires no running service. Suited to applications whose data is kept alongside the application.
Redis	Message queues, caching, publish-subscribe	A single lightweight service covering several needs that would otherwise require separate tools.

Vector Storage Inside PostgreSQL

Applications that need to store and search embeddings for agent features use the pgvector extension inside PostgreSQL. This avoids adding a separate vector database to the stack. One database engine handles both ordinary records and embeddings, keeping the number of running services to a minimum.

Redis for Queues and Notifications

Redis provides a small, fast layer for passing messages between parts of the application and for caching. It replaces what would otherwise be several separate services: a message broker, a cache, and a publish-subscribe system.

Redis changed its licence in 2024 and again in 2025; the version packaged in Debian 13 is under the AGPL, which is acceptable for a self-hosted service. Valkey, the Linux Foundation fork, is protocol-compatible and also packaged in Debian 13. If the Redis licence or its maintenance changes again, Valkey is the replacement, and the application code should not depend on anything Valkey lacks.

SQLite databases are exported while in use with the database’s own online backup command (sqlite3 db.sqlite ".backup export.sqlite", or VACUUM INTO), which produces a consistent copy without pausing writers. The filesystem snapshot collects the export, never the live file.

Version Control

Source code is held in a single Forgejo instance on the control plane, with one organisation per customer. Forgejo is a lightweight, self-hosted, Git-compatible version control server, a community fork of Gitea, actively maintained, and it runs as a single process.

One Forgejo instance, on the control plane, with an organisation per customer
The application source, the infrastructure and deployment definitions, and the agent run records live in the customer’s repositories
Customer servers run no git server and hold no repository credential; they receive built artefacts from the control node over SSH
Forgejo provides the web interface for reviewing changes and reading run records
At handover, or on request, each repository is push-mirrored to a repository the client owns, so the client holds a current copy of everything needed to rebuild their server, whether or not we are involved

An earlier version of this document placed a Forgejo instance on each customer server so that the history of the code lived beside the code. That put a second web application with its own login on every production machine, and it meant that when a server was lost, the automation needed to rebuild it was lost with it. The push mirror gives the client the same ownership of their history without either problem.

Secrets Management

Credentials, keys, and other sensitive values are managed using ansible-vault, the encryption facility built into Ansible. Keeping secrets management inside the tool that already manages configuration avoids adding another running service to the stack, and keeps the solution portable.

Storage

Secrets are stored in encrypted files inside the Ansible repository on the control plane
Encryption uses AES-256; the vault is decrypted only on the control node, only during a deployment
No secret is stored in plain text in any repository

Delivery

Not yet defined: how a decrypted value reaches a running service on the customer server, and in what form it rests there. An earlier draft said services would receive secrets as environment variables and that no secret would ever be in plain text on disk. Those two statements do not both hold: the usual Ansible pattern writes an environment file to disk, and environment variables are inherited by child processes and readable through the process table. The delivery mechanism, the at-rest form on the customer server, and whether any two services on one server ever share a credential are recorded as open questions below.

Rotation

Secret rotation happens as part of the deployment cycle. When a secret changes, the encrypted file is updated, the change is committed, and the deployment process picks up the new value on the next run. Critical values such as database passwords and third-party access tokens are rotated on a regular schedule as part of ordinary maintenance.

Scope

The secrets managed by this system include database credentials, keys used for communication between services on the same server, tokens for version control access, private keys for certificates where those are not handled automatically, backup encryption keys, and credentials for any third-party services the application depends on.

Backup and Recovery

Backups operate in two layers. The first layer captures the contents of each database using the database’s own export tools. The second layer captures the entire filesystem, including the exported database files. Recovery is performed by running the same automation that built the server in the first place, pointed at a blank machine and a set of backup files.

Database-Level Backups

Each server has a directory of utility scripts for maintenance tasks
These scripts produce consistent exports of each database using the appropriate tool (pg_dump for PostgreSQL, the online backup command for SQLite)
The scripts handle any locking, coordination, and compression required
Ansible deploys the scripts and schedules them through the system’s scheduler
The export schedule is set per customer from the agreed recovery point; the exported files are held locally on the server for the backup host to collect

Filesystem-Level Snapshots

rsnapshot (which uses rsync internally) on the backup host takes incremental snapshots of each customer server’s filesystem
The backup host pulls the backups, using an SSH key for each customer server that is restricted to read-only rsync of the backup paths and nothing else. Customer servers hold no key to the backup host or to anything outside themselves.
Each snapshot includes the application code, configuration, and the database exports from the layer above
Hard links between snapshots keep the storage cost manageable as generations accumulate
Snapshots run on a schedule, typically nightly, with a retention policy set per customer
The backup host’s disks are encrypted at rest. rsnapshot itself does not encrypt, so any copy sent to an external service or to another site is encrypted with age (packaged in Debian) before it leaves the backup host. The backup encryption keys held in the vault are the keys for these copies.

Recovery Objectives

The recovery point (how much data can be lost) follows from the export and snapshot schedules, and the recovery time (how long the service can be down) follows from the size of the data and the speed of the rebuild. Both are agreed with each customer, written into their contract, and recorded in the Ansible configuration for their server. The schedules are set to meet them. The phrase “a short period of downtime” is not an objective and is not used.

Restore Drills

Every quarter, each customer server is restored from its most recent backup onto a scratch virtual machine by the same automation used for real recovery, the application is checked against its acceptance tests, and the elapsed time and the result are recorded. The control plane is restored from its off-site copy on the same schedule. A drill that fails is treated as an incident.

Recovery Process

When a server is lost, the recovery sequence is:

From the control node, Terraform creates a replacement virtual machine on the Proxmox cluster
Ansible installs the standard stack on the new machine
Ansible restores the customer data from the most recent backup on the backup host
Ansible restores the databases from the exported files
Services are started and verified

Nothing needed for recovery lives on the server being recovered. The automation and the inventory are in Forgejo on the control plane, the backups are on the backup host, and the secrets are in the vault on the control node. The whole process is automated and repeatable, requiring no manual intervention beyond the instruction to start it.

If the control plane itself is lost, it is rebuilt first from its off-site copy, and customer servers are then recovered from it. Both sequences are drilled (see Restore Drills above).

Logging

Logging works at two levels. Each server keeps its own logs for local diagnosis and for anything with legal or contractual weight. In addition, logs from servers we operate ourselves are forwarded to a central location so that the infrastructure can be observed as a whole.

On Each Server

The application writes structured logs to disk in a machine-readable format
Operating system logs are handled by the standard system logger
Log files are rotated automatically to prevent disk exhaustion
Each server is a self-contained unit of logging, usable without any external service

Central Collection

For servers that we host and manage, logs are also forwarded to a central collection point. This gives us visibility across the estate for operational awareness, security analysis, and trend detection.

The per-server logging process forwards entries to the central collection point
Central collection covers every server we operate ourselves
Retention periods are set per customer at the point of provisioning

Retention

How long logs are kept is decided per customer at deployment and held in the Ansible configuration. It can be changed without redeployment where required.

Alerting

Logs record events but do not notify anyone. Alerting is a separate mechanism and works in two directions, so that a server that stops reporting raises an alert in the same way as one that reports a fault.

From the Server

Every service unit has a failure hook that sends a message to the control plane when the service fails or fails to restart
A scheduled health check on each server tests the application, the database, disk space, and certificate expiry, and reports the result to the control plane
Both use one outbound channel over HTTPS to the control plane and nothing else

From the Control Plane

The control plane probes each customer server’s public address over HTTPS at a short interval and raises an alert when the probe fails or the certificate is near expiry
A server whose scheduled health check does not arrive on time raises an alert, so silence is treated as a fault
Alerts go to a named person by a channel that does not depend on the servers being monitored

Where a client hosts themselves, the server-side hooks and health check are pointed at a collector the client runs, or at ours where the contract says so, and the external probe is run from whichever control plane operates the server. The tool that receives alerts on the control plane has not been chosen and is listed under Open Questions.

Security Baseline

Every server is hardened when it is first built. The baseline is part of the standard configuration; it is applied automatically and is identical across all customer servers.

Debian stable with automatic security updates enabled
Remote access by secure-shell key only; password authentication is disabled
Only three inbound ports are permitted: secure shell, web traffic redirect, and encrypted web traffic
NGINX handles transport encryption using certificates from Let’s Encrypt
All secrets are encrypted at rest using ansible-vault
Each service runs under its own dedicated account with only the privileges it needs
No unnecessary packages are installed; the base image is kept minimal. No git server, no npm, and no build tools on a customer server
Software versions are pinned and recorded in the Ansible configuration

Patching

Operating system security patches, including Node.js, are applied automatically through Debian’s unattended-upgrades mechanism. Application dependencies are patched through the weekly update described under Dependencies and Supply Chain: the change is made in the repository, reviewed, built on the control node, and deployed by Ansible, with Forgejo recording every step. Critical patches are deployed immediately, without waiting for the next cycle. In both cases the target is fourteen days from publication to deployment.

Full Stack Reference

The following table lists every piece of software permitted in the standard stack. Nothing else is installed on a customer server without explicit documentation of why the addition was necessary.

Component	Technology	Purpose
Hosting	Proxmox cluster	Hosts virtual machines
Operating System	Debian (stable)	Base system
Machine provisioning	Terraform	Creates virtual machines (control plane)
Configuration and deployment	Ansible	Installs software, deploys, recovers
Secrets	ansible-vault	Encrypted credentials
Web server	NGINX	Traffic handling, transport encryption
Certificates	Let’s Encrypt	Automated certificate management
Application runtime	Node.js	Runs the server-side code
Application language	TypeScript	Used for browser and server
Browser framework	React or Vue	Component-based interface
Main database	PostgreSQL	Relational storage
Embedding search	pgvector extension	Vector storage inside PostgreSQL
Small-scale database	SQLite	Where a full database service is unnecessary
Messages and cache	Redis	Queues, caching, publish-subscribe
Version control	Forgejo	Code, automation, and run records (control plane)
Remote access	OpenSSH	Development, deployment, retrieval
Filesystem backup	rsnapshot	Incremental snapshots pulled by the backup host (control plane)
Off-site backup encryption	age	Encrypts copies before they leave the backup host (control plane)
Database backup	pg_dump and companion scripts	Application-level exports
System logs	syslog / journald	Per-server system logging
Application logs	Structured records on disk	Machine-readable application logs
Log rotation	logrotate	Prevents log files from growing without limit
Security updates	unattended-upgrades	Automatic Debian security patches, including Node.js
Application build	npm on the control node only	Builds artefacts; never installed on a customer server
Alerting	systemd failure hooks, health check, external probe	Receiving tool on the control plane not yet chosen

Open Questions

The following items are not yet resolved. They are recorded here so that each one is decided explicitly.

Central Log Collection Tool: The architecture requires central log collection for servers we operate, but the specific tool has not been chosen. Candidates include Loki and a simple forwarding arrangement using the standard system logger. The selection must remain portable and as small as possible.

Secret Delivery to Running Services: How a secret decrypted on the control node is delivered to a service on the customer server, and in what form it rests there, is not yet decided. Environment variables and environment files are ruled out because they leave the value readable on disk or through the process table. Candidates include the credential facility built into systemd, which seals each value against the host and exposes it only to the one service that loads it, and an application-side read from a root-owned file at startup. Whether any two services on one server ever need the same credential is decided at the same time; with version control moved to the control plane, no such case is currently known.

Alerting Tool: The component on the control plane that receives failure hooks and health checks, runs the external probes, and delivers alerts has not been chosen. It must be self-hosted, small, and able to deliver to a person by more than one channel. Candidates include Uptime Kuma and a scheduled script with an email and messaging gateway.

Structure Gate Thresholds: Principle 10 sets ceilings on complexity, size, and duplication but the starting values, and the interval at which the trend is reviewed, are not yet set. They will be calibrated on the first two repositories built under this document and recorded there.

Log Retention Periods: The retention period for logs is set per customer, but the defaults and the legal or contractual minimums for each customer category are not yet defined.

Redis Persistence: Whether Redis should keep its data across a restart depends on the use. For caching, no persistence is needed. For message queues with work in progress, some level of persistence is required. The persistence configuration needs to be defined for each use.

Summary

The architecture uses few components. Each is present because no other component in the stack meets the same need. One server per customer keeps customers isolated from each other. Terraform and Ansible, run from a control plane that holds nothing a customer server needs at runtime, give repeatable builds and automated recovery. Using one language from browser to server means the people and agents working on the code deal with one language and one set of tools.

A technology proposed for this stack is added only if it meets a need that existing components cannot. If the capability can be met by something already here, the existing component is used.