The proof, published whether it flatters us or not.

Client databases restored in this drill: 1, of which 1 came back verified. Which database it was is deliberately not published. The drill ran on 2026-07-24T21:40:12Z. The backup it used was 19.2 hours old at the moment of restore, because the drill deliberately uses the dump that already existed rather than one made for the occasion. 399 tables came back. About 25092 rows were counted, estimated from the restored catalogue rather than compared row by row against the source. The restore and verification together took 3.8 seconds. The drill passed. The throwaway database was destroyed at the end of the run, so nothing is left behind and no copy of client data sits around waiting to be forgotten.

Read the raw restore log →

Nightly database dump

Every database is dumped locally at 02:30. Until 24 July 2026 that dump covered the database only: attachments and uploaded documents were not in it, and the filestore artefact in every stored backup taken before that date is 13 bytes. The code fix was deployed on 24 July 2026, and the first backup that contains documents is the run of the night of 24–25 July 2026. Engine: pg_dump. Retention: 30 days. Last local copy: 02:30.

Separate off-site copy

A second copy is taken every day at 03:30 and held with a different provider, so one provider losing a machine — or losing us as a customer — does not take the backups with it. The retention target is 30 daily, 4 weekly, 6 monthly and 2 yearly copies, and there is a three-day hole in it: the local backups and the off-site copies for 21, 22 and 23 July 2026 are missing after a disk-full incident on those days. Publishing that is the reason this page exists. Last off-site copy: 03:30. Provider is separate from the one running the databases.

Point-in-time recovery

Write-ahead logs are archived, so a database can be rolled forward to a chosen moment rather than only to last night's dump — which is what you need when the damage was a bad import at 14:10 rather than a dead disk. The local point-in-time window covers the period since 23 July 2026 at 15:03, and nothing earlier than that.

Backup encryption — a limitation

Transport is TLS. The backups themselves are not encrypted at rest, locally or off-site, and the local pgBackRest repository shares a disk with the database it protects. Both of those are being changed, and until they are, this is what you are buying.

Streaming replication — asynchronous today

PostgreSQL streaming replication to one standby machine, asynchronous today, with synchronous mode in preparation. If the primary is lost, work committed in the last seconds may not have reached the standby. Mode: async. One standby is configured.

Replication lag alerts

A warning is raised at 30 seconds of lag and a critical alert at 120 seconds, because a standby that has quietly fallen behind is a backup you do not have. Both thresholds are live and recorded; like the other 60-second checks, they are not yet routed to a channel that wakes anybody up.

Fault detection — and where it stops

Checks run every 60 seconds across the databases, the containers, the disks and the backup timers, and every result is recorded. Today only two of them reach a human: a heartbeat and a disk watchdog. Routing for the remaining checks is not yet wired up to any channel, so a non-fatal fault may sit in the log until the next working morning.

Autonomous repair, with a rollback

Routine faults — a stalled worker, a container that will not come back, a filling disk — are repaired automatically. The repair then re-reads the real state of the machine, and if the state has got worse it rolls its own change back and escalates to the operator instead of trying again.

Failover is operator-assisted — a limitation

There is no automatic promotion. When the primary dies a human notices the missing heartbeat, confirms how far the standby has replayed and switches over, so recovery is measured in minutes rather than seconds. Anyone selling you a percentage instead of a mechanism is selling you the percentage. Confirmed in the document: automatic failover is off.

1. Detection A gap in the heartbeat

   The primary stops sending the heartbeat it is expected to send, and the gap is what raises the alarm — the heartbeat and the disk watchdog are the two checks that notify a human today. The 60-second checks record the rest of the state, including replication lag past 30 seconds, but their routing is not yet wired up to any channel, so a fault that is not fatal may sit in the log until the next working morning.

2. Assessment by a human 2 to 10 minutes, longer overnight

   The operator decides whether the primary is dead or merely slow, and reads how far the standby has replayed. This is the step that costs the most time, and it is the honest reason we publish no uptime percentage: one operator asleep at 04:00 is a real delay, and pretending otherwise would be the part of this page you should not believe.

3. Promotion of the standby 1 to 3 minutes once the decision is made

   Replication is asynchronous today, so the standby holds what it has managed to replay and no more: work committed in the last seconds before the failure may not have reached it. The operator reads how far the standby got, promotes it, the application is pointed at it, and sessions reconnect. Anything that did not reach the standby has to be re-entered, and work in flight at the moment of failure is retried by the user.

4. Verification and rebuild Same working day

   The promoted database is checked, the backup timers are re-pointed at it, and the failed machine is rebuilt and re-attached as the new standby so the pair is whole again. The 02:30 dump and the 03:30 off-site copy resume on their normal schedule, and the next weekly drill restores from the new primary.

Next scheduled drill: 2026-07-26T03:30:00Z.