# Backups and upgrades

> Source: https://docs.clonepartner.com/deployment/backups-and-upgrades/

A recoverable Envoy backup includes more than a container image. Protect the application state, the encryption material required to read it, and every file-backed connector database your workloads depend on.

## Backup inventory

For a standard Docker deployment, account for:

| Asset | Typical location | Why it matters |
|---|---|---|
| Server configuration | `/opt/envoy/state/envoy-server.yaml` | Backend, auth, retention, executor, and branding settings |
| SQLite state | `/opt/envoy/state/envoy-server.db` | Users, connectors, tasks, jobs, runs, triggers, dashboards |
| Data files | `/opt/envoy/data` | CSV/YAML inputs and outputs |
| Connector databases | `/opt/envoy/dbs` | Mapping, staging, and workload-specific SQLite data |
| Encryption key | Protected secret store | Required to decrypt in-database secrets |
| Executor key | Protected secret store | Required for split-runtime communication |

If state or logs are in PostgreSQL or MSSQL, use that service's backup inventory instead of assuming `/app/state` contains them.

## SQLite with Litestream

The Docker entrypoint can generate a Litestream configuration for SQLite state.

For an S3-compatible target, configure:

```dotenv
LITESTREAM_S3_ENDPOINT=https://...
LITESTREAM_S3_BUCKET=envoy-backups
LITESTREAM_S3_ACCESS_KEY_ID=...
LITESTREAM_S3_SECRET_ACCESS_KEY=...
LITESTREAM_S3_PATH=production/envoy-server.db
```

The entrypoint enables this path only when the state backend is SQLite and all required replica credentials are present.

To include connector SQLite databases:

```dotenv
LITESTREAM_DBS_DIR=/app/dbs
LITESTREAM_S3_DBS_PATH=production/dbs
LITESTREAM_DBS_PATTERN=*.db
```

The directory watcher discovers matching databases created after startup.

On startup, if the state database does not exist and a Litestream config is present, the entrypoint attempts restore before starting the server. It does not overwrite an existing database.

:::callout{type="warning"}
Replication is not proven recovery until you restore it in an isolated environment. Test the state database, connector databases, and encryption key together.
:::

## Managed databases

For PostgreSQL, MSSQL, and Azure SQL:

- enable the provider's automated backups;
- document point-in-time recovery limits;
- protect connection credentials separately;
- include the server YAML and encryption key in the recovery set;
- confirm run-log and metrics databases are covered if they use a separate backend.

Azure Key Vault recovery settings are independent from Azure SQL recovery. Enable soft-delete and purge protection according to your policy.

## Upgrade preparation

Before changing an image or binary:

1. Review release-specific migration instructions.
2. Take or verify a current backup.
3. Record the deployed image tag and configuration.
4. Check for active task and job runs.
5. Confirm the new build includes every connector used by stored tasks and jobs.
6. Plan control-plane and executor restart order.

Review [Seed import and upgrades](/deployment/seed-import-and-upgrades) before relying on bundled definitions to update existing database records.

## Drain active work

Do not recreate an executor while it owns active task processes. The remote deployment script checks active leases and can wait:

```bash
WAIT_FOR_DRAIN=true ./scripts/deploy-remote.sh
```

If a run must be stopped, stop it through Envoy and wait for a terminal status before deploying.

## Apply the upgrade

Use immutable image tags where possible:

```bash
./scripts/deploy-remote.sh
```

For a scoped change:

```bash
DEPLOY_SCOPE=cp ./scripts/deploy-remote.sh
DEPLOY_SCOPE=ex ./scripts/deploy-remote.sh
```

The Azure script supports the same `DEPLOY_SCOPE` values. Terraform users should review `terraform plan` before applying.

## Verify after upgrade

Check:

- `/api/health` reports a healthy server;
- expected executors are healthy and heartbeating;
- the UI loads under the configured browser origin;
- stored connectors can be decrypted and tested;
- bundled seed import did not create unexpected duplicates;
- one bounded run completes with logs and metrics;
- triggers remain enabled as intended;
- public dashboard controls have not become more permissive.

## Rollback

Rollback normally means redeploying the prior immutable image against the same state and keys. Do not restore an old database solely because an image rollback is needed unless schema compatibility requires it and the rollback procedure explicitly says so.

Never run `docker compose down -v` as an upgrade or rollback step.