# Contain an incident on a self-hosted server

Source: https://codeherder.com/docs/self-host-incident-containment/

Stop agent work, end one person's access, and know when to stop the server instead. Use these steps when you suspect a compromise on a self-hosted server.

This page tells an instance operator how to contain an incident on a self-hosted CodeHerder server. Every step uses the instance plane. You must be an instance operator to run it. Every call is written to the instance audit log.

Pick the smallest step that stops the harm. Each step below is stronger than the one before it.

## Step 1: Pause new work

A pause stops new agent sessions. Running sessions keep running.

```
ch instance spawn-pause pause --root <workspaceRef> --reason "incident 42"
```

Use `--fleet` instead of `--root` to pause the whole server. A refused start answers `spawn_paused`. Read the current state at any time:

```
ch instance spawn-pause show
```

## Step 2: Freeze a root

A freeze is a pause that also stops the running sessions. Use it when a running agent is the problem.

```
ch instance spawn-pause freeze --root <workspaceRef> --reason "incident 42" --yes
```

The server stops the running sessions at once when it holds their device tunnels. A sweep repeats this every 30 seconds. That covers a device that reconnects later, and a tunnel held by a second server process. So allow up to 30 seconds after the command.

A frozen root runs no session and starts none. A task whose session was stopped parks with `spawn_paused`. Use `--fleet` to freeze every root.

A freeze persists in the database. It survives a server restart.

## Step 3: End one person’s access

`revoke-all` ends every credential one person holds. Use it when an account or a laptop is compromised.

```
ch human revoke-all <humanRef> --yes
```

It ends these credentials:

- Every API key, including MCP access tokens.
- Every MCP grant, so no refresh token mints a new access token.
- Every CLI installation.
- Every device token. It also closes that person’s device tunnels.
- Every browser sign-in made before now. A refresh token does not restore one.
- The person’s running sessions that they started on a device.

Every server process refuses the revoked credentials within 60 seconds. The command prints the counts and the bound.

`revoke-all` does not disable the person. It does not block a new sign-in. The person can sign in again and get new credentials.

## Step 4: Block a new sign-in

To keep the person out, do both of these:

1. Disable the user in your identity provider. That is the Cognito user pool, or your OIDC provider.
2. Deprovision the member in CodeHerder.

Do step 1 first. Then run `revoke-all`. Otherwise the person can sign in again in between.

## Step 5: Stop the server

Stop the server instead of the steps above in these cases:

- The host, the database, or a server key is compromised.
- You do not trust the instance plane itself.
- You must suspend the whole install. On a one-tenant install, stopping the server is the supported way to suspend everything.

Stopping the server does not stop agent processes on devices. A device keeps its running sessions when the server connection drops. It reattaches them when the server returns. So this order matters:

1. If you still trust the server, run `ch instance spawn-pause freeze --fleet --yes` first. Wait 30 seconds. Check with `ch instance spawn-pause show`.
2. Stop the server.
3. If you do not trust the server, also stop `ch device-server` on each device. Stop each device that runs agents. Those processes are the only thing that ends the running sessions.

A fleet freeze stays in force after the restart. The first sweep after the server returns stops any session that reattached.

## Step 6: Undo

Clear a pause or a freeze:

```
ch instance spawn-pause unpause --root <workspaceRef>
ch instance spawn-pause unpause --fleet
```

Sessions that a freeze stopped do not restart. Their tasks resume on the next dispatch. `revoke-all` cannot be undone. The person needs new credentials.

## Find the audit record

Every instance call is a row in the `admin_audit_log` table in PostgreSQL, with `plane = 'instance'`. A freeze is a separate route, so it is a separate row from a pause. See [Self-hosted logs](https://codeherder.com/docs/self-host-logs/) for how to read it and how to ship the server logs off the host.

## Related guides

- [Self-hosting CodeHerder](https://codeherder.com/docs/self-hosting/) — set up and run your own server
- [Self-hosted logs](https://codeherder.com/docs/self-host-logs/) — log formats and the instance audit table
- [How do I add a device?](https://codeherder.com/docs/adding-a-device/) — the `ch device-server` process on each device