# Git host request budget

Source: https://codeherder.com/docs/self-host-git-host-budget/

See how many requests your server and devices send to GitLab, why work pauses on a 429, and which server settings tune the pacing.

A shared GitLab token has a request limit. On gitlab.com, many agents and devices can reach it fast. CodeHerder counts every git-host request and paces them on gitlab.com so work slows down before GitLab refuses it. This page shows how the pacing works, where to read the numbers and what to change.

It applies to a self-hosted server. For a full git-host outage, see [When GitHub or GitLab is down](https://codeherder.com/docs/self-host-git-host-outage/).

## How requests are counted

The server and its devices count each request to the git host. They count it against two things:

- **The identity** that sent it. This is a connection, the GitHub App, a device’s own token or a short fingerprint of a credential. It is never the token itself.
- **The top-level group** of the repository. A group is the first part of the repository path.

## How GitLab is paced

On gitlab.com the server spends a token from a bucket for each request. A bucket refills at a steady rate. These are the defaults:

| Bucket | Default |
| --- | --- |
| One identity | 80 requests a minute |
| One group | 350 requests a minute and 4,000 requests an hour |

GitLab allows 100 a minute per identity, 400 a minute per group and 5,000 an hour per group. The defaults leave some room.

When a bucket runs low, the work that matters most goes first:

- **Interactive** work never queues behind lower work. It waits up to 30 seconds for a token. This covers agent git, provisioning and actions you take.
- **Event** work waits up to 10 seconds. This covers webhooks, status checks and merge request notes.
- **Background** work never waits. The server skips it for now, and it runs again later. This covers merge polling, repository reconcile, mirror clean-up and credential probes.

Other git hosts are counted but not paced by these buckets.

## What a 429 does

When the git host answers `429`, CodeHerder pauses that identity. It does not send more requests with it until the pause ends.

- If the reply names a reset time, the pause lasts until then.
- If it names none, the pause is 30 seconds. Each further `429` doubles it, up to 10 minutes.
- Up to 20% extra time is added, so devices do not all resume together.

A device that gets a `429` also pauses the repository’s group. It tells the server, and the server tells the group’s other devices. Git, probes and proxied requests all wait.

Paused work parks. It does not count as a failure, and the task keeps its stage. It tries again after the reset. A start that the git host refuses this way shows the reason `git_host_rate_limited`. The device does not count against the provisioning breaker. See [Release a provisioning breaker](https://codeherder.com/docs/self-host-provision-breaker/).

## Read the budget

You need an instance operator login. Run:

```
ch instance githost
```

For the same data as JSON, add `--json`. The app shows it under **Instance** → **Git host budget**.

The numbers cover the last 60 minutes. Each API process keeps its own window. A restart clears them.

The command prints three tables.

**GROUPS** has one row for each host and top-level group.

| Column | Meaning |
| --- | --- |
| `LAST HOUR` | Requests in the last hour |
| `CAP/H` and `CAP/MIN` | The hourly and per-minute limits for the group. `—` means no limit applies |
| `PEAK MIN` | The busiest minute in the window |
| `429s` | Replies of `429` in the last hour |
| `PAUSED UNTIL (UTC)` | When the pause ends. `—` means the group is not paused |
| `TOP SOURCES` | The three biggest causes of requests, with counts |

**IDENTITIES** has one row for each host and identity. It has the same columns, without the hourly cap. It adds `REMAINING/LIMIT`, the quota the host last reported for that identity.

**BUCKETS** has one row for each bucket window.

| Column | Meaning |
| --- | --- |
| `SCOPE` and `KEY` | `group` or `identity`, and which one |
| `WINDOW` | `minute` or `hour` |
| `TOKENS/BURST` | Tokens left and the most the bucket holds |
| `LIMIT` | The refill limit for the window |
| `SHED` | Requests dropped in the last hour, by class |
| `WAITED` | Requests that waited in the last hour, by class |

Hosts the server does not pace have no `BUCKETS` rows.

## Change the pacing

Set these in the server environment. Restart the server to apply them. Self-hosted servers keep the defaults unless you change them. See [Self-hosted deployment](https://codeherder.com/docs/self-hosting/).

| Setting | Default | What it does |
| --- | --- | --- |
| `CH_GITLAB_IDENTITY_PER_MIN` | `80` | Most gitlab.com requests a minute for one identity |
| `CH_GITLAB_GROUP_PER_MIN` | `350` | Most gitlab.com requests a minute for one group |
| `CH_GITLAB_GROUP_PER_HOUR` | `4000` | Most gitlab.com requests an hour for one group |
| `CH_GITHOST_PACE_BACKOFF_MIN` | `30s` | First pause after a `429` with no reset time |
| `CH_GITHOST_PACE_BACKOFF_MAX` | `10m` | Longest pause after a `429` with no reset time |
| `CH_GITLAB_MINTS_PER_MIN` | `12` | Most sandbox token requests a minute for one GitLab connection. `0` turns the limit off |
| `CH_GITHOST_BUDGET_LOG_INTERVAL` | `1h` | How often the server logs one line of request totals for each group |

Do not set a limit above what GitLab allows for your plan. The server then hits `429` instead of pacing itself. A self-managed GitLab sets its own limits, and these buckets do not apply to it.

## When work is paused

1. Run `ch instance githost`. Look for a value in `PAUSED UNTIL (UTC)`.
2. Wait for that time. Parked tasks resume by themselves. You do not need to unblock them.
3. If `429s` keeps rising, look at `TOP SOURCES`. One source that dominates tells you which work uses the budget.
4. If `SHED` or `WAITED` lists `interactive` or `event` in the last hour, the limits may be too low for your load. Raise them, within what GitLab allows.

## Limits

- The token buckets cover gitlab.com only.
- Pauses and counts live in memory. A restart clears them. A new `429` sets a fresh pause.
- With more than one API process, each keeps its own window and pauses.
- The page and the command show counts. They never show a token or a repository path.
