CodeHerderSearch⌘KRequest access →

Git host request budget

See how many requests your server and devices send to GitLab, why work pauses on a 429, and which server settings tune the pacing.

A shared GitLab token has a request limit. On gitlab.com, many agents and devices can reach it fast. CodeHerder counts every git-host request and paces them on gitlab.com so work slows down before GitLab refuses it. This page shows how the pacing works, where to read the numbers and what to change.

It applies to a self-hosted server. For a full git-host outage, see When GitHub or GitLab is down.

How requests are counted

The server and its devices count each request to the git host. They count it against two things:

  • The identity that sent it. This is a connection, the GitHub App, a device’s own token or a short fingerprint of a credential. It is never the token itself.
  • The top-level group of the repository. A group is the first part of the repository path.

How GitLab is paced

On gitlab.com the server spends a token from a bucket for each request. A bucket refills at a steady rate. These are the defaults:

Bucket Default
One identity 80 requests a minute
One group 350 requests a minute and 4,000 requests an hour

GitLab allows 100 a minute per identity, 400 a minute per group and 5,000 an hour per group. The defaults leave some room.

When a bucket runs low, the work that matters most goes first:

  • Interactive work never queues behind lower work. It waits up to 30 seconds for a token. This covers agent git, provisioning and actions you take.
  • Event work waits up to 10 seconds. This covers webhooks, status checks and merge request notes.
  • Background work never waits. The server skips it for now, and it runs again later. This covers merge polling, repository reconcile, mirror clean-up and credential probes.

Other git hosts are counted but not paced by these buckets.

What a 429 does

When the git host answers 429, CodeHerder pauses that identity. It does not send more requests with it until the pause ends.

  • If the reply names a reset time, the pause lasts until then.
  • If it names none, the pause is 30 seconds. Each further 429 doubles it, up to 10 minutes.
  • Up to 20% extra time is added, so devices do not all resume together.

A device that gets a 429 also pauses the repository’s group. It tells the server, and the server tells the group’s other devices. Git, probes and proxied requests all wait.

Paused work parks. It does not count as a failure, and the task keeps its stage. It tries again after the reset. A start that the git host refuses this way shows the reason git_host_rate_limited. The device does not count against the provisioning breaker. See Release a provisioning breaker.

Read the budget

You need an instance operator login. Run:

ch instance githost

For the same data as JSON, add --json. The app shows it under Instance → Git host budget.

The numbers cover the last 60 minutes. Each API process keeps its own window. A restart clears them.

The command prints three tables.

GROUPS has one row for each host and top-level group.

Column Meaning
LAST HOUR Requests in the last hour
CAP/H and CAP/MIN The hourly and per-minute limits for the group. — means no limit applies
PEAK MIN The busiest minute in the window
429s Replies of 429 in the last hour
PAUSED UNTIL (UTC) When the pause ends. — means the group is not paused
TOP SOURCES The three biggest causes of requests, with counts

IDENTITIES has one row for each host and identity. It has the same columns, without the hourly cap. It adds REMAINING/LIMIT, the quota the host last reported for that identity.

BUCKETS has one row for each bucket window.

Column Meaning
SCOPE and KEY group or identity, and which one
WINDOW minute or hour
TOKENS/BURST Tokens left and the most the bucket holds
LIMIT The refill limit for the window
SHED Requests dropped in the last hour, by class
WAITED Requests that waited in the last hour, by class

Hosts the server does not pace have no BUCKETS rows.

Change the pacing

Set these in the server environment. Restart the server to apply them. Self-hosted servers keep the defaults unless you change them. See Self-hosted deployment.

Setting Default What it does
CH_GITLAB_IDENTITY_PER_MIN 80 Most gitlab.com requests a minute for one identity
CH_GITLAB_GROUP_PER_MIN 350 Most gitlab.com requests a minute for one group
CH_GITLAB_GROUP_PER_HOUR 4000 Most gitlab.com requests an hour for one group
CH_GITHOST_PACE_BACKOFF_MIN 30s First pause after a 429 with no reset time
CH_GITHOST_PACE_BACKOFF_MAX 10m Longest pause after a 429 with no reset time
CH_GITLAB_MINTS_PER_MIN 12 Most sandbox token requests a minute for one GitLab connection. 0 turns the limit off
CH_GITHOST_BUDGET_LOG_INTERVAL 1h How often the server logs one line of request totals for each group

Do not set a limit above what GitLab allows for your plan. The server then hits 429 instead of pacing itself. A self-managed GitLab sets its own limits, and these buckets do not apply to it.

When work is paused

  1. Run ch instance githost. Look for a value in PAUSED UNTIL (UTC).
  2. Wait for that time. Parked tasks resume by themselves. You do not need to unblock them.
  3. If 429s keeps rising, look at TOP SOURCES. One source that dominates tells you which work uses the budget.
  4. If SHED or WAITED lists interactive or event in the last hour, the limits may be too low for your load. Raise them, within what GitLab allows.

Limits

  • The token buckets cover gitlab.com only.
  • Pauses and counts live in memory. A restart clears them. A new 429 sets a fresh pause.
  • With more than one API process, each keeps its own window and pauses.
  • The page and the command show counts. They never show a token or a repository path.

Last updated

Try it on one repository for two weeks.

File a task. An agent builds it. A person reads every request. You see the cost of each task.

Try "pricing", "connect a device", or "who reviews the code"

↑↓ move · ↵ open · esc close