What a concurrency limit costs you
A concurrency limit caps how many jobs run at the same time. Anything past the cap waits in a queue. GitHub-hosted runners, for example, allow 60 concurrent jobs on the Team plan and 5 concurrent macOS jobs (GitHub’s Actions limits). The cap bites when you push for speed. Shard a test suite 32 ways to finish faster, and if teammates’ pushes leave your pull request three free slots, most shards wait in line for over an hour while the first checks pass in a minute or two.Queues compound
When pushes arrive faster than the cap can run them, the queue never clears. Each push waits behind everything queued before it, so the wait grows all afternoon. In this simulated workday, every push starts 48 five-minute jobs, so a 60-job limit gets through about 15 pushes an hour. From late morning the team pushes closer to 20 an hour. By 4:30pm a new push waits over an hour and a half to start, and a hotfix pushed then doesn’t start until after 6pm. On Blacksmith, every push starts within seconds.CI is bursty
CI traffic spikes on every push and goes quiet in between. One Blacksmith customer went from 7 running jobs to 1,435 in nine minutes. Another went from 16 to 1,467 in two minutes, then back under 200 a minute later. Our busiest customers peak above 3,000 concurrent jobs, or 25,000 vCPUs. Every customer runs on one shared fleet of bare-metal machines, and their spikes rarely line up. From How the economics of multitenancy work:From a distance, what once looked like sharp spikes from individual customers smooths into a predictable pattern. The more customers we serve, the less intense each individual spike appears.So even a spike of 1,400 jobs is a small slice of the fleet, and we have no reason to queue it. A cap sized to your average would block your peaks, and a cap sized to your peaks would sit idle most of the day.
Shard as wide as you want
Pick the shard count that makes CI fastest, not one that fits a concurrency budget. Every job gets its own runner in seconds: microVMs boot in under 3 seconds, every runner size is available on demand, and when the fleet has room, jobs get a free upgrade to the next size..github/workflows/test.yml