Scaling a Shopify App: Webhooks, Queues and the Five-Second Rule

A Shopify app that works beautifully for fifty stores can fall over at five hundred without a single line of code changing. Nothing was wrong with the code. What changed is that the assumptions it was built on — that API calls always succeed, that webhooks arrive at a manageable rate, that a sync can happen inline — stop being true somewhere between those two numbers.

The failure is rarely dramatic. It looks like a support queue slowly filling with merchants reporting that their inventory is a few hours stale, and an engineering team that cannot reproduce it because on their test store everything is instant.

Shopify's Admin GraphQL API uses a leaky-bucket cost model: every query has a calculated cost, you hold a bucket of points, and the bucket refills at a fixed rate. The REST API works on a similar principle with a request bucket. Either way, the limit is per shop, not per app — which is the detail that changes how you should think about it.

Because it is per shop, adding merchants does not shrink each merchant's budget. What it does is multiply the number of independent budgets your infrastructure has to track. A single global rate limiter across your whole app is the wrong shape: it throttles shop A because shop B is busy, which is a self-inflicted outage.

Read the cost extensions Shopify returns on every GraphQL response. They tell you your current bucket state exactly. An app that reads them and paces itself will outperform one that fires requests until it gets a 429 and then backs off, because by the time you see the 429 you have already lost the round trip.

Shopify gives a webhook handler five seconds to respond. Not five seconds to finish the job — five seconds to acknowledge receipt. Exceed it enough times and Shopify stops delivering to that endpoint entirely, which is a silent failure mode: your app does not error, it simply stops hearing about orders.

The correct handler is almost boring. Verify the HMAC, write the payload to a queue, return 200. Everything else happens somewhere else, where it can take as long as it needs, retry on failure, and be observed.

This also solves a problem teams often try to solve separately. During a flash sale, a single store can emit orders/create faster than any synchronous handler could process them. With a queue in front, that burst becomes a depth measurement on a graph rather than an incident.

Once work is queued, the questions that matter are ordering, isolation and poison handling. Ordering, because processing an inventory update out of sequence produces a number that is wrong in a way that self-heals only if another update happens to arrive. Isolation, because one merchant's backlog must not starve everyone else's. Poison handling, because one malformed payload should not stall a partition forever.

Partitioning by shop domain handles the first two at once: ordering is preserved within a shop, which is the only place it matters, and a slow shop occupies its own lane. For the third, a dead-letter queue with an alert is the whole answer, and the alert is the important half — a dead-letter queue nobody looks at is a data-loss mechanism with extra steps.

Retries need jitter. Synchronised retries after a transient Shopify outage produce a thundering herd precisely when the platform is least able to absorb it.

Paginating a hundred thousand products through the standard API is the wrong tool, and it is the most common cause of an app that takes six hours to onboard a large merchant. Shopify's bulk operations API exists for exactly this: you submit a query, Shopify runs it asynchronously, and you download a JSONL file when it is ready.

The trade-off is that bulk operations are asynchronous and there is a limit on concurrent operations per shop, so your onboarding flow has to be built around polling and resumability rather than a progress bar that assumes linear progress. That is more work up front and dramatically less work later.

The same applies in reverse for writes. bulkOperationRunMutation exists, and hand-rolling ten thousand sequential mutations instead is how apps end up permanently in rate-limit backoff.

None of this is exotic infrastructure. It is a queue, a per-tenant rate limiter, a dead-letter path and five dashboards. What makes it hard is that every one of them is unnecessary at ten stores, and retrofitting them at a thousand means doing it while the support queue is on fire.

The honest middle ground is to build the seams early and the sophistication late. Put the queue in from day one even if it has one consumer and no partitioning, because the expensive part is not the queue — it is untangling a codebase that assumed synchronous execution everywhere.