SD notes

Microservices and API Gateway

Intermediate · 7 min read · updated 29 October 2026

Quick answer

What are microservices and what does an API gateway do?

Microservices split an application into small services that each own one business capability and its data, and are deployed on their own. An API gateway is the single entry point in front of them: it routes each request to the right service and handles shared concerns such as authentication, rate limiting and TLS once, at the edge.

On this page

Monolith vs microservices
API gateway
Service discovery
Circuit breaker
Communication and data
Interview questions
Common questions

Microservices trade one big deployable for many small ones; the gateway, the registry and the circuit breaker are the parts that keep many small ones usable.

Monolith vs microservices

Monolith: one codebase, one deploy, in-process calls, one database.
Microservices: one service per business capability, each with its own code, data and deploy.
A change ships as its deploy unit — the whole app, or one service.

Monolith vs microservices: what one change redeploys

The same four features built two ways: one monolith, deployed as a single unit, and four services, each with its own code, database and deploy.

0:01 / 0:18Step 1 of 6

Transcript · 6 steps

	Monolith	Microservices
Deploy	Everything at once	One service at a time
Scaling	Whole app	Per service
Calls	Function calls	Network calls that can fail
Data	One database, joins and transactions	Database per service, events
Best for	Small teams, new products	Many teams, clear boundaries

Start with a well-structured monolith; split along boundaries that have proven stable.

API gateway

One entry point for every client (web, mobile, partners).
Routes by path or host: /orders/* → orders service.
Cross-cutting work once: authentication, rate limiting, TLS, logging, CORS.
Can aggregate several service calls into one response for a client.
Run it as a redundant fleet behind a load balancer — it sits on every request path.

API gateway: one entry point that authenticates, limits and routes

Clients send every request to one address, the gateway. It checks the token, counts each client's requests against a limit of 2, then routes by the path.

0:01 / 0:18Step 1 of 7

Transcript · 7 steps

Service discovery

Instances change address as they scale, deploy and crash.
Registry: instances register and heartbeat; entries expire after a TTL.
Client-side: the caller looks up the list and picks an instance.
Server-side: the caller hits a load balancer that does the lookup.

Service discovery: instances register, callers look them up

Instances come and go as they scale and crash, so the caller cannot hard-code addresses. It asks a registry, which knows who is alive.

0:01 / 0:25Step 1 of 10

Transcript · 10 steps

Circuit breaker

A slow dependency ties up its callers' threads and connections, and the failure cascades upward.
Closed: calls pass; failures are counted.
Open: calls fail fast for a cool-down.
Half-open: one probe call; success closes, failure opens again.
Pair it with timeouts, retries with backoff and a fallback (cached or default answer).

Circuit breaker: closed, open, half-open, run on a failing service

Each call to a downstream service goes through a breaker. Closed passes calls; 3 failures in a row open it, and an open breaker refuses calls at once.

0:01 / 0:36Step 1 of 13

Transcript · 13 steps

Communication and data

Synchronous: REST or gRPC — simple, but the caller waits and shares the callee's failures.
Asynchronous: events through a message queue — decoupled, eventually consistent.
Database per service; no reaching into another service's tables.
Cross-service workflows use sagas with compensating steps.

Interview questions

Would you start with microservices? Usually not; a modular monolith first, then split where team or scaling needs prove it.
What does an API gateway do that a load balancer does not? It understands requests: authentication, rate limits, routing by path, aggregation.
How do services find each other? A registry with heartbeats and TTLs, or the platform's DNS, as in Kubernetes.
How do you stop one slow service taking down the rest? Timeouts, a circuit breaker that fails fast, and a fallback.

Next: Design a URL Shortener · Before this: Rate Limiting

Common questions

What is the difference between a monolith and microservices?

A monolith is one codebase deployed as one unit, so any change redeploys everything but calls between modules are cheap function calls. Microservices are separate deployables talking over the network, so teams ship independently and scale parts separately, at the cost of network failures, distributed data and more operations work.

When should you not use microservices?

For a small team or a new product whose boundaries are still moving. The network calls, deployments, monitoring and data consistency work cost more than they save until the system and the organisation are large enough that one deployable slows everyone down.

Why use an API gateway?

Clients get one address instead of knowing every service, and cross-cutting work such as authentication, rate limiting, TLS termination, request logging and response aggregation is done once instead of in every service. Services can then be split or moved without changing clients.

What is service discovery?

A way for services to find each other's current addresses. Instances register with a registry such as Consul, etcd or Eureka and send heartbeats; callers look up a service by name. In Kubernetes, DNS names and Services do this for you.

What is a circuit breaker?

A wrapper around calls to a dependency that stops calling it after repeated failures. While open it fails fast instead of waiting on timeouts, then lets a probe through after a cool-down and closes again if the dependency has recovered. It stops one failing service from exhausting its callers.

How do microservices share data?

Each service owns its database and others reach that data only through its API or events. Work that spans services uses events and sagas, with compensating actions instead of one distributed transaction.

Microservices and API Gateway in System Design

Microservices vs monolith, the API gateway, service discovery and circuit breakers for system design interviews, each shown as a looping animation.

Subject: System Design
Level: Intermediate
Reading time: 7 min
Updated: 2026-10-29

What are microservices and what does an API gateway do?

Microservices trade one big deployable for many small ones; the gateway, the registry and the circuit breaker are the parts that keep many small ones usable.

Monolith vs microservices

Monolith: one codebase, one deploy, in-process calls, one database.
Microservices: one service per business capability, each with its own code, data and deploy.
A change ships as its deploy unit — the whole app, or one service.

Monolith vs microservices: what one change redeploys.

The same four features built two ways: one monolith, deployed as a single unit, and four services, each with its own code, database and deploy.
A developer changes the payments code. In both designs only the payments logic is touched.
Shipping it: the monolith is rebuilt, retested and redeployed whole as app v2 — all 4 modules restart. Only the payments service is redeployed, as payments v2.
A developer changes the search code. In both designs only the search logic is touched.
Shipping it: the monolith is rebuilt, retested and redeployed whole as app v3 — all 4 modules restart. Only the search service is redeployed, as search v2.
After 2 changes the monolith has restarted 8 module copies and the services 2. Each team ships alone, at the price of a network between the parts.

	Monolith	Microservices
Deploy	Everything at once	One service at a time
Scaling	Whole app	Per service
Calls	Function calls	Network calls that can fail
Data	One database, joins and transactions	Database per service, events
Best for	Small teams, new products	Many teams, clear boundaries

Start with a well-structured monolith; split along boundaries that have proven stable.

API gateway

One entry point for every client (web, mobile, partners).
Routes by path or host: /orders/* → orders service.
Cross-cutting work once: authentication, rate limiting, TLS, logging, CORS.
Can aggregate several service calls into one response for a client.
Run it as a redundant fleet behind a load balancer — it sits on every request path.

API gateway: one entry point that authenticates, limits and routes.

Clients send every request to one address, the gateway. It checks the token, counts each client's requests against a limit of 2, then routes by the path.
A1 reaches the gateway, which reads its token and path.
A1 (/users/7) is forwarded to the users service; A2 reaches the gateway.
A2 (/orders/12) is forwarded to the orders service; B3 reaches the gateway.
B3 carries no token, so the gateway answers 401 and no service sees it; B4 reaches the gateway.
B4 (/orders/15) is forwarded to the orders service; A5 reaches the gateway.
A5 is client A's request number 3, over the limit: 429 at the gateway. Auth and rate limiting ran once, at the edge, so the services hold only their own logic.

Service discovery

Instances change address as they scale, deploy and crash.
Registry: instances register and heartbeat; entries expire after a TTL.
Client-side: the caller looks up the list and picks an instance.
Server-side: the caller hits a load balancer that does the lookup.

Service discovery: instances register, callers look them up.

Instances come and go as they scale and crash, so the caller cannot hard-code addresses. It asks a registry, which knows who is alive.
orders-1 starts and registers its address, 10.0.1.5, with the registry.
orders-2 starts and registers its address, 10.0.2.8, with the registry.
checkout asks the registry for "orders" and gets 2 addresses: 10.0.1.5 and 10.0.2.8.
checkout picks orders-1 from the list and calls it directly; the registry is not in the request path.
orders-2 sends its heartbeat. orders-1 crashed at 15s and has gone quiet for 10s, short of the 30s TTL, so it is still listed.
orders-2 sends its heartbeat. orders-1 crashed at 15s and has gone quiet for 20s, short of the 30s TTL, so it is still listed.
orders-1 has been silent for 30 seconds, so its entry expires and is removed; orders-2 still heartbeats.
checkout asks the registry for "orders" and gets 1 address: 10.0.2.8.
The new list has only orders-2, so the call reaches a live instance — no one edited a config file.

Circuit breaker

A slow dependency ties up its callers' threads and connections, and the failure cascades upward.
Closed: calls pass; failures are counted.
Open: calls fail fast for a cool-down.
Half-open: one probe call; success closes, failure opens again.
Pair it with timeouts, retries with backoff and a fallback (cached or default answer).

Circuit breaker: closed, open, half-open, run on a failing service.

Each call to a downstream service goes through a breaker. Closed passes calls; 3 failures in a row open it, and an open breaker refuses calls at once.
Tick 1: the call succeeds; the breaker stays closed.
Tick 2: the downstream call fails (failure 1 in a row). Still under 3, so the breaker stays closed.
Tick 3: the downstream call fails (failure 2 in a row). Still under 3, so the breaker stays closed.
Tick 4: failure 3 in a row reaches the threshold. The breaker opens, stopping the failure from spreading to every caller.
Tick 5: the breaker is open, so the call fails fast without touching the struggling service — no threads wait on timeouts.
Tick 6: the breaker is open, so the call fails fast without touching the struggling service — no threads wait on timeouts.
Tick 7: 3 ticks have passed, so one probe call is let through. It fails, and the breaker opens again for another 3 ticks.
Tick 8: the breaker is open, so the call fails fast without touching the struggling service — no threads wait on timeouts.
Tick 9: the breaker is open, so the call fails fast without touching the struggling service — no threads wait on timeouts.
Tick 10: another probe, and the service answers. The breaker closes and normal traffic resumes.
Tick 11: calls pass normally again, and the failure count stays at 0.
Tick 12: closed and healthy. Over 12 calls the breaker spared the failing service 4 calls it could not have answered.

Communication and data

Synchronous: REST or gRPC — simple, but the caller waits and shares the callee's failures.
Asynchronous: events through a message queue — decoupled, eventually consistent.
Database per service; no reaching into another service's tables.
Cross-service workflows use sagas with compensating steps.

Interview questions

Would you start with microservices? Usually not; a modular monolith first, then split where team or scaling needs prove it.
What does an API gateway do that a load balancer does not? It understands requests: authentication, rate limits, routing by path, aggregation.
How do services find each other? A registry with heartbeats and TTLs, or the platform's DNS, as in Kubernetes.
How do you stop one slow service taking down the rest? Timeouts, a circuit breaker that fails fast, and a fallback.