What is an API Gateway?
There's a good chance you've interacted with an API Gateway today, even if you didn't realize it. They're a core component in modern architectures, especially with microservices.
Think of it as the front desk at a luxury hotel. Guests don't need to know where housekeeping or maintenance is — and clients shouldn't need to know your internal service topology. The gateway is a single entry point that routes requests and handles centralized middleware: authentication, routing, logging, and more.
API gateways rose alongside microservices. As monoliths split into specialized services, clients needed a centralized control point instead of talking to every service directly. Gateways are thin, purposeful components — this deep dive covers what interviews actually need without overcomplicating it.
Route requests to services.
Auth · rate limit · SSL.
Microservices architecture.
Simple monolith · one client.
Core responsibilities
The gateway's primary function is request routing — determining which backend service handles each incoming request. Funny enough, candidates often introduce a gateway and emphasize middleware but forget the core reason it exists: routing.
Gateways also handle cross-cutting concerns: authentication, rate limiting, caching, SSL termination, CORS, IP allow/deny lists, request size validation, API versioning, and service discovery integration. Mention middleware with purpose — don't spend half the interview here.
/users/* to user-service — auth and rate limits happen on the way.Tracing a request
Requests arrive via HTTP (or gRPC at the edge in some setups). The gateway validates, applies middleware, routes to a backend, transforms the response, and optionally caches. Six steps:
- Request validation
- Middleware (auth, rate limiting, etc.)
- Routing to the appropriate backend
- Backend processes and returns a response
- Response transformation for the client
- Optional caching for future requests
Validation and middleware
1) Request validation
Before anything else, check URL validity, required headers, and body format. Malformed JSON or a missing API key should fail fast at the gateway — not waste backend CPU on requests that were never going to succeed.
2) Middleware
Common gateway middleware includes JWT authentication, rate limiting, SSL termination, traffic logging, response compression, CORS, IP whitelisting/blacklisting, request size limits, API versioning, throttling, and service discovery hooks.
For interviews, the big three are authentication, rate limiting, and IP allow/deny. See Design a rate limiter for token-bucket enforcement at the edge.
Routing
The gateway maintains a routing table mapping requests to services — typically by URL path, HTTP method, query parameters, and headers.
routes:
- path: /users/*
service: user-service
port: 8080
- path: /orders/*
service: order-service
port: 8081
- path: /payments/*
service: payment-service
port: 8082
Backend communication and response transformation
Most backends speak HTTP; some use gRPC internally. The gateway can translate protocols so clients stay on HTTP/JSON while services use whatever's efficient — uncommon in practice but fair to mention.
The response transformation layer presents a clean external API while internal services use their preferred formats:
// Client: HTTP GET /users/123/profile
// Gateway → internal gRPC:
userService.getProfile({ userId: "123" })
// Gateway → client JSON:
{
"userId": "123",
"name": "John Doe",
"email": "john@example.com"
}
Caching
Before returning a response, the gateway can cache it — useful for frequently accessed, non-user-specific data where the same input yields the same output. Strategies include full response caching, partial caching of stable fields, and TTL or event-based invalidation. Store in memory or a distributed cache like Redis — see Redis when the design already has it.
Scaling an API Gateway
Horizontal scaling
Gateways are typically stateless — add instances behind a load balancer (AWS ELB, NGINX). The gateway may also load-balance across backend service instances. In interviews, one box labeled "API Gateway + Load Balancer" is usually enough; don't let entry-point details eat your clock.
Global distribution
For global user bases, deploy regional gateway instances with GeoDNS routing users to the nearest region. Keep routing rules and policies synchronized across regions — similar in spirit to CDN edge deployment.
Popular API Gateways
Managed
- AWS API Gateway — REST/WebSocket, throttling, Lambda integration, CloudWatch.
- Azure API Management — OAuth/OIDC, policy config, developer portal.
- Google Cloud Endpoints — GCP integration, gRPC, OpenAPI docs.
Easiest ops, highest cost at scale.
Open source
- Kong — NGINX-based, plugin ecosystem.
- Tyk — GraphQL, analytics, multi-DC.
- Express Gateway — Node.js, lightweight for JS microservices.
- nginx / Apache — how early Amazon did it; still valid.
When to propose an API Gateway
Use it when you have a microservices architecture. Without a gateway, clients must know every service endpoint — tighter coupling, complex client code, no clean separation between internal topology and external API.
Skip it for simple monoliths or single-client systems. A gateway adds complexity with no payoff when there's one backend and one app.
Cost and performance levers
Gateway responsibility map
Retry amplification
Gateway retries × service retries × dependency retries = thundering herd. Prefer bounded retries at one layer, idempotent APIs, and circuit breakers.
Failure modes to mention
Call out at least one dependency failure (DB down, cache stampede, queue lag, region outage) and your mitigation (timeouts, retries with jitter, degraded mode, circuit breaker).
Interview Q&A by level
Practice saying these out loud for API gateways. Interviewers grade clarity and judgment more than buzzwords.
Match depth to the bar: define → trade off → operate. Don't dump principal answers in an entry-level screen.
Wrapping up
An API Gateway is the front door for microservices: validate early, apply middleware, route by path, transform responses, cache when safe. Scale it statelessly; deploy regionally when users are global. Name AWS API Gateway or Kong if asked — but lead with why it's on the diagram.
For where the gateway sits among load balancers and the rest of the stack, see Key technologies. For running stateless services behind the gateway at scale, see Kubernetes. For API shape and auth choices clients see, see API design.