Building a custom API gateway for microservices

A custom API gateway gives clients a stable entry point while internal services evolve behind it. Instead of exposing every microservice directly, the gateway can route requests, authenticate users, apply traffic policies, combine responses, and translate between public and private API contracts. This makes it especially useful when a platform serves web applications, mobile users, business partners, and internal tools with different integration needs.

The strongest gateway designs begin with business boundaries rather than technology choices. A team should identify which services need protection, which operations require low latency, how failures should appear to customers, and where sensitive data travels. For an Australian organisation, those decisions also need to account for the Privacy Act 1988, the Australian Privacy Principles, local payment expectations, and traffic patterns across cities such as Sydney, Melbourne, Brisbane, and Perth.

Start with a clear gateway boundary

An API gateway should own concerns that are common across many services. Authentication, request validation, rate limiting, routing, protocol translation, correlation identifiers, and consistent error responses are good candidates. Business rules such as calculating an insurance premium or approving a refund should remain inside the relevant domain service, where they can be tested and changed without turning the gateway into a central monolith.

Begin by documenting each client type and its access pattern. A mobile application may need a compact response assembled from several services, while a partner may require a carefully versioned REST endpoint. An internal operations portal might need richer filtering and longer request timeouts. Mapping these use cases helps determine whether the gateway should perform response aggregation, expose separate client-facing APIs, or simply forward requests.

The gateway contract should be explicit about methods, status codes, pagination, idempotency, and error formats. OpenAPI is a practical way to publish that contract and generate validation or client libraries. Versioning should be deliberate: a path such as /v1/orders is easy to understand, while header-based versioning can be cleaner when many resources share a release cycle. Either approach works when deprecation dates and migration rules are visible to consumers.

A gateway should also have a defined failure position. If a catalogue service is unavailable, the gateway might return a partial response or a clear dependency error. If a payment authorisation is uncertain, it should not automatically retry an operation that could create a duplicate charge. These decisions belong in the API design before implementation begins.

Choose an architecture that fits the traffic

A common design places a public gateway at the edge, with internal routing to domain services through a private network. A web application firewall, load balancer, or cloud edge service can absorb basic internet traffic before requests reach the gateway. The gateway then handles application-level policies, while service-to-service communication remains protected by network controls and service identity.

A modular application is often a sensible starting point. Authentication, routing, throttling, transformation, and observability can be separate modules deployed as one gateway process. This keeps operational complexity manageable while allowing later extraction of components that need independent scaling. Splitting every plugin into a separate service from the beginning may create additional network calls, deployments, and failure modes without clear value.

Use asynchronous messaging for work that does not need an immediate response. Creating an order may require a quick acknowledgement, while invoicing, notifications, and warehouse updates can be handled through a queue or event stream. The gateway should return a traceable operation identifier so clients can check progress. For workflows involving account balances or payouts, careful status modelling matters; a cash-out timing guide illustrates why users need clear expectations around pending, approved, and completed states.

Response aggregation can reduce mobile round trips, which is useful when customers connect through congested networks or switch between Wi-Fi and mobile data. It also introduces coupling: the gateway must understand several downstream responses and decide what to do when one is slow. Apply strict per-service timeouts, limit the amount of data collected, and avoid making aggregation a substitute for a properly designed client-facing resource.

Choose the implementation language based on team capability, runtime performance, available libraries, and operational support. Go, Java, .NET, Node.js, and other established platforms can all support a reliable gateway. The important qualities are predictable concurrency, strong HTTP support, safe dependency handling, and straightforward instrumentation rather than a fashionable framework.

Build security into every request path

Authentication establishes who is calling, while authorisation determines what that caller may do. OAuth 2.0 and OpenID Connect are common choices for user-facing applications. Short-lived access tokens reduce the impact of theft, and scopes should express meaningful permissions rather than broad access to an entire system. For machine-to-machine integrations, signed credentials, client certificates, or workload identity can be more appropriate than shared secrets.

The gateway should validate tokens locally when practical, but it must still handle key rotation and issuer failures safely. Claims such as tenant, role, audience, and token expiry require strict validation. Never trust an identifier from a URL simply because the request is authenticated; the downstream service must still check whether that user or organisation may access the specific resource.

Transport encryption should cover every hop, including connections from the gateway to internal services. Mutual TLS or signed service requests can provide stronger workload authentication in a zero-trust design. Secrets belong in a managed secret store, not in source code, container images, or ordinary configuration files. Administrative endpoints should be isolated from public traffic and protected with stronger controls.

Australian organisations handling personal information should map data flows against the Australian Privacy Principles and the requirements of the Privacy Act 1988. The Notifiable Data Breaches scheme also makes incident preparation important. A gateway can help by minimising logged personal data, masking tokens and payment details, enforcing retention limits, and attaching audit records to privileged actions. Data residency should be assessed against contractual and sector-specific obligations rather than assumed from the location of a cloud region.

Payment-related endpoints deserve additional controls. Use idempotency keys for operations that create charges, refunds, or transfers. Apply stricter rate limits to credential and payment routes, and align card-handling processes with PCI DSS obligations where card data is involved. A secure gateway reduces exposure, but it does not remove the responsibility of the underlying services or the organisation operating the platform.

Design for resilience and observable performance

A gateway is a critical dependency, so it should be deployed across multiple availability zones or failure domains. Stateless instances make horizontal scaling easier, with shared state kept in suitable stores rather than local memory. Health checks should distinguish between a process that is alive and one that can safely accept traffic. A shallow health endpoint may report success while every important downstream dependency is failing.

Timeouts, retries, circuit breakers, and bulkheads need to work together. A short timeout protects a caller from waiting indefinitely, while a circuit breaker prevents repeated calls to an unhealthy service. Retries should be limited, use backoff with jitter, and apply only to operations known to be safe. Retrying a non-idempotent request at several layers can multiply load and produce duplicate business actions.

Set service-level objectives for availability, latency, and error rates. Track gateway latency separately from downstream latency so the team can identify whether a problem lies in routing, authentication, network transport, or a domain service. Useful metrics include request volume by route, status-code distribution, timeout counts, rejected requests, token validation failures, and the number of open circuit breakers.

Distributed tracing is essential when a single customer request crosses several services. Propagate a trace ID and request ID, but avoid placing sensitive values in either identifier. Structured logs should record route, method, outcome, duration, dependency timings, and a carefully selected tenant or account reference. Logs must be useful for incident response without becoming an ungoverned copy of customer data.

Australian traffic has its own practical patterns. A retailer may see sharp demand during local evening shopping hours, end-of-financial-year promotions, or major sporting events. A service used across Sydney, Melbourne, and Perth needs to account for time-zone differences when setting maintenance windows and interpreting daily reports. Capacity tests should use realistic mobile behaviour, including repeated retries caused by a dropped connection rather than assuming every client has a stable fixed-line connection.

Deliver safely and evolve the platform

Treat gateway configuration as code. Routes, upstream targets, authentication policies, rate limits, and transformation rules should pass through pull requests, automated tests, and an auditable deployment process. A staging environment should reproduce production routing and security behaviour closely enough to expose contract errors before release. Secrets and production credentials must remain outside ordinary test fixtures.

Contract testing helps prevent a gateway release from breaking a downstream service or external partner. Consumer-driven tests can verify required fields, status codes, and error behaviour. Integration tests should cover expired tokens, malformed payloads, unavailable dependencies, duplicate idempotency keys, slow responses, and oversized requests. Load tests need to examine both normal traffic and bursts, since a gateway can fail through connection exhaustion before CPU usage appears high.

Use progressive delivery for changes with meaningful risk. A canary release can send a small percentage of traffic to a new gateway version, while feature flags allow a route or transformation to be enabled gradually. Keep rollback procedures simple, documented, and tested. Database migrations and API changes should be compatible with the previous gateway version long enough to support a safe deployment sequence.

Partner integrations require special care because the organisation cannot control every client release. Publish clear documentation, sample requests, authentication instructions, quotas, and deprecation timelines. Give partners a sandbox with representative responses and test credentials. Webhooks should be signed, retried with backoff, and accompanied by an event identifier so receivers can process them idempotently.

Operations teams should review gateway policies as the service catalogue changes. Remove abandoned routes, rotate credentials, examine unused scopes, and check whether rate limits still reflect real usage. A route that began as a simple read operation may later expose personal or financial information and require stronger controls. Ownership should be recorded for each route, dependency, alert, and contract.

A well-built gateway remains deliberately boring to its users: requests are authenticated consistently, failures are understandable, latency is predictable, and internal service changes rarely force client rewrites. Its value comes from disciplined boundaries, secure defaults, measurable reliability, and an operating model that suits the organisation’s customers, partners, and regulatory environment.