Change Log

changelog:
  - version: 0.1
    description: 
      - Added foundational RFCs for HTTP and API Design.
      - Included RFC 7230 covering HTTP/1.1, Message Syntax and Routing.
      - Added RFC 7231 on HTTP/1.1 Semantics and Content.
      - Included RFC 7232 for HTTP/1.1 Conditional Requests.
      - Added RFC 7233 focusing on HTTP/1.1 Range Requests.
      - Included RFC 7234 for HTTP/1.1 Caching.
      - Added RFC 7235 covering HTTP/1.1 Authentication.
      - Included RFC 3986 for URI Syntax.
      - Added RFC 8259 for JSON (JavaScript Object Notation) Data Format.
      - Included RFC 6749 for OAuth 2.0 Authorization Framework.
      - Added RFC 7540 covering HTTP/2.
  - version: 1.0
    description: 
      - Added OpenAPI Guidelines.
      - Included OpenAPI Specification (OAS) 3.1.
      - Added JSON Schema (IETF Draft) for API schema validation.
  - version: 1.3
    description:
      - Finalised guidance and documentation.
      - Reviewed all included RFCs for accuracy and relevance.
      - Enhanced details around OpenAPI Specification and JSON Schema usage.
      - Clarified best practices for implementing RFC guidelines in modern API design.

API Design Guidelines

1. Design APIs with a Consumer-Centric Approach

APIs should be built with the consumers’ needs as a priority. Start by understanding the different personas that will interact with your API—whether it’s frontend developers, backend services, or third-party integrators. Recognise their pain points and design APIs that address these concerns.

Design endpoints to be intuitive and predictable. Resources should be clearly named, and their relationships should be easy to infer. APIs that are simple to use and understand improve adoption and reduce errors in integration. Ensure that your API documentation is concise, with clear examples that help developers quickly grasp usage patterns. Avoid over-complicating the API by adding excessive logic into the endpoints.

Be aware of the scalability of your API from the perspective of the consumer. As the user base grows, the API must be capable of handling different workloads while maintaining a consistent experience for all consumers.

TIP: Use tools like Postman or Swagger UI to provide interactive documentation, allowing consumers to test API endpoints directly from the documentation. This makes onboarding smoother and helps identify any potential issues early in the integration phase.

2. Adopt Standard HTTP Methods for Resource Actions

HTTP methods serve as the foundation of any RESTful API. To ensure clarity, adopt the following methods:

• GET for retrieving resources. A GET request should never modify the state of a resource. Ensure responses are cacheable where appropriate to reduce unnecessary load.
• POST for creating new resources. Responses to POST requests should return the appropriate status code (201 Created) along with a link to the newly created resource in the Location header.
• PUT for replacing or updating an existing resource. PUT operations should be idempotent, meaning multiple identical requests should result in the same resource state.
• PATCH for partial updates. Use PATCH when you only need to update a subset of the resource’s properties.
• DELETE for removing a resource. A DELETE request should also be idempotent, meaning multiple delete requests to the same resource should yield the same result.

Adhering to these standard methods maintains consistency across APIs, making them easier for consumers to understand and use.

TIP: Be explicit in the usage of PATCH versus PUT. PATCH is useful for partial updates, while PUT should only be used when fully replacing the resource.

3. Utilise Proper HTTP Status Codes

HTTP status codes provide essential feedback to API consumers. For every request, return the appropriate status code based on the operation’s success or failure. Here are the key codes to include:

• 2xx for success: Use 200 (OK) for successful GET requests, 201 (Created) for new resources, and 204 (No Content) for successful DELETE requests.
• 4xx for client errors: Use 400 (Bad Request) for malformed requests, 401 (Unauthorized) when authentication is required, and 404 (Not Found) when a requested resource doesn’t exist.
• 5xx for server errors: Use 500 (Internal Server Error) when the server encounters an issue processing the request, and 503 (Service Unavailable) if the API is temporarily offline for maintenance or overloaded.

Correct use of status codes allows consumers to handle responses programmatically and ensures smooth operation between services.

TIP: Include descriptive error messages in the body of 4xx and 5xx responses. The message should provide enough information for consumers to resolve issues without relying on your support team.

4. Ensure Statelessness in API Calls

A RESTful API must be stateless, meaning each request from the client should contain all the necessary information for the server to fulfil it. The server should not retain any session state between requests. Statelessness ensures that scaling is easier since any server can process any request without needing to refer to previous interactions.

To achieve this, avoid using server-side sessions for authentication. Instead, employ stateless authentication tokens such as JSON Web Tokens (JWT). Each request should include a token in the header (typically via Authorization: Bearer) so that the server can validate the client’s identity with every request. This approach decouples authentication from server state.

By adhering to stateless design, the API becomes more scalable, resilient, and easier to maintain.

TIP: When using JWTs, ensure that token expiration and rotation are well-defined. Implement refresh tokens to allow users to obtain a new token without re-authenticating, while avoiding long-lived tokens for security purposes.

5. Use Proper Nouns for Resource Naming Conventions

In RESTful API design, the naming of resources is important for clarity and usability. Use nouns, not verbs, to name resources, as the HTTP method itself represents the action. For instance, use /users for accessing a collection of user data, and /users/{id} to access a specific user by their unique identifier.

Always use plural nouns for resource collections. Even if a single entity is returned, the collection is still plural because the endpoint represents a list or group of items. Keep names simple, lowercase, and hyphenated if necessary (e.g., /order-items instead of /orderItems or /OrderItems).

Be mindful of relationships between resources. For example, a list of orders for a specific user might be represented as /users/{id}/orders. However, avoid deep nesting beyond two or three levels, as it complicates routing and reduces readability.

TIP: Ensure that resource names are consistent across different APIs in your ecosystem. This makes the APIs more intuitive for consumers, reducing the learning curve for developers working across services.

6. Maintain Consistent Resource Naming Across Endpoints

Consistency is crucial in REST API design, especially for resource naming. All endpoints should follow a unified naming convention to ensure they are predictable and easy to use. For example, if /users is used for retrieving a list of users, maintain that naming standard across related resources like /users/{id}/orders for retrieving user orders. Consistent naming aids in discoverability and usability, making it easier for developers to understand the relationships between different resources.

A common mistake is mixing conventions, such as using camelCase in one resource (/userDetails) and snake_case in another (/user_orders). This inconsistency can lead to confusion, especially when APIs are used in multiple contexts or by different teams. Establishing naming guidelines early in the design process will reduce errors and misinterpretations.

Path parameters should also follow clear conventions. When referring to resources by ID, use simple, readable names, such as {user_id} or {order_id}, ensuring clarity on what the ID represents in the resource hierarchy.

7. Limit Resource Nesting for Improved Readability

While resource nesting can represent relationships between objects, excessive nesting can lead to unreadable and complex API structures. For instance, deeply nested URLs like /users/{id}/orders/{order_id}/items/{item_id}/reviews/{review_id} may represent accurate relationships but are unnecessarily complex and difficult to manage.

Limit nesting to two or three levels. A better approach would be to have a flatter structure where separate endpoints manage different resources, such as /orders/{order_id}/items and /reviews/{review_id}. Flatter structures make the API more intuitive and easier to maintain in the long run.

If your API deals with related data, consider using query parameters for filtering or expanding related data instead of relying on deeply nested paths. For example, /orders?include=items could return orders with item details, reducing the need for complicated URL paths.

8. Design API Responses to be Minimal and Efficient

An API response should be concise and contain only the necessary information. Overly verbose responses not only increase bandwidth usage but can also slow down the performance of client applications. For instance, if an API consumer requests user details, the response should only include relevant fields such as name, email, and ID, rather than sending the entire user object with unnecessary data.

Additionally, consider offering partial responses where clients can request only specific fields. This can be achieved using query parameters, such as /users/{id}?fields=name,email. This technique allows consumers to tailor the response based on their needs, improving both performance and efficiency.

Responses should also adhere to a consistent structure, making them predictable for API consumers. Each response should include metadata (e.g., status, pagination information) where necessary, ensuring clarity and usability.

9. Implement Pagination for Large Data Sets

When dealing with large collections, always implement pagination to improve API performance and usability. Returning massive data sets in a single response can lead to slow client-side processing, unnecessary server load, and increased bandwidth usage. Use pagination mechanisms to limit the number of items returned at once, allowing clients to fetch additional data in smaller, more manageable chunks.

The two most common pagination techniques are:

• Offset-based pagination: Use parameters like ?page=2&limit=20 to specify the page and number of items per page.
• Cursor-based pagination: This method is more efficient for large datasets, using a cursor (usually a unique identifier or timestamp) to fetch the next set of results. For example, ?cursor=abc123&limit=20.

Always return pagination metadata in your API responses, including total items, total pages, and a link to the next and previous pages.

10. Ensure Proper Use of Query Parameters for Filtering

Filtering allows clients to retrieve a specific subset of data by applying conditions to the request. Use query parameters for filtering rather than adding additional endpoints or deeply nested paths. For example, instead of creating multiple endpoints like /users/active and /users/inactive, use query parameters such as /users?status=active or /users?status=inactive.

Be mindful of how multiple filters are handled. The API should support combining filters with logical operators (e.g., AND, OR). For instance, /orders?status=pending&min_price=100 could retrieve all pending orders with a minimum price of £100.

Ensure that query parameters are well-documented so that consumers know exactly which filters are available and how to use them correctly.

11. Support Partial Responses through Query Parameters

Supporting partial responses allows consumers to request only the data they need, improving efficiency. This can be achieved by introducing a fields query parameter. For example, /users/{id}?fields=name,email would return only the name and email properties of the user, excluding other data that might be irrelevant for the specific request.

Partial responses reduce payload size, improve performance, and help conserve bandwidth, especially in environments with limited network resources such as mobile apps. However, ensure that the API handles this feature efficiently and doesn’t create unnecessary processing overhead on the server.

TIP: When using partial responses, ensure that sensitive fields are protected, even if requested by the client. Introduce role-based access control (RBAC) to control which fields can be returned based on the user’s permission level.

12. Implement Caching for Optimised Performance

Caching is essential for improving the performance and scalability of APIs. Implementing proper caching strategies reduces the load on your servers by storing frequently requested data in a cache for a short period. Common caching mechanisms involve using HTTP headers such as Cache-Control and ETag. Cache-Control can be used to define how long a resource should be cached, while ETag helps in conditional GET requests by ensuring only changed data is sent back to the client.

For example, if a client requests a resource with an unchanged ETag, the server can return a 304 (Not Modified) status, saving bandwidth. Ensure to invalidate or update caches appropriately to avoid serving stale data to clients.

TIP: Implement caching carefully with endpoints that return frequently changing data. Misuse of caching on dynamic data can lead to clients receiving outdated information, causing inconsistencies in the user experience.

13. Use JSON as the Default Data Format

While RESTful APIs can theoretically support multiple formats (XML, YAML, etc.), JSON has become the de facto standard for modern APIs due to its lightweight structure and wide compatibility. JSON is easy to parse in virtually every programming language, making it the most accessible format for developers working with your API.

Be sure to include Content-Type: application/json in responses and allow clients to explicitly request JSON by using the Accept: application/json header in requests. JSON’s structure should also be consistent and well-formed, with no unnecessary fields or redundant nesting.

Avoid returning deeply nested objects as they can complicate the client-side parsing and increase processing time.

Lessons Learnt: We once left out content negotiation, allowing clients to request multiple formats, and encountered issues with malformed responses. This opened up vulnerabilities that allowed actors to exploit endpoints for unintended data formats.

Understood. I will proceed without adding any borders or horizontal rules between the sections.

14. Define Clear and Concise Error Messages

Error handling is vital for any API, and providing clear, concise, and actionable error messages helps clients resolve issues quickly. Use a structured approach to error messages, including an error code, a brief message, and an explanation.

For example, a 400 Bad Request might return the following response:

{
  "error_code": "400",
  "message": "Invalid input format",
  "details": "The 'email' field is missing or incorrectly formatted."
}

This structure ensures that both developers and automated systems can interpret and handle errors effectively. Standardise error responses across your entire API to avoid confusion.

TIP: Avoid exposing sensitive information in error messages, such as internal system logs or stack traces, which could be exploited by malicious users.

15. Employ Versioning to Manage API Changes

Versioning your API is essential for maintaining backward compatibility while introducing new features or changes. Use a clear versioning strategy, typically included in the URL path, such as /v1/ or /v2/, to indicate the version of the API being used. This allows consumers to continue using older versions without being forced to immediately upgrade when new versions are released.

It’s important to communicate version deprecation policies clearly to API consumers, providing ample time for migrations. Avoid making breaking changes within the same version.

Lessons Learnt: We once made breaking changes to an API without versioning it, which caused disruptions in client applications that were dependent on the older implementation. This led to complaints from customers and delayed releases.

16. Use Hypermedia as the Engine of Application State (HATEOAS)

HATEOAS (Hypermedia as the Engine of Application State) is a REST constraint that enables API responses to contain links that guide the client on how to interact with the API further. Instead of hardcoding endpoint paths, a hypermedia-driven API provides dynamic links based on the current state of the resource.

For example, when retrieving a list of orders, the response could include links to update or cancel an order:

{
  "order_id": 1234,
  "status": "pending",
  "links": {
    "cancel": "/orders/1234/cancel",
    "update": "/orders/1234"
  }
}

HATEOAS promotes a more flexible and dynamic interaction with the API, where clients can discover actions available on a resource without needing pre-knowledge of endpoint paths. It also enables the API to evolve without breaking existing client integrations.

17. Enable Secure Authentication Using OAuth 2.0

OAuth 2.0 is the industry standard for secure API authentication, providing token-based access and ensuring users can authorise applications without exposing credentials. Implement OAuth 2.0 to allow users to delegate access to their data through token-based authorisation.

The flow should involve the client obtaining an access token after authenticating with an authorisation server. The token is then used in API requests by including it in the Authorization: Bearer header. This ensures that the server can verify the user’s identity for each request without storing session information.

Make sure to include refresh tokens for longer-lived access and limit the token’s scope to minimise the risk of over-authorisation.

Lessons Learnt: We once deployed an API without properly limiting the OAuth 2.0 token’s scope, which led to clients gaining unintended access to sensitive resources. This was flagged as a security vulnerability that required immediate remediation.

18. Ensure End-to-End Encryption (HTTPS)

To ensure data security, always use HTTPS for all API communications. HTTPS encrypts the data exchanged between the client and server, protecting it from being intercepted by malicious actors. Never allow communication over unsecured HTTP.

API servers should redirect all HTTP traffic to HTTPS and require SSL/TLS certificates for secure communication. This is especially important when dealing with sensitive data such as user credentials, financial information, or personally identifiable information (PII). Implementing HTTPS ensures data integrity and confidentiality during transmission.

TIP: Set up HSTS (HTTP Strict Transport Security) headers to enforce HTTPS in all client interactions with the API, further securing communications against protocol downgrades.

19. Avoid Long-Lived Connections for Statelessness

To preserve statelessness in your API, avoid relying on long-lived connections such as WebSockets unless absolutely necessary. While WebSockets are suitable for real-time communication, they are not inherently stateless and can complicate horizontal scaling efforts.

APIs should focus on using stateless, short-lived HTTP requests that include all necessary information within each request, avoiding the need for session tracking or reliance on previous interactions. This approach enables better scalability, load balancing, and easier management of API instances.

If real-time updates are required, consider implementing Server-Sent Events (SSE) or offering webhook subscriptions for clients to receive asynchronous updates.

Lessons Learnt: We once relied on long-lived connections for an API handling real-time notifications, which caused scaling issues during traffic spikes. This led to system instability and resource exhaustion across the infrastructure.

20. Include Request and Response Metadata for Clarity

API responses should provide clear metadata, especially when dealing with collections or paginated data. Metadata can include information such as the total number of items, current page, page size, and links to the next and previous pages in paginated responses.

For instance, an API response for a paginated list of users might look like this:

{
  "data": [
    { "id": 1, "name": "John Doe" },
    { "id": 2, "name": "Jane Smith" }
  ],
  "metadata": {
    "total_items": 100,
    "page": 1,
    "page_size": 20,
    "next_page": "/users?page=2"
  }
}

Including metadata enhances the usability of your API by providing clients with important contextual information about the response and how they can request additional data if needed.

21. Design APIs for Scalability

Your API should be designed to scale as usage grows, ensuring high availability and performance even during traffic spikes. One approach is to use statelessness, allowing for easy load balancing across multiple servers. Another strategy is to design APIs that are horizontally scalable, meaning that additional servers can be added as traffic increases without changing the architecture.

Implement rate limiting to manage excessive requests and prevent abuse. Design your API to degrade gracefully, ensuring that even under heavy load, essential services remain operational. Use caching and CDNs to reduce the load on your primary servers.

TIP: Build your API with cloud-native architectures in mind, leveraging auto-scaling features in cloud platforms to handle demand surges efficiently.

22. Use Asynchronous Requests for Long-Running Tasks

Long-running tasks should not block the client’s request or degrade performance. Instead, use asynchronous processing by offloading such tasks to background jobs. The API should return an immediate response indicating that the task has been initiated, along with a link or job ID that the client can poll to check the status of the task.

For example, after uploading a large file or initiating a batch process, return a 202 (Accepted) status with a job resource that provides the task’s current progress:

{
  "status": "processing",
  "job_id": "12345",
  "status_url": "/jobs/12345/status"
}

This pattern keeps your API responsive and ensures better resource management on the server.

Example: Async API (Async HTTP)

23. Support Rate Limiting to Prevent Abuse

Implement rate limiting to control the number of requests clients can make to your API within a defined time frame. Rate limiting helps prevent abuse or accidental overuse, particularly in public APIs or those with high traffic volumes. Use HTTP headers such as X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset to inform clients of their current usage and when they can expect limits to reset.

For example:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 50
X-RateLimit-Reset: 1372700873

Return a 429 (Too Many Requests) status when the rate limit is exceeded, along with information on when the limit will reset.

Lessons Learnt: We once deployed an API without proper rate limiting, which led to a distributed denial-of-service (DDoS) attack that overwhelmed our infrastructure. Rate limiting would have mitigated the damage. Why didn’t we rate limit at the start? Well the API was a Backoffice function and what we didn’t realise at the time was that Users were familiar with Python, and they would use this to speed up their daily tasks which led to a CSV file being processed, which in turn called the API endpoint.

24. Optimise API for Mobile Networks

Mobile networks can be unreliable and have limited bandwidth. To optimise APIs for mobile clients, reduce payload sizes, minimise round trips, and avoid overfetching. Support partial responses with the fields query parameter to allow clients to request only the data they need.

Additionally, consider using HTTP/2 for better efficiency, as it allows multiple requests and responses to be multiplexed over a single connection. This reduces latency and improves performance on mobile networks.

25. Provide API Documentation Using OpenAPI Specification

Comprehensive documentation is key to API adoption. Use the OpenAPI Specification (formerly known as Swagger) to create machine-readable API documentation. The OpenAPI Specification describes the API’s endpoints, request parameters, response formats, error codes, and authentication methods.

Make sure your documentation is always up-to-date with the API itself and includes code examples for common use cases. Tools like Swagger UI or Redoc can render interactive documentation that allows users to try out the API directly.

TIP: Provide examples for different scenarios such as successful responses, client errors, and edge cases, ensuring that developers have a clear understanding of how to use your API.

26. Use Semantic Versioning for Backward Compatibility

Use semantic versioning to clearly communicate the impact of API changes. Follow the MAJOR.MINOR.PATCH format:

• MAJOR version increments indicate breaking changes.
• MINOR version increments indicate backward-compatible new features.
• PATCH version increments indicate backward-compatible bug fixes.

For example, if your API version changes from 1.2.3 to 2.0.0, this should signal to clients that breaking changes have been introduced. Communicate version changes clearly and provide ample deprecation time for older versions.

Lessons Learnt: Failing to follow semantic versioning once resulted in confusion among clients who were not prepared for the breaking changes introduced. This led to integration issues and delayed deployments.

27. Adopt Idempotency in POST, PUT, and DELETE Methods

For operations that create or modify resources, ensure that they are idempotent, meaning that repeated identical requests produce the same result. This is particularly important for PUT and DELETE methods.

For example, submitting a DELETE request multiple times to /orders/123 should result in the same outcome each time—the order should be deleted without additional side effects.

POST requests should be idempotent when generating unique resources by using idempotency keys. This prevents accidental duplication of resource creation in cases where the client might retry the request due to network issues.

TIP: Implement idempotency keys for POST requests to prevent resource duplication during retries. It’s far easier to do this in a POST request that implementing a compensating transaction in the back-end solution.

28. Ensure Resource Representation is Properly Normalised

APIs should follow data normalisation principles to avoid redundant data and improve efficiency. Instead of embedding large objects within resource representations, use links or references to other resources.

For example, instead of embedding an entire user object within an order response, return a link to the user’s details:

{
  "order_id": 1234,
  "user": {
    "id": 5678,
    "link": "/users/5678"
}

This approach reduces response payload size and ensures data is only fetched when needed.

29. Design APIs with Error Codes for Machine Parsing

Error responses should be designed to allow machine parsing, helping clients to handle errors programmatically. Standardise error formats to include an error code, message, and optionally, additional details. The error code should correspond to an internal system-specific identifier that can be referenced in documentation for further troubleshooting.

For instance, a 404 response could return:

{
  "error_code": "404_NOT_FOUND",
  "message": "The requested resource was not found."
}

Standardising error responses allows developers to write better error-handling logic.

30. Implement Circuit Breaker Pattern for API Resilience

The Circuit Breaker pattern is critical for preventing cascading failures in distributed systems. If an API endpoint is experiencing failures or performance issues, the circuit breaker will temporarily block further requests, allowing the system to recover. During the block period, the API should return a fallback response (e.g., 503 Service Unavailable) until the issue is resolved.

The circuit breaker prevents overloading a struggling service and gives time for recovery or alerts to be triggered for intervention. Once the service recovers, the circuit breaker gradually allows traffic through, monitoring the success rate to determine full restoration. This should be implemented with RATE Limiting feature.

31. Provide APIs with Consistent HTTP Headers

HTTP headers are essential for transmitting metadata between clients and servers. Ensure consistency in headers used across all endpoints. Common headers include Authorization for authentication, Content-Type for specifying data formats, Cache-Control for caching directives, and X-Request-ID for tracking unique requests.

Custom headers should follow naming conventions such as X-YourService-Header, avoiding potential conflicts with future HTTP standards.

Lessons Learnt: We once used inconsistent custom headers across different APIs, which led to confusion among consumers and increased integration complexity.

32. Avoid Including Business Logic in the API Layer

Business logic should reside in the application or service layer, not the API layer. The API layer acts as an interface to validate requests, route them to the correct service, and return responses. By keeping the API layer thin and free of complex logic, you make it easier to maintain, test, and scale your system.

Embedding business logic in the API can lead to a tightly coupled architecture where changes to the logic require changes to the API itself. This makes the system harder to update and reduces flexibility. For instance, if the same logic is needed across different interfaces (like web or mobile), duplicating it in multiple APIs increases the risk of inconsistency.

Separating concerns ensures the reuse of business logic across various channels, preventing code duplication. It also simplifies testing, as the business logic can be tested independently of the API. This improves the system’s modularity and scalability. Additionally, keeping business logic out of the API enhances security, as critical processes remain isolated from potential API vulnerabilities.

By adhering to this principle, you make your system more adaptable, ensuring that APIs can evolve without disrupting underlying business operations.

33. Handle Resource Creation with Appropriate Response Codes

When creating resources, return the appropriate HTTP response codes. A successful POST request should return 201 Created along with a link to the newly created resource in the Location header. For example:

HTTP/1.1 201 Created
Location: /users/1234

This practice ensures clients can easily locate the new resource and confirms that the operation succeeded.

34. Design APIs to Support Both Sync and Async Workflows

APIs should be designed to handle both synchronous and asynchronous workflows depending on the nature of the tasks. Synchronous workflows are ideal for operations that can be completed immediately, such as fetching a user profile or submitting a form. For such tasks, the API should respond promptly with the appropriate status code (e.g., 200 OK for success or 400 Bad Request for errors).

On the other hand, asynchronous workflows are necessary for long-running tasks such as file uploads, report generation, or batch processing. In these cases, the API should immediately return a 202 Accepted status to indicate that the request has been received and is being processed. Additionally, the response should include a URL or token that the client can use to poll or check the status of the operation.

By supporting both sync and async workflows, the API becomes more flexible, allowing it to handle a wide variety of use cases.

Synchronous Workflow Example:

// Sync operation - simple user lookup
const getUser = async (userId) => {
  try {
    const response = await fetch(`/api/users/${userId}`);
    if (response.ok) {
      const user = await response.json();
      console.log("User data:", user);
    } else {
      console.log(`Error: ${response.statusText}`);
    }
  } catch (error) {
    console.error("Failed to fetch user data", error);
  }
};

getUser(123); // Immediately returns the user data

In this example, the client makes a synchronous call to fetch a user’s details. The server processes the request and returns the response immediately, providing a 200 OK status if the user data is successfully retrieved.

Asynchronous Workflow Example:

// Async operation - file processing
const startFileProcessing = async (fileId) => {
  try {
    // Initiate the file processing request
    const response = await fetch(`/api/files/${fileId}/process`, {
      method: 'POST',
    });
    
    if (response.status === 202) {
      // The process has started, and a location to check status is provided
      const { statusUrl } = await response.json();
      console.log("File processing initiated. Check status at:", statusUrl);
      
      // Poll the status URL until the process is completed
      let completed = false;
      while (!completed) {
        const statusResponse = await fetch(statusUrl);
        const status = await statusResponse.json();
        
        if (status.completed) {
          console.log("File processing completed:", status.result);
          completed = true;
        } else {
          console.log("Processing... Checking again in 5 seconds.");
          await new Promise(res => setTimeout(res, 5000)); // Wait before polling again
        }
      }
    } else {
      console.error("Failed to start file processing:", response.statusText);
    }
  } catch (error) {
    console.error("Error starting file processing", error);
  }
};

startFileProcessing('file123'); // Starts an asynchronous file processing operation

In this example, the client initiates a file processing operation. The server responds with a 202 Accepted status, acknowledging that the request has been received and that processing will occur asynchronously. The client is given a statusUrl to poll and check the progress of the operation, which is periodically checked until completion.

Lessons Learnt: We initially handled long-running tasks synchronously, causing client timeouts and resource exhaustion during heavy operations. After implementing asynchronous handling for these tasks with 202 Accepted responses, client performance improved dramatically, and the system became more resilient to load.

35. Avoid Overloading APIs with Multi-Functional Endpoints

Each API endpoint should be designed with a single, clear purpose. Overloading an endpoint with multiple, unrelated actions creates ambiguity and confusion for consumers, making it harder to use and maintain. For example, an endpoint like /user/action that performs both updates and deletions based on the payload (e.g., using a flag to indicate which action to take) can become difficult for clients to understand and prone to errors.

A better approach is to use distinct endpoints for each operation, ensuring that each endpoint has a well-defined function. For example:

• POST /users – to create a new user
• PUT /users/{id} – to update a user’s information
• DELETE /users/{id} – to delete a user

By clearly separating actions into different endpoints, you reduce the cognitive load on developers using the API and ensure that each request has a predictable outcome. This also aligns with REST principles, where HTTP methods like GET, POST, PUT, and DELETE are meant to perform specific operations on resources.

For example, instead of using /user/action with a body that specifies whether to update or delete:

{
  "action": "update",
  "name": "John Doe"
}

Use /users/{id} for updates:

{
  "name": "John Doe"
}

And /users/{id} for deletions using DELETE:

DELETE /users/123

This method ensures clarity, reduces ambiguity, and makes the API easier to document and maintain.

Lessons Learnt: In a previous project, we overloaded an endpoint to handle multiple operations based on the request body, which led to bugs and unclear usage patterns. After switching to single-purpose endpoints, clients found the API much easier to integrate, and we saw fewer support issues related to incorrect API usage.

36. Return Meaningful HTTP Headers with Metadata

HTTP headers provide valuable metadata that can enhance the client-server interaction. Ensure that every API response includes relevant headers. For example, include Cache-Control headers to manage caching, ETag headers for versioning and validation, and Link headers for pagination.

When dealing with paginated responses, return a Link header to guide consumers on how to navigate between pages:

Link: </users?page=2>; rel="next", </users?page=1>; rel="prev"

This makes it easier for clients to navigate your API efficiently without relying solely on response bodies for metadata.

37. Handle Timeouts Gracefully in API Requests

Timeouts can occur when processing requests takes too long, especially for long-running tasks. Handle these timeouts gracefully by providing a clear response to the client. Use appropriate status codes such as 504 Gateway Timeout to indicate that the server didn’t respond within the required time.

Ensure that your API provides a mechanism for clients to retry or continue a request in case of a timeout. Where possible, return partial progress or a reference to the current state of the request so the client knows how to proceed without losing data.

38. Use JWT (JSON Web Tokens) for Stateless Authentication

JSON Web Tokens (JWT) are an ideal solution for stateless authentication in RESTful APIs. A JWT is a token that is signed and encoded, allowing the server to verify the user’s identity without storing session data. When a user authenticates, the server generates a JWT that includes claims about the user’s identity and permissions.

The client sends this token in the Authorization: Bearer header with each request, and the server validates the token on each call. JWTs are ideal for distributed systems, enabling stateless and scalable authentication without centralised session management.

TIP: Keep JWTs short-lived and implement refresh tokens for extended sessions to minimise the risk of compromised tokens being used.

39. Design with Rate Limiting Response Headers

When implementing rate limiting, always provide clear feedback to clients regarding their current usage and how much quota remains. Use the following HTTP headers to inform clients:

• X-RateLimit-Limit: The maximum number of requests allowed in the window.
• X-RateLimit-Remaining: The number of requests remaining in the current window.
• X-RateLimit-Reset: The time when the rate limit resets, provided as a Unix timestamp.

These headers enable clients to adjust their behaviour proactively and avoid exceeding rate limits. Otherwise it’s difficult to establish if the API is broken somewhere or something else has gone wrong.

40. Avoid Overusing Query Strings for Action Triggers

Query strings should be used for filtering, pagination, and sorting—not for triggering actions. Avoid patterns like /users?action=delete&id=123 to initiate actions on resources. Instead, use appropriate HTTP methods (DELETE, POST, PUT) and define clear resource paths to perform these operations.

For example, deleting a user should be done through a dedicated DELETE method to /users/{id}, not via query strings, ensuring clarity in resource actions and making the API more intuitive.

41. Ensure Proper Validation of User Inputs

Validating user inputs is critical for both security and data integrity. Always perform input validation on both the client and server sides. Validate parameters like query strings, path variables, and request bodies against expected formats and constraints. This ensures that only valid data reaches your API, reducing the risk of invalid or malicious input.

For example, if your API expects an email, ensure it is properly validated and follows the standard email format. Return a clear error message in cases of invalid input, using HTTP status codes such as 400 Bad Request.

Lessons Learnt: We once neglected to validate user input, resulting in malformed data being stored in our database, which later required a costly data clean-up and system fixes.

42. Maintain Backward Compatibility Through API Evolution

As your API evolves, it’s essential to maintain backward compatibility so that existing clients are not forced to upgrade immediately. Introduce new features or changes in a non-breaking way, ensuring older clients can continue to use the API without modifications.

For instance, when introducing new fields in responses, make them optional so that older clients can continue to function without expecting those fields. Always provide ample notice when deprecating features and allow for a smooth transition to newer versions.

43. Standardise Error Codes and Format Across APIs

Establish a standard error format across all your APIs to ensure consistency. Each error response should include an error code, a user-friendly message, and optionally, detailed information about the error:

{
  "error_code": "USER_NOT_FOUND",
  "message": "The requested user does not exist."
}

This format allows clients to handle errors programmatically and provides them with the necessary information to resolve issues. Additionally, standardising error codes across APIs ensures that consumers know what to expect regardless of which service they are using.

44. Employ Retry-After Header for Rate-Limited Responses

When a client exceeds the rate limit, return a 429 Too Many Requests status along with a Retry-After header indicating how long the client should wait before making another request:

HTTP/1.1 429 Too Many Requests
Retry-After: 120

This reduces unnecessary retries and allows clients to adjust their behaviour based on the rate limit policy.

45. Implement Logging and Monitoring for API Requests

Logging and monitoring are essential for maintaining visibility into your API’s performance and usage. Log all API requests, including important details such as request method, resource accessed, status code, and execution time. Ensure that logs capture enough information to help diagnose issues without exposing sensitive data.

Use monitoring tools to track metrics like request latency, error rates, and throughput. Real-time monitoring enables proactive identification of issues before they impact users.

TIP: Implement correlation IDs across services so that logs for related requests can be easily traced across different components of your system.

46. Design APIs with Event-Driven Models for Scalability

For APIs that need to handle large volumes of events or data, consider adopting an event-driven architecture. Instead of clients polling the API for updates, use asynchronous methods like webhooks to notify clients of changes or updates. This reduces unnecessary traffic and ensures timely updates.

APIs can publish events to a message broker such as Kafka, allowing consumers to subscribe and act on events in real-time without directly polling the API.

47. Utilise Content Negotiation for Resource Format Flexibility

Content negotiation allows clients to request different formats for the same resource. Support this by respecting the Accept header in client requests. For example, a client might request JSON or XML:

Accept: application/json
Accept: application/xml

Design your API to return the requested format or an appropriate error if the format is unsupported. JSON should be the default format unless otherwise specified by the client.

48. Return Pagination Metadata with List Responses

When returning a list of resources, include pagination metadata to help clients understand the size of the dataset and how to navigate through it. This metadata should include:

• Total count of items
• Current page
• Page size
• Next and previous page links

For example:

{
  "data": [
    { "id": 1, "name": "John Doe" },
    { "id": 2, "name": "Jane Smith" }
  ],
  "pagination": {
    "total": 100,
    "page": 1,
    "page_size": 20,
    "next_page": "/users?page=2"
  }
}

This structure improves usability, ensuring that clients can easily traverse large datasets.

49. Define API Timeouts Based on Use Case

Every API request should have a well-defined timeout to prevent requests from hanging indefinitely. Set timeouts based on the expected time required for an operation. For instance, simple GET requests should have short timeouts (e.g., 5-10 seconds), while complex operations, such as bulk processing, might require longer timeouts.

Use status codes like 504 Gateway Timeout to inform clients when their request exceeds the allowed time.

50. Minimise Latency with CDNs for API Delivery

To reduce latency and improve response times, use Content Delivery Networks (CDNs) for delivering static assets such as documentation, images, or other large resources that your API may reference. CDNs ensure that these assets are delivered from servers closer to the client, reducing the time it takes to access these resources.

For APIs with global reach, CDNs are critical for ensuring performance consistency across regions.

51. Support Pagination, Filtering, and Sorting in Query Parameters

Clients should have control over how they retrieve data, especially when working with large datasets. Support query parameters for pagination (?page=2&limit=20), filtering (?status=active), and sorting (?sort=name_asc). These features allow clients to fetch, filter, and sort data more efficiently.

For example:

GET /users?page=2&limit=20&status=active&sort=name_asc

By supporting these parameters, your API becomes more flexible and efficient, enabling clients to retrieve exactly the data they need.

52. Design APIs to Work with Microservices Architecture

In a microservices architecture, each service should handle a distinct piece of the application’s functionality. When designing your API, ensure that it communicates efficiently with other services, maintaining a clear separation of concerns. Each microservice should be autonomous and capable of being deployed or updated independently without affecting other services.

To maintain a well-structured API ecosystem, design services to communicate through APIs using HTTP, gRPC, or event-driven architectures. API gateways can manage traffic across services, handling common concerns such as rate limiting, security, and load balancing. Consider how services interact to avoid tight coupling, ensuring scalability and resilience across the system.

When possible, use asynchronous communication for cross-service interactions, such as through message brokers like Kafka or RabbitMQ. This reduces latency and enables services to decouple themselves from direct synchronous calls, improving fault tolerance.

Lessons Learnt: We initially tightly coupled some services in our microservice-based API, which led to cascading failures when one service experienced downtime. This experience taught us to favour loose coupling and asynchronous messaging.

53. Avoid Including Credentials in API Requests

Never send sensitive information, such as credentials, in the URL or as part of query strings (e.g., /login?username=user&password=pass). This practice exposes sensitive data to logs, intermediaries, and caches, creating security risks.

Instead, always send credentials or tokens in secure HTTP headers, such as the Authorization: Bearer header for OAuth tokens. Additionally, use HTTPS to encrypt all communications and prevent credentials from being intercepted.

Adopt OAuth 2.0 and other modern token-based authentication mechanisms to reduce the reliance on traditional username-password authentication methods. Implement short-lived access tokens with refresh mechanisms to mitigate risks associated with token exposure.

Lessons Learnt: We once allowed sensitive data to be sent through query strings, which was later exposed in access logs, compromising security. Moving to header-based credentials resolved this issue and improved security.

54. Utilise API Gateway for Centralised Management

API gateways play a crucial role in managing, securing, and scaling APIs, especially in microservices architectures. An API gateway acts as a single entry point through which all API requests pass, enabling functionalities like rate limiting, authentication, logging, load balancing, and request transformation.

By centralising these responsibilities, the gateway offloads these concerns from individual microservices, allowing them to focus solely on business logic. Tools like Kong, Amazon API Gateway, or Azure API Management offer robust features for managing complex APIs.

The gateway can also serve as a reverse proxy, directing client requests to the appropriate service while abstracting the internal architecture. It helps manage versioning, throttling, and error handling across all services in a consistent manner.

TIP: Implement API gateway security features such as IP whitelisting, request validation, and DDoS protection to further secure your API ecosystem.

55. Ensure API Supports Internationalisation Standards

If your API serves global users, it’s critical to design it with internationalisation in mind. Support multiple languages by allowing clients to specify the Accept-Language header, and return content in the appropriate language. Ensure all date and time formats use ISO 8601 for standardisation, and handle timezone differences by including timezone information in timestamps.

For monetary data, include currency codes like ISO 4217 and always return amounts in a format that respects regional differences. Consider encoding responses in UTF-8 to support characters from various languages and ensure proper handling of non-ASCII characters.

When dealing with input, validate data in a locale-sensitive manner (e.g., phone numbers, addresses, and currencies) to ensure that the API is capable of handling region-specific variations.

Lessons Learnt: We once failed to include proper internationalisation support, resulting in misinterpretation of timestamps and numerical data, which led to several client-side errors across different regions.

56. Return ETag Headers for Efficient Resource Caching

ETags (Entity Tags) are a useful tool for optimising API responses through efficient caching. An ETag is a unique identifier assigned to a specific version of a resource, allowing clients to make conditional requests using the If-None-Match header. This reduces unnecessary data transfer, as the server can return a 304 Not Modified response if the resource has not changed, instead of retransmitting the entire payload.

For example, a client can include the ETag in its request:

If-None-Match: "xyz123"

The server checks the ETag and returns a 304 status if the resource hasn’t changed.

ETags improve efficiency and bandwidth usage, particularly for APIs serving large or frequently accessed resources.

57. Implement Retry Mechanism for Transient Failures

Transient failures, such as network outages or timeouts, are common in distributed systems. To ensure API reliability, implement a retry mechanism that allows clients to automatically retry failed requests. Use exponential backoff to avoid overwhelming the server with repeated retries in a short period of time.

For example, after the first failure, retry after 1 second, then 2 seconds, then 4 seconds, doubling the delay each time. Be sure to include a maximum retry limit to avoid indefinite retries.

Ensure that your API is idempotent where applicable, so that retrying a failed request won’t cause unintended side effects, such as creating duplicate resources.

Lessons Learnt: We once lacked a proper retry mechanism in our API, which resulted in degraded user experiences during temporary network outages. Implementing retries with exponential backoff significantly improved API reliability during transient failures.

58. Offer Real-Time Subscription to Resource Changes Using Webhooks

Instead of requiring clients to poll the API for updates, offer webhook subscriptions to notify them of resource changes in real time. With webhooks, the API can send HTTP POST requests to the client’s endpoint when certain events occur (e.g., order status updates, new user registrations).

This reduces unnecessary polling traffic, decreases latency, and ensures clients are notified of changes as they happen. Allow clients to subscribe to webhooks programmatically by providing an endpoint where they can register their webhook URLs and specify the types of events they wish to receive.

Ensure proper security measures, such as validating webhook payloads with signatures and restricting the IP addresses from which webhooks can be accepted.

TIP: Always provide retry mechanisms for webhook delivery to ensure that events are not lost in case the client’s server is temporarily unavailable.

59. Provide API Client SDKs for Ease of Integration

To enhance developer experience and promote faster adoption, provide client SDKs in popular programming languages such as Python, JavaScript, and Java. SDKs abstract the underlying API complexity, allowing developers to interact with your API using familiar methods and objects without having to manually craft HTTP requests.

SDKs also handle common tasks like authentication, pagination, and error handling, further simplifying the integration process. Ensure that SDKs are kept up to date with API changes, and provide clear documentation to accompany each SDK.

TIP: Include automated tests in your SDK repositories to ensure compatibility with the latest API versions and reduce the risk of introducing bugs during integration.

60. Use UUIDs for Unique Resource Identification

In distributed systems, it’s essential to generate unique identifiers for resources that do not collide. Use UUIDs (Universally Unique Identifiers) as a best practice for resource IDs, rather than relying on simple integer sequences. UUIDs are 128-bit numbers that are guaranteed to be unique, even across different databases or systems.

For example:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "John Doe"
}

This ensures that resource IDs remain unique across distributed environments, mitigating the risk of ID conflicts when merging data from multiple sources.

Lessons Learnt: We initially used auto-incrementing integers for resource IDs, which caused conflicts during database replication. Switching to UUIDs resolved the issue and improved the scalability of our API.

61. Avoid Blocking Calls in Synchronous Operations

Blocking calls can degrade the performance of APIs, especially when dealing with I/O-bound tasks such as database queries or external service calls. Where possible, use asynchronous programming techniques to free up server resources while waiting for these operations to complete. This improves the scalability and responsiveness of the API.

For example, in Node.js, use async/await or Promises to handle asynchronous tasks without blocking the event loop:

async function fetchData() {
  const data = await externalApi.getData();
  return data;
}

By avoiding blocking calls, you can serve more requests concurrently, improving throughput and reducing response times.

62. Design APIs to Handle Large Payloads Efficiently

When dealing with large payloads, such as file uploads or big data requests, implement strategies to efficiently manage the data without overwhelming the API. Break large payloads into smaller chunks and use streaming to handle the data incrementally, rather than loading everything into memory at once.

For example, use Content-Range headers to support file uploads in chunks, allowing clients to send large files in smaller parts:

Content-Range: bytes 0-999/5000

Additionally, use compression techniques like GZIP to reduce the size of payloads sent over the network, improving performance and bandwidth usage.

63. Ensure APIs Follow Security Best Practices (OWASP)

APIs must be built with security at the forefront, following industry standards such as the OWASP API Security Top 10. These best practices help to mitigate common vulnerabilities like injection attacks, broken authentication, and insufficient logging. Implement robust input validation, ensuring that all user input is sanitised before processing to prevent attacks like SQL injection and XSS.

Authentication should always use secure methods like OAuth 2.0, and sensitive data must be transmitted only over HTTPS. Implement rate limiting and IP whitelisting to prevent abuse and potential DDoS attacks. Employ proper logging mechanisms, ensuring that security events are captured without logging sensitive information like passwords or tokens.

Enforce least privilege for API access, restricting users to only the resources and actions they need to perform. Implement security headers like Content-Security-Policy (CSP), X-Frame-Options, and X-Content-Type-Options to further secure your API from vulnerabilities.

Lessons Learnt: In a previous deployment, we failed to implement proper security headers, leaving an API endpoint vulnerable to clickjacking and content sniffing attacks. We also faced a DDoS attack that could have been mitigated with proper rate limiting. After following OWASP best practices and implementing security controls, we drastically reduced the attack surface and prevented further breaches.

64. Provide Multi-Factor Authentication for API Access

Multi-factor authentication (MFA) provides an additional layer of security beyond just username and password. When implementing APIs that deal with sensitive data or financial transactions, use MFA to ensure that only authorised users can access critical endpoints.

MFA usually requires users to verify their identity with a second factor such as a mobile app, hardware token, or one-time password (OTP). APIs should support MFA by integrating with identity providers that handle the generation and verification of the second factor.

Implement MFA for login or sensitive actions (e.g., money transfers, account modifications). Once authenticated, issue short-lived tokens (JWT or OAuth tokens) to ensure that the session remains secure but does not require repeated re-authentication.

Lessons Learnt: We initially relied on simple username-password authentication for sensitive operations, which led to a data breach when several users’ credentials were compromised. After integrating MFA into the workflow, unauthorised access attempts were reduced by 95%, and the overall security of our API improved significantly.

65. Use Descriptive URI Patterns Without Action Verbs

In RESTful API design, URIs should represent resources, not actions. Avoid using verbs like create, delete, or update in your endpoint URIs, as the HTTP methods themselves imply the action. Instead, focus on nouns that represent the resources you’re working with.

For example:

• Use POST /users to create a new user, not POST /createUser.
• Use DELETE /users/{id} to delete a user, not DELETE /deleteUser/{id}.

This makes the API more intuitive and self-descriptive. The HTTP methods (GET, POST, PUT, DELETE) should indicate the intended action, while the URI should remain consistent across the API.

Ensure that URIs follow a consistent pattern. Use plural nouns for collections (/users), singular nouns for individual resources (/users/{id}), and hierarchical URIs to represent relationships (/users/{id}/orders).

Lessons Learnt: In one of our earlier APIs, we used action verbs in URIs, like /createUser and /deleteUser, which confused clients. They had to remember which URIs to call for each action, rather than relying on HTTP method conventions. Refactoring the API to use RESTful conventions significantly improved usability and reduced onboarding time for new developers.

66. Maintain Consistent Timezone and Date Formats Across APIs

Time-sensitive data should always be consistent across the API. Ensure that all dates and times use the ISO 8601 standard (YYYY-MM-DDTHH:mm:ssZ) and include timezone information. This guarantees that clients across different time zones can accurately interpret the data without the need for manual adjustments.

When clients submit time-related data, require them to send it in ISO 8601 format and validate the input before processing. For time-sensitive operations, such as bookings or event management, include both the server’s and client’s timezone information to ensure accuracy and avoid misinterpretation.

To handle varying locales, use the Accept-Language header to format date and time responses in a client-friendly manner if necessary.

Lessons Learnt: In one project, we failed to enforce a consistent date format across our APIs, leading to discrepancies in event timing. Some clients received timestamps in UTC, while others saw local server time. This caused confusion and scheduling issues. After switching to ISO 8601 with clear timezone data, the problem was eliminated, and client-side handling of time-sensitive data improved drastically.

67. Use Correlation IDs to Trace API Requests Across Systems

In distributed systems or microservices architectures, tracing requests across multiple services can be difficult. Implementing Correlation IDs allows you to track requests through different components of your system.

When a client makes a request, generate a unique correlation ID and pass it along with the request as a custom header (e.g., X-Correlation-ID). Every service that handles the request should log this ID, allowing developers to trace the entire lifecycle of the request across all systems involved.

For example:

• A client makes a request to Service A.
• Service A generates a correlation ID and forwards it to Service B.
• Service B logs the correlation ID and passes it to Service C.
• When debugging, developers can trace this correlation ID across all services to determine where errors occurred.

Lessons Learnt: We once deployed a system without correlation IDs, and when a request failed halfway through, it became almost impossible to track down the root cause across services. By implementing correlation IDs, we were able to trace issues through the entire request path, dramatically reducing debugging time and improving system observability.

68. Provide a Health Check Endpoint for API Monitoring

To ensure that your API is always available and operational, include a dedicated health check endpoint that monitoring tools can call to verify the API’s status. The health check should return a 200 (OK) status if the service is functioning correctly and include detailed information about the service’s health, such as database connections, service dependencies, and system load.

A simple health check might just return a 200 OK response, while more complex checks could return additional data in the response body, like:

{
  "status": "healthy",
  "uptime": "50000s",
  "db_status": "connected",
  "queue_status": "healthy"
}

The health check endpoint allows load balancers or monitoring services to automatically remove instances that are unresponsive or in a degraded state from the pool of available servers.

Lessons Learnt: In a previous deployment, we didn’t have a health check endpoint, and as a result, our load balancer continued to route traffic to instances that were unhealthy. This caused degraded performance for some users until we manually intervened. Adding a health check endpoint allowed the system to self-correct by automatically removing failing instances from the load balancer, significantly improving system resilience.

69. Include CORS Support for Cross-Domain API Requests

Cross-Origin Resource Sharing (CORS) is essential for APIs that will be accessed from web applications hosted on different domains. CORS allows servers to specify which domains are permitted to make API requests by setting the Access-Control-Allow-Origin header.

For example:

Access-Control-Allow-Origin: https://example.com

This ensures that only authorised domains can access your API from the browser. CORS headers should also be used to specify allowed methods (GET, POST, etc.), headers, and credentials to control the type of access clients can have.

Ensure you implement CORS securely by only allowing trusted domains, especially when dealing with APIs that expose sensitive data.

Lessons Learnt: Initially, we misconfigured CORS settings, allowing all domains to access our API (* wildcard), which exposed us to potential security vulnerabilities from malicious domains. After refining our CORS policy to only allow trusted origins, we closed this vulnerability and improved the overall security of our API.

70. Handle HTTP Redirects Appropriately in REST APIs

Sometimes, resources are moved, or clients need to be directed to a different location. Use HTTP status codes like 301 Moved Permanently or 302 Found to inform clients of the new location of a resource. Always include the Location header in the response to indicate the new URL.

For example, if a resource has moved, return:

HTTP/1.1 301 Moved Permanently
Location: /new-resource-location

Clients should handle these redirects gracefully, following the new location to retrieve the resource. Use redirects sparingly to avoid unnecessary complexity, and ensure that clients are aware of your redirect policy through proper documentation.

Lessons Learnt: We once moved a resource without providing a proper redirect, resulting in broken client integrations and unnecessary 404 errors. Implementing proper HTTP redirects allowed for a seamless migration to the new location, reducing client disruptions and integration issues.

71. Ensure Clear API Rate Limits for End-Users

Rate limiting is crucial for managing API resources and preventing abuse. Define and enforce rate limits based on your API’s capacity and the expected traffic volume. Return headers such as X-RateLimit-Limit (the maximum number of requests allowed), X-RateLimit-Remaining (remaining requests in the current window), and X-RateLimit-Reset (time when the limit resets) with each request.

For example:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 500
X-RateLimit-Reset: 1372700873

Make sure to return a 429 Too Many Requests status code when the client exceeds the limit, along with information on when they can resume sending requests. Rate limiting helps to ensure fair usage, prevent abuse, and maintain optimal API performance under high traffic.

Lessons Learnt: In one of our APIs, we initially deployed it without proper rate limits, which resulted in one customer generating excessive traffic, leading to degraded service for other clients. After implementing rate limiting, we ensured that all users received equitable access to resources, and it improved overall system stability during high traffic.

72. Design APIs to Handle Fault Tolerance in Distributed Systems

In distributed systems, failures are inevitable, whether from network issues, service outages, or hardware problems. Your API should be designed to handle these failures gracefully. Implement circuit breakers to stop requests to failing services and use fallback mechanisms, such as returning cached data or a generic response, to mitigate the impact of a failure.

When services are temporarily unavailable, return a 503 Service Unavailable status, along with the Retry-After header to inform clients when they can try again:

HTTP/1.1 503 Service Unavailable
Retry-After: 120

Additionally, use retries with exponential backoff to handle transient failures without overwhelming the system.

Lessons Learnt: We once built an API that directly called third-party services synchronously. When the external service experienced an outage, our API failed completely, causing downtime. After implementing fault-tolerant patterns like circuit breakers and retries, we were able to gracefully handle third-party service failures, improving the resilience of our API.

73. Use GraphQL Only When Necessary

While GraphQL offers flexibility by allowing clients to query exactly the data they need, it comes with additional complexity compared to REST. Use GraphQL only when your API needs to serve highly dynamic queries or when clients require the ability to aggregate multiple resources in a single request.

For example, a REST API may require multiple requests to get user information and their orders, but a GraphQL query can retrieve both in one call:

{
  user(id: "123") {
    name
    orders {
      id
      total
    }
  }
}

However, GraphQL introduces complexity in terms of query optimisation, security (e.g., preventing overly complex queries), and caching. Stick with REST for simpler use cases unless there’s a clear benefit to using GraphQL.

Lessons Learnt: We once migrated an entire API from REST to GraphQL, assuming it would simplify the client experience. Instead, it introduced performance issues due to unoptimised queries and made security more difficult to manage. We reverted some endpoints back to REST for simpler use cases while keeping GraphQL for complex, highly customised queries.

74. Ensure REST APIs are Discoverable Through Self-Descriptive URIs

REST APIs should be easy to navigate without requiring external documentation for every step. Implement self-descriptive URIs and meaningful HTTP status codes. For example, return hypermedia links in responses to guide clients on the next possible actions:

{
  "id": 123,
  "status": "active",
  "links": {
    "self": "/users/123",
    "update": "/users/123/update",
    "delete": "/users/123/delete"
  }
}

This allows clients to discover how to interact with the API without needing to refer to external documentation constantly. It also makes your API more flexible, as new features can be introduced without breaking existing functionality.

Lessons Learnt: Initially, our API did not include hypermedia links, which led to confusion for clients, as they had to constantly refer to the documentation. After adding hypermedia links, clients were able to navigate the API more easily, reducing support requests.

75. Leverage Distributed Tracing in Microservices-Based APIs

In a microservices architecture, tracking requests across multiple services can be challenging. Implement distributed tracing to track the flow of a request through various services. Use tools like Jaeger or Zipkin to collect and visualise trace data.

When a request enters the system, assign a unique trace ID and pass it along with the request to all downstream services. Each service logs this trace ID, making it easier to identify bottlenecks or failures in the system. Traces should include timing information to identify which services are contributing to latency.

Lessons Learnt: We initially deployed a microservices architecture without distributed tracing. When issues occurred, it was difficult to determine where the problem originated. After introducing tracing, we significantly reduced the time required to troubleshoot performance issues, as we could track the entire request flow across services.

76. Provide Fine-Grained Permissions Control for API Access

Not all API users need access to every endpoint or resource. Implement fine-grained permissions to control which users can access specific resources or perform certain actions. Use Role-Based Access Control (RBAC) to assign different roles to users (e.g., admin, read-only, user), and enforce these permissions at the API level.

For example, an admin might have access to /users/create and /users/delete, while a regular user might only have access to /users/{id} to view their own data. Permissions should be enforced both at the endpoint and data level to ensure that users can only interact with resources they are authorised for.

Lessons Learnt: We once gave users broad access to various API endpoints, which resulted in unintended data exposure. By implementing fine-grained access control, we limited users to only the actions and resources relevant to their roles, improving security and reducing potential misuse.

77. Offer API Throttling to Maintain Service Stability

Throttling helps ensure that no single client can overwhelm your API, preventing service degradation during high traffic periods. Throttling limits how quickly clients can make requests to the API, capping usage beyond a certain threshold.

For example, if your API allows up to 100 requests per minute, throttling ensures that once a client exceeds this limit, further requests will be delayed until they fall back within the limit:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

Throttling not only protects your API from malicious use but also ensures that system resources are distributed fairly among all clients, maintaining overall service stability.

Lessons Learnt: During a product launch, we experienced service disruptions due to high traffic from a small number of clients. Throttling could have prevented this issue by regulating traffic and ensuring that all users had fair access to the system. After implementing throttling, we were able to maintain consistent service levels during peak demand.

78. Utilise WebSockets for Real-Time Data in Specific Use Cases

While REST APIs are stateless, certain use cases—such as real-time notifications, live data updates, or chat applications—may require continuous data streams. In these cases, use WebSockets to establish a persistent, full-duplex connection between the client and server, enabling real-time communication.

WebSockets are more efficient than repeatedly polling a REST API for updates, as they allow the server to push updates to the client as soon as data changes. However, WebSockets introduce added complexity, including maintaining connection states and handling network failures.

Lessons Learnt: In one of our projects, we initially used REST polling for live data updates, which caused performance issues and high server load. After switching to WebSockets, we reduced the server load significantly and improved the real-time experience for users.

79. Support Multi-Tenant Architecture in API Design

If your API is intended to serve multiple clients (tenants) within a single system, ensure it is designed to handle multi-tenancy. Each tenant’s data should be isolated, either by using separate databases or by scoping data to tenant IDs.

For example, if two companies, Company A and Company B, use the same API, the data belonging to Company A should never be visible to users from Company B. Implement authentication and authorisation mechanisms that include tenant-specific validation to ensure that users can only access their own tenant’s data.

Lessons Learnt: Initially, we did not isolate tenant data properly, leading to cross-tenant data exposure. This was flagged as a major security breach, and we had to refactor the entire API to enforce stricter multi-tenancy rules, improving data isolation and security.

80. Use JSON Schema for Input Validation

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "email": { "type": "string", "format": "email" },
    "age": { "type": "integer", "minimum": 18 }
  },
  "required": ["name", "email"]
}

This schema ensures that the input contains a valid name, an email in the correct format, and an optional age that must be an integer and at least 18 if provided. Enforcing this validation at the API layer helps protect against invalid data entering the system and ensures that your business logic receives data in the expected structure.

Lessons Learnt: We initially failed to implement comprehensive input validation, resulting in corrupted data entering our database, which caused downstream failures. After introducing JSON Schema for strict validation, we reduced errors significantly and improved the reliability of our API.

81. Provide Batch Endpoints for Efficient Bulk Operations

Batch endpoints allow clients to submit multiple operations in a single request, improving efficiency and reducing the number of round trips required. For example, instead of sending separate POST requests to create 100 users, the client can submit a single batch request that processes all 100 users in one go.

Design batch endpoints carefully to handle large datasets. Include error handling for partial successes, where some operations may fail while others succeed, and return detailed feedback on each operation:

{
  "results": [
    { "id": 1, "status": "created" },
    { "id": 2, "status": "failed", "error": "email already exists" }
  ]
}

Batch processing not only reduces latency but also makes it easier for clients to manage large-scale operations with fewer requests.

Lessons Learnt: Initially, our API lacked batch endpoints, forcing clients to send multiple individual requests, which overloaded our servers and led to poor performance during high-traffic periods. Introducing batch processing dramatically improved the efficiency of the system and reduced the strain on our infrastructure.

82. Implement Data Masking for Sensitive Information

Data masking is a technique used to protect sensitive information (such as personally identifiable information, credit card numbers, or passwords) in API responses. Only expose necessary information to clients while masking or redacting sensitive data to reduce security risks.

For example, when returning user data, you can mask the user’s email and credit card information:

{
  "name": "John Doe",
  "email": "j***@example.com",
  "credit_card": "**** **** **** 1234"
}

Ensure that the full information is only available to users or services with the appropriate permissions. Data masking reduces the risk of exposing sensitive information in logs, UI displays, or to unauthorised users.

Lessons Learnt: In an early version of our API, we inadvertently exposed full credit card numbers in API responses, which became a serious security vulnerability. After implementing data masking, we minimised the risk of exposing sensitive data and improved the security of our API.

83. Use Circuit Breakers to Handle API Downtime

The circuit breaker pattern helps prevent cascading failures in distributed systems. When an API endpoint or service is experiencing issues, the circuit breaker temporarily “trips” and prevents additional requests from being routed to the failing service. This allows the system to recover without becoming overwhelmed by additional traffic.

Once the service is restored, the circuit breaker gradually allows requests to resume, ensuring that the system has recovered before returning to full traffic. Circuit breakers are especially useful in microservices architectures where one failing service could cause others to fail if not handled properly.

Lessons Learnt: We encountered a situation where a downstream service failed, and without a circuit breaker, our API continued to send requests, exacerbating the issue and causing widespread outages. Implementing circuit breakers reduced downtime and prevented service failures from spreading across the system.

84. Implement Secure API Key Management

API keys are commonly used to authenticate users, but they must be managed securely. Ensure that API keys are generated with sufficient entropy and are stored securely. Avoid hardcoding API keys in your codebase or exposing them in URLs, query strings, or logs.

Implement rotation policies for API keys, allowing clients to regenerate keys without downtime. For sensitive operations, ensure that API keys have limited scopes, so they can only access the resources and perform the actions they are explicitly authorised for.

Use headers like Authorization: Bearer <API_KEY> to securely transmit API keys, and always enforce HTTPS to encrypt the communication.

Lessons Learnt: We once had a case where API keys were hardcoded in public repositories, leading to unauthorised access and security breaches. After implementing secure API key management, including periodic rotation and proper scoping, we significantly improved the security of our system and protected against further unauthorised access.

85. Ensure Efficient Use of API Rate Limiting Techniques

While rate limiting is necessary to protect APIs from abuse, it should be implemented efficiently to avoid hampering legitimate use. Design rate limits to account for varying client needs and traffic patterns. Provide flexible rate-limiting policies based on factors such as API key, user type, or specific endpoints.

For example, allow higher limits for trusted enterprise clients while maintaining stricter limits for free-tier users or public access:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 500

Return appropriate error codes (429 Too Many Requests) when rate limits are exceeded and provide clients with guidance on how to adjust their usage or when to retry.

Lessons Learnt: We once implemented a strict rate limit across the board, which led to dissatisfaction from our premium customers who had legitimate needs for higher throughput. After introducing tiered rate limits, we were able to satisfy both public and enterprise clients, improving user experience while protecting our system from abuse.

86. Employ gRPC for Low-Latency API Use Cases

gRPC is a high-performance, open-source RPC framework that is well-suited for low-latency, high-throughput use cases. Unlike traditional REST APIs, gRPC uses Protocol Buffers (protobuf) for serialisation, resulting in smaller payloads and faster transmission. This makes gRPC ideal for microservices communication and real-time applications where latency is critical.

gRPC also supports bi-directional streaming, enabling clients and servers to send and receive messages in real-time over a single connection. However, gRPC adds complexity and is not always suitable for public-facing APIs due to limited browser support.

Lessons Learnt: Initially, we used REST for a high-throughput, low-latency application, but it struggled with performance under heavy load. After switching to gRPC, we saw a significant improvement in response times and throughput, especially for internal microservice communication.

87. Define Consistent and Predictable Field Names in Responses

Consistency in field names across API responses is key to providing a predictable experience for clients. Ensure that field names are meaningful and follow a standard convention throughout the API. Use snake_case or camelCase consistently, and avoid mixing the two.

For example:

• user_name (snake_case)
• userName (camelCase)

Choose one convention and apply it uniformly across all resources and responses. Avoid abbreviations or cryptic field names that could confuse users. Field names should clearly describe the data they represent, improving readability and reducing errors in client implementations.

Lessons Learnt: We once used inconsistent field names in our API, mixing snake_case and camelCase. This caused confusion for clients and led to integration errors. Standardising field names across all endpoints significantly improved usability and reduced client-side bugs.

88. Use OpenAPI 3.1 for Comprehensive API Documentation

The OpenAPI Specification (OAS) is the industry standard for describing REST APIs. Version 3.1 offers improved support for JSON Schema and enables you to define every aspect of your API, including endpoints, request parameters, response structures, and authentication methods.

Providing OpenAPI documentation allows developers to understand how to interact with your API without needing additional resources. Tools like Swagger UI or Redoc can generate interactive documentation from OpenAPI specs, allowing users to test endpoints directly from the documentation.

Ensure that your OpenAPI specification is kept up-to-date with API changes, and include detailed descriptions and examples for every operation.

Lessons Learnt: We initially provided documentation manually, which was difficult to maintain and often became outdated. After adopting OpenAPI, we were able to automate documentation updates, significantly reducing errors and improving the developer experience.

89. Provide a Clear Deprecation Strategy for Outdated Endpoints

As your API evolves, some endpoints will eventually become obsolete. To avoid breaking client integrations, implement a clear deprecation strategy. Announce deprecations well in advance, providing clients with enough time to migrate to new endpoints.

Include deprecation notices in your API responses, such as:

Deprecation: true
Deprecation-Date: "2024-12-31"
Link: "/new-endpoint"

Clearly communicate the timeline for deprecations and provide alternative endpoints to ensure a smooth transition for clients.

Lessons Learnt: In a previous version of our API, we deprecated several endpoints without proper notice, causing major disruptions for clients. After establishing a clear deprecation strategy with sufficient lead time and documentation, we were able to ensure smoother transitions for future API versions.

90. Optimise API for Edge Computing Use Cases

Edge computing brings data processing closer to the source of data, reducing latency and bandwidth usage by offloading tasks from centralised cloud servers to edge devices. For APIs that interact with edge computing infrastructure, design the API to be lightweight and capable of operating under intermittent or constrained network conditions.

Minimise large payloads and avoid operations that require constant cloud connectivity. Use techniques such as local caching and data synchronisation to handle periods of network unavailability. Additionally, consider building APIs that can operate in both online and offline modes, synchronising data to the cloud only when the network connection is restored.

Lessons Learnt: In one of our IoT projects, we initially designed the API with a centralised cloud-first approach, which caused significant delays in data processing due to high latency and network outages at the edge. After refactoring to support edge computing, with local caching and synchronisation, we improved response times and system reliability, especially in regions with poor connectivity.

91. Enable API Management Tools for Scaling

API management platforms such as Azure API Management, Amazon API Gateway, and Kong provide a centralised way to manage and scale APIs across multiple environments. These tools offer features like rate limiting, security enforcement, traffic monitoring, and analytics, helping you manage APIs at scale while maintaining reliability.

API management tools also provide versioning support, allowing for seamless transitions between different versions of the API. With built-in traffic shaping, these platforms can distribute load efficiently and handle spikes in usage without causing downtime.

Lessons Learnt: Initially, we scaled our API manually, leading to several operational bottlenecks and a lack of visibility into performance. After adopting API management tools, we were able to automate scaling, implement comprehensive rate limiting, and gain better insights into usage patterns, leading to improved service availability and user experience.

92. Support Webhooks for Event-Driven Applications

APIs designed for event-driven architectures can benefit from supporting webhooks. Webhooks allow your API to notify external systems when specific events occur, such as a completed transaction, an updated resource, or a new user registration. Clients register webhook URLs with your API, and when the event occurs, your system sends an HTTP POST request with the event data to the client’s URL.

Implement retries with exponential backoff for failed webhook deliveries, ensuring reliability even when the client’s system is temporarily unavailable. To secure webhook notifications, use techniques like signing payloads and validating incoming requests with tokens.

Lessons Learnt: In an earlier version of our API, we relied solely on polling for event updates, which caused unnecessary load on the server and delays for clients. After integrating webhooks, we significantly reduced traffic and provided real-time notifications to clients, improving the responsiveness and efficiency of the API.

93. Employ Graceful Error Handling for API Failures

Graceful error handling ensures that clients receive clear, actionable feedback when something goes wrong. Use appropriate HTTP status codes to indicate the nature of the error:

• 4xx for client-side issues (e.g., 400 Bad Request, 401 Unauthorized)
• 5xx for server-side failures (e.g., 500 Internal Server Error, 503 Service Unavailable)

In addition to status codes, provide meaningful error messages that describe the problem and how it can be resolved. Include detailed information in the response body, such as error codes, descriptions, and documentation links:

{
  "error_code": "INVALID_INPUT",
  "message": "The 'email' field is required."
}

Implement fallback mechanisms and retries where appropriate to handle temporary failures gracefully.

Lessons Learnt: Initially, we returned vague error messages without sufficient detail, which confused our clients and led to increased support requests. By improving error messages with detailed descriptions and documentation links, we reduced support queries by 30% and improved client satisfaction.

94. Ensure JSON-LD or HAL for Enhanced API Discoverability

APIs designed for hypermedia use cases should use standards like JSON-LD (Linked Data) or HAL (Hypertext Application Language) to enhance discoverability. These formats allow APIs to include metadata that describes how resources are connected, guiding clients on how to interact with the API.

For example, JSON-LD adds semantic meaning to the data by embedding links that clients can follow to navigate related resources:

{
  "@context": "http://schema.org",
  "@type": "Person",
  "name": "John Doe",
  "knows": {
    "@type": "Person",
    "name": "Jane Smith"
  }
}

This approach is particularly useful for APIs that expose complex, interconnected datasets, such as those used in e-commerce, social networks, or knowledge graphs.

Lessons Learnt: We initially used plain JSON responses without hypermedia links, which limited the API’s discoverability. After adopting JSON-LD, clients were able to navigate the API more intuitively, reducing the learning curve and enhancing the flexibility of client integrations.

95. Leverage GraphQL for Aggregating Multiple Resources

GraphQL allows clients to fetch multiple related resources in a single query, reducing the number of API calls needed to retrieve complex data. Unlike REST, where you might need to query several endpoints to get user details and their associated orders, GraphQL lets you query all this data at once.

A sample GraphQL query:

{
  user(id: "123") {
    name
    orders {
      id
      total
    }
  }
}

This flexibility can significantly reduce the amount of network traffic and improve performance for clients, especially in mobile applications where reducing latency and bandwidth usage is critical.

Lessons Learnt: In a previous REST-based API, our clients had to make multiple calls to gather related data, which increased latency and complexity. After migrating specific use cases to GraphQL, we saw a 40% reduction in API calls, which improved overall performance and client satisfaction.

96. Ensure Data Locality Considerations for Global APIs

For global APIs that serve users across multiple regions, ensure data locality by deploying API services in geographically distributed data centres. This reduces latency and improves performance for users who are far from your main servers. Use Content Delivery Networks (CDNs) to cache static resources and implement regional API instances for dynamic content.

API requests should be routed to the nearest available server, either through DNS-based routing or via API gateways that handle global traffic distribution. This approach ensures that data is processed closer to where it’s consumed, reducing delays and improving the user experience.

Lessons Learnt: Initially, our API was deployed in a single region, which resulted in poor performance for users in distant locations. By implementing region-specific deployments and leveraging CDNs, we improved response times by up to 60% for users in Europe, Asia, and other distant regions.

97. Support OAuth 2.1 for Secure API Access

OAuth 2.1 is the latest iteration of the OAuth framework, streamlining security features and addressing vulnerabilities found in earlier versions. Implement OAuth 2.1 for secure, token-based access to your API. It provides enhanced security features such as mandatory PKCE (Proof Key for Code Exchange), which prevents certain types of attacks during the authorisation flow.

OAuth 2.1 tokens are short-lived, reducing the risk of abuse, and refresh tokens can be used to renew access without re-authenticating. This ensures that user sessions remain secure while maintaining a smooth user experience.

Ensure that sensitive operations, such as account modifications or payment actions, are protected with scopes, allowing the API to restrict access to specific actions.

Lessons Learnt: We initially used OAuth 2.0 but did not enforce PKCE, which left our API vulnerable to attacks like authorisation code interception. After migrating to OAuth 2.1 with PKCE and better token lifecycle management, we significantly improved the security of our API and reduced the attack surface.

98. Implement Rate Limiting Based on Client Usage

Rate limiting should be flexible and tailored to the needs of different clients. Enterprise clients might require higher limits due to their usage patterns, while free-tier users should be subject to stricter limits to protect against abuse.

Design rate limiting policies that adapt to client-specific usage. For example, you might allow:

• 1000 requests/minute for enterprise clients
• 500 requests/minute for premium users
• 100 requests/minute for free users

Return appropriate HTTP headers to inform clients of their current usage and how long until their rate limit resets:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 100
X-RateLimit-Reset: 1372700873

Lessons Learnt: Initially, we implemented a one-size-fits-all rate limiting strategy, which led to poor performance for enterprise clients and under-utilisation for free users. After adopting client-based rate limiting, we improved fairness and performance across different user tiers.

99. Design APIs to be Cloud-Native and Scalable (continued)

Initially, we hosted our API on traditional servers, which required manual scaling during high-traffic periods. This led to delays in resource provisioning and service degradation during peak usage. After migrating to a cloud-native architecture with Kubernetes and auto-scaling, we achieved seamless scaling based on traffic demand, improving system availability and reducing operational overhead.

By leveraging cloud-native principles, such as microservices and event-driven architectures, your API can dynamically adjust resources, ensuring high availability and performance regardless of traffic patterns.

100. Ensure API Contracts Are Strictly Enforced

An API contract defines the expectations between the API provider and its consumers. This includes request parameters, response formats, and error codes. Once published, ensure that the contract is strictly enforced to avoid breaking changes that could disrupt client applications.

To maintain consistency, use tools such as Swagger or OpenAPI to document the API contract and automatically validate requests and responses against it. Implement contract testing to verify that API changes don’t violate the contract. Additionally, when evolving the API, ensure backward compatibility by versioning your API, allowing clients to upgrade at their own pace.

Lessons Learnt: In one case, we introduced a minor update to our API without validating it against the original contract, leading to unexpected errors for clients. By enforcing strict contract testing and validation, we ensured that future changes adhered to the API contract, preventing breaking changes and reducing disruptions for consumers.

101. Use Eventual Consistency in Distributed APIs

In distributed systems, achieving strong consistency across all services can be challenging and costly. Instead, design APIs with eventual consistency in mind. This means that while data may not be immediately consistent across all nodes, it will eventually become consistent.

For example, if an API performs a write operation in one region, it might take a short time for the changes to propagate to other regions. During this window, clients might see different versions of the data depending on which node they are connected to. While eventual consistency is more efficient, clients should be aware of the potential delay and design their applications accordingly.

Lessons Learnt: We initially aimed for strict consistency across our globally distributed services, which led to significant performance bottlenecks and latency issues. After switching to eventual consistency for non-critical operations, we reduced latency and improved throughput while maintaining acceptable levels of data accuracy.

102. Implement API Gateways for Rate Limiting and Security

An API gateway serves as a central entry point for all API traffic, providing key features like rate limiting, security, and logging. It helps abstract away the complexity of managing multiple microservices while centralising critical features like authentication, request validation, and load balancing.

Rate limiting at the gateway level ensures that all incoming traffic is subject to the same rules, preventing abuse and protecting backend services. API gateways like Kong, Nginx, or AWS API Gateway also provide enhanced security features such as IP whitelisting, request filtering, and SSL termination.

Lessons Learnt: Initially, we implemented rate limiting and security features within individual services, which led to inconsistencies and management overhead. After integrating an API gateway, we were able to enforce uniform policies across all services, simplifying management and improving security and scalability.

103. Consider Backward Compatibility When Evolving APIs

As your API evolves, ensure backward compatibility to avoid breaking client integrations. Always provide ample deprecation warnings when planning to remove or modify functionality. Use versioning to introduce new features or changes without disrupting existing clients.

For example, use semantic versioning (e.g., /v1, /v2) to signal major changes, and maintain older versions for a sufficient transition period. Communicate clearly with your clients about the deprecation timeline and provide migration guides to help them transition to new versions.

Lessons Learnt: We once made a breaking change to our API without versioning it, which caused major disruptions for clients still relying on the previous functionality. After implementing proper versioning and deprecation strategies, we reduced client disruptions and improved trust with our user base.