Establish the needs of the business to be resolved by software. This involves gathering the requirements and specifying the goals of the software to be met [Wikipedia].

Next to the business requirements, this document contains the minimal standards for software, including software license and source code management.

Architecture and design are where the Art of Building Software comes in; the architecture and design are the foundation of the software. This document will guide towards selecting the architecture and creating the design. There are some specific don’ts and do’s; however, it is up to the architect to create the optimal design by considering all business requirements [Wikipedia].

How much effort is required for another developer of comparable experience to pick up where the previous developer left off to fix, enhance or build upon the source code - without involving the former developer and considering the lifetime, quality, security, and the business impact of the application.

The proof of the pudding is in the eating; a good cook will sample the dish before serving the guests. The software builds upon requirements, and it is up to the software engineers to indicate the proven level of adherence to these requirements, preferably in an automated manner. It is up to the business to decide on the level of awareness regarding the software quality based on testing.

When deploying the latest version of the software, there are two continuously ongoing tasks. Firstly: bug fixing, updates, and new features; require the software to be flexible, adaptable, and logically structured. Note there is an overlap with software development: clean quality code will require a lower effort to adjust or fix instead of ill-structured low-quality code.

Secondly: the software’s level of availability and performance; requires the software to be monitorable and report its well-being. There is overlap with software testing: checking if the software is adhering to requirements can also be reused in the live environment to check if the software is operational and responding within a predefined timeframe.

Digital Solutions will provide a preferred code style configuration per language or application that is retrievable from this document’s repository.

Code styles are approved and maintained by the software development guild.

An additional "S" to the rule number and green left margin indicates a Pon standard.

Contracts relating to software development must indicate that the signing party has knowledge of the guidelines and agrees to comply with the standards as shown in this document.

All resources, APIs, documentation, comments, etc., must be written in the U.S. English language.

This section describes the guilds WOW regarding communication

All systems must have predifined KPIs on which the monitoring is to be based. These KPIs are the foundation for the monitoring of the infrastructure components.

For example: a website should have KPIs indicating response time and uptime. A low response time will require a fast scaling infrastructure and monitoring will have to be setup accordingly. A high uptime will require fast responding infrastructure monitoring.

All systems must have capabilities to allow itself to be monitored in an automated way and respond unambigous about the current operational state.

All systems should be able to report its operational state with enough detail for operational and support teams to take preventive measures. Basic reporting of "up" and "down" is not sufficient.

For example: system response time should be reported in milliseconds instead of "oke" and "not oke", allowing the business to set guidelines for the operational teams when to respond and take measures before the response time degrades to a level it will impact the effectiveness of its users.

In order to maintain software or integration quality the setup of an DTAP environment is preferred. This setup allows for an atomized way of working on features and bug fixes. These environments should be available before starting development.

This environment consists of four parts:

The quality of the code should be known, but we diffrentiate between frontends and api’s. GUI are more difficult to test, because you’ll need to automate the click flow through the frontend/GUI and therefor more labour intensive to maintain.

All tests and checks should be configured in the pipeline so the developer receives instant feedback of the commit.

API’s need to have a certain quality level which can be observed.

All systems must be compliant with the Pon Security Policy and Principles

All systems must be compliant with Binding Corportate Rules and local privacy legislation such as GDPR or CCPA.

Must have performed a Security & Privacy intake. When the requirements for the new system are clear, the Pon Security & Privacy by Design Process must be triggered via completing the Security & Privacy by Design Intake Form. The intake process is there to help you create APIs that are secure and privacy compliant and friendly. Your Group Security and Privacy manager will contact you to discuss the intake.

Must have the agreed security and privacy measures approved by Security and/or Privacy Manager before going live

Every tool, program or integration must include a comment, either in a file or in the source briefly indicating the purpose.

If the comment is included in the source, it is preferred to be included in the file containing the main function

Every tool, program or integration must include a comment, either in a file or in the source inidicating how the software operational state can be checked or monitored by automated systems.

Deployment of the tool, program or integration must be documented. This documentation describes the deployment procedures and required resources. Enabling an engineer to deploy without any support from the developers.

The onboarding procedure for new developers must include familiarization with these guidelines.

There should be pre-defined development environments, including OS and IDE. These environments will ensure a smooth onboarding and will make teams of developers more efficient.

The Pon guilds supply a repository of approved tools, techniques and frameworks both to inspire and guide teams. It is essential this repository is consulted when designing a solution architecture.

The source code of each project must include a Pon approved license file.

/** * Copyright © Pon Holding - All Rights Reserved */

All applicable source files must contain a Pon approved copyright notice at the start of the file and a reference to the project license file.

Example:

/** * Copyright © Pon Holding - All Rights Reserved * Unauthorized copying of this file, via any medium is strictly prohibited * Proprietary and confidential * * This file is subject to the terms and conditions defined in * file 'LICENSE.txt', which is part of this source code package. * * Written by Pon Employee <pon.employee@pon.com>, September 2020 */

Comparing SOA web service interfacing style of SOAP vs. REST, the former tend to be centered around operations that are usually use-case specific and specialized. In contrast, REST is centered around business (data) entities exposed as resources that are identified via URIs and can be manipulated via standardized CRUD-like methods using different representations, and hypermedia. RESTful APIs tend to be less use-case specific and comes with less rigid client / server coupling and are more suitable for an ecosystem of (core) services providing a platform of APIs to build diverse new business services. We apply the RESTful web service principles to all kind of application (micro-) service components, independently from whether they provide functionality via the internet or intranet.

An important principle for API design and usage is Postel’s Law, aka The Robustness Principle (see also RFC 1122):

Readings: Some interesting reads on the RESTful API design style and service architecture:

The design of our APIs should be based on the API as a Product principle:

Embracing 'API as a Product' facilitates a service ecosystem which can be evolved more easily, and used to experiment quickly with new business ideas by recombining core capabilities. It makes the difference between agile, innovative product service business built on a platform of APIs and ordinary enterprise integration business where APIs are provided as "appendix" of existing products to support system integration and optimised for local server-side realization.

Understand the concrete use cases of your customers and carefully check the trade-offs of your API design variants with a product mindset. Avoid short-term implementation optimizations at the expense of unnecessary client side obligations, and have a high attention on API quality and client developer experience.

API as a Product is closely related to our API First principle (see next chapter) which is more focused on how we engineer high quality APIs.

API First is one of our engineering and architecture principles. In a nutshell API First requires two aspects:

By defining APIs outside the code, we want to facilitate early review feedback and also a development discipline that focus service interface design on…​

Moreover, API definitions with standardized specification format also facilitate…​

Elements of API First are also this API Guidelines and a standardized API review process as to get early review feedback from peers and client developers. Peer review is important for us to get high quality APIs, to enable architectural and design alignment and to supported development of client applications decoupled from service provider engineering life cycle.

It is important to learn, that API First is not in conflict with the agile development principles that we love. Service applications should evolve incrementally — and so its APIs. Of course, our API specification will and should evolve iteratively in different cycles; however, each starting with draft status and early team and peer review feedback. API may change and profit from implementation concerns and automated testing feedback. API evolution during development life cycle may include breaking changes for not yet productive features and as long as we have aligned the changes with the clients. Hence, API First does not mean that you must have 100% domain and requirement understanding and can never produce code before you have defined the complete API and get it confirmed by peer review. On the other hand, API First obviously is in conflict with the bad practice of publishing API definition and asking for peer review after the service integration or even the service productive operation has started. It is crucial to request and get early feedback — as early as possible, but not before the API changes are comprehensive with focus to the next evolution step and have a certain quality (including API Guideline compliance), already confirmed via team internal reviews.

You must follow the API First Principle, more specifically:

Must provide API specification according to standards as specified for API platform.

The API specification files should be subject to version control using a source code management system - best together with the implementing sources.

You must / should publish the component external / internal API specification with the deployment of the implementing service, and, hence, make it discoverable.

Normally, API specification files must be self-contained, i.e. files should not contain references to local or remote content, e.g. ../fragment.yaml#/element or $ref: 'https://github.com/zalando/zally/blob/master/server/src/main/resources/api/zally-api.yaml#/schemas/LintingRequest'.

In addition to the API Specification, it is good practice to provide an API user manual to improve client developer experience, especially of engineers that are less experienced in using this API. A helpful API user manual typically describes the following API aspects:

The user manual must be published online, e.g. via our documentation hosting platform service, GHE pages, or specific team web servers. Please do not forget to include a link to the API user manual into the API specification using the #/externalDocs/url property.

API specifications must contain the following meta information to allow for API management:

Open API allows to specify the API specification version in #/info/version. To share a common semantic of version information we expect API designers to comply to Semantic Versioning 2.0 rules 1 to 8 and 11 restricted to the format <MAJOR>.<MINOR>.<PATCH> for versions as follows:

Additional Notes:

Example:

Each API specification may be provisioned with a globally unique and immutable API identifier.

API specifications will evolve and any aspect of an API specification may change. We require API identifiers because we want to support API clients and providers with API lifecycle management features, like change trackability and history or automated backward compatibility checks. The immutable API identifier allows the identification of all API specification versions of an API evolution. By using API semantic version information or API publishing date as order criteria you get the version or publication history as a sequence of API specifications.

Note: While it is nice to use human readable API identifiers based on self-managed URNs, it is recommend to stick to UUIDs to relief API designers from any urge of changing the API identifier while evolving the API. Example:

Each API must be classified with respect to the intended target audience supposed to consume the API, to facilitate differentiated standards on APIs for discoverability, changeability, quality of design and documentation, as well as permission granting. We differentiate the following API audience groups with clear organisational and legal boundaries:

Note: a smaller audience group is intentionally included in the wider group and thus does not need to be declared additionally.

The API audience is provided as API meta information in the info-block of the Open API specification and must conform to the following specification:

Note: Exactly one audience per API specification is allowed. For this reason a smaller audience group is intentionally included in the wider group and thus does not need to be declared additionally. If parts of your API have a different target audience, we recommend to split API specifications along the target audience.

Example:

For details and more information on audience groups see the Pon internal documentation

TODO: add link to internal documentation regarding API audience issue 2

Every API endpoint should be secured, also for anonymous access. The preferred authentication method is OAuth 2.0. For anonymous access the client credentials grant is preferred.

The following code snippet shows how to define the authorization scheme using a bearer token (e.g. JWT token).

The next code snippet applies this security scheme to all API endpoints. The bearer token of the client must have additionally the scopes "scope_1" and "scope_2".

APIs should define permissions to protect their resources. Thus, at least one permission must be assigned to each endpoint. Permissions are defined as shown in the previous section.

The naming schema for permissions corresponds to the naming schema for hostnames and event type names. Please refer to (RFP) MAY follow naming convention for permissions (scopes) for designing permission names.

APIs should stick to component specific permissions without resource extension to avoid governance complexity of too many fine grained permissions. For the majority of use cases, restricting access to specific API endpoints using read and write is sufficient for controlling access for client types like merchant or retailer business partners, customers or operational staff. However, in some situations, where the API serves different types of resources for different owners, resource specific scopes may make sense.

Some examples for standard and resource-specific permissions:

After permission names are defined and the permission is declared in the security definition at the top of an API specification, it should be assigned to each API operation by specifying a security requirement like this:

In very rare cases a whole API or some selected endpoints may not require specific access control. However, to make this explicit you should assign the uid pseudo permission in this case. It is the user id and always available as OAuth2 default scope.

Hint: you need not explicitly define the "Authorization" header; it is a standard header so to say implicitly defined via the security section.

As long as the functional naming is not supported for permissions, permission names in APIs must conform to the following naming pattern:

This pattern is compatible with the previous definition.

Change APIs, but keep all consumers running. Consumers usually have independent release lifecycles, focus on stability, and avoid changes that do not provide additional value. APIs are contracts between service providers and service consumers that cannot be broken via unilateral decisions.

There are two techniques to change APIs without breaking them:

We strongly encourage using compatible API extensions and discourage versioning (see (RFP) SHOULD avoid versioning and {STATUS-TODO} MUST API Versioning Has No “Right Way” below). The following guidelines for service providers ((RFP) SHOULD prefer compatible extensions) and consumers ((RFP) MUST prepare clients accept compatible API extensions) enable us (having Postel’s Law in mind) to make compatible changes without versioning.

Note: There is a difference between incompatible and breaking changes. Incompatible changes are changes that are not covered by the compatibility rules below. Breaking changes are incompatible changes deployed into operation, and thereby breaking running API consumers. Usually, incompatible changes are breaking changes when deployed into operation. However, in specific controlled situations it is possible to deploy incompatible changes in a non-breaking way, if no API consumer is using the affected API aspects (see also Deprecation guidelines).

Hint: Please note that the compatibility guarantees are for the "on the wire" format. Binary or source compatibility of code generated from an API definition is not covered by these rules. If client implementations update their generation process to a new version of the API definition, it has to be expected that code changes are necessary.

API designers may apply the following rules to evolve APIs for services in a backward-compatible way:

Service clients should apply the robustness principle:

Service clients must be prepared for compatible API extensions of service providers:

Designers of service provider APIs should be conservative and accurate in what they accept from clients:

Not ignoring unknown input fields is a specific deviation from Postel’s Law (e.g. see also
The Robustness Principle Reconsidered) and a strong recommendation. Servers might want to take different approach but should be aware of the following problems and be explicit in what is supported:

In specific situations, where a (known) input field is not needed anymore, it either can stay in the API definition with "not used anymore" description or can be removed from the API definition as long as the server ignores this specific parameter.

In a JSON response body, you must always return a JSON object (and not e.g. an array) as a top level data structure to support future extensibility. JSON objects support compatible extension by additional attributes. This allows you to easily extend your response and e.g. add pagination later, without breaking backwards compatibility. See SHOULD use pagination links where applicable for an example.

Maps (see SHOULD define maps using additionalProperties), even though technically objects, are also forbidden as top level data structures, since they don’t support compatible, future extensions.

{TODO} Add example

Enumerations are per definition closed sets of values, that are assumed to be complete and not intended for extension. This closed principle of enumerations imposes compatibility issues when an enumeration must be extended. To avoid these issues, we strongly recommend to use an open-ended list of values instead of an enumeration unless:

To specify an open-ended list of values use the marker x-extensible-enum as follows:

Note: x-extensible-enum is not JSON Schema conform but will be ignored by most tools.

See MUST declare enum values using UPPER_SNAKE_CASE format about enum value naming conventions.

When changing your APIs, do so in a compatible way and avoid generating additional API versions unless the API is non-functional or is degraded. Multiple versions can significantly complicate understanding, testing, maintaining, evolving, operating and releasing our systems (supplementary reading).

If changing an API can’t be done in a compatible way, then proceed in one of these three ways:

As we discourage versioning by all means because of the manifold disadvantages, we strongly recommend to only use the first two approaches.

API Versioning Has No "Right Way" provides an overview on different versioning approaches to handle breaking changes without being opinionated.

With URI versioning a (major) version number is included in the path, e.g. /v1/customers. The consumer has to wait until the provider has been released and deployed.

Before shutting down an API, version of an API, or API feature the producer must make sure, that all clients have given their consent on a sunset date. Producers should help consumers to migrate to a potential new API or API feature by providing a migration manual and clearly state the time line for replacement availability and sunset (see also (RFP) SHOULD add Deprecation and Sunset header to responses). Once all clients of a sunset API feature are migrated, the producer may shut down the deprecated API feature.

If the API is consumed by any external partner, the API owner must define a reasonable time span that the API will be maintained after the producer has announced deprecation. All external partners must state consent with this after-deprecation-life-span, i.e. the minimum time span between official deprecation and first possible sunset, before they are allowed to use the API.

The API deprecation must be part of the API specification.

If an API endpoint (operation object), an input argument (parameter object), an in/out data object (schema object), or on a more fine grained level, a schema attribute or property should be deprecated, the producers must set deprecated: true for the affected element and add further explanation to the description section of the API specification. If a future shut down is planned, the producer must provide a sunset date and document in details what consumers should use instead and how to migrate.

Owners of an API, API version, or API feature used in production that is scheduled for sunset must monitor the usage of the sunset API, API version, or API feature in order to observe migration progress and avoid uncontrolled breaking effects on ongoing consumers. See also SHOULD monitor API usage.

During the deprecation phase, the producer should add a Deprecation: <date-time> (see draft: RFC Deprecation HTTP Header) and - if also planned - a Sunset: <date-time> (see RFC 8594) header on each response affected by a deprecated element (see (RFP) MUST reflect deprecation in API specifications).

The Deprecation header can either be set to true - if a feature is retired -, or carry a deprecation time stamp, at which a replacement will become/became available and consumers must not on-board any longer (see (RFP) MUST not start using deprecated APIs). The optional Sunset time stamp carries the information when consumers latest have to stop using a feature. The sunset date should always offer an eligible time interval for switching to a replacement feature.

If multiple elements are deprecated the Deprecation and Sunset headers are expected to be set to the earliest time stamp to reflect the shortest interval consumers are expected to get active.

Note: adding the Deprecation and Sunset header is not sufficient to gain client consent to shut down an API or feature.

Hint: In earlier guideline versions, we used the Warning header to provide the deprecation info to clients. However, Warning header has a less specific semantics, will be obsolete with draft: RFC HTTP Caching, and our syntax was not compliant with RFC 7234 — Warning header.

Clients should monitor the Deprecation and Sunset headers in HTTP responses to get information about future sunset of APIs and API features (see (RFP) SHOULD add Deprecation and Sunset header to responses). We recommend that client owners build alerts on this monitoring information to ensure alignment with service owners on required migration task.

Hint: In earlier guideline versions, we used the Warning header to provide the deprecation info (see hint in (RFP) SHOULD add Deprecation and Sunset header to responses).

Clients must not start using deprecated APIs, API versions, or API features.

Use the following common money structure:

APIs are encouraged to include a reference to the global schema for Money.

Please note that APIs have to treat Money as a closed data type, i.e. it’s not meant to be used in an inheritance hierarchy. That means the following usage is not allowed:

There exist a variety of field types that are required in multiple places. To achieve consistency across all API implementations, you must use common field names and semantics whenever applicable.

Functional naming is a powerful, yet easy way to align global resources as host, permission, and event names within an the application landscape. It helps to preserve uniqueness of names while giving readers meaningful context information about the addressed component. Besides, the most important aspect is, that it allows to keep APIs stable in the case of technical and organizational changes (Pon for example maintains an internal naming convention).

To make use of this advantages for APIs with a larger audience we strongly recommended to follow the functional naming schema for hostnames, permission names, and event names in APIs as follows:

To conduct the functional naming schema, a unique functional-name is assigned to each functional component. It is built of the domain name of the functional group the component is belonging to and a unique a short identifier for the functional component itself:

Internal Hint: Use the simple functional name registry (internal link) to register your functional name before using it. The registry is a centralized infrastructure service to ensure uniqueness of your functional names (and available domains) and to support hostname DNS resolution.

Please see the following rules for detailed functional naming patterns:

Hostnames in APIs must, respectively should conform to the functional naming depending on the audience as follows (see MUST/SHOULD use functional naming schema for details and <functional-name> definition):

The following application specific legacy convention is only allowed for hostnames of component-internal APIs:

Example:

This applies to concrete path segments and not the names of path parameters. For example {shipment_order_id} would be ok as a path parameter.

Examples:

This is for consistency in your documentation (most other headers follow this convention). Avoid camelCase (without hyphens). Exceptions are common abbreviations like "ID."

Examples:

See also: HTTP Headers are case-insensitive (RFC 7230).

See Common headers and Proprietary headers sections for more guidance on HTTP headers.

Usually, a collection of resource instances is provided (at least API should be ready here). The special case of a resource singleton is a collection with cardinality 1.

In most cases, all resources provided by a service are part of the public API, and therefore should be made available under the root "/" base path.

If the service should also support non-public, internal APIs — for specific operational support functions, for example — we encourage you to maintain two different API specifications and provide API audience. For both APIs, you should not use /api as base path.

We see API’s base path as a part of deployment variant configuration. Therefore, this information has to be declared in the server object.

The trailing slash must not have specific semantics. Resource paths must deliver the same results whether they have the trailing slash or not.

If you provide query support for searching, sorting, filtering, and paginating, you must stick to the following naming conventions:

REST is all about your resources, so consider the domain entities that take part in web service interaction, and aim to model your API around these using the standard HTTP methods as operation indicators. For instance, if an application has to lock articles explicitly so that only one user may edit them, create an article lock with PUT or POST instead of using a lock action.

Request:

The added benefit is that you already have a service for browsing and filtering article locks.

An API should contain the complete business processes containing all resources representing the process. This enables clients to understand the business process, foster a consistent design of the business process, allow for synergies from description and implementation perspective, and eliminates implicit invisible dependencies between APIs.

In addition, it prevents services from being designed as thin wrappers around databases, which normally tends to shift business logic to the clients.

As a rule of thumb resources should be defined to cover 90% of all its client’s use cases. A useful resource should contain as much information as necessary, but as little as possible. A great way to support the last 10% is to allow clients to specify their needs for more/less information by supporting filtering and embedding.

The API describes resources, so the only place where actions should appear is in the HTTP methods. In URLs, use only nouns. Instead of thinking of actions (verbs), it’s often helpful to think about putting a message in a letter box: e.g., instead of having the verb cancel in the url, think of sending a message to cancel an order to the cancellations letter box on the server side.

API resources represent elements of the application’s domain model. Using domain-specific nomenclature for resource names helps developers to understand the functionality and basic semantics of your resources. It also reduces the need for further documentation outside the API definition. For example, "sales-order-items" is superior to "order-items" in that it clearly indicates which business object it represents. Along these lines, "items" is too general.

To simplify encoding of resource IDs in URLs, their representation must only consist of ASCII strings using letters, numbers, underscore, minus, colon, period, and - on rare occasions - slash. The corresponding regular expression is

[a-zA-Z0-9:._\-/]*

Note: slashes are only allowed to build and signal resource identifiers consisting of compound keys.

Some API resources may contain or reference sub-resources. Embedded sub-resources, which are not top-level resources, are parts of a higher-level resource and cannot be used outside of its scope. Sub-resources should be referenced by their name and identifier in the path segments as follows:

In order to improve the consumer experience, you should aim for intuitively understandable URLs, where each sub-path is a valid reference to a resource or a set of resources. E.g. if /customers/12ev123bv12v/addresses/DE_100100101 is valid, then /customers/12ev123bv12v/addresses, /customers/12ev123bv12v and /customers must be valid as well in principle. E.g.:

Note: resource identifiers may be build of multiple other resource identifiers (see MAY expose compound keys as resource identifiers).

If a resource is best identified by a compound key consisting of multiple other resource identifiers, it is allowed to reuse the compound key in its natural form containing slashes instead of technical resource identifier in the resource path without violating the above rule MUST identify resources and sub-resources via path segments as follows:

Example paths:

Warning: Exposing a compound key as described above limits ability to evolve the structure of the resource identifier as it is no longer opaque.

To compensate for this drawback, APIs must apply a compound key abstraction consistently in all requests and responses parameters and attributes allowing consumers to treat these as technical resource identifier replacement. The use of independent compound key components must be limited to search and creation requests, as follows:

Where id-1 is representing the opaque provision of the compound key sku-1/sid-1 as technical resource identifier.

Remark: A compound key component may itself be used as another resource identifier providing another resource endpoint, e.g /article-size-advices/{sku}.

If a sub-resource is only accessible via its parent resource and may not exist without parent resource, consider using a nested URL structure, for instance:

However, if the resource can be accessed directly via its unique id, then the API should expose it as a top-level resource. For example, customer has a collection for sales orders; however, sales orders have globally unique id and some services may choose to access the orders directly, for instance:

Generating IDs can be a scaling problem in high frequency and near real time use cases. UUIDs solve this problem, as they can be generated without collisions in a distributed, non-coordinated way and without additional server round trips.

However, they also come with some disadvantages:

UUIDs should be avoided when not needed for large scale id generation. Instead, for instance, server side support with id generation can be preferred (POST on id resource, followed by idempotent PUT on entity resource). Usage of UUIDs is especially discouraged as primary keys of master and configuration data, like brand-ids or attribute-ids which have low id volume but widespread steering functionality.

Please be aware that sequential, strictly monotonically increasing numeric identifiers may reveal critical, confidential business information, like order volume, to non-privileged clients.

In any case, we should always use string rather than number type for identifiers. This gives us more flexibility to evolve the identifier naming scheme. Accordingly, if used as identifiers, UUIDs should not be qualified using a format property.

Hint: Usually, random UUID is used - see UUID version 4 in RFC 4122. Though UUID version 1 also contains leading timestamps it is not reflected by its lexicographic sorting. This deficit is addressed by ULID (Universally Unique Lexicographically Sortable Identifier). You may favour ULID instead of UUID, for instance, for pagination use cases ordered along creation time.

To keep maintenance and service evolution manageable, we should follow "functional segmentation" and "separation of concern" design principles and do not mix different business functionalities in same API definition. In practice this means that the number of resource types exposed via an API should be limited. In this context a resource type is defined as a set of highly related resources such as a collection, its members and any direct sub-resources.

For example, the resources below would be counted as three resource types, one for customers, one for the addresses, and one for the customers' related addresses:

Note that:

Given this definition, our experience is that well defined APIs involve no more than 4 to 8 resource types. There may be exceptions with more complex business domains that require more resources, but you should first check if you can split them into separate subdomains with distinct APIs.

Nevertheless one API should hold all necessary resources to model complete business processes helping clients to understand these flows.

There are main resources (with root url paths) and sub-resources (or nested resources with non-root urls paths). Use sub-resources if their life cycle is (loosely) coupled to the main resource, i.e. the main resource works as collection resource of the subresource entities. You should use ⇐ 3 sub-resource (nesting) levels — more levels increase API complexity and url path length. (Remember, some popular web browsers do not support URLs of more than 2000 characters.)

APIs should support techniques for reducing bandwidth based on client needs. This holds for APIs that (might) have high payloads and/or are used in high-traffic scenarios like the public Internet and telecommunication networks. Typical examples are APIs used by mobile web app clients with (often) less bandwidth connectivity.

Common techniques include:

Each of these items is described in greater detail below.

Compress the payload of your API’s responses with gzip, unless there’s a good reason not to — for example, you are serving so many requests that the time to compress becomes a bottleneck. This helps to transport data faster over the network (fewer bytes) and makes frontends respond faster.

Though gzip compression might be the default choice for server payload, the server should also support payload without compression and its client control via Accept-Encoding request header — see also RFC 7231 Section 5.3.4. The server should indicate used gzip compression via the Content-Encoding header.

Depending on your use case and payload size, you can significantly reduce network bandwidth need by supporting filtering of returned entity fields. Here, the client can explicitly determine the subset of fields he wants to receive via the fields query parameter. (It is analogue to GraphQL fields and simple queries, and also applied, for instance, for Google Cloud API’s partial responses.)

Embedding related resources (also know as Resource expansion) is a great way to reduce the number of requests. In cases where clients know upfront that they need some related resources they can instruct the server to prefetch that data eagerly. Whether this is optimized on the server, e.g. a database join, or done in a generic way, e.g. an HTTP proxy that transparently embeds resources, is up to the implementation.

See MUST stick to conventional query parameters for naming, e.g. "embed" for steering of embedded resource expansion. Please use the BNF grammar, as already defined above for filtering, when it comes to an embedding query syntax.

Embedding a sub-resource can possibly look like this where an order resource has its order items as sub-resource (/order/{orderId}/items):

Caching has to take many aspects into account, e.g. general cacheability of response information, our guideline to protect endpoints using SSL and OAuth authorization, resource update and invalidation rules, existence of multiple consumer instances. As a consequence, caching is in best case complex, e.g. with respect to consistency, in worst case inefficient.

As a consequence, client side as well as transparent web caching should be avoided, unless the service supports and requires it to protect itself, e.g. in case of a heavily used and therefore rate limited master data service, i.e. data items that rarely or not at all change after creation.

As default, API providers and consumers should always set the Cache-Control header set to Cache-Control: no-store and assume the same setting, if no Cache-Control header is provided.

Note: There is no need to document this default setting. However, please make sure that your framework is attaching this header value by default, or ensure this manually, e.g. using the best practice of Spring Security as shown below. Any setup deviating from this default must be sufficiently documented.

If your service really requires to support caching, please observe the following rules:

Hint: For proper cache support, you must return 304 without content on a failed HEAD or GET request with If-None-Match: <entity-tag> instead of 412.

Hint: For ETag source see MAY consider to support ETag together with If-Match/If-None-Match header.

The default setting for Cache-Control should contain the private directive for endpoints with standard OAuth authorization, as well as the must-revalidate directive to ensure, that the client does not use stale cache entries. Last, the max-age directive should be set to a value between a few seconds (max-age=60) and a few hours (max-age=86400) depending on the change rate of your master data and your requirements to keep clients consistent.

The default setting for Vary is harder to determine correctly. It highly depends on the API endpoint, e.g. whether it supports compression, accepts different media types, or requires other request specific headers. To support correct caching you have to carefully choose the value. However, a good first default may be:

Anyhow, this is only relevant, if you encourage clients to install generic HTTP layer client and proxy caches.

Note: generic client and proxy caching on HTTP level is hard to configure. Therefore, we strongly recommend to attach the (possibly distributed) cache directly to the service (or gateway) layer of your application. This relieves from interpreting the Vary header and greatly simplifies interpreting the Cache-Control and ETag headers. Moreover, is highly efficient with respect to caching performance and overhead, and allows to support more advanced cache update and warm up patterns.

Anyhow, please carefully read RFC 7234 before adding any client or proxy cache.

We strive for a good implementation of REST Maturity Level 2 as it enables us to build resource-oriented APIs that make full use of HTTP verbs and status codes. You can see this expressed by many rules throughout these guidelines, e.g.:

Although this is not HATEOAS, it should not prevent you from designing proper link relationships in your APIs as stated in rules below.

We do not generally recommend to implement REST Maturity Level 3. HATEOAS comes with additional API complexity without real value in our SOA context where client and server interact via REST APIs and provide complex business functions as part of our e-commerce SaaS platform.

Our major concerns regarding the promised advantages of HATEOAS (see also RESTistential Crisis over Hypermedia APIs, Why I Hate HATEOAS and others for a detailed discussion):

However, we do not forbid HATEOAS; you could use it, if you checked its limitations and still see clear value for your usage scenario that justifies its additional complexity. If you use HATEOAS please share experience and present your findings.

Links to other resource must always use full, absolute URI.

Motivation: Exposing any form of relative URI (no matter if the relative URI uses an absolute or relative path) introduces avoidable client side complexity. It also requires clarity on the base URI, which might not be given when using features like embedding subresources. The primary advantage of non-absolute URI is reduction of the payload size, which is better achievable by following the recommendation to use gzip compression

When embedding links to other resources into representations you must use the common hypertext control object. It contains at least one attribute:

In API that contain any hypertext controls, the attribute name href is reserved for usage within hypertext controls.

The schema for hypertext controls can be derived from this model:

The name of an attribute holding such a HttpLink object specifies the relation between the object that contains the link and the linked resource. Implementations should use names from the IANA Link Relation Registry whenever appropriate. As IANA link relation names use hyphen-case notation, while this guide enforces snake_case notation for attribute names, hyphens in IANA names have to be replaced with underscores (e.g. the IANA link relation type version-history would become the attribute version_history)

Specific link objects may extend the basic link type with additional attributes, to give additional information related to the linked resource or the relationship between the source resource and the linked one.

E.g. a service providing "Person" resources could model a person who is married with some other person with a hypertext control that contains attributes which describe the other person (id, name) but also the relationship "spouse" between the two persons (since):

Hypertext controls are allowed anywhere within a JSON model. While this specification would allow HAL, we actually don’t recommend/enforce the usage of HAL anymore as the structural separation of meta-data and data creates more harm than value to the understandability and usability of an API.

For pagination and self-references a simplified form of the extensible common hypertext controls should be used to reduce the specification and cognitive overhead. It consists of a simple URI value in combination with the corresponding link relations, e.g. next, prev, first, last, or self.

See simple-hypertext-control-fields and SHOULD use pagination links where applicable for examples and more information.

For flexibility and precision, we prefer links to be directly embedded in the JSON payload instead of being attached using the uncommon link header syntax. As a result, the use of the Link Header defined by RFC 8288 in conjunction with JSON media types is forbidden.

Content or entity headers are headers with a Content- prefix. They describe the content of the body of the message and they can be used in both, HTTP requests and responses. Commonly used content headers include but are not limited to:

Use this list and mention its support in your Open API definition.

The Content-Location header is optional and can be used in successful write operations (PUT, POST, or PATCH) or read operations (GET, HEAD) to guide caching and signal a receiver the actual location of the resource transmitted in the response body. This allows clients to identify the resource and to update their local copy when receiving a response with this header.

The Content-Location header can be used to support the following use cases:

Note: When using the Content-Location header, the Content-Type header has to be set as well. For example:

As the correct usage of Content-Location with respect to semantics and caching is difficult, we discourage the use of Content-Location. In most cases it is sufficient to direct clients to the resource location by using the Location header instead without hitting the Content-Location specific ambiguities and complexities.

More details in RFC 7231 7.1.2 Location, 3.1.4.2 Content-Location

The Prefer header defined in RFC 7240 allows clients to request processing behaviors from servers. It pre-defines a number of preferences and is extensible, to allow others to be defined. Support for the Prefer header is entirely optional and at the discretion of API designers, but as an existing Internet Standard, is recommended over defining proprietary "X-" headers for processing directives.

The Prefer header can defined like this in an API definition:

Note: Please copy only the behaviors into your Prefer header specification that are supported by your API endpoint. If necessary, specify different Prefer headers for each supported use case.

Supporting APIs may return the Preference-Applied header also defined in RFC 7240 to indicate whether a preference has been applied.

When creating or updating resources it may be necessary to expose conflicts and to prevent the 'lost update' or 'initially created' problem. Following RFC 7232 "HTTP: Conditional Requests" this can be best accomplished by supporting the ETag header together with the If-Match or If-None-Match conditional header. The contents of an ETag: <entity-tag> header is either (a) a hash of the response body, (b) a hash of the last modified field of the entity, or (c) a version number or identifier of the entity version.

To expose conflicts between concurrent update operations via PUT, POST, or PATCH, the If-Match: <entity-tag> header can be used to force the server to check whether the version of the updated entity is conforming to the requested <entity-tag>. If no matching entity is found, the operation is supposed a to respond with status code 412 - precondition failed.

Beside other use cases, If-None-Match: * can be used in a similar way to expose conflicts in resource creation. If any matching entity is found, the operation is supposed a to respond with status code 412 - precondition failed.

The ETag, If-Match, and If-None-Match headers can be defined as follows in the API definition:

Please see Optimistic locking in RESTful APIs for a detailed discussion and options.

When creating or updating resources it can be helpful or necessary to ensure a strong idempotent behavior comprising same responses, to prevent duplicate execution in case of retries after timeout and network outages. Generally, this can be achieved by sending a client specific unique request key – that is not part of the resource – via Idempotency-Key header.

The unique request key is stored temporarily, e.g. for 24 hours, together with the response and the request hash (optionally) of the first request in a key cache, regardless of whether it succeeded or failed. The service can now look up the unique request key in the key cache and serve the response from the key cache, instead of re-executing the request, to ensure idempotent behavior. Optionally, it can check the request hash for consistency before serving the response. If the key is not in the key store, the request is executed as usual and the response is stored in the key cache.

This allows clients to safely retry requests after timeouts, network outages, etc. while receive the same response multiple times. Note: The request retry in this context requires to send the exact same request, i.e. updates of the request that would change the result are off-limits. The request hash in the key cache can protection against this misbehavior. The service is recommended to reject such a request using status code 400.

Important: To grant a reliable idempotent execution semantic, the resource and the key cache have to be updated with hard transaction semantics – considering all potential pitfalls of failures, timeouts, and concurrent requests in a distributed systems. This makes a correct implementation exceeding the local context very hard.

The Idempotency-Key header must be defined as follows, but you are free to choose your expiration time:

Hint: The key cache is not intended as request log, and therefore should have a limited lifetime, else it could easily exceed the data resource in size.

Note: The Idempotency-Key header unlike other headers in this section is not standardized in an RFC. Our only reference are the usage in the Stripe API. However, as it fit not into our section about Proprietary headers, and we did not want to change the header name and semantic, we decided to treat it as any other common header.

As a general rule, proprietary HTTP headers should be avoided. Still they can be useful in cases where context needs to be passed through multiple services in an end-to-end fashion. As such, a valid use-case for a proprietary header is providing context information, which is not a part of the actual API, but is needed by subsequent communication.

From a conceptual point of view, the semantics and intent of an operation should always be expressed by URLs path and query parameters, the method, and the content. Headers are more often used to implement functions close to the protocol considerations, such as flow control, content negotiation, and authentication. Thus, headers are reserved for general context information (RFC 7231).

X- headers were initially reserved for unstandardized parameters, but the usage of X- headers is deprecated (RFC 6648). This complicates the contract definition between consumer and producer of an API following these guidelines, since there is no aligned way of using those headers. Because of this, the guidelines restrict which X- headers can be used and how they are used.

The Internet Engineering Task Force’s states in RFC 6648 that company specific header' names should incorporate the organization’s name. We aim for backward compatibility, and therefore keep the X- prefix.

The following proprietary headers have been specified by this guideline for usage so far. Remember that HTTP header field names are not case-sensitive.

Exception: The only exception to this guideline are the conventional hop-by-hop X-RateLimit- headers which can be used as defined in MUST use code 429 with headers for rate limits.

All Pon’s proprietary headers are end-to-end headers. ^[2]

All headers specified above must be propagated to the services down the call chain. The header names and values must remain unchanged.

For example, the values of the custom headers like X-Device-Type can affect the results of queries by using device type information to influence recommendation results. Besides, the values of the custom headers can influence the results of the queries (e.g. the device type information influences the recommendation results).

Sometimes the value of a proprietary header will be used as part of the entity in a subsequent request. In such cases, the proprietary headers must still be propagated as headers with the subsequent request, despite the duplication of information.

The Flow-ID is a generic parameter to be passed through service APIs and events and written into log files and traces. A consequent usage of the Flow-ID facilitates the tracking of call flows through our system and allows the correlation of service activities initiated by a specific call. This is extremely helpful for operational troubleshooting and log analysis. Main use case of Flow-ID is to track service calls of our SaaS fashion commerce platform and initiated internal processing flows (executed synchronously via APIs or asynchronously via published events).

All service applications must publish Open API specifications of their external APIs. While this is optional for internal APIs, i.e. APIs marked with the component-internal API audience group, we still recommend to do so to profit from the API management infrastructure.

An API is published by copying its Open API specification into the reserved /pon-apis directory of the deployment artifact used to deploy the provisioning service. The directory must only contain self-contained YAML files that each describe one API (exception see (RFP) MUST only use durable and immutable remote references). We prefer this deployment artifact-based method over the past (now legacy) .well-known/schema-discovery service endpoint-based publishing process, that we only support for backward compatibility reasons.

Background: In our dynamic and complex service infrastructure, it is important to provide API client developers a central place with online access to the API specifications of all running applications. As a part of the infrastructure, the API publishing process is used to detect API specifications. The findings are published in the API Portal - the universal hub for all Pon APIs.

Note: To publish an API, it is still necessary to deploy the artifact successful, as we focus the discovery experience on APIs supported by running services.

Owners of APIs used in production should monitor API service to get information about its using clients. This information, for instance, is useful to identify potential review partner for API changes.

Hint: A preferred way of client detection implementation is by logging of the client-id retrieved from the OAuth token.

Events are defined using an item called an Event Type. The Event Type allows events to have their structure declared with a schema by producers and understood by consumers. An Event Type declares standard information, such as a name, an owning application (and by implication, an owning team), a schema defining the event’s custom data, and a compatibility mode declaring how the schema will be evolved. Event Types also allow the declaration of validation and enrichment strategies for events, along with supplemental information such as how events can be partitioned in an event stream.

Event Types belong to a well known Event Category (such as a data change category), which provides extra information that is common to that kind of event.

Event Types can be published and made available as API resources for teams to use, typically in an Event Type Registry. Each event published can then be validated against the overall structure of its event type and the schema for its custom data.

The basic model described above was originally developed in the Nakadi project, which acts as a reference implementation of the event type registry, and as a validating publish/subscribe broker for event producers and consumers.

Events are part of a service’s interface to the outside world equivalent in standing to a service’s REST API. Services publishing data for integration must treat their events as a first class design concern, just as they would an API. For example this means approaching events with the "API first" principle in mind as described in the Introduction.

Services publishing event data for use by others must make the event schema as well as the event type definition available for review.

To align the event schema specifications to API specifications, we use the Schema Object as defined by the Open API Specifications to define event schemas. This is particularly useful for events that represent data changes about resources also used in other APIs.

The Open API Schema Object is an extended subset of JSON Schema Draft 4. For convenience, we highlight some important differences below. Please refer to the Open API Schema Object specification for details.

As the Open API Schema Object specification removes some JSON Schema keywords, the following properties must not be used in event schemas:

On the other side Schema Object redefines some JSON Schema keywords:

Finally, the Schema Object extends JSON Schema with some keywords:

In Pon’s architecture, events are registered using a structure called an Event Type. The Event Type declares standard information as follows:

Event Types allow easier discovery of event information and ensure that information is well-structured, consistent, and can be validated.

Event type owners must pay attention to the choice of compatibility mode. The mode provides a means to evolve the schema. The range of modes are designed to be flexible enough so that producers can evolve schemas while not inadvertently breaking existing consumers:

The compatibility mode interact with revision numbers in the schema version field, which follows semantic versioning (MAJOR.MINOR.PATCH):

The following examples illustrate this relations:

The core Event Type structure is shown below as an Open API object definition:

APIs such as registries supporting event types, may extend the model, including the set of supported categories and schema formats. For example the Nakadi API’s event category registration also allows the declaration of validation and enrichment strategies for events, along with supplemental information, such as how events are partitioned in the stream (see SHOULD use the hash partition strategy for data change events).

An event category describes a generic class of event types. The guidelines define two such categories:

The set of categories is expected to evolve in the future.

A category describes a predefined structure that event publishers must conform to along with standard information about that kind of event (such as the operation for a data change event).

Events are intended to be used by other services including business process/data analytics and monitoring. They should be based around the resources and business processes you have defined for your service domain and adhere to its natural lifecycle (see also SHOULD model complete business processes and SHOULD define useful resources).

As there is a cost in creating an explosion of event types and topics, prefer to define event types that are abstract/generic enough to be valuable for multiple use cases, and avoid publishing event types without a clear need.

Similar to API permission scopes, there will be event type permissions passed via an OAuth token supported in near future. In the meantime, teams are asked to note the following:

When publishing events that represent steps in a business process, event types must be based on the General Event category.

All your events of a single business process will conform to the following rules:

At the moment we cannot state whether it’s best practice to publish all the events for a business process using a single event type and represent the specific steps with a state field, or whether to use multiple event types to represent each step. For now we suggest assessing each option and sticking to one for a given business process.

When publishing events that represents created, updated, or deleted data, change event types must be based on the Data Change Event category.

Some common error cases may require event consumers to reconstruct event streams or replay events from a position within the stream. Events should therefore contain a way to restore their partial order of occurrence.

This can be done – among other ways – by adding

In the event type definition, the ordering_key_fields property should be used to indicate which field(s) contains the ordering key, if any.

System timestamps are not necessarily a good choice, since exact synchronization of clocks in distributed systems is difficult, two events may occur in the same microsecond and system clocks may jump backward or forward to compensate drifts or leap-seconds. If you use system timestamps to indicate event ordering, you must carefully ensure that your designated event order is not messed up by these effects.

Also, if using timestamps, the producer must make sure that they are formatted for all events in the UTC time zone, to allow for a simple string-based comparison.

Note that basing events on data structures that can be converged upon in a distributed setting (such as CRDTs, logical clocks and vector clocks) is outside the scope of this guidance.

The hash partition strategy allows a producer to define which fields in an event are used as input to compute a logical partition the event should be added to. Partitions are useful as they allow supporting systems to scale their throughput while provide local ordering for event entities.

The hash option is particulary useful for data changes as it allows all related events for an entity to be consistently assigned to a partition, providing a relative ordered stream of events for that entity. This is because while each partition has a total ordering, ordering across partitions is not assured by a supporting system, thus it is possible for events sent across partitions to appear in a different order to consumers that the order they arrived at the server.

When using the hash strategy the partition key in almost all cases should represent the entity being changed and not a per event or change identifier such as the eid field or a timestamp. This ensures data changes arrive at the same partition for a given entity and can be consumed effectively by clients.

There may be exceptional cases where data change events could have their partition strategy set to be the producer defined or random options, but generally hash is the right option - that is while the guidelines here are a "should", they can be read as "must, unless you have a very good reason".

A data change event’s representation of an entity should correspond to the REST API representation.

There’s value in having the fewest number of published structures for a service. Consumers of the service will be working with fewer representations, and the service owners will have less API surface to maintain. In particular, you should only publish events that are interesting in the domain and abstract away from implementation or local details - there’s no need to reflect every change that happens within your system.

There are cases where it could make sense to define data change events that don’t directly correspond to your API resource representations. Some examples are -

Event definitions must have clear ownership - this can be indicated via the owning_application field of the EventType.

Typically there is one producer application, which owns the EventType and is responsible for its definition, akin to how RESTful API definitions are managed. However, the owner may also be a particular service from a set of multiple services that are producing the same kind of event.

Events must be consistent with other API data and the API Guidelines in general.

Everything expressed in the Introduction to these Guidelines is applicable to event data interchange between services. This is because our events, just like our APIs, represent a commitment to express what our systems do and designing high-quality, useful events allows us to develop new and interesting products and services.

What distinguishes events from other kinds of data is the delivery style used, asynchronous publish-subscribe messaging. But there is no reason why they could not be made available using a REST API, for example via a search request or as a paginated feed, and it will be common to base events on the models created for the service’s REST API.

The following existing guideline sections are applicable to events:

Changes to events must be based around making additive and backward compatible changes. This follows the guideline, "Must: Don’t Break Backward Compatibility" from the Compatibility guidelines.

In the context of events, compatibility issues are complicated by the fact that producers and consumers of events are highly asynchronous and can’t use content-negotiation techniques that are available to REST style clients and servers. This places a higher bar on producers to maintain compatibility as they will not be in a position to serve versioned media types on demand.

For event schema, these are considered backward compatible changes, as seen by consumers -

These are considered backwards-incompatible changes, as seen by consumers -

Event type schema should avoid using additionalProperties declarations, in order to support schema evolution.

Events are often intermediated by publish/subscribe systems and are commonly captured in logs or long term storage to be read later. In particular, the schemas used by publishers and consumers can
drift over time. As a result, compatibility and extensibility issues that happen less frequently with client-server style APIs become important and regular considerations for event design. The guidelines recommend the following to enable event schema evolution:

The above constraint does not mean fields can never be added in future revisions of an event type schema - additive compatible changes are allowed, only that the new schema for an event type must define the field first before it is published within an event. By the same turn the consumer must ignore fields it does not know about from its copy of the schema, just as they would as an API client - that is, they cannot treat the absence of an additionalProperties field as though the event type schema was closed for extension.

Requiring event publishers to define their fields ahead of publishing avoids the problem of field redefinition. This is when a publisher defines a field to be of a different type that was already being emitted, or, is changing the type of an undefined field. Both of these are prevented by not using additionalProperties.

See also rule 111 in the Compatibility section for further guidelines on the use of additionalProperties.

The eid (event identifier) value of an event must be unique.

The eid property is part of the standard metadata for an event and gives the event an identifier. Producing clients must generate this value when sending an event and it must be guaranteed to be unique from the perspective of the owning application. In particular events within a given event type’s stream must have unique identifiers. This allows consumers to process the eid to assert the event is unique and use it as an idempotency check.

Note that uniqueness checking of the eid might be not enforced by systems consuming events and it is the responsibility of the producer to ensure event identifiers do in fact distinctly identify events. A straightforward way to create a unique identifier for an event is to generate a UUID value.

Events that are designed for idempotent out-of-order processing allow for extremely resilient systems: If processing an event fails, consumers and producers can skip/delay/retry it without stopping the world or corrupting the processing result.

To enable this freedom of processing, you must explicitly design for idempotent out-of-order processing: Either your events must contain enough information to infer their original order during consumption or your domain must be designed in a way that order becomes irrelevant.

As common example similar to data change events, idempotent out-of-order processing can be supported by sending the following information:

A receiver that is interested in the current state can then ignore events that are older than the last processed event of each resource. A receiver interested in the history of a resource can use the ordering key to recreate a (partially) ordered sequence of events.

Event type names must (or should, see MUST/SHOULD use functional naming schema for details and definition) conform to the functional naming depending on the audience as follows:

The following application specific legacy convention is only allowed for internal event type names:

Note: consistent naming should be used whenever the same entity is exposed by a data change event and a RESTful API.

Event consumers must be able to process duplicate events.

Most message brokers and data streaming systems offer "at-least-once" delivery. That is, one particular event is delivered to the consumers one or more times. Other circumstances can also cause duplicate events.

For example, these situations occur if the publisher sends an event and doesn’t receive the acknowledgment (e.g. due to a network issue). In this case, the publisher will try to send the same event again. This leads to two identical events in the event bus which have to be processed by the consumers. Similar conditions can appear on consumer side: an event has been processed successfully, but the consumer fails to confirm the processing.

The following frameworks were specifically designed to support the API First workflow with Open API YAML files (sorted alphabetically):

The Swagger/Open API homepage lists more Community-Driven Language Integrations, but most of them do not fit our API First approach.

These utility libraries support you in implementing various parts of our RESTful API guidelines (sorted alphabetically):

This chapter is about coding, developing or anything related to using a programming language to solve a problem.

A problem is defined as a resolution for a challenge the business is facing. The mission of a coder is to write ”good”, ”clean” and ”secure” code. These terms are ill-defined.

At Pon we consider code ”good”, ”clean” and ”secure” based on the following rule of thumb

There is a clear relationship between the amount of bugs in the code and the development effort required, needlessly complex illogical code will in itself be a source of new bugs. Moreover clear documentation and requirements from the business are the only solid foundation for software development.

The following chapters will further explain the intricacies of this rule.

Rules and definition used in the development guidelines.

Refers to “How much effort”.

The amount of effort required for a change should be on-par with the apparent complexity of the change.

A logical structure, both for files and in the code itself, is essential. This also applies to the layout of the code itself, for example indents.

Refers to “How much effort”.

Code is simple and concise in order to increase the readability [praeng], rule 4 [nasa-safety-code]. Increasing performance by sacrificing readability is only an option if the performance increase is in-line with the business requirements.

Refers to "How much effort".

Or, simply put:

However keep in mind that if two distinct pieces of knowledge result in the same code the DRY principle should be reviewed carefully as shown in [accidental-doppelganger]

Refers to “without involving the previous developer”.

All code is sufficiently documented in order to reduce the effort²⁴⁴ required for updates and changes. Comments must explain the why, not the how [praeng].

When readability is sacrificed for performance²⁵⁴ it is reflected in the comments.

Refers to “taking into account the lifetime, quality, security and business impact”.

Solution design comes first, coding second. The solution design must address the following

Based on the quality as discussed in the solution design steps the code quality must be known.

This rule does not state that code must be fully automatically tested and scoring 100/100 on quality. This rule states that the quality, as agreed upon beforehand with the business, is known and documented.

Refers to “How much effort”.

Keep the number of conditional statements to a minimum; rule 1 of [nasa-safety-code].

Only use an else statement if required. Prefer a switch statement over multiple if-then-else constructs.

Each source file is self explanatory and must contain comments showing at least the following

Filenames

All commissioned code is intellectual property and must be stored in a Pon-managed source code repository, preferably in the Pon-maintained Github environment. Use of personal accounts to store and version code that is intellectual property is strictly prohibited.

All code must be reviewed according to documented and approved guidelines. Relates to Coding rule: logical structured code.

The linter configuration is selected from the solution architecture repository.

Code changes must be covered by automated tests. When choosing how to cover your changes, pick the most lightweight (execution time wise) test type that will provide sufficient coverage. If you encounter an existing test that insufficiently covers your changes, you can delete that test but you must write a proper test to replace it. For example, a method that interacts with database has a unit test. You can replace it with an integration test.

Be aware that while high-level tests may provide coverage to code, it is only indirect coverage. Tests with more direct usage of the changed code will likely still needs to be written to ensure that regardless of the which components are actually used in the black box, the individual components still have coverage.

These tests must test the concrete implementation’s behavior in a way that can not be inadvertently changed outside of the test itself.

Logging directly to the console or command line is not recommended; it reduces the effectivess of debugging, monitoring and checking the operational status of the software. It increases the security risk because it is challenging to detect if senstive data is being logged.

To effectively use the library logging levels (DEBUG, ERROR, NOTICE) should be used.

The international standard RFC-3339 must be used for time and date encoding unless not feasible due to technical constraints.

Date and time manipulation is very complex and involves many one-offs, and thus very error-prone. Date and time manipulation must be handled by a dedicated library, which is available for all main development languages.

Schema-based JSON properties that are by design durations and intervals could be strings formatted as recommended by RFC-3339 (Appendix A of RFC 3339 contains a grammar for durations).

The body of an if or else must be encapsulated it increases readability and decreases the introduction of bugs.

For single-line statements a ternary is preferred.

If using if statements where the conditionals are related, they should be ordered by increased complexity when possible. Using this ordering will significantly improve readability.

In most programming languages several options are available for the quoting of text, in the following examples Javascript style quoting is used which is applicable to a variety of other programming languages.

For the following examples the key question is: can the readbility be improved to make the code more simple and concise?

Tabs are hould never be used for indentation. A notable exception is Go where tabs are a language default.

Using predefined spacing for indentation results in clear readable code without sacrificing too much screen real estate; SHOULD never use tabs for indentation.

When using multiple languages a single project it is preferred to use the same indent for all languages involved.

Return values of functions should not be ignored, especially if error return values must be propagated up the function call chain. By checking return types exception justification is enforced, which will result in increased code stability.

Input parameters should not be assumed to be valid; by checking the validity code stability is increased.

All variables are in use, unused variables have no function and are cluttering the code.

Using < is preferred over using <=, using > is preferred over using >=. It improves readability and performance of code.

If a return value always results in true except for a single value using != is preferred over using > or >=.

In our IDE and CI flow we use golangci-lint as linter. Follow the installation steps for you system. Golangci-lint has integrations with various CI systems and IDE’s. golangci-lint runs the most important code checks by default. In our CI/CD the default lint settings of golangci-lint are mandatory.

We preferably configure our IDE to run go Vet on every save operation.

No code may be build without a code check. This is done by default when rule 270 is applied.

Unlike the development guidelines for other languages in Go we use tabs for indentation. To improve formatting of code we use gofmt to automatically format go code in de IDE.

In Go you don’t worry about formatting yourself and use gofmt for automatic formating. Gofmt directions are leading. GOFMT must part of the CI pipeline and is best used in the IDE on the "save" action.

In Go exported functions must be commented and is enforced by linting. Comments on unexported functions are not enforced by linting, but improve readablity of the code. See rule 297 for comment syntaxes.

For readability only one variable is declared in a single line

Global variables can cause al kinds of bugs in your code and result in poor readability and difficult debugging. It is hard to keep track of where the value is modified and will cause race conditions when used in combination with concurrency. Another problem with global variables is that it can cause conflicts when importing packages. When either the package itself or the imported packages has global variables they are available in the other.

When declaring a variable or a constant the type is explicitly declared. This avoids problems with int, int64 etc.

An uninitialized variable has the zero value for the declared type. This can be used to check if a variable is zero valued in an absolute and readable manner. These unitialized variabled can be used as global vars to avoid recursion, but be aware of the risks of global vars as stated in rule 300. This is an exception to rule

In some languages try/catch statements are used. Go does not have that method and we do not try to mimic it. Instead we handle the errors and the errors are just values. You need to assess the value and act upon it.

We handle errors only once.

When errors are passed it might eventualy be unclear what the origin of the error is. You can pass context to it, but be carefull with fmt.Errorf(), because that will override the initial error with just a string.