Skip to content

api_guidelines.md

Latest commit

 

History

History
486 lines (390 loc) · 19.3 KB

api_guidelines.md

File metadata and controls

486 lines (390 loc) · 19.3 KB

ACS API Guidelines

An overview of the set of standardized guidelines and practices to ensure that the APIs across different projects and teams adhere to a common structure and design. This consistency simplifies the process of understanding and maintaining APIs. These guidelines help identify potential pitfalls and design choices that could lead to issues to help developers avoid common mistakes and build more robust APIs. When in doubt, It is encouraged to seek resolution in slack channel #forum-acs-api-design.

Table of Contents

Decoupling

While certain data may exist in the database for internal purposes or other functionalities, it is important to exercise caution when deciding what data to make accessible through the API. The rule of thumb to follow is that the APIs must not expose data structures defined for internal purposes. Instead, define a data structure for the sole purpose of API use, generally in the same package where the API is defined. This applies to all APIs including the inter-component APIs.

Exposing internal data structures, especially the ones representing and affecting database schema directly at the API can pose security risks. By carefully selecting and limiting the data exposed via the API, we can prevent unauthorized access and potential data breaches which can have compliance ramifications.

Furthermore, exposing unnecessary data in the API can lead to overfetching, where the client receives more data than required. This inefficiency increases network traffic, consumes extra bandwidth, and impacts API performance. By limiting API data to what is actually needed by clients, we optimize resource usage and improve response times.

Consider the following database/internal structures as an example data structure used in Central.

Deployment {
  string id
  string name
  repeated KeyValue labels
  repeated KeyValue annotations
  repeated Container containers
}
  
Container {
  string name
  string image_name
  repeated Volume volumes
}

Exposing the above data structure in the API will always lead to reading all the Container bytes from the database, even if the user does not need them. Instead, design the API such that users can request the information they need, as below:

// GetDeploymentRequest requests deployment information. Deployment metadata is requested by default.
GetDeploymentRequest {
  Options {
    bool get_container_spec `json: "getContainerSpec"`
  }
}

// Default response
GetDeploymentResponse {
  Metadata {
    string name `json: "name"`
    string id `json: "id"`
  }
}

// If `"getContainerSpec": true`
GetDeploymentResponse {
  Metadata {
    string name `json: "name"`
    string id `json: "name"`
    ...
  }
  ContainerSpec {
    string name `json: "name"`
    string image_name `json: "imageName"`
    ...
  }
}

As APIs evolve over time, adding or removing data fields becomes inevitable. By limiting the exposure of internal database structures, we decouple the API from the database schema, allowing us to modify the internal structures without breaking existing client applications.

Consider the following database/internal structures as an example data structure exposed by APIs.

Alert {
  string id
  repeated Violation violations
  string policy_name
  string policy_description
  string policy_enforcement
}

Changing the above Alert data structure to the following structure may break the client application.

Alert {
  string id;
  repeated Violation violations
  
  Policy {
    string policy_name
    string policy_description
    string policy_enforcement
  }
}

Keeping the API focused and concise simplifies maintenance efforts and reduces the chances of introducing bugs. A clean and manageable codebase improves the overall maintainability and stability of the API.

Naming Guidelines

gRPC Service Name

The service name must be unique and use a noun that generally refers to a resource or product component and must end with Service e.g. DeploymentService, ReportService, ComplianceService. Intuitive and well-known short forms or abbreviations may be used in some cases (and could even be preferable) for succinctness e.g. ReportConfigService, RbacService. All gRPC methods grouped into a single service must generally pertain to the primary resource of the service.

gRPC Method Name

Methods should be named such that they provide insights into the functionality.

Let us look at a few examples.StartComplianceScan, RunComplianceScan, and GetComplianceScan are not the same.

  • StartComplianceScan should return without waiting for the compliance scan to complete.
  • RunComplianceScan is ambiguous because it is unclear if the call waits for the scan to complete. The ambiguity can be removed by adding a field to the request that helps clarify the expectation e.g. bool wait_for_scan_completion if set to true informs the method to wait for the compliance scan to complete. However, for long-running processes, it is recommended to create a job that finishes the process asynchronously and return the job ID to the users which can be tracked via dedicated job tracking method.
  • GetComplianceScan should not run a compliance scan but only fetches a stored one.

Typically, the method name should follow the VerbNoun convention.

Verb Noun Method name
List Deployment ListDeployments
Get Deployment GetDeployment
Update Deployment UpdateDeployment
Delete Deployment DeleteDeployment
Notify Violation NotifyViolation
Run ComplianceScan RunComplianceScan

It is recommended that the verbs be imperative instead of inquisitive. Generally, the noun should be the resource type. In some cases, the noun portion could be composed of multiple nouns e.g. GetVulnerabilityDeferralState, RunPolicyScan.

Inquisitive Imperative
IsRunComplete GetRunStatus
IsAdmin GetUserRole
IsVulnerabilityDeferred GetVulnerabilityDeferralState

The noun portion of methods that act on a single resource must be singular e.g. GetDeployment. Those methods that act on the collection of resources must be plural e.g. ListDeployments, DeleteDeployments. Avoid prepositions (e.g. for, by) in method names as much as possible. Typically, this can be addressed by using a distinct verb, adding a field to the request message, or restructuring VerbNoun.

Instead ofUse

GetBaselineGeneratedNetworkPolicyForDeployment

 
GenerateDeploymentNetworkPolicy

GenerateDeploymentNetworkPolicyRequest {
  bool from_baseline;  
  bool from_network_flows;
}

RunPolicyScanForDeployment

RunDeploymentPolicyScan

DeleteDeploymentsByQuery

 
DeleteDeployments

DeleteDeploymentsRequest {
  string query; 
}

GetBaselineGeneratedNetworkPolicyForDeployment

GetDeploymentBaselineNetworkPolicy or merely GetBaselineNetworkPolicy if the concept of baselines applies to deployments only.

The following example demonstrates design if that concept of baselines could apply to multiple resource types.

GetBaselineNetworkPolicy

GetBaselineNetworkPolicyRequest {
  oneof resource {
    string deployment_id;
    string cluster_id;
  }
}

gRPC Message Name

The request and response messages must be named after method names with suffix Request and Response unless the request/response type is an empty message. Generally, resource type as response message should be avoided e.g. use GetDeploymentResponse response instead of Deployment. This allows augmenting the response with supplemental information in the future.

Verb Noun Method name Request message Response message
List Deployment ListDeployments ListDeploymentRequest ListDeploymentResponse
Get Deployment GetDeployment GetDeploymentRequest GetDeploymentResponse
Update Deployment UpdateDeployment UpdateDeploymentRequest UpdateDeploymentResponse
Delete Deployment DeleteDeployment DeleteDeploymentRequest google.protobuf.Empty
Get ReportStatus GetReportStatus GetReportStatusRequest GetReportStatusResponse
Run ComplianceScan RunComplianceScan RunComplianceScanRequest RunComplianceScanResponse

Avoid prepositions as much as possible (e.g. “for”, “with”; DeploymentWithProcessInfo, DeploymentWithImageScan). In case such a need arises, add a field to the request message and response message.

Instead ofUse

GetDeploymentWithImageScanRequest

 
GetDeploymentRequest {
  bool with_image_scan;
}

GetDeploymentWithImageScanResponse

 
GetDeploymentImageScanResponse {
  Image image;
}

or,

GetDeploymentResponse {
  Deployment deployment;
  Image image;
}

RunPolicyScanForDeploymentRequest

RunDeploymentPolicyScanRequest

All fields in the message must be lowercase and underscore separated names. The JSON names for the fields are autogenerated by the proto compiler. By default, field names are converted to camel case notation.

Proto field name JSON field name
network_data_start_time networkDataStartTime
expiry_date expiryDate

Be explicit about conveying the specific purpose of fields e.g. instead of expires_on use expiry_date(/timestamp) as it informs users if the field returns the date portion of the timestamp or the full timestamp, and use network_data_start_time instead of network_data_since for a similar reason. The fields should convey their purpose without requiring users to read the documentation.

URL Guidelines

This section goes over key URL guidelines that could help avoid common mistakes when building APIs.

All APIs, except custom HTTP routes, must be prefixed with API version. Custom routes should not be version prefixed due to current design limitation of we handle gRPC vs HTTP-only endpoints (which may be mitigated in the future). The version is typically followed by the plural form of resource noun in the service name e.g. /v1/deployments, /v2/violations. The resource noun may be singular if it refers to non-acting resources encapsulating acting resources e.g. /v1/compliance, /v1/networkgraph, /v1/debug, /v1/auth.

Path parameters are variable components of a URL path. They are typically used to point to a specific resource. A URL can have several path parameters, each denoted with curly braces { }. If the request URL contains one or more path parameters, the path parameter should clearly indicate the resource type otherwise by default they are associated with resource type in the preceding URL component.

/v1/compliance/profiles/{id}

 Acts on a specific compliance profile

/v1/networkpolicies/{id}

 Acts on a specific network policy.

/v1/networkpolicies/{deployment_id}

 Not recommended. Consider query string parameter pattern instead.

Keep it simple and descriptive; avoid long-worded URL components. If one object can contain another object, design the endpoint to reflect that regardless of whether the data is structured like this in the database. If the URL describes the action, nest the action within the resource. Avoid stop words (the, and, or, of, a, an, to, for, etc.) in a URL to make it shorter and more readable.

Instead of Use
GET: /v1/depoymentsbynamespace GET: /v1/namespaces/{namespace}/deployments Get all deployments in a specific namespace
GET: /v1/reportsmetadata/{id} GET: /v1/reports/jobs/{id}/metadata Get metadata of report job
GET: /v1/reports/status/{id} GET: /v1/reports/jobs/{id}/status Get status of report job
GET: /v1/complianceprofiles GET: /v1/compliance/profiles Get all compliance profiles
POST: /v1/resetbaselineforcluster/{cluster_id} POST: /v1/baselines/cluster/{cluster_id}/reset Reset baseline for a specific cluster

If splitting the words into multiple URL components is not intuitive, it is recommended to use a hyphen(-).

Instead of Use
/v1/kernelsupport /v1/kernel-support
/v1/kernel_support /v1/kernel-support
/v1/securitypolicy /v1/security-policy

If the API acts on specific attribute of the resource, using URL query parameters is more intuitive over long-worded or hierarchical identifiers.

Instead of Use
/v1/deferredcves /v1/cves?deferred=true
/v1/inactiveviolations /v1/violations?inactive=true
/v1/fixablecves /v1/cves?fixable=true
/v1/runningreports /v1/reports?status=running

URLs should not be duplicated. A request URL and request method should uniquely identify an API. For example, GET: /v1/deployments/{id} conflicts with GET: /v1/deployments/violations.

A GET API must not declare a body but instead specify the response criteria using path and query parameters, therefore, the gRPC request message fields should map to the URL path or query parameters. The parameters could be used for identifying, filtering, sorting, paginating, tracking the source, translation, etc.

POST requests to create a resource must not accept the resource ID. IDs must be generated on the backend. Only POST requests to perform an action on a specific resource should accept a resource ID.

A GET API must use an HTTP GET verb. For example, the following API configuration is not recommended and should be avoided:

rpc GetExistingProbes(GetExistingProbesRequest) returns (GetExistingProbesResponse) {
option (google.api.http) = {
post: "/v1/probeupload/getexisting"
};
}

Deprecation guidelines

While we generally try to avoid deprecations, it is sometimes necessary.

This section only applies for General Available (GA) features. For Technology Preview, these guidelines do not apply and only need to be announced in release notes.

Before deprecating APIs, clarify with relevant stakeholders (e.g. product team, solution engineering) the deprecation. In addition, if available for your API, the analytics we collect may also be used to gauge whether deprecation is an option or not.

While there are no strict guidelines from Red Hat about announcing deprecations, historically we have announced deprecations two releases in advance. This gives users enough time to adjust potential usages of the deprecated API.

The deprecation announcement must be done at least within the release notes within the Deprecated Features section and the associated service proto (i.e. the API documentation).

In addition, there are other forms of announcement that have been used historically. Use these with your best judgement:

  • Within the UI
    • This can be a banner or other hints announcing the deprecation close to places where the respective API is being used.
    • As an example, the SAC resource consolidation efforts led to a banner being created in the UI where resources were being used.
  • Within Central logs
    • This can be done during Central startup or during API calls, informing the user that the API service will be removed in the future.
    • Be aware that high-cardinality APIs are not an ideal candidate for logging per API call.

Once the deprecation time has been met after two releases, the API can safely be fully removed (i.e. removing the service associated with the API), but it is up to your discretion to postpone or not to go through with the deprecation.