Skip to content
Staircase

Assess

A rules engine over infrastructure definitions, fetched at run time, with a version hash that blocks publication against a stale rule set.

Assess runs before anything is built. The service definition is checked against a rule set covering the deployment manifest, the API definition, and the security, distribution, identity and logging conventions every service is required to follow.

The rules are fetched at run time rather than vendored into each repository, so tightening a rule is one edit at the source and the next build everywhere picks it up.

How it works

One rule carries the most weight. Named EndpointDocumentationNotExists, it asserts that every endpoint declared in the deployment manifest also appears in the API definition — checked on every build, blocking the build where it did not hold.

No endpoint could be released undocumented, so the API definitions stayed at their canonical paths and each service's contract surface is readable off its repository.

Publication is gated on a version hash over the joined rule hashes. Three naming rules are excluded from the join, and their absence in any form makes every generated hash differ — the note on record says the hash is deliberately broken in that case, to stop a broken version publishing.

When a component linked to Assess publishes, the catalogue reads the bundle's hash, compares it against the last one recorded, and updates the hash required for publication when it differs. A build against a stale rule set therefore fails at publication rather than shipping quietly.

Operations

Assess

POST /assessments

Create Assessment

new-assessment

Assess validates a bundle of source code (Identified by a source_url) according to the rules configured for Assess.

Assess returns an assessment_id in the response body. To retrieve the results of the validation, call Retrieve Assessment Status with the assessment_id.

Request
application/json 
{
  "source_url": "https://code-dev-0258.s3.amazonaws.com/code-assessor/1506dbf2/source.zip?AWSAccessKeyId=2323"
}
Response
application/json

200 response

{
  "assessment_id": "devops-code-assessor:9dff4dc3-f700-4fc8-a51a-56d91d7ed7dg"
}
Request bodyapplication/json
1 fields
FieldTypeDescription
source_urlrequiredstringURL to project source code.
Response 200application/json
1 fields

200 response

FieldTypeDescription
assessment_idstringAssessment id, used for get assessment status
Other responses

400401403404

GET /assessments/{assessment_id}

Retrieve Assessment Status

assessment-status

Retrieve Assessment Status retrieves the results of the validation performed by Assess for the given assessment_id.

Response
application/json

200 response

{
  "assessment_id": "devops-code-assessor:9dff4dc3-f700-4fc8-a51a-56d91d7ed7dg",
  "logs": "[Container]2021/01/29 19:44:20 Waiting for agent ping [Container]",
  "metadata": {
    "service-assessor": {
      "id": "devops-code-assessor:9dff4dc3-f700-4fc8-a51a-56d91d7ed7dg",
      "issuer": "https://api.staircaseapi.com/code_assessor",
      "status": "SUCCEEDED",
      "timestamp": 1611949487.728646,
      "version": "1.0.0"
    },
    "service-code": {
      "commit_hash": "5a8d1bbd276306e73ad5fcb1c45dfe3bb2e6c5c2",
      "id": "1506dbf2-ea7b-4b76-9f51-2ebb0ac49af1",
      "issuer": "https://api.staircaseapi.com/code",
      "status": "SUCCEEDED",
      "timestamp": 1611948276.2954555,
      "version": 1.1
    }
  },
  "source_url": "https://code-assessor-oybto4.s3.amazonaws.com/assessments/devops-code-assessor-a51a-56d91d7ed7df/build.zip?AWSAccessKeyId=2525&Signature=TyI1LC0",
  "status": "SUCCEEDED",
  "warnings": [
    {
      "errors": [
        {
          "message": "Run pipeline was expected",
          "path": "jobs.Pipeline.name"
        }
      ],
      "file": ".github/workflows/pipeline.yml",
      "message": "Your pipeline configuration is not compliant",
      "rule_code": "pipeline"
    }
  ]
}
Parameters
1
ParameterTypeExampleDescription
assessment_id required string path devops-code-assessor:9dff4dc3-f700-4fc8-a51a-56d91d7ed7dg id of assessment
Response 200application/json
6 fields

200 response

FieldTypeDescription
errorsobjectContains errors on failed assessment
errorsobject[]Errors array
messagestringError message
pathstringPath of the property or location issue in your file
filestringAssessed file
messagestringError message
rule_codestringRule code
source_urlstringPresigned URL with the assessed bundle and including metadata in the header for Assess and other products. This metadata is in use as a signature of successful assessment.
statusstringStatus of assessmentFAILEDFAULTIN_PROGRESSNOT_FOUNDSTOPPEDSUCCEEDEDTIMED_OUT
metadataobjectAssessment metadata plus metadata included in the assessment presigned URL coming from other products
logsstringContains log on failed assessment
warningsobjectContains warnings on assessment
errorsobject[]Errors array
messagestringError message
pathstringPath of the property or location issue in your file
filestringAssessed file
messagestringError message
rule_codestringRule code
Other responses

400401403404

POST /assessments/overview

Create Overview Assessment

new-overview-assessment

Create Assessment Overview

Assess Overview validates an input of overview in String format according to the rules configured for Assess.

Assess returns an assessment_id in the response body. To retrieve the results of the validation, call Retrieve Assessment Status with the assessment_id.

Request
application/json 
{
  "overview": "<div>Overview html template</div>",
  "dictionary": [
    "source_url",
    "x-api-key"
  ]
}
Response
application/json

200 response

{
  "assessment_id": "devops-code-assessor:9dff4dc3-f700-4fc8-a51a-56d91d7ed7dg"
}
Request bodyapplication/json
2 fields
FieldTypeDescription
overviewstringOverview in string format
dictionarystring[]Words to omit in spell check
Response 200application/json
1 fields

200 response

FieldTypeDescription
assessment_idstringAssessment id, used for get assessment status
Other responses

400401403404

POST /assessments/swagger

Create Assessment Swagger

new-assessment-swagger

Assess Swagger validates an input of swagger in JSON format according to the rules configured for Assess. Assess returns an assessment_id in the response body. To retrieve the results of the validation, call Retrieve Assessment Status with the assessment_id.

Request
application/json 
{
  "swagger": {
    "info": {
      "title": "Assess",
      "x-product-category": "Ship",
      "x-product-family": "DevOps",
      "x-product-name": "Assess"
    }
  },
  "dictionary": [
    "source_url",
    "x-api-key"
  ]
}
Response
application/json

200 response

{
  "assessment_id": "devops-code-assessor:9dff4dc3-f700-4fc8-a51a-56d91d7ed7dg"
}
Request bodyapplication/json
2 fields
FieldTypeDescription
swaggerobjectSwagger in JSON format
dictionarystring[]Words to omit in spell check
Response 200application/json
1 fields

200 response

FieldTypeDescription
assessment_idstringAssessment id, used for get assessment status
Other responses

400401403404

GET /version-hash

Retrieve Version Hash

get-version-hash

Retrieve Version Hash retrieves the assessment version hash currently available in an environment.

Response
application/json

200 response

{
  "version_hash": "5a8d1bbd276306e73ad5fcb1c45dfe3bb2e6c5c2"
}
Response 200application/json
1 fields

200 response

FieldTypeDescription
version_hashrequiredstringVersion hash of the latest commit of the Assess repository

Assessment Rules

GET /rules-category

Retrieve List of Assessment Rules

get-rules

Retrieve List of Assessment Rules retrieves a list of all assessments rules configured for Assess.

Response
application/json

200 response

[
  {
    "code": "usagePlan",
    "description": "Usage Plan assessment rules"
  },
  {
    "code": "swagger",
    "description": "Swagger assessment rules"
  }
]
Response 200application/json
2 fields

200 response

FieldTypeDescription
coderequiredstringCode of the rule
descriptionrequiredstringDescription of the rule
Other responses

400401403404

GET /rule/{code}

Retrieve Assessment Rule

get-rule

Retrieve Assessment Rule retrieves an assessment rule for a given code. A list of code are returned by the service Retrieve List of Assessment Rules.

Response
application/json

200 response

{
  "type": "object",
  "description": "Usage Plan assessment rules",
  "required": [
    "handler"
  ],
  "properties": {
    "handler": {
      "const": "handler"
    }
  }
}
Parameters
1
ParameterTypeExampleDescription
code required string path usagePlan Rule code from the service Retrieve the list of all assessment rules
Response 200application/json
4 fields

200 response

FieldTypeDescription
typestringType of the schema
descriptionstringDescription of the schema
requiredstring[]Mandatory schema parameters
propertiesobjectSchema characteristics
Other responses

400401403404

POST /rule

Update Assessment Rule

post-rule

Update Assessment Rule updates an assessment rule configured for Assess. Assessment rules are configured in JSON format, as defined in

List of available actions:

  • insert_product_names: Add new product names.
  • remove_product_names: Delete product names.
  • update_server_url_pattern: Set server URL pattern using regular expression.
  • insert_schema_rule: Add a new rule in the schema.
  • remove_schema_rule: Delete a rule by name.
  • remove_schema: Delete a schema.
  • replace_all_schemas: Replace all schema action allows changing or add multiple rules at the same time by just replacing the complete schema.

Important considerations:

Show the rest
  • This feature only allows updating swaggerProductName, serviceProductName, serverlessProductName and swaggerBase schema rules.
  • You can get the rule code by the service Retrieve List of Assessment Rules.
  • You can see the complete schema by the service Retrieve Assessment Rule.
Request
application/json 
{
  "rule_name": [
    "swaggerProductName",
    "serviceProductName",
    "serverlessProductName"
  ],
  "action": "_insert_product_names",
  "product_names": [
    "Account",
    "Test"
  ]
}
Response
application/json

200 response

{
  "status": "SUCCEEDED"
}
Request bodyapplication/json
6 fields
FieldTypeDescription
rule_namerequiredstring[]Schema name
actionrequiredstringAction to execute on the rule
product_namesstring[]Product names to insert or delete
server_url_patternstringRegular expression to validate the server URL
schema_rule_namestringSchema rule name to insert or delete
schemaobjectSchema of the rule defined in this format
Response 200application/json
1 fields

200 response

FieldTypeDescription
statusstringSUCCEEDED
Other responses

400401403404

Errors

400401403404

Type to search.