CircleCI
Gets the details of the CircleCI workflows; including the details of the last runs and the jobs, and retrieves the artifacts of the jobs.
Utilities · CircleCI
Details
| ID | CircleCI |
|---|---|
| Provider | CircleCI |
| Category | Utilities |
| From Version | 5.5.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | Agentix XSIAM |
README
Gets the details of the CircleCI workflows; including the details of the last runs and the jobs, and retrieves the artifacts of the jobs.
This integration was integrated and tested with version v2 of CircleCI.
Configure CircleCI on Cortex XSOAR
- Navigate to Settings > Integrations > Servers & Services.
- Search for CircleCI.
-
Click Add Instance to create and configure a new integration instance.
Parameter Description Required Your server URL True API key The API key used to connect True Version control system type Type of version control system True Organization Name Name of the organization True Project Name Name of the project True Trust any certificate (not secure) True Use system proxy settings False - Click Test to validate the URLs, token, and connection.
Commands
You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
circleci-workflows-list
Gets information on workflows.
Base Command
circleci-workflows-list
Input
| Argument Name | Description | Required |
|---|---|---|
| limit | Maximum number of workflows to retrieve. Default is 20. | Optional |
| vcs_type | Version control system type of the project. Possible values: “github” and bitbucket”. Default is github. | Optional |
| organization | Organization from which to retrieve workflows. Defaults to the organization instance parameter. | Optional |
| project | Project from which to retrieve workflows. Defaults to the project instance parameter. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| CircleCI.Workflow.metrics.duration_metrics.max | Number | Maximum workflow run time. |
| CircleCI.Workflow.metrics.duration_metrics.mean | Number | Mean workflow run time. |
| CircleCI.Workflow.metrics.duration_metrics.median | Number | Median workflow run time. |
| CircleCI.Workflow.metrics.duration_metrics.min | Number | Minimum workflow run time. |
| CircleCI.Workflow.metrics.duration_metrics.p95 | Number | 95th percentile workflow run time. |
| CircleCI.Workflow.metrics.duration_metrics.standard_deviation | Number | The standard deviation of the workflow run time. |
| CircleCI.Workflow.metrics.duration_metrics.total_duration | Number | Total duration. |
| CircleCI.Workflow.metrics.failed_runs | Number | Number of failed workflow runs. |
| CircleCI.Workflow.metrics.median_credits_used | Number | Median credits used. |
| CircleCI.Workflow.metrics.mttr | Number | Mean recovery time. |
| CircleCI.Workflow.metrics.success_rate | Number | Success rate. |
| CircleCI.Workflow.metrics.successful_runs | Number | Number of successful runs. |
| CircleCI.Workflow.metrics.throughput | Number | Throughput. |
| CircleCI.Workflow.metrics.total_credits_used | Number | Total credits used. |
| CircleCI.Workflow.metrics.total_recoveries | Number | Total recoveries. |
| CircleCI.Workflow.metrics.total_runs | Number | Total runs. |
| CircleCI.Workflow.name | String | Workflow name. |
| CircleCI.Workflow.project_id | String | The project ID that the workflow belongs to. |
| CircleCI.Workflow.window_end | Date | When the workflow ended. |
| CircleCI.Workflow.window_start | Date | When the workflow started. |
Command Example
!circleci-workflows-list limit=2
Context Example
{
"CircleCI": {
"Workflow": {
"metrics": {
"duration_metrics": {
"max": 6011,
"mean": 4508,
"median": 4508,
"min": 3005,
"p95": 5860,
"standard_deviation": 2125,
"total_duration": 0
},
"failed_runs": 1,
"median_credits_used": 0,
"mttr": 0,
"success_rate": 0.5,
"successful_runs": 1,
"throughput": 0.2,
"total_credits_used": 1900,
"total_recoveries": 0,
"total_runs": 2
},
"name": "bucket_upload_trigger",
"project_id": "4eaba5af-8c43-43ec-b469-3968d8a76f68",
"window_end": "2021-04-22T14:28:57.252Z",
"window_start": "2021-04-13T12:31:14.409Z"
}
}
}
Human Readable Output
CircleCI Workflows
Metrics Name ProjectId WindowEnd WindowStart total_runs: 181
successful_runs: 136
mttr: 93519
total_credits_used: 323000
failed_runs: 43
median_credits_used: 0
success_rate: 0.7513812154696132
duration_metrics: {“min”: 202, “mean”: 8807, “median”: 8606, “p95”: 11307, “max”: 16317, “standard_deviation”: 1707.0, “total_duration”: 0}
total_recoveries: 0
throughput: 2.033707865168539bucket_upload 4eaba5af-8c43-43ec-b469-3968d8a76f68 2021-06-28T23:42:38.647Z 2021-03-31T09:01:11.412Z total_runs: 2
successful_runs: 1
mttr: 0
total_credits_used: 1900
failed_runs: 1
median_credits_used: 0
success_rate: 0.5
duration_metrics: {“min”: 3005, “mean”: 4508, “median”: 4508, “p95”: 5860, “max”: 6011, “standard_deviation”: 2125.0, “total_duration”: 0}
total_recoveries: 0
throughput: 0.2bucket_upload_trigger 4eaba5af-8c43-43ec-b469-3968d8a76f68 2021-04-22T14:28:57.252Z 2021-04-13T12:31:14.409Z
circleci-artifacts-list
Retrieves the artifacts list from the CircleCI job.
Base Command
circleci-artifacts-list
Input
| Argument Name | Description | Required |
|---|---|---|
| job_number | The number of the job from which to retrieve its artifacts, e.g., 31263. | Required |
| artifact_suffix | Returns only the artifacts for which the suffix corresponds to the given suffix, e.g., ‘test_failures.txt’ will only retrieve the artifacts for which the suffix ends with test_failures.txt. | Optional |
| limit | Maximum number of artifacts to retrieve. Default is 20. | Optional |
| vcs_type | The version control system type of the project. Possible values: “github” and “bitbucket”. Default is github. | Optional |
| organization | Organization from which to retrieve artifacts. Defaults to the organization instance parameter. | Optional |
| project | Project from which to retrieve artifacts. Defaults to the project instance parameter. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| CircleCI.Artifact.path | String | Artifact relative path. |
| CircleCI.Artifact.node_index | Number | Artifact node index. |
| CircleCI.Artifact.url | String | Artifact URL. |
Command Example
!circleci-artifacts-list job_number=390115 limit=2
Context Example
{
"CircleCI": {
"Artifact": [
{
"node_index": 0,
"path": "artifacts/env.json",
"url": "https://390115-12353212-gh.circle-artifacts.com/0/artifacts/env.json"
},
{
"node_index": 0,
"path": "artifacts/debug_log.log",
"url": "https://390115-12353212-gh.circle-artifacts.com/0/artifacts/debug_log.log"
}
]
}
}
Human Readable Output
CircleCI Artifacts
NodeIndex Path Url 0 artifacts/env.json https://390115-12353212-gh.circle-artifacts.com/0/artifacts/env.json 0 artifacts/debug_log.log https://390115-12353212-gh.circle-artifacts.com/0/artifacts/debug_log.log
circleci-workflow-jobs-list
Retrieves the jobs list from the CircleCI workflow.
Base Command
circleci-workflow-jobs-list
Input
| Argument Name | Description | Required |
|---|---|---|
| workflow_id | The workflow ID from which to retrieve its jobs, e.g., 12zxcase-12za-as51-123zs4sdgf12. | Required |
| limit | The maximum number of jobs to retrieve. Default is 20. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| CircleCI.WorkflowJob.id | String | Job ID. |
| CircleCI.WorkflowJob.job_number | Number | Job number. |
| CircleCI.WorkflowJob.name | String | Job name. |
| CircleCI.WorkflowJob.project_slug | String | Job project slug. |
| CircleCI.WorkflowJob.started_at | Date | Time the job started. |
| CircleCI.WorkflowJob.status | String | Job status. |
| CircleCI.WorkflowJob.stopped_at | Date | Time the job stopped. |
| CircleCI.WorkflowJob.type | String | Job type. |
| CircleCI.WorkflowJob.dependencies | String | Job dependencies. |
Command Example
!circleci-workflow-jobs-list workflow_id=f85efae0-cbf4-4b6d-b136-e3db67d41221 limit=2
Context Example
{
"CircleCI": {
"Workflow": {
"Job": [
{
"dependencies": [],
"id": "c7425325-bb57-4e78-968f-2c9867d31z11",
"job_number": 389133,
"name": "Setup Environment",
"project_slug": "gh/organization_name/repo_name",
"started_at": "2021-06-24T00:04:57Z",
"status": "success",
"stopped_at": "2021-06-24T00:06:32Z",
"type": "build"
},
{
"dependencies": [
"c7425325-bb57-4e78-968f-2c9867d31z11"
],
"id": "89d36e04-5481-48a3-8be4-ddcb2bdcz1q1",
"job_number": 389166,
"name": "Run Validations",
"project_slug": "gh/organization_name/repo_name",
"started_at": "2021-06-24T00:06:35Z",
"status": "success",
"stopped_at": "2021-06-24T00:38:00Z",
"type": "build"
}
]
}
}
}
Human Readable Output
CircleCI Workflow f85efae0-cbf4-4b6d-b136-e3db67d41221 Jobs
Dependencies Id JobNumber Name ProjectSlug StartedAt Status StoppedAt Type c7425325-bb57-4e78-968f-2c9867d31z11 389133 Setup Environment gh/organization_name/repo_name 2021-06-24T00:04:57Z success 2021-06-24T00:06:32Z build c7425325-bb57-4e78-968f-2c9867d31z11 89d36e04-5481-48a3-8be4-ddcb2bdcz1q1 389166 Run Validations gh/organization_name/repo_name 2021-06-24T00:06:35Z success 2021-06-24T00:38:00Z build
circleci-workflow-last-runs
Retrieves the jobs list from the CircleCI workflow.
Base Command
circleci-workflow-last-runs
Input
| Argument Name | Description | Required |
|---|---|---|
| workflow_name | Name of the workflow from which to retrieve its last runs details. | Required |
| limit | Maximum number of workflow runs to retrieve. Default is 20. | Optional |
| vcs_type | Version control system type of the project. Possible values: “github” and “bitbucket”. Default is github. | Optional |
| organization | Organization from which to retrieve workflow last runs. Defaults to the organization instance parameter. | Optional |
| project | Project from which to retrieve workflow last runs. Defaults to the project instance parameter. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| CircleCI.WorkflowRun.branch | String | Branch name. |
| CircleCI.WorkflowRun.created_at | Date | Time run created. |
| CircleCI.WorkflowRun.credits_used | Number | Credits used. |
| CircleCI.WorkflowRun.duration | Number | Duration of run in seconds. |
| CircleCI.WorkflowRun.id | String | ID of the run. |
| CircleCI.WorkflowRun.status | String | Run status. |
| CircleCI.WorkflowRun.stopped_at | Date | Time run stopped. |
Command Example
!circleci-workflow-last-runs workflow_name=nightly limit=2
Context Example
{
"CircleCI": {
"WorkflowRun": [
{
"branch": "master",
"created_at": "2021-06-29T00:04:56.069Z",
"credits_used": 2482,
"duration": 7743,
"id": "d832d004-0069-4412-8e6d-41265143411z",
"status": "failed",
"stopped_at": "2021-06-29T02:13:59.354Z"
},
{
"branch": "master",
"created_at": "2021-06-28T00:04:55.409Z",
"credits_used": 3129,
"duration": 9778,
"id": "531e678e-73e3-4f2a-ac80-55aa203461za",
"status": "failed",
"stopped_at": "2021-06-28T02:47:52.916Z"
}
]
}
}
Human Readable Output
CircleCI Workflow nightly Last Runs
Branch CreatedAt CreditsUsed Duration Id Status StoppedAt master 2021-06-29T00:04:56.069Z 2482 7743 d832d004-0069-4412-8e6d-41265143411z failed 2021-06-29T02:13:59.354Z master 2021-06-28T00:04:55.409Z 3129 9778 531e678e-73e3-4f2a-ac80-55aa203461za failed 2021-06-28T02:47:52.916Z
circleci-trigger-workflow
Triggers a new pipeline on the project.
Base Command
circleci-trigger-workflow
Input
| Argument Name | Description | Required |
|---|---|---|
| parameters | A JSON object containing pipeline parameters and their values. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| CircleCI.WorkflowTrigger.id | number | The workflow ID. |
| CircleCI.WorkflowTrigger.created_at | number | Date when the workflow was created. |
| CircleCI.WorkflowTrigger.number | number | The workflow number. |
| CircleCI.WorkflowTrigger.state | number | The workflow state. |
Command Example
!circleci-trigger-workflow parameters={"param1": "value"}
Context Example
{
"CircleCI": {
"WorkflowTrigger": {
"created_at": "2021-07-28T15:49:33.906Z",
"id": "9bc69b28-f241-4918",
"number": 103232,
"state": "pending"
}
}
}
Human Readable Output
CircleCI Workflow created successfully, ID=103232
Configuration parameters
url— Your server URL (required)api_key— The API keyapi_key_creds—vcs_type— Version control system type (required)organization— Organization name (required)project— Project Name (required)insecure— Trust any certificate (not secure)proxy— Use system proxy settings
Commands (5)
-
circleci-artifacts-listRetrieves the artifacts list from the CircleCI job.
-
circleci-trigger-workflowTriggers a new pipeline on the project.
-
circleci-workflow-jobs-listRetrieves the jobs list from the CircleCI workflow.
-
circleci-workflow-last-runsRetrieves the jobs list from the CircleCI workflow.
-
circleci-workflows-listGets information on workflows.
from collections.abc import Callable import urllib3 from CommonServerPython import * # noqa # pylint: disable=unused-wildcard-import # Disable insecure warnings urllib3.disable_warnings() # pylint: disable=no-member """ CONSTANTS """ DEFAULT_VCS_TYPE = "github" DEFAULT_LIMIT_VALUE = 20 """ CLIENT CLASS """ class Client(BaseClient): def __init__(self, base_url: str, api_key: str, verify: bool, proxy: bool, vc_type: str, organization: str, project: str): super().__init__(base_url=base_url, verify=verify, proxy=proxy, headers={"authorization": f"Basic {api_key}"}) self.vc_type = vc_type self.organization = organization self.project = project self.api_key = api_key def get_job_artifacts(self, vc_type: str, organization: str, project: str, job_name: str): return self._http_request(method="GET", url_suffix=f"/project/{vc_type}/{organization}/{project}/{job_name}/artifacts") def get_workflows_list(self, vc_type: str, organization: str, project: str, page_token: Optional[str] = None): url_suffix = f"insights/{vc_type}/{organization}/{project}/workflows" return self._http_request(method="GET", url_suffix=url_suffix, params={"page-token": page_token} if page_token else None) def get_last_workflow_runs( self, vc_type: str, organization: str, project: str, workflow_name: str, branch: str, page_token: Optional[str] = None ): url_suffix = f"insights/{vc_type}/{organization}/{project}/workflows/{workflow_name}" params = {} if page_token: params["page-token"] = page_token if branch: params["branch"] = branch return self._http_request( method="GET", url_suffix=url_suffix, params=params, ) def get_workflow_jobs(self, workflow_id: str, page_token: Optional[str] = None): url_suffix = f"/workflow/{workflow_id}/job" return self._http_request(method="GET", url_suffix=url_suffix, params={"page-token": page_token} if page_token else None) def trigger_workflow(self, vc_type: str, organization: str, project: str, parameters: str): url_suffix = f"project/{vc_type}/{organization}/{project}/pipeline" return self._http_request( method="POST", url_suffix=url_suffix, json_data=parameters, resp_type="text", headers={"Circle-Token": self.api_key}, ) """ HELPER FUNCTIONS """ def get_response_with_pagination(client_command: Callable, client_command_args: List, limit: int) -> List[dict]: """ Preforms API calls to CircleCI, using pagination mechanism given by CircleCI. CircleCI gives a page token for retrieving next page if more results exists. Args: client_command (Callable): Client function command which performs the API call. client_command_args (Dict): Args for client command. limit (int): Maximum number of results to retrieve. Returns: (List[Dict]): List of the results. """ results: List[dict] = [] response = client_command(*client_command_args) results.extend(response.get("items", [])) while len(results) < limit: next_page_token: Optional[str] = response.get("next_page_token") if not next_page_token: break response = client_command(*(client_command_args + [next_page_token])) data = response.get("items", []) results.extend(data) return results[:limit] def get_common_arguments(client: Client, args: dict[str, Any]) -> tuple[str, str, str, int]: """ Performs same logic for getting arguments. Args: client (Client): Client to retrieve instance parameter configurations in case argument was not given. args (Dict[str, Any]): XSOAR arguments. Returns: (Tuple[str, str, str]): (vcs_type, organization, project, limit). """ vc_type: str = args.get("vcs_type") or client.vc_type organization: str = args.get("organization") or client.organization project: str = args.get("project") or client.project limit: int = arg_to_number(args.get("limit")) or DEFAULT_LIMIT_VALUE return vc_type, organization, project, limit """ COMMAND FUNCTIONS """ def test_module_command(client: Client) -> str: """ Tests connectivity to CircleCI services. Args: client (Client): Client to perform API request to test connection. Returns: (str): 'ok' upon success, (str): 'Error connecting to CircleCI. Make sure your URL and API token are configured correctly.' upon failure. """ message: str = "ok" try: vc_type, organization, project, _ = get_common_arguments(client, {}) client.get_workflows_list(vc_type, organization, project) except DemistoException as e: if "not found" in str(e).lower(): message = ( "Error connecting to CircleCI. Check if your organization and repository names are correct." " If it is a private repository, make sure your API token is valid." ) else: raise e return message def circleci_workflows_list_command(client: Client, args: dict[str, Any]) -> CommandResults: """ Retrieves workflows list details from CircleCI. Args: client (Client): Client to perform the API calls args (Dict[str, Any]): XSOAR arguments. - 'vc_type' (str): VC type. One of 'github', 'bitbucket'. - 'organization' (str): Organization to retrieve workflows from. Defaults to organization parameter is none is given. - 'project' (str): Project to retrieve workflows from. Defaults to project parameter is none is given. Returns: (CommandResults). """ vc_type, organization, project, limit = get_common_arguments(client, args) response = get_response_with_pagination(client.get_workflows_list, [vc_type, organization, project], limit) return CommandResults( outputs_prefix="CircleCI.Workflow", outputs_key_field="id", readable_output=tableToMarkdown("CircleCI Workflows", response, removeNull=True, headerTransform=camelize_string), outputs=response, ) def circleci_artifacts_list_command(client: Client, args: dict[str, Any]) -> CommandResults: """ Retrieves artifacts list from CircleCI job. Args: client (Client): Client to perform the API calls args (Dict[str, Any]): XSOAR arguments. - 'job_number' (str): Number of the job to retrieve its artifacts. - 'artifact_suffix' (Optional[str]): Will return only artifact whom suffix corresponds to suffix given. - 'vc_type' (str): VC type. One of 'github', 'bitbucket'. - 'organization' (str): Organization to retrieve artifacts from. Defaults to artifacts parameter is none is given. - 'project' (str): Project to retrieve artifacts from. Defaults to project parameter is none is given. - 'limit' (int): Maximum number of results to return. Returns: (CommandResults). """ vc_type, organization, project, limit = get_common_arguments(client, args) job_number: str = args.get("job_number", "") artifact_suffix: Optional[str] = args.get("artifact_suffix") response = get_response_with_pagination(client.get_job_artifacts, [vc_type, organization, project, job_number], limit) if artifact_suffix: response = [artifact for artifact in response if artifact.get("path", "").endswith(artifact_suffix)] else: response = response[:limit] return CommandResults( outputs_prefix="CircleCI.Artifact", outputs_key_field="url", readable_output=tableToMarkdown("CircleCI Artifacts", response, removeNull=True, headerTransform=camelize_string), outputs=response, ) def circleci_workflow_jobs_list_command(client: Client, args: dict[str, Any]) -> CommandResults: """ Retrieve jobs list from CircleCI workflow. Args: client (Client): Client to perform the API calls args (Dict[str, Any]): XSOAR arguments. - 'workflow_id' (str): Workflow ID to retrieve its jobs. - 'limit' (int): Maximum number of results to return. Returns: (CommandResults). """ workflow_id: str = args.get("workflow_id", "") limit = arg_to_number(args.get("limit")) or DEFAULT_LIMIT_VALUE response = get_response_with_pagination(client.get_workflow_jobs, [workflow_id], limit) return CommandResults( outputs_prefix="CircleCI.WorkflowJob", outputs_key_field="id", readable_output=tableToMarkdown( f"CircleCI Workflow {workflow_id} Jobs", response, removeNull=True, headerTransform=camelize_string ), outputs=response, ) def circleci_workflow_last_runs_command(client: Client, args: dict[str, Any]) -> CommandResults: """ Retrieve jobs list from CircleCI workflow. Args: client (Client): Client to perform the API calls args (Dict[str, Any]): XSOAR arguments. - 'workflow_name' (str): Name of workflow to retrieve its last runs details. - 'vc_type' (str): VC type. One of 'github', 'bitbucket'. - 'organization' (str): Organization to retrieve workflow last runs from. Defaults to artifacts parameter is none is given. - 'project' (str): Project to retrieve workflow last runs from. Defaults to project parameter is none is given. - 'limit' (int): Maximum number of results to return. Returns: (CommandResults). """ vc_type, organization, project, limit = get_common_arguments(client, args) workflow_name: str = args.get("workflow_name", "") branch: str = args.get("branch", "") response = get_response_with_pagination( client.get_last_workflow_runs, [vc_type, organization, project, workflow_name, branch], limit ) return CommandResults( outputs_prefix="CircleCI.WorkflowRun", outputs_key_field="id", readable_output=tableToMarkdown( f"CircleCI Workflow {workflow_name} Last Runs", response, removeNull=True, headerTransform=camelize_string ), outputs=response, ) def circleci_trigger_workflow_command(client: Client, args: dict[str, Any]) -> CommandResults: vc_type, organization, project, _ = get_common_arguments(client, args) parameters_json: str = args.get("parameters", "") try: parameters = json.loads(parameters_json) except ValueError: raise DemistoException("Failed to parse the 'parameters' argument.") response_json = client.trigger_workflow(vc_type, organization, project, parameters) response = json.loads(response_json) return CommandResults( outputs_prefix="CircleCI.WorkflowTrigger", outputs_key_field="id", readable_output=f"CircleCI Workflow created successfully, ID={response.get('number')}", outputs=response, ) """ MAIN FUNCTION """ def main() -> None: command = demisto.command() params = demisto.params() args = demisto.args() base_url: str = urljoin(params.get("url", ""), "/api/v2") verify_certificate: bool = not params.get("insecure", False) proxy: bool = params.get("proxy", False) api_key: str = params.get("api_key_creds", {}).get("password") or params.get("api_key", "") vc_type: str = params.get("vcs_type", "") organization: str = params.get("organization", "") project: str = params.get("project", "") demisto.debug(f"Command being called is {demisto.command()}") try: client = Client( base_url=base_url, api_key=api_key, verify=verify_certificate, proxy=proxy, vc_type=vc_type, organization=organization, project=project, ) if command == "test-module": return_results(test_module_command(client)) elif command == "circleci-workflows-list": return_results(circleci_workflows_list_command(client, args)) elif command == "circleci-artifacts-list": return_results(circleci_artifacts_list_command(client, args)) elif command == "circleci-workflow-jobs-list": return_results(circleci_workflow_jobs_list_command(client, args)) elif command == "circleci-workflow-last-runs": return_results(circleci_workflow_last_runs_command(client, args)) elif command == "circleci-trigger-workflow": return_results(circleci_trigger_workflow_command(client, args)) else: raise NotImplementedError(f'Command "{command}" is not implemented.') # Log exceptions and return errors except Exception as e: return_error(f"Failed to execute {demisto.command()} command.\nError:\n{e!s}") """ ENTRY POINT """ if __name__ in ("__main__", "__builtin__", "builtins"): main()