Arcanna.AI
Arcanna integration for using the power of AI in SOC.
Data Enrichment & Threat Intelligence · ArcannaAI
Details
| ID | Arcanna.AI |
|---|---|
| Provider | Arcanna.ai INC |
| Category | Data Enrichment & Threat Intelligence |
| From Version | 5.5.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | Agentix XSIAM |
README
Arcanna integration for using the power of AI in SOC
This integration was integrated and tested with version 1.45.1 of Arcanna.AI
Configure Arcanna.AI in Cortex
| Parameter | Description | Required |
|---|---|---|
| Server URL (e.g. https://<your arcanna ai api>) | URL of Arcanna API | True |
| API Key | Api Key for Arcanna API | True |
| Trust any certificate (not secure) | False | |
| Use system proxy settings | False | |
| Default Arcanna Job Id | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
arcanna-get-jobs
Get jobs list
Base Command
arcanna-get-jobs
Input
There are no input arguments for this command.
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Jobs.job_id | Number | Arcanna Job id |
| Arcanna.Jobs.data_type | String | Arcanna Job type |
| Arcanna.Jobs.title | String | Arcanna Job title |
| Arcanna.Jobs.status | String | Arcanna job status |
| Arcanna.Jobs.last_processed_timestamp | date | Last processed time |
Command Example
#### Context Example
```json
{
"Arcanna": {
"Jobs": [
{
"data_type": "",
"job_id": 1201,
"status": "STARTED",
"title": "dev1"
},
{
"data_type": "",
"job_id": 1202,
"status": "STARTED",
"title": "marian-demo"
}
]
}
}
Human Readable Output
Arcanna Jobs
job_id title data_type status 1201 dev1 STARTED 1202 marian-demo STARTED
arcanna-send-event
Sends a raw event to Arcanna
Base Command
arcanna-send-event
Input
| Argument Name | Description | Required |
|---|---|---|
| job_id | An Arcanna running job_id. | Optional |
| event_json | json event for arcanna to inference. | Required |
| title | event title. | Required |
| severity | event severity. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Event.event_id | Number | Arcanna event id |
| Arcanna.Event.status | String | Arcanna ingestion status |
| Arcanna.Event.ingest_timestamp | date | Arcanna ingestion timestamp |
| Arcanna.Event.error_message | String | Arcanna error message if any |
| Arcanna.Event.job_id | Number | An Arcanna Job id used for sending. |
Command Example
!arcanna-send-event job_id=1202 event_json="{\"destination\":{ \"address\":\"10.128.0.6\", \"port\":53720, \"bytes\":836035, \"ip\":\"10.128.0.6\", \"packets\":6459},\"rule\":{ \"name\":\"SURICATA HTTP unable to match response to request\", \"id\":\"2221010\", \"category\":\"Generic Protocol Command Decode\"},\"source\":{ \"address\":\"8.8.8.8\", \"port\":9200, \"bytes\":4833843, \"ip\":\"8.8.8.8\", \"packets\":9677},\"event\":{ \"severity\":3, \"ingested\":\"2021-06-15T07:56:50.654225827Z\", \"created\":\"2021-06-15T07:56:49.649Z\", \"kind\":\"alert\", \"module\":\"suricata\", \"start\":\"2021-06-14T14:02:06.280Z\", \"category\":[ \"network\", \"intrusion_detection\" ], \"type\":[ \"allowed\" ], \"dataset\":\"suricata.eve\"},\"fileset\":{ \"name\":\"eve\"},\"message\":\"Generic Protocol Command Decode\",\"url\":{ \"path\":\"/libhtp::request_uri_not_seen\", \"original\":\"/libhtp::request_uri_not_seen\"},\"@timestamp\":\"2021-06-15T07:56:49.647Z\",\"suricata\":{ \"eve\":{ \"in_iface\":\"ens4\", \"metadata\":{ \"flowints\":{ \"http.anomaly.count\":2419 } }, \"event_type\":\"alert\", \"alert\":{ \"signature_id\":2221010, \"rev\":1, \"gid\":1, \"signature\":\"SURICATA HTTP unable to match response to request\", \"category\":\"Generic Protocol Command Decode\" }, \"flow_id\":576330410117303, \"tx_id\":3224, \"flow\":{ } }}}" title=Test_alert severity=3
Context Example
{
"Arcanna": {
"Event": {
"error_message": "",
"event_id": "12023636421762",
"ingest_timestamp": "2021-09-02T09:46:22.363642Z",
"job_id": 1202,
"status": "Pending inference"
}
}
}
Human Readable Output
{‘event_id’: ‘12023636421762’, ‘job_id’: 1202, ‘ingest_timestamp’: ‘2021-09-02T09:46:22.363642Z’, ‘status’: ‘Pending inference’, ‘error_message’: ‘’}
arcanna-trigger-train
Trigger AI Train for specified Arcanna.ai Job.
Base Command
arcanna-trigger-train
Input
| Argument Name | Description | Required |
|---|---|---|
| job_id | Job ID to trigger train for. | Required |
| username | Username for audit. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Train.status | String | Action status. |
| Arcanna.Train.error_message | String | Message in case of error. |
Command Example
!arcanna-trigger-train job_id="1201" username="admin"
Context Example
{
"status": "OK",
"error_message": ""
}
Human Readable Output
Arcanna trigger train results: {‘status’: ‘OK’, ‘error_message’: ‘’}
arcanna-get-decision-set
Retrieve avaiable decision points for specified AI Job.
Base Command
arcanna-get-decision-set
Input
| Argument Name | Description | Required |
|---|---|---|
| job_id | Job ID to use for exporting event. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Event.decision_set | List | Available decisions for specified AI Job. |
Command Example
!arcanna-get-decision-set job_id="1201"
Context Example
{
"decision_set": ["Drop", "Escalate"]
}
Human Readable Output
Arcanna get decision set results: {‘decision_set’: [‘Drop’,’Escalate’]}
arcanna-export-event
Export full event with metadata from Arcanna.ai based on specified Job ID and Event ID.
Base Command
arcanna-export-event
Input
| Argument Name | Description | Required |
|---|---|---|
| job_id | Job ID to use for exporting event. | Required |
| event_id | Event ID to use for exporting event. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Event.arcanna_event | unknown | Full export for specified event. |
Command Example
!arcanna-export-event job_id="1201" event_id="12011938471583"
Context Example
{
"result": "Escalate with Priority",
"metadata": {
"id": "12011938471583",
...
}
...
}
Human Readable Output
Arcanna export event results: {‘result’: ‘Escalate with Priority’, ‘metadata’: {‘id’: ‘12011938471583’, .. }, ..}
arcanna-get-event-status
Retrieves Arcanna Inference result.
Base Command
arcanna-get-event-status
Input
| Argument Name | Description | Required |
|---|---|---|
| job_id | Arcanna Job Id. | Optional |
| event_id | Arcanna generated unique event id. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Event.event_id | String | Arcanna event id |
| Arcanna.Event.ingest_timestamp | String | Arcanna ingestion timestamp. |
| Arcanna.Event.result | String | Arcanna event result |
| Arcanna.Event.error_message | String | Arcanna error message if any. |
| Arcanna.Event.status | String | Arcanna event status. |
| Arcanna.Event.result_label | String | Arcanna event result label |
| Arcanna.Event.is_duplicated | boolean | Deprecated. Arcanna signalling if event is duplicated by another alert. |
| Arcanna.Event.confidence_level | Number | Deprecated. Arcanna ML confidence_level |
| Arcanna.Event.confidence_score | Number | Arcanna ML confidence_score |
| Arcanna.Event.bucket_state | String | Flag to indicate the current event’s state in the AI Model |
| Arcanna.Event.outlier | boolean | Arcanna signalling if event is an outlier based on historical data |
Command Example
!arcanna-get-event-status job_id="1201" event_id="12011938471583"
Context Example
{
"Arcanna": {
"Event": {
"confidence_score": 0.9999464750289917,
"error_message": null,
"event_id": "12011938471583",
"ingest_timestamp": "2021-08-26T12:53:47.193847Z",
"result": "escalate_alert",
"result_label": "Escalate",
"outlier": False
"status": "OK"
}
}
}
Human Readable Output
{‘event_id’: ‘12011938471583’, ‘ingest_timestamp’: ‘2021-08-26T12:53:47.193847Z’, ‘status’: ‘OK’, ‘confidence_score’: 0.9999464750289917, ‘result’: ‘escalate_alert’, ‘result_label’: ‘Escalate’, ‘outlier’: False, ‘error_message’: None}
arcanna-send-event-feedback
Send Arcanna feedback for a previous inferred event.
Base Command
arcanna-send-event-feedback
Input
| Argument Name | Description | Required |
|---|---|---|
| job_id | An Arcanna job id. | Optional |
| event_id | An Arcanna event id. | Required |
| feedback | An Arcanna feedback label. | Required |
| username | A username providing the feedback. | Required |
| decision_set | List of possible decisions to be used as feedback values. | Required |
| closing_notes | Deprecated. Prior used for audit. | Optional |
| label | Deprecated. Replaced by feedback. |
Required |
| indicators | Deprecated. Prior used for metadata. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| Arcanna.Event.feedback_status | String | An Arcanna feedback status response. |
Command Example
!arcanna-send-event-feedback job_id="1201" event_id="12011938471583" feedback="Escalate with Priority" username="dbot"
Context Example
{
"Arcanna": {
"Event": {
"status": "updated"
}
}
}
Human Readable Output
Arcanna send event feedback results: {‘status’: ‘updated’}
Configuration parameters
url— Server URL (e.g. https://<your arcanna ai api>/ ) (required)credentials— Authenticationapikey— API Key ( Deprecated )ssl_verification— SSL Certificate verificationinsecure— Trust any certificate (not secure)proxy— Use system proxy settingsfeature_mapping— A closing reason to Arcanna labels (deprecated).closing_reason_field— Alert closing reasondefault_job_id— Default Arcanna Job Id
Commands (11)
-
arcanna-export-eventExport full event with metadata from Arcanna.ai based on specified Job ID and Event ID.
-
arcanna-get-decision-setRetrieve avaiable decision points for specified AI Job.
-
arcanna-get-default-job-idDeprecatedRetrieves Arcanna Default Job id.
-
arcanna-get-event-statusRetrieves Arcanna Inference result.
-
arcanna-get-feedback-fieldDeprecatedReturns the Feedback field set on integration.
-
arcanna-get-jobsGet jobs list.
-
arcanna-send-bulk-eventsDeprecatedSend to Arcanna a bulk of events as a dict encoded JSON string filled with XSOAR fields.
-
arcanna-send-eventSends a raw event to Arcanna.
-
arcanna-send-event-feedbackSend Arcanna feedback for a previous inferred event.
-
arcanna-set-default-job-idDeprecatedSets Arcanna Default Job id.
-
arcanna-trigger-trainTrigger AI Train for specified Arcanna.ai Job.
import json import traceback from typing import Any import demistomock as demisto # noqa: F401 import urllib3 from CommonServerPython import * # noqa: F401 from CommonServerUserPython import * # noqa: F401 # Disable insecure warnings urllib3.disable_warnings() """ CLIENT CLASS """ class Client(BaseClient): """ Implements Arcanna API """ def test_arcanna(self) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix="/api/v1/health/") def list_jobs(self): return self._http_request(method="GET", url_suffix="/api/v1/jobs/", ok_codes=(200, 201, 422)) def export_event(self, job_id: int, event_id: str) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix=f"/api/v1/events/{job_id}/{event_id}/export", ok_codes=(200, 201, 422)) def trigger_training(self, job_id: int, username: str) -> Dict[str, Any]: return self._http_request( method="POST", url_suffix=f"/api/v1/jobs/{job_id}/train?username={username}", ok_codes=(200, 201, 422) ) def get_decision_set(self, job_id: int) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix=f"/api/v1/jobs/{job_id}/labels", ok_codes=(200, 201, 422)) def send_raw_event(self, job_id: int, payload: Dict[str, Any], id_value="") -> Dict[str, Any]: return self._http_request( method="POST", url_suffix=f"/api/v1/events/{id_value}", json_data=payload, ok_codes=(200, 201, 422) ) def get_event_status(self, job_id: int, event_id: str) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix=f"/api/v1/events/{job_id}/{event_id}", ok_codes=(200, 201, 422)) def send_feedback(self, job_id: int, event_id: str, payload: Dict[str, Any]) -> Dict[str, Any]: return self._http_request( method="PUT", url_suffix=f"/api/v1/events/{job_id}/{event_id}/feedback", json_data=payload, ok_codes=(200, 201, 422) ) """ COMMAND FUNCTIONS """ def get_default_job_id(client: Client) -> CommandResults: response = 1201 readable_output = f"## {response}" return CommandResults(readable_output=readable_output, outputs_prefix="Arcanna.Default_Job_Id", outputs=response) def set_default_job_id(client: Client, args: Dict[str, Any]) -> CommandResults: return get_default_job_id(client) def send_bulk_events(client: Client, feature_mapping_field: Dict[str, Any], args: Dict[str, Any]) -> CommandResults: response = "mock" readable_output = f" ## Arcanna send bulk results: {response}" return CommandResults( readable_output=readable_output, outputs_prefix="Arcanna.Bulk", outputs_key_field="status", outputs={"status": "deprecated"}, ) def get_feedback_field(params: Dict[str, Any]) -> CommandResults: response = "deprecated" readable_output = f" ## Get feedback returned results: {response}" return CommandResults(readable_output=readable_output, outputs_prefix="Arcanna.FeedbackField", outputs=response) def test_module(client: Client) -> str: try: response = client.test_arcanna() demisto.info(f"test_module response={response}") if "status" in response and response.get("status", False): return "ok" else: return "Authentication Error. Please check the API Key you provided." except DemistoException as e: raise e def get_jobs(client: Client) -> CommandResults: result = client.list_jobs() headers = ["job_id", "title", "status", "last_processed_timestamp"] readable_output = tableToMarkdown(name="Arcanna Jobs", headers=headers, t=result, removeNull=True) outputs = {"Arcanna.Jobs(val.job_id && val.job_id === obj.job_id)": createContext(result)} return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result) def export_event(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults: job_id = args.get("job_id", default_job_id) event_id = str(args.get("event_id")) result = client.export_event(job_id=job_id, event_id=event_id) headers = ["arcanna_event", "status", "ingest_timestamp", "event_id", "error_message"] readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True) outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(result)} return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result) def trigger_training(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults: job_id = args.get("job_id", default_job_id) username = args.get("username", None) result = client.trigger_training(job_id=job_id, username=username) headers = ["status", "error_message"] readable_output = tableToMarkdown(name="Arcanna.ai training outcome", headers=headers, t=result, removeNull=True) outputs = {"Arcanna.Training(val.job_id && val.job_id === obj.job_id)": createContext(result)} return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result) def get_decision_set(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults: job_id = args.get("job_id", default_job_id) result = client.get_decision_set(job_id=job_id) headers = ["decision_set", "error_message"] readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True) outcome = {"decision_set": result} outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(outcome)} return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result) def map_to_arcanna_raw_event(job_id, raw, severity, title): body = {"job_id": int(job_id), "title": title, "raw_body": json.loads(raw)} if severity: severity = int(float(severity)) body["severity"] = severity return body def post_event(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults: job_id = args.get("job_id", default_job_id) title = args.get("title") raw_payload = args.get("event_json", {}) severity = args.get("severity", 0) id_value = args.get("id_value", "") payload = map_to_arcanna_raw_event(job_id, raw_payload, severity, title) result = client.send_raw_event(job_id=job_id, payload=payload, id_value=id_value) headers = ["event_id", "ingest_timestamp", "status", "error_message", "job_id"] readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True) outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(result)} return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result) @polling_function( name="arcanna-get-event-status", timeout=arg_to_number(demisto.args().get("timeout", 120)), interval=arg_to_number(demisto.args().get("interval", 15)), ) def get_event_status(args: Dict[str, Any], client: Client, default_job_id) -> CommandResults | PollResult: job_id = args.get("job_id", default_job_id) event_id = str(args.get("event_id")) result = client.get_event_status(job_id, event_id) polling = args.get("polling") successful_response = result.get("status", "pending_inference") != "pending_inference" outputs = {"Arcanna.Event": createContext(result)} headers = [ "event_id", "ingest_timestamp", "status", "error_message", "bucket_state", "result", "confidence_score", "outlier", "arcanna_label", ] return_data = CommandResults( outputs_prefix="Arcanna.Event", outputs_key_field="event_id", outputs=outputs, readable_output=tableToMarkdown(name="Arcanna Event Status", headers=headers, t=result), raw_response=result, ) if successful_response or polling == "false": return PollResult(response=return_data, continue_to_poll=False) else: return PollResult(continue_to_poll=True, response=return_data) def map_to_arcanna_label(arcanna_label, username): body = {"cortex_user": username, "feedback": arcanna_label} return body def send_event_feedback(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults: job_id = args.get("job_id", default_job_id) event_id = str(args.get("event_id")) username = str(args.get("username")) feedback = args.get("feedback", "<empty>") decision_set = args.get("valid_decisions", []) if feedback in decision_set is False: raise Exception(f"Error in arcanna-send-event-feedback.Unknown decision - {feedback} ; Valid options: {decision_set} ") payload = map_to_arcanna_label(feedback, username) result = client.send_feedback(job_id, event_id, payload) headers = ["feedback_status", "status", "details"] readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True) outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(result)} return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result) """ MAIN FUNCTION """ def main() -> None: """main function, parses params and runs command functions :return: :rtype: """ api_key = demisto.params().get("credentials", {}).get("password") or demisto.params().get("apikey") # get the service API url base_url = urljoin(demisto.params()["url"]) verify_certificate = demisto.params().get("ssl_verification", False) proxy = demisto.params().get("proxy", False) default_job_id = int(demisto.params().get("default_job_id", 1201)) demisto.debug(f"Command being called is {demisto.command()}") headers = {"accept": "application/json", "x-arcanna-api-key": api_key} try: client = Client(base_url=base_url, headers=headers, verify=verify_certificate, proxy=proxy) # Deprecated. For BC purposes feature_mapping: Dict[str, Any] = {} if demisto.command() == "test-module": # This is the call made when pressing the integration Test button. result_test = test_module(client) return_results(result_test) elif demisto.command() == "arcanna-get-jobs": result_get_jobs = get_jobs(client) return_results(result_get_jobs) elif demisto.command() == "arcanna-send-event": result_send_event = post_event(client, default_job_id, demisto.args()) return_results(result_send_event) elif demisto.command() == "arcanna-export-event": result_export_event = export_event(client, default_job_id, demisto.args()) return_results(result_export_event) elif demisto.command() == "arcanna-trigger-train": result_trigger_train = trigger_training(client, default_job_id, demisto.args()) return_results(result_trigger_train) elif demisto.command() == "arcanna-get-decision-set": result_decision_set = get_decision_set(client, default_job_id, demisto.args()) return_results(result_decision_set) elif demisto.command() == "arcanna-get-event-status": result_get_event = get_event_status(demisto.args(), client, default_job_id) return_results(result_get_event) elif demisto.command() == "arcanna-send-event-feedback": result_send_feedback = send_event_feedback(client, default_job_id, demisto.args()) return_results(result_send_feedback) elif demisto.command() == "arcanna-get-default-job-id": result_get_default_id = get_default_job_id(client) return_results(result_get_default_id) elif demisto.command() == "arcanna-set-default-job-id": result_set_default_id = set_default_job_id(client, demisto.args()) return_results(result_set_default_id) elif demisto.command() == "arcanna-send-bulk-events": result_bulk = send_bulk_events(client, feature_mapping, demisto.args()) return_results(result_bulk) elif demisto.command() == "arcanna-get-feedback-field": result_feedback_field = get_feedback_field(demisto.params()) return_results(result_feedback_field) # Log exceptions and return errors except Exception as e: demisto.error(traceback.format_exc()) # print the traceback return_error(f"Failed to execute {demisto.command()} command.\nError:\n{e!s}") """ ENTRY POINT """ if __name__ in ("__main__", "__builtin__", "builtins"): main()