Arcanna.AI

Arcanna integration for using the power of AI in SOC.

Data Enrichment & Threat Intelligence · ArcannaAI

Details

IDArcanna.AI
ProviderArcanna.ai INC
CategoryData Enrichment & Threat Intelligence
From Version5.5.0
Docker Imagedemisto/python3:3.12.13.10116658
Supported ModulesAgentix XSIAM

README

Arcanna integration for using the power of AI in SOC
This integration was integrated and tested with version 1.45.1 of Arcanna.AI

Configure Arcanna.AI in Cortex

Parameter Description Required
Server URL (e.g. https://<your arcanna ai api>) URL of Arcanna API True
API Key Api Key for Arcanna API True
Trust any certificate (not secure)   False
Use system proxy settings   False
Default Arcanna Job Id   False

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

arcanna-get-jobs


Get jobs list

Base Command

arcanna-get-jobs

Input

There are no input arguments for this command.

Context Output

Path Type Description
Arcanna.Jobs.job_id Number Arcanna Job id
Arcanna.Jobs.data_type String Arcanna Job type
Arcanna.Jobs.title String Arcanna Job title
Arcanna.Jobs.status String Arcanna job status
Arcanna.Jobs.last_processed_timestamp date Last processed time

Command Example


#### Context Example

```json
{
    "Arcanna": {
        "Jobs": [
            {
                "data_type": "",
                "job_id": 1201,
                "status": "STARTED",
                "title": "dev1"
            },
            {
                "data_type": "",
                "job_id": 1202,
                "status": "STARTED",
                "title": "marian-demo"
            }
        ]
    }
}

Human Readable Output

Arcanna Jobs

job_id title data_type status
1201 dev1   STARTED
1202 marian-demo   STARTED

arcanna-send-event


Sends a raw event to Arcanna

Base Command

arcanna-send-event

Input

Argument Name Description Required
job_id An Arcanna running job_id. Optional
event_json json event for arcanna to inference. Required
title event title. Required
severity event severity. Optional

Context Output

Path Type Description
Arcanna.Event.event_id Number Arcanna event id
Arcanna.Event.status String Arcanna ingestion status
Arcanna.Event.ingest_timestamp date Arcanna ingestion timestamp
Arcanna.Event.error_message String Arcanna error message if any
Arcanna.Event.job_id Number An Arcanna Job id used for sending.

Command Example

!arcanna-send-event job_id=1202 event_json="{\"destination\":{ \"address\":\"10.128.0.6\", \"port\":53720, \"bytes\":836035, \"ip\":\"10.128.0.6\", \"packets\":6459},\"rule\":{ \"name\":\"SURICATA HTTP unable to match response to request\", \"id\":\"2221010\", \"category\":\"Generic Protocol Command Decode\"},\"source\":{ \"address\":\"8.8.8.8\", \"port\":9200, \"bytes\":4833843, \"ip\":\"8.8.8.8\", \"packets\":9677},\"event\":{ \"severity\":3, \"ingested\":\"2021-06-15T07:56:50.654225827Z\", \"created\":\"2021-06-15T07:56:49.649Z\", \"kind\":\"alert\", \"module\":\"suricata\", \"start\":\"2021-06-14T14:02:06.280Z\", \"category\":[ \"network\", \"intrusion_detection\" ], \"type\":[ \"allowed\" ], \"dataset\":\"suricata.eve\"},\"fileset\":{ \"name\":\"eve\"},\"message\":\"Generic Protocol Command Decode\",\"url\":{ \"path\":\"/libhtp::request_uri_not_seen\", \"original\":\"/libhtp::request_uri_not_seen\"},\"@timestamp\":\"2021-06-15T07:56:49.647Z\",\"suricata\":{ \"eve\":{ \"in_iface\":\"ens4\", \"metadata\":{ \"flowints\":{ \"http.anomaly.count\":2419 } }, \"event_type\":\"alert\", \"alert\":{ \"signature_id\":2221010, \"rev\":1, \"gid\":1, \"signature\":\"SURICATA HTTP unable to match response to request\", \"category\":\"Generic Protocol Command Decode\" }, \"flow_id\":576330410117303, \"tx_id\":3224, \"flow\":{ } }}}" title=Test_alert severity=3

Context Example

{
    "Arcanna": {
        "Event": {
            "error_message": "",
            "event_id": "12023636421762",
            "ingest_timestamp": "2021-09-02T09:46:22.363642Z",
            "job_id": 1202,
            "status": "Pending inference"
        }
    }
}

Human Readable Output

{‘event_id’: ‘12023636421762’, ‘job_id’: 1202, ‘ingest_timestamp’: ‘2021-09-02T09:46:22.363642Z’, ‘status’: ‘Pending inference’, ‘error_message’: ‘’}

arcanna-trigger-train


Trigger AI Train for specified Arcanna.ai Job.

Base Command

arcanna-trigger-train

Input

Argument Name Description Required
job_id Job ID to trigger train for. Required
username Username for audit. Required

Context Output

Path Type Description
Arcanna.Train.status String Action status.
Arcanna.Train.error_message String Message in case of error.

Command Example

!arcanna-trigger-train job_id="1201" username="admin"

Context Example

{
  "status": "OK",
  "error_message": ""
}

Human Readable Output

Arcanna trigger train results: {‘status’: ‘OK’, ‘error_message’: ‘’}

arcanna-get-decision-set


Retrieve avaiable decision points for specified AI Job.

Base Command

arcanna-get-decision-set

Input

Argument Name Description Required
job_id Job ID to use for exporting event. Required

Context Output

Path Type Description
Arcanna.Event.decision_set List Available decisions for specified AI Job.

Command Example

!arcanna-get-decision-set job_id="1201"

Context Example

{
  "decision_set": ["Drop", "Escalate"]
}

Human Readable Output

Arcanna get decision set results: {‘decision_set’: [‘Drop’,’Escalate’]}

arcanna-export-event


Export full event with metadata from Arcanna.ai based on specified Job ID and Event ID.

Base Command

arcanna-export-event

Input

Argument Name Description Required
job_id Job ID to use for exporting event. Required
event_id Event ID to use for exporting event. Required

Context Output

Path Type Description
Arcanna.Event.arcanna_event unknown Full export for specified event.

Command Example

!arcanna-export-event job_id="1201" event_id="12011938471583"

Context Example

{
  "result": "Escalate with Priority",
  "metadata": {
    "id": "12011938471583",
    ...
  }
  ... 
}

Human Readable Output

Arcanna export event results: {‘result’: ‘Escalate with Priority’, ‘metadata’: {‘id’: ‘12011938471583’, .. }, ..}

arcanna-get-event-status


Retrieves Arcanna Inference result.

Base Command

arcanna-get-event-status

Input

Argument Name Description Required
job_id Arcanna Job Id. Optional
event_id Arcanna generated unique event id. Required

Context Output

Path Type Description
Arcanna.Event.event_id String Arcanna event id
Arcanna.Event.ingest_timestamp String Arcanna ingestion timestamp.
Arcanna.Event.result String Arcanna event result
Arcanna.Event.error_message String Arcanna error message if any.
Arcanna.Event.status String Arcanna event status.
Arcanna.Event.result_label String Arcanna event result label
Arcanna.Event.is_duplicated boolean Deprecated. Arcanna signalling if event is duplicated by another alert.
Arcanna.Event.confidence_level Number Deprecated. Arcanna ML confidence_level
Arcanna.Event.confidence_score Number Arcanna ML confidence_score
Arcanna.Event.bucket_state String Flag to indicate the current event’s state in the AI Model
Arcanna.Event.outlier boolean Arcanna signalling if event is an outlier based on historical data

Command Example

!arcanna-get-event-status job_id="1201" event_id="12011938471583"

Context Example

{
    "Arcanna": {
        "Event": {
            "confidence_score": 0.9999464750289917,
            "error_message": null,
            "event_id": "12011938471583",
            "ingest_timestamp": "2021-08-26T12:53:47.193847Z",
            "result": "escalate_alert",
            "result_label": "Escalate",
            "outlier": False
            "status": "OK"
        }
    }
}

Human Readable Output

{‘event_id’: ‘12011938471583’, ‘ingest_timestamp’: ‘2021-08-26T12:53:47.193847Z’, ‘status’: ‘OK’, ‘confidence_score’: 0.9999464750289917, ‘result’: ‘escalate_alert’, ‘result_label’: ‘Escalate’, ‘outlier’: False, ‘error_message’: None}

arcanna-send-event-feedback


Send Arcanna feedback for a previous inferred event.

Base Command

arcanna-send-event-feedback

Input

Argument Name Description Required
job_id An Arcanna job id. Optional
event_id An Arcanna event id. Required
feedback An Arcanna feedback label. Required
username A username providing the feedback. Required
decision_set List of possible decisions to be used as feedback values. Required
closing_notes Deprecated. Prior used for audit. Optional
label Deprecated. Replaced by feedback. Required
indicators Deprecated. Prior used for metadata. Optional

Context Output

Path Type Description
Arcanna.Event.feedback_status String An Arcanna feedback status response.

Command Example

!arcanna-send-event-feedback job_id="1201" event_id="12011938471583" feedback="Escalate with Priority" username="dbot"

Context Example

{
    "Arcanna": {
        "Event": {
            "status": "updated"
        }
    }
}

Human Readable Output

Arcanna send event feedback results: {‘status’: ‘updated’}

Configuration parameters

  • url — Server URL (e.g. https://<your arcanna ai api>/ ) (required)
  • credentials — Authentication
  • apikey — API Key ( Deprecated )
  • ssl_verification — SSL Certificate verification
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings
  • feature_mapping — A closing reason to Arcanna labels (deprecated).
  • closing_reason_field — Alert closing reason
  • default_job_id — Default Arcanna Job Id

Commands (11)

  • arcanna-export-event

    Export full event with metadata from Arcanna.ai based on specified Job ID and Event ID.

  • arcanna-get-decision-set

    Retrieve avaiable decision points for specified AI Job.

  • arcanna-get-default-job-id Deprecated

    Retrieves Arcanna Default Job id.

  • arcanna-get-event-status

    Retrieves Arcanna Inference result.

  • arcanna-get-feedback-field Deprecated

    Returns the Feedback field set on integration.

  • arcanna-get-jobs

    Get jobs list.

  • arcanna-send-bulk-events Deprecated

    Send to Arcanna a bulk of events as a dict encoded JSON string filled with XSOAR fields.

  • arcanna-send-event

    Sends a raw event to Arcanna.

  • arcanna-send-event-feedback

    Send Arcanna feedback for a previous inferred event.

  • arcanna-set-default-job-id Deprecated

    Sets Arcanna Default Job id.

  • arcanna-trigger-train

    Trigger AI Train for specified Arcanna.ai Job.

import json
import traceback
from typing import Any

import demistomock as demisto  # noqa: F401
import urllib3
from CommonServerPython import *  # noqa: F401

from CommonServerUserPython import *  # noqa: F401

# Disable insecure warnings
urllib3.disable_warnings()

""" CLIENT CLASS """


class Client(BaseClient):
    """
    Implements Arcanna API
    """

    def test_arcanna(self) -> Dict[str, Any]:
        return self._http_request(method="GET", url_suffix="/api/v1/health/")

    def list_jobs(self):
        return self._http_request(method="GET", url_suffix="/api/v1/jobs/", ok_codes=(200, 201, 422))

    def export_event(self, job_id: int, event_id: str) -> Dict[str, Any]:
        return self._http_request(method="GET", url_suffix=f"/api/v1/events/{job_id}/{event_id}/export", ok_codes=(200, 201, 422))

    def trigger_training(self, job_id: int, username: str) -> Dict[str, Any]:
        return self._http_request(
            method="POST", url_suffix=f"/api/v1/jobs/{job_id}/train?username={username}", ok_codes=(200, 201, 422)
        )

    def get_decision_set(self, job_id: int) -> Dict[str, Any]:
        return self._http_request(method="GET", url_suffix=f"/api/v1/jobs/{job_id}/labels", ok_codes=(200, 201, 422))

    def send_raw_event(self, job_id: int, payload: Dict[str, Any], id_value="") -> Dict[str, Any]:
        return self._http_request(
            method="POST", url_suffix=f"/api/v1/events/{id_value}", json_data=payload, ok_codes=(200, 201, 422)
        )

    def get_event_status(self, job_id: int, event_id: str) -> Dict[str, Any]:
        return self._http_request(method="GET", url_suffix=f"/api/v1/events/{job_id}/{event_id}", ok_codes=(200, 201, 422))

    def send_feedback(self, job_id: int, event_id: str, payload: Dict[str, Any]) -> Dict[str, Any]:
        return self._http_request(
            method="PUT", url_suffix=f"/api/v1/events/{job_id}/{event_id}/feedback", json_data=payload, ok_codes=(200, 201, 422)
        )


""" COMMAND FUNCTIONS """


def get_default_job_id(client: Client) -> CommandResults:
    response = 1201
    readable_output = f"## {response}"

    return CommandResults(readable_output=readable_output, outputs_prefix="Arcanna.Default_Job_Id", outputs=response)


def set_default_job_id(client: Client, args: Dict[str, Any]) -> CommandResults:
    return get_default_job_id(client)


def send_bulk_events(client: Client, feature_mapping_field: Dict[str, Any], args: Dict[str, Any]) -> CommandResults:
    response = "mock"
    readable_output = f" ## Arcanna send bulk results: {response}"

    return CommandResults(
        readable_output=readable_output,
        outputs_prefix="Arcanna.Bulk",
        outputs_key_field="status",
        outputs={"status": "deprecated"},
    )


def get_feedback_field(params: Dict[str, Any]) -> CommandResults:
    response = "deprecated"
    readable_output = f" ## Get feedback returned results: {response}"

    return CommandResults(readable_output=readable_output, outputs_prefix="Arcanna.FeedbackField", outputs=response)


def test_module(client: Client) -> str:
    try:
        response = client.test_arcanna()
        demisto.info(f"test_module response={response}")
        if "status" in response and response.get("status", False):
            return "ok"
        else:
            return "Authentication Error. Please check the API Key you provided."
    except DemistoException as e:
        raise e


def get_jobs(client: Client) -> CommandResults:
    result = client.list_jobs()

    headers = ["job_id", "title", "status", "last_processed_timestamp"]

    readable_output = tableToMarkdown(name="Arcanna Jobs", headers=headers, t=result, removeNull=True)

    outputs = {"Arcanna.Jobs(val.job_id && val.job_id === obj.job_id)": createContext(result)}
    return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result)


def export_event(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults:
    job_id = args.get("job_id", default_job_id)
    event_id = str(args.get("event_id"))
    result = client.export_event(job_id=job_id, event_id=event_id)
    headers = ["arcanna_event", "status", "ingest_timestamp", "event_id", "error_message"]
    readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True)
    outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(result)}

    return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result)


def trigger_training(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults:
    job_id = args.get("job_id", default_job_id)
    username = args.get("username", None)
    result = client.trigger_training(job_id=job_id, username=username)
    headers = ["status", "error_message"]
    readable_output = tableToMarkdown(name="Arcanna.ai training outcome", headers=headers, t=result, removeNull=True)
    outputs = {"Arcanna.Training(val.job_id && val.job_id === obj.job_id)": createContext(result)}

    return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result)


def get_decision_set(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults:
    job_id = args.get("job_id", default_job_id)
    result = client.get_decision_set(job_id=job_id)
    headers = ["decision_set", "error_message"]
    readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True)
    outcome = {"decision_set": result}
    outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(outcome)}
    return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result)


def map_to_arcanna_raw_event(job_id, raw, severity, title):
    body = {"job_id": int(job_id), "title": title, "raw_body": json.loads(raw)}
    if severity:
        severity = int(float(severity))
        body["severity"] = severity
    return body


def post_event(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults:
    job_id = args.get("job_id", default_job_id)
    title = args.get("title")
    raw_payload = args.get("event_json", {})
    severity = args.get("severity", 0)
    id_value = args.get("id_value", "")

    payload = map_to_arcanna_raw_event(job_id, raw_payload, severity, title)
    result = client.send_raw_event(job_id=job_id, payload=payload, id_value=id_value)

    headers = ["event_id", "ingest_timestamp", "status", "error_message", "job_id"]
    readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True)

    outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(result)}

    return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result)


@polling_function(
    name="arcanna-get-event-status",
    timeout=arg_to_number(demisto.args().get("timeout", 120)),
    interval=arg_to_number(demisto.args().get("interval", 15)),
)
def get_event_status(args: Dict[str, Any], client: Client, default_job_id) -> CommandResults | PollResult:
    job_id = args.get("job_id", default_job_id)
    event_id = str(args.get("event_id"))
    result = client.get_event_status(job_id, event_id)
    polling = args.get("polling")
    successful_response = result.get("status", "pending_inference") != "pending_inference"
    outputs = {"Arcanna.Event": createContext(result)}
    headers = [
        "event_id",
        "ingest_timestamp",
        "status",
        "error_message",
        "bucket_state",
        "result",
        "confidence_score",
        "outlier",
        "arcanna_label",
    ]
    return_data = CommandResults(
        outputs_prefix="Arcanna.Event",
        outputs_key_field="event_id",
        outputs=outputs,
        readable_output=tableToMarkdown(name="Arcanna Event Status", headers=headers, t=result),
        raw_response=result,
    )
    if successful_response or polling == "false":
        return PollResult(response=return_data, continue_to_poll=False)
    else:
        return PollResult(continue_to_poll=True, response=return_data)


def map_to_arcanna_label(arcanna_label, username):
    body = {"cortex_user": username, "feedback": arcanna_label}
    return body


def send_event_feedback(client: Client, default_job_id: int, args: Dict[str, Any]) -> CommandResults:
    job_id = args.get("job_id", default_job_id)
    event_id = str(args.get("event_id"))
    username = str(args.get("username"))
    feedback = args.get("feedback", "<empty>")
    decision_set = args.get("valid_decisions", [])
    if feedback in decision_set is False:
        raise Exception(f"Error in arcanna-send-event-feedback.Unknown decision - {feedback} ; Valid options: {decision_set} ")

    payload = map_to_arcanna_label(feedback, username)
    result = client.send_feedback(job_id, event_id, payload)
    headers = ["feedback_status", "status", "details"]
    readable_output = tableToMarkdown(name="Arcanna Event", headers=headers, t=result, removeNull=True)
    outputs = {"Arcanna.Event(val.job_id && val.job_id === obj.job_id)": createContext(result)}

    return CommandResults(readable_output=readable_output, outputs=outputs, raw_response=result)


""" MAIN FUNCTION """


def main() -> None:
    """main function, parses params and runs command functions

    :return:
    :rtype:
    """
    api_key = demisto.params().get("credentials", {}).get("password") or demisto.params().get("apikey")

    # get the service API url
    base_url = urljoin(demisto.params()["url"])
    verify_certificate = demisto.params().get("ssl_verification", False)
    proxy = demisto.params().get("proxy", False)
    default_job_id = int(demisto.params().get("default_job_id", 1201))
    demisto.debug(f"Command being called is {demisto.command()}")
    headers = {"accept": "application/json", "x-arcanna-api-key": api_key}

    try:
        client = Client(base_url=base_url, headers=headers, verify=verify_certificate, proxy=proxy)
        # Deprecated. For BC purposes
        feature_mapping: Dict[str, Any] = {}
        if demisto.command() == "test-module":
            # This is the call made when pressing the integration Test button.
            result_test = test_module(client)
            return_results(result_test)
        elif demisto.command() == "arcanna-get-jobs":
            result_get_jobs = get_jobs(client)
            return_results(result_get_jobs)
        elif demisto.command() == "arcanna-send-event":
            result_send_event = post_event(client, default_job_id, demisto.args())
            return_results(result_send_event)
        elif demisto.command() == "arcanna-export-event":
            result_export_event = export_event(client, default_job_id, demisto.args())
            return_results(result_export_event)
        elif demisto.command() == "arcanna-trigger-train":
            result_trigger_train = trigger_training(client, default_job_id, demisto.args())
            return_results(result_trigger_train)
        elif demisto.command() == "arcanna-get-decision-set":
            result_decision_set = get_decision_set(client, default_job_id, demisto.args())
            return_results(result_decision_set)
        elif demisto.command() == "arcanna-get-event-status":
            result_get_event = get_event_status(demisto.args(), client, default_job_id)
            return_results(result_get_event)
        elif demisto.command() == "arcanna-send-event-feedback":
            result_send_feedback = send_event_feedback(client, default_job_id, demisto.args())
            return_results(result_send_feedback)
        elif demisto.command() == "arcanna-get-default-job-id":
            result_get_default_id = get_default_job_id(client)
            return_results(result_get_default_id)
        elif demisto.command() == "arcanna-set-default-job-id":
            result_set_default_id = set_default_job_id(client, demisto.args())
            return_results(result_set_default_id)
        elif demisto.command() == "arcanna-send-bulk-events":
            result_bulk = send_bulk_events(client, feature_mapping, demisto.args())
            return_results(result_bulk)
        elif demisto.command() == "arcanna-get-feedback-field":
            result_feedback_field = get_feedback_field(demisto.params())
            return_results(result_feedback_field)
    # Log exceptions and return errors
    except Exception as e:
        demisto.error(traceback.format_exc())  # print the traceback
        return_error(f"Failed to execute {demisto.command()} command.\nError:\n{e!s}")


""" ENTRY POINT """

if __name__ in ("__main__", "__builtin__", "builtins"):
    main()