Cybersixgill_Actionable_Alerts

Cybersixgill automatically collects intelligence in real-time on all items that appear in the underground sources which we monitor. By using various rules and machine learning models, Cybersixgill automatically correlates these intelligence items with pre defined organization assets, and automatically alerts users in real time of any relevant intelligence items.

Data Enrichment & Threat Intelligence · Cybersixgill Actionable Alerts

Details

ID	`Cybersixgill_Actionable_Alerts`
Provider	Bitsight
Category	Data Enrichment & Threat Intelligence
From Version	5.0.0
Docker Image	`demisto/sixgill:1.0.0.10120494`
Supported Modules	Agentix XSIAM

README

Cybersixgill automatically collects intelligence in real-time on all items that appear in the underground sources which we monitor. By using various rules and machine learning models, Cybersixgill automatically correlates these intelligence items with pre defined
organization assets, and automatically alerts users in real time of any relevant intelligence items.

The integration will focus on retrieving Cybersixgill’s Actionable Alerts as incidents

Use Cases

Fetch Incidents & Events

Configure Cybersixgill on XSOAR

Parameter	Description	Required
client_id	Cybersixgill API client ID	True
client_secret	Cybersixgill API client secret	True
threat_level	Filter by alert threat level	False
threat_type	Filter by alert threat type	False

Fetch incidents

You can execute these commands from the XSOAR CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

output

[{
'name': "<alert name>",
'occurred': '<occurred>',
'details': '<details>',
'severity': <severity>,
'rawJSON': '{
    "alert_name": "<alert name>",
    "category": "regular",
    "content": "<some content>",
    "date": "<date>",
    "id": "<id>",
    "lang": "English",
    "langcode": "en",
    "read": false,
    "threat_level": "imminent",
    "threats": ["Fraud"],
    "title": "<title>",
    "user_id": "<id>",
    "sixgill_severity": 10}'
}]

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

cybersixgill-update-alert-status

updates the existing actionable alert status

Base Command

cybersixgill-update-alert-status

Input

Argument Name	Description	Required
alert_id	The alert id to update.	Required
alert_status	The new status.	Required
aggregate_alert_id	The aggregate alert id.	Optional

Context Output

There is no context output for this command.

Additional Information

Configuration parameters

client_id — Cybersixgill API client ID (required)
client_secret — Cybersixgill API client secret (required)
org_id — Cybersixgill Organization ID
max_fetch — Maximum number of incidents to fetch - maximum is 25
first_fetch_days — How many days back to fetch incidents on the first run - maximum is 30
threat_level — Filter by alert threat level
threat_type — Filter by alert threat type
proxy — Use system proxy settings
insecure — Trust any certificate (not secure)
isFetch — Fetch incidents
incidentType — Incident type
incidentFetchInterval — Incidents Fetch Interval

Commands (1)

cybersixgill-update-alert-status
updates the existing actionable alert status.

Source

CybersixgillActionableAlerts.py CybersixgillActionableAlerts_test.py CybersixgillActionableAlerts.yml CybersixgillActionableAlerts_description.md

Download History Raw

import demistomock as demisto
from CommonServerPython import *

from CommonServerUserPython import *

""" IMPORTS """

import copy
import json

import requests
import urllib3
from sixgill.sixgill_actionable_alert_client import SixgillActionableAlertClient
from sixgill.sixgill_request_classes.sixgill_auth_request import SixgillAuthRequest

# Disable insecure warnings
urllib3.disable_warnings()

""" GLOBALS/PARAMS """

CHANNEL_CODE = "7457a04d972fceb8e0cc2192ba4abc66" if is_xsiam() else "7698e8287dfde53dcd13082be750a85a"
MAX_INCIDENTS = 25
DEFAULT_INCIDENTS = "25"
MAX_DAYS_BACK = 30
DEFAULT_DAYS_BACK = "1"
DATETIME_FORMAT = "%Y-%m-%d %H:%M:%S"
DEMISTO_DATETIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%fZ"
THREAT_LEVEL_TO_SEVERITY = {"imminent": 3, "emerging": 2, "unknown": 0}
TO_DEMISTO_STATUS = {"in_treatment": 1, "resolved": 2, "treatment_required": 0}
VERIFY = not demisto.params().get("insecure", True)
SESSION = requests.Session()

""" HELPER FUNCTIONS """


def get_incident_init_params():
    params_dict = {
        "threat_level": demisto.params().get("threat_level", None),
        "threat_type": demisto.params().get("threat_type", None),
    }
    return {param_k: param_v for param_k, param_v in params_dict.items() if param_v}


def item_to_incidents(item_info, sixgill_alerts_client):
    incident: Dict[str, Any] = {}
    incidents = []
    items = []
    org = demisto.params().get("org_id", "")
    # get fields that are shared in case of sub alerts
    add_sub_alerts_shared_fields(incident, item_info)
    sub_alerts = item_info.pop("sub_alerts", None)
    if sub_alerts:
        alert_id = item_info.get("id", "")
        # add any sub alert as incident
        for sub_alert in sub_alerts:
            aggregate_alert_id = sub_alert.get("aggregate_alert_id", "")
            sub_item = copy.deepcopy(item_info)
            sub_item.update(sub_alert)
            alert_post_url = (
                f"https://portal.cybersixgill.com/#/alerts?actionable_alert_content_id={alert_id}"
                f"&aggregatedIndex={aggregate_alert_id}&filters.alert_id={alert_id}"
            )
            if org:
                alert_post_url += f"&org={org}"
            sub_item["alert_post_url"] = alert_post_url
            items.append(sub_item)
    else:
        items.append(item_info)
    for item in items:
        sub_incident = copy.deepcopy(incident)
        # add all other fields
        add_sub_alerts_fields(sub_incident, item, sixgill_alerts_client)
        item["org_id"] = demisto.params().get("org_id", "")
        sub_incident["rawJSON"] = json.dumps(item)
        incidents.append(sub_incident)
    return incidents


def add_sub_alerts_shared_fields(incident, item_info):
    incident["name"] = item_info.get("title", "Cybersixgill Alert")
    incident_date = datetime.strptime(item_info.get("date"), DATETIME_FORMAT)
    incident["occurred"] = incident_date.strftime(DEMISTO_DATETIME_FORMAT)
    incident["severity"] = THREAT_LEVEL_TO_SEVERITY[item_info.get("threat_level", "unknown")]
    org = demisto.params().get("org_id", "")
    alert_id = item_info.get("id", "")
    incident_link = f"https://portal.cybersixgill.com/#/alerts?filters.alert_id={alert_id}"
    if org:
        incident_link += f"&org={org}"

    incident["CustomFields"] = {
        "cybersixgillthreatlevel": item_info.get("threat_level", "unknown"),
        "cybersixgillthreattype": item_info.get("threats", []),
        "cybersixgillassessment": item_info.get("assessment", None),
        "cybersixgillrecommendations": "\n\n-----------\n\n".join(item_info.get("recommendations", [])),
        "incidentlink": incident_link,
        "cve": None,
        "cybersixgillattributes": None,
    }


def add_sub_alerts_fields(incident, item_info, sixgill_alerts_client):
    status = item_info.get("status", {}).get("name", "treatment_required")
    incident["status"] = TO_DEMISTO_STATUS[status]

    content_item = {"creator": None, "title": "", "content": "", "description": item_info.get("description", "")}
    try:
        get_alert_content(content_item, item_info, incident, sixgill_alerts_client)
    except Exception as e:
        demisto.error(f"Could not get alert content: {e}")
    detail_sections = (
        str(content_item.get("description", "")),
        str(content_item.get("title", "")),
        str(content_item.get("content", "")),
        *content_item.get("entries", []),
    )
    incident["details"] = "\n\n".join(section for section in detail_sections if section)
    triggered_assets = []
    for key, value in item_info.get("additional_info", {}).items():
        if "matched_" in key:
            triggered_assets.extend(value)
    incident["CustomFields"].update(
        {
            "cybersixgillstatus": status.replace("_", " ").title(),
            "cybersixgillsite": item_info.get("site", None),
            "cybersixgillactor": content_item.get("creator", None),
            "cybersixgilltriggeredassets": triggered_assets,
        }
    )


def format_content_item_entry(item: dict) -> list[str]:
    """Format a single alert content item into detail lines, based on the alert type it belongs to."""
    if item.get("Additional Keywords") is not None:  # Github Alert
        return [
            "Repository name: " + item.get("Repository name", ""),
            "Customer Keywords: " + item.get("Customer Keywords", ""),
            "GitURL: " + item.get("URL", ""),
        ]
    if item.get("Actor") is not None:  # Compromised Alerts
        return [
            "Actor: " + item.get("Actor", ""),
            "BIN: " + item.get("BIN", ""),
            "Site: " + item.get("Site", ""),
            "Text: " + item.get("Text", ""),
        ]
    if item.get("Detection time") is not None:  # Phishing Alerts
        return [
            "Detection time: " + item.get("Detection time", ""),
            "IP addresses: " + item.get("IP addresses", ""),
            "Suspicious domain: " + item.get("Suspicious domain", ""),
            "Triggered domain: " + item.get("Triggered domain", ""),
        ]
    if item.get("breach_date") is not None:  # Leaked Credentials Alerts
        return [
            "Already Seen: " + item.get("already_seen", ""),
            "Breach Date: " + item.get("breach_date", ""),
            "Description: " + item.get("description", ""),
            "Email: " + item.get("email", ""),
            "Created Time: " + item.get("create_time", ""),
        ]
    return []


def get_alert_content(content_item, item_info, incident, sixgill_alerts_client):
    # cve alert
    cve_id = item_info.get("additional_info").get("cve_id")
    es_id = item_info.get("es_id")
    if cve_id:
        content_item["content"] = f"https://portal.cybersixgill.com/#/cve/{cve_id}"
        additional_info = item_info.get("additional_info", {})
        incident["CustomFields"]["cve"] = cve_id
        attributes = []
        for attribute in additional_info.get("attributes", []):
            if attribute.get("value", False):
                attributes.append(additional_info.get("description"))
        attributes = "\n\n-----------\n\n".join(attributes)
        incident["CustomFields"]["cybersixgillattributes"] = attributes
    elif es_id == "Not Applicable":
        content = sixgill_alerts_client.get_actionable_alert_content(
            actionable_alert_id=item_info.get("id"),
            fetch_only_current_item=True,
            organization_id=demisto.params().get("org_id", None),
        )
        content_items = content.get("items")
        if content_items:
            entries = ("\n".join(format_content_item_entry(item)) for item in content_items)
            content_item["entries"] = [entry for entry in entries if entry]
    else:
        aggregate_alert_id = item_info.get("aggregate_alert_id", None)
        if not isinstance(aggregate_alert_id, int):
            aggregate_alert_id = None
        content = sixgill_alerts_client.get_actionable_alert_content(
            actionable_alert_id=item_info.get("id"),
            aggregate_alert_id=aggregate_alert_id,
            fetch_only_current_item=True,
            organization_id=demisto.params().get("org_id", None),
        )
        # get item full content
        content = content.get("items", None)
        if content and content[0].get("_id"):
            es_items = content[0].get("_source")
            if es_items:
                content_item["title"] = es_items.get("title")
                content_item["content"] = es_items.get("content")
                content_item["creator"] = es_items.get("creator")


""" COMMANDS + REQUESTS FUNCTIONS """


def test_module():
    """
    Performs basic Auth request
    """
    response = SESSION.send(
        request=SixgillAuthRequest(demisto.params()["client_id"], demisto.params()["client_secret"], CHANNEL_CODE).prepare(),
        verify=VERIFY,
    )
    if not response.ok:
        raise Exception("Auth request failed - please verify client_id, and client_secret.")


def fetch_incidents():
    last_run = demisto.getLastRun()

    if "last_fetch_time" in last_run:
        last_fetch_time = last_run["last_fetch_time"]
        demisto.info(f"Found last run, fetching new alerts from {last_fetch_time}")
    else:
        days_back = int(demisto.params().get("first_fetch_days", DEFAULT_DAYS_BACK))
        if days_back > MAX_DAYS_BACK:
            demisto.info(f"Days back({days_back}) is larger than the maximum, setting to {MAX_DAYS_BACK}")
            days_back = MAX_DAYS_BACK
        last_fetch_time = (datetime.now() - timedelta(days=days_back)).strftime(DATETIME_FORMAT)
        demisto.info(f"First run, fetching alerts from {last_fetch_time}")

    max_incidents_to_return = int(demisto.params().get("max_fetch", DEFAULT_INCIDENTS))
    if max_incidents_to_return > MAX_INCIDENTS:
        demisto.info(f"Max incidents({max_incidents_to_return}) is larger than the maximum, setting to {MAX_INCIDENTS}")
        max_incidents_to_return = MAX_INCIDENTS

    sixgill_alerts_client = SixgillActionableAlertClient(
        client_id=demisto.params()["client_id"],
        client_secret=demisto.params()["client_secret"],
        channel_id=CHANNEL_CODE,
        logger=demisto,
        session=SESSION,
        verify=VERIFY,
        num_of_attempts=3,
    )

    filter_alerts_kwargs = get_incident_init_params()
    items = sixgill_alerts_client.get_actionable_alerts_bulk(
        limit=max_incidents_to_return,
        from_date=last_fetch_time,
        sort_order="asc",
        **filter_alerts_kwargs,
        organization_id=demisto.params().get("org_id", None),
    )
    if len(items) > 0:
        demisto.info(f"Found {len(items)} new alerts since {last_fetch_time}")

        # getting more info about oldest ~max_incidents_to_return(can be more because of sub alerts)
        newest_incident_date = items[-1].get("date")
        incidents = []
        for item in items:
            try:
                item_info = sixgill_alerts_client.get_actionable_alert(
                    actionable_alert_id=item.get("id"), organization_id=demisto.params().get("org_id", None)
                )
                item_info["date"] = item.get("date")
                new_incidents = item_to_incidents(item_info, sixgill_alerts_client)
                incidents.extend(new_incidents)
                # can increase because of sub alerts
                if len(incidents) >= max_incidents_to_return:
                    newest_incident_date = item.get("date")
                    break
            except Exception as e:
                demisto.error(f"Could not get alert info: {e}")

        if len(incidents) > 0:
            demisto.info(f"Adding {len(incidents)} to demisto")
            demisto.incidents(incidents)

            demisto.info(f"Update last fetch time to: {newest_incident_date}")
            demisto.setLastRun({"last_fetch_time": newest_incident_date})
    else:
        demisto.info(f"No new alerts since {last_fetch_time}")
        demisto.incidents([])


def update_alert_status():
    """
    Updates the actionable alert status.
    """
    args = demisto.args()
    alert_status = args.get("alert_status")
    alert_id = args.get("alert_id")
    aggregate_alert_id = args.get("aggregate_alert_id")
    demisto.info(f"update_alert_status: status- {alert_status}, alert_id - {alert_id}, aggregate_alert_id - {aggregate_alert_id}")
    aggregate_alert_id = [int(aggregate_alert_id)] if aggregate_alert_id else aggregate_alert_id
    alert_body = {"status": {"status": alert_status}}

    sixgill_alerts_client = SixgillActionableAlertClient(
        client_id=demisto.params()["client_id"],
        client_secret=demisto.params()["client_secret"],
        channel_id=CHANNEL_CODE,
        logger=demisto,
        session=SESSION,
        verify=VERIFY,
    )

    res = sixgill_alerts_client.update_actionable_alert(
        actionable_alert_id=alert_id,
        json_body=alert_body,
        sub_alert_indexes=aggregate_alert_id,
        organization_id=demisto.params().get("org_id", None),
    )

    if res.get("status") == 200:
        demisto.results("Actionable alert status updated")


""" COMMANDS MANAGER / SWITCH PANEL """

if __name__ in ("__main__", "__builtin__", "builtins"):
    try:
        SESSION.proxies = handle_proxy()
        command = demisto.command()

        if command == "test-module":
            test_module()
            demisto.results("ok")

        elif command == "fetch-incidents":
            fetch_incidents()

        elif command == "cybersixgill-update-alert-status":
            update_alert_status()

    except Exception as e:
        return_error(f"Failed to execute {demisto.command()} command. Error: {e!s}")