SpyCloudEnterpriseProtectionFeed

Fetch SpyCloud watchlist data (breach, malware and access records) for daily monitoring, incident response, and mitigation.

Data Enrichment & Threat Intelligence · SpyCloud Enterprise Protection

Details

IDSpyCloudEnterpriseProtectionFeed
ProviderSpyCloud
CategoryData Enrichment & Threat Intelligence
From Version6.10.0
Docker Imagedemisto/python3:3.12.13.10116658
Supported ModulesXSIAM Agentix

README

SpyCloud Enterprise Protection Feed

Create breach, malware and access incidents in Cortex® XSOAR™ using the SpyCloud Enterprise Protection API.
This integration was integrated and tested with version 3.5 of SpyCloud Enterprise Protection API

Configure SpyCloud Enterprise Protection Feed in Cortex

Parameter Description Required
API URL SpyCloud Enterprise Protection API Base URL True
API Key SpyCloud Enterprise Protection API Key True
Fetch incidents This is a required field by XSOAR to fetch new Watchlist events from SpyCloud watchlist API True
Since This parameter allows you to define the starting point for a date range query on the spycloud_publish_date field.
Example: -1days, now, YYYY-MM-DD.
False
Until This parameter allows you to define the ending point for a date range query on the spycloud_publish_date field.
Example: -1days, now, YYYY-MM-DD.
False
Since Modification Date This parameter allows you to define the starting point for a date range query on the when an already published record was modified (record_modification_date).
Example: -1days, now, YYYY-MM-DD.
False
Until Modification Date This parameter allows you to define the ending point for a date range query on the when an already published record was modified (record_modification_date).
Example: -1days, now, YYYY-MM-DD.
False
Severity This parameter allows you to filter based on the numeric severity code. The codes map to the following record types: 2 = email_only, 5 = informational, 20 = breach, 25 = malware, 30 = access data. Each selected code creates incidents of the matching SpyCloud incident type. False
Source ID This parameter allows you to filter based on a particular breach source.This parameter allows you to filter based on a particular breach source. False
Salt If hashing is enabled for your API key, you have the option to provide a 10 to 24 character, high entropy salt otherwise the pre-configured salt will be used. False
Type This parameter lets you filter results by type. The allowed values are ‘corporate’ for corporate records, and ‘infected’ for infected user records (from botnet data). If no value has been provided the API function will, by default, return all record types. False
Watchlist Type This parameters lets you filter results for only emails or only domains on your watchlist. The allowed values are: [‘email’, ‘domain’, ‘subdomain’, ‘ip’]. If no value has been provided, the API will return all watchlist types. False
Trust any certificate (not secure) Trust any certificate (not secure) False
Use system proxy settings Use system proxy settings False
Incidents Fetch Interval Incidents Fetch Interval False
Incident type Incident type False
Domain Search Please enter the domains to search here, if left empty, your full watchlist will be pulled False
Fetch Limit Volume of incidents captured at a single time. By default this is set to 200 False

Configuration parameters

  • url — API URL (required)
  • apikey — API Key (required)
  • isFetch — Fetch incidents (required)
  • first_fetch — Since
  • until — Until
  • since_modification_date — Since Modification Date
  • until_modification_date — Until Modification Date
  • severity — Severity
  • source_id — Source ID
  • salt — Salt
  • type — Type
  • watchlist_type — Watchlist Type
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings
  • incidentFetchInterval — Incidents Fetch Interval
  • incidentType — Incident type
  • domain_search — Domain Search
  • fetch_limit — Fetch Limit

Commands (0)

This integration defines no commands.

import json

import pytest
from CommonServerPython import DemistoException
from SpyCloudEnterpriseProtectionFeed import (
    Client,
    create_spycloud_args,
    fetch_incident,
    fetch_domain_or_watchlist_data,
    INCIDENT_NAME,
    INCIDENT_TYPE,
    SEVERITY_VALUE,
    LIMIT_EXCEED,
    MONTHLY_QUOTA_EXCEED_MSG,
    TOO_MANY_REQUESTS,
)
from CommonServerPython import IncidentSeverity


def util_load_json(path):
    with open(path) as f:
        return json.loads(f.read())


client = Client(base_url="http://test.com/", apikey="test_123", proxy=False, verify=False)
WATCHLIST_DATA = util_load_json("test_data/breach_data_by_indicator.json")
INCIDENTS = util_load_json("test_data/incidents.json")
MODIFIED_RESPONSE = util_load_json("test_data/modified_response.json")
DOMAIN_DATA = util_load_json("test_data/domain_data.json")


class MockResponse:
    def __init__(self, status_code, headers=None, json_data=None, url=None):
        self.status_code = status_code
        self.headers = headers or {}
        self.json_data = json_data or {}
        self.url = "https://api.spycloud.com/test"

    def json(self):
        return self.json_data


def test_spy_cloud_error_handler(mocker):
    # --- TOO_MANY_REQUESTS should trigger retry via query_spy_cloud_api ---
    response = MockResponse(
        status_code=429, headers={"x-amzn-ErrorType": TOO_MANY_REQUESTS}, json_data={"message": "Too many requests"}
    )

    retry_mock = mocker.patch.object(client, "query_spy_cloud_api")

    assert client.spy_cloud_error_handler(response) is None
    retry_mock.assert_called_once_with(response.url, is_retry=True)

    # --- LIMIT_EXCEED should raise monthly quota exception ---
    response = MockResponse(
        status_code=429, headers={"x-amzn-ErrorType": LIMIT_EXCEED}, json_data={"message": "Monthly quota exceeded"}
    )

    with pytest.raises(DemistoException, match=MONTHLY_QUOTA_EXCEED_MSG):
        client.spy_cloud_error_handler(response)

    # --- 429 without Amazon error type should return None ---
    response = MockResponse(status_code=429, headers={}, json_data={"message": "Rate limit exceeded"})

    assert client.spy_cloud_error_handler(response) is None

    # --- 403 should raise Authorization or IP error ---
    response = MockResponse(
        status_code=403, headers={"SpyCloud-Error": "Invalid IP"}, json_data={"message": "Invalid IP address"}
    )

    with pytest.raises(DemistoException, match="Authorization or IP error"):
        client.spy_cloud_error_handler(response)

    # --- Non-403, non-429 should raise generic SpyCloud API error ---
    response = MockResponse(status_code=500, json_data={"message": "Internal server error"})

    with pytest.raises(DemistoException, match="SpyCloud API error: Internal server error"):
        client.spy_cloud_error_handler(response)


def test_query_spy_cloud_api_success(requests_mock):
    endpoint = "watchlist"
    req_url = f"{client._base_url}{endpoint}"
    requests_mock.get(req_url, json=WATCHLIST_DATA)
    response = client.query_spy_cloud_api(endpoint, {})
    assert response == WATCHLIST_DATA


@pytest.mark.parametrize(
    "raw_response, modified",
    [
        (WATCHLIST_DATA, MODIFIED_RESPONSE),
    ],
)
def test_fetch_domain_or_watchlist_data(mocker, raw_response, modified):
    mocker.patch.object(client, "query_spy_cloud_api", return_value=raw_response)
    response = fetch_domain_or_watchlist_data(client, {}, {})
    assert response == modified.get("results")[:6]


@pytest.mark.parametrize(
    "raw_response, modified",
    [
        (DOMAIN_DATA, DOMAIN_DATA),
    ],
)
def test_fetch_domain_or_watchlist_data_with_domain(mocker, raw_response, modified):
    mocker.patch.object(client, "query_spy_cloud_api", return_value=raw_response)
    response = fetch_domain_or_watchlist_data(client, {"domain_search": "dummy.com"}, {})
    assert response == modified.get("results")


@pytest.mark.parametrize(
    "raw_response, expected",
    [
        (WATCHLIST_DATA, INCIDENTS),
    ],
)
def test_fetch_incident_command(mocker, raw_response, expected):
    mocker.patch.object(client, "query_spy_cloud_api", return_value=raw_response)
    mocker.patch.object(client, "get_last_run", return_value="2023-05-30")
    response = fetch_incident(client, {})
    assert response == expected


def test_create_spycloud_args():
    args = {"severity": "2, 1"}
    with pytest.raises(DemistoException):
        create_spycloud_args(args, client)


def test_create_spycloud_args_accepts_severity_30(mocker):
    # Severity 30 (SpyCloud Access Data) must be accepted, not rejected as invalid.
    mocker.patch.object(client, "get_last_run", return_value="2023-05-30")
    result = create_spycloud_args({"severity": "30"}, client)
    assert result["severity"] == "30"

    # The full set of supported severities, including 30, passes validation.
    result = create_spycloud_args({"severity": "2, 5, 20, 25, 30"}, client)
    assert result["severity"] == "2,5,20,25,30"


def test_severity_30_mappings():
    # Severity 30 maps to the SpyCloud Access Data incident type at CRITICAL severity.
    assert INCIDENT_TYPE[30] == "SpyCloud Access Data"
    assert INCIDENT_NAME[30] == "SpyCloud Access Alert on"
    assert SEVERITY_VALUE[30] == IncidentSeverity.CRITICAL