SpyCloudEnterpriseProtectionFeed
Fetch SpyCloud watchlist data (breach, malware and access records) for daily monitoring, incident response, and mitigation.
Data Enrichment & Threat Intelligence · SpyCloud Enterprise Protection
Details
| ID | SpyCloudEnterpriseProtectionFeed |
|---|---|
| Provider | SpyCloud |
| Category | Data Enrichment & Threat Intelligence |
| From Version | 6.10.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | XSIAM Agentix |
README
SpyCloud Enterprise Protection Feed
Create breach, malware and access incidents in Cortex® XSOAR™ using the SpyCloud Enterprise Protection API.
This integration was integrated and tested with version 3.5 of SpyCloud Enterprise Protection API
Configure SpyCloud Enterprise Protection Feed in Cortex
| Parameter | Description | Required |
|---|---|---|
| API URL | SpyCloud Enterprise Protection API Base URL | True |
| API Key | SpyCloud Enterprise Protection API Key | True |
| Fetch incidents | This is a required field by XSOAR to fetch new Watchlist events from SpyCloud watchlist API | True |
| Since | This parameter allows you to define the starting point for a date range query on the spycloud_publish_date field. Example: -1days, now, YYYY-MM-DD. |
False |
| Until | This parameter allows you to define the ending point for a date range query on the spycloud_publish_date field. Example: -1days, now, YYYY-MM-DD. |
False |
| Since Modification Date | This parameter allows you to define the starting point for a date range query on the when an already published record was modified (record_modification_date). Example: -1days, now, YYYY-MM-DD. |
False |
| Until Modification Date | This parameter allows you to define the ending point for a date range query on the when an already published record was modified (record_modification_date). Example: -1days, now, YYYY-MM-DD. |
False |
| Severity | This parameter allows you to filter based on the numeric severity code. The codes map to the following record types: 2 = email_only, 5 = informational, 20 = breach, 25 = malware, 30 = access data. Each selected code creates incidents of the matching SpyCloud incident type. | False |
| Source ID | This parameter allows you to filter based on a particular breach source.This parameter allows you to filter based on a particular breach source. | False |
| Salt | If hashing is enabled for your API key, you have the option to provide a 10 to 24 character, high entropy salt otherwise the pre-configured salt will be used. | False |
| Type | This parameter lets you filter results by type. The allowed values are ‘corporate’ for corporate records, and ‘infected’ for infected user records (from botnet data). If no value has been provided the API function will, by default, return all record types. | False |
| Watchlist Type | This parameters lets you filter results for only emails or only domains on your watchlist. The allowed values are: [‘email’, ‘domain’, ‘subdomain’, ‘ip’]. If no value has been provided, the API will return all watchlist types. | False |
| Trust any certificate (not secure) | Trust any certificate (not secure) | False |
| Use system proxy settings | Use system proxy settings | False |
| Incidents Fetch Interval | Incidents Fetch Interval | False |
| Incident type | Incident type | False |
| Domain Search | Please enter the domains to search here, if left empty, your full watchlist will be pulled | False |
| Fetch Limit | Volume of incidents captured at a single time. By default this is set to 200 | False |
Configuration parameters
url— API URL (required)apikey— API Key (required)isFetch— Fetch incidents (required)first_fetch— Sinceuntil— Untilsince_modification_date— Since Modification Dateuntil_modification_date— Until Modification Dateseverity— Severitysource_id— Source IDsalt— Salttype— Typewatchlist_type— Watchlist Typeinsecure— Trust any certificate (not secure)proxy— Use system proxy settingsincidentFetchInterval— Incidents Fetch IntervalincidentType— Incident typedomain_search— Domain Searchfetch_limit— Fetch Limit
Commands (0)
This integration defines no commands.
import json import pytest from CommonServerPython import DemistoException from SpyCloudEnterpriseProtectionFeed import ( Client, create_spycloud_args, fetch_incident, fetch_domain_or_watchlist_data, INCIDENT_NAME, INCIDENT_TYPE, SEVERITY_VALUE, LIMIT_EXCEED, MONTHLY_QUOTA_EXCEED_MSG, TOO_MANY_REQUESTS, ) from CommonServerPython import IncidentSeverity def util_load_json(path): with open(path) as f: return json.loads(f.read()) client = Client(base_url="http://test.com/", apikey="test_123", proxy=False, verify=False) WATCHLIST_DATA = util_load_json("test_data/breach_data_by_indicator.json") INCIDENTS = util_load_json("test_data/incidents.json") MODIFIED_RESPONSE = util_load_json("test_data/modified_response.json") DOMAIN_DATA = util_load_json("test_data/domain_data.json") class MockResponse: def __init__(self, status_code, headers=None, json_data=None, url=None): self.status_code = status_code self.headers = headers or {} self.json_data = json_data or {} self.url = "https://api.spycloud.com/test" def json(self): return self.json_data def test_spy_cloud_error_handler(mocker): # --- TOO_MANY_REQUESTS should trigger retry via query_spy_cloud_api --- response = MockResponse( status_code=429, headers={"x-amzn-ErrorType": TOO_MANY_REQUESTS}, json_data={"message": "Too many requests"} ) retry_mock = mocker.patch.object(client, "query_spy_cloud_api") assert client.spy_cloud_error_handler(response) is None retry_mock.assert_called_once_with(response.url, is_retry=True) # --- LIMIT_EXCEED should raise monthly quota exception --- response = MockResponse( status_code=429, headers={"x-amzn-ErrorType": LIMIT_EXCEED}, json_data={"message": "Monthly quota exceeded"} ) with pytest.raises(DemistoException, match=MONTHLY_QUOTA_EXCEED_MSG): client.spy_cloud_error_handler(response) # --- 429 without Amazon error type should return None --- response = MockResponse(status_code=429, headers={}, json_data={"message": "Rate limit exceeded"}) assert client.spy_cloud_error_handler(response) is None # --- 403 should raise Authorization or IP error --- response = MockResponse( status_code=403, headers={"SpyCloud-Error": "Invalid IP"}, json_data={"message": "Invalid IP address"} ) with pytest.raises(DemistoException, match="Authorization or IP error"): client.spy_cloud_error_handler(response) # --- Non-403, non-429 should raise generic SpyCloud API error --- response = MockResponse(status_code=500, json_data={"message": "Internal server error"}) with pytest.raises(DemistoException, match="SpyCloud API error: Internal server error"): client.spy_cloud_error_handler(response) def test_query_spy_cloud_api_success(requests_mock): endpoint = "watchlist" req_url = f"{client._base_url}{endpoint}" requests_mock.get(req_url, json=WATCHLIST_DATA) response = client.query_spy_cloud_api(endpoint, {}) assert response == WATCHLIST_DATA @pytest.mark.parametrize( "raw_response, modified", [ (WATCHLIST_DATA, MODIFIED_RESPONSE), ], ) def test_fetch_domain_or_watchlist_data(mocker, raw_response, modified): mocker.patch.object(client, "query_spy_cloud_api", return_value=raw_response) response = fetch_domain_or_watchlist_data(client, {}, {}) assert response == modified.get("results")[:6] @pytest.mark.parametrize( "raw_response, modified", [ (DOMAIN_DATA, DOMAIN_DATA), ], ) def test_fetch_domain_or_watchlist_data_with_domain(mocker, raw_response, modified): mocker.patch.object(client, "query_spy_cloud_api", return_value=raw_response) response = fetch_domain_or_watchlist_data(client, {"domain_search": "dummy.com"}, {}) assert response == modified.get("results") @pytest.mark.parametrize( "raw_response, expected", [ (WATCHLIST_DATA, INCIDENTS), ], ) def test_fetch_incident_command(mocker, raw_response, expected): mocker.patch.object(client, "query_spy_cloud_api", return_value=raw_response) mocker.patch.object(client, "get_last_run", return_value="2023-05-30") response = fetch_incident(client, {}) assert response == expected def test_create_spycloud_args(): args = {"severity": "2, 1"} with pytest.raises(DemistoException): create_spycloud_args(args, client) def test_create_spycloud_args_accepts_severity_30(mocker): # Severity 30 (SpyCloud Access Data) must be accepted, not rejected as invalid. mocker.patch.object(client, "get_last_run", return_value="2023-05-30") result = create_spycloud_args({"severity": "30"}, client) assert result["severity"] == "30" # The full set of supported severities, including 30, passes validation. result = create_spycloud_args({"severity": "2, 5, 20, 25, 30"}, client) assert result["severity"] == "2,5,20,25,30" def test_severity_30_mappings(): # Severity 30 maps to the SpyCloud Access Data incident type at CRITICAL severity. assert INCIDENT_TYPE[30] == "SpyCloud Access Data" assert INCIDENT_NAME[30] == "SpyCloud Access Alert on" assert SEVERITY_VALUE[30] == IncidentSeverity.CRITICAL