IgniteFeed
Flashpoint Ignite Feed Integration allows importing indicators of compromise that occur in the context of an event on the Flashpoint Ignite platform which contains finished intelligence reports data, data from illicit forums, marketplaces, chat services, blogs, paste sites, technical data, card shops, and vulnerabilities. The indicators of compromise are ingested as indicators on the Cortex XSOAR and displayed in the War Room using a command.
Data Enrichment & Threat Intelligence · Flashpoint Feed · Feed
Details
| ID | IgniteFeed |
|---|---|
| Provider | Flashpoint |
| Category | Data Enrichment & Threat Intelligence |
| From Version | 6.10.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | Agentix XSIAM |
README
Flashpoint Ignite Feed Integration allows importing indicators of compromise that occur in the context of an event on the Flashpoint Ignite platform which contains finished intelligence reports data, data from illicit forums, marketplaces, chat services, blogs, paste sites, technical data, card shops, and vulnerabilities. The indicators of compromise are ingested as indicators on the Cortex XSOAR and displayed in the War Room using a command.
This integration was integrated and tested with API v1 of Ignite.
Fetch Indicators
Fetching the Ignite indicators. The indicators that are created or updated after the provided “First fetch time” will be fetched in the ascending order.
If you are upgrading from a Flashpoint Feed integration, please refer to the Migration Guide for guidance.
Configure Flashpoint Ignite Feed in Cortex
| Parameter | Description | Required |
|---|---|---|
| Server URL | Server URL to connect to Ignite. | True |
| API Key | API key used for secure communication with the Ignite platform. | True |
| Types of the indicators to fetch | Supports multiple values such as url, domain. Supports comma separated values. If not specified, it fetches all the indicators. See all available types: https://www.circl.lu/doc/misp/categories-and-types/#types. | False |
| First fetch time | Backfill indicators by providing date or relative timestamp. (Formats accepted: 2 minutes, 2 hours, 2 days, 2 weeks, 2 months, 2 years, yyyy-mm-dd, yyyy-mm-ddTHH:MM:SSZ, etc) | False |
| Fetch indicators | False | |
| Indicator Reputation | Indicators from this integration instance will be marked with this reputation. | False |
| Source Reliability | Reliability of the source providing the intelligence data. | True |
| feedExpirationPolicy | False | |
| feedExpirationInterval | False | |
| Feed Fetch Interval | False | |
| Bypass exclusion list | When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system. | False |
| Tags | Supports CSV values. | False |
| Traffic Light Protocol Color | The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed. | False |
| feedIncremental | To indicate to the Cortex XSOAR server that a feed is incremental. Generally feeds that fetch based on a time range. For example, a daily feed which provides new indicators for the last day or a feed which is immutable and provides indicators from a search date onwards. | False |
| Create relationships | Create relationships between indicators as part of Enrichment. | False |
| Default Indicator Mapping | When selected, all the incoming indicators will map to the Flashpoint Indicator. | False |
| Trust any certificate (not secure) | Indicates whether to allow connections without verifying SSL certificate’s validity. | False |
| Use system proxy settings | Indicates whether to use XSOAR’s system proxy settings to connect to the API. | False |
Troubleshooting
Error: The maximum indicators to fetch for the given first fetch can not exceed 10,000
- The maximum number of indicators that can be fetched using the first fetch time is limited to 10,000 by the API.
- To resolve this issue, you can reduce the first fetch time to a shorter time period, ensuring that the total number of indicators fetched during the specified time falls within the 10,000 limit.
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
flashpoint-ignite-get-indicators
Retrieves indicators from the Ignite API. It displays the content of the fetch-indicators command.
Base Command
flashpoint-ignite-get-indicators
Input
| Argument Name | Description | Required |
|---|---|---|
| limit | The maximum number of result objects to return. Maximum allowed limit is 1000. Default is 10. | Optional |
| updated_since | Only retrieve values after the given timestamp. This parameter operates on the timestamp when an IOC was last updated, i.e. enriched with more metadata. When the user wants the most recent IOCs shared in the past week, the freshest data, we recommend using this parameter. Formats accepted: 2 minutes, 2 hours, 2 days, 2 weeks, 2 months, 2 years, yyyy-mm-dd, yyyy-mm-ddTHH:MM:SSZ, etc. Default is 3 days. |
Optional |
| types | Search by Attribute types. Can have multiple terms. See all available types: https://www.circl.lu/doc/misp/categories-and-types/#types. Possible values are: IP, Domain, URL, Email, File. | Optional |
Context Output
There is no context output for this command.
Command example
!flashpoint-ignite-get-indicators limit=2 types=URL updated_since="3 days"
Human Readable Output
Indicator(s)
FPID Indicator Type Indicator Value Category Event Name Event Tags Created Timestamp (UTC) First Observed Date dummy_fpid1 url https://dummy_url1.com/attachments/1234/1234/dummy_file1.exe Payload delivery Analysis: dummy_event1 “dummy_value1” [2024-04-13 17:10:40] analysis_id:12345,
event:analysis,
malware:dummy_event1,
misp-galaxy:mitre-enterprise-attack-attack-pattern=”Software Packing - T1045”,
os:windows2024-04-13T18:01:24Z 2024-04-13T18:02:08+00:00 dummy_fpid2 url http://dummy_url2.com/dummy_file2.exe Payload delivery Analysis: dummy_event1 “dummy_value2” [2024-04-13 14:00:26] analysis_id:56789,
event:analysis,
file_type:exe,
malware:remcos,
misp-galaxy:mitre-enterprise-attack-attack-pattern=”Code Signing - T1116”2024-04-13T14:10:50Z 2024-04-13T14:11:08+00:00
Migration Guide
Note:
For fetching indicator, set the First Fetch time to the previous integration’s Feed Fetch Interval time. This might create duplicate indicators, but it will ensure that no indicators is lost.
Migrated Commands
Some of the previous integration’s commands have been migrated to new commands. Below is the table showing the commands that have been migrated to the new ones.
| Flashpoint Command | Migrated Ignite Command |
|---|---|
| flashpoint-get-indicators | flashpoint-ignite-get-indicators |
Configuration parameters
url— Server URL (required)credentials— (required)types— Types of the indicators to fetchfirst_fetch— First fetch timefeed— Fetch indicatorsfeedReputation— Indicator ReputationfeedReliability— Source Reliability (required)feedExpirationPolicy—feedExpirationInterval—feedFetchInterval— Feed Fetch IntervalfeedBypassExclusionList— Bypass exclusion listfeedTags— Tagstlp_color— Traffic Light Protocol ColorfeedIncremental—createRelationship— Create relationshipsdefaultMap— Default Indicator Mappinginsecure— Trust any certificate (not secure)proxy— Use system proxy settings
Commands (1)
-
flashpoint-ignite-get-indicatorsRetrieves indicators from the Ignite API. It displays the content of the fetch-indicators command.
"""Flashpoint Ignite Feed Integration for Cortex XSOAR - Unit Tests file""" import json import os import sys import IgniteFeed import pytest from CommonServerPython import DemistoException from IgniteFeed import ( HTTP_ERRORS, MAX_FETCH, MAX_INDICATORS, MESSAGES, URL_SUFFIX, Client, demisto, fetch_indicators_command, flashpoint_ignite_get_indicators_command, main, ) from IgniteFeed import test_module as main_test_module from requests.exceptions import HTTPError """ CONSTANTS """ API_KEY = "dummy_api_key" MOCK_URL = "https://mock_dummy.com" """ UTILITY FUNCTIONS AND FIXTURES """ def util_load_json(path): """ Load json file into dictionary. :param path: Takes file path. :return: Dictionary. """ with open(path, encoding="utf-8") as f: return json.loads(f.read()) @pytest.fixture def mock_client(): """Mock a client object with required data to mock.""" client = Client(url=MOCK_URL, headers={}, proxy=False, verify=False) return client class MockResponse: """Creates mock response.""" def __init__(self, status_code): """Initialize class object.""" self.status_code = status_code def raise_for_status(self): """Raise status code error.""" if self.status_code != 200: raise HTTPError("test") """ TEST CASES """ def test_test_module_with_invalid_apikey(requests_mock, mock_client): """ Test case scenario for the execution of test_module with invalid apikey. Given: - mocked client with invalid apikey. When: - Calling `test_module` function. Then: - Returns exception. """ indicator_list_response_401: str = util_load_json( os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/invalid_apikey_401.json") ) requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_401, status_code=401) with pytest.raises(DemistoException) as err: main_test_module(client=mock_client) assert str(err.value) == HTTP_ERRORS[401] def test_test_module_with_valid_apikey(requests_mock, mock_client): """ Test case scenario for successful execution of test_module. Given: - mocked client with valid apikey. When: - Calling `test_module` function. Then: - Returns an ok message. """ indicator_list_response_200: dict = util_load_json( os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json") ) requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, status_code=200) assert main_test_module(client=mock_client) == "ok" @pytest.mark.parametrize( "params, err_msg", [ ( {"url": "", "credentials": {"password": API_KEY}, "integrationReliability": "B - Usually reliable"}, MESSAGES["NO_PARAM_PROVIDED"].format("Server URL"), ), ( {"url": MOCK_URL, "credentials": "", "integrationReliability": "B - Usually reliable"}, MESSAGES["NO_PARAM_PROVIDED"].format("API Key"), ), ], ) def test_test_module_when_invalid_params_provided(params, err_msg, mocker, capfd): """ Test case scenario for execution of test_module when invalid argument provided. Given: - Params for test_module. When: - Calling `main` function. Then: - Returns exception. """ mocker.patch.object(demisto, "params", return_value=params) mocker.patch.object(demisto, "command", return_value="test-module") mocker.patch.object(sys, "exit", return_value=None) return_error = mocker.patch.object(IgniteFeed, "return_error") capfd.close() main() assert err_msg in return_error.call_args[0][0] @pytest.mark.parametrize("status_code", [400, 401, 403, 404, 500]) def test_http_request_when_error_is_returned(requests_mock, mock_client, status_code): """ Tests http_request method of Client class. Given: - Status codes of requests. When: - Calling `http_request` function. Then: - Returns exception. """ requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), status_code=status_code) with pytest.raises(DemistoException) as e: mock_client.http_request(method="GET", url_suffix=URL_SUFFIX["ATTRIBUTES"], params={}) assert str(e.value) == HTTP_ERRORS[status_code] def test_http_request_when_raise_for_status(mock_client): """Tests http_request when error raised for status.""" resp = MockResponse(status_code=503) with pytest.raises(HTTPError): mock_client.handle_errors(resp) def test_flashpoint_ignite_get_indicators_command_when_invalid_argument_provided(mock_client): """ Test case scenario for execution of flashpoint-ignite-get-indicators command when invalid argument provided. Given: - command arguments for flashpoint_ignite_get_indicators_command. When: - Calling `flashpoint_ignite_get_indicators_command` function. Then: - Returns a valid error message. """ with pytest.raises(ValueError) as err: flashpoint_ignite_get_indicators_command(client=mock_client, params={}, args={"limit": -1}) assert str(err.value) == MESSAGES["LIMIT_ERROR"].format(-1, MAX_FETCH) def test_flashpoint_ignite_get_indicators_command_when_valid_response_is_returned(requests_mock, mock_client): """ Test case scenario for execution of flashpoint-ignite-get-indicators command when valid response is returned. Given: - command arguments for flashpoint_ignite_get_indicators_command. When: - Calling `flashpoint_ignite_get_indicators_command` function. Then: - Returns a valid output. """ args = {"limit": 2} indicator_list_response_200: dict = util_load_json( os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json") ) requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, status_code=200) with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), "./test_data/indicator_list_hr.md")) as file: hr_output = file.read() indicators: dict = util_load_json(os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicators.json")) actual = flashpoint_ignite_get_indicators_command(client=mock_client, params={"tlp_color": "AMBER"}, args=args) assert actual.raw_response == indicators assert actual.readable_output == hr_output def test_flashpoint_ignite_get_indicators_command_when_create_relationship_is_true(requests_mock, mock_client): """ Test case scenario for execution of flashpoint-ignite-get-indicators command when create relationship is true. Given: - command arguments for flashpoint_ignite_get_indicators_command. When: - Calling `flashpoint_ignite_get_indicators_command` function. Then: - Returns a valid output. """ args = {"limit": 2} indicator_list_response_200: dict = util_load_json( os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json") ) requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, status_code=200) with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), "./test_data/indicator_list_hr.md")) as file: hr_output = file.read() indicators: dict = util_load_json( os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicators_with_relationship.json") ) actual = flashpoint_ignite_get_indicators_command(client=mock_client, params={"createRelationship": True}, args=args) assert actual.raw_response == indicators assert actual.readable_output == hr_output def test_flashpoint_ignite_get_indicators_command_when_empty_response_is_returned(requests_mock, mock_client): """ Test case scenario for execution of flashpoint-ignite-get-indicators command when empty response is returned. Given: - command arguments for flashpoint_ignite_get_indicators_command. When: - Calling `flashpoint_ignite_get_indicators_command` function. Then: - Returns a no indicator message. """ args = {"limit": 2} requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json={}, status_code=200) actual = flashpoint_ignite_get_indicators_command(client=mock_client, params={}, args=args) assert actual.readable_output == MESSAGES["NO_INDICATORS_FOUND"] def test_fetch_indicators_command_when_valid_response_is_returned(requests_mock, mock_client): """ Test case scenario for execution of fetch-indicators command when valid response is returned. Given: - command arguments for fetch_indicators_command. When: - Calling `fetch_indicators_command` function. Then: - Returns a valid output. """ indicator_list_response_200: dict = util_load_json( os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json") ) requests_mock.get( os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, headers={"x-fp-total-hits": "5000"}, status_code=200, ) indicators: dict = util_load_json(os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicators.json")) assert fetch_indicators_command(client=mock_client, params={"tlp_color": "AMBER"}, last_run={})[0] == indicators def test_test_module_with_isfetch(requests_mock, mocker, mock_client): """ Test case scenario for successful execution of test_module. Given: - mocked client of type one with valid apikey. When: - Calling `test_module` function. Then: - Returns an ok message. """ requests_mock.get( os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json={}, headers={"x-fp-total-hits": "5000"}, status_code=200 ) mocker.patch.object(demisto, "params", return_value={"feed": True, "url": MOCK_URL, "credentials": {"password": API_KEY}}) assert main_test_module(mock_client) == "ok" def test_test_module_with_isfetch_when_error(requests_mock, mocker, mock_client): """ Test case scenario for successful execution of test_module. Given: - mocked client of type one with valid apikey. When: - Calling `test_module` function. Then: - Returns an exception. """ requests_mock.get( os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json={}, headers={"x-fp-total-hits": f"{MAX_INDICATORS + 1}"}, status_code=200, ) mocker.patch.object(demisto, "params", return_value={"feed": True, "url": MOCK_URL, "credentials": {"password": API_KEY}}) with pytest.raises(ValueError) as err: main_test_module(client=mock_client) assert str(err.value) == MESSAGES["TIME_RANGE_ERROR"].format(MAX_INDICATORS + 1)