IgniteFeed

Flashpoint Ignite Feed Integration allows importing indicators of compromise that occur in the context of an event on the Flashpoint Ignite platform which contains finished intelligence reports data, data from illicit forums, marketplaces, chat services, blogs, paste sites, technical data, card shops, and vulnerabilities. The indicators of compromise are ingested as indicators on the Cortex XSOAR and displayed in the War Room using a command.

Data Enrichment & Threat Intelligence · Flashpoint Feed · Feed

Details

IDIgniteFeed
ProviderFlashpoint
CategoryData Enrichment & Threat Intelligence
From Version6.10.0
Docker Imagedemisto/python3:3.12.13.10116658
Supported ModulesAgentix XSIAM

README

Flashpoint Ignite Feed Integration allows importing indicators of compromise that occur in the context of an event on the Flashpoint Ignite platform which contains finished intelligence reports data, data from illicit forums, marketplaces, chat services, blogs, paste sites, technical data, card shops, and vulnerabilities. The indicators of compromise are ingested as indicators on the Cortex XSOAR and displayed in the War Room using a command.
This integration was integrated and tested with API v1 of Ignite.

Fetch Indicators

Fetching the Ignite indicators. The indicators that are created or updated after the provided “First fetch time” will be fetched in the ascending order.

If you are upgrading from a Flashpoint Feed integration, please refer to the Migration Guide for guidance.

Configure Flashpoint Ignite Feed in Cortex

Parameter Description Required
Server URL Server URL to connect to Ignite. True
API Key API key used for secure communication with the Ignite platform. True
Types of the indicators to fetch Supports multiple values such as url, domain. Supports comma separated values. If not specified, it fetches all the indicators. See all available types: https://www.circl.lu/doc/misp/categories-and-types/#types. False
First fetch time Backfill indicators by providing date or relative timestamp. (Formats accepted: 2 minutes, 2 hours, 2 days, 2 weeks, 2 months, 2 years, yyyy-mm-dd, yyyy-mm-ddTHH:MM:SSZ, etc) False
Fetch indicators   False
Indicator Reputation Indicators from this integration instance will be marked with this reputation. False
Source Reliability Reliability of the source providing the intelligence data. True
feedExpirationPolicy   False
feedExpirationInterval   False
Feed Fetch Interval   False
Bypass exclusion list When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system. False
Tags Supports CSV values. False
Traffic Light Protocol Color The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed. False
feedIncremental To indicate to the Cortex XSOAR server that a feed is incremental. Generally feeds that fetch based on a time range. For example, a daily feed which provides new indicators for the last day or a feed which is immutable and provides indicators from a search date onwards. False
Create relationships Create relationships between indicators as part of Enrichment. False
Default Indicator Mapping When selected, all the incoming indicators will map to the Flashpoint Indicator. False
Trust any certificate (not secure) Indicates whether to allow connections without verifying SSL certificate’s validity. False
Use system proxy settings Indicates whether to use XSOAR’s system proxy settings to connect to the API. False

Troubleshooting

Error: The maximum indicators to fetch for the given first fetch can not exceed 10,000

  • The maximum number of indicators that can be fetched using the first fetch time is limited to 10,000 by the API.
  • To resolve this issue, you can reduce the first fetch time to a shorter time period, ensuring that the total number of indicators fetched during the specified time falls within the 10,000 limit.

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

flashpoint-ignite-get-indicators


Retrieves indicators from the Ignite API. It displays the content of the fetch-indicators command.

Base Command

flashpoint-ignite-get-indicators

Input

Argument Name Description Required
limit The maximum number of result objects to return. Maximum allowed limit is 1000. Default is 10. Optional
updated_since Only retrieve values after the given timestamp. This parameter operates on the timestamp when an IOC was last updated, i.e. enriched with more metadata. When the user wants the most recent IOCs shared in the past week, the freshest data, we recommend using this parameter.

Formats accepted: 2 minutes, 2 hours, 2 days, 2 weeks, 2 months, 2 years, yyyy-mm-dd, yyyy-mm-ddTHH:MM:SSZ, etc. Default is 3 days.
Optional
types Search by Attribute types. Can have multiple terms. See all available types: https://www.circl.lu/doc/misp/categories-and-types/#types. Possible values are: IP, Domain, URL, Email, File. Optional

Context Output

There is no context output for this command.

Command example

!flashpoint-ignite-get-indicators limit=2 types=URL updated_since="3 days"

Human Readable Output

Indicator(s)

FPID Indicator Type Indicator Value Category Event Name Event Tags Created Timestamp (UTC) First Observed Date
dummy_fpid1 url https://dummy_url1.com/attachments/1234/1234/dummy_file1.exe Payload delivery Analysis: dummy_event1 “dummy_value1” [2024-04-13 17:10:40] analysis_id:12345,
event:analysis,
malware:dummy_event1,
misp-galaxy:mitre-enterprise-attack-attack-pattern=”Software Packing - T1045”,
os:windows
2024-04-13T18:01:24Z 2024-04-13T18:02:08+00:00
dummy_fpid2 url http://dummy_url2.com/dummy_file2.exe Payload delivery Analysis: dummy_event1 “dummy_value2” [2024-04-13 14:00:26] analysis_id:56789,
event:analysis,
file_type:exe,
malware:remcos,
misp-galaxy:mitre-enterprise-attack-attack-pattern=”Code Signing - T1116”
2024-04-13T14:10:50Z 2024-04-13T14:11:08+00:00

Migration Guide

Note:
For fetching indicator, set the First Fetch time to the previous integration’s Feed Fetch Interval time. This might create duplicate indicators, but it will ensure that no indicators is lost.

Migrated Commands

Some of the previous integration’s commands have been migrated to new commands. Below is the table showing the commands that have been migrated to the new ones.

Flashpoint Command Migrated Ignite Command
flashpoint-get-indicators flashpoint-ignite-get-indicators

Configuration parameters

  • url — Server URL (required)
  • credentials — (required)
  • types — Types of the indicators to fetch
  • first_fetch — First fetch time
  • feed — Fetch indicators
  • feedReputation — Indicator Reputation
  • feedReliability — Source Reliability (required)
  • feedExpirationPolicy
  • feedExpirationInterval
  • feedFetchInterval — Feed Fetch Interval
  • feedBypassExclusionList — Bypass exclusion list
  • feedTags — Tags
  • tlp_color — Traffic Light Protocol Color
  • feedIncremental
  • createRelationship — Create relationships
  • defaultMap — Default Indicator Mapping
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings

Commands (1)

  • flashpoint-ignite-get-indicators

    Retrieves indicators from the Ignite API. It displays the content of the fetch-indicators command.

"""Flashpoint Ignite Feed Integration for Cortex XSOAR - Unit Tests file"""

import json
import os
import sys

import IgniteFeed
import pytest
from CommonServerPython import DemistoException
from IgniteFeed import (
    HTTP_ERRORS,
    MAX_FETCH,
    MAX_INDICATORS,
    MESSAGES,
    URL_SUFFIX,
    Client,
    demisto,
    fetch_indicators_command,
    flashpoint_ignite_get_indicators_command,
    main,
)
from IgniteFeed import test_module as main_test_module
from requests.exceptions import HTTPError

""" CONSTANTS """

API_KEY = "dummy_api_key"
MOCK_URL = "https://mock_dummy.com"

""" UTILITY FUNCTIONS AND FIXTURES """


def util_load_json(path):
    """
    Load json file into dictionary.

    :param path: Takes file path.

    :return: Dictionary.
    """
    with open(path, encoding="utf-8") as f:
        return json.loads(f.read())


@pytest.fixture
def mock_client():
    """Mock a client object with required data to mock."""
    client = Client(url=MOCK_URL, headers={}, proxy=False, verify=False)
    return client


class MockResponse:
    """Creates mock response."""

    def __init__(self, status_code):
        """Initialize class object."""
        self.status_code = status_code

    def raise_for_status(self):
        """Raise status code error."""
        if self.status_code != 200:
            raise HTTPError("test")


""" TEST CASES """


def test_test_module_with_invalid_apikey(requests_mock, mock_client):
    """
    Test case scenario for the execution of test_module with invalid apikey.

    Given:
       - mocked client with invalid apikey.
    When:
       - Calling `test_module` function.
    Then:
       - Returns exception.
    """

    indicator_list_response_401: str = util_load_json(
        os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/invalid_apikey_401.json")
    )
    requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_401, status_code=401)

    with pytest.raises(DemistoException) as err:
        main_test_module(client=mock_client)

    assert str(err.value) == HTTP_ERRORS[401]


def test_test_module_with_valid_apikey(requests_mock, mock_client):
    """
    Test case scenario for successful execution of test_module.

    Given:
       - mocked client with valid apikey.
    When:
       - Calling `test_module` function.
    Then:
       - Returns an ok message.
    """

    indicator_list_response_200: dict = util_load_json(
        os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json")
    )
    requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, status_code=200)

    assert main_test_module(client=mock_client) == "ok"


@pytest.mark.parametrize(
    "params, err_msg",
    [
        (
            {"url": "", "credentials": {"password": API_KEY}, "integrationReliability": "B - Usually reliable"},
            MESSAGES["NO_PARAM_PROVIDED"].format("Server URL"),
        ),
        (
            {"url": MOCK_URL, "credentials": "", "integrationReliability": "B - Usually reliable"},
            MESSAGES["NO_PARAM_PROVIDED"].format("API Key"),
        ),
    ],
)
def test_test_module_when_invalid_params_provided(params, err_msg, mocker, capfd):
    """
    Test case scenario for execution of test_module when invalid argument provided.

    Given:
        - Params for test_module.
    When:
        - Calling `main` function.
    Then:
        - Returns exception.
    """
    mocker.patch.object(demisto, "params", return_value=params)
    mocker.patch.object(demisto, "command", return_value="test-module")
    mocker.patch.object(sys, "exit", return_value=None)

    return_error = mocker.patch.object(IgniteFeed, "return_error")
    capfd.close()
    main()

    assert err_msg in return_error.call_args[0][0]


@pytest.mark.parametrize("status_code", [400, 401, 403, 404, 500])
def test_http_request_when_error_is_returned(requests_mock, mock_client, status_code):
    """
    Tests http_request method of Client class.

    Given:
        - Status codes of requests.
    When:
        - Calling `http_request` function.
    Then:
        - Returns exception.
    """
    requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), status_code=status_code)

    with pytest.raises(DemistoException) as e:
        mock_client.http_request(method="GET", url_suffix=URL_SUFFIX["ATTRIBUTES"], params={})

    assert str(e.value) == HTTP_ERRORS[status_code]


def test_http_request_when_raise_for_status(mock_client):
    """Tests http_request when error raised for status."""
    resp = MockResponse(status_code=503)

    with pytest.raises(HTTPError):
        mock_client.handle_errors(resp)


def test_flashpoint_ignite_get_indicators_command_when_invalid_argument_provided(mock_client):
    """
    Test case scenario for execution of flashpoint-ignite-get-indicators command when invalid argument provided.

    Given:
        - command arguments for flashpoint_ignite_get_indicators_command.
    When:
        - Calling `flashpoint_ignite_get_indicators_command` function.
    Then:
        - Returns a valid error message.
    """
    with pytest.raises(ValueError) as err:
        flashpoint_ignite_get_indicators_command(client=mock_client, params={}, args={"limit": -1})

    assert str(err.value) == MESSAGES["LIMIT_ERROR"].format(-1, MAX_FETCH)


def test_flashpoint_ignite_get_indicators_command_when_valid_response_is_returned(requests_mock, mock_client):
    """
    Test case scenario for execution of flashpoint-ignite-get-indicators command when valid response is returned.

    Given:
        - command arguments for flashpoint_ignite_get_indicators_command.
    When:
        - Calling `flashpoint_ignite_get_indicators_command` function.
    Then:
        - Returns a valid output.
    """
    args = {"limit": 2}

    indicator_list_response_200: dict = util_load_json(
        os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json")
    )

    requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, status_code=200)

    with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), "./test_data/indicator_list_hr.md")) as file:
        hr_output = file.read()

    indicators: dict = util_load_json(os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicators.json"))

    actual = flashpoint_ignite_get_indicators_command(client=mock_client, params={"tlp_color": "AMBER"}, args=args)

    assert actual.raw_response == indicators
    assert actual.readable_output == hr_output


def test_flashpoint_ignite_get_indicators_command_when_create_relationship_is_true(requests_mock, mock_client):
    """
    Test case scenario for execution of flashpoint-ignite-get-indicators command when create relationship is true.

    Given:
        - command arguments for flashpoint_ignite_get_indicators_command.
    When:
        - Calling `flashpoint_ignite_get_indicators_command` function.
    Then:
        - Returns a valid output.
    """
    args = {"limit": 2}

    indicator_list_response_200: dict = util_load_json(
        os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json")
    )

    requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json=indicator_list_response_200, status_code=200)

    with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), "./test_data/indicator_list_hr.md")) as file:
        hr_output = file.read()

    indicators: dict = util_load_json(
        os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicators_with_relationship.json")
    )

    actual = flashpoint_ignite_get_indicators_command(client=mock_client, params={"createRelationship": True}, args=args)

    assert actual.raw_response == indicators
    assert actual.readable_output == hr_output


def test_flashpoint_ignite_get_indicators_command_when_empty_response_is_returned(requests_mock, mock_client):
    """
    Test case scenario for execution of flashpoint-ignite-get-indicators command when empty response is returned.

    Given:
        - command arguments for flashpoint_ignite_get_indicators_command.
    When:
        - Calling `flashpoint_ignite_get_indicators_command` function.
    Then:
        - Returns a no indicator message.
    """
    args = {"limit": 2}

    requests_mock.get(os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json={}, status_code=200)

    actual = flashpoint_ignite_get_indicators_command(client=mock_client, params={}, args=args)

    assert actual.readable_output == MESSAGES["NO_INDICATORS_FOUND"]


def test_fetch_indicators_command_when_valid_response_is_returned(requests_mock, mock_client):
    """
    Test case scenario for execution of fetch-indicators command when valid response is returned.

    Given:
        - command arguments for fetch_indicators_command.
    When:
        - Calling `fetch_indicators_command` function.
    Then:
        - Returns a valid output.
    """
    indicator_list_response_200: dict = util_load_json(
        os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicator_list_200.json")
    )

    requests_mock.get(
        os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]),
        json=indicator_list_response_200,
        headers={"x-fp-total-hits": "5000"},
        status_code=200,
    )

    indicators: dict = util_load_json(os.path.join(os.path.dirname(os.path.realpath(__file__)), "test_data/indicators.json"))

    assert fetch_indicators_command(client=mock_client, params={"tlp_color": "AMBER"}, last_run={})[0] == indicators


def test_test_module_with_isfetch(requests_mock, mocker, mock_client):
    """
    Test case scenario for successful execution of test_module.

    Given:
       - mocked client of type one with valid apikey.
    When:
       - Calling `test_module` function.
    Then:
       - Returns an ok message.
    """
    requests_mock.get(
        os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]), json={}, headers={"x-fp-total-hits": "5000"}, status_code=200
    )
    mocker.patch.object(demisto, "params", return_value={"feed": True, "url": MOCK_URL, "credentials": {"password": API_KEY}})

    assert main_test_module(mock_client) == "ok"


def test_test_module_with_isfetch_when_error(requests_mock, mocker, mock_client):
    """
    Test case scenario for successful execution of test_module.

    Given:
       - mocked client of type one with valid apikey.
    When:
       - Calling `test_module` function.
    Then:
       - Returns an exception.
    """
    requests_mock.get(
        os.path.join(MOCK_URL, URL_SUFFIX["ATTRIBUTES"]),
        json={},
        headers={"x-fp-total-hits": f"{MAX_INDICATORS + 1}"},
        status_code=200,
    )
    mocker.patch.object(demisto, "params", return_value={"feed": True, "url": MOCK_URL, "credentials": {"password": API_KEY}})

    with pytest.raises(ValueError) as err:
        main_test_module(client=mock_client)

    assert str(err.value) == MESSAGES["TIME_RANGE_ERROR"].format(MAX_INDICATORS + 1)