RecordedFutureLists
Search and manage watchlists in Recorded Future.
Data Enrichment & Threat Intelligence · Recorded Future Intelligence
Details
| ID | RecordedFutureLists |
|---|---|
| Provider | Mastercard |
| Category | Data Enrichment & Threat Intelligence |
| From Version | 6.5.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | Agentix XSIAM |
README
Search and manage watchlists in Recorded Future
This integration was integrated and tested with version 1.1.1 of RecordedFutureLists
Configure Recorded Future - Lists in Cortex
| Parameter | Description | Required |
|---|---|---|
| API URL | Default URL: https://api.recordedfuture.com/gw/xsoar/ | True |
| API Token | True | |
| Trust any certificate (not secure) | False | |
| Use system proxy settings | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
recordedfuture-lists-search
Search for lists in Recorded Future
Base Command
recordedfuture-lists-search
Input
| Argument Name | Description | Required |
|---|---|---|
| list_names | Freetext name to search for. | Optional |
| contains | Filter lists based on entity types, will only include lists with the entity types specified. Default value “” includes all types. Possible values are: entity, source, text, custom, ip, domain, tech_stack, industry, brand, partner, industry_peer, location, supplier, vulnerability, company, hash, operation, attacker, target, method. | Optional |
| limit | Limits the amount of returned results. | Optional |
| include | Include all search results. Default is to exclude all lists owned by the system user. Possible values are: all. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| RecordedFuture.List.id | String | Unique id of the list in Recorded Future |
| RecordedFuture.List.name | String | Name of the list in Recorded Future |
| RecordedFuture.List.type | String | Recorded future entity type |
| RecordedFuture.List.created | String | Timestamp of creation |
| RecordedFuture.List.updated | String | Timestamp of last update to the list |
| RecordedFuture.List.owner_id | String | Unique id of the owner in Recorded Future |
| RecordedFuture.List.owner_name | String | Readable name of list in Recorded Future |
recordedfuture-lists-add-entities
Add entities to a list, separate entities by commas. “NOTE:” if entity type is specified, only one entity type can be added with each action.
When adding IDs use the following for Recorded Future light entities:
- IPaddress: “ip:x.x.x.x”
- Domain: “idn:example.xyz”
- Hash: “hash:examplehashvalue”
- Email: “email:example@example.xyz”
- Url: “url:https://example.xyz”
Base Command
recordedfuture-lists-add-entities
Input
| Argument Name | Description | Required |
|---|---|---|
| list_id | Id of the list that should be added, can be found by running !recordedfuture-lists-search with the corresponding filters or in the Recorded Future portal. | Required |
| entity_ids | Specific ids from Recorded Future separated by comma, For urls containing commas: replace comma with %2C. | Optional |
| freetext_names | Freetext names will be matched to Recorded Future ids separated by comma, this alernative will add the best match in the Recorded Future data. For urls containing commas: escape with %2C. | Optional |
| entity_type | Type of the entities that should be added. Use together with freetext_names to improve entity resolution. Possible values are: ip, domain, malware, url, hash, cve, company, person, product, industry, country, attack-vector, operation, mitre-identifier, malware-category. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| RecordedFuture.List.Entities.name | String | Name of the entity in the list |
| RecordedFuture.List.Entities.type | String | The Recorded Future entity type resolved during the action |
| RecordedFuture.List.Entities.id | String | Unique id of the entity in Recorded Future |
| RecordedFuture.List.Entities.input_value | String | The value inputted to the command |
| RecordedFuture.List.Entities.action_result | String | Entity specific result for the action |
recordedfuture-lists-remove-entities
Remove entities from a list. Separate entities with commas. “NOTE:” If entity type is specified, only one entity type can be added with each action.
When adding IDs use the following for Recorded Future light entities:
- IPaddress: “ip:x.x.x.x”
- Domain: “idn:example.xyz”
- Hash: “hash:examplehashvalue”
- Email: “email:example@example.xyz”
- Url: “url:https://example.xyz”
Base Command
recordedfuture-lists-remove-entities
Input
| Argument Name | Description | Required |
|---|---|---|
| list_id | ID of the list that should be removed. Can be found by running !recordedfuture-lists-search with the corresponding filters or in the Recorded Future portal. | Required |
| entity_ids | A comma-separated list of specific IDs from Recorded Future. For URLs containing commas, replace comma with %2C. | Optional |
| freetext_names | A comma-separated list of freetext names to be matched to Recorded Future IDs. This will remove the best match in the Recorded Future data. For URLs containing commas, escape with %2C. | Optional |
| entity_type | Type of the entities that should be removed. Use together with freetext_names to improve entity resolution. Possible values are: ip, domain, malware, url, hash, cve, company, person, product, industry, country, attack-vector, operation, mitre-identifier, malware-category. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| RecordedFuture.List.Entities.name | String | Name of the entity in the list |
| RecordedFuture.List.Entities.type | String | The Recorded Future entity type resolved during the action. |
| RecordedFuture.List.Entities.id | String | Unique ID of the entity in Recorded Future. |
| RecordedFuture.List.Entities.input_value | String | The value inputted to the command. |
| RecordedFuture.List.Entities.action_result | String | Entity specific result for the action. |
recordedfuture-lists-entities
Get the entities that are currently in the given lists.
Base Command
recordedfuture-lists-entities
Input
| Argument Name | Description | Required |
|---|---|---|
| list_ids | A comma-separated list of Recorded Future list IDs. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| RecordedFuture.List.id | String | Unique ID of the list in Recorded Future. |
| RecordedFuture.List.name | String | Name of the list in Recorded Future. |
| RecordedFuture.List.type | String | Recorded Future entity type. |
| RecordedFuture.List.Entities.name | String | Name of the entity in the list. |
| RecordedFuture.List.Entities.type | String | The Recorded Future entity type resolved during the action. |
| RecordedFuture.List.Entities.id | String | Unique ID of the entity in Recorded Future. |
Configuration parameters
server_url— API URL (required)token— (required)insecure— Trust any certificate (not secure)proxy— Use system proxy settings
Commands (4)
-
recordedfuture-lists-add-entitiesAdd entities to a list, separate entities by commas. "NOTE:" if entity type is specified, only one entity type can be added with each action.
-
recordedfuture-lists-entitiesGet the entities that are currently in the given lists.
-
recordedfuture-lists-remove-entitiesRemove entities from a list. Separate entities with a comma. "NOTE:" If entity type is specified, only one entity type can be added with each action.
-
recordedfuture-lists-searchSearch for lists in Recorded Future.
import demistomock as demisto # noqa: F401 from CommonServerPython import * # noqa: F401 """Recorded Future Lists Integration for Demisto.""" import platform import json # flake8: noqa: F402,F405 lgtm STATUS_TO_RETRY = [500, 501, 502, 503, 504] # disable insecure warnings requests.packages.urllib3.disable_warnings() # type: ignore __version__ = "1.1.1" # === === === === === === === === === === === === === === === # === === === === Recorded Future API Client === === === ==== # === === === === === === === === === === === === === === === class Client(BaseClient): def whoami(self) -> Dict[str, Any]: return self._http_request( method="get", url_suffix="info/whoami", timeout=60, ) def _call(self, url_suffix, **kwargs): json_data = { "demisto_command": demisto.command(), "demisto_args": demisto.args(), } if "demisto_args" in kwargs: if args := kwargs.get("demisto_args"): json_data.update({"demisto_args": args}) kwargs.pop("demisto_args") method = kwargs.get("method", "post") request_kwargs = { "method": method, "url_suffix": url_suffix, "json_data": json_data, "timeout": 90, "retries": 3, "status_list_to_retry": STATUS_TO_RETRY, } request_kwargs.update(kwargs) try: response = self._http_request(**request_kwargs) if isinstance(response, dict) and response.get("return_error"): # This will raise the Exception or call "demisto.results()" for the error and sys.exit(0). return_error(**response["return_error"]) except DemistoException as err: if "404" in str(err): return CommandResults( outputs_prefix="", outputs={}, raw_response={}, readable_output="No results found.", outputs_key_field="", ) else: raise err return response #################################################### ################## List operations ################# #################################################### def list_search(self) -> Dict[str, Any]: parsed_args = demisto.args() if list_names := parsed_args.get("list_names"): parsed_args["list_names"] = list_names.split(",") if types := parsed_args.get("contains"): parsed_args["contains"] = types.split(",") """Search for lists in Recorded Future""" return self._call(url_suffix="/v2/lists/search", demisto_args=parsed_args) #################################################### ################ Entity operations ################# #################################################### def entity_operation(self, operation) -> Dict[str, Any]: parsed_args = demisto.args() list_id = parsed_args.pop("list_id") if entity_ids := parsed_args.get("entity_ids"): parsed_args["entity_ids"] = entity_ids.split(",") if freetext_names := parsed_args.get("freetext_names"): parsed_args["freetext_names"] = freetext_names.split(",") if not ((entity_ids and not freetext_names) or (not entity_ids and freetext_names)): # XOR entity_ids and freetext_names raise ValueError( "Command expected 1 of parmeters: entity_ids or freetext_names, to be specified." f" Got {len([x for x in [entity_ids, freetext_names] if x])} specified." ) return self._call( url_suffix=f"/v2/lists/{list_id}/entities/{operation}", demisto_args=parsed_args, ) def entity_fetch(self) -> Dict[str, Any]: parsed_args = demisto.args() if list_ids := parsed_args.get("list_ids"): parsed_args["list_ids"] = list_ids.split(",") return self._call(url_suffix="/v2/lists/entities/lookup", demisto_args=parsed_args) # === === === === === === === === === === === === === === === # === === === === === === ACTIONS === === === === === === === # === === === === === === === === === === === === === === === class Actions: def __init__(self, rf_client: Client): self.client = rf_client def _process_result_actions(self, response: Union[dict, CommandResults]) -> List[CommandResults]: if isinstance(response, CommandResults): # Case when we got 404 on response, and it was processed in self.client._call() method. return [response] elif not isinstance(response, dict): # In case API returned a str - we don't want to call "response.get()" on a str object. return None # type: ignore result_actions: Union[List[dict], None] = response.get("result_actions") if not result_actions: return None # type: ignore command_results: List[CommandResults] = [] for action in result_actions: if "CommandResults" in action: command_results.append(CommandResults(**action["CommandResults"])) return command_results ####################################################### #################### List actions ##################### ####################################################### def list_search_command(self) -> List[CommandResults]: response = self.client.list_search() return self._process_result_actions(response=response) ####################################################### ################### Entity actions #################### ####################################################### def entity_add_command(self) -> List[CommandResults]: response = self.client.entity_operation("add") return self._process_result_actions(response=response) def entity_remove_command(self) -> List[CommandResults]: response = self.client.entity_operation("remove") return self._process_result_actions(response=response) def entities_get_command(self) -> List[CommandResults]: response = self.client.entity_fetch() return self._process_result_actions(response=response) # === === === === === === === === === === === === === === === # === === === === === === === MAIN === === === === === === == # === === === === === === === === === === === === === === === def main() -> None: """Main method used to run actions.""" try: demisto_params = demisto.params() base_url = demisto_params.get("server_url", "").rstrip("/") verify_ssl = not demisto_params.get("insecure", False) proxy = demisto_params.get("proxy", False) headers = { "X-RFToken": demisto_params["token"].get("password"), "X-RF-User-Agent": ( f'RecordedFutureLists.py/{__version__} ({platform.platform()}) ' f'XSOAR/{__version__} ' f'RFClient/{__version__} (Cortex_XSOAR_{demisto.demistoVersion()["version"]})' ), } client = Client(base_url=base_url, verify=verify_ssl, headers=headers, proxy=proxy) command = demisto.command() actions = Actions(client) if command == "test-module": # This is the call made when pressing the integration Test button. # Returning 'ok' indicates that the integration works like it suppose to and # connection to the service is successful. # Returning 'ok' will make the test result be green. # Any other response will make the test result be red. try: client.whoami() return_results("ok") except Exception as err: message = str(err) try: error = json.loads(str(err).split("\n")[1]) if "fail" in error.get("result", {}).get("status", ""): message = error.get("result", {})["message"] except Exception: message = ( f"Unknown error. Please verify that the API URL and Token are correctly configured. RAW Error: {err}" ) raise DemistoException(f"Failed due to - {message}") ####################################################### ################### List commands ##################### ####################################################### elif command == "recordedfuture-lists-search": return_results(actions.list_search_command()) ####################################################### ################## Entity commands #################### ####################################################### elif command == "recordedfuture-lists-add-entities": return_results(actions.entity_add_command()) elif command == "recordedfuture-lists-remove-entities": return_results(actions.entity_remove_command()) elif command == "recordedfuture-lists-entities": return_results(actions.entities_get_command()) except Exception as e: return_error(message=f"Failed to execute {demisto.command()} command: {str(e)}") if __name__ in ("__main__", "__builtin__", "builtins"): main()