Recorded Future Feed

Ingests indicators from Recorded Future feeds into Demisto.

Data Enrichment & Threat Intelligence · Recorded Future Feed · Feed

Details

IDRecorded Future Feed
ProviderMastercard
CategoryData Enrichment & Threat Intelligence
From Version5.5.0
Docker Imagedemisto/python3:3.12.13.10404775
Supported ModulesAgentix XSIAM

README

Ingests indicators from Recorded Future feeds into Cortex XSOAR.
This integration was integrated and tested with Recorded Future Feed

Configure Recorded Future Feed on Cortex XSOAR

  1. Navigate to Settings > Integrations > Servers & Services.
  2. Search for Recorded Future Feed.

  3. Click Add instance to create and configure a new integration instance.

    Parameter Description Required
    Fetch indicators   False
    Indicator Reputation Indicators from this integration instance will be marked with this reputation False
    Source Reliability Reliability of the source providing the intelligence data True
    Traffic Light Protocol Color The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed False
    Feed Fetch Interval   False
    Bypass exclusion list When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system. False
    Trust any certificate (not secure)   False
    Use system proxy settings   False
    Indicator Type Type of the indicator in the feed. True
    API token   True
    Services   True
    Risk Rule A comma-separated list of risk rules which limits the indicators list to a specific risk rule. For example: ‘dhsAis,phishingUrl’. If more than one risk rule is set, the indicators fetching and the ‘rf-feed-get-indicators’ command will be executed for each risk rule. To see available risk rules run the rf-feed-get-risk-rules command. This parameter will only be used for the ‘connectApi’ service. Using the ‘large’ risk rule is not recommended. False
    Fusion File Path Load a custom risklist from a specified Recorded Future file path.
    If no file path is specified, the default risklist file is used. This parameter
    will only be used for the ‘fusion’ service.    		 False
    Tags Supports CSV values. False
    Request Timeout Time in seconds before HTTP requests timeout. True
    Malicious Threshold The minimum score from the feed in order to to determine whether the indicator is malicious. Default is “65”. For more information about Recorded Future scoring go to integration details. False
    IOC Risk Score Threshold If selected, will be used to filter out the ingested indicators, and only indicators with equivalent and higher risk score will be ingested into XSOAR. False
  4. Click Test to validate the URLs, token, and connection.

Notes

  1. It is highly recommended to not create multiple instances of the same indicator type, even when fetching both from fusion and connectApi. Creating multiple instances with same indicator type will lead to duplicate indicators being fetched which can cause performance issues for the server.
  2. Because of restrictions on the API side, it is strongly advisable to maintain the number of indicators below 100,000 per instance. Exceeding this limit may result in unforeseen expiration of indicators beyond that threshold.

  3. Recommended interval for fetching indicators according to Recorded Future documentation:

    Indicator Type Recommended Fetch Interval
    IP 1 Hour.
    Domain 2 Hours.
    Hash 1 Day.
    URL 2 Hours.
    Vulnerability 2 Hours.
  4. Per instance configuration, it is recommended to use either connectApi or fusion as a service for chosen indicator type, and not both, as most of the data between both services is duplicated.
  5. The feed size can be changed according to the chosen indicator type:
    • IP - As of September 24, 2020, this risk list includes over 5.9k records.
    • Domain - Due to additional sources of malicious domains added recently, the number of high risk domains collected and analyzed in Recorded Future has dramatically increased. As a result, now cap this risklist at 100,000 domains.
    • Hash - In the second half of 2018, improvements and enhancements to our hash collection and analysis processes led to a dramatic increase in risky hashes that meet the above criteria. As a result, now cap this risklist at 100,000 hashes.
    • URL - This risk list includes 100,000 records.

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

rf-feed-get-indicators

Gets indicators from the feed.

Base Command

rf-feed-get-indicators

Input

Argument Name Description Required
limit The maximum number of results to return. The default value is 10. Default is 10. Required
indicator_type The indicator type. Can be “ip”, “domain”, “hash”, “vulnerability” or “url”. Possible values are: ip, domain, hash, url, vulnerability. Optional

Context Output

There is no context output for this command.

rf-feed-get-risk-rules

Get a list of the risk rules available for an indicator,
To limit the ‘connectApi’ service indicators list.

Base Command

rf-feed-get-risk-rules

Input

Argument Name Description Required
indicator_type The indicator type. Possible values are: ip, domain, hash, url, vulnerability. Required

Context Output

Path Type Description
RecordedFutureFeed.RiskRule.Name String The risk rule name.
RecordedFutureFeed.RiskRule.Description String The risk rule description.
RecordedFutureFeed.RiskRule.Criticality String The risk rule criticality.

Troubleshooting

If indicators expire unexpectedly, ensure that the feed is not receiving more than 100,000 indicators per fetch. As it is discouraged to use “large” as a risk rule, we currently receive indicators in a single large CSV file containing up to 100,000 indicators. If Recorded Future has additional indicators to send, the CSV will be sorted in descending order based on the highest score. Consequently, some indicators may not pass through, leading to their expiration, particularly if the expiration is configured as “When removed from the feed” and they were present in our system from previous fetches.

Configuration parameters

  • feed — Fetch indicators
  • feedReputation — Indicator Reputation
  • feedReliability — Source Reliability (required)
  • tlp_color — Traffic Light Protocol Color
  • feedExpirationPolicy —
  • feedExpirationInterval —
  • feedFetchInterval — Feed Fetch Interval
  • feedBypassExclusionList — Bypass exclusion list
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings
  • indicator_type — Indicator Type (required)
  • api_token — API token
  • credentials_api_token —
  • services — Services (required)
  • risk_rule — Risk Rule
  • fusion_file_path — Fusion File Path
  • feedTags — Tags
  • polling_timeout — Request Timeout (required)
  • threshold — Malicious Threshold
  • suspicious_threshold — Suspicious Threshold
  • risk_score_threshold — IOC Risk Score Threshold
  • performance — Remove rawJSON from indicators

Commands (2)

  • rf-feed-get-indicators

    Gets indicators from the feed.

  • rf-feed-get-risk-rules

    Get a list of the risk rules available for an indicator, To limit the 'connectApi' service indicators list.

Source

FeedRecordedFuture.py FeedRecordedFuture_test.py FeedRecordedFuture.yml FeedRecordedFuture_description.md
Download History Raw 
## Notes
1. It is highly recommended to not create multiple instances of the same indicator type, even when fetching both from fusion and connectApi. Creating multiple instances with same indicator type will lead to duplicate indicators being fetched which can cause performance issues for the server.
2. Because of restrictions on the API side, it is strongly advisable to maintain the number of indicators below 100,000 per instance. Exceeding this limit may result in unforeseen expiration of indicators beyond that threshold.
3. Recommended interval for fetching indicators according to Recorded Future documentation:

| **Indicator Type** | **Recommended Fetch Interval**
| --- | --- |
| IP | 1 Hour. |
| Domain | 2 Hours. |
| Hash | 1 Day. |
| URL | 2 Hours. |
| Vulnerability | 2 Hours. |
3. Per instance configuration, it is recommended to use either `connectApi` or `fusion` as a service for chosen indicator type, and not both, as most of the data between both services is duplicated.
## Recorded Future Feed
This integration downloads from Recorded Future a list of IP addresses, domains, URLs, CVEs or file hashes with known risk associations.
The risk list includes risk scores and supporting evidence details.
- The 'Connect Api' feed can be configured to load a specific risk list by specifying a risk rule.
If no risk rule is specified, Recorded Future’s default risk list is used, which only contains indicators with score above 65.
To find the available risk rules for every indicator you can use the ‘rf-get-risk-rules’ command.
- The 'Fusion' feed be configured to load a custom risk list from a user-specified Recorded Future file path.
If no file path is specified, the default risk list file is used.

Recorded Future's indicators scoring:
- Above 90 - Very Malicious
- Between 65 to 90 - Malicious
- Between 25 to 90 - Suspicious
- Between 5 to 25 - Unusual

To access this resource, a valid Recorded Future API token is required.