Recorded Future Feed

Details

ID	`Recorded Future Feed`
Provider	Mastercard
Category	Data Enrichment & Threat Intelligence
From Version	5.5.0
Docker Image	`demisto/python3:3.12.13.10404775`
Supported Modules	Agentix XSIAM

README

Ingests indicators from Recorded Future feeds into Cortex XSOAR.
This integration was integrated and tested with Recorded Future Feed

Configure Recorded Future Feed on Cortex XSOAR

Navigate to Settings > Integrations > Servers & Services.
Search for Recorded Future Feed.

Click Add instance to create and configure a new integration instance.

Parameter	Description	Required
Fetch indicators		False
Indicator Reputation	Indicators from this integration instance will be marked with this reputation	False
Source Reliability	Reliability of the source providing the intelligence data	True
Traffic Light Protocol Color	The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed	False
Feed Fetch Interval		False
Bypass exclusion list	When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system.	False
Trust any certificate (not secure)		False
Use system proxy settings		False
Indicator Type	Type of the indicator in the feed.	True
API token		True
Services		True
Risk Rule	A comma-separated list of risk rules which limits the indicators list to a specific risk rule. For example: ‘dhsAis,phishingUrl’. If more than one risk rule is set, the indicators fetching and the ‘rf-feed-get-indicators’ command will be executed for each risk rule. To see available risk rules run the rf-feed-get-risk-rules command. This parameter will only be used for the ‘connectApi’ service. Using the ‘large’ risk rule is not recommended.	False
Fusion File Path	Load a custom risklist from a specified Recorded Future file path. If no file path is specified, the default risklist file is used. This parameter will only be used for the ‘fusion’ service.	False
Tags	Supports CSV values.	False
Request Timeout	Time in seconds before HTTP requests timeout.	True
Malicious Threshold	The minimum score from the feed in order to to determine whether the indicator is malicious. Default is “65”. For more information about Recorded Future scoring go to integration details.	False
IOC Risk Score Threshold	If selected, will be used to filter out the ingested indicators, and only indicators with equivalent and higher risk score will be ingested into XSOAR.	False

Click Test to validate the URLs, token, and connection.

Notes

It is highly recommended to not create multiple instances of the same indicator type, even when fetching both from fusion and connectApi. Creating multiple instances with same indicator type will lead to duplicate indicators being fetched which can cause performance issues for the server.
Because of restrictions on the API side, it is strongly advisable to maintain the number of indicators below 100,000 per instance. Exceeding this limit may result in unforeseen expiration of indicators beyond that threshold.

Recommended interval for fetching indicators according to Recorded Future documentation:

Indicator Type	Recommended Fetch Interval
IP	1 Hour.
Domain	2 Hours.
Hash	1 Day.
URL	2 Hours.
Vulnerability	2 Hours.

Per instance configuration, it is recommended to use either connectApi or fusion as a service for chosen indicator type, and not both, as most of the data between both services is duplicated.
The feed size can be changed according to the chosen indicator type:
- IP - As of September 24, 2020, this risk list includes over 5.9k records.
- Domain - Due to additional sources of malicious domains added recently, the number of high risk domains collected and analyzed in Recorded Future has dramatically increased. As a result, now cap this risklist at 100,000 domains.
- Hash - In the second half of 2018, improvements and enhancements to our hash collection and analysis processes led to a dramatic increase in risky hashes that meet the above criteria. As a result, now cap this risklist at 100,000 hashes.
- URL - This risk list includes 100,000 records.

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

rf-feed-get-indicators

Gets indicators from the feed.

Base Command

rf-feed-get-indicators

Input

Argument Name	Description	Required
limit	The maximum number of results to return. The default value is 10. Default is 10.	Required
indicator_type	The indicator type. Can be “ip”, “domain”, “hash”, “vulnerability” or “url”. Possible values are: ip, domain, hash, url, vulnerability.	Optional

Context Output

There is no context output for this command.

rf-feed-get-risk-rules

Get a list of the risk rules available for an indicator,
To limit the ‘connectApi’ service indicators list.

Base Command

rf-feed-get-risk-rules

Input

Argument Name	Description	Required
indicator_type	The indicator type. Possible values are: ip, domain, hash, url, vulnerability.	Required

Context Output

Path	Type	Description
RecordedFutureFeed.RiskRule.Name	String	The risk rule name.
RecordedFutureFeed.RiskRule.Description	String	The risk rule description.
RecordedFutureFeed.RiskRule.Criticality	String	The risk rule criticality.

Troubleshooting

If indicators expire unexpectedly, ensure that the feed is not receiving more than 100,000 indicators per fetch. As it is discouraged to use “large” as a risk rule, we currently receive indicators in a single large CSV file containing up to 100,000 indicators. If Recorded Future has additional indicators to send, the CSV will be sorted in descending order based on the highest score. Consequently, some indicators may not pass through, leading to their expiration, particularly if the expiration is configured as “When removed from the feed” and they were present in our system from previous fetches.

Configuration parameters

feed — Fetch indicators
feedReputation — Indicator Reputation
feedReliability — Source Reliability (required)
tlp_color — Traffic Light Protocol Color
feedExpirationPolicy —
feedExpirationInterval —
feedFetchInterval — Feed Fetch Interval
feedBypassExclusionList — Bypass exclusion list
insecure — Trust any certificate (not secure)
proxy — Use system proxy settings
indicator_type — Indicator Type (required)
api_token — API token
credentials_api_token —
services — Services (required)
risk_rule — Risk Rule
fusion_file_path — Fusion File Path
feedTags — Tags
polling_timeout — Request Timeout (required)
threshold — Malicious Threshold
suspicious_threshold — Suspicious Threshold
risk_score_threshold — IOC Risk Score Threshold
performance — Remove rawJSON from indicators

Commands (2)

rf-feed-get-indicators
Gets indicators from the feed.
rf-feed-get-risk-rules
Get a list of the risk rules available for an indicator, To limit the 'connectApi' service indicators list.

Source

FeedRecordedFuture.py FeedRecordedFuture_test.py FeedRecordedFuture.yml FeedRecordedFuture_description.md

Download History Raw

category: Data Enrichment & Threat Intelligence
provider: Mastercard
sectionorder:
- Connect
- Collect
commonfields:
  id: Recorded Future Feed
  version: -1
configuration:
- display: Fetch indicators
  name: feed
  defaultvalue: 'true'
  type: 8
  required: false
  section: Collect
- display: Indicator Reputation
  name: feedReputation
  defaultvalue: feedInstanceReputationNotSet
  type: 18
  options:
  - None
  - Good
  - Suspicious
  - Bad
  additionalinfo: Indicators from this integration instance will be marked with this reputation
  required: false
  section: Collect
- defaultvalue: A - Completely reliable
  display: Source Reliability
  name: feedReliability
  options:
  - A - Completely reliable
  - B - Usually reliable
  - C - Fairly reliable
  - D - Not usually reliable
  - E - Unreliable
  - F - Reliability cannot be judged
  required: true
  type: 15
  additionalinfo: Reliability of the source providing the intelligence data
  section: Collect
- additionalinfo: The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed
  display: Traffic Light Protocol Color
  name: tlp_color
  options:
  - RED
  - AMBER
  - GREEN
  - WHITE
  type: 15
  required: false
  section: Collect
- display: ""
  name: feedExpirationPolicy
  defaultvalue: indicatorType
  type: 17
  options:
  - never
  - interval
  - indicatorType
  - suddenDeath
  required: false
  section: Collect
- name: feedExpirationInterval
  defaultvalue: "20160"
  type: 1
  display: ""
  required: false
  section: Collect
- defaultvalue: '60'
  display: Feed Fetch Interval
  name: feedFetchInterval
  type: 19
  required: false
  section: Collect
- display: Bypass exclusion list
  name: feedBypassExclusionList
  type: 8
  additionalinfo: |-
    When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system.
  required: false
  section: Collect
  advanced: true
- display: Trust any certificate (not secure)
  name: insecure
  type: 8
  required: false
  section: Connect
  advanced: true
- display: Use system proxy settings
  name: proxy
  type: 8
  required: false
  section: Connect
  advanced: true
- additionalinfo: Type of the indicator in the feed.
  display: Indicator Type
  name: indicator_type
  options:
  - domain
  - ip
  - hash
  - url
  - CVE(vulnerability)
  required: true
  type: 15
  section: Collect
- display: API token
  name: api_token
  type: 4
  hidden: true
  required: false
  section: Connect
- displaypassword: API Token
  name: credentials_api_token
  hiddenusername: true
  type: 9
  required: false
  display: ''
  section: Connect
- defaultvalue: connectApi
  display: Services
  name: services
  options:
  - connectApi
  - fusion
  required: true
  type: 16
  section: Connect
- additionalinfo: |-
    A comma-separated list of risk rules which limits the indicators list to a specific risk rule.
    For example: 'dhsAis,phishingUrl'.
    If more than one risk rule is set, the indicators fetching and the 'rf-feed-get-indicators' command will be executed for each risk rule.
    To see available risk rules run the rf-feed-get-risk-rules command.
    This parameter will only be used for the 'connectApi' service.
    Using the 'large' risk rule is not recommended.
  display: Risk Rule
  name: risk_rule
  type: 0
  required: false
  section: Collect
- additionalinfo: |-
    Load a custom risklist from a specified Recorded Future file path.
    If no file path is specified, the default risklist file is used. This parameter
    will only be used for the 'fusion' service.
  display: Fusion File Path
  name: fusion_file_path
  type: 0
  required: false
  section: Collect
  advanced: true
- additionalinfo: Supports CSV values.
  display: Tags
  name: feedTags
  type: 0
  required: false
  section: Collect
  advanced: true
- additionalinfo: Time in seconds before HTTP requests timeout.
  defaultvalue: '20'
  display: Request Timeout
  name: polling_timeout
  required: true
  type: 0
  section: Collect
- additionalinfo: |-
    The minimum score from the feed in order to determine whether the indicator is malicious. Default is "65". For more information about Recorded Future scoring go to integration details.
  defaultvalue: '65'
  display: Malicious Threshold
  name: threshold
  type: 0
  required: false
  section: Collect
  advanced: true
- additionalinfo: The minimum score from the feed in order to determine whether the indicator is Suspicious. Ranges up to the Malicious Threshold. Default is "25". For more information about Recorded Future scoring go to integration details.
  defaultvalue: '25'
  display: Suspicious Threshold
  name: suspicious_threshold
  type: 0
  required: false
  section: Collect
  advanced: true
- additionalinfo: If selected, will be used to filter out the ingested indicators, and only indicators with equivalent and higher risk score will be ingested into XSOAR. Scores greater than the IOC Risk Score Threshold and less than the Suspicious Threshold will have an unknown verdict.
  defaultvalue: '0'
  display: IOC Risk Score Threshold
  name: risk_score_threshold
  required: false
  type: 0
  section: Collect
  advanced: true
- additionalinfo: 'Improve the feed performance by removing the rawJSON from the indicators. Note: A loss of data may occur if selecting this checkbox.'
  display: Remove rawJSON from indicators
  name: performance
  defaultvalue: "true"
  type: 8
  required: false
  section: Collect
  advanced: true
description: Ingests indicators from Recorded Future feeds into Demisto.
display: Recorded Future RiskList Feed
name: Recorded Future Feed
script:
  commands:
  - arguments:
    - default: true
      defaultValue: '10'
      description: The maximum number of results to return. The default value is 10.
      name: limit
      required: true
    - auto: PREDEFINED
      description: The indicator type. Can be "ip", "domain", "hash", "vulnerability" or "url".
      name: indicator_type
      predefined:
      - ip
      - domain
      - hash
      - url
      - vulnerability
    description: Gets indicators from the feed.
    name: rf-feed-get-indicators
  - arguments:
    - auto: PREDEFINED
      description: The indicator type.
      name: indicator_type
      predefined:
      - ip
      - domain
      - hash
      - url
      - vulnerability
      required: true
    description: |-
      Get a list of the risk rules available for an indicator,
      To limit the 'connectApi' service indicators list.
    name: rf-feed-get-risk-rules
    outputs:
    - contextPath: RecordedFutureFeed.RiskRule.Name
      description: The risk rule name.
      type: String
    - contextPath: RecordedFutureFeed.RiskRule.Description
      description: The risk rule description.
      type: String
    - contextPath: RecordedFutureFeed.RiskRule.Criticality
      description: The risk rule criticality.
      type: String
  dockerimage: demisto/python3:3.12.13.10404775
  feed: true
  runonce: false
  script: ''
  subtype: python3
  type: python
tests:
- RecordedFutureFeed - Test
fromversion: 5.5.0