JSON Feed

Fetches indicators from a JSON feed.

Data Enrichment & Threat Intelligence · JSON Feed · Feed

Details

ID	`JSON Feed`
Provider	Open Source
Category	Data Enrichment & Threat Intelligence
From Version	5.5.0
Docker Image	`demisto/py3-tools:1.0.0.114656`
Supported Modules	Agentix XSIAM

README

Use the JSON feed integration to fetch indicators from a JSON feed. This integration allows for a wide variety of user configuration to support different types of JSON feeds.

Configure JSON Feed in Cortex

Parameter	Description
Name	A meaningful name for the integration instance.
Fetch indicators	Whether to fetch indicators, if checked.
Indicator Reputation	The reputation applied to indicators from this integration instance. The default value is “Bad”.
Source Reliability	The reliability of the source providing the intelligence data. The default value is “C - Fairly reliable”
Traffic Light Protocol Color	The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed. More information about the protocol can be found at https://us-cert.cisa.gov/tlp
Indicator Expiration Method	The method by which to expire indicators from this feed for this integration instance.
Feed Fetch Interval	How often to fetch indicators from the feed for this integration instance (in minutes). The default value is 60.
URL	The URL of the feed.
Auto detect indicator type	Whether a type auto detection mechanism will take place for each indicator, if checked.
Indicator Type	The type of the indicator in the feed. This is relevant only if `Auto detect` is not checked.
Username + Password	The credentials used to access feeds that require basic authentication. These fields also support the use of API key headers. To use API key headers, specify the header name and value in the following format: `_header:<header_name>` in the Username field and the header value in the Password field.
JMESPath Extractor	The JMESPath expression for extracting the indicators from. You can check the expression in the JMESPath site to verify this expression will return the following array of objects.
JSON Indicator Attribute	The JSON attribute whose value is the indicator. The default is “indicator”.
POST Data	Send specified data in a POST request. When specified, by default will add the header: `Content-Type: application/x-www-form-urlencoded`. To specify a different Content-Type (for example: application/json) use the Headers config param.
Headers	Headers to add to the http request. Specify each header on a single line in the format: `Name: Value`.
Include indicator type for mapping	When using a custom classifier and mapper with this feed, use this option to include the indicator type in the raw json used for classification and mapping. The type will be included under the key `_indicator_type`.
Bypass Exclusion List	Whether the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system.

Step-by-step configuration

IP address ranges from Amazon AWS will be used as examples. The feed will ingest indicators of the CIDR type. These are the feed instance configuration parameters for our example.

URL: https://ip-ranges.amazonaws.com/ip-ranges.json

Auto detect indicator type: Checked.

Indicator Type - Leave this empty and the system will identify the indicator type.

Credentials - This feed does not require authentication.

The following parameters will be configured based on the feed in the web browser.

JMESPath Extractor - prefixes[?service==’AMAZON’] This means that the desired objects to extract the indicators from is
prefixes, and the objects will be filtered by where the field service is equal to AMAZON.

JSON Indicator Attribute - The ip_prefix.

At this point, an instance for the IP ranges from Amazon AWS has been successfully configured. After Fetches indicators have been enabled, the instance will start pulling indicators.

By clicking Mapping in the integration instance, the field names we previously configured can be mapped to the actual indicator fields (except value which is the indicator value).
We can use Set up a new classification rule using actual data from the feed.

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

Get indicators from the feed

Gets the feed indicators.

Base Command

!json-get-indicators

Input

Argument Name	Description	Required
limit	The maximum number of results to return. The default value is 50.	Optional

Context Output

There is no context output for this command.

Demo Video

Sorry, your browser doesn't support embedded videos. You can download the video at: https://github.com/demisto/content-assets/blob/7982404664dc68c2035b7c701d093ec026628802/Assets/FeedJSON/Json_generic_feed_demo.mp4

Configuration parameters

feed — Fetch indicators
feedReputation — Indicator Reputation
feedReliability — Source Reliability (required)
tlp_color — Traffic Light Protocol Color
feedExpirationPolicy —
feedExpirationInterval —
feedFetchInterval — Feed Fetch Interval
url — URL (required)
auto_detect_type — Auto detect indicator type
remove_ports — Remove IPv4 Ports
indicator_type — Indicator Type
credentials — Username
extractor — JMESPath Extractor (required)
indicator — JSON Indicator Attribute
data — POST Data
headers — Headers
rawjson_include_indicator_type — Include indicator type for mapping
insecure — Trust any certificate (not secure)
proxy — Use system proxy settings
feedBypassExclusionList — Bypass exclusion list
feedTags — Tags
enrichmentExcluded — Enrichment Excluded

Commands (1)

json-get-indicators
Gets the feed indicators.

Source

FeedJSON.py FeedJSON.yml FeedJSON_description.md

Download History Raw

category: Data Enrichment & Threat Intelligence
provider: Open Source
commonfields:
  id: JSON Feed
  version: -1
configuration:
- defaultvalue: 'true'
  display: Fetch indicators
  name: feed
  type: 8
  required: false
  section: Collect
- additionalinfo: Indicators from this integration instance will be marked with this reputation
  defaultvalue: feedInstanceReputationNotSet
  display: Indicator Reputation
  name: feedReputation
  options:
  - None
  - Good
  - Suspicious
  - Bad
  type: 18
  required: false
  section: Collect
- additionalinfo: Reliability of the source providing the intelligence data
  defaultvalue: F - Reliability cannot be judged
  display: Source Reliability
  name: feedReliability
  options:
  - A - Completely reliable
  - B - Usually reliable
  - C - Fairly reliable
  - D - Not usually reliable
  - E - Unreliable
  - F - Reliability cannot be judged
  required: true
  type: 15
  section: Collect
- additionalinfo: The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed
  display: Traffic Light Protocol Color
  name: tlp_color
  options:
  - RED
  - AMBER
  - GREEN
  - WHITE
  type: 15
  required: false
  section: Collect
- defaultvalue: indicatorType
  display: ''
  name: feedExpirationPolicy
  options:
  - never
  - interval
  - indicatorType
  - suddenDeath
  type: 17
  required: false
  section: Collect
  advanced: true
- defaultvalue: '20160'
  display: ''
  name: feedExpirationInterval
  type: 1
  required: false
  section: Collect
  advanced: true
- defaultvalue: '240'
  display: Feed Fetch Interval
  name: feedFetchInterval
  type: 19
  required: false
  section: Collect
  advanced: true
- display: URL
  name: url
  required: true
  type: 0
  section: Connect
- additionalinfo: If selected, the indicator type will be auto detected for each indicator.
  defaultvalue: 'true'
  display: Auto detect indicator type
  name: auto_detect_type
  type: 8
  required: false
  section: Collect
  advanced: true
- additionalinfo: Remove ports from IPv4 type indicators in the fetch command (e.g. ,192.168.1.1:8080 -> 192.168.1.1)
  display: Remove IPv4 Ports
  name: remove_ports
  type: 8
  required: false
  defaultvalue: 'false'
  section: Collect
  advanced: true
- display: Indicator Type
  name: indicator_type
  type: 0
  required: false
  additionalinfo: Type of the indicator in the feed. If auto-detect is checked then the value set as Indicator Type will be ignored.
  section: Collect
  advanced: true
- display: Username
  name: credentials
  required: false
  type: 9
  section: Connect
- additionalinfo: JMESPath expression for extracting the indicators. You can use http://jmespath.org/ to identify the proper expression.
  display: JMESPath Extractor
  name: extractor
  type: 0
  required: true
  section: Collect
- additionalinfo: "The JSON attribute that holds the indicator value. Default value is 'indicator'."
  display: JSON Indicator Attribute
  name: indicator
  type: 0
  required: false
  section: Collect
  advanced: true
- additionalinfo: "Send specified data in a POST request. When specified, by default will add the header: 'Content-Type: application/x-www-form-urlencoded'. To specify a different Content-Type (for example: application/json) use the Headers config param."
  display: POST Data
  name: data
  type: 0
  required: false
  section: Collect
  advanced: true
- additionalinfo: "Headers to add to the http request. Specify each header on a single line in the format: 'Name: Value'. For example: 'User-Agent: XSOAR Feed'"
  display: Headers
  name: headers
  type: 12
  required: false
  section: Collect
  advanced: true
- display: Include indicator type for mapping
  name: rawjson_include_indicator_type
  type: 8
  required: false
  additionalinfo: "When using a custom classifier and mapper with this feed, use this option to include the indicator type in the raw JSON used for classification and mapping."
  section: Collect
  advanced: true
- display: Trust any certificate (not secure)
  name: insecure
  type: 8
  required: false
  section: Connect
  advanced: true
- display: Use system proxy settings
  name: proxy
  type: 8
  required: false
  section: Connect
  advanced: true
- additionalinfo: "When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system."
  display: Bypass exclusion list
  name: feedBypassExclusionList
  type: 8
  required: false
  section: Collect
  advanced: true
- additionalinfo: Supports CSV values.
  display: Tags
  name: feedTags
  type: 0
  required: false
  section: Collect
  advanced: true
- display: Enrichment Excluded
  name: enrichmentExcluded
  type: 8
  required: false
  defaultvalue: 'false'
  additionalinfo: Select this option to exclude the fetched indicators from the enrichment process.
  hidden:
  - xsoar_on_prem
  section: Collect
description: Fetches indicators from a JSON feed.
display: JSON Feed
name: JSON Feed
script:
  commands:
  - arguments:
    - defaultValue: '50'
      description: The maximum number of results to return. The default value is 50.
      name: limit
    - defaultValue: 'false'
      description: Remove ports from IPv4s.
      name: remove_ports
    description: Gets the feed indicators.
    name: json-get-indicators
  dockerimage: demisto/py3-tools:1.0.0.114656
  feed: true
  runonce: false
  script: '-'
  subtype: python3
  type: python
tests:
- JSON_Feed_Test
fromversion: 5.5.0
sectionorder:
- Connect
- Collect