ElasticsearchFeed

Fetches indicators stored in an Elasticsearch database.

Data Enrichment & Threat Intelligence · Elasticsearch Feed · Feed

Details

IDElasticsearchFeed
ProviderElasticNV
CategoryData Enrichment & Threat Intelligence
From Version5.5.0
Docker Imagedemisto/elasticsearch:1.0.0.5954979
Supported ModulesAgentix XSIAM

README

Overview

Fetch indicators stored in an Elasticsearch database.

  1. The Cortex XSOAR Feed contains system indicators saved in an Elasticsearch index.
  2. The Cortex XSOAR MT Shared Feed contains indicators shared by a tenant account in a multi-tenant environment.
  3. The Generic Feed contains a feed in a format specified by the user.

Supports version 6 and later.
This integration was integrated and tested with versions 6.6.2, 7.3, 8.4.1 of Elasticsearch.

Configure Elasticsearch Feed in Cortex

Parameter Description Required
Server URL   True
Name (see ?->Authentication) Provide Username + Passoword instead of API key + API ID False
Password   False
Client type For Elasticsearch version 7 and below, select ‘Elasticsearch’. For Elasticsearch server version 8, select ‘Elasticsearch_v8’. In some hosted ElasticSearch environments, the standard ElasticSearch client is not supported. If you encounter any related client issues, please consider using the OpenSearch client type. False
Trust any certificate (not secure)   False
Use system proxy settings   False
Feed Type The Cortex XSOAR Feed contains system indicators saved in an Elasticsearch index. The Cortex XSOAR MT Shared Feed contains indicators shared by a tenant account in a multi-tenant environment. Generic Feed contains a feed in a format specified by the user False
Fetch indicators   False
First Fetch Time Determine how far to look back for fetched indicators (<number> <time unit>, e.g., 12 hours, 7 days). False
Fetch Limit The maximal number of indicators that could be fetched in a fetch cycle. False
Indicator Reputation Indicators from this integration instance will be marked with this reputation False
Source Reliability Reliability of the source providing the intelligence data True
Traffic Light Protocol Color The Traffic Light Protocol (TLP) designation to apply to indicators fetched from the feed False
Feed Fetch Interval   False
Tags Supports CSV values. False
Bypass exclusion list When selected, the exclusion list is ignored for indicators from this feed. This means that if an indicator from this feed is on the exclusion list, the indicator might still be added to the system. False
Indicator Value Field Source field that contains the indicator value in the index. Relevant for generic feed type only. False
Indicator Type Field Source field that contains the indicator type in the index. Relevant for generic feed type only. False
Indicator Type Default indicator type used in case no “Indicator Type Field” was provided. Relevant for generic feed type only. False
Index from Which To Fetch Indicators A comma-separated list of indexes. If empty, searches all indexes. False
Time Field Type   False
Index Time Field Used for sorting and limiting data. If empty, results are not sorted. Relevant for generic feed type only. False
Query Elasticsearch query to execute when fetching indicators from Elasticsearch False

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

es-get-indicators

Gets indicators available in the configured Elasticsearch database.

Base Command

es-get-indicators

Input

Argument Name Description Required
limit The maximum number of indicators to fetch. Default is 50. Required

Context Output

There is no context output for this command.

Known Limitations

Indicator Type Case Sensitivity: The indicator type field in Elasticsearch must exactly match the indicator type in Cortex XSOAR, including case sensitivity. For example, if the indicator type in XSOAR is File (with uppercase ‘F’), then the indicator type in Elasticsearch must also be File, not file (lowercase). If there is a case mismatch, indicators will not be created in XSOAR.

To resolve this issue:

  1. Check the existing indicator types in XSOAR by navigating to: <XSOAR Domain>/configuration/indicator/types
  2. Update your Elasticsearch documents so that the indicator type field matches the exact case of the type in XSOAR
  3. For the Generic Feed type, ensure the field specified in the “Indicator Type Field” parameter returns values that match XSOAR’s indicator types exactly

Configuration parameters

  • url — Server URL (required)
  • credentials — Name (see ?->Authentication)
  • client_type — Client type
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings
  • feed_type — Feed Type
  • feed — Fetch indicators
  • fetch_time — First Fetch Time
  • fetch_limit — Fetch Limit
  • feedReputation — Indicator Reputation
  • feedReliability — Source Reliability (required)
  • tlp_color — Traffic Light Protocol Color
  • feedExpirationPolicy —
  • feedExpirationInterval —
  • feedFetchInterval — Feed Fetch Interval
  • feedTags — Tags
  • feedBypassExclusionList — Bypass exclusion list
  • src_val — Indicator Value Field
  • src_type — Indicator Type Field
  • default_type — Indicator Type
  • fetch_index — Index from Which To Fetch Indicators
  • time_method — Time Field Type
  • time_field — Index Time Field
  • es_query — Query
  • feedIncremental — Incremental Feed
  • enrichmentExcluded — Enrichment Excluded

Commands (1)

  • es-get-indicators

    Gets indicators available in the configured Elasticsearch database.

Source

FeedElasticsearch.py FeedElasticsearch_test.py FeedElasticsearch.yml FeedElasticsearch_description.md
Download History Raw 
### Setting up the integration
* __Server URL__: Elasticsearch database URL. 
* __Client type__: For Elasticsearch version 7 and below, select **Elasticsearch**. For Elasticsearch server version 8, select **Elasticsearch_v8**. In some hosted ElasticSearch environments, the standard ElasticSearch client is not supported. If you encounter any related client issues, please consider using the **OpenSearch** client type.
* __Trust any certificate (not secure)__: Ignore HTTPS certificates.
* __Use system proxy settings__: Enable/Disable
* __Fetch indicators__: Enable/Disable
* __First Fetch Time__: Determine how far to look back for fetched indicators (<number> <time unit>, e.g., 12 hours, 7 days).
* __Indicator Reputation__: Indicators from this integration instance will be marked with this reputation.
* __Source Reliability__: Reliability of the source providing the intelligence data.
* __Indicator Value Field__: Source field that contains the indicator value in the index. Relevant for generic feed type only.
* __Indicator Type Field__: Source field that contains the indicator type in the index. Relevant for generic feed type only.
* __Indicator Type__: Default indicator type used in case no "Indicator Type Field" was provided. Relevant for generic feed type only.
* __Index From Which To Fetch Indicators__: A comma-separated list of indexes. If empty, searches all indexes.
* __Time Field Type__: Time field type used in the database.
* __Index Time Field__: Used for sorting sort and limiting data. If left empty, no sorting is applied. Relevant for generic feed type only.
* __Query__: Elasticsearch query to execute when fetching indicators from Elasticsearch.

#### Authentication
* __Name__: Used for authentication via Username + Password or API ID + API Key (If you wish to use API Key authorization enter **_api_key_id:** followed by your API key ID).
* __Password__: Used for authentication via Username + Password or API ID + API Key (If you wish to use API Key authorization enter your API key).]()

If you wish to use API Key authorization, please enter into the **Password** field your API key, and into the **Name** parameter **_api_key_id:** followed by your API key ID.
For example, for API Key with ID: _VuaCfGcBCdbkQm-e5aOx_ and key value: _ui2lp2axTNmsyakw9tvNnw_ you'll need to enter into name **_api_key_id:VuaCfGcBCdbkQm-e5aOx** and into password you'll need to enter the value: **ui2lp2axTNmsyakw9tvNnw**. For more info about API Key management see: https://www.elastic.co/guide/en/elasticsearch/reference/7.6/security-api-create-api-key.html

#### Feed Type
Fetch indicators stored in an Elasticsearch database. 
1. The Cortex XSOAR Feed contains system indicators saved in an Elasticsearch index. 
2. The Cortex XSOAR MT Shared Feed contains indicators shared by a tenant account in a multi-tenant environment. 
3. The Generic Feed contains a feed in a format specified by the user.