ElasticsearchEventCollector

Details

ID	`ElasticsearchEventCollector`
Provider	Elastic
Category	Database
From Version	8.4.0
Docker Image	`demisto/elasticsearch:1.0.0.10133006`
Supported Modules	Agentix Cloud Runtime Security XSIAM EDR Cortex Cloud Attack Surface Management Cloud Posture Security Exposure Management

README

Elasticsearch Event Collector integration is used to search for and analyze data in real time.
Supports version 6 and later.

Configure Elasticsearch Event Collector in Cortex

Parameter	Description	Required
Server URL	The Elasticsearch server to which the integration connects. Ensure that the URL includes the correct Elasticsearch port. The default port for Elasticsearch v7 and below is 9200. Use the Server URL for on-premises deployments.	False
Authorization type	Select the authentication type and enter the appropriate credentials: - Basic Auth: Enter Username and Password. - Bearer Auth: Enter Username and Password. - API Key Auth: Enter the API Key ID and API Key.	True
API key ID	Use for API key auth	False
API Key	Use for API key auth	False
Username	Use for Basic auth. Optionally you can use Username as an API key ID and Password as an API key for API Key auth.	False
Password	Use for Basic auth. Optionally you can use Username as an API key ID and Password as an API key for API Key auth.	False
Trust any certificate (not secure)		False
Use system proxy settings		False
Client type	In some hosted ElasticSearch environments, the standard ElasticSearch client is not supported. If you encounter any related client issues, please consider using the OpenSearch client type.	False
Index to fetch events from	CSV	False
Query String	Query string uses the Lucene syntax. For more information about the Lucene syntax see here. The query will be used when fetching events. Index time field will be used as a filter in the query.	False
Index time field	The time field used for sorting and limiting results. If using a nested field, separate field names with dot notation.	False
Raw Query	Raw Query allows raw DSL queries and will override the ‘Query String’ Lucene syntax string. Results will not be filtered or sorted. Use additional parameters in the raw query for these purposes. For more information about Query DSL see here.	False
Time field type	3 formats supported: Simple-Date - A plain date string. You must specify the format in which the date is stored. For more information about time formatting, see here. Timestamp-Second - A numeric value representing the number of seconds since the Unix epoch (00:00:00 UTC on 1 January 1970). Example: ‘1572164838’ * Timestamp-Milliseconds - A numeric value representing the number of milliseconds since the Unix epoch. Example: ‘1572164838123’	False
Map JSON fields into labels		False
The maximum number of results per fetch		False
Request timeout (in seconds).		False
Fetch events		False

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

es-get-events

Gets events from Elasticsearch.
This command is used for developing/ debugging and is to be used with caution, as it can cause the API request limit to be exceeded.

Base Command

es-get-events

Input

Argument Name	Description	Required
start_time	Start time for fetching events. Supports ISO format (“2023-01-01T23:59:59”) or natural language (“2 hours ago”, “now”).	Required
end_time	End time for fetching events. Supports ISO format (“2023-01-01T23:59:59”) or natural language (“2 hours ago”, “now”).	Optional
time_method	For more information see the explanation in the help section.	Required
fetch_size	The maximum number of results per fetch, default 10.	Optional
fetch_index	CSV.	Optional
fetch_time_field	The time field used for sorting and limiting results. If using a nested field, separate field names with dot notation.	Required
fetch_query	Query string uses the Lucene syntax.	Optional
raw_query	Raw Query allows raw DSL queries and will override the ‘Query String’ Lucene syntax string.	Optional

Context Output

There is no context output for this command.

Configuration parameters

url — Server URL (required)
auth_type — Authorization type (required)
api_key_auth_credentials — API key ID
credentials — Username
insecure — Trust any certificate (not secure)
proxy — Use system proxy settings
client_type — Client type
fetch_index — Index to fetch incidents from
fetch_query — Query String
fetch_time_field — Index time field
raw_query — Raw Query
time_method — Time field type
map_labels — Map JSON fields into labels
fetch_size — The maximum number of results per fetch
timeout — Request timeout (in seconds).
isFetchEvents — Fetch events

Commands (1)

es-get-events
This command is used for developing/ debugging and is to be used with caution, as it can cause the API request limit to be exceeded.

Source

ElasticsearchEventCollector.py ElasticsearchEventCollector_test.py ElasticsearchEventCollector.yml ElasticsearchEventCollector_description.md

Download History Raw

The Elasticsearch Event Collector integration supports Elasticsearch 6.0.0 and later.
This integration was integrated and tested with versions 6.6.2, 7.3, 8.4.1 of Elasticsearch.

## Authentication

There are 3 different authentication [methods](https://www.elastic.co/docs/api/doc/elasticsearch#doc-authentication)

### Basic Auth (http)

To use **Basic Authentication**:

* Select **Basic Auth** from the *Authorization type* dropdown.
* Enter your **Username** in the *Username* field.
* Enter your **Password** in the *Password* field.

### API Key Auth (http_api_key)

To use **API Key Authentication**:

* Select **API Key Auth** from the *Authorization type* dropdown.
* Enter your **API key ID** in the *API key ID* field.
* Enter your **API key** in the *API key* field.

For more info about API Key management see [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/security-api-create-api-key.html)

**Note:** Alternatively, you can select the **Basic Auth** type and enter the API key ID in the *Username* field and the API key in the *Password* field.
Example:
for *API Key ID* kQme5aOx enter: _api_key_id:kQme5aOx
for *API Key* ui2lp2axT enter: ui2lp2axT

### Bearer Auth (http)

To use **Bearer Authentication**:

* Select **Bearer Auth** type from the *Authorization type* dropdown.
* Enter your **Username** in the *Username* field.
* Enter your **Password** in the *Password* field.

For more info see [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/security-api-get-token.html#security-api-get-token-prereqs)

## Notes

* Not all fields can be used for sorting in Elasticsearch. Sorting is only supported for fields of the following types: **boolean**, **numeric**, **date**, and **keyword**.
* The "Test" button does not fully validate the fetch events functionality.

## Additional Configuration Parameters Details

Fetch events requires:

* Index
* Index time field
* Query String or Raw Query

For further information about type mapping, see [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/mapping.html#mapping-type).

**Query String**
Query String is queried using the Lucene syntax. For more information about Lucene syntax see [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/query-dsl-query-string-query.html#query-string-syntax).

**Raw Query**
Allows raw DSL queries. For more information about Query DSL see [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html).

**Time field type**
3 formats supported:

* Simple-Date - A plain date string. You must specify the format in which the date is stored. For more information about time formatting, see [here](http://strftime.org/).
* Timestamp-Second - A numeric value representing the number of seconds since the Unix epoch (00:00:00 UTC on 1 January 1970). Example: '1572164838'
* Timestamp-Milliseconds - A numeric value representing the number of milliseconds since the Unix epoch. Example: '1572164838123'