TAXII2 Server

This integration provides TAXII2 Services for system indicators (Outbound feed).

Data Enrichment & Threat Intelligence · TAXII Server

Details

IDTAXII2 Server
ProviderOpen Source
CategoryData Enrichment & Threat Intelligence
From Version6.1.0
Docker Imagedemisto/flask-nginx:1.0.0.11047721
Supported ModulesAgentix XSIAM

README

TAXII2 Server Integration

This integration provides TAXII2 Services for system indicators (outbound feed).
You can choose to use TAXII v2.0 or TAXII v2.1.

The TAXII2 Server integration is a long-running integration. For more information about long-running integrations, see the Cortex XSOAR 8 Cloud, Cortex XSOAR 8 On-prem or Cortex XSIAM documentation.

Configure Collections

Each TAXII collection in the integration is represented by a Cortex XSOAR indicator query.

The collections are defined by a JSON object in the following format:

{
  "collection1_name":{
    "query": "<Cortex XSOAR indicator query>",
    "description": "<Custom collection description>"
  },
  "collection2_name": "<Cortex XSOAR indicator query>"
}

You can add a collection description as is done in collection1_name, or enter only a collection query, as in collection2_name.

How to Access the TAXII2 Server

For Cortex XSOAR 6.x

Use one of the following options:

  • https://<xsoar_address>/instance/execute/<instance_name>/<taxii2_api_endpoint>/
  • http://<xsoar_address>:<listen_port>/<taxii2_api_endpoint>/

For Cortex XSOAR 8 On-prem, Cortex XSOAR 8 Cloud, or Cortex XSIAM

Use https://ext-<tenant>/xsoar/instance/execute/<instance-name>/<taxii2_api_endpoint>/

Note:

  • For Cortex XSOAR 8 On-prem, you need to add the ext- FQDN DNS record to map the Cortex XSOAR DNS name to the external IP address.
    For example, ext-xsoar.mycompany.com.
  • The instance name cannot be changed after saving the integration configuration.

Access the TAXII Service by Instance Name

To access the TAXII service by instance name, make sure Instance execute external is enabled.

In Cortex XSOAR 6.x:

  1. Navigate to Settings > About > Troubleshooting.
  2. In the Server Configuration section, verify that the instance.execute.external key is set to true. If this key does not exist, click + Add Server Configuration, add the instance.execute.external and set the value to true.

How to Use HTTPS

To use HTTPS, a certificate and private key have to be supplied in the integration configuration.

Set up Authentication

For Cortex XSOAR 8 Cloud Tenant or Cortex XSIAM Tenant

The TAXII2 Server integration running on a Cortex XSOAR 8 Cloud tenant or Cortex XSIAM tenant enables using basic authentication in the requests.
To enable basic authentication, a user and password must be supplied in the Credentials parameters in the integration configuration.
The server will then authenticate the requests by the Authorization header, expecting basic authentication encrypted in base64 to match the given credentials.

For Cortex XSOAR On-prem (6.x or 8) or When Using Engines

For Cortex XSOAR On-prem (6.x or 8) or when using engines, you can set up authentication using custom certificates. For more information on setting up a custom certificate for Cortex XSOAR 8 On-prem, see HTTPS with a signed certificate. For more information on setting up a custom certificate for Cortex XSOAR 6.x, see HTTPS with a Signed Certificate.

TAXII v2.0 API Endpoints

URL Method Response TAXII2 Documentation
/taxii/ GET Server Discovery Information. Server Discovery
/{api_root}/ GET Cortex XSOAR API root is threatintel. API Root Information
/{api_root}/collections/ GET All Cortex XSOAR collections that configure in Collection JSON parameter. Collections Resource
/{api_root}/collections/{collection_id}/ GET Cortex XSOAR Collection with given collection_id. Collection Response
/{api_root}/collections/{collection_id}/manifest/ GET Object manifests from the given collection. Objects Manifest Resource
/{api_root}/collections/{collection_id}/objects/ GET Objects (Cortex XSOAR Indicators) from the given collection. Object Resource

For more information, visit TAXII2 Documentation.

TAXII v2.1 API Endpoints

URL Method Response TAXII2 Documentation
/taxii2/ GET Server Discovery Information. Server Discovery
/{api_root}/ GET XSOAR API root is threatintel. API Root Information
/{api_root}/collections/ GET All Cortex XSOAR collections that configure in Collection JSON parameter. Collections Resource
/{api_root}/collections/{collection_id}/ GET Cortex XSOAR Collection with given collection_id. Collection Response
/{api_root}/collections/{collection_id}/manifest/ GET Object manifests from the given collection. Objects Manifest Resource
/{api_root}/collections/{collection_id}/objects/ GET Objects (Cortex XSOAR Indicators and Relationships) from the given collection. Object Resource

For more information, visit TAXII2 Documentation.

  • When retrieving Objects (Cortex XSOAR Indicators and Relationships) from a collection, use the next parameter to paginate through additional records.

Known Limitations

  • GET objects by ID is not allowed.
  • Filtering objects by ID or version not allowed.
  • POST and DELETE objects are not allowed. Cannot add or delete indicators using TAXII2 Server.

How UUIDs Work for TAXII2 in Cortex XSOAR


STIX Cyber Objects (SCO)

All STIX SCOs UUIDs follow STIX 2.1 guidelines and use UUID5 with STIX unique namespace
(00abedb4-aa42-466c-9c01-fed23315a9b7). This is used so all SCOs created have persistent UUID across all producers.

STIX Domain Objects (SDO)

Unlike SCOs, STIX 2.1 specs for SDOs require a UUID4. While this solution works if the UUID is part of the database,
it is not the case in Cortex XSOAR. If the SDO already has a unique UUID stored it will use it, if not it will generate a unique and persistent UUID using the following method.

A general UUID5 is created using the NameSpace_URL as follows:

PAWN_UUID = uuid.uuid5(uuid.NAMESPACE_URL, 'https://www.paloaltonetworks.com')

The generated UUID is then used to create a unique UUID5 per customer:

UNIQUE_UUID = uuid.uuid5(PAWN_UUID, <UniqueCostumerString>)

We then use this UUID as a base namespace to generate UUIDs for SDOs following the STIX 2.1 specs. Using this method,
we create unique and persistent UUIDs per customer.

Cortex XSOAR TIM Extension Fields


When selected in the integration settings (Cortex XSOAR Extension Fields) the TAXII2 integration will generate an extension object and an extension attribute that holds Cortex XSOAR additional
TIM fields (system generated and custom). An example of these two related objects:

{
  "id": "extension-definition--<UUID>",
  "type": "extension-definition",
  "spec_version": "2.1",
  "name": "XSOAR TIM <Cortex XSOAR Type>",
  "description": "This schema adds TIM data to the object",
  "created": "<creation date>",
  "modified": "<modification date>",
  "created_by_ref": "identity--<UUID of creator>",
  "schema": "https://github.com/demisto/content/blob/4265bd5c71913cd9d9ed47d9c37d0d4d3141c3eb/Packs/TAXIIServer/doc_files/XSOAR_indicator_schema.json",
  "version": "1.0",
  "extension_types": ["property-extension"]
},
{
    "type": "ipv4-addr",
    "spec_version": "2.1",
    "id": "ipv4-addr--2f689bf9-0ff2-545f-aa61-e495eb8cecc7",
    "value": "8.8.8.8",
    "extensions":{
        "extension-definition--<UUID>": {
           "Extension_type": "property_extension",
           "Field_1": "Value1",
           "Field_2": "Value2",
           "Field_3": "Value3"
        }
    }
}

Performance Benchmark

Indicators Amount Request time (seconds)
10,000 5-10
50,000 30-40
100,000 50-90

Microsoft Sentinel Configuration Guide

Configure the TAXII2 Server Instance

  1. Set TAXII2 Server version to 2.0 (The integration currently doesn’t work with Microsoft Sentinel in TAXII Version 2.1).

  2. Under STIX types for STIX indicator Domain Object select the indicator types you want to ingest.

  3. Set the Listen Port and Collection JSON to your linking.

Find the Information Required for the Sentinel TAXII Connector

For Cortex XSOAR 6.x

  1. All your server info can be found by running !taxii-server-info, the default API root for you server will usually be - https://<xsoar-server>/instance/execute/<instance_name>/threatintel/
  2. You can use the !taxii-server-list-collections command in order to get a list of your server’s collections and their IDs. You can also do it manually by running curl https://<xsoar-server>/instance/execute/<instance_name>/threatintel/collections/ | jq . to get a list of the collections available and on your TAXII server. From the list, copy the correct ID of the collection you want to ingest.

For Cortex XSOAR 8 On-prem, Cortex XSOAR Cloud, or Cortex XSIAM

  1. All your server info can be found by running !taxii-server-info, the default API root for you server will usually be - https://ext-<tenant>.crtx.<region>.paloaltonetworks.com/xsoar/instance/execute/<instance-name>/threatintel/
  2. You can use the !taxii-server-list-collections command in order to get a list of your server’s collections and their IDs. You can also do it manually by running curl https://ext-<tenant>.crtx.<region>.paloaltonetworks.com/xsoar/instance/execute/<instance-name>/threatintel/collections/ | jq . to get a list of the collections available and on your TAXII server. From the list, copy the correct ID of the collection you want to ingest.

Response Example:

  {
    "collections": [
      {
        "can_read": true,
        "can_write": false,
        "description": "",
        "id": "3709e6df-bbe1-5ece-b683-e99a42f31fce",
        "media_types": [
          "application/vnd.oasis.stix+json; version=2.0"
        ],
        "query": "type:IP",
        "title": "AllIPs"
      },
      {
        "can_read": true,
        "can_write": false,
        "description": "",
        "id": "16763019-5ee6-59bc-b4d9-be586235b308",
        "media_types": [
          "application/vnd.oasis.stix+json; version=2.0"
        ],
        "query": "type:File",
        "title": "AllHashes"
      }
    ]
  }

Set up the Microsoft Sentinel TAXII Connector

Now that we have the API root URL and the collection ID we can configure the Threat intelligence - TAXII Connector in Microsoft Sentinel.

Paste your API root URL in the field marked API Root URL and the desired collection ID we got in step (2) under Collection ID.

Example:
Microsoft Sentinel TI Configuration

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

taxii-server-list-collections


Returns all the collections.

Base Command

taxii-server-list-collections

Input

There are no input arguments for this command.

Context Output

Path Type Description
TAXIIServer.Collection.id String The collection ID.
TAXIIServer.Collection.query String The collection query.
TAXIIServer.Collection.title String The collection title.
TAXIIServer.Collection.description String The collection description.

Command example


#### Context Example

```json
{
    "TAXIIServer": {
        "Collection": {
            "can_read": true,
            "can_write": false,
            "description": "",
            "id": "2eb7bfae-7739-5863-9b00-1681309c3d8c",
            "media_types": [
                "application/stix+json;version=2.1"
            ],
            "query": "",
            "title": "ALL"
        }
    }
}

Human Readable Output

Collections

id title query description
2eb7bfae-7739-5863-9b00-1681309c3d8c ALL    

taxii-server-info


Returns the TAXII server info, default URL, title, etc.

Base Command

taxii-server-info

Input

There are no input arguments for this command.

Context Output

Path Type Description
TAXIIServer.ServerInfo.title String The server title
TAXIIServer.ServerInfo.api_roots Unknown The server API roots URLs.
TAXIIServer.ServerInfo.default String The default URL.
TAXIIServer.ServerInfo.description String The server description

Command Example


#### Context Example

```json
{
    "TAXIIServer": {
        "ServerInfo": {
            "api_roots": [
                "https://foo.cooo.com/inc/threatintel/"
            ],
            "default": "https://foo.cooo.com/inc/threatintel/",
            "description": "This integration provides TAXII Services for system indicators (Outbound feed).",
            "title": "Cortex XSOAR TAXII2 Server"
        }
    }
}

Human Readable Output

In case the default URL is incorrect, you can override it by setting the “TAXII2 Service URL Address” field in the integration configuration

Server Info

api_roots default description title
https://foo.cooo.com/inc/threatintel/ https://foo.cooo.com/inc/threatintel/ This integration provides TAXII Services for system indicators (Outbound feed). Cortex XSOAR TAXII2 Server

Troubleshooting

429 Too Many Requests error

NGINX prevents concurrent builds of the same cache entry. If multiple requests for the same cache entry (matching the URL and parameters) arrive simultaneously, NGINX builds the entry for the first request and rejects the others with an HTTP 429 Too Many Requests rather than queuing them. Requests for different entries are still processed in parallel. If a request triggers a refresh of an existing cache entry, the previous data is served (HTTP 200) with no 429.

This is expected behavior. Retry the request after a short delay; once the initial build finishes populating the cache, retries are served from the cache (HTTP 200).

[Errno 98] Address in use error

Each instance uses three consecutive ports: the configured Listen Port, port + 1, and port + 2. NGINX listens on the configured port (public), the Python process listens on port + 1, and NGINX uses port + 2 internally for its fail-fast cache fetch tier. For example, if configured for port 9009, ports 9009, 9010, and 9011 must all be free. Ensure no other instance uses a Listen Port within 2 of another, or an [Errno 98] Address in use error will occur. When running without --network=host, ports + 1 and + 2 are not exposed to the machine.

Configuration parameters

  • longRunning — Long Running Instance
  • version — TAXII2 Server version (required)
  • credentials — Username
  • collections — Collection JSON (required)
  • fields_filter — Cortex XSOAR Extension fields
  • res_size — Response Size (required)
  • certificate — Certificate (Required for HTTPS)
  • key — Private Key (Required for HTTPS)
  • hsts_header — Add HSTS header
  • service_address — TAXII2 Service URL Address
  • longRunningPort — Listen Port (required)
  • nginx_global_directives — NGINX Global Directives
  • nginx_server_conf — NGINX Server Conf
  • cache_refresh_rate — Refresh Rate
  • cache_lock_timeout — Cache Lock Timeout (Deprecated)
  • cache_lock_age — Cache Lock Age (Deprecated)
  • cache_404_ttl — Cache 404 TTL
  • cache_default_ttl — Cache Default TTL
  • provide_as_indicator — STIX types for STIX indicator Domain Object
  • cache_duration_hours — Search-after cache duration hours (required)

Commands (2)

  • taxii-server-info

    Returns the TAXII server info, default URL, title, etc.

  • taxii-server-list-collections

    Returns all the collections.

category: Data Enrichment & Threat Intelligence
provider: Open Source
sectionorder:
- Connect
- Collect
commonfields:
  id: TAXII2 Server
  version: -1
configuration:
- defaultvalue: 'true'
  display: Long Running Instance
  name: longRunning
  type: 8
  hidden: true
  section: Connect
  advanced: true
  required: false
- display: TAXII2 Server version
  name: version
  defaultvalue: "2.1"
  options:
  - "2.0"
  - "2.1"
  required: true
  type: 15
  section: Connect
- display: Username
  additionalinfo: Credentials to use for the basic authentication.
  name: credentials
  type: 9
  section: Connect
  required: false
- additionalinfo: JSON string of indicator query collections. Dictionary of the collection name as the key and the query as the value.
  defaultvalue: |-
    {
       "Collection1": "type:SomeType and sourceBrands:\"Some Feed\"",
        "Collection2": {"query": "type:SomeType", "description": "IP from Cortex XSOAR Threat Intel" }
    }
  display: Collection JSON
  name: collections
  required: true
  type: 12
  section: Collect
- additionalinfo: Comma-separated fields to return in the extension. Leave empty for no extension fields, 'All' for all existing fields.
  display: Cortex XSOAR Extension fields
  name: fields_filter
  type: 0
  section: Collect
  advanced: true
  required: false
- additionalinfo: Maximum number of items to return.
  defaultvalue: '2000'
  display: Response Size
  name: res_size
  required: true
  type: 0
  section: Collect
- display: Certificate (Required for HTTPS)
  name: certificate
  type: 12
  section: Connect
  advanced: true
  required: false
- display: Private Key (Required for HTTPS)
  name: key
  type: 14
  section: Connect
  advanced: true
  required: false
- display: Add HSTS header
  additionalinfo: When set to true and certificate & private key are provided will add the HSTS header to all requests.
  name: hsts_header
  type: 8
  required: false
  defaultvalue: 'false'
  section: Connect
- additionalinfo: "Full URL address to set in the TAXII2 service response. If not set, the integration will try to auto-detect the URL."
  display: TAXII2 Service URL Address
  name: service_address
  type: 0
  section: Connect
  required: false
- additionalinfo: Will run the TAXII2 Server on this port from within Cortex XSOAR. Requires a unique port for each long-running integration instance. Do not use the same port for multiple instances. (For Cortex XSOAR 8 and Cortex XSIAM) If you do not enter a Listen Port, an unused port for TAXII2 Server will automatically be generated when the instance is saved. However, if using an engine, you must enter a Listen Port.
  display: Listen Port
  name: longRunningPort
  required: true
  type: 0
  section: Connect
- additionalinfo: "NGINX global directives to be passed on the command line using the -g option. Each directive should end with `;`. For example: `worker_processes 4; timer_resolution 100ms;`. Advanced configuration to be used only if instructed by Cortex XSOAR Support."
  display: NGINX Global Directives
  name: nginx_global_directives
  type: 0
  section: Connect
  advanced: true
  required: false
- display: NGINX Server Conf
  name: nginx_server_conf
  type: 12
  additionalinfo: "NGINX server configuration. To be used instead of the default NGINX_SERVER_CONF used in the integration code. Advanced configuration to be used only if instructed by Cortex XSOAR Support."
  section: Connect
- additionalinfo: How often to refresh the list (e.g., less than 1 minute, 5 minutes, 12 hours, 7 days, 3 months, 1 year). For performance reasons, we do not recommend setting this value at less than 1 minute.
  defaultvalue: 30 minutes
  display: Refresh Rate
  name: cache_refresh_rate
  type: 0
  section: Collect
  advanced: true
  required: false
- additionalinfo: 'The parameter is deprecated and no longer used. Cache locking was replaced by a two-tier fail-fast design that rejects excess concurrent cache-building requests instead of queuing them.'
  defaultvalue: 1h
  display: Cache Lock Timeout (Deprecated)
  name: cache_lock_timeout
  type: 0
  section: Connect
  advanced: true
  required: false
  hidden: true
- additionalinfo: 'The parameter is deprecated and no longer used. Cache locking was replaced by a two-tier fail-fast design that rejects excess concurrent cache-building requests instead of queuing them.'
  defaultvalue: 1h
  display: Cache Lock Age (Deprecated)
  name: cache_lock_age
  type: 0
  section: Connect
  advanced: true
  required: false
  hidden: true
- additionalinfo: The TTL for 404 responses in the cache.
  defaultvalue: 1m
  display: Cache 404 TTL
  name: cache_404_ttl
  type: 0
  section: Connect
  advanced: true
  required: false
- additionalinfo: The default TTL for responses in the cache.
  defaultvalue: 1m
  display: Cache Default TTL
  name: cache_default_ttl
  type: 0
  section: Connect
  advanced: true
  required: false
- display: STIX types for STIX indicator Domain Object
  name: provide_as_indicator
  type: 16
  options:
  - ipv4-addr
  - domain-name
  - ipv6-addr
  - user-account
  - email-addr
  - windows-registry-key
  - file
  - url
  - x509-certificate
  additionalinfo: The selected types will be outputted as a "STIX Indicator type" with a STIX pattern instead of an SCO. This may be necessary based on the tool you use to ingest data.
  section: Collect
  advanced: true
  required: false
- display: Search-after cache duration hours
  additionalinfo: Defines how many hours the search-after values is kept in cache so pagination can be faster.
  name: cache_duration_hours
  required: true
  type: 0
  section: Collect
  advanced: true
  defaultvalue: "48"
description: This integration provides TAXII2 Services for system indicators (Outbound feed).
display: TAXII2 Server
name: TAXII2 Server
script:
  commands:
  - description: Returns all the collections.
    name: taxii-server-list-collections
    outputs:
    - contextPath: TAXIIServer.Collection.id
      description: The collection ID.
      type: String
    - contextPath: TAXIIServer.Collection.query
      description: The collection query.
      type: String
    - contextPath: TAXIIServer.Collection.title
      description: The collection title.
      type: String
    - contextPath: TAXIIServer.Collection.description
      description: The collection description.
      type: String
  - description: Returns the TAXII server info, default URL, title, etc.
    name: taxii-server-info
    outputs:
    - contextPath: TAXIIServer.ServerInfo.title
      description: The server title.
      type: String
    - contextPath: TAXIIServer.ServerInfo.api_roots
      description: The server API roots URLs.
      type: Unknown
    - contextPath: TAXIIServer.ServerInfo.default
      description: The default URL.
      type: String
    - contextPath: TAXIIServer.ServerInfo.description
      description: The server description.
      type: String
  dockerimage: demisto/flask-nginx:1.0.0.11047721
  longRunning: true
  longRunningPort: true
  script: '-'
  subtype: python3
  type: python
tests:
- TAXII2 Server Performance Test
fromversion: 6.1.0
triggers:
- conditions:
  - name: engine
    operator: not_exists
  - name: isEngineGroup
    operator: not_exists
  effects:
  - name: longRunningPort
    action:
      hidden: true
      required: false