Devo Deprecated

Deprecated. Use the Devo v2 integration instead.

Analytics & SIEM · Devo

Details

IDDevo
ProviderDevo Technology
CategoryAnalytics & SIEM
From Version5.0.0
Docker Imagedemisto/python:2.7.18.24066
Supported ModulesAgentix XSIAM

README

Devo (Deprecated)

This integration is now deprecated. Please use the Devo v2 integration instead.


Overview


Use the Devo integration to query data in Devo.

This integration was integrated and tested with API v2 version of Devo.

 

Configure Devo on Cortex XSOAR


To use the Devo integration, a user with the administrator role is required.

You can access the API key and API secret in the Devo UI under Administration > Credentials.

  1. Navigate to Settings > Integrations > Servers & Services.
  2. Search for Devo.
  3. Click Add instance to create and configure a new integration instance.
    • Name: a textual name for the integration instance.
    • Server URL (e.g https://api-us.logtrust.com/)
    • API key
    • API secret
    • Trust any certificate (not secure)
    • Use system proxy settings
  4. Click Test to validate the URLs, token, and connection.

Commands


You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.

  1. Perform a query in Devo: devo-query

1. Perform a query in Devo


Perform a query. For more information about querying in Devo, see the Devo documentation.

Base Command
devo-query
Input
Argument Name Description Required
query A LINQ query to launch. The body must have a query or queryId parameter. Optional
queryId A query Id to launch. The body must have a query or queryId parameter. For information about obtaining the queryID, see the Additional Information section. Optional
from The start date as a UTC timestamp in the format: 2012-03-01T10:00:00Z. Required
to The end date as a UTC timestamp in the format: 2012-03-01T10:00:00Z. Default is the cur Optional
skip Skip the first "X" number of elements of the query. Optional
limit Limit the results of the query. The query will stop after returning the first X elements of the query or reaching its end. Optional
writeToContext Whether to write results to context or not Optional
 
Context Output
Path Description
Devo.Results The query results
 
Command Example
!devo-query from=2018-10-07T08:00:00Z to=2018-10-07T08:30:00Z limit=5 query="from demo.ecommerce.data select eventdate, referralUri, userAgent where method=\"GET\""
Context Example
{
    "Devo": {
        "Results": [
            {
                "eventcount": 1,
                "eventdate": "2018-10-07T08:00:00Z",
                "referralUri": "http://www.logtrust.com/oldlink?item_id=LOG-77\u0026port=161\u0026JSESSIONID=SD5SL4FF3ADFF5",
                "userAgent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1"
            },
            {
                "eventcount": 1,
                "eventdate": "2018-10-07T08:00:01Z",
                "referralUri": "http://www.bing.com/cart.do?action=purchase\u0026itemId=LOG-66\u0026product_id=L98-72BOK-SKD00\u0026JSESSIONID=SD1SL6FF8ADFF4",
                "userAgent": "Opera/9.20 (Windows NT 6.0; U; en)"
            },
            {
                "eventcount": 1,
                "eventdate": "2018-10-07T08:00:01Z",
                "referralUri": "http://www.logtrust.com/cart.do?action=purchase\u0026itemId=LOG-29\u0026product_id=99J-SALKS-ASKD0\u0026JSESSIONID=SD5SL7FF7ADFF1",
                "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko)"
            },
            {
                "eventcount": 1,
                "eventdate": "2018-10-07T08:00:01Z",
                "referralUri": "http://www.yahoo.com/product.screen?product_id=235-40LSZ-09823\u0026JSESSIONID=SD1SL4FF10ADFF7",
                "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko)"
            },
            {
                "eventcount": 1,
                "eventdate": "2018-10-07T08:00:02Z",
                "referralUri": "http://www.logtrust.com/cart.do?action=addtocart\u0026itemId=LOG-66\u0026product_id=235-40ER0-J3308\u0026JSESSIONID=SD9SL1FF5ADFF8",
                "userAgent": "Debian APT-HTTP/1.3 (1.0.1ubuntu2)"
            }
        ]
    }
}
Human Readable Output

image

 

Additional Information


Follow these steps to get the query ID

  1. Access your Devo environment.
  2. Navigate to the gear icon > Query Info > Get Id.

image

Troubleshooting


If you receive HTTP Error 401 (Unauthorized), the API key or API secret might be incorrect.

Configuration parameters

  • url — Server URL (e.g https://api-us.logtrust.com/) (required)
  • api_key — API key (required)
  • api_secret — API secret (required)
  • unsecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings

Commands (1)

  • devo-query

    Perform a query.

import demistomock as demisto  # noqa: F401
from CommonServerPython import *  # noqa: F401

''' IMPORTS '''
import hashlib
import hmac
import json
import socket
import struct
import time
from collections import Counter
from datetime import datetime

import requests

# disable insecure warnings
requests.packages.urllib3.disable_warnings()

if not demisto.params().get('proxy', False):
    del os.environ['HTTP_PROXY']
    del os.environ['HTTPS_PROXY']
    del os.environ['http_proxy']
    del os.environ['https_proxy']

''' GLOBAL VARS '''
API_KEY = demisto.params()['api_key']
API_SECRET = demisto.params()['api_secret']
VERIFY_SSL = not demisto.params().get('unsecure', False)
SERVER = demisto.params().get('url')[:-1] if demisto.params().get('url').endswith('/') \
    else demisto.params().get('url')

SERVER_URL = SERVER + '/search'

''' HELPER FUNCTIONS '''


def get_api_sign(timestamp, body):
    message = API_KEY
    if body:
        message += json.dumps(body)
    message += str(timestamp)
    sign = hmac.new(API_SECRET.encode("utf-8"), message.encode("utf-8"), hashlib.sha256)

    return sign.hexdigest()


def get_headers(body=''):
    headers = {
        'Content-Type': 'application/json'
    }
    timestamp = int(round(time.time() * 1000))

    headers['x-logtrust-apikey'] = API_KEY
    headers['x-logtrust-timestamp'] = str(timestamp)
    headers['x-logtrust-sign'] = get_api_sign(timestamp, body)

    return headers


def send_request(path, headers=get_headers(), method='get', body=None, params=None):
    body = body if body is not None else {}
    params = params if params is not None else {}
    url = '{}/{}'.format(SERVER_URL, path)

    res = requests.request(method, url, headers=headers, data=json.dumps(body), params=params, verify=VERIFY_SSL)

    if res.status_code < 200 or res.status_code >= 300:
        raise Exception('Got status code {} with url {} with body {} with headers {}'.format(
            str(res.status_code), url, res.content, str(res.headers)))

    try:
        return res.json()
    except Exception:
        return res.content


def get_timestamp_in_seconds(timestamp):
    dt = datetime.strptime(timestamp, '%Y-%m-%dT%H:%M:%SZ').timetuple()
    return int(time.mktime(dt))


''' FUNCTIONS '''


def query_command():
    query = demisto.args().get('query')
    query_id = demisto.args().get('queryId')
    timestamp_from = demisto.args()['from']
    timestamp_to = demisto.args().get('to', datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ'))
    skip = demisto.args().get('skip')
    limit = demisto.args().get('limit')
    write_context = demisto.args()['writeToContext'].lower()

    if (not query and not query_id) or (query and query_id):
        raise ValueError('Query or query id must be specified')

    res = do_query(query, query_id, timestamp_from, timestamp_to, skip, limit)

    entry = {
        'Type': entryTypes['note'],
        'Contents': res,
        'ContentsFormat': formats['json'],
        'ReadableContentsFormat': formats['markdown']
    }

    if 'object' not in res or not res['object'] or len(res['object']) == 0:
        entry['HumanReadable'] = 'No results found'
        return entry

    results = []
    for result in res['object']:
        if 'eventdate' in result:
            result['eventdate'] = datetime.fromtimestamp(result['eventdate'] / 1000).strftime('%Y-%m-%dT%H:%M:%SZ')
        # temporary
        if 'srcIp' in result:
            try:
                decimal_ip = int(result['srcIp'])
                # Convert decimal IP to IP
                result['srcIp'] = socket.inet_ntoa(struct.pack('!L', int(decimal_ip))) if decimal_ip else ''
            except Exception:
                pass
        results.append(result)

    # remove duplicates and add a count of them instead
    unique_results = [dict(t + (('eventcount', c),)) for t, c in Counter(tuple(r.items()) for r in results).items()]

    headers = res['object'][0].keys()
    if 'eventdate' in headers:
        # set event date as first column
        headers.remove('eventdate')
        headers = ['eventdate'] + headers
    if 'eventcount' not in headers:
        headers.append('eventcount')

    md = tableToMarkdown('Devo query results', unique_results, headers, removeNull=True)

    entry['HumanReadable'] = md
    if write_context == 'true':
        entry['EntryContext'] = {
            'Devo.Results': createContext(unique_results, removeNull=True)
        }

    return entry


def do_query(query, query_id, timestamp_from, timestamp_to, skip, limit):
    query_body = {
        'from': get_timestamp_in_seconds(timestamp_from)
    }

    if query:
        query_body['query'] = query
    elif query_id:
        query_body['queryId'] = query_id

    if timestamp_to:
        query_body['to'] = get_timestamp_in_seconds(timestamp_to)
    if skip:
        query_body['skip'] = skip
    if limit:
        query_body['limit'] = limit

    path = 'query'
    headers = get_headers(query_body)

    return send_request(path, headers, 'post', query_body)


''' EXECUTION CODE '''
try:
    if demisto.command() == 'test-module':
        path = 'system/ping'
        send_request(path)
        demisto.results('ok')
    elif demisto.command() == 'devo-query':
        demisto.results(query_command())
except Exception as e:
    LOG(e)
    LOG.print_log(False)
    return_error(e.message)