DNSDB_v2
Farsight Security DNSDB DNSDB is a Passive DNS (pDNS) historical database that provides a unique, fact-based, multifaceted view of the configuration of the global Internet infrastructure DNSDB leverages the richness of Farsight’s Security Information Exchange (SIE) data-sharing platform and is engineered and operated by leading DNS experts.
Data Enrichment & Threat Intelligence · Farsight DNSDB
Details
| ID | DNSDB_v2 |
|---|---|
| Provider | DomainTools |
| Category | Data Enrichment & Threat Intelligence |
| From Version | 5.0.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | Agentix XSIAM |
README
Farsight Security DNSDB
Farsight Security DNSDB® is the world’s largest DNS intelligence database that
provides a unique, fact-based, multifaceted view of the configuration of the
global Internet infrastructure. DNSDB leverages the richness of Farsight’s
Security Information Exchange (SIE) data-sharing platform and is engineered and
operated by leading DNS experts. Farsight collects Passive DNS data from its
global sensor array. It then filters and verifies the DNS transactions before
inserting them into the DNSDB, along with ICANN-sponsored zone file access
download data. The end result is the highest-quality and most comprehensive DNS
intelligence data service of its kind - with more than 100 billion DNS records
since 2010.
This integration uses Farsight Security’s DNSDB solution to interactively
lookup rich, historical DNS information – either as playbook tasks or through
API calls in the War Room – to access rdata and rrset records. It was
integrated and tested with version 2 of the DNSDB API.
Configure DNSDB v2 in Cortex
| Parameter | Description | Required |
|---|---|---|
| apikey | API Key | True |
| url | DNSDB Service URL | False |
| insecure | Trust any certificate (not secure) | False |
| useproxy | Use system proxy settings | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
dnsdb-flex
DNSDB flex search
Base Command
dnsdb-flex
Input
| Argument Name | Description | Required |
|---|---|---|
| method | query value type | Required |
| key | search over rrnames or rdata | Required |
| value | query regex or glob | Required |
| rrtype | query rrtype | Optional |
| limit | Limit the number of returned records | Optional |
| time_first_before | Filter results for entries seen for first time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_before | Filter results for entries seen for last time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_first_after | Filter results for entries seen for first time after (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_after | Filter results for entries seen for last time after (ISO or UNIX timestamp, relative if negative) | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DNSDB.Record.RRName | string | The owner name of the resource record in DNS presentation format. |
| DNSDB.Record.RRType | string | The resource record type of the resource record, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal RRtype number. |
| DNSDB.Record.RData | string | The record data value. The Rdata value is converted to the standard presentation format based on the rrtype value. If the encoder lacks a type-specific presentation format for the resource record’s type, then the RFC 3597 generic Rdata encoding will be used. |
| DNSDB.Record.Count | number | The number of times the resource record was observed via passive DNS replication. |
| DNSDB.Record.TimeFirst | date | The first time that the resource record was observed. |
| DNSDB.Record.TimeLast | date | The most recent time that the resource record was observed. |
| DNSDB.Record.FromZoneFile | bool | False if the resource record was observed via passive DNS replication, True if by zone file import. |
Command Example
!dnsdb-flex method=regex key=rrnames value=farsightsecurity limit=5
Context Example
{
"DNSDB": {
"Record": [
{
"FromZoneFile": false,
"RRName": "farsightsecurity.yahoo.com.au",
"RRType": "CNAME"
},
{
"FromZoneFile": false,
"RRName": "farsightsecurity-2432183.starbucks.com.cn",
"RRType": "A"
},
{
"FromZoneFile": false,
"RRName": "farsightsecurity.com.cn",
"RRType": "NS"
},
{
"FromZoneFile": false,
"RRName": "farsightsecurity.com.cn",
"RRType": "SOA"
},
{
"FromZoneFile": false,
"RRName": "farsightsecurity.com.cn",
"RRType": "CNAME"
},
{
"FromZoneFile": false,
"RRName": "www.farsightsecurity.com.cn",
"RRType": "CNAME"
},
{
"FromZoneFile": false,
"RRName": "farsightsecurity.damai.cn",
"RRType": "CNAME"
}
]
}
}
Human Readable Output
Farsight DNSDB Flex Search
RRName RRType Count TimeFirst TimeLast farsightsecurity.yahoo.com.au CNAME farsightsecurity-2432183.starbucks.com.cn A farsightsecurity.com.cn NS farsightsecurity.com.cn SOA farsightsecurity.com.cn CNAME www.farsightsecurity.com.cn CNAME farsightsecurity.damai.cn CNAME
dnsdb-rdata
Lookup RData records
Base Command
dnsdb-rdata
Input
| Argument Name | Description | Required |
|---|---|---|
| type | Query type | Required |
| value | Query value | Required |
| rrtype | query rrtype | Optional |
| limit | Limit the number of returned records | Optional |
| time_first_before | Filter results for entries seen for first time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_before | Filter results for entries seen for last time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_first_after | Filter results for entries seen for first time after (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_after | Filter results for entries seen for last time after (ISO or UNIX timestamp, relative if negative) | Optional |
| aggr | Aggregate identical RRsets | Optional |
| offset | How many rows to offset in the results. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DNSDB.Record.RRName | string | The owner name of the resource record in DNS presentation format. |
| DNSDB.Record.RRType | string | The resource record type of the resource record, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal RRtype number. |
| DNSDB.Record.RData | string | The record data value. The Rdata value is converted to the standard presentation format based on the rrtype value. If the encoder lacks a type-specific presentation format for the resource record’s type, then the RFC 3597 generic Rdata encoding will be used. |
| DNSDB.Record.Count | number | The number of times the resource record was observed via passive DNS replication. |
| DNSDB.Record.TimeFirst | date | The first time that the resource record was observed. |
| DNSDB.Record.TimeLast | date | The most recent time that the resource record was observed. |
| DNSDB.Record.FromZoneFile | bool | False if the resource record was observed via passive DNS replication, True if by zone file import. |
Command Example
!dnsdb-rdata type=name value=www.farsightsecurity.com limit=5
Context Example
{
"DNSDB": {
"Record": [
{
"Count": 606,
"FromZoneFile": false,
"RData": [
"www.farsightsecurity.com."
],
"RRName": "scout.dnsdb.info",
"RRType": "CNAME",
"TimeFirst": "2020-03-27T18:37:24Z",
"TimeLast": "2020-10-21T18:11:47Z"
},
{
"Count": 121,
"FromZoneFile": false,
"RData": [
"www.farsightsecurity.com."
],
"RRName": "scout-beta.dnsdb.info",
"RRType": "CNAME",
"TimeFirst": "2020-08-20T22:52:29Z",
"TimeLast": "2020-10-20T17:54:58Z"
},
{
"Count": 546,
"FromZoneFile": false,
"RData": [
"www.farsightsecurity.com."
],
"RRName": "81.64-26.140.160.66.in-addr.arpa",
"RRType": "PTR",
"TimeFirst": "2013-12-10T01:20:08Z",
"TimeLast": "2020-10-10T15:47:19Z"
}
]
}
}
Human Readable Output
Farsight DNSDB Lookup
RRName RRType RData Count TimeFirst TimeLast FromZoneFile scout.dnsdb.info CNAME www.farsightsecurity.com. 606 2020-03-27T18:37:24Z 2020-10-21T18:11:47Z False scout-beta.dnsdb.info CNAME www.farsightsecurity.com. 121 2020-08-20T22:52:29Z 2020-10-20T17:54:58Z False 81.64-26.140.160.66.in-addr.arpa PTR www.farsightsecurity.com. 546 2013-12-10T01:20:08Z 2020-10-10T15:47:19Z False
dnsdb-summarize-rdata
Summarize RData records
Base Command
dnsdb-summarize-rdata
Input
| Argument Name | Description | Required |
|---|---|---|
| type | Query type | Required |
| value | Query value | Required |
| rrtype | query rrtype | Optional |
| limit | Limit the number of returned records | Optional |
| time_first_before | Filter results for entries seen for first time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_before | Filter results for entries seen for last time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_first_after | Filter results for entries seen for first time after (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_after | Filter results for entries seen for last time after (ISO or UNIX timestamp, relative if negative) | Optional |
| aggr | Aggregate identical RRsets | Optional |
| max_count | Stop when the summary count is reached | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DNSDB.Summary.Count | number | The number of times the resource record was observed via passive DNS replication. |
| DNSDB.Summary.NumResults | number | The number of results (resource records) that would be returned from a Lookup. |
| DNSDB.Summary.TimeFirst | date | The first time that the resource record was observed via passive DNS replication. |
| DNSDB.Summary.TimeLast | date | The most recent time that the resource record was observed via passive DNS replication. |
| DNSDB.Summary.ZoneTimeFirst | date | The first time that the resource record was observed in a zone file. |
| DNSDB.Summary.ZoneTimeLast | date | The most recent time that the resource record was observed in a zone file. |
Command Example
!dnsdb-summarize-rdata type=name value=www.farsightsecurity.com limit=5
Context Example
{
"DNSDB": {
"Summary": {
"Count": 1273,
"FromZoneFile": false,
"NumResults": 3,
"TimeFirst": "2013-12-10T01:20:08Z",
"TimeLast": "2020-10-21T18:11:47Z"
}
}
}
Human Readable Output
Farsight DNSDB Summarize
Count NumResults TimeFirst TimeLast 1273 3 2013-12-10T01:20:08Z 2020-10-21T18:11:47Z
dnsdb-rrset
Lookup RRset records
Base Command
dnsdb-rrset
Input
| Argument Name | Description | Required |
|---|---|---|
| owner_name | Owner name to query | Required |
| rrtype | rrtype value to query | Optional |
| bailiwick | Bailiwick value to query | Optional |
| limit | Limit the number of returned records | Optional |
| time_first_before | Filter results for entries seen for first time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_first_after | Filter results for entries seen for first time after (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_before | Filter results for entries seen for last time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_after | Filter results for entries seen for last time after (ISO or UNIX timestamp, relative if negative) | Optional |
| aggr | Aggregate identical RRsets | Optional |
| offset | How many rows to offset in the results. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DNSDB.Record.RRName | string | The owner name of the RRset in DNS presentation format. |
| DNSDB.Record.RRType | string | The resource record type of the RRset, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal RRtype number. |
| DNSDB.Record.Bailiwick | string | The closest enclosing zone delegated to a nameserver which served the RRset, or the name of the zone containing the RRset if FromZoneFile is True. |
| DNSDB.Record.RData | string | An array of one or more Rdata values. The Rdata values are converted to the standard presentation format based on the rrtype value. If the encoder lacks a type-specific presentation format for the RRset’s rrtype, then the RFC 3597 generic Rdata encoding will be used. |
| DNSDB.Record.Count | number | The number of times the RRset was observed via passive DNS replication. |
| DNSDB.Record.TimeFirst | date | The first time that the RRset was observed. |
| DNSDB.Record.TimeLast | date | The most recent time that the RRset was observed. |
| DNSDB.Record.FromZoneFile | bool | False if the RRset was observed via passive DNS replication, True if by zone file import. |
Command Example
!dnsdb-rrset owner_name=*.farsightsecurity.com type=NS limit=5
Context Example
{
"DNSDB": {
"Record": [
{
"Bailiwick": "com",
"Count": 19,
"FromZoneFile": true,
"RData": [
"ns.lah1.vix.com.",
"ns1.isc-sns.net.",
"ns2.isc-sns.com.",
"ns3.isc-sns.info."
],
"RRName": "farsightsecurity.com",
"RRType": "NS",
"TimeFirst": "2013-06-30T16:21:41Z",
"TimeLast": "2013-07-18T16:22:47Z"
},
{
"Bailiwick": "com",
"Count": 157,
"FromZoneFile": true,
"RData": [
"ns.sjc1.vix.com.",
"ns.sql1.vix.com."
],
"RRName": "farsightsecurity.com",
"RRType": "NS",
"TimeFirst": "2013-01-24T17:18:05Z",
"TimeLast": "2013-06-29T16:19:01Z"
},
{
"Bailiwick": "com",
"Count": 1890,
"FromZoneFile": true,
"RData": [
"ns5.dnsmadeeasy.com.",
"ns6.dnsmadeeasy.com.",
"ns7.dnsmadeeasy.com."
],
"RRName": "farsightsecurity.com",
"RRType": "NS",
"TimeFirst": "2013-07-19T16:22:00Z",
"TimeLast": "2020-07-24T16:02:05Z"
},
{
"Bailiwick": "farsightsecurity.com",
"Count": 6350,
"FromZoneFile": false,
"RData": [
"66.160.140.81"
],
"RRName": "farsightsecurity.com",
"RRType": "A",
"TimeFirst": "2013-09-25T15:37:03Z",
"TimeLast": "2015-04-01T06:17:25Z"
},
{
"Bailiwick": "farsightsecurity.com",
"Count": 36770,
"FromZoneFile": false,
"RData": [
"104.244.13.104"
],
"RRName": "farsightsecurity.com",
"RRType": "A",
"TimeFirst": "2015-04-01T14:17:52Z",
"TimeLast": "2018-09-27T00:29:43Z"
}
]
}
}
Human Readable Output
Farsight DNSDB Lookup
RRName RRType Bailiwick RData Count TimeFirst TimeLast FromZoneFile farsightsecurity.com NS com ns.lah1.vix.com.
ns1.isc-sns.net.
ns2.isc-sns.com.
ns3.isc-sns.info.19 2013-06-30T16:21:41Z 2013-07-18T16:22:47Z True farsightsecurity.com NS com ns.sjc1.vix.com.
ns.sql1.vix.com.157 2013-01-24T17:18:05Z 2013-06-29T16:19:01Z True farsightsecurity.com NS com ns5.dnsmadeeasy.com.
ns6.dnsmadeeasy.com.
ns7.dnsmadeeasy.com.1890 2013-07-19T16:22:00Z 2020-07-24T16:02:05Z True farsightsecurity.com A farsightsecurity.com 66.160.140.81 6350 2013-09-25T15:37:03Z 2015-04-01T06:17:25Z False farsightsecurity.com A farsightsecurity.com 104.244.13.104 36770 2015-04-01T14:17:52Z 2018-09-27T00:29:43Z False
dnsdb-summarize-rrset
Lookup RRset records
Base Command
dnsdb-summarize-rrset
Input
| Argument Name | Description | Required |
|---|---|---|
| owner_name | Owner name to query | Required |
| rrtype | rrtype value to query | Optional |
| bailiwick | Bailiwick value to query | Optional |
| limit | Limit the number of returned records | Optional |
| time_first_before | Filter results for entries seen for first time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_first_after | Filter results for entries seen for first time after (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_before | Filter results for entries seen for last time before (ISO or UNIX timestamp, relative if negative) | Optional |
| time_last_after | Filter results for entries seen for last time after (ISO or UNIX timestamp, relative if negative) | Optional |
| aggr | Aggregate identical RRsets | Optional |
| max_count | Stop when the summary count is reached | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DNSDB.Summary.Count | number | The number of times the resource record was observed via passive DNS replication. |
| DNSDB.Summary.NumResults | number | The number of results (resource records) that would be returned from a Lookup. |
| DNSDB.Summary.TimeFirst | date | The first time that the resource record was observed via passive DNS replication. |
| DNSDB.Summary.TimeLast | date | The most recent time that the resource record was observed via passive DNS replication. |
| DNSDB.Summary.ZoneTimeFirst | date | The first time that the resource record was observed in a zone file. |
| DNSDB.Summary.ZoneTimeLast | date | The most recent time that the resource record was observed in a zone file. |
Command Example
!dnsdb-summarize-rrset owner_name=*.farsightsecurity.com type=NS limit=5
Context Example
{
"DNSDB": {
"Summary": {
"Count": 45186,
"FromZoneFile": true,
"NumResults": 5,
"TimeFirst": "2013-01-24T17:18:05Z",
"TimeLast": "2020-07-24T16:02:05Z"
}
}
}
Human Readable Output
Farsight DNSDB Summarize
Count NumResults TimeFirst TimeLast ZoneTimeFirst ZoneTimeLast 45186 5 2013-09-25T15:37:03Z 2018-09-27T00:29:43Z 2013-01-24T17:18:05Z 2020-07-24T16:02:05Z
dnsdb-rate-limit
Retrieve service limits
Base Command
dnsdb-rate-limit
Input
There are no input arguments for this command.
Context Output
| Path | Type | Description |
| — | — | — |
| DNSDB.Rate.Limit | number | The maximum number of API lookups that may be performed. This is the initial quota. |
| DNSDB.Rate.Unlimited | bool | True if there is no maximum number of API lookups that may be performed. |
| DNSDB.Rate.Remaining | number | For time-based quotas: the remaining number of API lookups that may be performed until the reset time.
For block-based quotas: the remaining number of API lookups in the block quota.
|
| DNSDB.Rate.Reset | date | For time-based quotas: When the quota limit will be reset. Usually this is at 00:00 (midnight) UTC.
|
| DNSDB.Rate.NeverResets | bool | True for block-based quotas that do not reset. |
| DNSDB.Rate.Expires | date | Only present for block-based quota. When the quota will expire. |
| DNSDB.Rate.ResultsMax | number | The maximum number of results that can be returned by these lookup methods. This overrides a “limit” query parameter if provided. For example, if “?limit=20000” is appended to the URL path but results_max=1000 then only up to 1000 results will be returned.
|
| DNSDB.Rate.OffsetMax | number | The maximum value that the offset query parameter can be. If it is higher then an HTTP 416 “Requested Range Not Satisfiable” response code will be returned with message “Error: offset value greater than maximum allowed.”
|
| DNSDB.Rate.OffsetNotAllowed | number | True if the offset parameter is not allowed for this API key, and similar 416 error will be generated. |
| DNSDB.Rate.BurstSize | number | The maximum number of API lookups that may be performed within this burst_window number of seconds. |
| DNSDB.Rate.BurstWindow | number | The number of seconds over which a burst of queries is measured. |
Command Example
#### Context Example
```json
{
"DNSDB": {
"Rate": {
"Unlimited": true
}
}
}
Human Readable Output
Farsight DNSDB Service Limits
Unlimited true
Configuration parameters
apikey— API Key (required)url— DNSDB Service URLinsecure— Trust any certificate (not secure)useproxy— Use system proxy settings
Commands (6)
-
dnsdb-flexDNSDB flex search.
-
dnsdb-rate-limitRetrieve service limits.
-
dnsdb-rdataLookup RData records.
-
dnsdb-rrsetLookup RRset records.
-
dnsdb-summarize-rdataSummarize RData records.
-
dnsdb-summarize-rrsetLookup RRset records.
import inspect import json import locale import re import urllib import urllib.parse from collections.abc import Iterable, Iterator from datetime import datetime import urllib3 from CommonServerPython import * # noqa: E402 lgtm [py/polluting-import] # Disable insecure warnings urllib3.disable_warnings() INTEGRATION_NAME = "Farsight DNSDB" INTEGRATION_COMMAND_NAME = "dnsdb" INTEGRATION_CONTEXT_NAME = "DNSDB" RECORD_SUBCONTEXT_NAME = "Record" SUMMARY_SUBCONTEXT_NAME = "Summary" RATE_SUBCONTEXT_NAME = "Rate" # CONSTANTS DEFAULT_DNSDB_SERVER = "https://api.dnsdb.info" TIMEOUT = 60 SWCLIENT = "demisto" VERSION = "v2.1.2" PATH_PREFIX = "dnsdb/v2" IDN_REGEX = re.compile(r"(?:^|(?<=[\s=.:@]))xn--[a-z0-9\-]+\.") FALSE_REGEX = re.compile(r"^(?i:f(alse)?)$") COND_BEGIN = "begin" COND_ONGOING = "ongoing" COND_SUCCEEDED = "succeeded" COND_LIMITED = "limited" COND_FAILED = "failed" locale.setlocale(locale.LC_ALL, "") """ HELPER FUNCTIONS """ class QueryError(Exception): pass class timeval(int): pass class Client(BaseClient): def __init__(self, base_url: str, apikey: str, verify=None, proxy=None): BaseClient.__init__( self, base_url, verify=verify, headers={ "Accept": "application/x-ndjson", "X-Api-Key": apikey, }, proxy=proxy, ok_codes=(200,), ) self.apikey = apikey @staticmethod def base_params() -> dict: return { "swclient": SWCLIENT, "version": VERSION, } def rate_limit(self) -> dict: params = self.base_params() url_suffix = "dnsdb/v2/rate_limit" return self._http_request("GET", url_suffix=url_suffix, params=params) def lookup_rrset( self, owner_name: str, rrtype: str = None, bailiwick: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, ) -> Iterator[dict]: return self._query_rrset( "lookup", owner_name=owner_name, rrtype=rrtype, bailiwick=bailiwick, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, ) def summarize_rrset( self, owner_name: str, rrtype: str = None, bailiwick: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, max_count: int = None, ) -> dict: try: return next( self._query_rrset( "summarize", owner_name=owner_name, rrtype=rrtype, bailiwick=bailiwick, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, max_count=max_count, ) ) except StopIteration: raise QueryError("no data") def _query_rrset( self, mode: str, owner_name: str, rrtype: str = None, bailiwick: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, max_count: int = None, ) -> Iterator[dict]: owner_name = quote(to_ascii(owner_name)) if bailiwick: if not rrtype: rrtype = "ANY" bailiwick = quote(to_ascii(bailiwick)) path = f"{PATH_PREFIX}/{mode}/rrset/name/{owner_name}/{rrtype}/{bailiwick}" elif rrtype: path = f"{PATH_PREFIX}/{mode}/rrset/name/{owner_name}/{rrtype}" else: path = f"{PATH_PREFIX}/{mode}/rrset/name/{owner_name}" return self._query( path, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, max_count=max_count, ) def lookup_rdata_name( self, value: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, ) -> Iterator[dict]: return self._query_rdata_name( "lookup", name=value, rrtype=rrtype, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, ) def summarize_rdata_name( self, value: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, max_count: int = None, ) -> dict: try: return next( self._query_rdata_name( "summarize", name=value, rrtype=rrtype, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, max_count=max_count, ) ) except StopIteration: raise QueryError("no data") def _query_rdata_name( self, mode: str, name: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, max_count: int = None, ) -> Iterator[dict]: rdata_name = quote(to_ascii(name)) if rrtype: path = f"{PATH_PREFIX}/{mode}/rdata/name/{rdata_name}/{rrtype}" else: path = f"{PATH_PREFIX}/{mode}/rdata/name/{rdata_name}" return self._query( path, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, max_count=max_count, ) def lookup_rdata_ip( self, value: str, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, ) -> Iterator[dict]: return self._query_rdata_ip( "lookup", ip=value, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, ) def summarize_rdata_ip( self, value: str, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, max_count: int = None, ) -> dict: try: return next( self._query_rdata_ip( "summarize", ip=value, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, max_count=max_count, ) ) except StopIteration: raise QueryError("no data") def _query_rdata_ip( self, mode: str, ip: str, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, max_count: int = None, ) -> Iterator[dict]: ip = ip.replace("/", ",") path = f"{PATH_PREFIX}/{mode}/rdata/ip/{ip}" return self._query( path, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, max_count=max_count, ) def lookup_rdata_raw( self, value: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, ) -> Iterator[dict]: return self._query_rdata_raw( "lookup", raw=value, rrtype=rrtype, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, ) def summarize_rdata_raw( self, value: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, max_count: int = None, ) -> dict: try: return next( self._query_rdata_raw( "summarize", raw=value, rrtype=rrtype, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, max_count=max_count, ) ) except StopIteration: raise QueryError("no data") def _query_rdata_raw( self, mode: str, raw: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, max_count: int = None, ) -> Iterator[dict]: if rrtype: path = f"{PATH_PREFIX}/{mode}/rdata/raw/{quote(raw)}/{rrtype}" else: path = f"{PATH_PREFIX}/{mode}/rdata/raw/{quote(raw)}" return self._query( path, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, max_count=max_count, ) def flex( self, method: str, key: str, value: str, rrtype: str = None, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, ): path = f"{PATH_PREFIX}/{method}/{key}/{quote(value)}" if rrtype: path += f"/{rrtype}" return self._query( path, limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, ) def _query( self, path: str, limit: int = None, time_first_before: timeval = None, time_first_after: timeval = None, time_last_before: timeval = None, time_last_after: timeval = None, aggr: bool = None, offset: int = None, max_count: int = None, ) -> Iterator[dict]: params = self.base_params() params.update( assign_params( limit=limit, time_first_before=time_first_before, time_first_after=time_first_after, time_last_before=time_last_before, time_last_after=time_last_after, aggr=aggr, offset=offset, max_count=max_count, ) ) res = self._http_request( "GET", path, params=params, stream=True, resp_type="response", timeout=TIMEOUT, ) return _handle_saf(res.iter_lines(decode_unicode=True)) def _handle_saf(i: Iterable[str]): for line in i: if not line: continue try: saf_msg = json.loads(line) except json.JSONDecodeError as e: raise DemistoException(f"saf protocol error: could not decode json: {line}") from e cond = saf_msg.get("cond") obj = saf_msg.get("obj") msg = saf_msg.get("msg") if cond == COND_BEGIN: continue elif cond == COND_SUCCEEDED: return if obj: yield obj if cond == COND_ONGOING or not cond: continue elif cond == COND_LIMITED: return elif cond == COND_FAILED: raise QueryError(f"saf query failed: {msg}") else: raise QueryError(f"saf protocol error: invalid cond: {cond}") raise QueryError("saf query truncated") def quote(path: str) -> str: return urllib.parse.quote(path, safe="") @logger def _run_query(f, args): sig = inspect.signature(f) kwargs = {} # type: Dict[str, Any] for name, p in sig.parameters.items(): if name in args: if p.annotation != p.empty: if p.annotation is bool: if FALSE_REGEX.match(args[name]): kwargs[name] = False else: kwargs[name] = True elif p.annotation == timeval: try: kwargs[name] = int(args[name]) except ValueError: kwargs[name] = date_to_timestamp(args[name]) else: kwargs[name] = p.annotation(args[name]) else: kwargs[name] = args[name] elif p.kind == p.POSITIONAL_ONLY: raise Exception(f"Missing argument: {name}") return f(**kwargs) def to_unicode(domain: str) -> str: try: return domain.encode("utf8").decode("idna") except UnicodeError: return domain def to_ascii(domain: str) -> str: try: return domain.encode("idna").decode("utf8") except UnicodeError: return domain def format_name_for_context(domain: str) -> str: return domain.rstrip(".") def format_name_for_markdown(domain: str) -> str: return to_unicode(domain.rstrip(".")) def parse_rdata(rdata: str | list[str]): if isinstance(rdata, list): return [parse_rdata(entry) for entry in rdata] # pragma: no cover def f(m): return to_unicode(m.group(0)) return str(IDN_REGEX.sub(f, rdata)) def format_rdata_for_markdown(rdata: str | list[str]): rdata = parse_rdata(rdata) if isinstance(rdata, str): return rdata return "<br>".join(rdata) def parse_rate_limit_int(i): try: return int(i) except ValueError: return i def parse_unix_time(ts) -> str: try: return datetime.utcfromtimestamp(ts).strftime("%Y-%m-%dT%H:%M:%SZ") except TypeError: return ts def nop(x): return x @logger def build_result_context(results: dict) -> dict: ctx = {} for ckey, rkey, f in ( ("RRName", "rrname", format_name_for_context), ("RRType", "rrtype", str), ("Bailiwick", "bailiwick", format_name_for_context), ("RData", "rdata", nop), ("RawRData", "raw_rdata", nop), ("Count", "count", int), ("NumResults", "num_results", int), ("TimeFirst", "time_first", parse_unix_time), ("TimeLast", "time_last", parse_unix_time), ("TimeFirst", "zone_time_first", parse_unix_time), ("TimeLast", "zone_time_last", parse_unix_time), ): if rkey in results: ctx[ckey] = f(results[rkey]) # type: ignore[operator] if "zone_time_first" in results or "time_first" in results: ctx["FromZoneFile"] = "zone_time_first" in results return ctx @logger def build_rate_limits_context(results: dict) -> dict: """Formatting results from Rate Limit API to Demisto Context""" rate = results.get("rate") if rate is None: raise ValueError("Missing rate key") ctx = {} if rate["limit"] == "unlimited": return {"Unlimited": True} for ckey, rkey, f in ( ("Limit", "limit", parse_rate_limit_int), ("Remaining", "remaining", parse_rate_limit_int), ("Expires", "expires", parse_unix_time), ("ResultsMax", "results_max", parse_rate_limit_int), ("BurstSize", "burst_size", parse_rate_limit_int), ("BurstWindow", "burst_window", parse_rate_limit_int), ): if rkey in rate: ctx[ckey] = f(rate[rkey]) if "reset" in rate: if rate["reset"] == "n/a": ctx["NeverResets"] = True else: ctx["Reset"] = parse_unix_time(rate["reset"]) if "offset_max" in rate: if rate["offset_max"] == "n/a": ctx["OffsetNotAllowed"] = True else: ctx["OffsetMax"] = parse_rate_limit_int(rate["offset_max"]) return ctx @logger def lookup_to_markdown( results: list[dict], title: str = "Farsight DNSDB Lookup", want_bailiwick=True, header_filter=None, ) -> str: # TODO this should be more specific, include arguments? out = [] keys = [ ("RRName", "rrname", format_name_for_context), ("RRType", "rrtype", str), ("Bailiwick", "bailiwick", format_name_for_context), ("RData", "rdata", format_rdata_for_markdown), ("Count", "count", str), ] if not want_bailiwick: keys = list(filter(lambda r: r[1] != "bailiwick", keys)) headers = [k[0] for k in keys] + ["TimeFirst", "TimeLast", "FromZoneFile"] if header_filter: headers = list(filter(header_filter, headers)) for result in results: row = {} # type: Dict[str, Any] for ckey, rkey, f in keys: if rkey in result: row[ckey] = f(result[rkey]) # type: ignore[operator] if "time_first" in result: row["TimeFirst"] = parse_unix_time(result["time_first"]) elif "zone_time_first" in result: row["TimeFirst"] = parse_unix_time(result["zone_time_first"]) if "time_last" in result: row["TimeLast"] = parse_unix_time(result["time_last"]) elif "zone_time_last" in result: row["TimeLast"] = parse_unix_time(result["zone_time_last"]) row["FromZoneFile"] = str("zone_time_first" in result) out.append(row) return tableToMarkdown(title, out, headers=headers) @logger def summarize_to_markdown(summary: dict) -> str: headers = [] out = {} # type: Dict[str, Any] for ckey, rkey, f in ( ("Count", "count", int), ("NumResults", "num_results", int), ("TimeFirst", "time_first", parse_unix_time), ("TimeLast", "time_last", parse_unix_time), ("ZoneTimeFirst", "zone_time_first", parse_unix_time), ("ZoneTimeLast", "zone_time_last", parse_unix_time), ): if rkey in summary: headers.append(ckey) out[ckey] = f(summary[rkey]) # type: ignore[operator] return tableToMarkdown("Farsight DNSDB Summarize", out, headers=headers) @logger def rate_limit_to_markdown(results: dict) -> str: rate = results.get("rate") if rate is None: return "### Error" out = {} # type: Dict[str, Any] headers = [] if rate["limit"] != "unlimited": for ckey, rkey, f in ( ("Limit", "limit", parse_rate_limit_int), ("Remaining", "remaining", parse_rate_limit_int), ("Reset", "reset", parse_unix_time), ("Expires", "expires", parse_unix_time), ("ResultsMax", "results_max", parse_rate_limit_int), ("OffsetMax", "offset_max", parse_rate_limit_int), ("BurstSize", "burst_size", parse_rate_limit_int), ("BurstWindow", "burst_window", parse_rate_limit_int), ): if rkey in rate: headers.append(ckey) if rkey == "reset": if rate[rkey] == "n/a": NEVER_RESETS = "NeverResets" out[NEVER_RESETS] = True headers.append(NEVER_RESETS) else: out[f"{ckey}"] = f(rate[rkey]) elif rkey == "offset_max": if rate[rkey] == "n/a": OFFSET_NOT_ALLOWED = "OffsetNotAllowed" out[OFFSET_NOT_ALLOWED] = True headers.append(OFFSET_NOT_ALLOWED) else: out[f"{ckey}"] = f(rate[rkey]) else: out[f"{ckey}"] = f(rate[rkey]) else: UNLIMITED = "Unlimited" out[UNLIMITED] = True headers.append(UNLIMITED) return tableToMarkdown("Farsight DNSDB Service Limits", out, headers=headers) """ COMMANDS """ @logger def test_module(client, _): try: client.rate_limit() except DemistoException as e: if "forbidden" in str(e): return "Authorization Error: make sure API Key is correctly set" else: raise e return "ok" @logger def dnsdb_flex(client, args): res = list(_run_query(client.flex, args)) def skip_rrname(header) -> bool: return header.lower() not in ("rrname", "fromzonefile") def skip_rdata(header) -> bool: return header.lower() not in ("rdata", "fromzonefile") if args.get("key") == "rdata": skip = skip_rrname else: skip = skip_rdata return CommandResults( readable_output=lookup_to_markdown( res, title="Farsight DNSDB Flex Search", want_bailiwick=False, header_filter=skip, ), outputs_prefix=f"{INTEGRATION_CONTEXT_NAME}.{RECORD_SUBCONTEXT_NAME}", outputs_key_field="", outputs=[build_result_context(r) for r in res], ) @logger def dnsdb_rdata(client, args): type = args.get("type") if type == "name": res = list(_run_query(client.lookup_rdata_name, args)) elif type == "ip": res = list(_run_query(client.lookup_rdata_ip, args)) elif type == "raw": res = list(_run_query(client.lookup_rdata_raw, args)) else: raise Exception(f"Invalid rdata query type: {type}") return CommandResults( readable_output=lookup_to_markdown(res, want_bailiwick=False), outputs_prefix=f"{INTEGRATION_CONTEXT_NAME}.{RECORD_SUBCONTEXT_NAME}", outputs_key_field="", outputs=[build_result_context(r) for r in res], ) @logger def dnsdb_summarize_rdata(client, args): type = args.get("type") if type == "name": res = _run_query(client.summarize_rdata_name, args) elif type == "ip": res = _run_query(client.summarize_rdata_ip, args) elif type == "raw": res = _run_query(client.summarize_rdata_raw, args) else: raise Exception(f"Invalid rdata query type: {type}") return CommandResults( readable_output=summarize_to_markdown(res), outputs_prefix=f"{INTEGRATION_CONTEXT_NAME}.{SUMMARY_SUBCONTEXT_NAME}", outputs_key_field="", outputs=build_result_context(res), ) @logger def dnsdb_rrset(client, args): q = _run_query(client.lookup_rrset, args) res = list(q) return CommandResults( readable_output=lookup_to_markdown(res), outputs_prefix=f"{INTEGRATION_CONTEXT_NAME}.{RECORD_SUBCONTEXT_NAME}", outputs_key_field="", outputs=[build_result_context(r) for r in res], ) @logger def dnsdb_summarize_rrset(client, args): res = _run_query(client.summarize_rrset, args) return CommandResults( readable_output=summarize_to_markdown(res), outputs_prefix=f"{INTEGRATION_CONTEXT_NAME}.{SUMMARY_SUBCONTEXT_NAME}", outputs_key_field="", outputs=build_result_context(res), ) @logger def dnsdb_rate_limit(client, _): res = client.rate_limit() return CommandResults( readable_output=rate_limit_to_markdown(res), outputs_prefix=f"{INTEGRATION_CONTEXT_NAME}.{RATE_SUBCONTEXT_NAME}", outputs_key_field="", outputs=build_rate_limits_context(res), ) def main(): """ PARSE AND VALIDATE INTEGRATION PARAMS """ apikey = demisto.params().get("apikey") base_url = demisto.params().get("url") if not base_url: base_url = DEFAULT_DNSDB_SERVER verify_certificate = not demisto.params().get("insecure", False) proxy = demisto.params().get("useproxy", False) client = Client(base_url, apikey, verify=verify_certificate, proxy=proxy) command = demisto.command() LOG(f"Command being called is {command}") try: if command == "test-module": return_results(test_module(client, demisto.args())) elif command == f"{INTEGRATION_COMMAND_NAME}-flex": return_results(dnsdb_flex(client, demisto.args())) elif command == f"{INTEGRATION_COMMAND_NAME}-rdata": return_results(dnsdb_rdata(client, demisto.args())) elif command == f"{INTEGRATION_COMMAND_NAME}-summarize-rdata": return_results(dnsdb_summarize_rdata(client, demisto.args())) elif command == f"{INTEGRATION_COMMAND_NAME}-rrset": return_results(dnsdb_rrset(client, demisto.args())) elif command == f"{INTEGRATION_COMMAND_NAME}-summarize-rrset": return_results(dnsdb_summarize_rrset(client, demisto.args())) elif command == f"{INTEGRATION_COMMAND_NAME}-rate-limit": return_results(dnsdb_rate_limit(client, demisto.args())) # Log exceptions except Exception as e: demisto.error(traceback.format_exc()) # print the traceback err_msg = f"Error in {INTEGRATION_NAME} Integration [{e}]" return_error(err_msg, error=e) if __name__ in ("__main__", "__builtin__", "builtins"): main()