CIRCLEHashlookup
CIRCL hash lookup is a public API to lookup hash values against known database of files. NSRL RDS database is included and many others are also included. The API is accessible via HTTP ReST API and the API is also described as an OpenAPI. The service is free and served as a best-effort basis.
Data Enrichment & Threat Intelligence · CIRCL hashlookup (hashlookup.circl.lu)
Details
| ID | CIRCLEHashlookup |
|---|---|
| Provider | LHC |
| Category | Data Enrichment & Threat Intelligence |
| From Version | 6.2.0 |
| Docker Image | demisto/python3:3.12.8.3296088 |
| Supported Modules | Agentix XSIAM |
README
CIRCL hash lookup is a public API to lookup hash values against known database of files. NSRL RDS database is included and many others are also included. The API is accessible via HTTP ReST API and the API is also described as an OpenAPI. The service is free and served as a best-effort basis.
This integration was integrated and tested with online version of CIRCLEHashlookup
Configure CIRCLEHashlookup in Cortex
| Parameter | Description | Required |
|---|---|---|
| Server URL (e.g. https://hashlookup.circl.lu) | True | |
| Trust any certificate (not secure) | False | |
| Use system proxy settings | False | |
| Source Reliability | Reliability of the source providing the intelligence data. | True |
| Create relationships | Create relationships between indicators as part of Enrichment. | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
circl-info
Get information about the hash lookup database
Base Command
circl-info
Input
| Argument Name | Description | Required |
| — | — | — |
Context Output
| Path | Type | Description |
|---|---|---|
| Circl.Info | string | Info about the hashlookup database |
Command Example
#### Human Readable Output
### circl-bulk-md5
***
Bulk search of MD5 hashes
#### Base Command
`circl-bulk-md5`
#### Input
| **Argument Name** | **Description** | **Required** |
| --- | --- | --- |
| md5_list | List of MD5s to query. | Required |
#### Context Output
| **Path** | **Type** | **Description** |
| --- | --- | --- |
| Circl.MD5 | string | Results of bulk MD5 query |
#### Command Example
Human Readable Output
circl-bulk-sha1
Bulk search of SHA1 hashes
Base Command
circl-bulk-sha1
Input
| Argument Name | Description | Required |
|---|---|---|
| sha1_list | List of SHA1 to search. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| Circl.SHA1 | string | Results of bulk SHA1 query |
Command Example
#### Human Readable Output
### file
***
Checks the file reputation of the specified hash.
#### Base Command
`file`
#### Input
| **Argument Name** | **Description** | **Required** |
| --- | --- | --- |
| file | Hash to query. | Required |
#### Context Output
| **Path** | **Type** | **Description** |
| --- | --- | --- |
| File.Name | string | Name of the file |
| File.Size | number | Size of the file |
| File.MD5 | string | MD5 hash of the file |
| File.SHA1 | string | SHA1 hash of the file |
| File.SHA256 | string | SHA256 hash of the file |
| File.SHA512 | string | SHA512 hash of the file |
| File.SSDeep | string | SSDeep of the file |
| DbotScore.Indicator | string | The indicator value. |
| DbotScore.Reliability | string | The reliability of the source providing the intelligence data |
| DbotScore.Score | number | An integer regarding the status of the indicator |
| DbotScore.Type | string | The indicator type |
| DbotScore.Vendor | string | The vendor used to calculate the score |
| DBotScore.Indicator | String | The indicator that was tested. |
| DBotScore.Score | Number | The actual score. |
| DBotScore.Type | String | Type of indicator. |
| DBotScore.Vendor | String | Vendor used to calculate the score. |
#### Command Example
Human Readable Output
circl-top
Return the top 100 of most queried values.
Base Command
circl-top
Input
| Argument Name | Description | Required |
| — | — | — |
Context Output
| Path | Type | Description |
|---|---|---|
| Circl.Top | string | The top 100 of most queried values |
Command Example
``````
Human Readable Output
Configuration parameters
url— Server URL (e.g. https://hashlookup.circl.lu) (required)insecure— Trust any certificate (not secure)proxy— Use system proxy settingsintegrationReliability— Source Reliability (required)create_relationships— Create relationships
Commands (5)
-
circl-bulk-md5Bulk search of MD5 hashes.
-
circl-bulk-sha1Bulk search of SHA1 hashes.
-
circl-infoGet information about the hash lookup database.
-
circl-topReturn the top 100 of most queried values.
-
fileChecks the file reputation of the specified hash.
import demistomock as demisto # noqa: F401 from CommonServerPython import * # noqa: F401 """ CLIENT CLASS """ class Client(BaseClient): def __init__(self, base_url: str, headers: dict, proxy: bool = False, verify: bool = False): super().__init__(base_url=base_url, headers=headers, proxy=proxy, verify=verify) def get_info(self) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix="/info") def get_top_stats(self) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix="/stats/top") def bulk_search(self, hashtype: str, data: Dict[str, List[str]]) -> Dict[str, Any]: return self._http_request(method="POST", url_suffix=f"/bulk/{hashtype}", json_data=data) def file_search(self, hashtype: str, hashvalue: str) -> Dict[str, Any]: return self._http_request(method="GET", url_suffix=f"/lookup/{hashtype}/{hashvalue}") def test_module(client) -> str: result = client._http_request("GET", "/info") if result: return "ok" else: return "Test failed: " + str(result) def correct_output_keys(results: Dict[str, str]) -> Dict[str, str]: newoutput = {} if "FileName" in results: newoutput["Name"] = results.pop("FileName") if "FileSize" in results: newoutput["Size"] = results.pop("FileSize") if "MD5" in results: newoutput["MD5"] = results.pop("MD5") if "SHA-1" in results: newoutput["SHA1"] = results.pop("SHA-1") if "SHA-256" in results: newoutput["SHA1"] = results.pop("SHA-256") if "SHA-512" in results: newoutput["SHA1"] = results.pop("SHA-512") if "SSDEEP" in results: newoutput["SSDeep"] = results.pop("SSDEEP") newoutput.update(results) return newoutput def relationship_creator(entity_a, entity_b_list, reliability) -> List[EntityRelationship]: list_of_relationships = [] for item in entity_b_list: if item.get("SHA-256"): entity_b = item.get("SHA-256") elif item.get("SHA-1"): entity_b = item.get("SHA-1") else: entity_b = item.get("MD5") relation_by_type = "related-to" list_of_relationships.append( EntityRelationship( name=relation_by_type, entity_a=entity_a, entity_a_type="File", entity_b=entity_b, entity_b_type="File", source_reliability=reliability, brand="Circl", ) ) return list_of_relationships def create_file_output(results: Dict[str, str], hashtype: str, reliability: str, create_relationships: bool) -> CommandResults: relationships = [] if hashtype == "sha256": file_hash = results.get("SHA-256") elif hashtype == "sha1": file_hash = results.get("SHA-1") elif hashtype == "md5": file_hash = results.get("MD5") else: file_hash = "" demisto.debug(f"{hashtype=} doesn't match any type. {file_hash=}") if "KnownMalicious" in results: dbot_score_object = Common.DBotScore( indicator=file_hash, indicator_type=DBotScoreType.FILE, integration_name="Circl", score=3, reliability=reliability ) else: dbot_score_object = Common.DBotScore( indicator=file_hash, indicator_type=DBotScoreType.FILE, integration_name="Circl", score=0, reliability=reliability ) file = Common.File( dbot_score=dbot_score_object, name=results.get("FileName"), file_type=results.get("mimetype"), md5=results.get("MD5"), sha1=results.get("SHA-1"), sha256=results.get("SHA-256"), size=results.get("FileSize"), ssdeep=results.get("SSDEEP"), ) if create_relationships: parents = results.get("parents") if parents: relationships = relationship_creator(file_hash, parents, reliability) modifiedresults = correct_output_keys(results) human_readable = tableToMarkdown("Cirlc hashlookup results", modifiedresults) output = CommandResults( outputs_prefix="File", outputs_key_field=file_hash, outputs=modifiedresults, indicator=file, relationships=relationships, readable_output=human_readable, ) return output def create_output(results: Dict[str, str], endpoint: str, keyfield: str = "") -> CommandResults: human_readable = tableToMarkdown("Circl results", results) output = CommandResults( outputs_prefix=f"CIRCL.{endpoint}", outputs_key_field=keyfield, outputs=results, readable_output=human_readable ) return output def main(): # get the service API url reliability = demisto.params().get("integrationReliability", DBotScoreReliability.B) create_relationships = demisto.params().get("create_relationships", True) base_url = demisto.params()["url"] verify_certificate = not demisto.params().get("insecure", False) proxy = demisto.params().get("proxy", False) headers = {"Accept": "application/json", "Content-type": "application/json"} demisto.info(f"Command being called is {demisto.command()}") try: client = Client(base_url=base_url, verify=verify_certificate, headers=headers, proxy=proxy) args = demisto.args() if demisto.command() == "test-module": # This is the call made when pressing the integration Test button. result = test_module(client) return_results(result) elif demisto.command() == "circl-info": results = client.get_info() return_results(create_output(results, "Info")) elif demisto.command() == "circl-top": results = client.get_top_stats() return_results(create_output(results, "Top")) elif demisto.command() == "circl-bulk-md5": md5list = argToList(args.get("md5_list")) data = {"hashes": md5list} results = client.bulk_search("md5", data) return_results(create_output(results, "MD5")) elif demisto.command() == "circl-bulk-sha1": sha1list = argToList(args.get("sha1_list")) data = {"hashes": sha1list} results = client.bulk_search("sha1", data) return_results(create_output(results, "SHA1")) elif demisto.command() == "file": file_list = argToList(args.get("file")) if len(file_list) == 0: raise ValueError("Hash(es) not specified") for item in file_list: if len(item) == 32: results = client.file_search("md5", item) return_results(create_file_output(results, "md5", reliability, create_relationships)) elif len(item) == 40: results = client.file_search("sha1", item) return_results(create_file_output(results, "sha1", reliability, create_relationships)) elif len(item) == 64: results = client.file_search("sha256", item) return_results(create_file_output(results, "sha256", reliability, create_relationships)) else: return_error("Hash value not valid md5, sha1 or sha256") # Log exceptions except Exception as e: return_error(f"Failed to execute {demisto.command()} command. Error: {e!s}") if __name__ in ("__main__", "__builtin__", "builtins"): main()