urlscan.io

Use urlscan.io integration to perform scans on suspected URLs and see their reputation.

Data Enrichment & Threat Intelligence · URLScan.io

Details

ID	`urlscan.io`
Provider	Urlscan io
Category	Data Enrichment & Threat Intelligence
From Version	5.0.0
Docker Image	`demisto/python3:3.12.8.3720084`
Supported Modules	Agentix XSIAM

README

Use urlscan.io integration to perform scans on suspected urls and see their reputation.

Configure urlscan.io on Cortex XSOAR

Navigate to Settings > Integrations > Servers & Services.
Search for urlscan.io.
Click Add instance to create and configure a new integration instance.
- Name: a textual name for the integration instance.
- Server URL (e.g. https://urlscan.io/api/v1/ )
- API Key (needed only for submitting URLs for scanning)
- Scan Visibility: Determines the visibility level of the scan. This will override the 'public submissions' setting.
- Source Reliability. Reliability of the source providing the intelligence data. (The default value is C - Fairly reliable)
- Scan Country. Specify which country the scan should be performed from. If you omit this value, urlscan will try to do automatic country detection based on the TLD of the URL, GeoIP information of the server and of the user.
- Trust any certificate (not secure)
- Use system proxy settings
- URL Threshold. Minimum number of positive results from urlscan.io to consider the URL malicious.
- User Agent: User Agent used during scans with this integration.
Click Test to validate the URLs, token, and connection.

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.

Search for indicators: urlscan-search
(Deprecated) Submit a URL: urlscan-submit
Submit a URL (specify the "using" argument): url

1. Search for indicators

Search for an indicator that is related to previous urlscan.io scans.

Notice: Submitting indicators using this command might make the indicator data publicly available. See the vendor’s documentation for more details.

Base Command

urlscan-search

Input

Argument Name	Description	Required
searchParameter	Enter a parameter to search as a string (IP, File name, sha256, url, domain)	Required
searchType	Allows querying multiple search parameters	Optional

Context Output

Path	Description
URLScan.URL	Bad URLs found
URLScan.Domain	Domain of the URL scanned
URLScan.ASN	ASN of the URL scanned
URLScan.IP	IP of the url scanned
URLScan.ScanID	Scan ID for the URL scanned
URLScan.ScanDate	Latest scan date for the URL
URLScan.Hash	SHA-256 of file scanned
URLScan.FileName	Filename of the file scanned
URLScan.FileSize	File size of the file scanned
URLScan.FileType	File type of the file scanned

Command Example

!urlscan-search searchParameter=8.8.8.8

!urlscan-search searchType=advanced searchParameter="filename:logo.png AND date:>now-24h"

!urlscan-search searchType=raw searchParameter="q=meta%3Asearchhit.search.04eb755f-468d-4421-ab86-210a01ee1bdd&datasource=hostnames&search_after="

2. (Deprecated) Submit a URL directly to urlscan.io

Submits a URL to urlscan.io.

This command is deprecated, but will still work if it is used in a playbook.

Base Command

urlscan-submit

Input

Argument Name	Description	Required
url	URL to scan	Required
timeout	How many seconds to wait to the scan id result. Default is 30 seconds.	Optional
public	Will the submission be public or private	Optional
useragent	User Agent used to perform scans	Optional
scan_visibility	The submission visibility. If specified, overrides the 'public' parameter	Optional

Context Output

Path	Description
URLScan.URLs	URLs related to the scanned URL
URLScan.RelatedIPs	IPs related to the scanned URL
URLScan.RelatedASNs	ASNs related to the scanned URL
URLScan.Countries	Countries associated with the scanned URL
URLScan.relatedhashes	IOCs found for the scanned URL
URLScan.Subdomains	Associated subdomains for the url scanned
URLScan.ASN	ASN of the URL scanned
URLScan.Data	URL of the file found
URLScan.Malicious.Vendor	Vendor reporting the malicious indicator for the file
URLScan.Malicious.Description	Description of the malicious indicator
URLScan.File.Hash	SHA256 of file found
URLScan.File.FileName	File name of file found
URLScan.File.FileType	File type of the file found
URLScan.File.Hostname	URL where the file was found
URLScan.Certificates	Certificates found for the scanned URL

Command Example

!urlscan-submit url=http://www.github.com/

3. Submit a URL (specify using urlscan.io)

Submit a URL to scan and specify the using argument as urlscan.io.

Base Command

url

Input

Argument Name	Description	Required
url	URL to scan	Required
timeout	How many seconds to wait for the scan ID result. Default is 30 seconds.	Optional
public	Whether the submission will be public or private	Optional
retries	Number of retries if the API rate limit is reached. This argument is optional, but if you specify this argument, you need to specify the wait argument.	Optional
wait	Time interval (in seconds) between retries, if the API rate limit is reached. This argument is optional, but if you specify the retries argument, you need to specify this argument.	Optional
useragent	User Agent used to perform scans	Optional
scan_visibility	The submission visibility. If specified, overrides the 'public' parameter	Optional
use_url_as_name	Whether to use the URL as the screenshot name. Default is false which sets screenshot name to screenshot.png	Optional

Context Output

Path	Description
URLScan.URLs	URLs related to the scanned URL
URLScan.RelatedIPs	IPs related to the URL scanned
URLScan.RelatedASNs	ASNs related to the scanned URL
URLScan.Countries	Countries associated with the scanned URL
URLScan.relatedhashes	IOCs found for the scanned URL
URLScan.Subdomains	Associated sub-domains for the scanned URL
URLScan.ASN	ASN of the scanned URL
URLScan.Data	URL of the file found
URLScan.Malicious.Vendor	Vendor reporting the malicious indicator for the file
URLScan.Malicious.Description	Description of the malicious indicator
URLScan.File.Hash	SHA-256 of file found
URLScan.File.FileName	File name of file found
URLScan.File.FileType	File type of the file found
URLScan.File.Hostname	URL where the file was found
URLScan.Certificates	Certificates found for the scanned URL
URLScan.RedirectedURLS	Redirected URLs from the URL scanned
URLScan.EffectiveURL	Effective URL of the original URL
URL.ASN	The URL ASN.
URL.FeedRelatedIndicators.value	Indicators that are associated with the URL.
URL.FeedRelatedIndicators.type	The type of the indicators that are associated with the URL.
URL.Geo.Country	The URL country.
URL.ASOwner	The URL AS owner.
URL.Tags	Tags that are associated with the URL.

Command Example

!url url=http://www.github.com/ using="urlscan.io"

Configuration parameters

creds_apikey —
scan_visibility — Scan Visibility (required)
integrationReliability — Source Reliability (required)
country — Scan Country
url_threshold — URL Threshold. Minimum number of positive results from urlscan.io to consider the URL malicious.
useragent — User Agent
create_relationships — Create relationships
insecure — Trust any certificate (not secure)
proxy — Use system proxy settings
is_public — Enable public submissions by default.
apikey — API Key (only required for scanning URLs)

Commands (7)

url
Submits a URL to scan.
urlscan-get-http-transaction-list Deprecated
Returns the HTTP transaction list for the specified URL. Do not use this command in conjunction with the urlscan-get-http-transactions script.
urlscan-get-result-page Deprecated
Returns the results page for the specified UUID.
urlscan-poll-uri Deprecated
Polls the urlscan service regarding the results of the specified URI.
urlscan-search
Search for an indicator that is related to former urlscan.io scans.
urlscan-submit Deprecated
Deprecated. Use the url command instead.
urlscan-submit-url-command Deprecated
Submits a URL to retrieve its UUID.

Source

UrlScan.py UrlScan_test.py UrlScan.yml UrlScan_description.md

Download History Raw

category: Data Enrichment & Threat Intelligence
provider: Urlscan io
sectionorder:
- Connect
- Collect
commonfields:
  id: urlscan.io
  version: -1
configuration:
- displaypassword: API Key (only required for scanning URLs)
  name: creds_apikey
  type: 9
  hiddenusername: true
  display: ''
  section: Connect
  required: false
- additionalinfo: Determines the visibility level of the scan.
  display: Scan Visibility
  name: scan_visibility
  options:
  - public
  - private
  - unlisted
  required: true
  type: 15
  section: Collect
- additionalinfo: Reliability of the source providing the intelligence data.
  defaultvalue: C - Fairly reliable
  display: Source Reliability
  name: integrationReliability
  options:
  - A+ - 3rd party enrichment
  - A - Completely reliable
  - B - Usually reliable
  - C - Fairly reliable
  - D - Not usually reliable
  - E - Unreliable
  - F - Reliability cannot be judged
  required: true
  type: 15
  section: Collect
- additionalinfo: Specify which country the scan should be performed 
  display: Scan Country
  name: country
  options:
    - "AT - Austria"
    - "AU - Australia"
    - "CA - Canada"
    - "CH - Switzerland"
    - "DE - Germany"
    - "DK - Denmark"
    - "ES - Spain"
    - "FI - Finland"
    - "FR - France"
    - "GB - United Kingdom"
    - "IL - Israel"
    - "IS - Iceland"
    - "IT - Italy"
    - "JP - Japan"
    - "NL - Netherlands"
    - "NO - Norway"
    - "NZ - New Zealand"
    - "PL - Poland"
    - "PT - Portugal"
    - "SE - Sweden"
    - "SG - Singapore"
    - "US - United States"
  required: false
  type: 15
  section: Collect
- defaultvalue: '1'
  display: URL Threshold. Minimum number of positive results from urlscan.io to consider the URL malicious.
  name: url_threshold
  type: 0
  section: Collect
  advanced: true
  required: false
- display: User Agent
  name: useragent
  type: 0
  additionalinfo: User Agent to perform requests
  section: Connect
  advanced: true
  required: false
- defaultvalue: 'true'
  additionalinfo: Create relationships between indicators as part of Enrichment.
  display: Create relationships
  name: create_relationships
  type: 8
  section: Collect
  advanced: true
  required: false
- display: Trust any certificate (not secure)
  name: insecure
  type: 8
  section: Connect
  advanced: true
  required: false
- display: Use system proxy settings
  name: proxy
  type: 8
  section: Connect
  advanced: true
  required: false
- defaultvalue: 'false'
  display: Enable public submissions by default.
  name: is_public
  type: 8
  hidden: true
  section: Connect
  advanced: true
  required: false
- display: API Key (only required for scanning URLs)
  name: apikey
  type: 4
  hidden: true
  section: Connect
  advanced: true
  required: false
description: Use urlscan.io integration to perform scans on suspected URLs and see their reputation.
display: urlscan.io
name: urlscan.io
script:
  commands:
  - arguments:
    - default: true
      description: A parameter for which to search (as a string), for example an IP address, file name, SHA256 hash, URL, domain, and so on.
      name: searchParameter
      required: true
    - auto: PREDEFINED
      description: The search type. When advanced, allows to query multiple search parameters.
      predefined:
      - advanced
      - raw
      name: searchType
    - defaultValue: '20'
      description: The maximum number of results to return. Default is 20.
      name: limit
    description: Search for an indicator that is related to former urlscan.io scans.
    name: urlscan-search
    outputs:
    - contextPath: URLScan.URL
      description: The URL.
      type: string
    - contextPath: URLScan.Domain
      description: The domain of the scanned URL.
      type: string
    - contextPath: URLScan.ASN
      description: The ASN of the scanned URL.
      type: string
    - contextPath: URLScan.IP
      description: The IP address of the scanned URL.
      type: string
    - contextPath: URLScan.ScanID
      description: The scan ID of the scanned URL.
      type: string
    - contextPath: URLScan.ScanDate
      description: The date that the URL was last scanned.
      type: string
    - contextPath: URLScan.Hash
      description: The SHA256 hash of the scanned file.
      type: string
    - contextPath: URLScan.FileName
      description: The file name of the scanned file.
      type: string
    - contextPath: URLScan.FileSize
      description: The size of the scanned file.
      type: number
    - contextPath: URLScan.FileType
      description: File type of the file scanned.
      type: string
    - contextPath: URLScan.Search.Results
      description: The unformatted search results from a raw search.
    polling: true
  - arguments:
    - description: The URL to scan.
      isArray: true
      name: url
      required: true
    - defaultValue: '60'
      description: The amount of time (in seconds) to wait for the scan ID result before timeout. Default is 60.
      name: timeout
    - auto: PREDEFINED
      description: The submission visibility. If specified, overrides the 'public' parameter.
      predefined:
      - public
      - private
      - unlisted
      name: scan_visibility
    - description: The submission type. Can be "public" or "private".
      name: public
    - defaultValue: '20'
      description: The maximum number of Limits the returned list of Certificates, IP's and ASN's.
      name: limit
    - auto: PREDEFINED
      description: Determines whether a scan should continue if one of the URLs is on block list.
      isArray: true
      name: continue_on_blacklisted_urls
      predefined:
      - 'true'
      - 'false'
      defaultValue: 'false'
    - name: useragent
      description: User agent to perform request.
    - name: use_url_as_name
      description: Whether to use the URL as the file name.
      auto: PREDEFINED
      predefined:
      - "false"
      - "true"
      defaultValue: "false"
    deprecated: true
    description: Deprecated. Use the url command instead.
    name: urlscan-submit
    outputs:
    - contextPath: URL.Data
      description: The URL submitted for scanning.
      type: string
    - contextPath: URL.Malicious.Vendor
      description: For malicious URLs, the vendor that made the decision.
      type: string
    - contextPath: URL.Malicious.Description
      description: For malicious URLs, the reason that the vendor made the decision.
      type: string
    - contextPath: URLScan.RelatedIPs
      description: IP addresses related to the the scanned URL.
      type: string
    - contextPath: URLScan.RelatedASNs
      description: ASNs related to the scanned URL.
      type: string
    - contextPath: URLScan.Countries
      description: Countries associated with the scanned URL.
      type: string
    - contextPath: URLScan.RelatedHash
      description: File hashes related to the scanned URL.
      type: string
    - contextPath: URLScan.Subdomains
      description: Subdomains related to the scanned URL.
      type: string
    - contextPath: URLScan.ASN
      description: ASN of the scanned URL.
      type: string
    - contextPath: URLScan.Data
      description: URL of the file.
      type: string
    - contextPath: URLScan.Malicious.Vendor
      description: The vendor reporting the malicious indicator for the file.
      type: string
    - contextPath: URLScan.Malicious.Description
      description: A description of the malicious indicator.
      type: string
    - contextPath: URLScan.File.Hash
      description: SHA256 of file found.
      type: string
    - contextPath: URLScan.File.FileName
      description: File name of file found.
      type: string
    - contextPath: URLScan.File.FileType
      description: File type of the file found.
      type: string
    - contextPath: URLScan.File.Hostname
      description: URL where the file was found.
      type: string
    - contextPath: URLScan.Certificates
      description: Certificates found for the URL scanned.
      type: string
    - contextPath: DBotScore.Score
      description: Score retrieved for Dbot.
      type: number
    - contextPath: DBotScore.Type
      description: Type of indicator tested for.
      type: string
    - contextPath: DBotScore.Vendor
      description: Vendor who provided DBot Score.
      type: string
    - contextPath: DBotScore.Indicator
      description: Indicator URLScan tested for.
      type: string
    - contextPath: URLScan.RedirectedURLs
      description: Redirected URLs from the URL scanned.
      type: string
    - contextPath: URLScan.EffectiveURL
      description: Effective URL of the original URL.
      type: string
    polling: true
  - arguments:
    - default: true
      description: Url to scan.
      isArray: true
      name: url
      required: true
    - auto: PREDEFINED
      description: The submission visibility. If specified, overrides the 'public' parameter.
      predefined:
      - public
      - private
      - unlisted
      name: scan_visibility
    - defaultValue: '60'
      description: The amount of time (in seconds) to wait for the scan ID result before timeout. Default is 60.
      name: timeout
    - description: The submission type. Can be "public" or "private".
      name: public
    - defaultValue: '20'
      description: The maximum number of results to return.
      name: limit
    - defaultValue: '5'
      description: The amount of time (in seconds) to wait between tries if the API rate limit is exceeded.
      name: wait
    - defaultValue: '0'
      description: Number of retries for the API rate limit. Default is 0.
      name: retries
    - auto: PREDEFINED
      description: Determines whether a scan should continue if one of the URLs is on block list.
      isArray: true
      name: continue_on_blacklisted_urls
      predefined:
      - 'true'
      - 'false'
      defaultValue: 'false'
    - name: useragent
      description: User agent to perform request.
    - name: use_url_as_name
      description: Whether to use the URL as the file name.
      auto: PREDEFINED
      predefined:
      - "false"
      - "true"
      defaultValue: "false"
    description: Submits a URL to scan.
    name: url
    outputs:
    - contextPath: URL.Data
      description: The URL submitted for scanning.
      type: string
    - contextPath: URL.Malicious.Vendor
      description: For malicious URLs, the vendor that made the decision.
      type: string
    - contextPath: URL.Malicious.Description
      description: For malicious URLs, the reason that the vendor made the decision.
      type: string
    - contextPath: URL.Relationships.EntityA
      description: The source of the relationship.
      type: string
    - contextPath: URL.Relationships.EntityB
      description: The destination of the relationship.
      type: string
    - contextPath: URL.Relationships.Relationship
      description: The name of the relationship.
      type: string
    - contextPath: URL.Relationships.EntityAType
      description: The type of the source of the relationship.
      type: string
    - contextPath: URL.Relationships.EntityBType
      description: The type of the destination of the relationship.
      type: string
    - contextPath: URLScan.RelatedIPs
      description: The IP addresses related to the scanned URL.
      type: string
    - contextPath: URLScan.RelatedASNs
      description: The ASNs related to the scanned URL.
      type: string
    - contextPath: URLScan.Countries
      description: The countries associated with the scanned URL.
      type: string
    - contextPath: URLScan.RelatedHash
      description: File hashes related to the scanned URL.
      type: string
    - contextPath: URLScan.Subdomains
      description: Subdomains associated with the scanned URL.
      type: string
    - contextPath: URLScan.ASN
      description: The ASN of the scanned URL.
      type: string
    - contextPath: URLScan.Data
      description: The URL of the file.
      type: string
    - contextPath: URLScan.Malicious.Vendor
      description: The vendor that reported the malicious indicator for the file.
      type: string
    - contextPath: URLScan.Malicious.Description
      description: A description of the malicious indicator.
      type: string
    - contextPath: URLScan.File.Hash
      description: The SHA256 hash of file.
      type: string
    - contextPath: URLScan.File.FileName
      description: The name of the file.
      type: string
    - contextPath: URLScan.File.FileType
      description: The file type.
      type: string
    - contextPath: URLScan.File.Hostname
      description: The URL of the file.
      type: string
    - contextPath: File.SHA256
      description: The SHA256 hash of the file.
      type: string
    - contextPath: File.Name
      description: The name of the file.
      type: string
    - contextPath: File.Type
      description: The file type.
      type: string
    - contextPath: File.Hostname
      description: The URL of the file.
      type: string
    - contextPath: URLScan.Certificates
      description: The certificates found for the scanned URL.
      type: string
    - contextPath: DBotScore.Score
      description: The actual score.
      type: string
    - contextPath: DBotScore.Type
      description: The indicator type.
      type: string
    - contextPath: DBotScore.Vendor
      description: The vendor used to calculate the score.
      type: string
    - contextPath: DBotScore.Indicator
      description: The indicator that was tested.
      type: string
    - contextPath: URLScan.RedirectedURLs
      description: Redirected URLs from the scanned URL.
      type: string
    - contextPath: URLScan.EffectiveURL
      description: Effective URL of the original URL.
      type: string
    - contextPath: URL.ASN
      description: The URL ASN.
      type: String
    - contextPath: URL.FeedRelatedIndicators.value
      description: Indicators that are associated with the URL.
      type: String
    - contextPath: URL.FeedRelatedIndicators.type
      description: the type of the indicators that are associated with the URL.
      type: String
    - contextPath: URL.Geo.Country
      description: The URL country.
      type: String
    - contextPath: URL.ASOwner
      description: The URL AS owner.
      type: String
    - contextPath: URL.Tags
      description: Tags that are associated with the URL.
      type: String
    - contextPath: URL.ResultPage
      description: Page in the URLScan UI displaying the scan result.
      type: String
    polling: true
  - arguments:
    - description: The UUID of the URL for which to search the transaction list.
      name: uuid
      required: true
    - defaultValue: '20'
      description: The maximum number of results to return to the War Room. Maximum is 100. Default is 20.
      name: limit
    - description: The URL for which to search the transaction list.
      name: url
      required: true
    deprecated: true
    description: Returns the HTTP transaction list for the specified URL. Do not use this command in conjunction with the urlscan-get-http-transactions script.
    name: urlscan-get-http-transaction-list
    outputs:
    - contextPath: URLScan.URL
      description: The URL address that was scanned.
      type: string
    - contextPath: URLScan.httpTransaction
      description: A link to the HTTP transaction made during the search for the specified URL.
      type: string
    polling: true
  - arguments:
    - description: The URL sought after.
      name: url
      required: true
    deprecated: true
    description: Submits a URL to retrieve its UUID.
    name: urlscan-submit-url-command
    polling: true
  - arguments:
    - description: The URI for which to get the results.
      name: uri
      required: true
    deprecated: true
    description: Polls the urlscan service regarding the results of the specified URI.
    name: urlscan-poll-uri
  - arguments:
    - description: The UUID of the URL for which to search.
      name: uuid
      required: true
    deprecated: true
    description: Returns the results page for the specified UUID.
    name: urlscan-get-result-page
  script: ''
  subtype: python3
  type: python
  dockerimage: demisto/python3:3.12.8.3720084
fromversion: 5.0.0
tests:
- urlscan_malicious_Test