FindDuplicateEmailIncidents

Details

ID	`FindDuplicateEmailIncidents`
Language	python
From Version	5.0.0
Docker Image	`demisto/sklearn:1.0.0.12545527`
Tags	ml phishing

README

Can be used to find duplicate emails for incidents of type phishing, including malicious, spam, and legitimate emails.

Script Data

Name	Description
Script Type	python3
Tags	ml, phishing
Cortex XSOAR Version	5.0.0

Inputs

Argument Name	Description
incidentTypeFieldName	The name of the incident field where its type is stored. Default is “type”. Change this argument only in case you use a custom field for specifying incident type.
incidentTypes	A comma-separated list of incident types by which to filter. The default is the current incident type. Specify “None” to ignore incident type from deduplication logic.
existingIncidentsLookback	The start date by which to search for duplicated existing incidents. Date format is the same as in the incidents query page. For example, “3 days ago”, “2019-01-01T00:00:00 +0200”).
query	Additional text by which to query incidents.
limit	The maximum number of incidents to fetch.
emailSubject	Subject of the email.
emailBody	Body of the email.
emailBodyHTML	HTML body of the email.
emailFrom	Incident fields contains the email from value.
fromPolicy	Whether to take into account the email from field for deduplication. “TextOnly” - incidents will be considered as duplicated based on test similarity only, ignoring the sender’s address. “Exact” - incidents will be considered as duplicated if their text is similar and their sender is the same. “Domain” - incidents will be considered as duplicated if their text is similar and their senders’ address has the same domain. Default is “Domain”.
statusScope	Whether to compare the new incident to past closed or non closed incidents only.
closeAsDuplicate	Whether to close the current incident if a duplicate incident is found. Only supported in Cortex XSOAR.
threshold	Threshold to consider incident as duplication, number between 0-1
maxIncidentsToReturn	Maximum number of duplicate incidents IDs to return.
populateFields	A comma-separated list of incident fields to populate.
exsitingIncidentsLookback	Deprecated. Use the existingIncidentsLookback argument instead.

Outputs

Path	Description	Type
duplicateIncident	The oldest duplicate incident found with the highest similarity to the current incident.	unknown
duplicateIncident.id	Duplicate incident ID.	string
duplicateIncident.rawId	Duplicate incident ID.	Unknown
duplicateIncident.name	Duplicate incident name.	Unknown
duplicateIncident.similarity	Number in range 0-1 which describe the similarity between the existing incident and the new incident.	Unknown
isDuplicateIncidentFound	Whether a duplicate incident was found (“true” or “false”).	boolean
allDuplicateIncidents	All duplicate incidents found where their similarity with the new incident exceeds the threshold.	Unknown
allDuplicateIncidents.id	A list of all duplicate incidents IDs found.	Unknown
allDuplicateIncidents.rawId	A list of all duplicate incidents IDs found.	Unknown
allDuplicateIncidents.name	A list of all duplicate incidents names found.	Unknown
allDuplicateIncidents.similarity	A list of the similarity between duplicate incidents and new the incident of all duplicate incidents names found.	Unknown

Source

FindDuplicateEmailIncidents.py FindDuplicateEmailIncidents_test.py FindDuplicateEmailIncidents.yml

Download History Raw

args:
- defaultValue: type
  description: The name of the incident field where its type is stored. Default is "type". Change this argument only in case you use a custom field for specifying incident type.
  name: incidentTypeFieldName
- description: A comma-separated list of incident types by which to filter. The default is the current incident type. Specify "None" to ignore incident type from deduplication logic.
  name: incidentTypes
- defaultValue: 30 days ago
  description: The start date by which to search for duplicated existing incidents. Date format is the same as in the incidents query page. For example, "3 days ago", "2019-01-01T00:00:00 +0200").
  name: existingIncidentsLookback
- description: Additional text by which to query incidents.
  name: query
- defaultValue: '3000'
  description: The maximum number of incidents to fetch.
  name: limit
- auto: PREDEFINED
  defaultValue: emailsubject
  description: Subject of the email.
  name: emailSubject
  predefined:
  - Subject of the email.
- defaultValue: emailbody
  description: Body of the email.
  name: emailBody
  predefined:
  - ''
- defaultValue: emailbodyhtml
  description: HTML body of the email.
  name: emailBodyHTML
- defaultValue: emailfrom
  description: Incident fields contains the email from value.
  name: emailFrom
- auto: PREDEFINED
  defaultValue: TextOnly
  description: Whether to take into account the email from field for deduplication. "TextOnly" - incidents will be considered as duplicated based on test similarity only, ignoring the sender's address. "Exact" - incidents will be considered as duplicated if their text is similar and their sender is the same. "Domain" -  incidents will be considered as duplicated if their text is similar and their senders' address has the same domain. Default is "Domain".
  name: fromPolicy
  predefined:
  - TextOnly
  - Exact
  - Domain
- auto: PREDEFINED
  defaultValue: All
  description: Whether to compare the new incident to past closed or non closed incidents only.
  name: statusScope
  predefined:
  - All
  - ClosedOnly
  - NonClosedOnly
- auto: PREDEFINED
  defaultValue: 'false'
  description: Whether to close the current incident if a duplicate incident is found. Only supported in Cortex XSOAR.
  name: closeAsDuplicate
  predefined:
  - 'true'
  - 'false'
- defaultValue: '0.99'
  description: Threshold to consider incident as duplication, number between 0-1.
  name: threshold
- defaultValue: '20'
  description: Maximum number of duplicate incidents IDs to return.
  name: maxIncidentsToReturn
- description: A comma-separated list of incident fields to populate.
  name: populateFields
- defaultValue: 30 days ago
  description: Deprecated. Use the *existingIncidentsLookback* argument instead.
  name: exsitingIncidentsLookback
comment: Can be used to find duplicate emails for incidents of type phishing, including  malicious, spam, and legitimate emails.
commonfields:
  id: FindDuplicateEmailIncidents
  version: -1
name: FindDuplicateEmailIncidents
outputs:
- contextPath: duplicateIncident
  description: The oldest duplicate incident found with the highest similarity to the current incident.
  type: unknown
- contextPath: duplicateIncident.id
  description: Duplicate incident ID.
  type: string
- contextPath: duplicateIncident.rawId
  description: Duplicate incident ID.
  type: Unknown
- contextPath: duplicateIncident.name
  description: Duplicate incident name.
  type: Unknown
- contextPath: duplicateIncident.similarity
  description: Number in range 0-1 which describe the similarity between the existing incident and the new incident.
  type: Unknown
- contextPath: isDuplicateIncidentFound
  description: Whether a duplicate incident was found ("true" or "false").
  type: boolean
- contextPath: allDuplicateIncidents
  description: All duplicate incidents found where their similarity with the new incident exceeds the threshold.
  type: Unknown
- contextPath: allDuplicateIncidents.id
  description: A list of all duplicate incidents IDs found.
  type: Unknown
- contextPath: allDuplicateIncidents.rawId
  description: A list of all duplicate incidents IDs found.
  type: Unknown
- contextPath: allDuplicateIncidents.name
  description: A list of all duplicate incidents names found.
  type: Unknown
- contextPath: allDuplicateIncidents.similarity
  description: A list of the similarity between duplicate incidents and new the incident of all duplicate incidents names found.
  type: Unknown
script: '-'
subtype: python3
tags:
- ml
- phishing
timeout: 600ns
type: python
dockerimage: demisto/sklearn:1.0.0.12545527
tests:
- Detect & Manage Phishing Campaigns - Test
fromversion: 5.0.0