DBot Create Phishing Classifier V2

Create a phishing classifier using machine learning techniques, based on email content.

Machine Learning · 8 tasks · 15 inputs · 5 outputs

Inputs

  • modelName — The model name to store in the system.
  • emailTextKey — A comma-separated list of incident fields names with the email body or html body. You can also use "|" if you want to choose the first non-empty value from a list of fields.
  • emailSubjectKey — A comma-separated list of incident fields names with the email subject. You can also use "|" if you want to choose the first non-empty value from a list of fields.
  • emailTagKey — The field name with the email tag. Supports a comma-separated list. The first non-empty value will be taken.
  • phishingLabels — A comma-separated list of email tags values and mapping. The script considers only the tags specified in this field. You can map the label to another value by using this format: LABEL:MAPPED_LABEL. For example, for 4 values in an email tag: malicious, credentials harvesting, inner communication, external legit email, unclassified. While training, we want to ignore the "unclassified" tag, and refer to "credentials harvesting" as "malicious" too. Also, we want to merge "inner communication" and "external legit email" to a single tag called "non-malicious". The input will be: malicious, credentials harvesting:malicious, inner communication:non-malicious, external legit email:non-malicious.
  • incidentsQuery — The incidents query to fetch the training data for the model.
  • maxIncidentsToFetchOnTraining — The maximum number of incidents to fetch.
  • hashSeed — If non-empty, hash every word with this seed.
  • historicalDataFileListName — The name of the Cortex XSOAR list that contains historical data training samples for the model.
  • overrideModel — Whether to override the existing model if a model with the same name exists. Default is "false".
  • incidentTypes — A common-separated list of incident types by which to filter.
  • dedupThreshold — Remove emails with similarity greater than this threshold, range 0-1, where 1 is completely identical.
  • removeShortTextThreshold — Sample text of which the total number words that are less than or equal to this number will be ignored.
  • modelTargetAccuracy — The model target accuracy, between 0 and 1.
  • outputFormat — The output file format. Can be "json" or "pickle".

Outputs

  • DBotPhishingClassifier.EvaluationScores.Precision.All — Average binary precision over all classes (0-1).
  • DBotPhishingClassifier.EvaluationScores.TP.All — The number of instances of all classes that were predicted correctly.
  • DBotPhishingClassifier.EvaluationScores.Coverage.All — The number of instances that were predicted at a probability greater than the threshold.
  • DBotPhishingClassifier.EvaluationScores.Total.All — The total number of instances.
  • DBotPhishingClassifier.ModelName — The name of the model in Cortex XSOAR.

Flowchart

yes Start Start Done Done Get Incidents - GetIncidentsByQuery Get Incidents GetIncidentsByQuery Pre-process file - DBotPreProcessTextData Pre-process file DBotPreProcessTextData Train Model - DBotTrainTextClassifierV2 Train Model DBotTrainTextClassifierV2 Historical data exist? Historical data exist? Get Historical File - Base64ListToFile Get Historical File Base64ListToFile Extend Query - ExtendQueryBasedOnPhishingLabels Extend Query ExtendQueryBasedOnPhishingLabels