Symantec Data Loss Prevention Deprecated
Deprecated. Use the Symantec Data Loss Prevention V2 integration instead. Symantec Data Loss Prevention enables you to discover, monitor and protect your sensitive corporate information.
Network Security · Symantec Data Loss Prevention
Details
| ID | Symantec Data Loss Prevention |
|---|---|
| Provider | Broadcom |
| Category | Network Security |
| From Version | 5.0.0 |
| Docker Image | demisto/zeep:1.0.0.23423 |
| Supported Modules | Agentix XSIAM |
README
Symantec Data Loss Prevention let's you discover, monitor and protect your sensitive corporate information.
This integration is now deprecated. Please use the Symantec DLP v2 integration instead.
This integration was integrated and tested with Symantec DLP version 15.5. The integration uses the SOAP-based Incident Reporting and Update API.
Important Note: Symantec DLP 15.7 introduced a new RESTful API, which this integration does not support. If you want to use the new REST API, please file a feature request that we can track. For more information about the Symantec DLP REST API see the Synantec documentation</>. </p>
Permissions
Symantec Data Loss Prevention requires that the integration user be assigned to the "Incident Reporting API Web Service" role. Make sure to follow Symantec's documentation on how to create such a role and assign it to the user:
- Symantec DLP 15.5: Configuring Roles
- Symantec DLP 15.7: Creating a user and role for an Incident Reporting API client
If you are using an AD User to authenticate as the API user make sure to follow the proper naming convention of:
<Username>:<Active_Directory_Domain_In_Upper_Case> OR <Role>\<Username>:<Active_Directory_Domain_In_Upper_Case>More details at: https://knowledge.broadcom.com/external/article/159761/unable-to-authenticate-to-reporting-api.html
Fetch Incidents
The Symantec Data Loss Prevention integration is configured to fetch incidents and integrate them into Cortex XSOAR's incidents and has the fetch limit parameter.
Configure Symantec Data Loss Prevention on Cortex XSOAR
- Navigate to Settings > Integrations > Servers & Services.
- Search for Symantec Data Loss Prevention.
-
Click Add instance to create and configure a new integration instance.
- Name: a textual name for the integration instance.
- Enforce Server (e.g. https://192.168.0.1)
- Username
- Trust any certificate (not secure)
- Use system proxy settings
- Fetch incidents
- Incident type
- First fetch timestamp (e.g., 12 hours, 7 days)</strong></li>
- Saved Report ID
- Fetch limit </ul>
- Click Test to validate the new instance. </ol>
- Log on to the Enforce Server administration console with Administator access mode.
- Go to System > Login Management > Roles > Add Role
- Enter a name for the new role in the Name field.
- In the User Privileges section, select the privileges you want.
- Click on the Save button.
- Go to System > Login Management > DLP Users
- Click on the Add User button and create a user.
- Go to the Roles section, select the new role being created.
- Select the same role in the Default Role menu.
- Click on the Save button.
- Log on to the Enforce Server administration console.
- Go to Incidents > Incident Reports
- Select an incident from the list of reports.
- Click Advanced Filters & Summarization.
- In the Summarize By menu, verify that no primary summary selected and no secondary summary selected are chosen.
- Select > Report > Save, and enter the report name in the Name field
- Click Save
- To retrive the ID of the saved report, move your mouse cursor over the report name.
- symantec-dlp-list-incidents: symantec-dlp-list-incidents
- symantec-dlp-get-incident-details: symantec-dlp-get-incident-details
- symantec-dlp-update-incident: symantec-dlp-update-incident
- symantec-dlp-incident-binaries: symantec-dlp-incident-binaries
- symantec-dlp-list-custom-attributes: symantec-dlp-list-custom-attributes
- symantec-dlp-list-incident-status: symantec-dlp-list-incident-status
- symantec-dlp-incident-violations: symantec-dlp-incident-violations
Please note that for Active Directoy accounts, the username must follow the format
username:DOMAIN. </li>In order that the integration will work you must create a Web Service user, role and saved report in the Enforce Server administration.
To create an user and role do the following:
To create a saved report do the following:
Commands
You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.
1. symantec-dlp-list-incidents
Returns a list of incidents.
Base Command
symantec-dlp-list-incidentsInput
Argument Name Description Required creation_date Get incidents with creation date later than specified. Given in free text (e.g. '2 days') Optional Context Output
Path Type Description SymantecDLP.Incident.ID Number The ID of the Incident Command Example
!symantec-dlp-list-incidentsContext Example
[ "SymantecDLP.Incident.ID": [ 1111, 2222, 3333 ] ]Human Readable Output
Symantec DLP incidents
ID 1111 2222 3333 Additional Information
2. symantec-dlp-get-incident-details
Returns the details of the specified incident.
Base Command
symantec-dlp-get-incident-detailsInput
Argument Name Description Required incident_id Incident ID to get details of. Required custom_attributes This argument can get the following values: all - If all custom attributes are needed none - If none of the custom attributes are needed specific attributes - A list of custom attributes names, seperated by comma. For example: ca1,ca2,ca3 custom attribute group name - A list of custom attributes group names, seperated by comma. For example: cag1, cag2, cag3. This value will retrive all custom attributes in the mentioned group. The value "none" is default. Optional custom_data A list of custom attributes names / custom attribute group names. List should be comma seperated. For example: item1,item2,item3 Optional Context Output
Path Type Description SymantecDLP.Incident.ID Number The ID of the incident. SymantecDLP.Incident.LongID Number The long ID of the incident. SymantecDLP.Incident.StatusCode String The status code of the incident. SymantecDLP.Incident.CreationDate Date The creation date of the incident. SymantecDLP.Incident.DetectionDate Date The detection date of the incident. SymantecDLP.Incident.Severity String The severity of the incident. SymantecDLP.Incident.MessageSource String The localized label that corresponds to the Symantec DLP product that generated the incident. SymantecDLP.Incident.MessageSourceType String Indicates the Symantec DLP product that generated the incident. Can be: NETWORK, DISCOVER, ENDPOINT, DIM, DAR. SymantecDLP.Incident.MessageType String Indicates the Symantec DLP product component that generated the incident. SymantecDLP.Incident.MessageTypeID Number The ID of the Message Type. SymantecDLP.Incident.Policy.Name String The name of the policy. SymantecDLP.Incident.Policy.Version String The version of the policy. SymantecDLP.Incident.Policy.Label String The label of the policy. SymantecDLP.Incident.Policy.ID Number The ID of the policy. SymantecDLP.Incident.BlockedStatus String Indicates whether the message was blocked or not. SymantecDLP.Incident.MatchCount Number Indicates the number of detection rule matches in the incident. SymantecDLP.Incident.RuleViolationCount Number Indicates the number of policy rules that were violated. SymantecDLP.Incident.DetectionServer String The name of the detection server that created the incident. SymantecDLP.Incident.DataOwner.Name String The name of the data owner. SymantecDLP.Incident.DataOwner.Email String The email of the data owner. SymantecDLP.Incident.EventDate Date The date and time at which the violation event occurred. SymantecDLP.Incident.ViolatedPolicyRule.Name String The name of the rule within the policy that the message violated. SymantecDLP.Incident.ViolatedPolicyRule.ID Number The ID of the rule within the policy that the message violated. SymantecDLP.Incident.OtherViolatedPolicy.Name String The name of any additional policies that the message violated. SymantecDLP.Incident.OtherViolatedPolicy.Version String The version of any additional policies that the message violated. SymantecDLP.Incident.OtherViolatedPolicy.Label String The label of any additional policies that the message violated. SymantecDLP.Incident.OtherViolatedPolicy.ID Number The ID of any additional policies that the message violated. SymantecDLP.Incident.CustomAttribute.Name String The custom attribute name. SymantecDLP.Incident.CustomAttribute.Value String The custom attribute value. Command Example
!symantec-dlp-get-incident-details incident_id=2222 custom_attributes="specific attributes" custom_data=ca1,ca2,ca3Context Example
"SymantecDLP.Incident: { 'ID': 2222, 'LongID': 2222, 'StatusCode': 'SUCCESS', 'CreationDate': '2018-08-01T11:50:16', 'DetectionDate': '2018-08-01T11:50:16', 'Severity': 'high', 'MessageSource': 'Endpoint', 'MessageSourceType': 'ENDPOINT', 'MessageType': 'Endpoint Copy to Network Share', 'MessageTypeID': 33, 'Policy.Name': 'CCN number', 'Policy.Version': 1, 'Policy.Label': 'label', 'Policy.ID': '2203', 'ViolatedPolicyRule': [ 'Name': CCN number, 'ID': '334' ], 'OtherViolatedPolicy': [ 'Name': 'CREDIT CARD POLICY TEST', 'Version': 13, 'Label': 'label12' 'ID': '2134' ], 'BlockedStatus': 'Passed', 'MatchCount': 1, 'RuleViolationCount': 1, 'DetectionServer': 'Local - Endpoint', 'DataOwner': { 'Name': 'name', 'Email': 'email', }, 'EventDate': '2018-08-01T11:50:16', 'CustomAttribute': [ { 'Name': 'ca1' 'Value': 'val1' }, { 'Name': 'ca2' 'Value': 'val2' }, { 'Name': 'ca3' 'Value': 'val3' }, ] }Human Readable Output
Symantec DLP incident 2222 details
ID Creation Date Detection Date Severity Status DLP Module DLP Module subtype Policy Name 2222 2018-08-01T11:50:16 2018-08-01T11:50:16 high SUCCESS ENDPOINT Endpoint Copy to Network Share CCN number 3. symantec-dlp-update-incident
Updates the details of a specific incident.
Base Command
symantec-dlp-update-incidentInput
Argument Name Description Required incident_id Incident ID to update. Optional severity Represents the severity level of the incident. Optional status Represents the status value of the incident. You define incident status values using the Enforce Server administration console. Optional remediation_status Represents the remediation status of an incident. Optional remediation_location Represents the remediation location of the incident. Values can be user-defined. Optional custom_attribute_name The custom attribute name. Optional custom_attribute_value The custom attribute value. Optional data_owner_name The data owner name. Optional data_owner_email The data owner email. Optional note The note to be added. Optional note_time The time of the note in ISO format. Optional Context Output
There are no context output for this command.Command Example
!symantec-dlp-update-incident incident_id=2222 data_owner_email=EMAIL data_owner_name=NAME note=NOTE note_time=2018-08-01T11:50:16Human Readable Output
Symantec DLP incident 2222 details
Batch ID Inaccessible Incident Long ID Inaccessible Incident ID Status Code 44102 [] [] SUCCESS 4. symantec-dlp-incident-binaries
Retrieves additional components of the message that generated the incident, might include binary files.
Base Command
symantec-dlp-incident-binariesInput
Argument Name Description Required incident_id Incident ID to get binaries of. Optional include_original_message Indicates whether the Web Service should include the original message in the response document or not. Optional include_all_components Indicates whether the Web Service should include all message components (for example, headers and file attachments) in the response document or not. Optional Context Output
Path Type Description SymantecDLP.Incident.ID Number The ID of the incident. SymantecDLP.Incident.LongID Number The long ID of the incident. SymantecDLP.Incident.OriginalMessage String The original message of the incident. SymantecDLP.Component.ID Number The ID of the component SymantecDLP.Component.Name String The name of the component. SymantecDLP.Component.TypeID Number The ID of the type of the component. SymantecDLP.Component.Type String The type of the component SymantecDLP.Component.Content String The content of the component SymantecDLP.Component.LongID Number The long ID of the component. Command Example
!symantec-dlp-incident-binaries incident_id=2222Context Example
"SymantecDLP.Incident": { 'ID': 2222, 'OriginalMessage': 'msg', 'Component': [ 'ID': 69065, 'Name': 'CCN.txt', 'TypeID': 'ATTACHMENT_TEXT', 'Content': '4386280016300125', 'LongID': 69065 ], 'LongID': 2222 }Human Readable Output
Symantec DLP incident 2222 binaries
ID Original Message Long ID 2222 msg 2222 5. symantec-dlp-list-custom-attributes
Returns a list of all custom attribute names defined in the Symantec DLP deployment.
Base Command
symantec-dlp-list-custom-attributesContext Output
There are no context output for this command.Command Example
!symantec-dlp-list-custom-attributesHuman Readable Output
Symantec DLP custom attributes
Custom Attribute ca1 ca2 ca3 6. symantec-dlp-list-incident-status
Returns a list of the custom status values defined in the Symantec DLP deployment.
Base Command
symantec-dlp-list-incident-statusContext Output
There are no context output for this command.Command Example
!symantec-dlp-list-incident-statusHuman Readable Output
Symantec DLP incident status
Incident Status status1 status2 status3 7. symantec-dlp-incident-violations
Returns the highlighted matches of a specific incident.
Base Command
symantec-dlp-incident-violationsInput
Argument Name Description Required incident_id The ID of the incident. Optional include_image_violations Indicates whether image violations should be included in the Incident Violations Response. Optional Context Output
Path Type Description SymantecDLP.Incident.ID Number The ID of the incident. SymantecDLP.Incident.LongID Number The long ID of the incident. SymantecDLP.Incident.StatusCode String The status code of the incident. SymantecDLP.Incident.ViolatingComponent.Name String The name of the violationg component. SymantecDLP.Incident.ViolatingComponent.DocumentFormat String The document format of the violationg component. SymantecDLP.Incident.ViolatingComponent.Type String The type of the violationg component. SymantecDLP.Incident.ViolatingComponent.TypeID Number The type ID of the violationg component. SymantecDLP.Incident.ViolatingComponent.ViolatingCount Number Indicates the number of policy rules that were violated. SymantecDLP.Incident.ViolatingComponent.ViolatingSegment.DocumentViolation String Details about the document violation. SymantecDLP.Incident.ViolatingComponent.ViolatingSegment.FileSizeViolation Number Details about the file size violation. SymantecDLP.Incident.ViolatingComponent.ViolatingSegment.Text.Data String The data that triggered the violation. SymantecDLP.Incident.ViolatingComponent.ViolatingSegment.Text.Type String The type of data that triggered the violation. SymantecDLP.Incident.ViolatingComponent.ViolatingSegment.Text.RuleID Number The rule ID which triggered the violation. SymantecDLP.Incident.ViolatingComponent.ViolatingSegment.Text.RuleName String The rule name which triggered the violation. Command Example
!symantec-dlp-incident-violations incident_id=35364Context Example
"SymantecDLP.Incident": { 'ID': 35364, 'LongID': 35364, 'StatusCode': 'SUCCESS', 'ViolatingComponent': [ { 'Name': 'C:\\Users\\Administrator\\Desktop\\CCN.txt', 'DocumentFormat': 'ascii', 'Type': 'Attachment', 'TypeID' 3, 'ViolatingCount': 1, 'ViolatingSegment': [ { 'DocumentViolation': None, 'FileSizeViolation': None, 'Text': [ { 'Data': '4386280016300125', 'Type': 'Violation', 'RuleID': 12288, 'RuleName': 'CCN' } ] } ] } ] }Human Readable Output
Symantec DLP incident status
ID 35364
Troubleshooting
If you are encountering issues authenticating the configured API user, you can perform a test that the user you've configured has the proper role by running the following curl command:
curl -i --user YOUR_DLP_USER:YOUR_PASS https://YOUR_DLP_SERVER/ProtectManager/services/v2011/incidentsNote: you may need to add to the curl command the "-k" option if the certificate is not trusted.
If the authentication fails you will receive a response similar to:
HTTP/1.1 200 Cache-Control: no-cache, no-store, max-age=0, must-revalidate Pragma: no-cache Expires: 0 Strict-Transport-Security: max-age=31536000 ; includeSubDomains X-XSS-Protection: 1; mode=block X-Frame-Options: DENY X-Content-Type-Options: nosniff Content-Type: text/xml;charset=utf-8 Content-Length: 769 Date: Wed, 17 Jun 2020 13:12:43 GMT <?xml version='1.0' encoding='UTF-8'?><S:Envelope xmlns:S='http://schemas.xmlsoap.org/soap/envelope/'><S:Body> <S:Fault xmlns:ns4='http://www.w3.org/2003/05/soap-envelope'> <faultcode>S:Server</faultcode> <faultstring>Authentication failed</faultstring><detail> <ns4:AuthenticationFault xmlns:ns4='http://www.vontu.com/v2011/enforce/webservice/incident/schema' xmlns:ns2='http://www.vontu.com/enforce/export/incident/common/schema' xmlns:ns3='http://www.vontu.com/enforce/export/incident/schema' xmlns:ns5='http://www.vontu.com/v2011/enforce/webservice/incident/common/schema' xmlns:ns6='http://www.vontu.com/v2011/enforce/webservice/incident'> <ns4:errorMessage>Authentication failed</ns4:errorMessage> </ns4:AuthenticationFault></detail></S:Fault></S:Body></S:Envelope>
If you encounter an authentication failure, make sure to follow the instructions at the Permissions section of this document.
Having both "Kerberos" and "form" type authentication methods enabled on your Symantec DLP may cause connection issues. Disabling the form-type may assist solving such issues.
Configuration parameters
server— Enforce Server (e.g. https://192.168.0.1) (required)credentials— Username (required)insecure— Trust any certificate (not secure)proxy— Use system proxy settingsisFetch— Fetch incidentsincidentType— Incident typefetch_time— First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days)saved_report_id— Saved Report ID (Required for list incidents)fetch_limit— Fetch limit
Commands (7)
-
symantec-dlp-get-incident-detailsReturns the details of the specified incident.
-
symantec-dlp-incident-binariesRetrieves additional components of the message that generated the incident. May include binary files.
-
symantec-dlp-incident-violationsReturns the highlighted matches of a specific incident.
-
symantec-dlp-list-custom-attributesReturns a list of all custom attribute names defined in the Symantec DLP deployment.
-
symantec-dlp-list-incident-statusReturns a list of the custom status values defined in the Symantec DLP deployment.
-
symantec-dlp-list-incidentsReturns a list of incidents.
-
symantec-dlp-update-incidentUpdates the details of a specific incident.
import demistomock as demisto from CommonServerPython import * from CommonServerUserPython import * ''' IMPORTS ''' from requests import Session from zeep import Client, Settings from zeep.transports import Transport from requests.auth import AuthBase, HTTPBasicAuth from zeep import helpers from zeep.cache import SqliteCache from datetime import datetime from typing import Dict, Tuple, Any from dateutil.parser import parse import urllib3 import uuid import tempfile import os import shutil # Disable insecure warnings urllib3.disable_warnings() def get_cache_path(): path = tempfile.gettempdir() + "/zeepcache" try: os.makedirs(path) except OSError: if os.path.isdir(path): pass else: raise db_path = os.path.join(path, "cache.db") try: if not os.path.isfile(db_path): static_init_db = os.getenv('ZEEP_STATIC_CACHE_DB', '/zeep/static/cache.db') if os.path.isfile(static_init_db): demisto.debug(f'copying static init db: {static_init_db} to: {db_path}') shutil.copyfile(static_init_db, db_path) except Exception as ex: # non fatal demisto.error(f'Failed copying static init db to: {db_path}. Error: {ex}') return db_path class SymantecAuth(AuthBase): def __init__(self, user, password, host): self.basic = HTTPBasicAuth(user, password) self.host = host def __call__(self, r): if r.url.startswith(self.host): return self.basic(r) else: return r ''' HELPER FUNCTIONS ''' def get_data_owner(data_owner: Any) -> dict: """ parses the data owner object :param data_owner: the data owner object, can be of any type :return: the parsed object """ if data_owner and isinstance(data_owner, dict): return {'Name': data_owner.get('name'), 'Email': data_owner.get('email')} if data_owner and not isinstance(data_owner, dict): LOG(f"A data owner was found in the incident, but did not match the expected format.\n " f"Found: {str(data_owner)}") return {} def get_incident_binaries(client: Client, incident_id: str, include_original_message: bool = True, include_all_components: bool = True) -> Tuple[str, dict, list, dict]: """ This function get's the binaries of a specific incident with the id incident_id It generates the human readable, entry context & raw response. It also generates the binary files. :param client: The client :param incident_id: The ID of the incident :param include_original_message: Indicates whether the Web Service should include the original message in the response document. :param include_all_components: Indicates whether the Web Service should include all message components (for example, headers and file attachments) in the response document. :return: The human readable, entry context, file entries & raw response """ raw_incident_binaries = client.service.incidentBinaries( incidentId=incident_id, includeOriginalMessage=include_original_message, includeAllComponents=include_all_components, ) human_readable: str entry_context: dict = {} raw_response: dict = {} file_entries: list = [] if raw_incident_binaries: serialized_incident_binaries: dict = helpers.serialize_object(raw_incident_binaries) raw_response = json.loads(json.dumps(serialized_incident_binaries, default=bytes_to_string)) raw_components = serialized_incident_binaries.get('Component') components: list = parse_component(raw_components) # type: ignore[arg-type] incident_binaries: dict = { 'ID': serialized_incident_binaries.get('incidentId'), 'OriginalMessage': serialized_incident_binaries.get('originalMessage'), 'Component(val.ID && val.ID === obj.ID)': components, 'LongID': serialized_incident_binaries.get('incidentLongId') } raw_headers: list = ['ID', 'OriginalMessage', 'LongID'] headers: list = ['ID', 'Original Message', 'Long ID'] outputs: dict = {} for raw_header in raw_headers: outputs[headers[raw_headers.index(raw_header)]] = incident_binaries.get(raw_header) human_readable = tableToMarkdown(f'Symantec DLP incident {incident_id} binaries', outputs, headers=headers, removeNull=True) for raw_component in raw_components: # type: ignore[union-attr] filename = raw_component.get('name') data = raw_component.get('content') if isinstance(data, (str, bytes)): file_entries.append(fileResult(filename=filename, data=data)) entry_context = {'SymantecDLP.Incident(val.ID && val.ID === obj.ID)': incident_binaries} else: human_readable = 'No incident found.' return human_readable, entry_context, file_entries, raw_response def parse_text(raw_text_list: list) -> list: """ Return the parsed text list :param raw_text_list: the raw text list :return: the parsed text list """ text_list: list = [] for raw_text in raw_text_list: text: dict = { 'Data': raw_text.get('_value_1'), 'Type': raw_text.get('type'), 'RuleID': raw_text.get('ruleId'), 'RuleName': raw_text.get('ruleName') } text_list.append({key: val for key, val in text.items() if val}) return text_list def parse_violation_segment(raw_violation_segment_list: list) -> list: """ Return the parsed violation segment list :param raw_violation_segment_list: the raw violating segment list :return: the parsed violation segment list """ violation_segment_list: list = [] for raw_violation_segment in raw_violation_segment_list: violation_segment: dict = { 'DocumentViolation': raw_violation_segment.get('documentViolation'), 'FileSizeViolation': raw_violation_segment.get('fileSizeViolation'), 'Text': parse_text(raw_violation_segment.get('text', [])) } violation_segment_list.append({key: val for key, val in violation_segment.items() if val}) return violation_segment_list def parse_violating_component(raw_violating_component_list: list) -> list: """ Return the parsed violating component list :param raw_violating_component_list: the raw violating component list :return: the parsed violating component list """ violating_component_list: list = [] for raw_violating_component in raw_violating_component_list: violating_component_type: dict = raw_violating_component.get('violatingComponentType', {}) violating_component: dict = { 'Name': raw_violating_component.get('name'), 'DocumentFormat': raw_violating_component.get('documentFormat'), 'Type': violating_component_type.get('_value_1'), 'TypeID': violating_component_type.get('id'), 'ViolatingCount': raw_violating_component.get('violationCount'), 'ViolationSegment': parse_violation_segment(raw_violating_component.get('violatingSegment', [])) } violating_component_list.append({key: val for key, val in violating_component.items() if val}) return violating_component_list def parse_violated_policy_rule(raw_violated_policy_rule_list: list) -> list: """ Parses a list of rules to context paths :param raw_violated_policy_rule_list: the raw rules list :return: the parsed rules list """ violated_policy_rule_list: list = [] for raw_violated_policy_rule in raw_violated_policy_rule_list: violated_policy_rule: dict = { 'Name': raw_violated_policy_rule.get('ruleName'), 'ID': raw_violated_policy_rule.get('ID') } violated_policy_rule_list.append({key: val for key, val in violated_policy_rule.items() if val}) return violated_policy_rule_list def parse_other_violated_policy(raw_other_violated_policy_list: list) -> list: """ Parses a list of policies to context paths :param raw_other_violated_policy_list: the raw policies list :return: the parsed policies list """ other_violated_policy_list: list = [] for raw_other_violated_policy in raw_other_violated_policy_list: other_violated_policy: dict = { 'Name': raw_other_violated_policy.get('name'), 'Version': raw_other_violated_policy.get('version'), 'Label': raw_other_violated_policy.get('label'), 'ID': raw_other_violated_policy.get('policyId') } other_violated_policy_list.append({key: val for key, val in other_violated_policy.items() if val}) return other_violated_policy_list def get_all_group_custom_attributes(group: dict) -> list: """ Returns a list of all the custom attributes in the group :param group: the group :return: the list of all custom attributes """ custom_attributes_list: list = [] for raw_custom_attribute in group.get('customAttribute', []): custom_attribute: dict = {'Name': raw_custom_attribute.get('name')} custom_attribute_value = raw_custom_attribute.get('value') if custom_attribute_value: custom_attribute['Value'] = custom_attribute_value custom_attributes_list.append(custom_attribute) return custom_attributes_list def parse_custom_attribute(custom_attribute_group_list: list, args: dict) -> list: """ Returns a list of all custom attributes chosen by the user. There are four options to choose from: all, none, specific attributes, custom attributes group name. The choosing flag is given in demisto.args value in the field custom_attributes. If the user has chosen "all" then the function will return all custom attributes possible (from all groups). If the user has chosen "none" then the function won't return any custom attributes. If the user has chosen "specific attributes" then he must also provide a list of all custom attribute names in the demisto.args dict under the field "custom_data". If not provided, an error msg will be shown. If provided, the function will return only the custom attributes mentioned in the custom_data list. If the user has chosen "custom attributes group name" the handling of this option is similar to the "custom" option. :param custom_attribute_group_list: the raw list of custom attributes group (as returned from the request) :param args: demisto.args :return: the parsed custom attributes list """ custom_attributes_flag = args.get('custom_attributes') custom_attributes_list: list = [] # all case if custom_attributes_flag == 'all': for group in custom_attribute_group_list: custom_attributes_list.extend(get_all_group_custom_attributes(group)) # custom attributes group name case elif custom_attributes_flag == 'custom attributes group name': custom_data = args.get('custom_data') if not custom_data: raise DemistoException('When choosing the group value for custom_attributes argument - the custom_data' ' list must be filled with group names. For example: custom_value=g1,g2,g3') group_name_list: list = argToList(custom_data, ',') for group in custom_attribute_group_list: if group.get('name') in group_name_list: custom_attributes_list.extend(get_all_group_custom_attributes(group)) # specific attributes case elif custom_attributes_flag == 'specific attributes': custom_data = args.get('custom_data') if not custom_data: raise DemistoException('When choosing the custom value for custom_attributes argument - the custom_data' ' list must be filled with custom attribute names.' ' For example: custom_value=ca1,ca2,ca3') custom_attribute_name_list: list = argToList(custom_data, ',') for group in custom_attribute_group_list: for raw_custom_attribute in group.get('customAttribute', []): custom_attribute_name: str = raw_custom_attribute.get('name') if custom_attribute_name in custom_attribute_name_list: custom_attribute: dict = {'Name': custom_attribute_name} custom_attribute_value = raw_custom_attribute.get('value') if custom_attribute_value: custom_attribute['Value'] = custom_attribute_value custom_attributes_list.append(custom_attribute) # none case - If custom_attributes_flag == 'none' than we return empty list return custom_attributes_list def get_incident_details(raw_incident_details: dict, args: dict) -> dict: """ Parses the needed incident details into context paths :param raw_incident_details: the raw response of the incident details :param args: demisto.args :return: the parsed dict """ incident: dict = raw_incident_details.get('incident', {}) message_source: dict = incident.get('messageSource', {}) message_type: dict = incident.get('messageType', {}) policy: dict = incident.get('policy', {}) incident_details: dict = { 'ID': raw_incident_details.get('incidentID'), 'LongID': raw_incident_details.get('incidentLongId'), 'StatusCode': raw_incident_details.get('statusCode'), 'CreationDate': incident.get('incidentCreationDate'), 'DetectionDate': incident.get('detectionDate'), 'Severity': incident.get('severity'), 'Status': incident.get('status'), 'MessageSource': message_source.get('_value_1'), 'MessageSourceType': message_source.get('sourceType'), 'MessageType': message_type.get('_value_1'), 'MessageTypeID': message_type.get('typeId'), 'Policy(val.ID && val.ID === obj.ID)': { 'Name': policy.get('name'), 'Version': policy.get('version'), 'Label': policy.get('label'), 'ID': policy.get('policyId') }, 'ViolatedPolicyRule(val.ID && val.ID === obj.ID)': parse_violated_policy_rule(incident.get('violatedPolicyRule', [])), 'OtherViolatedPolicy(val.ID && val.ID === obj.ID)': parse_other_violated_policy(incident.get('otherViolatedPolicy', [])), 'BlockedStatus': incident.get('blockedStatus'), 'MatchCount': incident.get('matchCount'), 'RuleViolationCount': incident.get('ruleViolationCount'), 'DetectionServer': incident.get('detectionServer'), 'CustomAttribute': parse_custom_attribute(incident.get('customAttributeGroup', []), args), 'DataOwner': get_data_owner(incident.get('dataOwner', {})), 'EventDate': incident.get('eventDate') } return {key: val for key, val in incident_details.items() if val} def get_incident_attributes(attributes: dict) -> dict: """ Transforms the demisto args entered by the user into a dict representing the attributes of the updated incidents :param attributes: the demisto args dict :return: the attributes dict by the API design """ # Verify Custom Attribute custom_attribute: dict = {} custom_attribute_name: str = attributes.get('custom_attribute_name', '') custom_attribute_value: str = attributes.get('custom_attribute_value', '') if custom_attribute_name and not custom_attribute_value or custom_attribute_value and not custom_attribute_name: raise DemistoException("If updating an incident's custom attribute," " both custom_attribute_name and custom_attribute_value must be provided.") elif custom_attribute_name and custom_attribute_value: custom_attribute['value'] = custom_attribute_value custom_attribute['name'] = custom_attribute_name # Verify Data Owner data_owner: dict = {} data_owner_name: str = attributes.get('data_owner_name', '') data_owner_email: str = attributes.get('data_owner_email', '') if data_owner_name and not data_owner_email or data_owner_email and not data_owner_name: raise DemistoException("If updating an incident's data owner," " both data_owner_name and data_owner_email must be provided.") elif data_owner_name and data_owner_email: data_owner['name'] = data_owner_name data_owner['email'] = data_owner_email # Verify Note note: dict = {} note_str: str = attributes.get('note', '') note_time_str: str = attributes.get('note_time', '') note_time = None if note_time_str: note_time = parse(note_time_str) if note_str and not note_time or note_time and not note_str: raise DemistoException("If adding an incident's note, both note and note_time must be provided.") elif note_str and note_time: note['note'] = note_str note['dateAndTime'] = note_time attributes: dict = { 'severity': attributes.get('severity'), 'status': attributes.get('status'), 'note': note, 'customAttribute': custom_attribute, 'dataOwner': data_owner, 'remediationStatus': attributes.get('remediation_status'), 'remediationLocation': attributes.get('remediation_location') } return {key: val for key, val in attributes.items() if val} def parse_component(raw_components: list) -> list: """ Parses a list of components into a list of context data :param raw_components: the components list before parsing :return: the parsed list """ components: list = [] for raw_component in raw_components: unfiltered_component: dict = { 'ID': raw_component.get('componentId'), 'Name': raw_component.get('name'), 'TypeID': raw_component.get('componentTypeId'), 'Type': raw_component.get('componentType'), 'Content': bytes_to_string(raw_component.get('content')), 'LongID': raw_component.get('componentLongId') } component: dict = {key: val for key, val in unfiltered_component.items() if val} if component: components.append(component) return components def datetime_to_iso_format(obj: Any): """ Converts a datetime object into an ISO string representation :param obj: Any type of object :return: If the object is of type datetime the return is it's ISO string representation """ if isinstance(obj, datetime): return obj.isoformat() def bytes_to_string(obj: Any): """ Converts a bytes object into a string :param obj: Any type of object :return: If the object is of type bytes it returns it's string representation, else returns the object itself """ if isinstance(obj, bytes): return obj.decode('utf-8') else: return obj ''' COMMANDS + REQUESTS FUNCTIONS ''' def test_module(client: Client, saved_report_id: int): """ Performs basic get request to get item samples """ helpers.serialize_object(client.service.incidentList( savedReportId=saved_report_id, incidentCreationDateLaterThan=parse_date_range('1 year')[0] )) demisto.results('ok') def get_incident_details_command(client: Client, args: dict) -> Tuple[str, dict, dict]: incident_id: str = args.get('incident_id', '') raw_incident: list = client.service.incidentDetail( incidentId=incident_id, includeHistory=True, includeViolations=True ) human_readable: str entry_context: dict = {} raw_response: dict = {} if raw_incident and isinstance(raw_incident, list): serialized_incident = helpers.serialize_object(raw_incident[0]) raw_response = json.loads(json.dumps(serialized_incident, default=datetime_to_iso_format)) incident_details: dict = get_incident_details(raw_response, args) raw_headers = ['ID', 'CreationDate', 'DetectionDate', 'Severity', 'Status', 'MessageSourceType', 'MessageType', 'Policy Name'] headers = ['ID', 'Creation Date', 'Detection Date', 'Severity', 'Status', 'DLP Module', 'DLP Module subtype', 'Policy Name'] outputs: dict = {} for raw_header in raw_headers: if raw_header == 'Policy Name': outputs['Policy Name'] = incident_details.get('Policy', {}).get('Name') else: outputs[headers[raw_headers.index(raw_header)]] = incident_details.get(raw_header) human_readable = tableToMarkdown(f'Symantec DLP incident {incident_id} details', outputs, headers=headers, removeNull=True) entry_context = {'SymantecDLP.Incident(val.ID && val.ID === obj.ID)': incident_details} else: human_readable = 'No incident found.' return human_readable, entry_context, raw_response def list_incidents_command(client: Client, args: dict, saved_report_id: str) -> Tuple[str, dict, dict]: if not saved_report_id: raise ValueError('Missing saved report ID. Configure it in the integration instance settings.') creation_date = parse_date_range(args.get('creation_date', '1 day'))[0] raw_incidents = client.service.incidentList( savedReportId=saved_report_id, incidentCreationDateLaterThan=creation_date ) human_readable: str entry_context: dict = {} raw_response: dict = {} if raw_incidents: serialized_incidents: dict = helpers.serialize_object(raw_incidents) incidents_ids_list = serialized_incidents.get('incidentId') if incidents_ids_list: raw_response = serialized_incidents incidents = [{'ID': str(incident_id)} for incident_id in incidents_ids_list] human_readable = tableToMarkdown('Symantec DLP incidents', incidents, removeNull=True) entry_context = {'SymantecDLP.Incident(val.ID && val.ID == obj.ID)': incidents} else: human_readable = 'No incidents found.' else: human_readable = 'No incidents found.' return human_readable, entry_context, raw_response def update_incident_command(client: Client, args: dict) -> Tuple[str, dict, dict]: incident_id: str = args.get('incident_id', '') incident_attributes: dict = get_incident_attributes(args) raw_incidents_update_response = client.service.updateIncidents( updateBatch={ 'batchId': '_' + str(uuid.uuid1()), 'incidentId': incident_id, 'incidentAttributes': incident_attributes } ) human_readable: str entry_context: dict = {} raw_response: dict = {} if raw_incidents_update_response and isinstance(raw_incidents_update_response, list): incidents_update_response = helpers.serialize_object(raw_incidents_update_response[0]) headers: list = ['Batch ID', 'Inaccessible Incident Long ID', 'Inaccessible Incident ID', 'Status Code'] outputs = { 'Batch ID': incidents_update_response.get('batchId'), 'Inaccessible Incident Long ID': incidents_update_response.get('InaccessibleIncidentLongId'), 'Inaccessible Incident ID': incidents_update_response.get('InaccessibleIncidentId'), 'Status Code': incidents_update_response.get('statusCode') } if outputs.get('Status Code') == 'VALIDATION_ERROR': raise DemistoException('Update was not successful. ADVICE: If status or custom attribute were changed,' ' check that they are configured in Symantec DLP.') human_readable = tableToMarkdown(f'Symantec DLP incidents {incident_id} update', outputs, headers=headers, removeNull=True) else: human_readable = 'Update was not successful' return human_readable, entry_context, raw_response def incident_binaries_command(client: Client, args: dict) -> Tuple[str, dict, list, dict]: incident_id: str = args.get('incident_id', '') include_original_message: bool = bool(args.get('include_original_message', 'True')) include_all_components: bool = bool(args.get('include_all_components', 'True')) human_readable, entry_context, file_entries, raw_response = get_incident_binaries(client, incident_id, include_original_message, include_all_components) return human_readable, entry_context, file_entries, raw_response def list_custom_attributes_command(client: Client) -> Tuple[str, dict, dict]: raw_custom_attributes_list = client.service.listCustomAttributes() human_readable: str entry_context: dict = {} raw_response: dict = {} if raw_custom_attributes_list: custom_attributes_list = helpers.serialize_object(raw_custom_attributes_list) raw_response = custom_attributes_list outputs: list = [{'Custom Attribute': custom_attribute} for custom_attribute in custom_attributes_list] human_readable = tableToMarkdown('Symantec DLP custom attributes', outputs, removeNull=True) else: human_readable = 'No custom attributes found.' return human_readable, entry_context, raw_response def list_incident_status_command(client: Client) -> Tuple[str, dict, dict]: raw_incident_status_list = client.service.listIncidentStatus() human_readable: str entry_context: dict = {} raw_response: dict = {} if raw_incident_status_list: incident_status_list = helpers.serialize_object(raw_incident_status_list) raw_response = incident_status_list outputs: list = [{'Incident Status': incident_status} for incident_status in incident_status_list] human_readable = tableToMarkdown('Symantec DLP incident status', outputs, removeNull=True) else: human_readable = 'No incident status found.' return human_readable, entry_context, raw_response def incident_violations_command(client: Client, args: dict) -> Tuple[str, dict, dict]: incident_id: str = args.get('incident_id', '') include_image_violations: bool = bool(args.get('include_image_violations', 'True')) raw_incident_violations = client.service.incidentViolations( incidentId=incident_id, includeImageViolations=include_image_violations ) human_readable: str entry_context: dict = {} raw_response: dict = {} if raw_incident_violations: raw_incident_violations = helpers.serialize_object(raw_incident_violations[0]) raw_response = raw_incident_violations incident_violations: dict = { 'ID': raw_incident_violations.get('incidentId'), 'LongID': raw_incident_violations.get('incidentLongId'), 'StatusCode': raw_incident_violations.get('statusCode'), 'ViolatingComponent': parse_violating_component(raw_incident_violations.get('violatingComponent', [])) } human_readable = tableToMarkdown(f'Symantec DLP incident {incident_id} violations', {'ID': incident_violations.get('ID')}, removeNull=True) entry_context = {'SymantecDLP.Incident(val.ID && val.ID === obj.ID)': incident_violations} else: human_readable = 'No incident status found.' return human_readable, entry_context, raw_response def fetch_incidents(client: Client, fetch_time: str, fetch_limit: int, last_run: dict, saved_report_id: str): """ Performs the fetch incidents functionality of Demisto, which means that every minute if fetches incidents from Symantec DLP and uploads them to Demisto server. :param client: Demisto Client :param fetch_time: For the first time the integration is enabled with the fetch incidents functionality, the fetch time indicates from what time to start fetching existing incidents in Symantec DLP system. :param fetch_limit: Indicates how many incidents to fetch every minute :param last_run: Demisto last run object :param saved_report_id: The report ID to retrieve the incidents from :return: A list of Demisto incidents """ # We use parse to get out time in datetime format and not iso, that's what Symantec DLP is expecting to get last_id_fetched = last_run.get('last_incident_id') if last_run and last_run.get('last_fetched_event_iso'): last_update_time = parse(last_run['last_fetched_event_iso']) else: last_update_time = parse_date_range(fetch_time)[0] incidents = [] incidents_ids = helpers.serialize_object(client.service.incidentList( savedReportId=saved_report_id, incidentCreationDateLaterThan=last_update_time )).get('incidentId', '') if incidents_ids: incidents_ids = incidents_ids[:fetch_limit] last_incident_time: str = '' last_incident_id: str = '' for incident_id in incidents_ids: if last_id_fetched and last_id_fetched == incident_id: # Skipping last incident from last cycle if fetched again continue incident_details = json.dumps(helpers.serialize_object(client.service.incidentDetail( incidentId=incident_id )[0]), default=datetime_to_iso_format) incident_creation_time = json.loads(incident_details).get('incident', {}).get('incidentCreationDate') incident: dict = { 'rawJSON': incident_details, 'name': f'Symantec DLP incident {incident_id}', 'occurred': incident_creation_time } _, _, file_entries, _ = get_incident_binaries(client, incident_id, False, False) if file_entries: attachments: list = [] for file_entry in file_entries: attachments.append({ 'path': file_entry['FileID'], 'name': file_entry['File'] }) incident['attachment'] = attachments incidents.append(incident) if incident_id == incidents_ids[-1]: last_incident_time = incident_creation_time last_incident_id = incident_id demisto.setLastRun( { 'last_fetched_event_iso': last_incident_time, 'last_incident_id': last_incident_id } ) demisto.incidents(incidents) ''' COMMANDS MANAGER / SWITCH PANEL ''' def main(): handle_proxy() params: Dict = demisto.params() server: str = params.get('server', '').rstrip('/') credentials: Dict = params.get('credentials', {}) username: str = credentials.get('identifier', '') password: str = credentials.get('password', '') fetch_time: str = params.get('fetch_time', '3 days').strip() try: fetch_limit: int = int(params.get('fetch_limit', '10')) except ValueError: raise DemistoException('Value for fetch_limit must be an integer.') saved_report_id: str = demisto.params().get('saved_report_id', '') last_run: dict = demisto.getLastRun() args: dict = demisto.args() verify_ssl = not params.get('insecure', False) wsdl: str = f'{server}/ProtectManager/services/v2011/incidents?wsdl' session: Session = Session() session.auth = SymantecAuth(username, password, server) session.verify = verify_ssl cache: SqliteCache = SqliteCache(path=get_cache_path(), timeout=None) transport: Transport = Transport(session=session, cache=cache) settings: Settings = Settings(strict=False, xsd_ignore_sequence_order=True) client: Client = Client(wsdl=wsdl, transport=transport, settings=settings) command = demisto.command() demisto.info(f'Command being called is {command}') commands = { 'test-module': test_module, 'fetch-incidents': fetch_incidents, 'symantec-dlp-get-incident-details': get_incident_details_command, 'symantec-dlp-list-incidents': list_incidents_command, 'symantec-dlp-update-incident': update_incident_command, 'symantec-dlp-incident-binaries': incident_binaries_command, 'symantec-dlp-list-custom-attributes': list_custom_attributes_command, 'symantec-dlp-list-incident-status': list_incident_status_command, 'symantec-dlp-incident-violations': incident_violations_command } try: if command == 'fetch-incidents': fetch_incidents(client, fetch_time, fetch_limit, last_run, saved_report_id) # type: ignore[operator] elif command == 'test-module': test_module(client, saved_report_id) # type: ignore[arg-type] elif command == 'symantec-dlp-list-incidents': human_readable, context, raw_response =\ commands[command](client, args, saved_report_id) # type: ignore[operator] return_outputs(human_readable, context, raw_response) elif command == 'symantec-dlp-list-incident-status' or command == 'symantec-dlp-list-custom-attributes': human_readable, context, raw_response = commands[command](client) # type: ignore[operator] return_outputs(human_readable, context, raw_response) elif command == 'symantec-dlp-incident-binaries': human_readable, context, file_entries, raw_response =\ commands[command](client, args) # type: ignore[operator] return_outputs(human_readable, context, raw_response) for file_entry in file_entries: demisto.results(file_entry) elif command in commands: human_readable, context, raw_response = commands[command](client, args) # type: ignore[operator] return_outputs(human_readable, context, raw_response) # Log exceptions except Exception as e: err_msg = f'Error in Symantec DLP integration: {str(e)}' if demisto.command() == 'fetch-incidents': LOG(err_msg) LOG.print_log() raise else: return_error(err_msg, error=e) if __name__ == 'builtins': main()