Palo Alto Networks Enterprise DLP
Palo Alto Networks Enterprise DLP discovers and protects company data across every data channel and repository. Integrated Enterprise DLP enables data protection and compliance everywhere without complexity.
Network Security · Enterprise DLP by Palo Alto Networks
Details
| ID | Palo Alto Networks Enterprise DLP |
|---|---|
| Provider | Palo Alto Networks |
| Category | Network Security |
| From Version | 5.0.0 |
| Docker Image | demisto/python3:3.12.13.10404775 |
| Supported Modules | Agentix XSIAM EDR Cloud Runtime Security |
README
Palo Alto Networks Enterprise DLP discovers and protects company data across every data channel and repository. Integrated Enterprise DLP enables data protection and compliance everywhere without complexity.
This integration was integrated and tested with version 2.0 of Palo Alto Networks Enterprise DLP.
Note:
This integration currently supports fetching DLP incidents from “NGFW” and “Prisma Access” channels only.
Setup
Go to the Settings tab on the DLP web interface.
Choose Alerts on the left menu. Follow all the steps under Setup Instructions.
Make sure the toggle at the bottom is switched on.
Configure Palo Alto Networks Enterprise DLP in Cortex
| Parameter | Description | Required |
|---|---|---|
| Server URL | Default value is https://api.dlp.paloaltonetworks.com/v1/ | False |
| Authentication URL | Default value is https://auth.apps.paloaltonetworks.com/auth/v1/oauth2/access_token | False |
| Access Token | Access token generated in the Enterprise DLP UI | True |
| Refresh Token | Refresh token generated in the Enterprise DLP UI | True |
| Trust any certificate (not secure) | False | |
| Use system proxy settings | False | |
| Fetch incidents | False | |
| Maximum number of incidents per fetch | Default value is 50. | False |
| First fetch timestamp | First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days). Default value is 60 minutes. | False |
| Fetch Lookback Window (minutes) | The number of minutes to look back during each fetch to capture late-indexed incidents. Default value is 0. | False |
| DLP Regions | Possible values: US (United States), EU (European Union), AP (Asia-Pacific), UK (United Kingdom), CA (Canada), AU (Australia), IN (India), JP (Japan), BR (Brazil), PAR (Paris), SUI (Switzerland). |
False |
| Data profiles to allow exemption | A comma-separated list of data profile names to request an exemption. Use “*” to allow everything. | False |
| Bot Message | The message to send to the user to ask for feedback. | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
pan-dlp-get-report
Fetches DLP reports associated with a report ID.
Base Command
pan-dlp-get-report
Input
| Argument Name | Description | Required |
|---|---|---|
| report_id | DLP report ID. | Required |
| fetch_snippets | If True, includes snippets with the reports. Possible values are: true, false. Default is false. | Optional |
| service_name | The DLP service that the report belongs to. Determines which backend the report is retrieved from. When empty, the request does not specify a service and the server retrieves the report from Prisma Access. Possible values are: ngfw, prisma-access, prisma-saas, prisma-access-browser, endpoint-dlp. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DLP.Report.DataProfile | unknown | The data profile name. |
| DLP.Report.DataPatternMatches.DataPatternName | unknown | The DLP data pattern name. |
| DLP.Report.DataPatternMatches.Detections | unknown | The DLP detection snippets. |
| DLP.Report.DataPatternMatches.HighConfidenceFrequency | unknown | The number of high confidence occurrences. |
| DLP.Report.DataPatternMatches.MediumConfidenceFrequency | unknown | The number of medium confidence occurrences. |
| DLP.Report.DataPatternMatches.LowConfidenceFrequency | unknown | The number of low confidence occurrences. |
| DLP.Report.DataPatternMatches.MatchedConfidenceLevel | String | The matched confidence level of the data pattern (e.g., “high”, “medium”, “low”). Only present for patterns that matched. |
| DLP.Report.DataProfiles.Name | String | The name of the data profile. |
| DLP.Report.DataProfiles.Id | Number | The ID of the data profile. |
| DLP.Report.DataProfiles.Version | Number | The version of the data profile. |
| DLP.Report.DataProfiles.IsTriggered | Boolean | Whether the data profile was triggered. |
| DLP.Report.DataProfiles.DataPatterns.Id | String | The data pattern ID within the profile. |
| DLP.Report.DataProfiles.DataPatterns.IsMatched | Boolean | Whether the data pattern matched. |
| DLP.Report.DataProfiles.DataPatterns.ConfidenceLevel | String | The confidence level configured for the pattern. |
| DLP.Report.DataProfiles.DataPatterns.OccurrenceCount | Number | The number of occurrences detected. |
| DLP.Report.DataProfiles.DataPatterns.OccurrenceOperatorType | String | The occurrence operator type (e.g., “more_than_equal_to”, “between”). |
| DLP.Report.DataProfiles.DataPatterns.OccurrenceLow | Number | The low bound for “between” operator type. |
| DLP.Report.DataProfiles.DataPatterns.OccurrenceHigh | Number | The high bound for “between” operator type. |
Command example
!pan-dlp-get-report report_id=3165792284 service_name=prisma-saas
Human Readable Output
DLP Report for profile: Sample-Data-Profile
DataPatternName ConfidenceFrequency MatchedConfidenceLevel National Id - US Social Security Number - SSN Low: 30
Medium: 0
High: 30high Credit Card Number Low: 30
Medium: 30
High: 30high
pan-dlp-update-incident
Updates a DLP incident with user feedback.
Base Command
pan-dlp-update-incident
Input
| Argument Name | Description | Required |
|---|---|---|
| incident_id | The ID of the incident to update. | Required |
| feedback | The user feedback. Possible values are: PENDING_RESPONSE, CONFIRMED_SENSITIVE, CONFIRMED_FALSE_POSITIVE, EXCEPTION_REQUESTED, EXCEPTION_GRANTED, EXCEPTION_NOT_REQUESTED, OPERATIONAL_ERROR, SEND_NOTIFICATION_FAILURE, EXCEPTION_DENIED. | Required |
| user_id | The ID of the user the feedback is collected from. | Required |
| region | The region where the incident originated. | Optional |
| report_id | The DLP report ID, needed only for granting exemptions. | Optional |
| dlp_channel | The DLP channel, needed only for granting exemptions. | Optional |
| error_details | Error details if status is SEND_NOTIFICATION_FAILURE. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| DLP.IncidentUpdate.success | boolean | Whether the update was successful. |
| DLP.IncidentUpdate.exemption_duration | number | The exemption duration, only available for “EXCEPTION_GRANTED”. |
pan-dlp-exemption-eligible
Determines whether exemption can be granted on incidents from a certain data profile.
Base Command
pan-dlp-exemption-eligible
Input
| Argument Name | Description | Required |
|---|---|---|
| data_profile | The name of the data profile. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| DLP.exemption.eligible | boolean | Whether the data profile is eligible for exemption. |
pan-dlp-slack-message
Gets the Slack bot message to send to the user for gathering feedback.
Base Command
pan-dlp-slack-message
Input
| Argument Name | Description | Required |
|---|---|---|
| user | The name of the user that receives this message. | Required |
| file_name | The name of the file that triggered the incident. | Required |
| data_profile_name | The data profile name associated with the incident. | Required |
| snippets | The snippets of the violation. | Optional |
| app_name | The name of the application that performed the activity. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| DLP.slack_message | string | The Slack bot message. |
pan-dlp-reset-last-run
Deprecated. Reset the “last run” timestamp via the integration instance configuration window.
Base Command
pan-dlp-reset-last-run
Input
There are no input arguments for this command.
Context Output
There is no context output for this command.
Troubleshooting
In case specific DLP incidents are not appearing on the Cortex tenant, verify the following:
- DLP Regions Configuration
- Check the Strata Cloud Manager to confirm which regions generated the incidents.
- Note: The DLP Regions dropdown menu shows all currently-supported regions.
- Ensure all regions where incidents originated are selected from the dropdown menu.
- Strata Cloud Manager (SCM) Channel Support
- Verify the channel in SCM console under the incident details.
- Note: Only incidents from the “NGFW” and “Prisma Access” SCM channels are supported.
- Incidents from other channels, such as Endpoint DLP, will not be fetched by this integration.
Configuration parameters
base_url— Server URLauth_url— Authentication URLcredentials— Access Token (required)insecure— Trust any certificate (not secure)proxy— Use system proxy settingsisFetch— Fetch incidentsincidentType— Incident typeincidentFetchInterval— Incidents Fetch Intervalfirst_fetch— First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days)max_fetch— Maximum number of incidents per fetchlook_back— Fetch Lookback Window (minutes)dlp_regions— DLP Regionsdlp_exemptible_list— Data profiles to allow exemptiondlp_slack_message— Bot Message
Commands (5)
-
pan-dlp-exemption-eligibleDetermines whether exemption can be granted on incidents from a certain data profile.
-
pan-dlp-get-reportFetches DLP reports associated with a report ID.
-
pan-dlp-reset-last-runDeprecatedDeprecated. Reset the "last run" timestamp via the integration instance configuration window.
-
pan-dlp-slack-messageGets the Slack bot message to send to the user for gathering feedback.
-
pan-dlp-update-incidentUpdates a DLP incident with user feedback.
import json from datetime import UTC import demistomock as demisto import pytest from freezegun import freeze_time from Palo_Alto_Networks_Enterprise_DLP import ( DEFAULT_BASE_URL as DLP_URL, DEFAULT_AUTH_URL as AUTH_URL, Client, exemption_eligible_command, fetch_notifications, main, parse_dlp_report, parse_incident_details, slack_bot_message_command, update_incident_command, create_incident, arg_to_datetime, compute_next_run, get_start_end_time_intervals, _migrate_last_run, START_TIMESTAMP_KEY, LAST_IDS_KEY, LAST_IDS_TIMESTAMPS_KEY, END_TIME_BUFFER, ) REPORT_DATA = { "txn_id": "2573778324", "report_id": "2573778324", "data_profile_id": "11995149", "data_profile_version": 1, "data_profile_name": "Credit Card Match 2", "type": "advanced", "tenant_id": "1128505801991063552", "fileSha": "9093980f84a22659207d6a7194fc10e22416c833044a4d23f292b3a666ee66d9", "file_name": "Test_file.txt", "file_type": "txt", "file_size_in_bytes": 7640, "extracted_file_size_in_bytes": 7649, "detection_time": "04/01/2022 20:21:50 UTC", "action": "block", "data_pattern_rule_1_verdict": "MATCHED", "data_pattern_rule_2_verdict": None, "scanContentRawReport": { "data_pattern_rule_1_results": [ { "data_pattern_id": "617b1867469e8924c80baeac", "version": 1, "name": "Credit Card Number", "technique": "regex", "type": "predefined", "strict_detection_frequency": 2, "proximity_detection_frequency": 10, "detection_frequency": 42, "unique_strict_detection_frequency": 1, "unique_checksum_detection_frequency": 0, "unique_proximity_detection_frequency": 5, "unique_detection_frequency": 7, "weighted_frequency": 0, "score": 0.0, "high_confidence_frequency": 10, "medium_confidence_frequency": 42, "low_confidence_frequency": 42, "unique_high_confidence_frequency": 5, "unique_medium_confidence_frequency": 7, "unique_low_confidence_frequency": 7, "matched_confidence_level": "low", "state": "EVALUATED", "detections": [ { "left": "mastercard ************4444 \r\n************1881\r\n*********2222\r\n***********0005\r\n", "right": "Cyprus CY17 0020 0128 0000 0012 0052 7600\r\nEs", "detection": "************1117", "origOffSet": 1484, "textLength": 0, } ], } ], "data_pattern_rule_2_results": None, "mlResponse": { "sha_256_original": None, "sha_256_extracted": None, "tenant_id": None, "report_id": None, "features": None, }, }, "data_profiles": [ { "name": "Test Profile", "id": 12345, "version": 1, "is_triggered": True, "data_patterns": [ { "id": "pattern_id_1", "is_matched": True, "confidence_level": "high", "occurrence_count": 5, "occurrence_operator_type": "more_than_equal_to", "occurrence_low": 1, }, { "id": "pattern_id_2", "confidence_level": "low", "occurrence_operator_type": "between", "occurrence_low": 1, "occurrence_high": 10, }, ], } ], } INCIDENT_JSON = { "incidentId": "1fd24b1e-05ff-46c1-b638-a79d284dc727", "userId": None, "tenantId": "1128505801991063552", "reportId": "2573778324", "dataProfileId": 11995149, "dataProfileVersion": 1, "action": "block", "channel": "ngfw", "filename": "Test_file.txt", "checksum": "9093980f84a22659207d6a7194fc10e22416c833044a4d23f292b3a666ee66d9", "source": "ngfw", "scanDate": "2022-Apr-01 20:21:50 UTC", "createdAt": "2022-Apr-01 20:21:50 UTC", "incidentDetails": "QlpoOTFBWSZTWVnl2RYAAKIfgFAFfBBEAoAKv+ffqjAA2CIpoZGjEDTIZBpgGGRpppkYTIwTQGBiSp/pTZGqe1T8qMQaaeo9Nqm3YdNAidgNoZcFEJmTIP+V1xQohhqNsWERYRnKAc3TlogFoteml94kUR+lVJzjB9uhEqOgfBMrQh34ox8qYCCQo2n9WoNceFBvtSCAfMeY7sIAvtXhGQZ7UToozWEQwedzu/MRtoFMK8+ucpSbK4O7zRnPU82E9etuWR5AtmDQF5muuAczVDMFREJd+AEsRAKqdBdyRThQkFnl2RY=", # noqa: E501 } CREDENTIALS = { "credential": "", "credentials": { "id": "", "locked": False, "modified": "0001-01-01T00:00:00Z", "name": "", "password": "", "sortValues": None, "sshkey": "", "sshkeyPass": "", "user": "", "vaultInstanceId": "", "version": 0, "workgroup": "", }, "identifier": "", "password": "", "passwordChanged": False, } def test_update_incident(requests_mock, mocker): incident_id = "abcdefg12345" user_id = "someone@somewhere.com" args = { "incident_id": incident_id, "feedback": "CONFIRMED_SENSITIVE", "user_id": user_id, "region": "us", "report_id": "A12345", "dlp_channel": "ngfw", } requests_mock.post(f"{DLP_URL}public/incident-feedback/{incident_id}?feedback_type=CONFIRMED_SENSITIVE®ion=us") client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) mocker.patch.object(demisto, "results") results = update_incident_command(client, args).to_context() request = requests_mock.last_request assert results["Contents"] == {"feedback": "CONFIRMED_SENSITIVE", "success": True} assert request.text == json.dumps({"user_id": user_id, "report_id": "A12345", "service_name": "ngfw"}) def test_update_incident_with_error_details(requests_mock, mocker): incident_id = "abcdefg12345" user_id = "someone@somewhere.com" args = { "incident_id": incident_id, "feedback": "SEND_NOTIFICATION_FAILURE", "user_id": user_id, "region": "us", "report_id": "A12345", "dlp_channel": "ngfw", "error_details": "Something went wrong", } requests_mock.post(f"{DLP_URL}public/incident-feedback/{incident_id}?feedback_type=SEND_NOTIFICATION_FAILURE®ion=us") client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) mocker.patch.object(demisto, "results") results = update_incident_command(client, args).to_context() request = requests_mock.last_request assert results["Contents"] == {"feedback": "SEND_NOTIFICATION_FAILURE", "success": True} assert request.text == json.dumps( {"user_id": user_id, "report_id": "A12345", "service_name": "ngfw", "error_details": "Something went wrong"} ) def test_get_dlp_report(requests_mock, mocker): report_id = 12345 requests_mock.get(f"{DLP_URL}public/report/{report_id}?fetchSnippets=true", json={"id": "test"}) mocker.patch.object(demisto, "command", return_value="pan-dlp-get-report") args = {"report_id": report_id, "fetch_snippets": "true"} params = {"credentials": CREDENTIALS} mocker.patch.object(demisto, "args", return_value=args) mocker.patch.object(demisto, "params", return_value=params) mocker.patch.object(demisto, "results") main() results = demisto.results.call_args_list[0][0] assert results[0]["Contents"] == {"id": "test"} def test_get_dlp_report_sends_service_name_header(requests_mock, mocker): """ Given: - A service_name argument. When: - Running the pan-dlp-get-report command. Then: - Ensure the service-name header is sent, so the report is retrieved from the requested service rather than the prisma-access default. """ report_id = 12345 requests_mock.get(f"{DLP_URL}public/report/{report_id}?fetchSnippets=true", json={"id": "test"}) mocker.patch.object(demisto, "command", return_value="pan-dlp-get-report") args = {"report_id": report_id, "fetch_snippets": "true", "service_name": "prisma-saas"} mocker.patch.object(demisto, "args", return_value=args) mocker.patch.object(demisto, "params", return_value={"credentials": CREDENTIALS}) mocker.patch.object(demisto, "results") main() assert requests_mock.last_request.headers["service-name"] == "prisma-saas" def test_get_dlp_report_omits_service_name_header_by_default(requests_mock, mocker): """ Given: - No service_name argument. When: - Running the pan-dlp-get-report command. Then: - Ensure no service-name header is sent, leaving existing calls unchanged. """ report_id = 12345 requests_mock.get(f"{DLP_URL}public/report/{report_id}?fetchSnippets=true", json={"id": "test"}) mocker.patch.object(demisto, "command", return_value="pan-dlp-get-report") args = {"report_id": report_id, "fetch_snippets": "true"} mocker.patch.object(demisto, "args", return_value=args) mocker.patch.object(demisto, "params", return_value={"credentials": CREDENTIALS}) mocker.patch.object(demisto, "results") main() assert "service-name" not in requests_mock.last_request.headers def test_parse_dlp_report(mocker): mocker.patch.object(demisto, "results") results = parse_dlp_report(REPORT_DATA).to_context() pattern_results = demisto.get(results["Contents"], "scanContentRawReport.data_pattern_rule_1_results", None) assert pattern_results is not None # Verify MatchedConfidenceLevel is present in DataPatternMatches contents = results["EntryContext"]["DLP.Report(val.DataPatternName && val.DataPatternName == obj.DataPatternName)"] data_pattern_matches = contents["DataPatternMatches"] assert len(data_pattern_matches) > 0 assert data_pattern_matches[0]["MatchedConfidenceLevel"] == "low" # Verify DataProfiles is present and correctly parsed data_profiles = contents["DataProfiles"] assert len(data_profiles) == 1 assert data_profiles[0]["Name"] == "Test Profile" assert data_profiles[0]["Id"] == 12345 assert data_profiles[0]["Version"] == 1 assert data_profiles[0]["IsTriggered"] is True assert len(data_profiles[0]["DataPatterns"]) == 2 assert data_profiles[0]["DataPatterns"][0]["Id"] == "pattern_id_1" assert data_profiles[0]["DataPatterns"][0]["IsMatched"] is True assert data_profiles[0]["DataPatterns"][0]["ConfidenceLevel"] == "high" assert data_profiles[0]["DataPatterns"][0]["OccurrenceCount"] == 5 assert data_profiles[0]["DataPatterns"][1]["OccurrenceOperatorType"] == "between" assert data_profiles[0]["DataPatterns"][1]["OccurrenceHigh"] == 10 def test_get_dlp_incidents(requests_mock): requests_mock.get(f"{DLP_URL}public/incident-notifications?regions=us", json={"us": []}) client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) result, status_code = client.get_dlp_incidents(regions="us") assert result == {"us": []} assert status_code == 200 @pytest.mark.parametrize( "error_code", [(401), (403)], ) def test_refresh_token(requests_mock, mocker, error_code): with pytest.raises(Exception): report_id = 12345 headers1 = {"Authorization": "Bearer 123", "Content-Type": "application/json"} requests_mock.get(f"{DLP_URL}public/report/{report_id}?fetchSnippets=true", headers=headers1, status_code=error_code) requests_mock.post(f"{DLP_URL}public/oauth/refreshToken", json={"access_token": "abc"}) credentials = ( { "credential": "", "credentials": { "id": "", "locked": False, "modified": "0001-01-01T00:00:00Z", "name": "", "password": "", "sortValues": None, "sshkey": "", "sshkeyPass": "", "user": "", "vaultInstanceId": "", "version": 0, "workgroup": "", }, "identifier": "123", "password": "", "passwordChanged": False, }, ) client = Client(DLP_URL, AUTH_URL, credentials, False, False) client.get_dlp_report(report_id, True) assert client.access_token == "abc" def test_refresh_token_with_access_token(requests_mock, mocker): requests_mock.post(f"{DLP_URL}public/oauth/refreshToken", json={"access_token": "abc"}) client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) client._refresh_token() assert client.access_token == "abc" def test_refresh_token_with_client_credentials(requests_mock): credentials = { "credential": "test credentials", "credentials": { "id": "test credentials", "locked": False, "name": "test credentials", "password": "test-pass", "sortValues": None, "sshkey": "", "sshkeyPass": "", "user": "test-user", "vaultInstanceId": "", "version": 1, "workgroup": "", }, "identifier": "test-user", "password": "test-pass", "passwordChanged": False, } requests_mock.post(AUTH_URL, json={"access_token": "abc"}) client = Client(DLP_URL, AUTH_URL, credentials, False, False) assert client.access_token == "abc" @pytest.mark.parametrize( "error_code", [(401), (403)], ) def test_handle_4xx_errors(requests_mock, mocker, error_code): credentials = { "credential": "test credentials", "credentials": { "id": "test credentials", "locked": False, "name": "test credentials", "password": "test-pass", "sortValues": None, "sshkey": "", "sshkeyPass": "", "user": "test-user", "vaultInstanceId": "", "version": 1, "workgroup": "", }, "identifier": "test-user", "password": "test-pass", "passwordChanged": False, } requests_mock.post(AUTH_URL, json={"access_token": "abc"}) client = Client(DLP_URL, AUTH_URL, credentials, False, False) response_mock = mocker.MagicMock() response_mock.status_code = error_code # mocker.PropertyMock(return_value=error_code) client._handle_4xx_errors(response_mock) assert client.access_token == "abc" client = Client(DLP_URL, AUTH_URL, CREDENTIALS, False, False) tokens_mocker = mocker.patch.object(client, "_refresh_token") client._handle_4xx_errors(response_mock) tokens_mocker.assert_called_with() def test_exemption_eligible(mocker): args = {"data_profile": "abc"} params = {"dlp_exemptible_list": "abc,aaa,bbb"} mocker.patch.object(demisto, "results") results = exemption_eligible_command(args, params).to_context() assert results["Contents"] == {"eligible": True} def test_exemption_eligible_wildcard(mocker): args = {"data_profile": "abc"} params = {"dlp_exemptible_list": "*"} mocker.patch.object(demisto, "results") results = exemption_eligible_command(args, params).to_context() assert results["Contents"] == {"eligible": True} def test_slack_bot_message(mocker): params = {"dlp_slack_message": "Hello $user, your file $file_name on $app_name violated $data_profile_name"} args = {"user": "John Doe", "file_name": "secrets.doc", "app_name": "Google Drive", "data_profile_name": "PCI"} mocker.patch.object(demisto, "results") results = slack_bot_message_command(args, params).to_context() assert results["Contents"] == {"message": "Hello John Doe, your file secrets.doc on Google Drive violated PCI"} def test_parse_incident_details(): compressed_str = "QlpoOTFBWSZTWVnl2RYAAKIfgFAFfBBEAoAKv+ffqjAA2CIpoZGjEDTIZBpgGGRpppkYTIwTQGBiSp/pTZGqe1T8qMQaaeo9Nqm3YdNAidgNoZcFEJmTIP+V1xQohhqNsWERYRnKAc3TlogFoteml94kUR+lVJzjB9uhEqOgfBMrQh34ox8qYCCQo2n9WoNceFBvtSCAfMeY7sIAvtXhGQZ7UToozWEQwedzu/MRtoFMK8+ucpSbK4O7zRnPU82E9etuWR5AtmDQF5muuAczVDMFREJd+AEsRAKqdBdyRThQkFnl2RY=" # noqa: E501 details = parse_incident_details(compressed_str) assert details["app_details"] == {"name": "Microsoft OneDrive"} def test_query_sleep_time(requests_mock): requests_mock.get(f"{DLP_URL}public/seconds-between-incident-notifications-pull", json=10) client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) time = client.query_for_sleep_time() assert time == 10 @pytest.mark.parametrize( "incident_type_input, expected_type", [ (None, "Data Loss Prevention"), ("custom type", "custom type"), ], ) def test_create_incident(incident_type_input, expected_type): """ Given: - A DLP notification containing an incident. When: - Calling `create_incident` with or without specifying an incident type. Then: - Ensure no errors due to the lack of `userId` in `INCIDENT_JSON`. - Ensure the incident is created with the correct type. """ import copy # Inputs notification = {"incident": copy.deepcopy(INCIDENT_JSON), "previous_notifications": []} region = "us" # Prepare parsed_details = parse_incident_details(INCIDENT_JSON["incidentDetails"]) occurred_time = arg_to_datetime(INCIDENT_JSON["createdAt"]).isoformat() user_id = parsed_details["headers"][0]["attribute_value"] # Take `attribute_value` where `attribute_name` = "username" raw_data = { **INCIDENT_JSON, "userId": user_id, "incidentDetails": parsed_details, "region": region, "previousNotification": None, } # Act if incident_type_input is None: result = create_incident(notification, region=region) else: result = create_incident(notification, region=region, incident_type=incident_type_input) # Assert - check standard fields assert result["name"] == f"Palo Alto Networks DLP Incident {INCIDENT_JSON['incidentId']}" assert result["type"] == expected_type assert result["occurred"] == occurred_time assert result["rawJSON"] == json.dumps(raw_data) assert result["details"] == json.dumps(raw_data) @pytest.mark.parametrize( "incident_ids_timestamps, last_run, has_new_incidents, last_queried_end_time, expected_timestamp, expected_ids", [ pytest.param( {"id1": 1000, "id2": 2000, "id3": 2000, "id4": 1500}, {START_TIMESTAMP_KEY: 500, LAST_IDS_TIMESTAMPS_KEY: {"old_id": 500}}, True, 2000, 2000, {"id2", "id3"}, # Both have timestamp 2000, within buffer (look_back=0 → cutoff = 2000-30 = 1970) id="multiple_incidents_different_timestamps", ), pytest.param( {}, {START_TIMESTAMP_KEY: 1234567890, LAST_IDS_TIMESTAMPS_KEY: {"id1": 1234567890}}, False, 1234567980, # last_queried_end_time advances by one interval (90s for this test) 1234567980, {"id1"}, # last_ids_timestamps preserved from last_run id="no_new_incidents_advances_start_timestamp", ), pytest.param( {"id1": 1000}, {START_TIMESTAMP_KEY: 500, LAST_IDS_TIMESTAMPS_KEY: {}}, True, 1000, 1000, {"id1"}, id="single_incident", ), pytest.param( {"id1": 2000, "id2": 2000 - END_TIME_BUFFER, "id3": 2000 - END_TIME_BUFFER - 1, "id4": 2000 - 15}, {START_TIMESTAMP_KEY: 500, LAST_IDS_TIMESTAMPS_KEY: {}}, True, 2000, 2000, {"id1", "id2", "id4"}, # id3 excluded (outside buffer: 2000-30-1=1969 < 1970) id="buffer_window_filtering", ), pytest.param( {"id1": 2000, "id2": 1999, "id3": 1998, "id4": 1971, "id5": 1970, "id6": 1969}, {START_TIMESTAMP_KEY: 500, LAST_IDS_TIMESTAMPS_KEY: {}}, True, 2000, 2000, {"id1", "id2", "id3", "id4", "id5"}, # id6 excluded (1969 < 1970 which is 2000-30) id="exact_buffer_boundary", ), ], ) def test_compute_next_run( incident_ids_timestamps, last_run, has_new_incidents, last_queried_end_time, expected_timestamp, expected_ids ): """ Given: - A dictionary of incident IDs mapped to their committed timestamps. - A boolean indicating whether new incidents were fetched. - The end_time of the last queried interval. When: - Calling compute_next_run. Then: - If new incidents were fetched: returns the correct timestamp and IDs within the buffer window. - If no new incidents were fetched: advances start_timestamp to last_queried_end_time and preserves last_ids_timestamps from last_run. """ result = compute_next_run( incident_ids_timestamps, last_run, has_new_incidents=has_new_incidents, last_queried_end_time=last_queried_end_time ) assert result[START_TIMESTAMP_KEY] == expected_timestamp assert set(result.get(LAST_IDS_TIMESTAMPS_KEY, {}).keys()) == expected_ids @pytest.mark.parametrize( "start, end, delta, expected_intervals", [ pytest.param( 0, 900, 300, [(0, 300), (300, 600), (600, 900)], id="even_intervals", ), pytest.param( 0, 1000, 300, [(0, 300), (300, 600), (600, 900), (900, 1000)], id="uneven_intervals_capped_at_end", ), pytest.param( 0, 100, 300, [(0, 100)], id="single_interval_delta_exceeds_range", ), pytest.param( 100, 100, 300, [], id="empty_range", ), ], ) def test_get_start_end_time_intervals(start, end, delta, expected_intervals): """ Given: - Start and end timestamps with a delta. When: - Calling get_start_end_time_intervals. Then: - Ensure it returns the correct time intervals. """ result = get_start_end_time_intervals(start, end, delta) assert result == expected_intervals @freeze_time("2022-04-01 20:25:00 UTC") def test_fetch_notifications_basic(requests_mock, mocker): """ Given: - A client and basic parameters with frozen time. When: - Calling fetch_notifications with no previous last_run. Then: - Ensure incidents are created and last_run is updated. """ import re from datetime import datetime from Palo_Alto_Networks_Enterprise_DLP import LOCAL_LAST_RUN LOCAL_LAST_RUN.clear() # Mock API response mock_notification = { "incident": { "incidentId": "test-id-1", "committedAt": "2022-Apr-01 20:21:50 UTC", "createdAt": "2022-Apr-01 20:21:50 UTC", "incidentDetails": INCIDENT_JSON["incidentDetails"], "tenantId": "1128505801991063552", "reportId": "2573778324", }, "previous_notifications": [], } requests_mock.get(re.compile(f"{DLP_URL}public/incident-notifications.*"), json={"us": [mock_notification]}) mocker.patch.object(demisto, "getIntegrationContext", return_value={}) mocker.patch.object(demisto, "getLastRun", return_value={}) mocker.patch.object(demisto, "createIncidents") mocker.patch.object(demisto, "setIntegrationContext") client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) # Use timestamp very close to frozen time (just 2 minutes before to minimize intervals) first_fetch_timestamp = int(datetime(2022, 4, 1, 20, 23, 0, tzinfo=UTC).timestamp()) next_run, incidents = fetch_notifications(client, "us", first_fetch_timestamp) assert len(incidents) == 1 assert "test-id-1" in incidents[0]["name"] assert next_run == {"start_timestamp": 1648844510, LAST_IDS_TIMESTAMPS_KEY: {"test-id-1": 1648844510}} @freeze_time("2026-04-01 20:25:00 UTC") def test_fetch_notifications_lookback(requests_mock, mocker): """ Given: - A last run with start_timestamp T and look_back_minutes=5. When: - Calling fetch_notifications. Then: - The first API interval starts at T - 5*60 (i.e. lookback is applied). """ import re from datetime import datetime start_timestamp = int(datetime(2026, 4, 1, 20, 23, 0, tzinfo=UTC).timestamp()) # T look_back_seconds = 5 * 60 expected_effective_start = start_timestamp - look_back_seconds requests_mock.get(re.compile(f"{DLP_URL}public/incident-notifications.*"), json={}) mocker.patch.object(demisto, "getIntegrationContext", return_value={}) mocker.patch.object(demisto, "getLastRun", return_value={START_TIMESTAMP_KEY: start_timestamp}) mocker.patch.object(demisto, "setIntegrationContext") client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) fetch_notifications(client, "us", first_fetch_timestamp=start_timestamp, look_back_minutes=5) # The very first request must use start_timestamp=expected_effective_start first_request_url = requests_mock.request_history[0].url assert f"start_timestamp={expected_effective_start}" in first_request_url @freeze_time("2026-04-01 20:25:00 UTC") def test_fetch_notifications_advances_start_timestamp_when_no_new_incidents(requests_mock, mocker): """ Given: - A last_run with a stale start_timestamp and all API responses returning empty results. When: - Calling fetch_notifications. Then: - Ensure start_timestamp in next_run is advanced to the end_time of the last queried interval, preventing the query window from growing unboundedly on subsequent fetches. """ import re from datetime import datetime start_timestamp = int(datetime(2026, 4, 1, 20, 23, 0, tzinfo=UTC).timestamp()) requests_mock.get(re.compile(f"{DLP_URL}public/incident-notifications.*"), json={}) mocker.patch.object(demisto, "getIntegrationContext", return_value={}) mocker.patch.object(demisto, "getLastRun", return_value={START_TIMESTAMP_KEY: start_timestamp, LAST_IDS_TIMESTAMPS_KEY: {}}) mocker.patch.object(demisto, "setIntegrationContext") client = Client(DLP_URL, AUTH_URL, CREDENTIALS, True, False) next_run, incidents = fetch_notifications(client, "us", first_fetch_timestamp=start_timestamp) assert incidents == [] # start_timestamp must advance beyond the stale value — it should equal the end_time of the # last queried interval (start_timestamp + MAX_API_CALLS_PER_FETCH * 180s), not remain frozen. assert next_run[START_TIMESTAMP_KEY] > start_timestamp @pytest.mark.parametrize( "last_run, start_timestamp, expected", [ pytest.param( {LAST_IDS_TIMESTAMPS_KEY: {"id1": 1000, "id2": 2000}}, 500, {"id1": 1000, "id2": 2000}, id="new_schema_returned_as_is", ), pytest.param( {LAST_IDS_KEY: ["id1", "id2"]}, 500, {"id1": 500, "id2": 500}, id="legacy_ids_seeded_with_start_timestamp", ), pytest.param( {}, 500, {}, id="empty_last_run_returns_empty_dict", ), pytest.param( {LAST_IDS_KEY: []}, 500, {}, id="legacy_empty_list_returns_empty_dict", ), ], ) def test_migrate_last_run(last_run: dict, start_timestamp: int, expected: dict): """ Given: - A last run dict in either the new (last_ids_timestamps) or legacy (last_ids) schema, or an empty dict. When: - Calling _migrate_last_run with a start_timestamp. Then: - New schema is returned unchanged as a plain dict copy. - Legacy IDs are migrated and each ID is seeded with start_timestamp. - Empty / missing keys produce an empty dict. """ result = _migrate_last_run(last_run, start_timestamp) assert result == expected