SaaS Security Event Collector

Palo Alto Networks SaaS Security Event Collector integration for XSIAM.

Analytics & SIEM · SaaS Security by Palo Alto Networks

Details

IDSaaS Security Event Collector
ProviderPalo Alto Networks
CategoryAnalytics & SIEM
From Version6.8.0
Docker Imagedemisto/python3:3.12.13.10116658
Supported ModulesAgentix XSIAM EDR Cloud Runtime Security

README

SaaS Security is an integrated CASB (Cloud Access Security Broker) solution that helps Security teams like yours meet
the challenges of:

  • protecting the growing availability of sanctioned and unsanctioned SaaS applications
  • maintaining compliance consistently in the cloud
  • stopping threats to sensitive information, users, and resources

This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM.

Configure SaaS Security on Cortex XSIAM

  1. Navigate to Settings > Configurations > Data Collection > Automations & Feed Integrations.
  2. Search for SaaS Security Event Collector.
  3. Click Add instance to create and configure a new integration instance.

    Parameter Description Required
    Your server URL The instance configuration URL based on the server location. True
    Client ID The SaaS Security Client ID. True
    Client Secret The SaaS Security Secret ID. True
    Trust any certificate (not secure) By default, SSL verification is enabled. If selected, the connection isn’t secure and all requests return an SSL error because the certificate cannot be verified. False
    Use system proxy settings Uses the system proxy server to communicate with the integration. If not selected, the integration will not use the system proxy server. False
    The maximum number of events per fetch. Applies only to the manual saas-security-get-events command. During scheduled Fetch Events, the collector drains as much of the queue as possible per cycle (bounded by The maximum number of iterations to retrieve events), so this value does not throttle live ingestion. Must be divisible by 10 due to SaaS Security API limitations. Default is 1000. False
    The maximum number of iterations to retrieve events. Each iteration retrieves up to 100 events from the SaaS Security queue (the API’s per-request limit). This parameter caps the number of iterations per fetch execution to prevent timeouts; the collector keeps draining across consecutive executions until the queue is empty. Increase this value if ingestion lag builds up under a high event rate. Default is 900. False
    Number of concurrent fetch requests The number of GET requests sent in parallel each iteration to drain the SaaS Security queue faster under high event rates. Maximum is 30. This is an advanced parameter. Default is 10. False
    Treat an empty Cortex XSIAM response as delivered Whether to treat a success (200) response from Cortex XSIAM that has an empty body as a successful delivery instead of retrying it indefinitely. Truncated or otherwise invalid responses are still retried. This is an advanced parameter. False
  4. Click Test to validate the URLs, token, and connection.

Create the Client ID and Client Secret on SaaS Security

In the SaaS Security UI, do the following:

  1. Navigate to Settings > External Service.
  2. Click Add API Client.
  3. Specify a unique name for the API client.
  4. Authorize the API client for the required scopes. You use these scopes in the POST request to the /oauth/token endpoint. The Required Scopes are:
    • Log access — Access log files. You can either provide the client log access API or add a syslog receiver.
    • Incident management — Retrieve and change the incident status.
    • Quarantine management — Quarantine assets and restore quarantined assets.
  5. Copy the client ID and client secret.
    Tip: Record your API client secret somewhere safe. For security purposes, it’s only shown when you create or reset the API client. If you lose your secret you must reset it, which removes access for any integrations that still use the previous secret.
  6. Add the Client ID and Client Secret to Cortex XSOAR.
    Note: For more information see the SaaS Security Administrator’s Guide

Limitations

1) Occurring events expire after one hour in the SaaS Security cache. During scheduled fetch the collector drains the queue continuously (it does not stop early at max_fetch), so under normal operation events are pulled well within the one-hour window. If the upstream event rate is very high and a single instance cannot keep up, increase The maximum number of iterations to retrieve events so each cycle drains more, and/or distribute the load across multiple instances.
2) The SaaS Security /log_events_bulk API returns at most 100 events per call. Each fetch iteration retrieves one such batch.
3) If the max_fetch is not divisible by 10, it will be rounded down to a number that is divisible by 10 due to SaaS Security API limits.
4) reset last fetch has no effect.
5) On initial activation this integration will pull events starting from one hour prior.
6) Using the saas-security-get-events command may take upwards of twenty seconds in some cases.
7) The max_fetch parameter applies only to the manual saas-security-get-events command; it does not limit the scheduled Fetch Events flow, which drains the full queue per cycle (bounded by the maximum number of iterations).
8) In case not providing the max_fetch argument to the saas-security-get-events command, the default will be 1000.

Fetch Events

Requires the scope of api_access in order to fetch log events. See Documentation
Since those events are saved only 1 hour at cache, it is highly recommended giving Events Fetch Interval in minutes rather than hours.

In case not stating a max fetch in the integration parameters, all available events will be fetched.

Log types could be one of policy_violation, activity_monitoring, remediation, incident, and admin_audit.
Every type returns a different api response that is unique.

Example Activity Monitoring Response

{
    "log_type" : "activity_monitoring",
    "item_type" : "File",
    "item_name" : "My File",
    "user" : "John Smith",
    "source_ip" : "10.10.10.10",
    "location" : "Somewhere, USA",
    "action" : "delete",
    "target_name" : null,
    "target_type" : null,
    "severity" : 1.0,
    "serial" : "mySerial",
    "cloud_app_instance" : "My Cloud App",
    "timestamp" : "2018-11-09T18:30:33.155Z"
}

Example Incident Response

{
    "log_type" : "incident",
    "severity" : 4.0,
    "item_type" : "File",
    "item_name" : "My File",
    "asset_id" : "ce7c9ed11e6f4891ae73c1601af7f741",
    "item_owner" : "John Smith",
    "container_name" : "Container",
    "item_creator" : "John Smith",
    "exposure" : "public",
    "occurrences_by_rule" : 5,
    "item_owner_email" : "owner@<--domain-->.com",
    "item_creator_email" : "creator@<--domain-->.com",
    "serial" : "mySerial",
    "cloud_app_instance" : "My Cloud App",
    "timestamp" : "2018-11-09T18:30:32.572Z",
    "incident_id" : "9610efdcd8a74a259bf031843eac0309",
    "policy_rule_name" : "PCI Policy",
    "incident_category" : "Testing",
    "incident_owner" : "John Smith"
}

Example Remediation Response

{
    "log_type" : "remediation",
    "item_type" : "File",
    "item_name" : "My File",
    "asset_id" : "ce7c9ed11e6f4891ae73c1601af7f741",
    "item_owner" : "John Smith",
    "container_name" : "Container",
    "item_creator" : "John Smith",
    "action_taken" : "quarantine",
    "action_taken_by" : "John Smith",
    "item_owner_email" : "owner@<--domain-->.com",
    "item_creator_email" : "creator@<--domain-->.com",
    "serial" : "mySerial",
    "cloud_app_instance" : "My Cloud App",
    "timestamp" : "2018-11-09T18:30:30.909Z",
    "incident_id" : "9610efdcd8a74a259bf031843eac0309",
    "policy_rule_name" : "PCI Policy"
}

Example Policy Violation Response

{
    "log_type" : "policy_violation",
    "severity" : 3.0,
    "item_type" : "File",
    "item_name" : "My File",
    "item_owner" : "John Smith",
    "item_creator" : "John Smith",
    "action_taken" : "download",
    "action_taken_by" : "John Smith",
    "asset_id" : "ce7c9ed11e6f4891ae73c1601af7f741",
    "item_owner_email" : null,
    "item_creator_email" : null,
    "serial" : "serial",
    "cloud_app_instance" : "My Cloud App",
    "timestamp" : "2017-01-06T19:04:06Z",
    "policy_rule_name" : "Policy Rule",
    "incident_id" : "1234"
}

Example Admin Audit Response

{
    "log_type" : "admin_audit",
    "admin_id" : "admin id",
    "admin_role" : "admin role",
    "ip" : "ip address",
    "event_type" : "event type",
    "item_type" : "File",
    "item_name" : "My File",
    "field" : "field",
    "action" : "action",
    "resource_value_old" : "old val",
    "resource_value_new" : "new val",
    "timestamp" : "2018-11-09T18:30:29.739Z",
    "serial" : "mySerial"
}

for more information see documentation

Commands

You can execute these commands from the Cortex XSIAM CLI as part of an automation or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

saas-security-get-events


Manual command to fetch events and display them. Use this command for development and debugging only, as it may produce duplicate events, exceed API rate limits, or disrupt the fetch mechanism.

Base Command

saas-security-get-events

Input

Argument Name Description Required
limit The maximum number of events to get. Must be divisible by 10 due to SaaS Security API limitations. Overrides the max-fetch parameter of the integration. Optional
should_push_events Set this argument to True in order to create events, otherwise the command will only display them. If setting to ‘False’ The returned events will be lost. Possible values are: True, False. Default is False. Required

Context Output

Path Type Description
SaasSecurity.Event.log_type String Event type.
SaasSecurity.Event.item_type String Item type (File, Folder, or User).
SaasSecurity.Event.item_name String Name of the file, folder, or user associated with the event.
SaasSecurity.Event.item_unique_id String Unique ID number for an asset’s related asset.
SaasSecurity.Event.user String Cloud app user that performed the action.
SaasSecurity.Event.source_ip String Original session source IP address.
SaasSecurity.Event.location String Location of the cloud app user that performed the event.
SaasSecurity.Event.action String Action performed.
SaasSecurity.Event.target_name String Target name.
SaasSecurity.Event.target_type String Target type.
SaasSecurity.Event.serial String Serial number of the organization using the service (tenant).
SaasSecurity.Event.cloud_app_instance String Cloud app name (not cloud app type).
SaasSecurity.Event.timestamp Date ISO8601 timestamp to show when the event occurred.
SaasSecurity.Event.severity Number Severity (0-5).
SaasSecurity.Event.incident_id String Incident/risk id.
SaasSecurity.Event.exposure String Exposure level (public, external, company, or internal).
SaasSecurity.Event.asset_id String The asset ID.
SaasSecurity.Event.item_owner String The item owner.
SaasSecurity.Event.container_name String Item’s container name.
SaasSecurity.Event.item_creator String Item creator.
SaasSecurity.Event.occurrences_by_rule Number Number of times the asset violated the policy.
SaasSecurity.Event.policy_rule_name String Violated policy’s name.
SaasSecurity.Event.incident_owner String Incident owner.
SaasSecurity.Event.incident_category String Incident category.
SaasSecurity.Event.item_creator_email String Item creator’s email.
SaasSecurity.Event.action_taken String Action taken.
SaasSecurity.Event.action_taken_by String Action taken by.
SaasSecurity.Event.field String Name of field (optional).
SaasSecurity.Event.resource_value_old String Old resource value. (optional).
SaasSecurity.Event.resource_value_new String New resource value. (optional).

Command example

!saas-security-get-events limit=200 should_push_events=False

Context Example

{
    "SaasSecurity": {
        "Event": [
            {
                "action": "preview",
                "cloud_app_instance": "Box 1",
                "item_name": "ssn_test3.txt",
                "item_type": "file",
                "item_unique_id": "123",
                "location": "somewhere, usa",
                "log_type": "activity_monitoring",
                "serial": null,
                "severity": 1,
                "source_ip": "2.2.2.2",
                "target_name": null,
                "target_type": "",
                "timestamp": "2022-05-30T06:40:59Z",
                "user": "some email"
            },
            {
                "action": "preview",
                "cloud_app_instance": "Box 1",
                "item_name": "SP0605 copy.java.txt",
                "item_type": "file",
                "item_unique_id": "1234",
                "location": "somewhere usa, Israel",
                "log_type": "activity_monitoring",
                "serial": null,
                "severity": 1,
                "source_ip": "1.1.1.1",
                "target_name": null,
                "target_type": "",
                "timestamp": "2022-05-30T06:40:47Z",
                "user": "some email"
            }
        ]
    }
}

Human Readable Output

SaaS Security Logs

LogType ItemType ItemName Timestamp
activity_monitoring file ssn_test3.txt 2022-05-30T06:40:59Z
activity_monitoring file SP0605 copy.java.txt 2022-05-30T06:40:47Z

Configuration parameters

  • url — Your server URL (required)
  • credentials — Client ID (required)
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings
  • max_fetch — The maximum number of events per fetch.
  • max_iterations — The maximum number of iterations to retrieve events
  • event_fetch_concurrency — Number of concurrent fetch requests
  • event_pass_over_empty_response — Treat an empty Cortex XSIAM response as delivered

Commands (1)

  • saas-security-get-events

    Manual command to fetch events and display them. Use this command for development and debugging only, as it may produce duplicate events, exceed API rate limits, or disrupt the fetch mechanism.

"""Unit tests for the SaaS Security Event Collector integration."""

import json

import pytest
import SaasSecurityEventCollector
from CommonServerPython import *  # noqa

BASE_PARAMS = {
    "url": "https://test.com/",
    "credentials": {"identifier": "1234", "password": "1234"},
}


@pytest.fixture
def mock_client():
    return SaasSecurityEventCollector.Client(
        base_url="https://test.com/api", client_id="", client_secret="", verify=False, proxy=False
    )


def create_events(start_id=1, end_id=100, should_dump=True):
    events = {"events": [{"id": i} for i in range(start_id, end_id + 1)]}
    return json.dumps(events) if should_dump else events


class MockedResponse:
    def __init__(self, status_code, text="{}"):
        self.status_code = status_code
        self.text = text

    def json(self):
        return json.loads(self.text)


def test_module(mocker, mock_client):
    """
    Given a valid access token, when testing the module, then it returns 'ok'.
    """
    mocker.patch.object(SaasSecurityEventCollector.Client, "get_token_request")
    assert SaasSecurityEventCollector.test_module(client=mock_client) == "ok"


def test_get_new_access_token(mocker, mock_client):
    mocker.patch.object(mock_client, "get_token_request", return_value=("123", "100"))
    assert mock_client.get_access_token() == "123"


# ---------------------------------------------------------------------------
# get_max_iterations - the code-level floor (fix #1)
# ---------------------------------------------------------------------------


@pytest.mark.parametrize(
    "configured, expected",
    [
        (None, 900),  # not configured -> default
        (0, 900),  # non-positive -> default
        (50, 900),  # stale legacy value below the floor -> raised to floor
        (300, 900),  # previous default, now below the floor -> raised
        (899, 900),  # just below the floor -> raised
        (900, 900),  # at the floor
        (1200, 1200),  # above the floor -> honored
    ],
)
def test_get_max_iterations_floor(configured, expected):
    """
    Given a configured max_iterations value (including a stale low one that cannot be edited on the instance),
    when resolving the effective value,
    then it is never below MIN_MAX_ITERATIONS so throughput cannot be capped by a stale instance param.
    """
    assert SaasSecurityEventCollector.get_max_iterations(configured) == expected


# ---------------------------------------------------------------------------
# get_concurrency - clamping
# ---------------------------------------------------------------------------


@pytest.mark.parametrize(
    "configured, expected",
    [(None, 10), (0, 10), (-5, 10), (5, 5), (30, 30), (100, 30)],
)
def test_get_concurrency(configured, expected):
    assert SaasSecurityEventCollector.get_concurrency(configured) == expected


# ---------------------------------------------------------------------------
# build_client - a fresh client (own session) per call (fix #2, thread safety)
# ---------------------------------------------------------------------------


def test_build_client_returns_independent_instances():
    """
    Given params, when building clients for concurrent workers,
    then each call returns a distinct Client with its own session (required for thread safety).
    """
    c1 = SaasSecurityEventCollector.build_client(BASE_PARAMS)
    c2 = SaasSecurityEventCollector.build_client(BASE_PARAMS)
    assert c1 is not c2
    assert c1._session is not c2._session


# ---------------------------------------------------------------------------
# get_events_batch
# ---------------------------------------------------------------------------


def test_get_events_batch_204_is_drained(mocker, mock_client):
    mocker.patch.object(SaasSecurityEventCollector.Client, "http_request", return_value=MockedResponse(status_code=204))
    events, drained = SaasSecurityEventCollector.get_events_batch(mock_client)
    assert events == []
    assert drained is True


def test_get_events_batch_200_returns_events(mocker, mock_client):
    mocker.patch.object(
        SaasSecurityEventCollector.Client,
        "http_request",
        return_value=MockedResponse(status_code=200, text=create_events(1, 100)),
    )
    events, drained = SaasSecurityEventCollector.get_events_batch(mock_client)
    assert len(events) == 100
    assert drained is False


# ---------------------------------------------------------------------------
# send_events_in_chunks - resilient chunked send (fix #3, context self-heal)
# ---------------------------------------------------------------------------


def test_send_events_in_chunks_success_empties_list(mocker):
    """
    Given events and a working send, when sending in chunks,
    then all events are sent and the source list is emptied (all acknowledged).
    """
    send_mock = mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam")
    events = [{"id": i} for i in range(2500)]
    sent = SaasSecurityEventCollector.send_events_in_chunks(events, send_batch_size=1000, vendor="v", product="p")
    assert sent == 2500
    assert events == []  # fully drained
    assert send_mock.call_count == 3  # 1000 + 1000 + 500


def test_send_events_in_chunks_failure_keeps_only_remainder(mocker):
    """
    Given a stashed batch and a send that fails on the 2nd chunk,
    when sending in chunks,
    then the successfully-sent chunk is removed and only the unsent remainder is left in the list.

    This is the self-heal guarantee: the poisoned/oversized stash shrinks every cycle instead of being
    re-stashed whole and retried forever.
    """
    mocker.patch.object(SaasSecurityEventCollector, "demisto")
    calls = {"n": 0}

    def flaky_send(events, vendor, product, **kwargs):
        calls["n"] += 1
        if calls["n"] == 2:
            raise ValueError("Expecting value: line 1 column 1 (char 0)")

    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam", side_effect=flaky_send)
    events = [{"id": i} for i in range(2500)]
    with pytest.raises(ValueError):
        SaasSecurityEventCollector.send_events_in_chunks(events, send_batch_size=1000, vendor="v", product="p")
    # First 1000 acknowledged and removed; remaining 1500 preserved for retry.
    assert len(events) == 1500
    assert events[0]["id"] == 1000


def _json_decode_error(body: str):
    """Build a real json.JSONDecodeError as raised by response.json() on the given (bad) body."""
    try:
        json.loads(body)
    except json.JSONDecodeError as exc:
        return exc
    raise AssertionError("body was valid JSON; expected a decode error")


# ---------------------------------------------------------------------------
# describe_xsiam_response_failure - capture the ACTUAL XSIAM response body
# ---------------------------------------------------------------------------


@pytest.mark.parametrize(
    "body, expect_benign",
    [
        ("", True),  # completely empty body -> benign empty-body case
        ("   \n\t ", True),  # whitespace-only body -> benign empty-body case
        ("<html>502</html>", False),  # non-empty unparseable body -> real failure
        ('{"ok":true}xtra', False),  # trailing garbage (Extra data) -> real failure
    ],
)
def test_describe_xsiam_response_failure_classification(body, expect_benign):
    """
    Given the JSONDecodeError raised by response.json() on a bad XSIAM response body,
    then the helper surfaces the ACTUAL body length and classifies blank bodies as benign
    (pass-over candidate) while non-empty unparseable bodies are NOT benign.
    """
    exc = _json_decode_error(body)
    description, benign = SaasSecurityEventCollector.describe_xsiam_response_failure(exc)
    assert benign is expect_benign
    assert f"response_body_len={len(body)}" in description
    assert "response_body_preview=" in description


def test_describe_xsiam_response_failure_non_decode_error():
    """
    Given an exception that is NOT a JSONDecodeError (e.g. a network/DemistoException),
    then there is no response body to show and it is not classified as a benign empty body.
    """
    description, benign = SaasSecurityEventCollector.describe_xsiam_response_failure(RuntimeError("boom"))
    assert benign is False
    assert "response_body=<unavailable>" in description
    assert "exc_type=RuntimeError" in description


def test_send_events_in_chunks_passes_over_empty_body(mocker):
    """
    Given XSIAM returns a 200 with an empty body (JSONDecodeError on char 0),
    when sending in chunks with pass_over_empty_response=True,
    then the chunk is treated as delivered: it is removed, counted as sent, and no exception propagates.

    This is the "catch and pass over a benign empty response while the platform team fixes it" behavior.
    """
    mocker.patch.object(SaasSecurityEventCollector, "demisto")

    def empty_body_send(events, vendor, product, **kwargs):
        raise _json_decode_error("")  # empty response body

    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam", side_effect=empty_body_send)
    events = [{"id": i} for i in range(2500)]
    sent = SaasSecurityEventCollector.send_events_in_chunks(
        events, send_batch_size=1000, vendor="v", product="p", pass_over_empty_response=True
    )
    assert sent == 2500
    assert events == []  # all passed over as delivered


def test_send_events_in_chunks_does_not_pass_over_non_empty_body(mocker):
    """
    Given XSIAM returns a 200 with a NON-empty unparseable body,
    when sending in chunks with pass_over_empty_response=True,
    then it is treated as a real failure: the exception propagates and the unsent remainder is preserved.
    """
    mocker.patch.object(SaasSecurityEventCollector, "demisto")

    def truncated_body_send(events, vendor, product, **kwargs):
        raise _json_decode_error("<html>bad gateway</html>")

    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam", side_effect=truncated_body_send)
    events = [{"id": i} for i in range(2500)]
    with pytest.raises(json.JSONDecodeError):
        SaasSecurityEventCollector.send_events_in_chunks(
            events, send_batch_size=1000, vendor="v", product="p", pass_over_empty_response=True
        )
    assert len(events) == 2500  # nothing acknowledged - all preserved for retry


def test_send_events_in_chunks_empty_body_not_passed_over_when_disabled(mocker):
    """
    Given the pass-over is disabled, when XSIAM returns an empty body,
    then even a benign empty body is treated as a failure (exception propagates) and events are preserved.
    """
    mocker.patch.object(SaasSecurityEventCollector, "demisto")

    def empty_body_send(events, vendor, product, **kwargs):
        raise _json_decode_error("")

    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam", side_effect=empty_body_send)
    events = [{"id": i} for i in range(2500)]
    with pytest.raises(json.JSONDecodeError):
        SaasSecurityEventCollector.send_events_in_chunks(
            events, send_batch_size=1000, vendor="v", product="p", pass_over_empty_response=False
        )
    assert len(events) == 2500


# ---------------------------------------------------------------------------
# fetch_and_send_events_concurrently
# ---------------------------------------------------------------------------


def test_concurrent_fetch_uses_own_client_per_worker(mocker):
    """
    Given the concurrent drain, when it issues GET calls,
    then it builds a fresh client per worker (never shares a session across threads).
    """
    build_client_mock = mocker.patch.object(
        SaasSecurityEventCollector, "build_client", wraps=SaasSecurityEventCollector.build_client
    )
    # First round returns events, second round drains.
    batches = [([{"id": 1}], False)] * 3 + [([], True)] * 10
    mocker.patch.object(SaasSecurityEventCollector, "get_events_batch", side_effect=batches)
    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam")

    SaasSecurityEventCollector.fetch_and_send_events_concurrently(params=BASE_PARAMS, max_iterations=6, concurrency=3)
    # build_client is called once per worker submission (never zero -> proves per-thread clients).
    assert build_client_mock.call_count >= 3


def test_concurrent_fetch_flushes_pending_first(mocker):
    """
    Given pending events restored from a poisoned context and an immediately-drained queue,
    when running the concurrent drain,
    then the pending events are flushed to XSIAM (context self-heal) before/independent of new fetches.
    """
    send_mock = mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam")
    mocker.patch.object(SaasSecurityEventCollector, "get_events_batch", return_value=([], True))
    pending = [{"id": i} for i in range(3077)]

    fetched, sent, drained, unsent, exc = SaasSecurityEventCollector.fetch_and_send_events_concurrently(
        params=BASE_PARAMS, max_iterations=150, concurrency=10, send_batch_size=2000, pending_events=pending
    )
    assert exc is None
    assert unsent == []
    assert sent == 3077  # the whole stuck stash was drained
    assert send_mock.called


def test_concurrent_fetch_send_failure_returns_shrunk_unsent(mocker):
    """
    Given a poisoned stash and a send that fails after the first chunk,
    when running the concurrent drain,
    then the exception is captured and the returned unsent list is smaller than the original stash
    (so the next cycle retries a strictly smaller batch - no infinite full-batch replay).
    """
    mocker.patch.object(SaasSecurityEventCollector, "demisto")
    calls = {"n": 0}

    def flaky_send(events, vendor, product, **kwargs):
        calls["n"] += 1
        if calls["n"] >= 2:
            raise ValueError("Extra data: line 1 column 4 (char 3)")

    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam", side_effect=flaky_send)
    mocker.patch.object(SaasSecurityEventCollector, "get_events_batch", return_value=([], True))
    pending = [{"id": i} for i in range(3077)]

    fetched, sent, drained, unsent, exc = SaasSecurityEventCollector.fetch_and_send_events_concurrently(
        params=BASE_PARAMS, max_iterations=150, concurrency=10, send_batch_size=2000, pending_events=pending
    )
    assert isinstance(exc, ValueError)
    assert 0 < len(unsent) < 3077  # stash shrank -> self-heals over cycles
    assert sent == 2000


def test_concurrent_fetch_stops_at_time_budget(mocker):
    """
    Given a queue that never drains, when the wall-clock budget is already exhausted (0s),
    then the drain stops before issuing any GET round and returns queue_drained=False (so main persists
    state and re-fires, instead of the engine hard-killing the execution at 5 minutes with progress lost).
    """
    get_batch_mock = mocker.patch.object(SaasSecurityEventCollector, "get_events_batch", return_value=([{"id": 1}], False))
    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam")

    fetched, sent, drained, unsent, exc = SaasSecurityEventCollector.fetch_and_send_events_concurrently(
        params=BASE_PARAMS, max_iterations=150, concurrency=10, time_budget_seconds=0
    )
    assert drained is False  # not drained -> caller will re-fire via nextTrigger
    assert exc is None
    assert fetched == 0  # exited before issuing any GET round
    assert get_batch_mock.call_count == 0  # budget check short-circuited the loop


def test_concurrent_fetch_stops_at_max_iterations(mocker):
    """
    Given a queue that never drains, when running the concurrent drain,
    then it stops at max_iterations and reports queue_drained=False (backlog signal).
    """
    mocker.patch.object(SaasSecurityEventCollector, "get_events_batch", return_value=([{"id": 1}], False))
    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam")

    fetched, sent, drained, unsent, exc = SaasSecurityEventCollector.fetch_and_send_events_concurrently(
        params=BASE_PARAMS, max_iterations=6, concurrency=3
    )
    assert drained is False
    assert exc is None


def test_concurrent_fetch_keeps_sibling_batches_when_one_worker_raises(mocker):
    """
    Given a round where one worker's ``get_events_batch`` raises while its siblings return already-dequeued
    batches, when running the concurrent drain, then the siblings' batches are NOT lost - they are preserved
    (returned as unsent so the caller stashes and retries them).

    /log_events_bulk is a destructive-read (pop) queue: a sibling batch that a completed worker already
    popped off the server queue is gone from the server. If a peer worker in the same round fails, the
    function must still carry those sibling events out (buffered, then stashed) instead of discarding them
    when the worker exception propagates - otherwise those events are lost forever (data-loss bug guard).
    """
    mocker.patch.object(SaasSecurityEventCollector, "demisto")
    mocker.patch.object(SaasSecurityEventCollector, "send_events_to_xsiam")

    # In the first round one worker raises while the two siblings return real, already-dequeued batches.
    outcomes = [
        RuntimeError("worker network blip"),
        ([{"id": "sibling-a"}], False),
        ([{"id": "sibling-b"}], False),
    ]

    def flaky_batch(client):
        outcome = outcomes.pop(0)
        if isinstance(outcome, Exception):
            raise outcome
        return outcome

    mocker.patch.object(SaasSecurityEventCollector, "get_events_batch", side_effect=flaky_batch)

    fetched, sent, drained, unsent, exc = SaasSecurityEventCollector.fetch_and_send_events_concurrently(
        params=BASE_PARAMS, max_iterations=3, concurrency=3, send_batch_size=1000
    )

    # The worker error propagated (so the caller will stash-and-retry)...
    assert isinstance(exc, RuntimeError)
    # ...but the sibling batches that were already popped off the queue survived in the returned unsent list.
    preserved_ids = {event["id"] for event in unsent}
    assert "sibling-a" in preserved_ids
    assert "sibling-b" in preserved_ids
    assert fetched == 2  # both sibling batches were counted as fetched, not dropped


# ---------------------------------------------------------------------------
# handle_fetch_events - fetch-events orchestration (extracted from main for coverage)
# ---------------------------------------------------------------------------


def _patch_fetch_result(mocker, *, fetched=0, sent=0, queue_drained=True, unsent=None, exception=None):
    """Patch fetch_and_send_events_concurrently to return a canned result tuple."""
    return mocker.patch.object(
        SaasSecurityEventCollector,
        "fetch_and_send_events_concurrently",
        return_value=(fetched, sent, queue_drained, unsent or [], exception),
    )


def test_handle_fetch_events_clean_drain_clears_backlog_state(mocker):
    """
    Given a fully drained queue and a prior backlog carried in last_run,
    when handling fetch-events,
    then nextTrigger and consecutive_backlog_cycles are cleared and the stash is emptied.
    """
    demisto_mock = mocker.patch.object(SaasSecurityEventCollector, "demisto")
    demisto_mock.getLastRun.return_value = {"nextTrigger": "1", "consecutive_backlog_cycles": 7}
    demisto_mock.getIntegrationContext.return_value = {}
    _patch_fetch_result(mocker, fetched=100, sent=100, queue_drained=True)

    last_run = SaasSecurityEventCollector.handle_fetch_events(
        params=BASE_PARAMS, max_iterations=900, concurrency=10, pass_over_empty_response=True
    )

    assert "nextTrigger" not in last_run
    assert "consecutive_backlog_cycles" not in last_run
    demisto_mock.setIntegrationContext.assert_called_once_with({})


def test_handle_fetch_events_backlog_sets_next_trigger_and_increments_counter(mocker):
    """
    Given a queue that did not drain (backlog) with no prior backlog state,
    when handling fetch-events,
    then nextTrigger is set and consecutive_backlog_cycles is incremented to 1.
    """
    demisto_mock = mocker.patch.object(SaasSecurityEventCollector, "demisto")
    demisto_mock.getLastRun.return_value = {}
    demisto_mock.getIntegrationContext.return_value = {}
    _patch_fetch_result(mocker, fetched=5000, sent=5000, queue_drained=False)

    last_run = SaasSecurityEventCollector.handle_fetch_events(
        params=BASE_PARAMS, max_iterations=900, concurrency=10, pass_over_empty_response=True
    )

    assert last_run["nextTrigger"] == SaasSecurityEventCollector.NEXT_TRIGGER_VALUE
    assert last_run["consecutive_backlog_cycles"] == 1


def test_handle_fetch_events_backlog_counter_accumulates(mocker):
    """
    Given a backlog and a prior consecutive_backlog_cycles in last_run,
    when handling fetch-events,
    then the counter is incremented (accumulates across back-to-back cycles).
    """
    demisto_mock = mocker.patch.object(SaasSecurityEventCollector, "demisto")
    demisto_mock.getLastRun.return_value = {"consecutive_backlog_cycles": 3}
    demisto_mock.getIntegrationContext.return_value = {}
    _patch_fetch_result(mocker, queue_drained=False)

    last_run = SaasSecurityEventCollector.handle_fetch_events(
        params=BASE_PARAMS, max_iterations=900, concurrency=10, pass_over_empty_response=True
    )

    assert last_run["consecutive_backlog_cycles"] == 4


@pytest.mark.parametrize(
    "prior_cycles, expect_warning",
    [
        (SaasSecurityEventCollector.BACKLOG_WARNING_THRESHOLD - 2, False),  # below threshold -> silent
        (SaasSecurityEventCollector.BACKLOG_WARNING_THRESHOLD - 1, True),  # reaches threshold -> warn
        (SaasSecurityEventCollector.BACKLOG_WARNING_THRESHOLD, False),  # one past threshold -> silent
        (
            SaasSecurityEventCollector.BACKLOG_WARNING_THRESHOLD + SaasSecurityEventCollector.BACKLOG_WARNING_INTERVAL - 1,
            True,
        ),  # exactly one interval later -> re-warn
    ],
)
def test_handle_fetch_events_backlog_warning_threshold_and_interval(mocker, prior_cycles, expect_warning):
    """
    Given a sustained backlog, when handling fetch-events,
    then demisto.error (the high-visibility backlog warning) fires exactly at BACKLOG_WARNING_THRESHOLD and
    then only every BACKLOG_WARNING_INTERVAL cycles, and stays silent otherwise (no log flooding).
    """
    demisto_mock = mocker.patch.object(SaasSecurityEventCollector, "demisto")
    demisto_mock.getLastRun.return_value = {"consecutive_backlog_cycles": prior_cycles}
    demisto_mock.getIntegrationContext.return_value = {}
    _patch_fetch_result(mocker, queue_drained=False)

    SaasSecurityEventCollector.handle_fetch_events(
        params=BASE_PARAMS, max_iterations=900, concurrency=10, pass_over_empty_response=True
    )

    backlog_warning_emitted = any(
        "ingestion backlog" in str(call.args[0]) for call in demisto_mock.error.call_args_list if call.args
    )
    assert backlog_warning_emitted is expect_warning


def test_handle_fetch_events_send_failure_stashes_unsent_and_forces_retry(mocker):
    """
    Given a send failure mid-drain that returns unsent events,
    when handling fetch-events,
    then the unsent events are persisted via setIntegrationContext and an immediate retry is forced
    (nextTrigger set), even though the concurrent drain reported queue_drained=True.
    """
    demisto_mock = mocker.patch.object(SaasSecurityEventCollector, "demisto")
    demisto_mock.getLastRun.return_value = {}
    demisto_mock.getIntegrationContext.return_value = {}
    unsent = [{"id": i} for i in range(500)]
    _patch_fetch_result(mocker, fetched=2500, sent=2000, queue_drained=True, unsent=unsent, exception=ValueError("send failed"))

    last_run = SaasSecurityEventCollector.handle_fetch_events(
        params=BASE_PARAMS, max_iterations=900, concurrency=10, pass_over_empty_response=True
    )

    demisto_mock.setIntegrationContext.assert_called_once_with({"events": unsent})
    assert last_run["nextTrigger"] == SaasSecurityEventCollector.NEXT_TRIGGER_VALUE
    assert last_run["consecutive_backlog_cycles"] == 1


def test_handle_fetch_events_stash_flushed_first_and_shrinks_next_cycle(mocker):
    """
    Given a stash of unsent events persisted from a prior failed cycle,
    when the next fetch-events cycle runs,
    then the stash is passed to the drain as pending_events (flushed first) and, on partial success, the
    re-stashed remainder is strictly smaller than the original stash (self-heals over cycles).
    """
    demisto_mock = mocker.patch.object(SaasSecurityEventCollector, "demisto")
    original_stash = [{"id": i} for i in range(1000)]
    demisto_mock.getLastRun.return_value = {}
    demisto_mock.getIntegrationContext.return_value = {"events": original_stash}

    # The drain flushes part of the stash, then fails again leaving a smaller remainder.
    shrunk_remainder = [{"id": i} for i in range(400)]
    fetch_mock = _patch_fetch_result(
        mocker, fetched=1000, sent=600, queue_drained=True, unsent=shrunk_remainder, exception=ValueError("boom")
    )

    SaasSecurityEventCollector.handle_fetch_events(
        params=BASE_PARAMS, max_iterations=900, concurrency=10, pass_over_empty_response=True
    )

    # The prior stash was handed to the drain to be flushed first.
    assert fetch_mock.call_args.kwargs["pending_events"] == original_stash
    # The re-stashed remainder shrank -> the stash self-heals instead of replaying forever.
    persisted = demisto_mock.setIntegrationContext.call_args.args[0]
    assert 0 < len(persisted["events"]) < len(original_stash)


# ---------------------------------------------------------------------------
# get_max_fetch (unchanged behavior, kept for coverage)
# ---------------------------------------------------------------------------


def test_get_max_fetch_default():
    assert SaasSecurityEventCollector.get_max_fetch(None) == 1000


def test_get_max_fetch_clamped_and_rounded():
    assert SaasSecurityEventCollector.get_max_fetch(99999) == 5000  # clamp to MAX_LIMIT
    assert SaasSecurityEventCollector.get_max_fetch(105) == 100  # round down to multiple of 10


def test_get_max_fetch_negative_number():
    with pytest.raises(DemistoException):
        SaasSecurityEventCollector.get_max_fetch(-1)