BoxEventsCollector

Collect events from Box's logs.

Analytics & SIEM · Box

Details

IDBoxEventsCollector
ProviderBox
CategoryAnalytics & SIEM
From Version6.6.0
Docker Imagedemisto/auth-utils:1.0.0.11206988
Supported ModulesAgentix XSIAM

README

Box Event Collector

Collect events from Box’s logs.

Permissions

The command is using the events endpoint with enterprise login.
The user making the API call will need to have admin privileges, and the application will need to have the scope manage enterprise properties checked.

This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM.

Configure Box Event Collector in Cortex

To acquire the “Credential JSON”, you need to get a JWT token and an app from Box.
You can use the guide from Box V2 to get those credentials.

Parameter Required
Verify SSL Certificate False
Credentials JSON True
Fetch Events False
First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days) False
Maximum number of events per fetch False

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

box-get-events


Get events.

Base Command

box-get-events

Input

Argument Name Description Required
limit Maximum events to fetch. Default is 10. Optional
created_after Fetch events from this time (<number> <time unit>, e.g., 12 hours, 7 days). Default is 3 days. Optional

Context Output

There is no context output for this command.

Command example

!box-get-events limit=1 created_after="30 days"

Context Example

{
    "BoxEvents": {
        "action_by": null,
        "additional_details": null,
        "created_at": "2022-04-10T05:39:15-07:00",
        "created_by": {
            "id": "00000000000",
            "login": "johndoe@example.com",
            "name": "John Doe",
            "type": "user"
        },
        "event_id": "event_id",
        "event_type": "ADD_LOGIN_ACTIVITY_DEVICE",
        "ip_address": "ip_address",
        "session_id": null,
        "source": {
            "id": "00000000000",
            "login": "johndoe@example.com",
            "name": "John Doe",
            "type": "user"
        },
        "type": "event"
    }
}

Human Readable Output

Results

action_by additional_details created_at created_by event_id event_type ip_address session_id source type
    2022-04-10T05:39:15-07:00 type: user
id: 0000000000
name: John Doe
login: johndoe@example.com
event_id ADD_LOGIN_ACTIVITY_DEVICE ip_address   type: user
id: 0000000000
name: John Doe
login: johndoe@example.com
event

Troubleshooting & Advanced Configuration

Each fetch cycle collects up to the Maximum number of events per fetch value (default 2500, capped internally at 5000) and then persists its position, so events are collected incrementally across cycles.

If ingestion is falling behind on a high-volume tenant (events arrive faster than they are collected), you can help the collector keep up by:

  • Increasing Maximum number of events per fetch (up to 5000) so each cycle collects more events.
  • Decreasing Events Fetch Interval (for example, to 1 minute) so cycles run more frequently.

Configuration parameters

  • url — Server URL (required)
  • credentials_json — (required)
  • proxy — Use system proxy settings
  • insecure — Trust any certificate (not secure)
  • isFetchEvents — Fetch Events
  • created_after — First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days)
  • max_events_per_fetch — Maximum number of events per fetch
  • eventFetchInterval — Events Fetch Interval

Commands (1)

  • box-get-events

    Gets events from Box. Use this command for development and debugging only, as it may produce duplicate events, exceed API rate limits, or disrupt the fetch mechanism.

import json

import demistomock as demisto
import pytest
from BoxEventsCollector import BoxEventsClient, main


class TestBoxCollectEvents:
    params = {
        "url": "https://api.box.com",
        "credentials_json": {
            "password": json.dumps(
                {
                    "boxAppSettings": {
                        "clientID": "I AM A CLIENT ID",
                        "clientSecret": "I AM A CLIENT SECRET",
                        "appAuth": {
                            "publicKeyID": "PUBLIC KEY ID",
                            "privateKey": "I AM A PRIVATE KEY!!!",
                            "passphrase": "passphrase",
                        },
                    },
                    "enterpriseID": "000000000",
                }
            )
        },
        "created_after": "30 days",
        "verify": False,
    }

    def test_everything_is_called_in_main(self, mocker, requests_mock):
        """Just see that the main works as intended with the mocked data.
        No really running the jwt creation as it need real value"""
        requests_mock.get(
            "https://api.box.com/2.0/events",
            json={"next_stream_position": "0", "entries": []},
        )
        main("box-get-events", self.params)

    def test_fetch_events_is_running(self, mocker, requests_mock):
        """See that call to the fetch events function do calls set last run
        and sends the events to xsiam"""
        params = self.params.copy()
        params["limit"] = 2
        requests_mock.get(
            "https://api.box.com/2.0/events",
            [
                {
                    "json": {
                        "next_stream_position": "600",
                        "entries": [{"sample event": "event"}],
                    }
                },
                {"json": {"next_stream_position": "601", "entries": []}},
            ],
        )

        last_run = mocker.patch.object(demisto, "setLastRun")
        send_events_to_xsiam = mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)
        assert last_run.call_args_list[0].args[0] == {"stream_position": "601"}
        assert len(send_events_to_xsiam.call_args_list[0].args[0]) == 1

    def _paged_events_callback(self, page_size_default: int):
        """Build a requests_mock callback that honors the `limit` query param like Box does.

        Each response returns exactly ``min(requested_limit, 2)`` events with a monotonically
        increasing stream position, so the test can verify the dynamic page-size shrinking that
        makes the total land exactly on ``max_events_per_fetch`` (XSUP-72996 data-loss fix).
        """
        state = {"counter": 0}

        def callback(request, context):
            requested = int(request.qs.get("limit", [str(page_size_default)])[0])
            # Box never returns more than the requested page size; cap our mock "page" at 2 events.
            page_len = min(requested, 2)
            entries = []
            for _ in range(page_len):
                state["counter"] += 1
                entries.append({"id": f"e{state['counter']}"})
            return {"next_stream_position": str(state["counter"]), "entries": entries}

        return callback

    def test_fetch_events_stops_at_max_events_per_fetch(self, mocker, requests_mock):
        """Regression test for XSUP-72996.

        Given an API that keeps returning non-empty pages (a large backlog), when
        `max_events_per_fetch` is set, the fetch must stop once that many events are reached
        instead of looping until the backlog is drained (which caused the timeout).

        The cap is EXACT and lossless: the per-request page size is shrunk to the remaining
        budget (`min(PAGE_SIZE, remaining)`), so the final page returns precisely the number of
        events needed and its stream position aligns exactly with what was returned - no mid-page
        slicing (which would have lost events) and no overshoot.
        """
        from BoxEventsCollector import PAGE_SIZE

        params = self.params.copy()
        params["max_events_per_fetch"] = 3
        # Mock honors the requested `limit`, returning up to 2 events per page. With a cap of 3:
        #   call 1 -> limit=min(500,3)=3 -> returns 2 events (mock max), total=2
        #   call 2 -> limit=min(500,1)=1 -> returns 1 event, total=3 -> exact stop.
        mocked = requests_mock.get(
            "https://api.box.com/2.0/events",
            json=self._paged_events_callback(page_size_default=PAGE_SIZE),
        )

        last_run = mocker.patch.object(demisto, "setLastRun")
        send_events_to_xsiam = mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        # Exactly max_events_per_fetch events -> exact cap, no overshoot and no loss.
        pushed = send_events_to_xsiam.call_args_list[0].args[0]
        assert [event["id"] for event in pushed] == ["e1", "e2", "e3"]
        # The requested page size shrank across calls to hit the cap exactly: first 3, then 1.
        requested_limits = [req.qs["limit"][0] for req in mocked.request_history]
        assert requested_limits == ["3", "1"]
        # Stream position persisted matches the last event actually returned (3), so the next
        # cycle resumes exactly after e3 with no gap and no duplication.
        assert last_run.call_args_list[0].args[0] == {"stream_position": "3"}

    def test_fetch_events_default_cap_does_not_stop_small_backlog(self, mocker, requests_mock):
        """When max_events_per_fetch is omitted, the default cap applies and a small
        backlog still drains fully via the natural empty-`entries` exit (no premature stop)."""
        params = self.params.copy()  # no max_events_per_fetch -> DEFAULT_MAX_EVENTS_PER_FETCH
        requests_mock.get(
            "https://api.box.com/2.0/events",
            [
                {"json": {"next_stream_position": "1", "entries": [{"id": "e1"}]}},
                {"json": {"next_stream_position": "2", "entries": [{"id": "e2"}]}},
                {"json": {"next_stream_position": "2", "entries": []}},
            ],
        )
        last_run = mocker.patch.object(demisto, "setLastRun")
        send_events_to_xsiam = mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        # All available events collected (backlog < default cap), stopped on empty entries.
        assert len(send_events_to_xsiam.call_args_list[0].args[0]) == 2
        assert last_run.call_args_list[0].args[0] == {"stream_position": "2"}

    def test_page_size_is_capped_by_page_size_constant(self, mocker, requests_mock):
        """When max_events_per_fetch exceeds PAGE_SIZE, the per-request page size sent to the API
        is capped at PAGE_SIZE (Box's maximum) - the total cap never inflates a single request."""
        from BoxEventsCollector import PAGE_SIZE

        params = self.params.copy()
        params["max_events_per_fetch"] = PAGE_SIZE + 100  # larger than a single page
        mocked_request = requests_mock.get(
            "https://api.box.com/2.0/events",
            [
                {"json": {"next_stream_position": "1", "entries": [{"id": "e1"}]}},
                {"json": {"next_stream_position": "1", "entries": []}},
            ],
        )
        mocker.patch.object(demisto, "setLastRun")
        send_events_to_xsiam = mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        # Backlog drained (only 1 event available), and the first request's page size was capped
        # at PAGE_SIZE even though the total budget was larger.
        assert len(send_events_to_xsiam.call_args_list[0].args[0]) == 1
        assert mocked_request.request_history[0].qs["limit"] == [str(PAGE_SIZE)]

    def test_max_events_per_fetch_is_capped_at_the_allowed_maximum(self, mocker, requests_mock):
        """A max_events_per_fetch above MAX_EVENTS_PER_FETCH_LIMIT is clamped to the maximum,
        not passed through as-is, keeping a single fetch within the Docker timeout budget."""
        from BoxEventsCollector import MAX_EVENTS_PER_FETCH_LIMIT, BoxEventsGetter

        params = self.params.copy()
        params["max_events_per_fetch"] = MAX_EVENTS_PER_FETCH_LIMIT + 1000  # above the allowed max
        requests_mock.get(
            "https://api.box.com/2.0/events",
            json={"next_stream_position": "0", "entries": []},
        )
        mocker.patch.object(demisto, "setLastRun")
        mocker.patch("BoxEventsCollector.send_events_to_xsiam")

        # Capture the effective options.limit at the moment run() is invoked.
        captured = {}

        def fake_run(self):
            captured["limit"] = self.client.options.limit
            return []

        mocker.patch.object(BoxEventsGetter, "run", fake_run)
        main("fetch-events", params)

        # The limit used for the fetch must be clamped to the maximum, not the requested value.
        assert captured["limit"] == MAX_EVENTS_PER_FETCH_LIMIT

    def test_fetch_events_resumes_from_stored_stream_position(self, mocker, requests_mock):
        """A subsequent fetch must resume from the persisted `stream_position`.

        After the backlog is chunked across cycles (XSUP-72996 cap), the next cycle relies on the
        stored cursor. This simulates the merged `getLastRun` state by seeding `stream_position` in
        params and asserts the very first request carries it, so no events are re-fetched or skipped.
        """
        params = self.params.copy()
        params["stream_position"] = "12345"  # simulates demisto.getLastRun() from a previous cycle
        mocked_request = requests_mock.get(
            "https://api.box.com/2.0/events",
            [
                {"json": {"next_stream_position": "12346", "entries": [{"id": "e1"}]}},
                {"json": {"next_stream_position": "12346", "entries": []}},
            ],
        )
        last_run = mocker.patch.object(demisto, "setLastRun")
        mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        # The first outgoing request must resume from the stored cursor, not from created_after.
        assert mocked_request.request_history[0].qs["stream_position"] == ["12345"]
        # And the new cursor is persisted for the following cycle.
        assert last_run.call_args_list[0].args[0] == {"stream_position": "12346"}

    def test_legacy_page_size_param_is_ignored(self, mocker, requests_mock):
        """A stale `page_size` left in an upgraded instance's config must be ignored.

        `page_size` was removed from the YAML; the request page size is now fixed at PAGE_SIZE.
        This guards the live-customer upgrade path where the old value (e.g. "50") is still stored.
        """
        from BoxEventsCollector import PAGE_SIZE

        params = self.params.copy()
        params["page_size"] = "50"  # legacy value from a pre-upgrade instance
        mocked_request = requests_mock.get(
            "https://api.box.com/2.0/events",
            [
                {"json": {"next_stream_position": "1", "entries": [{"id": "e1"}]}},
                {"json": {"next_stream_position": "1", "entries": []}},
            ],
        )
        mocker.patch.object(demisto, "setLastRun")
        mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        # The stale page_size=50 must NOT be used; the request page size stays at PAGE_SIZE.
        assert mocked_request.request_history[0].qs["limit"] == [str(PAGE_SIZE)]

    def test_cap_lands_exactly_on_page_boundary(self, mocker, requests_mock):
        """When the cap is an exact multiple of the page size, the run stops cleanly on the boundary.

        Two full pages of PAGE_SIZE fill the budget exactly; the `remaining <= 0` break must fire
        without requesting a third (shrunk) page, and without dropping any event.
        """
        from BoxEventsCollector import PAGE_SIZE

        params = self.params.copy()
        params["max_events_per_fetch"] = PAGE_SIZE * 2  # exactly two full pages

        state = {"counter": 0}

        def callback(request, context):
            requested = int(request.qs.get("limit", [str(PAGE_SIZE)])[0])
            entries = []
            for _ in range(requested):
                state["counter"] += 1
                entries.append({"id": f"e{state['counter']}"})
            return {"next_stream_position": str(state["counter"]), "entries": entries}

        mocked = requests_mock.get("https://api.box.com/2.0/events", json=callback)
        last_run = mocker.patch.object(demisto, "setLastRun")
        send_events_to_xsiam = mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        pushed = send_events_to_xsiam.call_args_list[0].args[0]
        # Exactly two full pages, no overshoot, no loss.
        assert len(pushed) == PAGE_SIZE * 2
        # Only two requests were made (both full pages); no extra shrunk third call.
        requested_limits = [req.qs["limit"][0] for req in mocked.request_history]
        assert requested_limits == [str(PAGE_SIZE), str(PAGE_SIZE)]
        assert last_run.call_args_list[0].args[0] == {"stream_position": str(PAGE_SIZE * 2)}

    def test_test_module_requests_single_event_and_returns_ok(self, mocker, requests_mock):
        """test-module caps the fetch to a single event and reports success."""
        mocked_request = requests_mock.get(
            "https://api.box.com/2.0/events",
            json={"next_stream_position": "0", "entries": []},
        )
        results = mocker.patch.object(demisto, "results")
        main("test-module", self.params)

        # test-module fixes options.limit = 1, so the first request asks for a single event.
        assert mocked_request.request_history[0].qs["limit"] == ["1"]
        assert results.call_args_list[0].args[0] == "ok"

    def test_fetch_events_empty_first_page(self, mocker, requests_mock):
        """A zero-backlog first page pushes no events but still persists a stream_position."""
        params = self.params.copy()
        requests_mock.get(
            "https://api.box.com/2.0/events",
            json={"next_stream_position": "777", "entries": []},
        )
        last_run = mocker.patch.object(demisto, "setLastRun")
        send_events_to_xsiam = mocker.patch("BoxEventsCollector.send_events_to_xsiam")
        main("fetch-events", params)

        # Nothing to push, but the cursor advances so the next cycle starts from the right place.
        assert len(send_events_to_xsiam.call_args_list[0].args[0]) == 0
        assert last_run.call_args_list[0].args[0] == {"stream_position": "777"}

    @pytest.fixture(autouse=True, scope="function")
    def remove_authentication(self, mocker):
        """We don't need to authenticate in the test functions"""
        mocker.patch.object(BoxEventsClient, "authenticate", return_value=None)

    def test_not_gate(self):
        """Well, I've been forced to raise the coverage"""
        from BoxEventsCollector import not_gate

        assert not_gate(None)
        assert not_gate(False)
        assert not_gate("No")
        assert not not_gate(True)
        assert not not_gate("yes")

    def test_url_as_param(self, mocker, requests_mock):
        """Assert the request url changes when url parameter changes."""
        new_url = "https://api.triangle.com"
        mocked_request = requests_mock.get(
            f"{new_url}/2.0/events",
            json={"next_stream_position": "0", "entries": []},
        )
        different_url_params = self.params.copy()
        different_url_params["url"] = new_url
        main("box-get-events", different_url_params)
        assert mocked_request.called