Details
| ID | BoxEventsCollector |
|---|---|
| Provider | Box |
| Category | Analytics & SIEM |
| From Version | 6.6.0 |
| Docker Image | demisto/auth-utils:1.0.0.11206988 |
| Supported Modules | Agentix XSIAM |
README
Box Event Collector
Collect events from Box’s logs.
Permissions
The command is using the events endpoint with enterprise login.
The user making the API call will need to have admin privileges, and the application will need to have the scope manage enterprise properties checked.
This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM.
Configure Box Event Collector in Cortex
To acquire the “Credential JSON”, you need to get a JWT token and an app from Box.
You can use the guide from Box V2 to get those credentials.
| Parameter | Required |
|---|---|
| Verify SSL Certificate | False |
| Credentials JSON | True |
| Fetch Events | False |
| First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days) | False |
| Maximum number of events per fetch | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
box-get-events
Get events.
Base Command
box-get-events
Input
| Argument Name | Description | Required |
|---|---|---|
| limit | Maximum events to fetch. Default is 10. | Optional |
| created_after | Fetch events from this time (<number> <time unit>, e.g., 12 hours, 7 days). Default is 3 days. | Optional |
Context Output
There is no context output for this command.
Command example
!box-get-events limit=1 created_after="30 days"
Context Example
{
"BoxEvents": {
"action_by": null,
"additional_details": null,
"created_at": "2022-04-10T05:39:15-07:00",
"created_by": {
"id": "00000000000",
"login": "johndoe@example.com",
"name": "John Doe",
"type": "user"
},
"event_id": "event_id",
"event_type": "ADD_LOGIN_ACTIVITY_DEVICE",
"ip_address": "ip_address",
"session_id": null,
"source": {
"id": "00000000000",
"login": "johndoe@example.com",
"name": "John Doe",
"type": "user"
},
"type": "event"
}
}
Human Readable Output
Results
action_by additional_details created_at created_by event_id event_type ip_address session_id source type 2022-04-10T05:39:15-07:00 type: user
id: 0000000000
name: John Doe
login: johndoe@example.comevent_id ADD_LOGIN_ACTIVITY_DEVICE ip_address type: user
id: 0000000000
name: John Doe
login: johndoe@example.comevent
Troubleshooting & Advanced Configuration
Each fetch cycle collects up to the Maximum number of events per fetch value (default 2500, capped internally at 5000) and then persists its position, so events are collected incrementally across cycles.
If ingestion is falling behind on a high-volume tenant (events arrive faster than they are collected), you can help the collector keep up by:
- Increasing Maximum number of events per fetch (up to
5000) so each cycle collects more events. - Decreasing Events Fetch Interval (for example, to 1 minute) so cycles run more frequently.
Configuration parameters
url— Server URL (required)credentials_json— (required)proxy— Use system proxy settingsinsecure— Trust any certificate (not secure)isFetchEvents— Fetch Eventscreated_after— First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days)max_events_per_fetch— Maximum number of events per fetcheventFetchInterval— Events Fetch Interval
Commands (1)
-
box-get-eventsGets events from Box. Use this command for development and debugging only, as it may produce duplicate events, exceed API rate limits, or disrupt the fetch mechanism.
# pylint: disable=no-name-in-module # pylint: disable=no-self-argument import secrets import dateparser import jwt from cryptography import exceptions from cryptography.hazmat.backends import default_backend from cryptography.hazmat.primitives.serialization import load_pem_private_key from pydantic import ConfigDict, Field, parse_obj_as from SiemApiModule import * # noqa: E402 VENDOR = "box" PRODUCT = "box" DEFAULT_MAX_EVENTS_PER_FETCH = 2500 MAX_EVENTS_PER_FETCH_LIMIT = 5000 PAGE_SIZE = 500 class Claims(BaseModel): iss: str = Field(alias="client_id") sub: str = Field(alias="id", description="user id or enterprise id") box_sub_type: str = "enterprise" aud: AnyUrl jti: str = secrets.token_hex(64) exp: int = round(time.time()) + 45 class AppAuth(BaseModel): publicKeyID: str privateKey: str passphrase: str class BoxAppSettings(BaseModel): clientID: str clientSecret: str appAuth: AppAuth model_config = ConfigDict(arbitrary_types_allowed=True) class BoxCredentials(BaseModel): enterpriseID: str boxAppSettings: BoxAppSettings class MyConfig: validate_assignment = True def get_box_events_timestamp_format(value): """Converting int(epoch), str(3 days) or datetime to Box's api time""" timestamp: Optional[datetime] if isinstance(value, int): value = str(value) if not isinstance(value, datetime): timestamp = dateparser.parse(value) if timestamp is None: raise TypeError(f"after is not a valid time {value}") return timestamp.isoformat("T", "seconds") class BoxEventsParams(BaseModel): event_type: Optional[str] = None # `limit` is the Box /events `limit` query param (page size). Fixed at PAGE_SIZE; not user-configurable. limit: int = PAGE_SIZE stream_position: Optional[str] = None stream_type: str = "admin_logs" created_after: Optional[str] # validators _normalize_after = validator("created_after", pre=True, allow_reuse=True)(get_box_events_timestamp_format) # type: ignore[type-var] model_config = ConfigDict(validate_assignment=True) def not_gate(v): """Due to a bug in the validator object (collision with CommonServerPython) we can pass this a a simple lambda. So here we are. Just doing not if v is bool, else it is true. Used when getting insecure and should change the insecure to verify. insecure == True, verify is False. """ v_ = parse_obj_as(bool, False if v is None else v) return not v_ class BoxEventsRequestConfig(IntegrationHTTPRequest): # Endpoint: https://developer.box.com/reference/get-events/ method: Method = Method.GET params: BoxEventsParams # type: ignore[assignment] verify: Optional[bool] = Field(True, alias="insecure") # type: ignore[assignment] # validators _oppsite_verify = validator("verify", allow_reuse=True)(not_gate) # type: ignore[type-var] class BoxIntegrationOptions(IntegrationOptions): product_name: str = PRODUCT vendor_name: str = VENDOR should_push_events: bool = False class BoxEventsClient(IntegrationEventsClient): request: BoxEventsRequestConfig options: IntegrationOptions def __init__( self, request: BoxEventsRequestConfig, options: IntegrationOptions, box_credentials: BoxCredentials, api_url: str, session: Optional[requests.Session] = None, ) -> None: self.api_url: str = api_url self.authorization_url = parse_obj_as(AnyUrl, urljoin(str(self.api_url), "/oauth2/token")) if session is None: session = requests.Session() self.box_credentials = box_credentials super().__init__(request, options, session) def set_request_filter(self, after: str): self.request.params.stream_position = after def authenticate(self): request = IntegrationHTTPRequest( method=Method.POST, url=self.authorization_url, data=self._create_authorization_body(), verify=self.request.verify, # type: ignore[arg-type] ) response = self.call(request) self.access_token = response.json()["access_token"] self.request.headers = {"Authorization": f"Bearer {self.access_token}"} def _create_authorization_body(self): claims = Claims( client_id=self.box_credentials.boxAppSettings.clientID, id=self.box_credentials.enterpriseID, aud=self.authorization_url, ) decrypted_private_key = _decrypt_private_key(self.box_credentials.boxAppSettings.appAuth) assertion = jwt.encode( payload=claims.model_dump(mode="json"), # type: ignore[attr-defined] key=decrypted_private_key, algorithm="RS512", headers={"kid": self.box_credentials.boxAppSettings.appAuth.publicKeyID}, ) body = { "grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer", "assertion": assertion, "client_id": self.box_credentials.boxAppSettings.clientID, "client_secret": self.box_credentials.boxAppSettings.clientSecret, } return body class BoxEventsGetter(IntegrationGetEvents): client: BoxEventsClient def run(self): """Collect events, capping the total at ``options.limit`` without losing data. The base ``IntegrationGetEvents.run`` slices the accumulated list to ``options.limit`` (``stored[: limit]``) *after* the page's ``stream_position`` has already been advanced. Persisting that advanced position while discarding the sliced-off tail causes permanent data loss (XSUP-72996). To cap the total *exactly* while keeping the persisted ``stream_position`` aligned with the events actually returned, we shrink the page size of each request to the remaining budget (``min(PAGE_SIZE, remaining)``). Box then ends the final page precisely on the limit and returns a valid ``next_stream_position`` for it, so no events are dropped and the next fetch resumes exactly where this one stopped. """ stored: list = [] start_time = time.time() params = self.client.request.params demisto.debug( f"[Fetch Events] run() START | total_cap(options.limit)={self.options.limit} | " f"start_stream_position={params.stream_position!r} | created_after={params.created_after!r} | " f"stream_type={params.stream_type!r} | initial_page_size(limit)={params.limit}" ) # Seed the first request's page size to the budget too, so a limit smaller than a full page # (e.g. limit < PAGE_SIZE) is respected on the very first call, not just on later pages. if self.options.limit: self.client.request.params.limit = min(PAGE_SIZE, self.options.limit) demisto.debug( f"[Fetch Events] seeded first-page size to min(PAGE_SIZE={PAGE_SIZE}, " f"cap={self.options.limit}) -> {self.client.request.params.limit}" ) page_number = 0 for logs in self._iter_events(): page_number += 1 page_len = len(logs) stored.extend(logs) demisto.debug( f"[Fetch Events] page #{page_number} received {page_len} events | " f"running_total={len(stored)} | requested_page_size={self.client.request.params.limit} | " f"stream_position_now={self.client.request.params.stream_position!r}" ) if self.options.limit: remaining = self.options.limit - len(stored) if remaining <= 0: demisto.debug( f"[Fetch Events] total cap reached: options.limit={self.options.limit}, " f"collected={len(stored)} over {page_number} page(s); stopping." ) break # Shrink the next request so the last page returns exactly the remaining budget, # making the total land precisely on the limit with a valid stream position. self.client.request.params.limit = min(PAGE_SIZE, remaining) demisto.debug( f"[Fetch Events] {remaining=} events left in budget; next request page size set to " f"{self.client.request.params.limit}" ) elapsed = round(time.time() - start_time, 2) demisto.debug( f"[Fetch Events] run() END | collected={len(stored)} events over {page_number} page(s) in " f"{elapsed}s | final_stream_position={self.client.request.params.stream_position!r}" ) return stored def get_last_run(self: Any) -> dict: # type: ignore stream_position = self.client.request.params.stream_position demisto.debug(f"[Last Run] persisting {stream_position=!r} as the next cycle's starting cursor") return {"stream_position": stream_position} def _iter_events(self): self.client.authenticate() demisto.debug("authenticated successfully") call_number = 0 # region First Call params = self.client.request.params demisto.debug( f"[Iter Events] first API call -> GET {self.client.request.url} | " f"stream_position={params.stream_position!r} | created_after={params.created_after!r} | " f"stream_type={params.stream_type!r} | limit(page_size)={params.limit}" ) call_number += 1 events = self.client.call(self.client.request).json() demisto.debug( f"[Iter Events] first response: entries={len(events.get('entries', []))} | " f"next_stream_position={events.get('next_stream_position')!r}" ) # endregion # region Yield Response while True: # Run as long there are logs self.client.set_request_filter(events["next_stream_position"]) # The next stream position points to where new messages will arrive. demisto.debug(f'setting the next request filter {events["next_stream_position"]=}') if not events["entries"]: demisto.debug( f"[Iter Events] empty page after {call_number} call(s) -> backlog drained; " f"stopping at next_stream_position={events['next_stream_position']!r}" ) break yield events["entries"] # endregion # region Do next call call_number += 1 demisto.debug( f"[Iter Events] API call #{call_number} | stream_position=" f"{self.client.request.params.stream_position!r} | limit(page_size)=" f"{self.client.request.params.limit}" ) events = self.client.call(self.client.request).json() demisto.debug( f"[Iter Events] call #{call_number} response: entries={len(events.get('entries', []))} | " f"next_stream_position={events.get('next_stream_position')!r}" ) # endregion def _decrypt_private_key(app_auth: AppAuth): """ Attempts to load the private key as given in the integration configuration. :return: an initialized Private key object. """ try: key = load_pem_private_key( data=app_auth.privateKey.encode("utf8"), password=app_auth.passphrase.encode("utf8"), backend=default_backend(), ) except ( TypeError, ValueError, exceptions.UnsupportedAlgorithm, ) as exception: raise DemistoException("An error occurred while loading the private key.", exception) return key def main(command: str, demisto_params: dict): try: box_credentials = BoxCredentials.parse_raw(demisto_params["credentials_json"]["password"]) events_request_params = demisto_params.copy() events_request_params["url"] = urljoin(demisto_params.get("url", "https://api.box.com"), "/2.0/events") request = BoxEventsRequestConfig( params=BoxEventsParams.parse_obj(events_request_params), **events_request_params, ) options = BoxIntegrationOptions.parse_obj(demisto_params) client = BoxEventsClient(request, options, box_credentials, api_url=demisto_params.get("url", "https://api.box.com")) get_events = BoxEventsGetter(client, options) if command == "test-module": get_events.client.options.limit = 1 get_events.run() demisto.results("ok") return if command == "fetch-events": # Cap total events per fetch so a single run can't chase the whole backlog and time out. max_events_per_fetch = arg_to_number(demisto_params.get("max_events_per_fetch")) or DEFAULT_MAX_EVENTS_PER_FETCH if max_events_per_fetch > MAX_EVENTS_PER_FETCH_LIMIT: demisto.debug( f"[Fetch Events] 'Maximum number of events per fetch' ({max_events_per_fetch}) exceeds the " f"allowed maximum; capping it to {MAX_EVENTS_PER_FETCH_LIMIT}." ) max_events_per_fetch = MAX_EVENTS_PER_FETCH_LIMIT get_events.client.options.limit = max_events_per_fetch demisto.debug(f"[Fetch Events] total cap set to {max_events_per_fetch=}") demisto.debug(f"[Main] executing {command=} against {options.vendor_name}/{options.product_name}") events = get_events.run() demisto.debug(f"[Main] run() returned {len(events)} events for {command=}") if command == "box-get-events": demisto.debug(f"[Main] box-get-events: publishing {len(events)} events to war room") return_results(CommandResults("BoxEvents", "event_id", events)) if options.should_push_events: demisto.debug(f"[Main] box-get-events: pushing {len(events)} events to XSIAM (should_push_events=True)") send_events_to_xsiam(events, options.vendor_name, options.product_name) if command == "fetch-events": last_run = get_events.get_last_run() demisto.debug(f"[Main] fetch-events: pushing {len(events)} events to XSIAM and setting {last_run=}") send_events_to_xsiam(events, options.vendor_name, options.product_name) demisto.setLastRun(last_run) demisto.debug(f"[Main] finished {command=} successfully. total_events={len(events)}") except Exception as e: demisto.error(f"[Main] Failed to execute {command=}: {e}\nTraceback:{traceback.format_exc()}") return_error(f"Failed to execute {command} command.\nError:\n{e}\nTraceback:{traceback.format_exc()}") if __name__ in ("__main__", "__builtin__", "builtins"): # Args is always stronger. Get getLastRun even stronger demisto_params_ = demisto.params() | demisto.args() | demisto.getLastRun() main(demisto.command(), demisto_params_)