Azure Storage Queue
Create and Manage Azure Storage Queues and Messages.
IT Services · Azure Storage Queue
Details
| ID | Azure Storage Queue |
|---|---|
| Provider | Microsoft |
| Category | IT Services |
| From Version | 6.0.0 |
| Docker Image | demisto/python3:3.12.13.10116658 |
| Supported Modules | Agentix XSIAM |
README
Azure Storage Queue
Create and Manage Azure Storage Queues and Messages.
This integration was integrated and tested with version “2020-10-02” of Azure Storage Queue
Configure Azure Storage Queue in Cortex
| Parameter | Description | Required |
|---|---|---|
| Storage account name | True | |
| Account SAS Token | False | |
| Use Azure Managed Identities | Relevant only if the integration is running on Azure VM. If selected, authenticates based on the value provided for the Azure Managed Identities Client ID field. If no value is provided for the Azure Managed Identities Client ID field, authenticates based on the System Assigned Managed Identity. For additional information, see the Help tab. | False |
| Azure Managed Identities Client ID | The Managed Identities client ID for authentication - relevant only if the integration is running on Azure VM. | False |
| Use system proxy settings | False | |
| Trust any certificate (not secure) | False | |
| Maximum incidents for one fetch. | Default is 10. Maximum is 32 (due to an API limit). | False |
| Queue name | The name of the queue from which the messages will be retrieved. | False |
| Incident type | False | |
| Fetch incidents | False |
Shared Access Signatures (SAS) Permissions
In order to use the integration use-cases,
please make sure your SAS token contains the following permissions:
- ‘Queue’ service.
- ‘Service’ and ‘Object’ resource types.
- ‘Read’, ‘Write’, ‘Delete’, ‘List’, ‘Create’, ‘Add’, ‘Update’, ‘Process’ and ‘Immutable storage’ permissions.
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
azure-storage-queue-list
List queues in storage account.
Base Command
azure-storage-queue-list
Input
| Argument Name | Description | Required |
|---|---|---|
| limit | Number of queues to retrieve. Default is 50. Default is 50. | Optional |
| prefix | Filters the results to return only queues with names that begin with the specified prefix. | Optional |
| page | Page number. Default is 1. Default is 1. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AzureStorageQueue.Queue.name | String | Queue name. |
Command Example
!azure-storage-queue-list limit="2" prefix="xs"
Context Example
{
"AzureStorageQueue": {
"Queue": [
{
"name": "xsoar-new-test"
},
{
"name": "xsoar-test"
}
]
}
}
Human Readable Output
Queues List
Current page size: 2
Showing page 1 out others that may exist
Name xsoar-new-test xsoar-test
azure-storage-queue-create
Create new queue in storage account.
Base Command
azure-storage-queue-create
Input
| Argument Name | Description | Required |
|---|---|---|
| queue_name | The name of the Queue to create. Rules for naming queues can be found here: https://docs.microsoft.com/en-us/rest/api/storageservices/naming-queues-and-metadata. | Required |
Context Output
There is no context output for this command.
Command Example
!azure-storage-queue-create queue_name="xsoar-test"
Human Readable Output
Queue xsoar-test successfully created.
azure-storage-queue-delete
Delete queue from storage account.
Base Command
azure-storage-queue-delete
Input
| Argument Name | Description | Required |
|---|---|---|
| queue_name | The name of the Queue to delete. | Required |
Context Output
There is no context output for this command.
Command Example
!azure-storage-queue-delete queue_name="xsoar-test"
Human Readable Output
Queue xsoar-test successfully deleted.
azure-storage-queue-message-create
Add a new message to the back of the queue.
Base Command
azure-storage-queue-message-create
Input
| Argument Name | Description | Required |
|---|---|---|
| message_content | The text content of the new message. | Required |
| queue_name | The name of the Queue. | Required |
| base64_encoding | Indicates whether the message should be encoded or not. Default is ‘False’. Possible values are: False, True. Default is False. | Optional |
| visibility_time_out | Specifies the new visibility timeout value, in seconds, relative to server time. Must be larger than or equal to 0, and cannot be larger than 7 days. The visibility timeout of a message cannot be set to a value later than the expiry time. Default is 0. |
Optional |
| expiration | Specifies the time-to-live (expiration) interval for the message, in seconds. the maximum time-to-live can be any positive number, as well as -1 indicating that the message does not expire. Default expiration time is 7 days. |
Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AzureStorageQueue.Queue.Message.MessageId | String | Message ID. |
| AzureStorageQueue.Queue.Message.InsertionTime | Date | Message insertion time. |
| AzureStorageQueue.Queue.Message.ExpirationTime | Date | Message expiration time. |
| AzureStorageQueue.Queue.Message.PopReceipt | String | Message pop receipt value. This value changes each time the message is retrieved or updated and used to ensure that message has not been dequeued by another user before deleting or updating the message. |
| AzureStorageQueue.Queue.Message.TimeNextVisible | Date | Message next visible time. |
| AzureStorageQueue.Queue.name | String | Queue name. |
Command Example
!azure-storage-queue-message-create message_content="test demo" queue_name="xsoar-test" base64_encoding="True"
Context Example
{
"AzureStorageQueue": {
"Queue": {
"Message": {
"ExpirationTime": "2021-12-05T11:55:31",
"InsertionTime": "2021-11-28T11:55:31",
"MessageId": "0d579602-222f-4256-9003-8772f0d65399",
"PopReceipt": "AgAAAAMAAAAAAAAAN734107k1wE=",
"TimeNextVisible": "2021-11-28T11:55:31"
},
"name": "xsoar-test"
}
}
}
Human Readable Output
xsoar-test Queue message
Message Id Expiration Time Insertion Time Time Next Visible Pop Receipt 0d579602-222f-4256-9003-8772f0d65399 2021-12-05T11:55:31 2021-11-28T11:55:31 2021-11-28T11:55:31 AgAAAAMAAAAAAAAAN734107k1wE=
azure-storage-queue-message-get
Retrieves messages from the front of the queue. Retrieved messages will move to the end of the queue,and will be visible after ‘TimeNextVisible’ param.
Base Command
azure-storage-queue-message-get
Input
| Argument Name | Description | Required |
|---|---|---|
| limit | Number of messages to retrieve. Default is 1, maximum is 32. Default is 1. | Optional |
| queue_name | The name of the Queue. | Required |
| visibility_time_out | Specifies the new visibility timeout value, in seconds, relative to server time. The default value is 30 seconds. A specified value must be larger than or equal to 1. |
Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AzureStorageQueue.Queue.Message.MessageId | String | Message ID. |
| AzureStorageQueue.Queue.Message.InsertionTime | Date | Message insertion time. |
| AzureStorageQueue.Queue.Message.ExpirationTime | Date | Message expiration time. |
| AzureStorageQueue.Queue.Message.PopReceipt | String | Message pop receipt value. This value changes each time the message is retrieved or updated and used to ensure that message has not been dequeued by another user before deleting or updating the message. |
| AzureStorageQueue.Queue.Message.TimeNextVisible | Date | Message next visible time. |
| AzureStorageQueue.Queue.Message.MessageText | String | Message text content. |
| AzureStorageQueue.Queue.name | String | Queue name. |
| AzureStorageQueue.Queue.Message.DequeueCount | Number | Indicates how many times a message has been retrieved. |
Command Example
!azure-storage-queue-message-get limit="1" queue_name="xsoar-test"
Context Example
{
"AzureStorageQueue": {
"Queue": {
"Message": [
{
"DequeueCount": "1",
"ExpirationTime": "2021-12-05T11:55:31",
"InsertionTime": "2021-11-28T11:55:31",
"MessageId": "0d579602-222f-4256-9003-8772f0d65399",
"MessageText": "test demo",
"PopReceipt": "AgAAAAMAAAAAAAAAziUx7U7k1wE=",
"TimeNextVisible": "2021-11-28T11:56:06"
}
],
"name": "xsoar-test"
}
}
}
Human Readable Output
xsoar-test Queue messages
Message Text Message Id Pop Receipt Dequeue Count Expiration Time Insertion Time Time Next Visible test demo 0d579602-222f-4256-9003-8772f0d65399 AgAAAAMAAAAAAAAAziUx7U7k1wE= 1 2021-12-05T11:55:31 2021-11-28T11:55:31 2021-11-28T11:56:06
azure-storage-queue-message-peek
Retrieves messages from the front of the queue. The command does not alter the visibility of the message.
Base Command
azure-storage-queue-message-peek
Input
| Argument Name | Description | Required |
|---|---|---|
| limit | Number of messages to retrieve. Default is 1, maximum is 32. Default is 1. | Optional |
| queue_name | The name of the Queue. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| AzureStorageQueue.Queue.Message.MessageId | String | Message ID. |
| AzureStorageQueue.Queue.Message.InsertionTime | Date | Message insertion time. |
| AzureStorageQueue.Queue.Message.ExpirationTime | Date | Message expiration time. |
| AzureStorageQueue.Queue.Message.MessageText | String | Message text content. |
| AzureStorageQueue.Queue.name | String | Queue name. |
| AzureStorageQueue.Queue.Message.DequeueCount | Number | Indicates how many times a message has been retrieved. |
Command Example
!azure-storage-queue-message-peek limit="1" queue_name="xsoar-test"
Context Example
{
"AzureStorageQueue": {
"Queue": {
"Message": [],
"name": "xsoar-test"
}
}
}
Human Readable Output
xsoar-test Queue messages
No entries.
azure-storage-queue-message-dequeue
Dequeue a message from the front of the queue.
Base Command
azure-storage-queue-message-dequeue
Input
| Argument Name | Description | Required |
|---|---|---|
| queue_name | The name of the Queue. | Required |
Context Output
There is no context output for this command.
Command Example
!azure-storage-queue-message-dequeue queue_name="xsoar-test"
Human Readable Output
There are no messages in xsoar-test queue.
azure-storage-queue-message-update
Update message content in the queue.
Base Command
azure-storage-queue-message-update
Input
| Argument Name | Description | Required |
|---|---|---|
| queue_name | The name of the Queue. | Required |
| message_content | New message content. | Required |
| message_id | The ID of the message to update. | Required |
| pop_receipt | Message PopReceipt parameter. This value changes each time the message is retrieved or updated and used to ensure that message has not been dequeued by another user before deleting or updating the message. | Required |
| base64_encoding | Indicates whether the message content should be encoded or not. Default is ‘False’. Possible values are: False, True. Default is False. | Optional |
| visibility_time_out | Specifies the new visibility timeout value of the message. The new value must be larger than or equal to 0, and cannot be larger than 7 days. The visibility timeout of a message cannot be set to a value later than the expiry time. Default is 0. Possible values are: . Default is 0. | Optional |
Context Output
There is no context output for this command.
Command Example
!azure-storage-queue-message-update queue_name="test-xsoar" message_content="new content" message_id="ea0db196-dad3-4c12-b845-dc6223739870" pop_receipt="AgAAAAMAAAAAAAAA0xNSmE7k1wE="
Human Readable Output
The message in test-xsoar successfully updated.
azure-storage-queue-message-delete
Delete message from a Queue.
Base Command
azure-storage-queue-message-delete
Input
| Argument Name | Description | Required |
|---|---|---|
| message_id | The ID of the message to delete. | Required |
| pop_receipt | Message PopReceipt parameter. This value changes each time the message is retrieved or updated and used to ensure that message has not been dequeued by another user before deleting or updating the message. | Required |
| queue_name | The name of the Queue. | Required |
Context Output
There is no context output for this command.
Command Example
!azure-storage-queue-message-delete queue_name="test-xsoar" message_id="66df94e9-4a66-428a-9c4a-e2a3c4fe7284" pop_receipt="AgAAAAMAAAAAAAAA0xNSmE7k1wE="
Human Readable Output
Message in test-xsoar successfully deleted.
azure-storage-queue-message-clear
Delete all messages from the specified Queue.
Base Command
azure-storage-queue-message-clear
Input
| Argument Name | Description | Required |
|---|---|---|
| queue_name | The name of the queue. | Required |
Context Output
There is no context output for this command.
Command Example
!azure-storage-queue-message-clear queue_name="xsoar-test"
Human Readable Output
xsoar-test was cleared of messages successfully.
Configuration parameters
credentials— Storage account nameuse_managed_identities— Use Azure Managed Identitiesmanaged_identities_client_id—proxy— Use system proxy settingsinsecure— Trust any certificate (not secure)max_fetch— Maximum incidents for one fetch.queue_name— Queue nameincidentType— Incident typeincidentFetchInterval— Incidents Fetch IntervalisFetch— Fetch incidents
Commands (10)
-
azure-storage-queue-createCreate new queue in storage account.
-
azure-storage-queue-deleteDelete queue from storage account.
-
azure-storage-queue-listList queues in storage account.
-
azure-storage-queue-message-clearDelete all messages from the specified Queue.
-
azure-storage-queue-message-createAdd a new message to the back of the queue.
-
azure-storage-queue-message-deleteDelete message from a Queue.
-
azure-storage-queue-message-dequeueDequeue a message from the front of the queue.
-
azure-storage-queue-message-getRetrieves messages from the front of the queue. Retrieved messages will move to the end of the queue,and will be visible after 'TimeNextVisible' param.
-
azure-storage-queue-message-peekRetrieves messages from the front of the queue. The command does not alter the visibility of the message.
-
azure-storage-queue-message-updateUpdate message content in the queue.
import base64 import copy from collections.abc import Callable import defusedxml.ElementTree as defused_ET import demistomock as demisto # noqa: F401 import urllib3 from CommonServerPython import * # noqa: F401 from requests import Response DATE_FORMAT = "%a, %d %b %Y %H:%M:%S GMT" account_sas_token = "" storage_account_name = "" class Client: """ API Client """ def __init__( self, server_url, verify, proxy, account_sas_token, storage_account_name, api_version, managed_identities_client_id: Optional[str] = None, ): self.ms_client = MicrosoftStorageClient( server_url, verify, proxy, account_sas_token, storage_account_name, api_version, managed_identities_client_id ) def list_queues_request(self, limit: str = None, prefix: str = None, marker: str = None) -> str: """ List queues in Azure storage account. Args: limit (str): Number of queues to retrieve. prefix (str): Filters the results to return only queues with names that begin with the specified prefix. marker (str): Identifies the portion of the list to be returned. Returns: str: API xml response from Azure. """ params = assign_params(comp="list", maxresults=limit, prefix=prefix, marker=marker) response = self.ms_client.http_request(method="GET", url_suffix="", params=params, resp_type="text") return response def create_queue_request(self, queue_name: str) -> Response: """ Create queue in storage account. Args: queue_name (str): New queue name. Returns: Response: API response from Azure. """ response = self.ms_client.http_request(method="PUT", url_suffix=f"/{queue_name}", return_empty_response=True) return response def delete_queue_request(self, queue_name: str) -> Response: """ Delete queue from storage account. Args: queue_name (str): New queue name. Returns: Response: API response from Azure. """ response = self.ms_client.http_request(method="DELETE", url_suffix=f"/{queue_name}", return_empty_response=True) return response def create_message_request( self, queue_name: str, xml_data: str, visibility_time_out: int = None, expiration: int = None ) -> str: """ Add a new message to the back of the message queue. Args: queue_name (str): Queue name. xml_data (str): Request XML data. visibility_time_out (int): Specifies the new visibility timeout value. expiration (int): Specifies the time-to-live interval for the message, in seconds. Returns: str: API response from Azure. """ params = assign_params(messagettl=expiration, visibilitytimeout=visibility_time_out) response = self.ms_client.http_request( method="POST", url_suffix=f"/{queue_name}/messages", params=params, resp_type="text", data=xml_data ) return response def get_messages_request(self, queue_name: str, limit: str = "1", visibility_time_out: int = 30) -> str: """ Retrieves messages from the front of the queue. Retrieved messages will move to the end of the queue,and will be visible after 'visibility_time_out' argument. Args: limit (str): Number of messages to retrieve. queue_name (str): Queue name. visibility_time_out (int): Specifies the new visibility timeout value. Returns: str: API response from Azure. """ params = assign_params(numofmessages=limit, visibilitytimeout=visibility_time_out) response = self.ms_client.http_request( method="GET", url_suffix=f"/{queue_name}/messages", resp_type="text", params=params ) return response def peek_messages_request(self, limit: str, queue_name: str) -> str: """ Retrieves messages from the front of the queue. Args: limit (str): Number of messages to retrieve queue_name (str): Queue name. Returns: str: API response from Azure. """ params = assign_params(numofmessages=limit, peekonly="true") response = self.ms_client.http_request( method="GET", url_suffix=f"/{queue_name}/messages", resp_type="text", params=params ) return response def delete_message_request(self, queue_name: str, message_id: str, pop_receipt: str) -> Response: """ Delete message from the queue. Args: queue_name (str): Queue name. message_id (str): Message ID. pop_receipt (str): Message ID pop-receipt. Returns: Response: API response from Azure. """ params = assign_params(popreceipt=pop_receipt.replace("+", "%2b")) url_suffix = f"/{queue_name}/messages/{message_id}" response = self.ms_client.http_request(method="DELETE", url_suffix=url_suffix, params=params, return_empty_response=True) return response def update_message_request( self, queue_name: str, xml_data: str, message_id: str, pop_receipt: str, visibility_time_out: str ) -> Response: """ Update message in the queue. Args: queue_name (str): Queue name. xml_data (str): Request XML data. message_id (str): Updated message ID. pop_receipt (str): Updated message ID pop-receipt. visibility_time_out (str): Specifies the new visibility timeout value. Returns: Response: API response text from Azure. """ params = assign_params(popreceipt=pop_receipt.replace("+", "%2b"), visibilitytimeout=visibility_time_out) url_suffix = f"/{queue_name}/messages/{message_id}" response = self.ms_client.http_request( method="PUT", url_suffix=url_suffix, params=params, data=xml_data, return_empty_response=True ) return response def clear_messages_request(self, queue_name: str) -> Response: """ Delete all messages from the queue. Args: queue_name (str): Queue name. Returns: Response: API response text from Azure. """ url_suffix = f"/{queue_name}/messages" response = self.ms_client.http_request(method="DELETE", url_suffix=url_suffix, return_empty_response=True) return response def parse_xml_response(xml_string_response: str, tag_path: str = "", find_tag: bool = False) -> list: """ Parse Azure XML response. Convert XML schema string to iterable list. For example: xml_string_response = Integration log: <?xml version="1.0" encoding="utf-8"?><QueueMessagesList> <QueueMessage><MessageId>e90f5f60-7a02-4b0b-a522-04ca8f3a00b9</MessageId> <InsertionTime>Thu, 14 Oct 2021 08:17:14 GMT</InsertionTime> <ExpirationTime>Thu, 21 Oct 2021 08:17:14 GMT</ExpirationTime> <DequeueCount>0</DequeueCount><MessageText>demo content</MessageText> </QueueMessage> </QueueMessagesList> The return value will be: [{'MessageId': 'e90f5f60-7a02-4b0b-a522-04ca8f3a00b9', 'InsertionTime': 'Thu, 14 Oct 2021 08:17:14 GMT', 'ExpirationTime': 'Thu, 21 Oct 2021 08:17:14 GMT', 'DequeueCount': '0', 'MessageText': 'demo content'}] Args: xml_string_response (str): XML response. tag_path (str): XML target Tag. find_tag (bool): Indicates parse operation type. Returns: list: XML iterable element. """ tree = ET.ElementTree(defused_ET.fromstring(xml_string_response)) root = tree.getroot() raw_response = [] if find_tag: return root.findall(tag_path) for message in root.iter(tag_path): message_data = {} for attribute in message: message_data[attribute.tag] = attribute.text raw_response.append(message_data) return raw_response def is_base_64(string: str) -> bool: """ Validate if string is base 64 encoded. Args: string (str): String to validate. Returns: bool: True if the string is base 64 encoded , else False. """ try: if isinstance(string, str): # If there's any unicode here, an exception will be thrown and the function will return false string_bytes = bytes(string, "ascii") elif isinstance(string, bytes): string_bytes = string else: raise ValueError("Argument must be string or bytes") return base64.b64encode(base64.b64decode(string_bytes)) == string_bytes except Exception: return False def decode_message(string: str) -> str: """ Decode string if it is encoded in base64. Args: string (str): String to decode. Returns: str : Decoded / origin string. """ if is_base_64(string): try: return base64.b64decode(string).decode("utf-8") except Exception: return string return string def encode_message(string: str) -> str: """ Encode string in base64. Args: string (str): String to decode. Returns: str: Encoded string. """ message_bytes = string.encode("utf-8") return base64.b64encode(message_bytes).decode("utf-8") def get_pagination_next_marker_element(limit: str, page: int, client_request: Callable, params: dict) -> str: """ Get next marker element for request pagination. 'marker' is a string value that identifies the portion of the list to be returned with the next list operation. The operation returns a NextMarker element within the response body if the list returned was not complete. This value may then be used as a query parameter in a subsequent call to request the next portion of the list items. Args: limit (str): Number of elements to retrieve. page (str): Page number. client_request (Callable): Client request function. params (dict): Request params. Returns: str: Next marker. """ offset = int(limit) * (page - 1) response = client_request(limit=str(offset), **params) tree = ET.ElementTree(defused_ET.fromstring(response)) root = tree.getroot() return root.findtext("NextMarker") # type: ignore def list_queues_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ List queues in Azure storage account. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: outputs, readable outputs and raw response for XSOAR. """ limit = args.get("limit") or "50" prefix = args.get("prefix") page = arg_to_number(args.get("page") or "1") marker = "" readable_message = f"Queues List:\n Current page size: {limit}\n Showing page {page} out others that may exist" if page > 1: # type: ignore marker = get_pagination_next_marker_element( limit=limit, page=page, # type: ignore client_request=client.list_queues_request, params={"prefix": prefix}, ) if not marker: return CommandResults( readable_output=readable_message, outputs_prefix="AzureStorageQueue.Queue", outputs=[], raw_response=[] ) response = client.list_queues_request(limit, prefix, marker) xml_response = parse_xml_response(xml_string_response=response, tag_path="./Queues/Queue/Name", find_tag=True) raw_response = [{"name": element.text} for element in xml_response] readable_output = tableToMarkdown(readable_message, raw_response, headers="name", headerTransform=pascalToSpace) command_results = CommandResults( readable_output=readable_output, outputs_prefix="AzureStorageQueue.Queue", outputs_key_field="name", outputs=raw_response, raw_response=raw_response, ) return command_results def create_queue_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Create queue in storage account. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: readable output for XSOAR. """ queue_name = args["queue_name"] queue_name_regex = "^[a-z0-9](?!.*--)[a-z0-9-]{1,61}[a-z0-9]$" # Rules for naming queues can be found here: # https://docs.microsoft.com/en-us/rest/api/storageservices/naming-queues-and-metadata if not re.search(queue_name_regex, queue_name): raise Exception("The specified queue name is invalid.") response = client.create_queue_request(queue_name) readable_output = ( f"Queue {queue_name} successfully created." if response.status_code == 201 else f"Queue {queue_name} already exists." ) command_results = CommandResults(readable_output=readable_output) return command_results def delete_queue_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Delete queue from storage account. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: readable output for XSOAR. """ queue_name = args["queue_name"] client.delete_queue_request(queue_name) readable_output = f"Queue {queue_name} successfully deleted." command_results = CommandResults(readable_output=readable_output) return command_results def date_values_to_iso(data: dict, keys: list): """ Convert time data values to ISO 8601 time format. input example: keys = ['InsertionTime','ExpirationTime'] , data = { 'InsertionTime': 'Wed, 13 Oct 2021 09:11:32 GMT', 'ExpirationTime': 'Wed, 20 Oct 2021 09:11:32 GMT', } the method will convert the data to: { 'InsertionTime': '2021-10-13T09:11:32', 'ExpirationTime': '2021-10-20T09:11:32' } Args: data (dict): Data. keys (list): Keys list to convert. """ for key in keys: if data.get(key): time_value = datetime.strptime(data.get(key), DATE_FORMAT) # type: ignore iso_time = FormatIso8601(time_value) data[key] = iso_time def create_message_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Add a new message to the back of the message queue. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: outputs, readable outputs and raw response for XSOAR. """ message_content = args["message_content"] queue_name = args["queue_name"] visibility_time_out = arg_to_number(args.get("visibility_time_out")) expiration = arg_to_number(args.get("expiration")) encode = argToBoolean(args.get("base64_encoding", False)) message_content = encode_message(message_content) if encode else message_content top = ET.Element("QueueMessage") child = ET.SubElement(top, "MessageText") child.text = message_content xml_data = ET.tostring(top, encoding="unicode") response = client.create_message_request(queue_name, xml_data, visibility_time_out, expiration) raw_response = parse_xml_response(xml_string_response=response, tag_path="QueueMessage") message_outputs = copy.deepcopy(raw_response)[0] date_values_to_iso(message_outputs, ["ExpirationTime", "InsertionTime", "TimeNextVisible"]) outputs = {"name": queue_name, "Message": message_outputs} readable_output = tableToMarkdown( f"{queue_name} Queue message:", message_outputs, headers=["MessageId", "ExpirationTime", "InsertionTime", "TimeNextVisible", "PopReceipt"], headerTransform=pascalToSpace, ) command_results = CommandResults( readable_output=readable_output, outputs_prefix="AzureStorageQueue.Queue", outputs_key_field="name", outputs=outputs, raw_response=raw_response, ) return command_results def get_messages_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Retrieves messages from the front of the queue. Retrieved messages will move to the end of the queue, and will be visible after the amount of time specified in the 'TimeNextVisible' param. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: outputs, readable outputs and raw response for XSOAR. """ limit = args.get("limit") or "1" queue_name = args["queue_name"] visibility_time_out = arg_to_number(args.get("visibility_time_out")) if int(limit) < 1 or int(limit) > 32: raise Exception("Invalid limit value. Minimum value is 1, maximum value is 32") response = client.get_messages_request(queue_name, limit, visibility_time_out) # type: ignore raw_response = parse_xml_response(xml_string_response=response, tag_path="QueueMessage") message_outputs = copy.deepcopy(raw_response) for message in message_outputs: message["MessageText"] = decode_message(message["MessageText"]) date_values_to_iso(message, ["ExpirationTime", "InsertionTime", "TimeNextVisible"]) outputs = {"name": queue_name, "Message": message_outputs} readable_output = tableToMarkdown( f"{queue_name} Queue messages:", message_outputs, headers=["MessageText", "MessageId", "PopReceipt", "DequeueCount", "ExpirationTime", "InsertionTime", "TimeNextVisible"], headerTransform=pascalToSpace, ) command_results = CommandResults( readable_output=readable_output, outputs_prefix="AzureStorageQueue.Queue", outputs_key_field="name", outputs=outputs, raw_response=raw_response, ) return command_results def peek_messages_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Retrieves messages from the front of the queue. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: outputs, readable outputs and raw response for XSOAR. """ limit = args.get("limit") or "1" queue_name = args["queue_name"] if int(limit) < 1 or int(limit) > 32: raise Exception("Invalid limit value. Minimum value is 1, maximum value is 32") response = client.peek_messages_request(limit, queue_name) raw_response = parse_xml_response(xml_string_response=response, tag_path="QueueMessage") message_outputs = copy.deepcopy(raw_response) for message in message_outputs: message["MessageText"] = decode_message(message["MessageText"]) date_values_to_iso(message, ["ExpirationTime", "InsertionTime"]) outputs = {"name": queue_name, "Message": message_outputs} readable_output = tableToMarkdown( f"{queue_name} Queue messages:", message_outputs, headers=["MessageText", "MessageId", "DequeueCount", "ExpirationTime", "InsertionTime"], headerTransform=pascalToSpace, ) command_results = CommandResults( readable_output=readable_output, outputs_prefix="AzureStorageQueue.Queue", outputs_key_field="name", outputs=outputs, raw_response=raw_response, ) return command_results def dequeue_message_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Dequeue message from the front of the queue. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: Readable output for XSOAR. """ queue_name = args["queue_name"] response = client.get_messages_request(queue_name=queue_name) message_response = parse_xml_response(xml_string_response=response, tag_path="QueueMessage") if len(message_response) == 0: return CommandResults(readable_output=f"There are no messages in {queue_name} queue.") message_id = message_response[0]["MessageId"] pop_receipt = message_response[0]["PopReceipt"] client.delete_message_request(queue_name=queue_name, message_id=message_id, pop_receipt=pop_receipt) readable_output = f"Message in {queue_name} successfully deleted." command_results = CommandResults(readable_output=readable_output) return command_results def delete_message_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Delete message from the queue. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: Readable output for XSOAR. """ queue_name = args["queue_name"] message_id = args["message_id"] pop_receipt = args["pop_receipt"] client.delete_message_request(queue_name=queue_name, message_id=message_id, pop_receipt=pop_receipt) readable_output = f"Message in {queue_name} successfully deleted." command_results = CommandResults(readable_output=readable_output) return command_results def update_message_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Update message in the the queue. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: readable output for XSOAR. """ message_content = args["message_content"] queue_name = args["queue_name"] message_id = args["message_id"] pop_receipt = args["pop_receipt"] encode = argToBoolean(args.get("base64_encoding", False)) visibility_time_out = args["visibility_time_out"] message_content = encode_message(message_content) if encode else message_content top = ET.Element("QueueMessage") child = ET.SubElement(top, "MessageText") child.text = message_content xml_data = ET.tostring(top, encoding="unicode") client.update_message_request(queue_name, xml_data, message_id, pop_receipt, visibility_time_out) readable_output = f"The message in {queue_name} successfully updated." command_results = CommandResults(readable_output=readable_output) return command_results def clear_messages_command(client: Client, args: Dict[str, Any]) -> CommandResults: """ Delete all messages from the queue. Args: client (Client): Azure Queue Storage API client. args (dict): Command arguments from XSOAR. Returns: CommandResults: Readable output for XSOAR. """ queue_name = args["queue_name"] client.clear_messages_request(queue_name=queue_name) readable_output = f"{queue_name} was cleared of messages successfully." command_results = CommandResults(readable_output=readable_output) return command_results def parse_incident(message: dict) -> dict: """ Parse message to XSOAR Incident. Args: message (dict): Message item. Returns: dict: XSOAR Incident """ time_headers = ["ExpirationTime", "InsertionTime", "TimeNextVisible"] message["MessageText"] = decode_message(message["MessageText"]) for header in time_headers: time_value = datetime.strptime(message.get(header), DATE_FORMAT) # type: ignore iso_time = FormatIso8601(time_value) + "Z" message[header] = iso_time incident = {} incident["name"] = "Azure Storage - Queue MessageId: " + message["MessageId"] incident["rawJSON"] = json.dumps(message) return incident def fetch_incidents(client: Client, queue_name: str, max_fetch: str) -> None: """ Fetch messages from the Queue. Args: client (Client): Azure Queue Storage API client. queue_name (str): Queue name. max_fetch (str): Maximum incidents for one fetch. """ response = client.get_messages_request(queue_name=queue_name, limit=max_fetch) raw_response = parse_xml_response(xml_string_response=response, tag_path="QueueMessage") incidents = [] for message in raw_response: message["queue_name"] = queue_name incidents.append(parse_incident(message)) demisto.incidents(incidents) for message in raw_response: client.delete_message_request(queue_name=queue_name, message_id=message["MessageId"], pop_receipt=message["PopReceipt"]) def test_module(client: Client, max_fetch: str) -> None: """ Tests API connectivity and authentication. Args: client (Client): Azure Queue Storage API client. max_fetch (str): Maximum incidents for one fetch. Returns: str : 'ok' if test passed, anything else will fail the test. """ try: client.list_queues_request() max_fetch_int = int(max_fetch) except Exception as exception: if "Error in API call" in str(exception): return return_results("Authorization Error: make sure API Credentials are correctly set") if "Error Type" in str(exception): return return_results( "Verify that the storage account name is correct and that you have access to the server from your host." ) if type(exception).__name__ == "ValueError": return return_results("Invalid Maximum fetch value.") raise exception if max_fetch_int <= 0 or max_fetch_int > 32: return return_results("Invalid Maximum fetch value. Minimum value is 1, maximum value is 32") return_results("ok") return None def main() -> None: """ Main function """ params: Dict[str, Any] = demisto.params() args: Dict[str, Any] = demisto.args() verify_certificate: bool = not params.get("insecure", False) proxy = params.get("proxy", False) global account_sas_token global storage_account_name account_sas_token = params.get("credentials", {}).get("password") storage_account_name = params["credentials"]["identifier"] api_version = "2020-10-02" base_url = f"https://{storage_account_name}.queue.core.windows.net" managed_identities_client_id = get_azure_managed_identities_client_id(params) command = demisto.command() demisto.debug(f"Command being called is {command}") try: urllib3.disable_warnings() client: Client = Client( base_url, verify_certificate, proxy, account_sas_token, storage_account_name, api_version, managed_identities_client_id, ) commands = { "azure-storage-queue-list": list_queues_command, "azure-storage-queue-create": create_queue_command, "azure-storage-queue-delete": delete_queue_command, "azure-storage-queue-message-create": create_message_command, "azure-storage-queue-message-get": get_messages_command, "azure-storage-queue-message-peek": peek_messages_command, "azure-storage-queue-message-dequeue": dequeue_message_command, "azure-storage-queue-message-update": update_message_command, "azure-storage-queue-message-delete": delete_message_command, "azure-storage-queue-message-clear": clear_messages_command, } if command == "test-module": test_module(client, params.get("max_fetch")) # type: ignore elif command == "fetch-incidents": fetch_incidents(client, params.get("queue_name"), params.get("max_fetch")) # type: ignore elif command in commands: return_results(commands[command](client, args)) else: raise NotImplementedError(f"{command} command is not implemented.") except Exception as e: return_error(str(e)) from MicrosoftAzureStorageApiModule import * # noqa: E402 if __name__ in ["__main__", "builtin", "builtins"]: main()