Details
| ID | AWS - SQS |
|---|---|
| Provider | Amazon |
| Category | IT Services |
| From Version | 5.0.0 |
| Docker Image | demisto/boto3py3:1.0.0.10325753 |
| Supported Modules | Agentix XSIAM |
README
Amazon Web Services Simple Queuing Service (SQS)
For more information regarding the AWS SQS service, please visit the official documentation found here.
For detailed instructions about setting up authentication, see: AWS Integrations - Authentication.
Configure AWS - SQS in Cortex
| Parameter | Description | Required |
|---|---|---|
| AWS Default Region | The AWS Region for this instance of the integration. For example, us-west-2 | False |
| Role Arn | The Amazon Resource Name (ARN) role used for EC2 instance authentication. If this is used, an access key and secret key are not required. | False |
| Role Session Name | A descriptive name for the assumed role session. For example, xsiam-IAM.integration-Role_SESSION | False |
| Access Key | The access key ID used for authentication, that was configured during IAM user configuration. If this is used, Role ARN is not required. | False |
| Secret Key | The secret key used for authentication, that was configured during IAM user configuration. If this is used, Role ARN is not required. | False |
| Role Session Duration | The maximum length of each session in seconds. Default: 900 seconds. The XSOAR integration will have the permissions assigned only when the session is initiated and for the defined duration. | False |
| Queue URL | URL of an existing Amazon SQS queue. | False |
| Timeout | The time in seconds till a timeout exception is reached. You can specify just the read timeout (for example 60) or also the connect timeout followed after a comma (for example 60,10). If a connect timeout is not specified a default of 10 second will be used. | False |
| Retries | The maximum number of retry attempts when connection or throttling errors are encountered. Set to 0 to disable retries. The default value is 5 and the limit is 10. Note: Increasing the number of retries will increase the execution time. More details about the retries strategy is available here. | False |
| Fetch incidents | False | |
| Maximum incidents for one fetch. Hard cap of 100. | Maximum number of incidents for a single fetch. | 10 |
| First fetch timestamp | First fetch query <number> <time unit>, e.g., 7 days. Default 3 days) |
False |
| Incident type | False | |
| Use system proxy settings | False | |
| Trust any certificate (not secure) | False | |
| Parse SQS message body as a JSON string | False |
There are three options to sign in to the service
- Provide Access Key ID and Secret Key ID.
- Provide Role ARN and Access Key ID and Secret Key ID.
- Do not provide any credentials or Role ARN other than permissions pulled from the service metadata.
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
aws-sqs-get-queue-url
Returns the URL of an existing queue. To access a queue that belongs to another AWS account, use the queueOwnerAWSAccountId parameter to specify the account ID of the queues owner. The queues owner must grant you permission to access the queue.
Base Command
aws-sqs-get-queue-url
Input
| Argument Name | Description | Required |
|---|---|---|
| queueName | The name of the queue. | Required |
| queueOwnerAWSAccountId | The AWS account ID of the account that created the queue. | Optional |
| region | The AWS Region, if not specified the default region will be used. | Optional |
| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional |
| roleSessionName | An identifier for the assumed role session. | Optional |
| roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AWS.SQS.Queues.QueueUrl | string | The URL of the queue. |
Command Example
!aws-sqs-get-queue-url queueName=test
Human Readable Output
AWS SQS Queues
| QueueUrl | test.queue.amazonaws.com/1234567/test |
| — | — |
aws-sqs-list-queues
Returns a list of your queues. The maximum number of queues that can be returned is 1,000. If you specify a value for the optional QueueNamePrefix parameter, only queues with a name that begins with the specified value are returned.
Base Command
aws-sqs-list-queues
Input
| Argument Name | Description | Required |
|---|---|---|
| queueNamePrefix | A string to use for filtering the list results. Only those queues whose name begins with the specified string are returned. Queue names are case-sensitive. | Optional |
| region | The AWS Region, if not specified the default region will be used. | Optional |
| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional |
| roleSessionName | An identifier for the assumed role session. | Optional |
| roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AWS.SQS.Queues.QueueUrl | string | The URL of the queue. |
Command Example
!aws-sqs-list-queues
Human Readable Output
AWS SQS Queues
| QueueUrl |
|---|
| test.queue.amazonaws.com/1234567/test1 |
| test.queue.amazonaws.com/1234567/test2 |
aws-sqs-send-message
Delivers a message to the specified queue.
Base Command
aws-sqs-send-message
Input
| Argument Name | Description | Required |
|---|---|---|
| queueUrl | The URL of the Amazon SQS queue to which a message is sent. | Required |
| messageBody | The message to send. The maximum string size is 256 KB. | Required |
| delaySeconds | The length of time, in seconds, for which to delay a specific message. Valid values 0 to 900. Maximum 15 minutes. | Optional |
| messageGroupId | This parameter applies only to FIFO queues. The tag that specifies that a message belongs to a specific message group. | Optional |
| region | The AWS Region, if not specified the default region will be used. | Optional |
| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional |
| roleSessionName | An identifier for the assumed role session. | Optional |
| roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AWS.SQS.Queues.SentMessages.MD5OfMessageBody | string | An MD5 digest of the non-URL-encoded message attribute string. |
| AWS.SQS.Queues.SentMessages.MD5OfMessageAttributes | string | An MD5 digest of the non-URL-encoded message attribute string. |
| AWS.SQS.Queues.SentMessages.MessageId | string | An attribute containing the MessageId of the message sent to the queue. |
| AWS.SQS.Queues.SentMessages.SequenceNumber | string | This parameter applies only to FIFO (first-in-first-out) queues. The large, non-consecutive number that Amazon SQS assigns to each message. |
Command Example
!aws-sqs-send-message queueUrl=test.queue.amazonaws.com/1234567/test2 messageBody="test"
Human Readable Output
AWS SQS Queues sent messages
| MD5OfMessageBody | 123a4bcd4621d373cade4e832627b4f6 |
|---|---|
| MessageId | 1a2bc456-1e23-45e6-b789-b1af23c4f56f |
| QueueUrl | test.queue.amazonaws.com/1234567/test2 |
aws-sqs-create-queue
Creates a new standard or FIFO queue. You can pass one or more attributes in the request.
Base Command
aws-sqs-create-queue
Input
| Argument Name | Description | Required |
|---|---|---|
| queueName | The name of the new queue. | Required |
| delaySeconds | The length of time, in seconds, for which the delivery of all messages in the queue is delayed. Valid values An integer from 0 to 900 seconds (15 minutes). The default is 0 (zero). | Optional |
| maximumMessageSize | The limit of how many bytes a message can contain before Amazon SQS rejects it. Valid values An integer from 1,024 bytes (1 KiB) to 262,144 bytes (256 KiB). The default is 262,144 (256 KiB). | Optional |
| messageRetentionPeriod | The length of time, in seconds, for which Amazon SQS retains a message. Valid values An integer from 60 seconds (1 minute) to 1,209,600 seconds (14 days). The default is 345,600 (4 days). | Optional |
| receiveMessageWaitTimeSeconds | The length of time, in seconds, for which a ReceiveMessage action waits for a message to arrive. Valid values An integer from 0 to 20 (seconds). The default is 0 (zero). | Optional |
| visibilityTimeout | The visibility timeout for the queue. Valid values An integer from 0 to 43,200 (12 hours). The default is 30. | Optional |
| kmsDataKeyReusePeriodSeconds | The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again. An integer representing seconds, between 60 seconds (1 minute) and 86,400 seconds (24 hours). The default is 300 (5 minutes). A shorter time period provides better security but results in more calls to KMS which might incur charges after Free Tier. | Optional |
| kmsMasterKeyId | The ID of an AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK. | Optional |
| policy | The queues policy. A valid AWS policy. | Optional |
| fifoQueue | Designates a queue as FIFO. Possible values are: True, False. | Optional |
| contentBasedDeduplication | Enables content-based deduplication. Possible values are: True, False. | Optional |
| region | The AWS Region, if not specified the default region will be used. | Optional |
| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional |
| roleSessionName | An identifier for the assumed role session. | Optional |
| roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| AWS.SQS.Queues.QueueUrl | unknown | The URL of the created Amazon SQS queue. |
Command Example
!aws-sqs-create-queue queueName=test3
Human Readable Output
AWS SQS Queues
| QueueUrl | test.queue.amazonaws.com/1234567/test3 |
| — | — |
aws-sqs-delete-queue
Deletes the queue specified by the QueueUrl , regardless of the queue’s contents. If the specified queue doesn’t exist, Amazon SQS returns a successful response.
Base Command
aws-sqs-delete-queue
Input
| Argument Name | Description | Required |
|---|---|---|
| queueUrl | The URL of the Amazon SQS queue to delete. | Required |
| region | The AWS Region, if not specified the default region will be used. | Optional |
| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional |
| roleSessionName | An identifier for the assumed role session. | Optional |
| roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional |
Context Output
There is no context output for this command.
Command Example
!aws-sqs-delete-queue queueUrl=example.com/123456789/test3
Human Readable Output
The Queue has been deleted
aws-sqs-purge-queue
Deletes the messages in a queue specified by the QueueURL parameter.
Base Command
aws-sqs-purge-queue
Input
| Argument Name | Description | Required |
|---|---|---|
| queueUrl | The URL of the queue from which the PurgeQueue action deletes messages. | Required |
| region | The AWS Region, if not specified the default region will be used. | Optional |
| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional |
| roleSessionName | An identifier for the assumed role session. | Optional |
| roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional |
Context Output
There is no context output for this command.
Command Example
!aws-sqs-purge-queue queueUrl=example.com/123456789/test2
Human Readable Output
The Queue has been Purged
Configuration parameters
defaultRegion— AWS Default Region (required)roleArn— Role ArnroleSessionName— Role Session Namecredentials— Access Keyaccess_key— Access Keysecret_key— Secret KeysessionDuration— Role Session DurationqueueUrl— QueueURLtimeout— Timeoutretries— RetriesisFetch— Fetch incidentsmax_fetch— Maximum incidents for one fetch. Hard cap of 100.first_fetch— First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days)incidentType— Incident typeincidentFetchInterval— Incidents Fetch Intervalsts_regional_endpoint— AWS STS Regional Endpointsproxy— Use system proxy settingsinsecure— Trust any certificate (not secure)parse_body_as_json— Parse SQS message body as a JSON string
Commands (6)
-
aws-sqs-create-queueCreates a new standard or FIFO queue. You can pass one or more attributes in the request.
-
aws-sqs-delete-queueDeletes the queue specified by the QueueUrl , regardless of the queue's contents. If the specified queue doesn't exist, Amazon SQS returns a successful response.
-
aws-sqs-get-queue-urlReturns the URL of an existing queue. To access a queue that belongs to another AWS account, use the queueOwnerAWSAccountId parameter to specify the account ID of the queues owner. The queues owner must grant you permission to access the queue.
-
aws-sqs-list-queuesReturns a list of your queues. The maximum number of queues that can be returned is 1,000. If you specify a value for the optional QueueNamePrefix parameter, only queues with a name that begins with the specified value are returned.
-
aws-sqs-purge-queueDeletes the messages in a queue specified by the QueueURL parameter.
-
aws-sqs-send-messageDelivers a message to the specified queue.
import demistomock as demisto # noqa: F401 from CommonServerPython import * # noqa: F401 def create_entry(title, data, ec): return { "ContentsFormat": formats["json"], "Type": entryTypes["note"], "Contents": data, "ReadableContentsFormat": formats["markdown"], "HumanReadable": tableToMarkdown(title, data) if data else "No result were found", "EntryContext": ec, } def raise_error(error): return {"Type": entryTypes["error"], "ContentsFormat": formats["text"], "Contents": str(error)} def get_queue_url(args, client): try: kwargs = {"QueueName": args.get("queueName")} if args.get("queueOwnerAWSAccountId"): kwargs.update({"QueueOwnerAWSAccountId": args.get("queueOwnerAWSAccountId")}) response = client.get_queue_url(**kwargs) data = {"QueueUrl": response["QueueUrl"]} ec = {"AWS.SQS.Queues": data} return create_entry("AWS SQS Queues", data, ec) except Exception as e: return raise_error(e) def list_queues(args, client): try: data = [] kwargs = {} if args.get("queueNamePrefix") is not None: kwargs.update({"QueueNamePrefix": args.get("queueNamePrefix")}) response = client.list_queues(**kwargs) for queue in response.get("QueueUrls", []): data.append({"QueueUrl": queue}) ec = {"AWS.SQS.Queues": data} return create_entry("AWS SQS Queues", data, ec) except Exception as e: return raise_error(e) def send_message(args, client): try: kwargs = { "QueueUrl": args.get("queueUrl"), "MessageBody": args.get("messageBody"), } if args.get("delaySeconds") is not None: kwargs.update({"DelaySeconds": int(args.get("delaySeconds"))}) if args.get("messageGroupId") is not None: kwargs.update({"MessageGroupId": int(args.get("messageGroupId"))}) response = client.send_message(**kwargs) data = { "QueueUrl": args.get("queueUrl"), "MessageId": response["MessageId"], } if "SequenceNumber" in response: data.update({"SequenceNumber": response["SequenceNumber"]}) if "MD5OfMessageBody" in response: data.update({"MD5OfMessageBody": response["MD5OfMessageBody"]}) if "MD5OfMessageAttributes" in response: data.update({"MD5OfMessageAttributes": response["MD5OfMessageAttributes"]}) ec = {"AWS.SQS.Queues(obj.QueueUrl === val.QueueUrl).SentMessages": data} return create_entry("AWS SQS Queues sent messages", data, ec) except Exception as e: return raise_error(e) def create_queue(args, client): try: attributes = {} kwargs = {"QueueName": args.get("queueName")} if args.get("delaySeconds") is not None: attributes.update({"DelaySeconds": args.get("delaySeconds")}) if args.get("maximumMessageSize") is not None: attributes.update({"MaximumMessageSize": args.get("maximumMessageSize")}) if args.get("messageRetentionPeriod") is not None: attributes.update({"MessageRetentionPeriod": args.get("messageRetentionPeriod")}) if args.get("receiveMessageWaitTimeSeconds") is not None: attributes.update({"ReceiveMessageWaitTimeSeconds": args.get("receiveMessageWaitTimeSeconds")}) if args.get("visibilityTimeout") is not None: attributes.update({"VisibilityTimeout": int(args.get("visibilityTimeout"))}) if args.get("kmsDataKeyReusePeriodSeconds") is not None: attributes.update({"KmsDataKeyReusePeriodSeconds": args.get("kmsDataKeyReusePeriodSeconds")}) if args.get("kmsMasterKeyId") is not None: attributes.update({"KmsMasterKeyId": args.get("kmsMasterKeyId")}) if args.get("policy") is not None: attributes.update({"Policy": args.get("policy")}) if args.get("fifoQueue") is not None: attributes.update({"FifoQueue": args.get("fifoQueue")}) if args.get("contentBasedDeduplication") is not None: attributes.update({"ContentBasedDeduplication": args.get("contentBasedDeduplication")}) if attributes: kwargs.update({"Attributes": attributes}) response = client.create_queue(**kwargs) data = {"QueueUrl": response["QueueUrl"]} ec = {"AWS.SQS.Queues": data} return create_entry("AWS SQS Queues", data, ec) except Exception as e: return raise_error(e) def delete_queue(args, client): try: response = client.delete_queue(QueueUrl=args.get("queueUrl")) if response["ResponseMetadata"]["HTTPStatusCode"] == 200: return "The Queue has been deleted" except Exception as e: return raise_error(e) def purge_queue(args, client): try: response = client.purge_queue(QueueUrl=args.get("queueUrl")) if response["ResponseMetadata"]["HTTPStatusCode"] == 200: return "The Queue has been Purged" except Exception as e: return raise_error(e) def parse_incident_from_finding(message, parse_body_as_json=False): incident = {} incident["name"] = "SQS MessageId: " + message["MessageId"] if parse_body_as_json: try: message["Body"] = json.loads(message["Body"]) except Exception: pass incident["rawJSON"] = json.dumps(message) return incident def fetch_incidents(aws_client, aws_queue_url, max_fetch, parse_body_as_json): """ Fetching the messages from the queue by following steps: 1. fetch. 2. create-incidents (while skipping previous fetch). 3. save fetch results to context. 4. try to delete all messages (if not successful, will continue next run). """ try: client = aws_client.aws_session(service="sqs") # The 'receipt_handles' of the messages that were received from the last call. last_receipt_handles = demisto.getLastRun().get("lastReceiptHandles") if last_receipt_handles: demisto.debug(f'last_receipt_handles before fetch occurred" -> {len(last_receipt_handles)} {last_receipt_handles}') last_receipt_handles = set(last_receipt_handles) incidents_created = 0 # type: int max_number_of_messages = min(max_fetch, 10) receipt_handles = [] # type: list incidents = [] # type: list while incidents_created < max_fetch: messages = client.receive_message( QueueUrl=aws_queue_url, MaxNumberOfMessages=max_number_of_messages, VisibilityTimeout=30, WaitTimeSeconds=5, ) if "Messages" not in messages: if incidents_created == 0: if demisto.command() == "fetch-incidents": demisto.incidents([]) return messages, incidents, receipt_handles else: break # Creating incidents and avoiding creating incidents that were already created previously for message in messages["Messages"]: receipt_handles.append(message["ReceiptHandle"]) if last_receipt_handles and message["ReceiptHandle"] in last_receipt_handles: continue incidents.append(parse_incident_from_finding(message, parse_body_as_json)) incidents_created += 1 if incidents_created == max_fetch: break # Save fetch results to context. demisto.incidents(incidents) # The "receipt_handles" of converted messages to the incidents are saved for next fetch demisto.setLastRun({"lastReceiptHandles": receipt_handles}) demisto.debug(f'last_receipt_handles after fetch occurred" -> {len(receipt_handles)} {receipt_handles}') # try to delete all messages (if not successful, will continue next run) for receipt_handle in receipt_handles: client.delete_message(QueueUrl=aws_queue_url, ReceiptHandle=receipt_handle) except Exception as e: return raise_error(e) def test_function(aws_client): try: client = aws_client.aws_session(service="sqs") response = client.list_queues() if response["ResponseMetadata"]["HTTPStatusCode"] == 200: return "ok" except Exception as e: return raise_error(e) def main(): params = demisto.params() aws_default_region = params.get("defaultRegion") aws_role_arn = params.get("roleArn") aws_role_session_name = params.get("roleSessionName") aws_role_session_duration = params.get("sessionDuration") aws_role_policy = None aws_access_key_id = params.get("credentials", {}).get("identifier") or params.get("access_key") aws_secret_access_key = params.get("credentials", {}).get("password") or params.get("secret_key") verify_certificate = not params.get("insecure", True) timeout = params.get("timeout") retries = params.get("retries") or 5 aws_queue_url = params.get("queueUrl") max_fetch = min(arg_to_number(params.get("max_fetch", 10)) or 10, 100) parse_body_as_json = params.get("parse_body_as_json", False) commands = { "aws-sqs-get-queue-url": get_queue_url, "aws-sqs-list-queues": list_queues, "aws-sqs-send-message": send_message, "aws-sqs-create-queue": create_queue, "aws-sqs-delete-queue": delete_queue, "aws-sqs-purge-queue": purge_queue, } try: validate_params(aws_default_region, aws_role_arn, aws_role_session_name, aws_access_key_id, aws_secret_access_key) aws_client = AWSClient( aws_default_region, aws_role_arn, aws_role_session_name, aws_role_session_duration, aws_role_policy, aws_access_key_id, aws_secret_access_key, verify_certificate, timeout, retries, ) command = demisto.command() args = demisto.args() demisto.debug(f"Command being called is {command}") if command == "test-module": return_results(test_function(aws_client)) elif demisto.command() == "fetch-incidents": fetch_incidents(aws_client, aws_queue_url, max_fetch, parse_body_as_json) sys.exit(0) elif command in commands: client = aws_client.aws_session( service="sqs", region=args.get("region"), role_arn=args.get("roleArn"), role_session_name=args.get("roleSessionName"), role_session_duration=args.get("roleSessionDuration"), ) return_results(commands[command](args, client)) else: raise NotImplementedError(f"{command} is not an existing AWS-SQS command") except Exception as e: return_error(f"Failed to execute {demisto.command()} command.\nError:\n{e!s}") from AWSApiModule import * # noqa: E402 if __name__ in ("__main__", "__builtin__", "builtins"): main()