AWS - SQS

Amazon Web Services Simple Queuing Service (SQS).

IT Services · AWS - SQS

Details

IDAWS - SQS
ProviderAmazon
CategoryIT Services
From Version5.0.0
Docker Imagedemisto/boto3py3:1.0.0.10325753
Supported ModulesAgentix XSIAM

README

Amazon Web Services Simple Queuing Service (SQS)

For more information regarding the AWS SQS service, please visit the official documentation found here.

For detailed instructions about setting up authentication, see: AWS Integrations - Authentication.

Configure AWS - SQS in Cortex

Parameter Description Required
AWS Default Region The AWS Region for this instance of the integration. For example, us-west-2 False
Role Arn The Amazon Resource Name (ARN) role used for EC2 instance authentication. If this is used, an access key and secret key are not required. False
Role Session Name A descriptive name for the assumed role session. For example, xsiam-IAM.integration-Role_SESSION False
Access Key The access key ID used for authentication, that was configured during IAM user configuration. If this is used, Role ARN is not required. False
Secret Key The secret key used for authentication, that was configured during IAM user configuration. If this is used, Role ARN is not required. False
Role Session Duration The maximum length of each session in seconds. Default: 900 seconds. The XSOAR integration will have the permissions assigned only when the session is initiated and for the defined duration. False
Queue URL URL of an existing Amazon SQS queue. False
Timeout The time in seconds till a timeout exception is reached. You can specify just the read timeout (for example 60) or also the connect timeout followed after a comma (for example 60,10). If a connect timeout is not specified a default of 10 second will be used. False
Retries The maximum number of retry attempts when connection or throttling errors are encountered. Set to 0 to disable retries. The default value is 5 and the limit is 10. Note: Increasing the number of retries will increase the execution time. More details about the retries strategy is available here. False
Fetch incidents   False
Maximum incidents for one fetch. Hard cap of 100. Maximum number of incidents for a single fetch. 10
First fetch timestamp First fetch query <number> <time unit>, e.g., 7 days. Default 3 days) False
Incident type   False
Use system proxy settings   False
Trust any certificate (not secure)   False
Parse SQS message body as a JSON string   False

There are three options to sign in to the service

  1. Provide Access Key ID and Secret Key ID.
  2. Provide Role ARN and Access Key ID and Secret Key ID.
  3. Do not provide any credentials or Role ARN other than permissions pulled from the service metadata.

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

aws-sqs-get-queue-url


Returns the URL of an existing queue. To access a queue that belongs to another AWS account, use the queueOwnerAWSAccountId parameter to specify the account ID of the queues owner. The queues owner must grant you permission to access the queue.

Base Command

aws-sqs-get-queue-url

Input

Argument Name Description Required
queueName The name of the queue. Required
queueOwnerAWSAccountId The AWS account ID of the account that created the queue. Optional
region The AWS Region, if not specified the default region will be used. Optional
roleArn The Amazon Resource Name (ARN) of the role to assume. Optional
roleSessionName An identifier for the assumed role session. Optional
roleSessionDuration The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. Optional

Context Output

Path Type Description
AWS.SQS.Queues.QueueUrl string The URL of the queue.

Command Example

!aws-sqs-get-queue-url queueName=test

Human Readable Output

AWS SQS Queues

| QueueUrl | test.queue.amazonaws.com/1234567/test |
| — | — |

aws-sqs-list-queues


Returns a list of your queues. The maximum number of queues that can be returned is 1,000. If you specify a value for the optional QueueNamePrefix parameter, only queues with a name that begins with the specified value are returned.

Base Command

aws-sqs-list-queues

Input

Argument Name Description Required
queueNamePrefix A string to use for filtering the list results. Only those queues whose name begins with the specified string are returned. Queue names are case-sensitive. Optional
region The AWS Region, if not specified the default region will be used. Optional
roleArn The Amazon Resource Name (ARN) of the role to assume. Optional
roleSessionName An identifier for the assumed role session. Optional
roleSessionDuration The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. Optional

Context Output

Path Type Description
AWS.SQS.Queues.QueueUrl string The URL of the queue.

Command Example

!aws-sqs-list-queues

Human Readable Output

AWS SQS Queues

QueueUrl
test.queue.amazonaws.com/1234567/test1
test.queue.amazonaws.com/1234567/test2

aws-sqs-send-message


Delivers a message to the specified queue.

Base Command

aws-sqs-send-message

Input

Argument Name Description Required
queueUrl The URL of the Amazon SQS queue to which a message is sent. Required
messageBody The message to send. The maximum string size is 256 KB. Required
delaySeconds The length of time, in seconds, for which to delay a specific message. Valid values 0 to 900. Maximum 15 minutes. Optional
messageGroupId This parameter applies only to FIFO queues. The tag that specifies that a message belongs to a specific message group. Optional
region The AWS Region, if not specified the default region will be used. Optional
roleArn The Amazon Resource Name (ARN) of the role to assume. Optional
roleSessionName An identifier for the assumed role session. Optional
roleSessionDuration The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. Optional

Context Output

Path Type Description
AWS.SQS.Queues.SentMessages.MD5OfMessageBody string An MD5 digest of the non-URL-encoded message attribute string.
AWS.SQS.Queues.SentMessages.MD5OfMessageAttributes string An MD5 digest of the non-URL-encoded message attribute string.
AWS.SQS.Queues.SentMessages.MessageId string An attribute containing the MessageId of the message sent to the queue.
AWS.SQS.Queues.SentMessages.SequenceNumber string This parameter applies only to FIFO (first-in-first-out) queues. The large, non-consecutive number that Amazon SQS assigns to each message.

Command Example

!aws-sqs-send-message queueUrl=test.queue.amazonaws.com/1234567/test2 messageBody="test"

Human Readable Output

AWS SQS Queues sent messages

MD5OfMessageBody 123a4bcd4621d373cade4e832627b4f6
MessageId 1a2bc456-1e23-45e6-b789-b1af23c4f56f
QueueUrl test.queue.amazonaws.com/1234567/test2

aws-sqs-create-queue


Creates a new standard or FIFO queue. You can pass one or more attributes in the request.

Base Command

aws-sqs-create-queue

Input

Argument Name Description Required
queueName The name of the new queue. Required
delaySeconds The length of time, in seconds, for which the delivery of all messages in the queue is delayed. Valid values An integer from 0 to 900 seconds (15 minutes). The default is 0 (zero). Optional
maximumMessageSize The limit of how many bytes a message can contain before Amazon SQS rejects it. Valid values An integer from 1,024 bytes (1 KiB) to 262,144 bytes (256 KiB). The default is 262,144 (256 KiB). Optional
messageRetentionPeriod The length of time, in seconds, for which Amazon SQS retains a message. Valid values An integer from 60 seconds (1 minute) to 1,209,600 seconds (14 days). The default is 345,600 (4 days). Optional
receiveMessageWaitTimeSeconds The length of time, in seconds, for which a ReceiveMessage action waits for a message to arrive. Valid values An integer from 0 to 20 (seconds). The default is 0 (zero). Optional
visibilityTimeout The visibility timeout for the queue. Valid values An integer from 0 to 43,200 (12 hours). The default is 30. Optional
kmsDataKeyReusePeriodSeconds The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again. An integer representing seconds, between 60 seconds (1 minute) and 86,400 seconds (24 hours). The default is 300 (5 minutes). A shorter time period provides better security but results in more calls to KMS which might incur charges after Free Tier. Optional
kmsMasterKeyId The ID of an AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK. Optional
policy The queues policy. A valid AWS policy. Optional
fifoQueue Designates a queue as FIFO. Possible values are: True, False. Optional
contentBasedDeduplication Enables content-based deduplication. Possible values are: True, False. Optional
region The AWS Region, if not specified the default region will be used. Optional
roleArn The Amazon Resource Name (ARN) of the role to assume. Optional
roleSessionName An identifier for the assumed role session. Optional
roleSessionDuration The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. Optional

Context Output

Path Type Description
AWS.SQS.Queues.QueueUrl unknown The URL of the created Amazon SQS queue.

Command Example

!aws-sqs-create-queue queueName=test3

Human Readable Output

AWS SQS Queues

| QueueUrl | test.queue.amazonaws.com/1234567/test3 |
| — | — |

aws-sqs-delete-queue


Deletes the queue specified by the QueueUrl , regardless of the queue’s contents. If the specified queue doesn’t exist, Amazon SQS returns a successful response.

Base Command

aws-sqs-delete-queue

Input

Argument Name Description Required
queueUrl The URL of the Amazon SQS queue to delete. Required
region The AWS Region, if not specified the default region will be used. Optional
roleArn The Amazon Resource Name (ARN) of the role to assume. Optional
roleSessionName An identifier for the assumed role session. Optional
roleSessionDuration The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. Optional

Context Output

There is no context output for this command.

Command Example

!aws-sqs-delete-queue queueUrl=example.com/123456789/test3

Human Readable Output

The Queue has been deleted

aws-sqs-purge-queue


Deletes the messages in a queue specified by the QueueURL parameter.

Base Command

aws-sqs-purge-queue

Input

Argument Name Description Required
queueUrl The URL of the queue from which the PurgeQueue action deletes messages. Required
region The AWS Region, if not specified the default region will be used. Optional
roleArn The Amazon Resource Name (ARN) of the role to assume. Optional
roleSessionName An identifier for the assumed role session. Optional
roleSessionDuration The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. Optional

Context Output

There is no context output for this command.

Command Example

!aws-sqs-purge-queue queueUrl=example.com/123456789/test2

Human Readable Output

The Queue has been Purged

Configuration parameters

  • defaultRegion — AWS Default Region (required)
  • roleArn — Role Arn
  • roleSessionName — Role Session Name
  • credentials — Access Key
  • access_key — Access Key
  • secret_key — Secret Key
  • sessionDuration — Role Session Duration
  • queueUrl — QueueURL
  • timeout — Timeout
  • retries — Retries
  • isFetch — Fetch incidents
  • max_fetch — Maximum incidents for one fetch. Hard cap of 100.
  • first_fetch — First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days)
  • incidentType — Incident type
  • incidentFetchInterval — Incidents Fetch Interval
  • sts_regional_endpoint — AWS STS Regional Endpoints
  • proxy — Use system proxy settings
  • insecure — Trust any certificate (not secure)
  • parse_body_as_json — Parse SQS message body as a JSON string

Commands (6)

  • aws-sqs-create-queue

    Creates a new standard or FIFO queue. You can pass one or more attributes in the request.

  • aws-sqs-delete-queue

    Deletes the queue specified by the QueueUrl , regardless of the queue's contents. If the specified queue doesn't exist, Amazon SQS returns a successful response.

  • aws-sqs-get-queue-url

    Returns the URL of an existing queue. To access a queue that belongs to another AWS account, use the queueOwnerAWSAccountId parameter to specify the account ID of the queues owner. The queues owner must grant you permission to access the queue.

  • aws-sqs-list-queues

    Returns a list of your queues. The maximum number of queues that can be returned is 1,000. If you specify a value for the optional QueueNamePrefix parameter, only queues with a name that begins with the specified value are returned.

  • aws-sqs-purge-queue

    Deletes the messages in a queue specified by the QueueURL parameter.

  • aws-sqs-send-message

    Delivers a message to the specified queue.

import demistomock as demisto  # noqa: F401
from CommonServerPython import *  # noqa: F401


def create_entry(title, data, ec):
    return {
        "ContentsFormat": formats["json"],
        "Type": entryTypes["note"],
        "Contents": data,
        "ReadableContentsFormat": formats["markdown"],
        "HumanReadable": tableToMarkdown(title, data) if data else "No result were found",
        "EntryContext": ec,
    }


def raise_error(error):
    return {"Type": entryTypes["error"], "ContentsFormat": formats["text"], "Contents": str(error)}


def get_queue_url(args, client):
    try:
        kwargs = {"QueueName": args.get("queueName")}
        if args.get("queueOwnerAWSAccountId"):
            kwargs.update({"QueueOwnerAWSAccountId": args.get("queueOwnerAWSAccountId")})

        response = client.get_queue_url(**kwargs)
        data = {"QueueUrl": response["QueueUrl"]}

        ec = {"AWS.SQS.Queues": data}
        return create_entry("AWS SQS Queues", data, ec)

    except Exception as e:
        return raise_error(e)


def list_queues(args, client):
    try:
        data = []
        kwargs = {}
        if args.get("queueNamePrefix") is not None:
            kwargs.update({"QueueNamePrefix": args.get("queueNamePrefix")})
        response = client.list_queues(**kwargs)
        for queue in response.get("QueueUrls", []):
            data.append({"QueueUrl": queue})

        ec = {"AWS.SQS.Queues": data}
        return create_entry("AWS SQS Queues", data, ec)

    except Exception as e:
        return raise_error(e)


def send_message(args, client):
    try:
        kwargs = {
            "QueueUrl": args.get("queueUrl"),
            "MessageBody": args.get("messageBody"),
        }
        if args.get("delaySeconds") is not None:
            kwargs.update({"DelaySeconds": int(args.get("delaySeconds"))})
        if args.get("messageGroupId") is not None:
            kwargs.update({"MessageGroupId": int(args.get("messageGroupId"))})

        response = client.send_message(**kwargs)
        data = {
            "QueueUrl": args.get("queueUrl"),
            "MessageId": response["MessageId"],
        }
        if "SequenceNumber" in response:
            data.update({"SequenceNumber": response["SequenceNumber"]})
        if "MD5OfMessageBody" in response:
            data.update({"MD5OfMessageBody": response["MD5OfMessageBody"]})
        if "MD5OfMessageAttributes" in response:
            data.update({"MD5OfMessageAttributes": response["MD5OfMessageAttributes"]})

        ec = {"AWS.SQS.Queues(obj.QueueUrl === val.QueueUrl).SentMessages": data}
        return create_entry("AWS SQS Queues sent messages", data, ec)

    except Exception as e:
        return raise_error(e)


def create_queue(args, client):
    try:
        attributes = {}
        kwargs = {"QueueName": args.get("queueName")}
        if args.get("delaySeconds") is not None:
            attributes.update({"DelaySeconds": args.get("delaySeconds")})
        if args.get("maximumMessageSize") is not None:
            attributes.update({"MaximumMessageSize": args.get("maximumMessageSize")})
        if args.get("messageRetentionPeriod") is not None:
            attributes.update({"MessageRetentionPeriod": args.get("messageRetentionPeriod")})
        if args.get("receiveMessageWaitTimeSeconds") is not None:
            attributes.update({"ReceiveMessageWaitTimeSeconds": args.get("receiveMessageWaitTimeSeconds")})
        if args.get("visibilityTimeout") is not None:
            attributes.update({"VisibilityTimeout": int(args.get("visibilityTimeout"))})
        if args.get("kmsDataKeyReusePeriodSeconds") is not None:
            attributes.update({"KmsDataKeyReusePeriodSeconds": args.get("kmsDataKeyReusePeriodSeconds")})
        if args.get("kmsMasterKeyId") is not None:
            attributes.update({"KmsMasterKeyId": args.get("kmsMasterKeyId")})
        if args.get("policy") is not None:
            attributes.update({"Policy": args.get("policy")})
        if args.get("fifoQueue") is not None:
            attributes.update({"FifoQueue": args.get("fifoQueue")})
        if args.get("contentBasedDeduplication") is not None:
            attributes.update({"ContentBasedDeduplication": args.get("contentBasedDeduplication")})
        if attributes:
            kwargs.update({"Attributes": attributes})

        response = client.create_queue(**kwargs)
        data = {"QueueUrl": response["QueueUrl"]}
        ec = {"AWS.SQS.Queues": data}
        return create_entry("AWS SQS Queues", data, ec)

    except Exception as e:
        return raise_error(e)


def delete_queue(args, client):
    try:
        response = client.delete_queue(QueueUrl=args.get("queueUrl"))
        if response["ResponseMetadata"]["HTTPStatusCode"] == 200:
            return "The Queue has been deleted"

    except Exception as e:
        return raise_error(e)


def purge_queue(args, client):
    try:
        response = client.purge_queue(QueueUrl=args.get("queueUrl"))
        if response["ResponseMetadata"]["HTTPStatusCode"] == 200:
            return "The Queue has been Purged"

    except Exception as e:
        return raise_error(e)


def parse_incident_from_finding(message, parse_body_as_json=False):
    incident = {}
    incident["name"] = "SQS MessageId: " + message["MessageId"]
    if parse_body_as_json:
        try:
            message["Body"] = json.loads(message["Body"])
        except Exception:
            pass
    incident["rawJSON"] = json.dumps(message)
    return incident


def fetch_incidents(aws_client, aws_queue_url, max_fetch, parse_body_as_json):
    """
    Fetching the messages from the queue by following steps:

    1. fetch.
    2. create-incidents (while skipping previous fetch).
    3. save fetch results to context.
    4. try to delete all messages (if not successful, will continue next run).
    """

    try:
        client = aws_client.aws_session(service="sqs")
        # The 'receipt_handles' of the messages that were received from the last call.
        last_receipt_handles = demisto.getLastRun().get("lastReceiptHandles")
        if last_receipt_handles:
            demisto.debug(f'last_receipt_handles before fetch occurred" -> {len(last_receipt_handles)} {last_receipt_handles}')
            last_receipt_handles = set(last_receipt_handles)
        incidents_created = 0  # type: int
        max_number_of_messages = min(max_fetch, 10)
        receipt_handles = []  # type: list
        incidents = []  # type: list
        while incidents_created < max_fetch:
            messages = client.receive_message(
                QueueUrl=aws_queue_url,
                MaxNumberOfMessages=max_number_of_messages,
                VisibilityTimeout=30,
                WaitTimeSeconds=5,
            )

            if "Messages" not in messages:
                if incidents_created == 0:
                    if demisto.command() == "fetch-incidents":
                        demisto.incidents([])
                    return messages, incidents, receipt_handles
                else:
                    break

            # Creating incidents and avoiding creating incidents that were already created previously
            for message in messages["Messages"]:
                receipt_handles.append(message["ReceiptHandle"])
                if last_receipt_handles and message["ReceiptHandle"] in last_receipt_handles:
                    continue
                incidents.append(parse_incident_from_finding(message, parse_body_as_json))
                incidents_created += 1
                if incidents_created == max_fetch:
                    break

        # Save fetch results to context.
        demisto.incidents(incidents)
        # The "receipt_handles" of converted messages to the incidents are saved for next fetch
        demisto.setLastRun({"lastReceiptHandles": receipt_handles})
        demisto.debug(f'last_receipt_handles after fetch occurred" -> {len(receipt_handles)} {receipt_handles}')

        # try to delete all messages (if not successful, will continue next run)
        for receipt_handle in receipt_handles:
            client.delete_message(QueueUrl=aws_queue_url, ReceiptHandle=receipt_handle)

    except Exception as e:
        return raise_error(e)


def test_function(aws_client):
    try:
        client = aws_client.aws_session(service="sqs")
        response = client.list_queues()
        if response["ResponseMetadata"]["HTTPStatusCode"] == 200:
            return "ok"
    except Exception as e:
        return raise_error(e)


def main():
    params = demisto.params()
    aws_default_region = params.get("defaultRegion")
    aws_role_arn = params.get("roleArn")
    aws_role_session_name = params.get("roleSessionName")
    aws_role_session_duration = params.get("sessionDuration")
    aws_role_policy = None
    aws_access_key_id = params.get("credentials", {}).get("identifier") or params.get("access_key")
    aws_secret_access_key = params.get("credentials", {}).get("password") or params.get("secret_key")
    verify_certificate = not params.get("insecure", True)
    timeout = params.get("timeout")
    retries = params.get("retries") or 5
    aws_queue_url = params.get("queueUrl")
    max_fetch = min(arg_to_number(params.get("max_fetch", 10)) or 10, 100)
    parse_body_as_json = params.get("parse_body_as_json", False)

    commands = {
        "aws-sqs-get-queue-url": get_queue_url,
        "aws-sqs-list-queues": list_queues,
        "aws-sqs-send-message": send_message,
        "aws-sqs-create-queue": create_queue,
        "aws-sqs-delete-queue": delete_queue,
        "aws-sqs-purge-queue": purge_queue,
    }

    try:
        validate_params(aws_default_region, aws_role_arn, aws_role_session_name, aws_access_key_id, aws_secret_access_key)
        aws_client = AWSClient(
            aws_default_region,
            aws_role_arn,
            aws_role_session_name,
            aws_role_session_duration,
            aws_role_policy,
            aws_access_key_id,
            aws_secret_access_key,
            verify_certificate,
            timeout,
            retries,
        )
        command = demisto.command()
        args = demisto.args()
        demisto.debug(f"Command being called is {command}")
        if command == "test-module":
            return_results(test_function(aws_client))
        elif demisto.command() == "fetch-incidents":
            fetch_incidents(aws_client, aws_queue_url, max_fetch, parse_body_as_json)
            sys.exit(0)
        elif command in commands:
            client = aws_client.aws_session(
                service="sqs",
                region=args.get("region"),
                role_arn=args.get("roleArn"),
                role_session_name=args.get("roleSessionName"),
                role_session_duration=args.get("roleSessionDuration"),
            )
            return_results(commands[command](args, client))
        else:
            raise NotImplementedError(f"{command} is not an existing AWS-SQS command")

    except Exception as e:
        return_error(f"Failed to execute {demisto.command()} command.\nError:\n{e!s}")


from AWSApiModule import *  # noqa: E402

if __name__ in ("__main__", "__builtin__", "builtins"):
    main()