GooglePubSub

Google Cloud Pub/Sub is a fully-managed real-time messaging service that enables you to send and receive messages between independent applications.

Messaging and Conferencing · Google Cloud Pub / Sub

Details

IDGooglePubSub
ProviderGoogle
CategoryMessaging and Conferencing
From Version5.0.0
Docker Imagedemisto/googleapi-python3:1.0.0.115338
Supported ModulesAgentix XSIAM

README

Google Cloud Pub / Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications.
This integration was integrated and tested with Google Cloud Pub/Sub

Required Permissions

To use this integration you must have a Service Account with one of the following roles:

  • Project-Owner
  • Project-Editor
  • Pub/Sub Admin
  • Pub/Sub Editor

Known Limitations

When clicking on Reset the “last run” timestamp, messages that were recently pulled (including pulls via classification mapper) might take a few minutes before they can be fetched again.
Because the fetch ignores older messages once newer ones were fetched, it’s recommended to wait a few minutes following a reset before trying to fetch again, to prevent older messages from being dropped.

Configure GooglePubSub in Cortex

Parameter Description Required
service_account_json Service Account Private Key File Contents (JSON) True
insecure Trust any certificate (not secure) False
proxy Use system proxy settings False
isFetch Fetch incidents False
incidentType Incident type False
default_subscription Fetch Incidents Subscription ID False
default_project Fetch Incidents Project ID False
default_max_msgs Max Incidents Per Fetch False

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

gcp-pubsub-topics-list


Get a list of the project’s topics.

Base Command

gcp-pubsub-topics-list

Input
Argument Name Description Required
project_id ID of the project to look in. Optional
page_size Max amount of entries to get. Optional
page_token Next page token as returned from "gcp-pubsub-topics-list" command Optional
Context Output
Path Type Description
GoogleCloudPubSubTopics.name String Name of the topic
GoogleCloudPubSub.Topics.nextPageToken String If not empty, indicates that there may be more topics that match the request.
Command Example

!gcp-pubsub-topics-list project_id=dmst-integrations

Context Example
{
    "GoogleCloudPubSubTopics": {
        "name": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Topics for project dmst-integrations

name
projects/dmst-integrations/topics/dmst-topic

gcp-pubsub-topic-publish-message


Publish a message in a topic.

Base Command

gcp-pubsub-topic-publish-message

Input
Argument Name Description Required
topic_id ID of the topic e.g. "projects/{project_id}/topics/topic_id". Required
data The message data field. If this field is empty, the message must contain at least one attribute. Optional
attributes Attributes for this message. If this field is empty, the message must contain non-empty data. Input format: "key=val" pairs sepearated by ",". Optional
project_id Project ID. Optional
delim_char_attributes Set delimiter of attributes split. Optional
Context Output
Path Type Description
GoogleCloudPubSubPublishedMessages.messageId String ID of the published message
GoogleCloudPubSubPublishedMessages.topic String Topic of the published message
GoogleCloudPubSubPublishedMessages.data String Text data of the published message.
GoogleCloudPubSubPublishedMessages.attributes Unknown The message attributes.
Command Example

!gcp-pubsub-topic-publish-message data="42 is the answer" project_id=dmst-integrations topic_id=dmst-topic delim_char_attributes=","

Context Example
{
    "GoogleCloudPubSubPublishedMessages": {
        "attributes": null,
        "data": "42 is the answer",
        "messageId": "874663628353499",
        "topic": "dmst-topic",
        "delim_char_attributes": ","
    }
}
Human Readable Output

Google Cloud PubSub has published the message successfully

Data Message Id Topic
42 is the answer 874663628353499 dmst-topic

gcp-pubsub-topic-subscription-get-by-name


Get subscription details by subscription ID.

Base Command

gcp-pubsub-topic-subscription-get-by-name

Input
Argument Name Description Required
project_id ID of the project from which the subscription is receiving messages. Optional
subscription_id ID of the subscription, without project/topic prefix. Required
Context Output
Path Type Description
GoogleCloudPubSubSubscriptions.ackDeadlineSeconds Number The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message.
GoogleCloudPubSubSubscriptions.expirationPolicy.ttl String The "time-to-live" duration for the subscription.
GoogleCloudPubSubSubscriptions.messageRetentionDuration String How long to retain unacknowledged messages in the subscription's backlog
GoogleCloudPubSubSubscriptions.name String Name of the subscription
GoogleCloudPubSubSubscriptions.topic String Name of the topic from which the subscription is receiving messages
Command Example

!gcp-pubsub-topic-subscription-get-by-name subscription_id=test_sub_2 project_id=dmst-integrations

Context Example
{
    "GoogleCloudPubSubSubscriptions": {
        "ackDeadlineSeconds": 10,
        "expirationPolicy": {
            "ttl": "9999999999s"
        },
        "messageRetentionDuration": "604800s",
        "name": "projects/dmst-integrations/subscriptions/test_sub_2",
        "pushConfig": {},
        "topic": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Subscription test_sub_2

ackDeadlineSeconds expirationPolicy messageRetentionDuration name pushConfig topic
10 ttl: 9999999999s 604800s projects/dmst-integrations/subscriptions/test_sub_2   projects/dmst-integrations/topics/dmst-topic

gcp-pubsub-topic-subscriptions-list


Get a list of subscriptions by project ID or topic ID.

Base Command

gcp-pubsub-topic-subscriptions-list

Input
Argument Name Description Required
project_id ID of the project from which the subscription is receiving messages. Optional
topic_id ID of the topic from which the subscription is receiving messages. Optional
page_size Max number of results Optional
page_token Next page token as returned from the API. Optional
Context Output
Path Type Description
GoogleCloudPubSubSubscriptions.ackDeadlineSeconds Number The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message.
GoogleCloudPubSubSubscriptions.expirationPolicy.ttl String The "time-to-live" duration for the subscription
GoogleCloudPubSubSubscriptions.messageRetentionDuration String How long to retain unacknowledged messages in the subscription's backlog
GoogleCloudPubSubSubscriptions.name String Name of the subscription
GoogleCloudPubSubSubscriptions.topic String Name of the topic from which the subscription is receiving messages.
GoogleCloudPubSubSubscriptions.pushConfig.pushEndpoint String A URL locating the endpoint to which messages should be pushed.
c Unknown If not empty, indicates that there may be more snapshot that match the request.
Command Example

!gcp-pubsub-topic-subscriptions-list project_id=dmst-integrations

Context Example
{
    "GoogleCloudPubSubSubscriptions": [
        {
            "ackDeadlineSeconds": 11,
            "expirationPolicy": {
                "ttl": "2678400s"
            },
            "messageRetentionDuration": "604800s",
            "name": "projects/dmst-integrations/subscriptions/dean-sub1",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "2678400s"
            },
            "messageRetentionDuration": "604800s",
            "name": "projects/dmst-integrations/subscriptions/dean-sub2",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "22678400s"
            },
            "messageRetentionDuration": "604800s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "2678400s"
            },
            "messageRetentionDuration": "604800s",
            "name": "projects/dmst-integrations/subscriptions/test_sub",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "9999999999s"
            },
            "messageRetentionDuration": "604800s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_2",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587031883059",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587032827289",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587039285961",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587038878685",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587039587203",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587040075117",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587042146495",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "test": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/test_sub_1587043084505",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 10,
            "expirationPolicy": {
                "ttl": "86400s"
            },
            "labels": {
                "doc": "true"
            },
            "messageRetentionDuration": "86400s",
            "name": "projects/dmst-integrations/subscriptions/doc_sub",
            "pushConfig": {},
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        },
        {
            "ackDeadlineSeconds": 600,
            "expirationPolicy": {},
            "messageRetentionDuration": "604800s",
            "name": "projects/dmst-integrations/subscriptions/gcf-function-1-us-central1-dmst-topic",
            "pushConfig": {
                "attributes": {
                    "x-goog-version": "v1"
                },
                "pushEndpoint": "https://d4d1290519676f29baf13a7bf18a25bf-dot-j40fd5d18d8c290e1p-tp.appspot.com/_ah/push-handlers/pubsub/projects/dmst-integrations/topics/dmst-topic?pubsub_trigger=true"
            },
            "topic": "projects/dmst-integrations/topics/dmst-topic"
        }
    ]
}
Human Readable Output

Subscriptions in project dmst-integrations

Name Topic Ack Deadline Seconds Labels
projects/dmst-integrations/subscriptions/dean-sub1 projects/dmst-integrations/topics/dmst-topic 11  
projects/dmst-integrations/subscriptions/dean-sub2 projects/dmst-integrations/topics/dmst-topic 10  
projects/dmst-integrations/subscriptions/test_sub_1 projects/dmst-integrations/topics/dmst-topic 10  
projects/dmst-integrations/subscriptions/test_sub projects/dmst-integrations/topics/dmst-topic 10  
projects/dmst-integrations/subscriptions/test_sub_2 projects/dmst-integrations/topics/dmst-topic 10  
projects/dmst-integrations/subscriptions/test_sub_1587031883059 projects/dmst-integrations/topics/dmst-topic 10  
projects/dmst-integrations/subscriptions/test_sub_1587032827289 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/test_sub_1587039285961 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/test_sub_1587038878685 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/test_sub_1587039587203 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/test_sub_1587040075117 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/test_sub_1587042146495 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/test_sub_1587043084505 projects/dmst-integrations/topics/dmst-topic 10 test: true
projects/dmst-integrations/subscriptions/doc_sub projects/dmst-integrations/topics/dmst-topic 10 doc: true
projects/dmst-integrations/subscriptions/gcf-function-1-us-central1-dmst-topic projects/dmst-integrations/topics/dmst-topic 600  

gcp-pubsub-topic-messages-pull


Pull messages that were published.

Base Command

gcp-pubsub-topic-messages-pull

Input
Argument Name Description Required
project_id Project ID to pull messages from. Optional
subscription_id Subscription ID to pull messages from. Required
max_messages The maximum number of messages to return for this request. Must be a positive integer. Optional
ack Acknowledge the messages pulled. Optional
Context Output
Path Type Description
GoogleCloudPubSubPulledMessages.data String Text data of the pulled message.
GoogleCloudPubSubPulledMessages.messageId String ID of the message
GoogleCloudPubSubPulledMessages.publishTime Date The time the message was published
GoogleCloudPubSubPulledMessages.attributes Unknown The message attributes.
Command Example

!gcp-pubsub-topic-messages-pull ack=true max_messages=1 project_id=dmst-integrations subscription_id=test_sub_2

Context Example
{
    "GoogleCloudPubSubPulledMessages": {
        "data": "42 is the answer",
        "messageId": "874662740221427",
        "publishTime": "2020-04-16T13:32:41.398Z"
    }
}
Human Readable Output

Google Cloud PubSub Messages

data messageId publishTime
42 is the answer 874662740221427 2020-04-16T13:32:41.398Z

gcp-pubsub-topic-subscription-create


Create a pull or push subscription.

Base Command

gcp-pubsub-topic-subscription-create

Input
Argument Name Description Required
project_id ID of the project from which the subscription is receiving messages. Optional
subscription_id ID of the created subscription. Required
topic_id ID of the topic from which the subscription is receiving messages. Required
push_endpoint A URL locating the endpoint to which messages should be pushed. Optional
push_attributes Endpoint configuration attributes that can be used to control the message delivery, such as "x-goog-version", which you can use to change the format of the pushed message. Input format: "key=val" pairs sepearated by ",". Optional
ack_deadline_seconds The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. Optional
retain_acked_messages Indicates whether to retain acknowledged messages. Optional
message_retention_duration How long to retain unacknowledged messages in the subscription's backlog. A duration of seconds e.g. "4.2s" Optional
labels Input format: "key=val" pairs sepearated by ",". Optional
expiration_ttl The "time-to-live" duration for the subscription. Optional
Context Output
Path Type Description
GoogleCloudPubSubSubscriptions.ackDeadlineSeconds Number The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message.
GoogleCloudPubSubSubscriptions.expirationPolicy.ttl String The "time-to-live" duration for the subscription.
GoogleCloudPubSubSubscriptions.messageRetentionDuration String How long to retain unacknowledged messages in the subscription's backlog
GoogleCloudPubSubSubscriptions.name String Name of the subscription
GoogleCloudPubSubSubscriptions.topic String Name of the topic from which the subscription is receiving messages
GoogleCloudPubSubSubscriptions.projectName String Name of the project from which the subscription is receiving messages
GoogleCloudPubSubSubscriptions.subscriptionName String Name of the newly created subscription
GoogleCloudPubSubSubscriptions.labels String An object containing a list of "key": value pairs.
Command Example

!gcp-pubsub-topic-subscription-create expiration_ttl=86400s project_id=dmst-integrations topic_id=dmst-topic subscription_id=doc_sub_1

Context Example
{
    "GoogleCloudPubSubSubscriptions": {
        "ackDeadlineSeconds": 10,
        "expirationPolicy": {
            "ttl": "86400s"
        },
        "messageRetentionDuration": "86400s",
        "name": "projects/dmst-integrations/subscriptions/doc_sub_1",
        "projectName": "dmst-integrations",
        "pushConfig": {},
        "subscriptionName": "doc_sub_1",
        "topic": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Subscription doc_sub_1 was created successfully

ackDeadlineSeconds expirationPolicy messageRetentionDuration name pushConfig topic
10 ttl: 86400s 86400s projects/dmst-integrations/subscriptions/doc_sub_1   projects/dmst-integrations/topics/dmst-topic

gcp-pubsub-topic-create


Create a topic.

Base Command

gcp-pubsub-topic-create

Input
Argument Name Description Required
project_id ID of the project the topic will belong to. Optional
topic_id ID of the newly created topic. Required
labels 'Input format: "key=val" pairs sepearated by ",".' Optional
allowed_persistence_regions A comma separated list of IDs of GCP regions where messages that are published to the topic may be persisted in storage. e.g. "us-east4,asia-1".
https://cloud.google.com/compute/docs/regions-zones#locations
Optional
kms_key_name The full name of the Cloud KMS CryptoKey to be used to restrict access to messages published on this topic.

Full name format: projects//locations//keyRings//cryptoKeys/.
Optional
Context Output
Path Type Description
GoogleCloudPubSubTopics.name String Name of the topic
GoogleCloudPubSubTopics.labels Unknown An object containing a list of "key": value pairs.
GoogleCloudPubSubTopics.messageStoragePolicy.allowedPersistenceRegions Unknown A list of IDs of GCP regions where messages that are published to the topic may be persisted in storage.
GoogleCloudPubSubTopics.kmsKeyName String The resource name of the Cloud KMS CryptoKey to be used to restrict access.
Command Example

!gcp-pubsub-topic-create project_id=dmst-integrations topic_id=dmst-doc-topic

Context Example
{
    "GoogleCloudPubSubTopics": {
        "name": "projects/dmst-integrations/topics/dmst-doc-topic"
    }
}
Human Readable Output

Topic dmst-doc-topic was created successfully

Name
projects/dmst-integrations/topics/dmst-doc-topic

gcp-pubsub-topic-delete


Delete a topic.

Base Command

gcp-pubsub-topic-delete

Input
Argument Name Description Required
project_id ID of the project the topic will belong to. Optional
topic_id ID of the newly created topic. Required
Context Output

There is no context output for this command.

Command Example

!gcp-pubsub-topic-delete project_id=dmst-integrations topic_id=dmst-doc-topic

Context Example
{}
Human Readable Output

Topic dmst-doc-topic was deleted successfully

gcp-pubsub-topic-update


Updates a topic.

Base Command

gcp-pubsub-topic-update

Input
Argument Name Description Required
project_id ID of the project the topic belongs to. Optional
topic_id ID of the topic. Required
labels 'Input format: "key=val" pairs sepearated by ",".' Optional
allowed_persistence_regions A comma separated list of IDs of GCP regions where messages that are published to the topic may be persisted in storage. e.g. "us-east4,asia-1".
https://cloud.google.com/compute/docs/regions-zones#locations
Optional
kms_key_name The full name of the Cloud KMS CryptoKey to be used to restrict access to messages published on this topic.

Full name format: projects//locations//keyRings//cryptoKeys/.
Optional
update_mask Indicates which fields in the provided topic to update.
A comma-separated list of fields. Example: "labels,messageStoragePolicy".
Required
Context Output
Path Type Description
GoogleCloudPubSubTopics.name String Name of the topic
GoogleCloudPubSubTopics.labels Unknown An object containing a list of "key": value pairs.
GoogleCloudPubSubTopics.messageStoragePolicy.allowedPersistenceRegions Unknown A list of IDs of GCP regions where messages that are published to the topic may be persisted in storage.
GoogleCloudPubSubTopics.kmsKeyName String The resource name of the Cloud KMS CryptoKey to be used to restrict access.
Command Example

!gcp-pubsub-topic-update project_id=dmst-integrations topic_id=dmst-doc-topic labels="doc=true" update_mask=labels

Context Example
{
    "GoogleCloudPubSubTopics": {
        "labels": {
            "doc": "true"
        },
        "name": "projects/dmst-integrations/topics/dmst-doc-topic"
    }
}
Human Readable Output

Topic dmst-doc-topic was updated successfully

Labels Name
doc: true projects/dmst-integrations/topics/dmst-doc-topic

gcp-pubsub-topic-subscription-update


Update a subscription.

Base Command

gcp-pubsub-topic-subscription-update

Input
Argument Name Description Required
update_mask Indicates which fields in the provided subscription to update.
A comma-separated list of fully qualified names of fields.
Example: "pushConfig.pushEndpoint,ackDeadlineSeconds".
Required
project_id ID of the project from which the subscription is receiving messages. Optional
subscription_id ID of the updated subscription. Required
topic_id ID of the topic from which the subscription is receiving messages. Required
push_endpoint A URL locating the endpoint to which messages should be pushed. Optional
push_attributes Endpoint configuration attributes that can be used to control the message delivery. Input format: "key=val" pairs sepearated by ",". Optional
ack_deadline_seconds The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. Optional
retain_acked_messages Indicates whether to retain acknowledged messages. Optional
message_retention_duration How long to retain unacknowledged messages in the subscription's backlog. A duration of seconds e.g. "4.2s" Optional
labels Input format: "key=val" pairs sepearated by ",". Optional
expiration_ttl The "time-to-live" duration for the subscription. Optional
Context Output
Path Type Description
GoogleCloudPubSubSubscriptions.ackDeadlineSeconds Number The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message.
GoogleCloudPubSubSubscriptions.expirationPolicy.ttl String The "time-to-live" duration for the subscription.
GoogleCloudPubSubSubscriptions.messageRetentionDuration String How long to retain unacknowledged messages in the subscription's backlog.
GoogleCloudPubSubSubscriptions.name String Name of the subscription.
GoogleCloudPubSubSubscriptions.topic String Name of the topic from which the subscription is receiving messages.
GoogleCloudPubSubSubscriptions.projectName String Name of the project from which the subscription is receiving messages.
GoogleCloudPubSubSubscriptions.subscriptionName String Name of the subscription.
GoogleCloudPubSubSubscriptions.labels String An object containing a list of "key": value pairs.
Command Example

!gcp-pubsub-topic-subscription-update labels="doc=true" project_id=dmst-integrations subscription_id=doc_sub_1 topic_id=dmst-topic update_mask=labels

Context Example
{
    "GoogleCloudPubSubSubscriptions": {
        "ackDeadlineSeconds": 10,
        "expirationPolicy": {
            "ttl": "86400s"
        },
        "labels": {
            "doc": "true"
        },
        "messageRetentionDuration": "86400s",
        "name": "projects/dmst-integrations/subscriptions/doc_sub_1",
        "projectName": "dmst-integrations",
        "pushConfig": {
            "attributes": {
                "x-goog-version": "v1"
            }
        },
        "subscriptionName": "doc_sub_1",
        "topic": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Subscription doc_sub_1 was updated successfully

ackDeadlineSeconds expirationPolicy labels messageRetentionDuration name pushConfig topic
10 ttl: 86400s doc: true 86400s projects/dmst-integrations/subscriptions/doc_sub_1 attributes: {“x-goog-version”: “v1”} projects/dmst-integrations/topics/dmst-topic

gcp-pubsub-topic-messages-seek


Seeks a subscription to a given point in time or to a given snapshot.

Base Command

gcp-pubsub-topic-messages-seek

Input
Argument Name Description Required
project_id ID of the project from which the subscription is receiving messages. Optional
subscription_id ID of the subscription, without project/topic prefix. Required
time_string A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2014-10-02T15:01:23.045123456Z". Optional
snapshot The snapshot to seek to. Optional
Context Output

There is no context output for this command.

Command Example

!gcp-pubsub-topic-messages-seek time_string="2020-04-16T13:27:55.117Z" project_id=dmst-integrations topic_id=dmst-topic subscription_id=doc_sub_1

Context Example
{}
Human Readable Output

Message seek was successful for time: 2020-04-16T13:27:55.117Z

gcp-pubsub-topic-snapshots-list


Get a list of snapshots by project ID and topic ID.

Base Command

gcp-pubsub-topic-snapshots-list

Input
Argument Name Description Required
project_id The ID of the project from which this snapshot is retaining messages. Optional
topic_id The ID of the topic from which this snapshot is retaining messages. Optional
page_size Max number of results Optional
page_token Next page token as returned from the API. Optional
Context Output
Path Type Description
GoogleCloudPubSubSnapshots.name String The name of the snapshot.
GoogleCloudPubSubSnapshots.topic Unknown The name of the topic from which this snapshot is retaining messages.
GoogleCloudPubSubSnapshots.expireTime Date The snapshot is guaranteed to exist up until this time.
GoogleCloudPubSubSnapshots.labels Unknown An object containing a list of "key": value pairs.
GoogleCloudPubSub.Snapshots.nextPageToken String If not empty, indicates that there may be more snapshot that match the request.
Command Example

!gcp-pubsub-topic-snapshots-list project_id=dmst-integrations

Context Example
{
    "GoogleCloudPubSubSnapshots": {
        "expireTime": "2020-04-23T13:37:26.199Z",
        "labels": {
            "doc": "true"
        },
        "name": "projects/dmst-integrations/snapshots/doc_snapshot",
        "topic": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Snapshots for project dmst-integrations

name
projects/dmst-integrations/snapshots/doc_snapshot

gcp-pubsub-topic-snapshot-create


Creates a snapshot from the requested subscription. Snapshots are used in gcp-pubsub-topic-messages-seek command.

Base Command

gcp-pubsub-topic-snapshot-create

Input
Argument Name Description Required
project_id ID of the project from which the subscription is receiving messages. Optional
subscription_id The subscription whose backlog the snapshot retains. Required
labels Input format: "key=val" pairs sepearated by ",". Optional
snapshot_id The id of the snapshot. Required
Context Output
Path Type Description
GoogleCloudPubSubSnapshots.name String The name of the snapshot.
GoogleCloudPubSubSnapshots.topic Unknown The name of the topic from which this snapshot is retaining messages.
GoogleCloudPubSubSnapshots.expireTime Date The snapshot is guaranteed to exist up until this time.
GoogleCloudPubSubSnapshots.labels Unknown An object containing a list of "key": value pairs
Command Example

!gcp-pubsub-topic-snapshot-create project_id=dmst-integrations subscription_id=test_sub_2 snapshot_id=doc_snapshot

Context Example
{
    "GoogleCloudPubSubSnapshots": {
        "expireTime": "2020-04-23T13:37:26.199Z",
        "name": "projects/dmst-integrations/snapshots/doc_snapshot",
        "topic": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Snapshot doc_snapshot was created successfully

Expire Time Name Topic
2020-04-23T13:37:26.199Z projects/dmst-integrations/snapshots/doc_snapshot projects/dmst-integrations/topics/dmst-topic

gcp-pubsub-topic-snapshot-update


Updates an existing snapshot. Snapshots are used in gcp-pubsub-topic-messages-seek command.

Base Command

gcp-pubsub-topic-snapshot-update

Input
Argument Name Description Required
project_id The ID of the project from which the subscription is receiving messages. Optional
expire_time The snapshot is guaranteed to exist up until this time. A newly-created snapshot expires no later than 7 days from the time of its creation.

A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2020-04-01T08:01:23.045678910Z"
Optional
labels Input format: "key=val" pairs sepearated by ",". Optional
snapshot_id The id of the snapshot. Required
update_mask Indicates which fields in the provided snapshot to update.
A comma-separated list of fields. Example: "labels,topic,expireTime".
Required
topic_id The ID of the topic from which this snapshot is retaining messages. Required
Context Output
Path Type Description
GoogleCloudPubSubSnapshots.name String The name of the snapshot.
GoogleCloudPubSubSnapshots.topic Unknown The name of the topic from which this snapshot is retaining messages.
GoogleCloudPubSubSnapshots.expireTime Date The snapshot is guaranteed to exist up until this time.
GoogleCloudPubSubSnapshots.labels Unknown An object containing a list of "key": value pairs
Command Example

!gcp-pubsub-topic-snapshot-update project_id=dmst-integrations subscription_id=test_sub_2 snapshot_id=doc_snapshot labels="doc=true" update_mask=labels topic_id=dmst-topic

Context Example
{
    "GoogleCloudPubSubSnapshots": {
        "expireTime": "2020-04-23T13:37:26.199Z",
        "labels": {
            "doc": "true"
        },
        "name": "projects/dmst-integrations/snapshots/doc_snapshot",
        "topic": "projects/dmst-integrations/topics/dmst-topic"
    }
}
Human Readable Output

Snapshot doc_snapshot was updated successfully

Expire Time Labels Name Topic
2020-04-23T13:37:26.199Z doc: true projects/dmst-integrations/snapshots/doc_snapshot projects/dmst-integrations/topics/dmst-topic

gcp-pubsub-topic-snapshot-delete


Removes an existing snapshot.

Base Command

gcp-pubsub-topic-snapshot-delete

Input
Argument Name Description Required
project_id The ID of the project from which the subscription is receiving messages. Optional
snapshot_id The id of the snapshot. Required
Context Output

There is no context output for this command.

Command Example

!gcp-pubsub-topic-snapshot-delete project_id=dmst-integrations snapshot_id=doc_snapshot

Context Example
{}
Human Readable Output

Snapshot doc_snapshot was deleted successfully

gcp-pubsub-topic-snapshot-delete


Removes an existing snapshot.

Base Command

gcp-pubsub-topic-ack-messages

Input
Argument Name Description Required
subscription_id The subscription that will have the messages acked. Required
ack_ids List of comma separated ids to ACK, as received from “gcp-pubsub-topic-messages-pull” or from “fetch-incidents”. Required
project_id The project id that the messages were pulled from. Optional
Context Output

There is no context output for this command.

Command Example

!gcp-pubsub-topic-ack-messages ack_ids=example_ack_id subscription_id=test_sub_2

Context Example
{}
Human Readable Output

Subscription test_sub_2 had the following ids acknowledged

ACK ID
example_ack_id

Configuration parameters

  • isFetch — Fetch incidents
  • incidentType — Incident type
  • incidentFetchInterval — Incidents Fetch Interval
  • service_account_json — Service account private key file contents (JSON).
  • credentials
  • insecure — Trust any certificate (not secure)
  • proxy — Use system proxy settings
  • default_project — Default project ID. (required)
  • default_subscription — Fetch incidents using the subscription ID.
  • default_max_msgs — Maximum number of incidents per fetch.
  • ack_incidents — Acknowledge incidents.
  • first_fetch_time — First fetch time.

Commands (16)

  • gcp-pubsub-topic-ack-messages

    Acknowledge previously pulled message or messages.

  • gcp-pubsub-topic-create

    Create a topic.

  • gcp-pubsub-topic-delete

    Delete a topic.

  • gcp-pubsub-topic-messages-pull

    Pull messages that were published.

  • gcp-pubsub-topic-messages-seek

    Seeks a subscription to a given point in time or to a given snapshot.

  • gcp-pubsub-topic-publish-message

    Publish a message in a topic.

  • gcp-pubsub-topic-snapshot-create

    Creates a snapshot from the requested subscription. Snapshots are used in the gcp-pubsub-topic-messages-seek command.

  • gcp-pubsub-topic-snapshot-delete

    Removes an existing snapshot.

  • gcp-pubsub-topic-snapshot-update

    Updates an existing snapshot. Snapshots are used in gcp-pubsub-topic-messages-seek command.

  • gcp-pubsub-topic-snapshots-list

    Get a list of snapshots by project ID and topic ID.

  • gcp-pubsub-topic-subscription-create

    Create a pull or push subscription.

  • gcp-pubsub-topic-subscription-get-by-name

    Get subscription details by the subscription ID.

  • gcp-pubsub-topic-subscription-update

    Update a subscription.

  • gcp-pubsub-topic-subscriptions-list

    Get a list of subscriptions by project ID or topic ID.

  • gcp-pubsub-topic-update

    Updates a topic.

  • gcp-pubsub-topics-list

    Get a list of the topics of the project.

import json
import traceback
import urllib.parse

# IMPORTS
import dateparser
import demistomock as demisto
import httplib2
import urllib3
from CommonServerPython import *  # noqa: E402 lgtm [py/polluting-import]
from googleapiclient import discovery
from oauth2client import service_account

from CommonServerUserPython import *  # noqa: E402 lgtm [py/polluting-import]

# Disable insecure warnings
urllib3.disable_warnings()

# CONSTANTS
SERVICE_NAME = "pubsub"
SERVICE_VERSION = "v1"
SCOPES = ["https://www.googleapis.com/auth/cloud-platform"]
ISO_DATE_FORMAT = "%Y-%m-%dT%H:%M:%S.%fZ"
LAST_RUN_TIME_KEY = "fetch_time"
LAST_RUN_FETCHED_KEY = "fetched_ids"

""" HELPER CLASSES """


class GoogleNameParser:
    """
    Used to easily transform Google Cloud Pub/Sub names
    """

    FULL_PROJECT_PREFIX = "projects/{}"
    FULL_TOPIC_PREFIX = "/topics/{}"
    FULL_SUBSCRIPTION_PREFIX = "/subscriptions/{}"
    FULL_SNAPSHOT_PREFIX = "/snapshots/{}"

    @staticmethod
    def get_project_name(project_id):
        return GoogleNameParser.FULL_PROJECT_PREFIX.format(project_id)

    @staticmethod
    def get_topic_name(project_id, topic_id):
        return GoogleNameParser.get_project_name(project_id) + GoogleNameParser.FULL_TOPIC_PREFIX.format(topic_id)

    @staticmethod
    def get_subscription_project_name(project_id, subscription_id):
        return GoogleNameParser.get_project_name(project_id) + GoogleNameParser.FULL_SUBSCRIPTION_PREFIX.format(subscription_id)

    @staticmethod
    def get_subscription_topic_name(project_id, topic_id, subscription_id):
        return GoogleNameParser.get_topic_name(project_id, topic_id) + GoogleNameParser.FULL_SUBSCRIPTION_PREFIX.format(
            subscription_id
        )

    @staticmethod
    def get_snapshot_project_name(project_id, snapshot_id):
        return GoogleNameParser.get_project_name(project_id) + GoogleNameParser.FULL_SNAPSHOT_PREFIX.format(snapshot_id)


# disable-secrets-detection-start
class BaseGoogleClient:
    """
    A Client class to wrap the google cloud api library as a service.
    """

    def __init__(
        self,
        service_name: str,
        service_version: str,
        client_secret: dict,
        scopes: list,
        proxy: bool,
        insecure: bool,
        **kwargs,
    ):
        """
        :param service_name: The name of the service. You can find this and the service  here
         https://github.com/googleapis/google-api-python-client/blob/master/docs/dyn/index.md
        :param service_version: The version of the API.
        :param client_secret: A string of the generated credentials.json
        :param scopes: The scope needed for the project. (i.e. ['https://www.googleapis.com/auth/cloud-platform'])
        :param proxy: Proxy flag
        :param insecure: Insecure flag
        :param kwargs: Potential arguments dict
        """
        credentials = service_account.ServiceAccountCredentials.from_json_keyfile_dict(client_secret, scopes=scopes)
        if proxy or insecure:
            http_client = credentials.authorize(self.get_http_client_with_proxy(proxy, insecure))
            self.service = discovery.build(service_name, service_version, http=http_client)
        else:
            self.service = discovery.build(service_name, service_version, credentials=credentials)

    @staticmethod
    def get_http_client_with_proxy(proxy, insecure):
        """
        Create an http client with proxy with whom to use when using a proxy.
        :param proxy: Whether to use a proxy.
        :param insecure: Whether to disable ssl and use an insecure connection.
        :return:
        """
        if proxy:
            proxies = handle_proxy()
            https_proxy = proxies.get("https")
            http_proxy = proxies.get("http")
            proxy_conf = https_proxy if https_proxy else http_proxy
            # if no proxy_conf - ignore proxy
            if proxy_conf:
                if not proxy_conf.startswith("https") and not proxy_conf.startswith("http"):
                    proxy_conf = "https://" + proxy_conf
                parsed_proxy = urllib.parse.urlparse(proxy_conf)
                proxy_info = httplib2.ProxyInfo(
                    proxy_type=httplib2.socks.PROXY_TYPE_HTTP,
                    proxy_host=parsed_proxy.hostname,
                    proxy_port=parsed_proxy.port,
                    proxy_user=parsed_proxy.username,
                    proxy_pass=parsed_proxy.password,
                )
                return httplib2.Http(proxy_info=proxy_info, disable_ssl_certificate_validation=insecure)
        return httplib2.Http(disable_ssl_certificate_validation=insecure)


# disable-secrets-detection-end


class PubSubClient(BaseGoogleClient):
    def __init__(
        self,
        default_project,
        default_subscription,
        default_max_msgs,
        client_secret,
        **kwargs,
    ):
        super().__init__(client_secret=client_secret, **kwargs)
        self.default_project = default_project
        if not default_project:
            self.default_project = self._extract_project_from_client_secret(client_secret)
        self.default_subscription = default_subscription
        self.default_max_msgs = default_max_msgs

    def _extract_project_from_client_secret(self, client_secret):
        """Extracts project name from a client secret json"""
        project_id = client_secret.get("project_id")
        if isinstance(project_id, list):
            project_id = project_id[0]
        return project_id

    def _create_subscription_body(
        self,
        ack_deadline_seconds,
        expiration_ttl,
        labels,
        message_retention_duration,
        push_attributes,
        push_endpoint,
        retain_acked_messages,
        topic_name,
    ):
        """Create a subscription body"""
        if push_endpoint or push_attributes:
            push_config = assign_params(
                pushEndpoint=push_endpoint,
                attributes=push_attributes,
            )
        else:
            push_config = None
        body = assign_params(
            topic=topic_name,
            pushConfig=push_config,
            ackDeadlineSeconds=ack_deadline_seconds,
            retainAckedMessages=retain_acked_messages,
            messageRetentionDuration=message_retention_duration,
            labels=labels,
            expirationPolicy=assign_params(ttl=expiration_ttl),
        )
        return body

    def _create_topic_body(self, allowed_persistence_regions, kms_key_name, labels):
        """Create a topic body"""
        message_storage_policy = assign_params(allowedPersistenceRegions=allowed_persistence_regions)
        body = assign_params(
            labels=labels,
            messageStoragePolicy=message_storage_policy,
            kmsKeyName=kms_key_name,
        )
        return body

    def list_topic(self, project_id, page_size, page_token=None):
        """Get topic list from GoogleClient"""
        return self.service.projects().topics().list(project=project_id, pageSize=page_size, pageToken=page_token).execute()

    def list_topic_subs(self, topic_id, page_size, page_token=None):
        """Get topic subscriptions from GoogleClient"""
        return (
            self.service.projects()
            .topics()
            .subscriptions()
            .list(topic=topic_id, pageSize=page_size, pageToken=page_token)
            .execute()
        )

    def list_project_subs(self, project_id, page_size, page_token=None):
        """Get project subscriptions list from GoogleClient"""
        return (
            self.service.projects().subscriptions().list(project=project_id, pageSize=page_size, pageToken=page_token).execute()
        )

    def get_sub(self, sub_name):
        """Get subscription by name from GoogleClient"""
        return self.service.projects().subscriptions().get(subscription=sub_name).execute()

    def publish_message(self, project_id, topic_id, req_body):
        """Publish a topic message via GoogleClient"""
        return (
            self.service.projects()
            .topics()
            .publish(
                topic=GoogleNameParser.get_topic_name(project_id, topic_id),
                body=req_body,
            )
            .execute()
        )

    def pull_messages(self, sub_name, max_messages, ret_immediately=True):
        """
        Pull messages for the subscription
        :param sub_name: Subscription name
        :param max_messages: The maximum number of messages to return for this request. Must be a positive integer
        :param ret_immediately: when set to true will return immediately, otherwise will be async
        :return: Messages
        """
        req_body = {"returnImmediately": ret_immediately, "maxMessages": max_messages}
        return self.service.projects().subscriptions().pull(subscription=sub_name, body=req_body).execute()

    def ack_messages(self, sub_name, acks):
        """
        Ack a list of messages
        :param sub_name: subscription name
        :param acks: ack ids to ack
        :return:
        """
        body = {"ackIds": acks}
        return self.service.projects().subscriptions().acknowledge(subscription=sub_name, body=body).execute()

    def create_subscription(
        self,
        sub_name,
        topic_name,
        push_endpoint,
        push_attributes,
        ack_deadline_seconds,
        retain_acked_messages,
        message_retention_duration,
        labels,
        expiration_ttl,
    ):
        """
        Creates a subscription
        :param sub_name: full sub name
        :param topic_name: full topic name
        :param push_endpoint: A URL locating the endpoint to which messages should be pushed.
        :param push_attributes: Input format: "key=val" pairs sepearated by ",".
        :param ack_deadline_seconds: The amount of time Pub/Sub waits for the subscriber to ack.
        :param retain_acked_messages: if 'true' then retain acknowledged messages
        :param message_retention_duration: How long to retain unacknowledged messages
        :param labels: Input format: "key=val" pairs sepearated by ",".
        :param expiration_ttl: The "time-to-live" duration for the subscription.

        :return: Subscription
        """
        body = self._create_subscription_body(
            ack_deadline_seconds,
            expiration_ttl,
            labels,
            message_retention_duration,
            push_attributes,
            push_endpoint,
            retain_acked_messages,
            topic_name,
        )
        return self.service.projects().subscriptions().create(name=sub_name, body=body).execute()

    def update_subscription(
        self,
        sub_name,
        topic_name,
        update_mask,
        push_endpoint,
        push_attributes,
        ack_deadline_seconds,
        retain_acked_messages,
        message_retention_duration,
        labels,
        expiration_ttl,
    ):
        """
        Updates a subscription
        :param sub_name: full sub name
        :param topic_name: full topic name
        :param update_mask: Indicates which fields in the provided subscription to update.
        :param push_endpoint: A URL locating the endpoint to which messages should be pushed.
        :param push_attributes: Input format: "key=val" pairs sepearated by ",".
        :param ack_deadline_seconds: The amount of time Pub/Sub waits for the subscriber to ack.
        :param retain_acked_messages: if 'true' then retain acknowledged messages
        :param message_retention_duration: How long to retain unacknowledged messages
        :param labels: Input format: "key=val" pairs sepearated by ",".
        :param expiration_ttl: The "time-to-live" duration for the subscription.

        :return: Subscription
        """
        sub_body = self._create_subscription_body(
            ack_deadline_seconds,
            expiration_ttl,
            labels,
            message_retention_duration,
            push_attributes,
            push_endpoint,
            retain_acked_messages,
            topic_name,
        )
        body = assign_params(subscription=sub_body, updateMask=update_mask)
        return self.service.projects().subscriptions().patch(name=sub_name, body=body).execute()

    def create_topic(self, topic_name, labels, allowed_persistence_regions, kms_key_name):
        """
        Create a topic in the project
        :param topic_name: name of the topic to be created
        :param labels: "key=val" pairs sepearated by ",".'
        :param allowed_persistence_regions: an str representing a list of IDs of GCP regions
        :param kms_key_name: The full name of the Cloud KMS CryptoKey to be used to restrict access on this topic.
        :return: Topic
        """
        body = self._create_topic_body(allowed_persistence_regions, kms_key_name, labels)
        return self.service.projects().topics().create(name=topic_name, body=body).execute()

    def delete_topic(self, topic_name):
        """
        Deletes a topic in the project
        :param topic_name: name of the topic to be created
        :return: Delete response
        """
        return self.service.projects().topics().delete(topic=topic_name).execute()

    def update_topic(self, topic_name, labels, allowed_persistence_regions, kms_key_name, update_mask):
        """
        Updates a topic in the project
        :param topic_name: name of the topic to be updated
        :param labels: "key=val" pairs sepearated by ",".'
        :param allowed_persistence_regions: an str representing a list of IDs of GCP regions
        :param kms_key_name: The full name of the Cloud KMS CryptoKey to be used to restrict access on this topic.
        :param update_mask: Indicates which fields in the provided topic to update.
        :return: Topic
        """
        topic = self._create_topic_body(allowed_persistence_regions, kms_key_name, labels)
        body = assign_params(topic=topic, updateMask=update_mask)
        return self.service.projects().topics().patch(name=topic_name, body=body).execute()

    def subscription_seek_message(self, subscription_name, time_string, snapshot=None):
        """
        Seeks messages in subscription
        :param subscription_name: Subscription to seek messages for
        :param time_string: A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds,
        :param snapshot: The snapshot to seek to.
        :return: Empty string if successful
        """
        body = assign_params(time=time_string, snapshot=snapshot)
        return self.service.projects().subscriptions().seek(subscription=subscription_name, body=body).execute()

    def get_topic_snapshots_list(self, topic_name, page_size, page_token=None):
        """
        Get snapshots list
        :param topic_name: The name of the topic from which this snapshot is retaining messages.
        :param page_size: Max number of results
        :param page_token: Next page token as returned from the API.
        :return:
        """
        return (
            self.service.projects()
            .topics()
            .snapshots()
            .list(topic=topic_name, pageSize=page_size, pageToken=page_token)
            .execute()
        )

    def get_project_snapshots_list(self, project_name, page_size, page_token):
        """
        Get snapshots list
        :param project_name: The name of the project from which this snapshot is retaining messages.
        :param page_size: Max number of results
        :param page_token: Next page token as returned from the API.
        :return: Snapshot list
        """
        return self.service.projects().snapshots().list(project=project_name, pageSize=page_size, pageToken=page_token).execute()

    def create_snapshot(self, subscription_name, snapshot_name, labels):
        """
        Create a snapshot
        :param subscription_name: The subscription whose backlog the snapshot retain
        :param snapshot_name: The name of the snapshot
        :param labels: labels dict
        :return: Snapshot
        """
        body = assign_params(subscription=subscription_name, labels=labels)
        return self.service.projects().snapshots().create(name=snapshot_name, body=body).execute()

    def update_snapshot(self, snapshot_name, topic_name, update_mask, expire_time, labels):
        """
        :param snapshot_name: The name of the snapshot
        :param topic_name: The ID of the topic from which this snapshot is retaining messages.
        :param update_mask: Indicates which fields in the provided snapshot to update.
        :param expire_time: A timestamp in RFC3339 UTC "Zulu" format
        :param labels: labels dict
        :return: Snapshot
        """
        snapshot = assign_params(name=snapshot_name, topic=topic_name, expireTime=expire_time, labels=labels)
        body = assign_params(snapshot=snapshot, updateMask=update_mask)
        return self.service.projects().snapshots().patch(name=snapshot_name, body=body).execute()

    def delete_snapshot(self, snapshot_name):
        """
        Delete a snapshot
        :param snapshot_name: full snapshot name
        :return: Empty response
        """
        return self.service.projects().snapshots().delete(snapshot=snapshot_name).execute()


""" HELPER FUNCTIONS"""


def init_google_client(
    default_subscription,
    default_project,
    default_max_msgs,
    insecure,
    credentials: dict = None,
    service_account_json: str = None,
    **kwargs,
) -> PubSubClient:
    """
    Initializes google client
    :param service_account_json: A string of the generated credentials.json
    :param default_subscription: Default subscription to use
    :param default_project: Default project to use
    :param default_max_msgs: Max messages to pull per fetch
    :param insecure: Flag - do not validate https certs
    :param kwargs:
    :return:
    """
    try:
        service_account_json = json.loads(
            str(credentials.get("password") if isinstance(credentials, dict) else service_account_json)
        )
        client = PubSubClient(
            default_project=default_project,
            default_subscription=default_subscription,
            default_max_msgs=default_max_msgs,
            service_name=SERVICE_NAME,
            service_version=SERVICE_VERSION,
            client_secret=service_account_json,
            scopes=SCOPES,
            insecure=insecure,
            **kwargs,
        )
        return client
    except ValueError as e:
        return_error("Failed to parse Service Account Private Key in json format, please make sure you entered it correctly")
        raise e


def message_to_incident(message):
    """
    Create incident from a message
    """
    published_time_dt = dateparser.parse(message.get("publishTime"))
    incident = {
        "name": f'Google PubSub Message {message.get("messageId")}',
        "rawJSON": json.dumps(message),
        "occurred": convert_datetime_to_iso_str(published_time_dt),
    }
    return incident


def get_messages_ids_and_max_publish_time(msgs):
    """
    Get message IDs and max publish time from given pulled messages
    """
    msg_ids = set()
    max_publish_time = None
    for msg in msgs:
        msg_ids.add(msg.get("messageId"))
        publish_time = msg.get("publishTime")
        if publish_time:
            publish_time = dateparser.parse(msg.get("publishTime"))
        if not max_publish_time:
            max_publish_time = publish_time
        else:
            max_publish_time = max(max_publish_time, publish_time)
    if max_publish_time:
        max_publish_time = convert_datetime_to_iso_str(max_publish_time)
    return msg_ids, max_publish_time


def convert_datetime_to_iso_str(publish_time):
    """
    Converts datetime to str in "%Y-%m-%dT%H:%M:%S.%fZ" format
    :param publish_time: Datetime
    :return: date str in "%Y-%m-%dT%H:%M:%S.%fZ" format
    """
    try:
        return publish_time.strftime(ISO_DATE_FORMAT)
    except ValueError:
        return publish_time.strftime("%Y-%m-%dT%H:%M:%SZ")


def attribute_pairs_to_dict(attrs_str: str | None, delim_char: str = ","):
    """
    Transforms a string of multiple inputs to a dictionary list

    :param attrs_str: attributes separated by key=val pairs sepearated by ','
    :param delim_char: delimiter character between atrribute pairs
    :return:
    """
    if not attrs_str:
        return attrs_str
    attrs = {}
    regex = re.compile(r"(.*)=(.*)")
    for f in attrs_str.split(delim_char):
        match = regex.match(f)
        if match is None:
            raise ValueError(f"Could not parse field: {f}")

        attrs.update({match.group(1): match.group(2)})

    return attrs


""" COMMAND FUNCTIONS """


def test_module(client: PubSubClient, is_fetch: bool):
    """
    Returning 'ok' indicates that the integration works like it is supposed to:
        1. Connection to the service is successful.
        2. Fetch incidents is configured properly
    :param client: GoogleClient
    :return: 'ok' if test passed, anything else will fail the test.
    """
    client.list_topic(GoogleNameParser.get_project_name(client.default_project), page_size=1)
    if is_fetch:
        client.pull_messages(
            GoogleNameParser.get_subscription_project_name(client.default_project, client.default_subscription),
            max_messages=1,
        )
    return "ok"


def topics_list_command(
    client: PubSubClient,
    project_id: str,
    page_size: str = None,
    page_token: str = None,
) -> tuple[str, dict, dict]:
    """
    Get topics list by project_id
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: project name
    :param page_size: page size
    :param page_token: page token, as returned from the api
    :return: list of topics
    """
    full_project_name = GoogleNameParser.get_project_name(project_id)
    res = client.list_topic(full_project_name, page_size, page_token)

    topics = list(res.get("topics", []))
    next_page_token = res.get("nextPageToken")
    readable_output = tableToMarkdown(f"Topics for project {project_id}", topics, ["name"])
    outputs = {"GoogleCloudPubSubTopics(val && val.name === obj.name)": topics}
    if next_page_token:
        outputs["GoogleCloudPubSub.Topics.nextPageToken"] = next_page_token
        readable_output += f"**Next Page Token: {next_page_token}**"
    return readable_output, outputs, res


def publish_message_command(
    client: PubSubClient,
    topic_id: str,
    project_id: str,
    data: str = None,
    attributes: str = None,
    delim_char_attributes: str = ",",
) -> tuple[str, dict, dict]:
    """
    Publishes message in the topic
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param project_id: project name
    :param topic_id: topic name without project name prefix
    :param attributes: message attributes separated by key=val pairs sepearated by ','
    :param data: message data str
    :param client: GoogleClient
    :param delim_char_attributes: delimiter character between atrribute pairs
    :return: list of topics
    """
    body = get_publish_body(attributes, data, delim_char_attributes)
    published_messages = client.publish_message(project_id, topic_id, body)

    output = []
    for msg_id in published_messages.get("messageIds"):
        output.append(
            {
                "topic": topic_id,
                "messageId": msg_id,
                "data": data,
                "attributes": body.get("attributes"),
            }
        )

    ec = {"GoogleCloudPubSubPublishedMessages(val.messageId === obj.messageId)": output}
    return (
        tableToMarkdown(
            "Google Cloud PubSub has published the message successfully",
            output,
            removeNull=True,
            headerTransform=pascalToSpace,
        ),
        ec,
        published_messages,
    )


def get_publish_body(message_attributes, message_data, delim_char):
    """
    Creates publish messages body from given arguments
    :param message_attributes: message attributes
    :param message_data: message data
    :param delim_char: delimiter character between atrribute pairs
    :return: publish message body
    """
    message = {}
    if message_data:
        # convert to base64 string
        message["data"] = base64.b64encode(message_data.encode("utf8")).decode("utf8")
    if message_attributes:
        message["attributes"] = attribute_pairs_to_dict(message_attributes, delim_char)
    body = {"messages": [message]}
    return body


def pull_messages_command(
    client: PubSubClient,
    subscription_id: str,
    project_id: str,
    max_messages: str = None,
    ack: str = None,
) -> tuple[str, dict, list]:
    """
    Pulls messages from the subscription
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: project name
    :param subscription_id: Subscription name to pull messages from
    :param max_messages: The maximum number of messages to return for this request. Must be a positive integer
    :param ack: Acknowledge the messages pulled if set to true.
    :return: list of messages
    """
    full_subscription_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    raw_msgs = client.pull_messages(full_subscription_name, max_messages)
    if "receivedMessages" in raw_msgs:
        acknowledges, msgs = extract_acks_and_msgs(raw_msgs)
        ec = {"GoogleCloudPubSubPulledMessages(val && val.messageId === obj.messageId)": msgs}
        if ack == "true":
            client.ack_messages(full_subscription_name, acknowledges)
        hr = tableToMarkdown("Google Cloud PubSub Messages", msgs, removeNull=True)
        return hr, ec, raw_msgs
    else:
        return "No new messages found", {}, raw_msgs


def ack_messages_command(
    client: PubSubClient,
    ack_ids: str,
    subscription_id: str,
    project_id: str,
) -> tuple[str, dict, list]:
    """
    ACKs previously pulled messages using ack Ids
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param ack_ids: csv str with ack ids
    :param project_id: project name
    :param subscription_id: Subscription name to pull messages from
    :return: Success message
    """
    sub_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    ack_ids = argToList(ack_ids)
    raw_res = client.ack_messages(sub_name, ack_ids)
    title = f"Subscription {subscription_id} had the following ids acknowledged"
    readable_output = tableToMarkdown(title, ack_ids, headers=["ACK ID"])
    return readable_output, {}, raw_res


def extract_acks_and_msgs(raw_msgs, add_ack_to_msg=True):
    """
    Extracts acknowledges and message data from raw_msgs
    :param raw_msgs: Raw messages object
    :param add_ack_to_msg: Boolean flag - if true, will add ack to message under "ackId"
    :return:
    """
    msg_list = []
    acknowledges = []
    if isinstance(raw_msgs, dict):
        rcvd_msgs = raw_msgs.get("receivedMessages", [])
        for raw_msg in rcvd_msgs:
            msg = raw_msg.get("message", {})
            decoded_data = str(msg.get("data", ""))
            try:
                decoded_data = base64.b64decode(decoded_data).decode("utf-8")
            except Exception as e:
                # display message with b64 value
                demisto.debug(f"Unable to encode {decoded_data}:\n{e}")

            msg["data"] = decoded_data
            ack_id = raw_msg.get("ackId")
            if ack_id:
                acknowledges.append(ack_id)
                if add_ack_to_msg:
                    msg["ackId"] = ack_id
            msg_list.append(msg)
    return acknowledges, msg_list


def subscriptions_list_command(
    client: PubSubClient,
    project_id: str,
    page_size: str = None,
    page_token: str = None,
    topic_id: str = None,
) -> tuple[str, dict, dict]:
    """
    Get subscription list by project_id or by topic_id
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: project name
    :param page_size: page size
    :param page_token: page token, as returned from the api
    :param topic_id: topic name
    :return: list of subscriptions
    """
    title = "Subscriptions"
    if topic_id:
        full_topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
        raw_response = client.list_topic_subs(full_topic_name, page_size, page_token)
        subs = [{"name": sub} for sub in raw_response.get("subscriptions", [])]
        next_page_token = raw_response.get("nextPageToken")
        title += f" for topic {topic_id} in project {project_id}"
        readable_output = tableToMarkdown(title, subs, headers=["name"], headerTransform=pascalToSpace)
    else:
        full_project_name = GoogleNameParser.get_project_name(project_id)
        raw_response = client.list_project_subs(full_project_name, page_size, page_token)
        subs = raw_response.get("subscriptions", "")
        next_page_token = raw_response.get("nextPageToken")
        title += f" in project {project_id}"
        for sub in subs:
            sub["deliveryType"] = "Push" if sub.get("pushConfig") else "Pull"
        readable_output = tableToMarkdown(
            title,
            subs,
            headers=["name", "topic", "ackDeadlineSeconds", "labels"],
            headerTransform=pascalToSpace,
        )
    outputs = {"GoogleCloudPubSubSubscriptions(val && val.name === obj.name)": subs}
    if next_page_token:
        outputs["GoogleCloudPubSubSubscriptions.nextPageToken"] = next_page_token
        readable_output += f"**Next Page Token: {next_page_token}**"

    return readable_output, outputs, raw_response


def get_subscription_command(client: PubSubClient, subscription_id: str, project_id: str) -> tuple[str, dict, dict]:
    """
    Get subscription list by project_id or by topic_id
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param subscription_id:
    :param client: GoogleClient
    :param project_id: project name
    :return: subscription
    """
    full_sub_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    sub = client.get_sub(full_sub_name)
    sub["deliveryType"] = "Push" if sub.get("pushConfig") else "Pull"
    title = f"Subscription {subscription_id}"
    readable_output = tableToMarkdown(title, sub, headerTransform=pascalToSpace)
    outputs = {"GoogleCloudPubSubSubscriptions(val && val.name === obj.name)": sub}
    return readable_output, outputs, sub


def create_subscription_command(
    client: PubSubClient,
    subscription_id: str,
    topic_id: str,
    project_id: str,
    push_endpoint: str = "",
    push_attributes: str = "",
    ack_deadline_seconds: str = "",
    retain_acked_messages: str = "",
    message_retention_duration: str = "",
    labels: str = "",
    expiration_ttl: str = "",
) -> tuple[str, dict, dict]:
    """
    Creates a subscription
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: Name of the project from which the subscription is receiving messages.
    :param subscription_id: Name of the created subscription.
    :param topic_id: Name of the topic from which the subscription is receiving messages.
    :param push_endpoint: A URL locating the endpoint to which messages should be pushed.
    :param push_attributes: Input format: "key=val" pairs sepearated by ",".
    :param ack_deadline_seconds: The amount of time Pub/Sub waits for the subscriber to ack.
    :param retain_acked_messages: if 'true' then retain acknowledged messages
    :param message_retention_duration: How long to retain unacknowledged messages
    :param labels: Input format: "key=val" pairs sepearated by ",".
    :param expiration_ttl: The "time-to-live" duration for the subscription.
    :return: Created subscription
    """
    full_sub_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    full_topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
    labels = attribute_pairs_to_dict(labels)
    push_attributes = attribute_pairs_to_dict(push_attributes)
    raw_sub = client.create_subscription(
        full_sub_name,
        full_topic_name,
        push_endpoint,
        push_attributes,
        ack_deadline_seconds,
        retain_acked_messages,
        message_retention_duration,
        labels,
        expiration_ttl,
    )
    sub = dict(raw_sub)
    title = f"Subscription {subscription_id} was created successfully"
    readable_output = tableToMarkdown(title, sub)
    sub["projectName"] = project_id
    sub["subscriptionName"] = subscription_id
    sub["deliveryType"] = "Push" if sub.get("pushConfig") else "Pull"
    outputs = {"GoogleCloudPubSubSubscriptions": sub}
    return readable_output, outputs, raw_sub


def update_subscription_command(
    client: PubSubClient,
    subscription_id: str,
    topic_id: str,
    update_mask: str,
    project_id: str,
    push_endpoint: str = "",
    push_attributes: str = "",
    ack_deadline_seconds: str = "",
    retain_acked_messages: str = "",
    message_retention_duration: str = "",
    labels: str = "",
    expiration_ttl: str = "",
) -> tuple[str, dict, dict]:
    """
    Creates a subscription
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: Name of the project from which the subscription is receiving messages.
    :param subscription_id: Name of the created subscription.
    :param topic_id: Name of the topic from which the subscription is receiving messages.
    :param update_mask: Indicates which fields in the provided subscription to update.
    :param push_endpoint: A URL locating the endpoint to which messages should be pushed.
    :param push_attributes: Input format: "key=val" pairs sepearated by ",".
    :param ack_deadline_seconds: The amount of time Pub/Sub waits for the subscriber to ack.
    :param retain_acked_messages: if 'true' then retain acknowledged messages
    :param message_retention_duration: How long to retain unacknowledged messages
    :param labels: Input format: "key=val" pairs sepearated by ",".
    :param expiration_ttl: The "time-to-live" duration for the subscription.
    :return: Created subscription
    """
    full_sub_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    full_topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
    labels = attribute_pairs_to_dict(labels)
    push_attributes = attribute_pairs_to_dict(push_attributes)
    raw_sub = client.update_subscription(
        full_sub_name,
        full_topic_name,
        update_mask,
        push_endpoint,
        push_attributes,
        ack_deadline_seconds,
        retain_acked_messages,
        message_retention_duration,
        labels,
        expiration_ttl,
    )
    sub = dict(raw_sub)
    title = f"Subscription {subscription_id} was updated successfully"
    readable_output = tableToMarkdown(title, sub)
    sub["projectName"] = project_id
    sub["subscriptionName"] = subscription_id
    sub["deliveryType"] = "Push" if sub.get("pushConfig") else "Pull"
    outputs = {"GoogleCloudPubSubSubscriptions(val && val.name === obj.name)": sub}
    return readable_output, outputs, raw_sub


def create_topic_command(
    client: PubSubClient,
    topic_id: str,
    project_id: str,
    allowed_persistence_regions: str = "",
    kms_key_name: str = None,
    labels: str = None,
) -> tuple[str, dict, dict]:
    """
    Creates a topic
    :param client: PubSub client instance
    :param project_id: project ID
    :param topic_id: topic ID
    :param labels: "key=val" pairs sepearated by ",".'
    :param allowed_persistence_regions: an str representing a list of IDs of GCP regions
    :param kms_key_name: The full name of the Cloud KMS CryptoKey to be used to restrict access on this topic.
    :return: Created topic
    """
    topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
    allowed_persistence_regions = argToList(allowed_persistence_regions)
    labels = attribute_pairs_to_dict(labels)
    raw_topic = client.create_topic(topic_name, labels, allowed_persistence_regions, kms_key_name)
    title = f"Topic **{topic_id}** was created successfully"
    readable_output = tableToMarkdown(title, raw_topic, headerTransform=pascalToSpace)
    outputs = {"GoogleCloudPubSubTopics": raw_topic}
    return readable_output, outputs, raw_topic


def delete_topic_command(client: PubSubClient, project_id: str, topic_id: str) -> tuple[str, dict, dict]:
    """
    Delete a topic
    :param client: PubSub client instance
    :param project_id: project ID
    :param topic_id: topic ID
    :return: Command success/error message
    """
    topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
    raw_topic = client.delete_topic(topic_name)
    readable_output = f"Topic **{topic_id}** was deleted successfully"
    return readable_output, {}, raw_topic


def update_topic_command(
    client: PubSubClient,
    project_id: str,
    topic_id: str,
    update_mask: str,
    allowed_persistence_regions: str = "",
    kms_key_name: str = None,
    labels: str = None,
) -> tuple[str, dict, dict]:
    """
    Creates a topic
    :param client: PubSub client instance
    :param project_id: project ID
    :param topic_id: topic ID
    :param labels: "key=val" pairs sepearated by ",".'
    :param allowed_persistence_regions: an str representing a list of IDs of GCP regions
    :param kms_key_name: The full name of the Cloud KMS CryptoKey to be used to restrict access on this topic.
    :param update_mask: Indicates which fields in the provided topic to update.
    :return: Created topic
    """
    topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
    allowed_persistence_regions = argToList(allowed_persistence_regions)
    labels = attribute_pairs_to_dict(labels)
    raw_topic = client.update_topic(topic_name, labels, allowed_persistence_regions, kms_key_name, update_mask)
    title = f"Topic {topic_id} was updated successfully"
    readable_output = tableToMarkdown(title, raw_topic, headerTransform=pascalToSpace)
    outputs = {"GoogleCloudPubSubTopics(val && val.name === obj.name)": raw_topic}
    return readable_output, outputs, raw_topic


def seek_message_command(
    client: PubSubClient,
    project_id: str,
    subscription_id: str,
    time_string: str = None,
    snapshot: str = None,
) -> tuple[str, dict, dict]:
    """
    Get topics list by project_id
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: ID of the subscription, without project/topic prefix.
    :param subscription_id: ID of the project from which the subscription is receiving messages.
    :param time_string: A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds,
    :param snapshot: The snapshot to seek to.
    :return: list of topics
    """
    if not time_string and not snapshot:
        return_error("Please provide either a time_string or a snapshot")
    sub_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    raw_res = client.subscription_seek_message(sub_name, time_string, snapshot)
    readable_output = (
        "Message seek was successful for **" + (f"time: {time_string}" if time_string else f"snapshot:{snapshot}") + "**"
    )
    return readable_output, {}, raw_res


def snapshot_list_command(
    client: PubSubClient,
    project_id: str,
    topic_id: str = None,
    page_size: str = None,
    page_token: str = None,
) -> tuple[str, dict, dict]:
    """
    Get snapshots list by project_id or topic_id
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: project id
    :param topic_id:
    :param page_size: page size
    :param page_token: page token, as returned from the api
    :return: list of snapshots
    """
    if topic_id:
        topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
        res = client.get_topic_snapshots_list(topic_name, page_size, page_token)
        title = f"Snapshots for topic {topic_id}"
    else:
        project_name = GoogleNameParser.get_project_name(project_id)
        res = client.get_project_snapshots_list(project_name, page_size, page_token)
        title = f"Snapshots for project {project_id}"
    snapshots = list(res.get("snapshots", []))
    next_page_token = res.get("nextPageToken")
    readable_output = tableToMarkdown(title, snapshots, ["name"])
    outputs = {"GoogleCloudPubSubSnapshots(val && val.name === obj.name)": snapshots}
    if next_page_token:
        outputs["GoogleCloudPubSub.Snapshots.nextPageToken"] = next_page_token
        readable_output += f"**Next Page Token: {next_page_token}**"
    return readable_output, outputs, res


def snapshot_create_command(
    client: PubSubClient,
    project_id: str,
    subscription_id: str,
    snapshot_id: str,
    labels: str = None,
) -> tuple[str, dict, dict]:
    """
    Create a snapshot
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: project id
    :param subscription_id: The subscription whose backlog the snapshot retains.
    :param snapshot_id: The id of the snapshot.
    :param labels: Input format: "key=val" pairs sepearated by ",".
    :return: list of topics
    """
    subscription_name = GoogleNameParser.get_subscription_project_name(project_id, subscription_id)
    snapshot_name = GoogleNameParser.get_snapshot_project_name(project_id, snapshot_id)
    labels = attribute_pairs_to_dict(labels)
    raw_snapshot = client.create_snapshot(subscription_name, snapshot_name, labels)
    title = f"Snapshot **{snapshot_id}** was created successfully"
    readable_output = tableToMarkdown(title, raw_snapshot, headerTransform=pascalToSpace)
    outputs = {"GoogleCloudPubSubSnapshots": raw_snapshot}
    return readable_output, outputs, raw_snapshot


def snapshot_update_command(
    client: PubSubClient,
    project_id: str,
    topic_id: str,
    snapshot_id: str,
    update_mask: str,
    expire_time: str = None,
    labels: str = None,
) -> tuple[str, dict, dict]:
    """
    Updates a snapshot
    Requires one of the following OAuth scopes:

        https://www.googleapis.com/auth/pubsub
        https://www.googleapis.com/auth/cloud-platform

    :param client: GoogleClient
    :param project_id: ID of the project from which the subscription is receiving messages.
    :param topic_id: The ID of the topic from which this snapshot is retaining messages.
    :param snapshot_id: The id of the snapshot.
    :param update_mask: Indicates which fields in the provided snapshot to update.
    :param expire_time: The snapshot is guaranteed to exist up until this time
    :param labels: An object containing a list of "key": value pairs
    :return:
    """
    snapshot_name = GoogleNameParser.get_snapshot_project_name(project_id, snapshot_id)
    topic_name = GoogleNameParser.get_topic_name(project_id, topic_id)
    labels = attribute_pairs_to_dict(labels)
    raw_snapshot = client.update_snapshot(snapshot_name, topic_name, update_mask, expire_time, labels)
    title = f"Snapshot **{snapshot_id}** was updated successfully"
    readable_output = tableToMarkdown(title, raw_snapshot, headerTransform=pascalToSpace)
    outputs = {"GoogleCloudPubSubSnapshots(val && val.name === obj.name)": raw_snapshot}
    return readable_output, outputs, raw_snapshot


def snapshot_delete_command(client: PubSubClient, project_id: str, snapshot_id: str) -> tuple[str, dict, dict]:
    """
    Delete a topic
    :param client: PubSub client instance
    :param project_id: The ID of the project from which the subscription is receiving messages.
    :param snapshot_id: The id of the snapshot.
    :return: Command success/error message
    """
    snapshot_name = GoogleNameParser.get_snapshot_project_name(project_id, snapshot_id)
    raw_res = client.delete_snapshot(snapshot_name)
    readable_output = f"Snapshot **{snapshot_id}** was deleted successfully"
    return readable_output, {}, raw_res


def fetch_incidents(client: PubSubClient, last_run: dict, first_fetch_time: str, ack_incidents: bool):
    """
    This function will execute each interval (default is 1 minute).
    :param client: GoogleClient initialized with default_project, default_subscription and default_max_msgs
    :param last_run: last run dict containing last run data
    :param first_fetch_time: how long ago should the subscription seek in first fetch
    :param ack_incidents: Boolean flag - when set to True will ack back the fetched messages
    :return: incidents: Incidents that will be created in Demisto
    """
    sub_name = GoogleNameParser.get_subscription_project_name(client.default_project, client.default_subscription)

    # Setup subscription for fetch
    last_run_fetched_ids, last_run_time = setup_subscription_last_run(client, first_fetch_time, last_run, sub_name, ack_incidents)

    # Pull unique messages if available
    msgs, msg_ids, acknowledges, max_publish_time = try_pull_unique_messages(
        client, sub_name, last_run_fetched_ids, last_run_time, ack_incidents, retry_times=1
    )

    # Handle fetch results
    return handle_fetch_results(
        client,
        sub_name,
        last_run,
        acknowledges,
        last_run_time,
        max_publish_time,
        msg_ids,
        msgs,
        ack_incidents,
    )


def setup_subscription_last_run(client, first_fetch_time, last_run, sub_name, ack_incidents):
    """
    Setups the subscription last run data, and seeks the subscription to a previous time if relevant
    :param client: PubSub client
    :param first_fetch_time: First fetch time provided by the user
    :param last_run: Last run dict
    :param sub_name: Name of the subscription
    :param ack_incidents: ACK flag - if true, will not use seek except for first time fetch
    :return:
    """
    last_run_fetched_ids = set()
    # Handle first time fetch
    if not last_run or LAST_RUN_TIME_KEY not in last_run:
        last_run_time, _ = parse_date_range(first_fetch_time, ISO_DATE_FORMAT)
        # Seek previous message state
        client.subscription_seek_message(sub_name, last_run_time)
    else:
        last_run_time = last_run.get(LAST_RUN_TIME_KEY)
        last_run_fetched_val = last_run.get(LAST_RUN_FETCHED_KEY)
        if last_run_fetched_val:
            last_run_fetched_ids = set(last_run_fetched_val)
        if not ack_incidents:
            # Seek previous message state
            client.subscription_seek_message(sub_name, last_run_time)
    return last_run_fetched_ids, last_run_time


def try_pull_unique_messages(client, sub_name, previous_msg_ids, last_run_time, ack_incidents, retry_times=0):
    """
    Tries to pull unique messages for the subscription
    :param client: PubSub client
    :param sub_name: Subscription name
    :param previous_msg_ids: Previous message ids set
    :param last_run_time: previous run time
    :param ack_incidents: should ack incidents
    :param retry_times: How many times to retry pulling
    :return:
        1. Unique list of messages
        2. Unique  set of message ids
        3. Messages acks
        4. max_publish_time
    """
    res_msgs = None
    res_msg_ids = None
    res_acks = None
    res_max_publish_time = None
    raw_msgs = client.pull_messages(sub_name, client.default_max_msgs)
    if "receivedMessages" in raw_msgs:
        res_acks, msgs = extract_acks_and_msgs(raw_msgs)
        # continue only if messages were extracted successfully
        if msgs:
            msg_ids, max_publish_time = get_messages_ids_and_max_publish_time(msgs)
            new_msg_ids = msg_ids.difference(previous_msg_ids)
            # all messages are unique - return as is
            if len(new_msg_ids) == len(msg_ids):
                return msgs, msg_ids, res_acks, max_publish_time
            # no new messages - retry -1
            elif len(new_msg_ids) == 0 and retry_times > 0:
                demisto.debug(f"GCP_PUBSUB_MSG Duplicates with max_publish_time: {max_publish_time}")
                return try_pull_unique_messages(client, sub_name, previous_msg_ids, retry_times - 1, ack_incidents)
            # clean non-unique ids from raw_msgs (in case non-ack)
            elif not ack_incidents:
                filtered_raw_msgs = filter_non_unique_messages(raw_msgs, previous_msg_ids, last_run_time)
                res_acks, res_msgs = extract_acks_and_msgs(filtered_raw_msgs)
                (
                    res_msg_ids,
                    res_max_publish_time,
                ) = get_messages_ids_and_max_publish_time(res_msgs)
    return res_msgs, res_msg_ids, res_acks, res_max_publish_time


def is_unique_msg(msg, previous_msg_ids, previous_run_time):
    """
    Determines if message is unique given previous message ids, and that it's greater than previous run time
    :param msg: raw Message object
    :param previous_msg_ids: set of previously fetched message ids
    :param previous_run_time: previous run time string
    :return: True if message is unique
    """
    message_dict = msg.get("message", {})
    if message_dict:
        msg_id = message_dict.get("messageId")
        msg_pub_time = message_dict.get("publishTime", "")
        unique_msg = msg_id not in previous_msg_ids and msg_pub_time > previous_run_time
        if not unique_msg:
            demisto.debug(f"Dropping {msg_id=}. {msg_pub_time=}")
        return msg_id not in previous_msg_ids and msg_pub_time > previous_run_time
    return False


def filter_non_unique_messages(raw_msgs, previous_msg_ids, previous_run_time):
    """
    Filters messages that appear in previous_msg_ids or are older than the previous_run_time
    :param raw_msgs: Raw message object
    :param previous_msg_ids:
    :param previous_run_time:
    :return:
    """
    raw_msgs = raw_msgs.get("receivedMessages", [])
    # filter messages using `previous_msg_ids` and `previous_run_time`
    filtered_raw_msgs = list(
        filter(
            lambda msg: is_unique_msg(msg, previous_msg_ids, previous_run_time),
            raw_msgs,
        )
    )
    return {"receivedMessages": filtered_raw_msgs}


def handle_fetch_results(
    client,
    sub_name,
    last_run,
    acknowledges,
    last_run_time,
    max_publish_time,
    pulled_msg_ids,
    pulled_msgs,
    ack_incidents,
):
    """
    Handle the fetch results
    :param client: PubSub Client
    :param sub_name: Subscription name
    :param last_run: last run dict
    :param acknowledges: acknowledges to make given ack_incidents is True
    :param last_run_time: last run time
    :param max_publish_time: max publish time of pulled messages
    :param pulled_msg_ids: pulled message ids
    :param pulled_msgs: pulled messages
    :param ack_incidents: ack incidents flag
    :return: incidents and last run
    """
    incidents = []
    if pulled_msg_ids and max_publish_time:
        # Create incidents
        for msg in pulled_msgs:
            incident = message_to_incident(msg)
            incidents.append(incident)
        # ACK messages if relevant
        if ack_incidents:
            demisto.debug(f"ACK: {acknowledges}")
            client.ack_messages(sub_name, acknowledges)
        # Recreate last run to return with new values
        last_run = {
            LAST_RUN_TIME_KEY: max_publish_time,
            LAST_RUN_FETCHED_KEY: list(pulled_msg_ids),
        }
    # We didn't manage to pull any unique messages, so we're trying to increment micro seconds - not relevant for ack
    elif not ack_incidents:
        last_run_time_dt = dateparser.parse(max_publish_time if max_publish_time else last_run_time)
        assert last_run_time_dt is not None
        last_run_time = convert_datetime_to_iso_str(last_run_time_dt + timedelta(microseconds=1))
        # Update last run time
        last_run[LAST_RUN_TIME_KEY] = last_run_time
    return incidents, last_run


def main():
    params = demisto.params()
    client = init_google_client(**params)
    command = demisto.command()
    LOG(f"Command being called is {command}")
    try:
        commands = {
            "gcp-pubsub-topic-publish-message": publish_message_command,
            "gcp-pubsub-topic-messages-pull": pull_messages_command,
            "gcp-pubsub-topic-ack-messages": ack_messages_command,
            "gcp-pubsub-topic-subscriptions-list": subscriptions_list_command,
            "gcp-pubsub-topic-subscription-get-by-name": get_subscription_command,
            "gcp-pubsub-topic-subscription-create": create_subscription_command,
            "gcp-pubsub-topic-subscription-update": update_subscription_command,
            "gcp-pubsub-topics-list": topics_list_command,
            "gcp-pubsub-topic-create": create_topic_command,
            "gcp-pubsub-topic-delete": delete_topic_command,
            "gcp-pubsub-topic-update": update_topic_command,
            "gcp-pubsub-topic-messages-seek": seek_message_command,
            "gcp-pubsub-topic-snapshots-list": snapshot_list_command,
            "gcp-pubsub-topic-snapshot-create": snapshot_create_command,
            "gcp-pubsub-topic-snapshot-update": snapshot_update_command,
            "gcp-pubsub-topic-snapshot-delete": snapshot_delete_command,
        }
        if command == "test-module":
            demisto.results(test_module(client, params.get("isFetch")))
        elif command == "fetch-incidents":
            ack_incidents = params.get("ack_incidents")
            first_fetch_time = params.get("first_fetch_time").rstrip()
            last_run = demisto.getLastRun()
            incidents, last_run = fetch_incidents(client, last_run, first_fetch_time, ack_incidents)
            demisto.incidents(incidents)
            demisto.setLastRun(last_run)
        else:
            args = demisto.args()
            # project_id is expected to be in all commands. If not provided defaults on client.default_project
            if "project_id" not in args:
                args["project_id"] = client.default_project
            return_outputs(*commands[command](client, **args))  # type: ignore[operator]

    # Log exceptions
    except Exception as e:
        return_error(f"Failed to execute {demisto.command()} command. Error: {e!s} , traceback: {traceback.format_exc()}")


if __name__ in ("__main__", "__builtin__", "builtins"):
    main()