GooglePubSub
Google Cloud Pub/Sub is a fully-managed real-time messaging service that enables you to send and receive messages between independent applications.
Messaging and Conferencing · Google Cloud Pub / Sub
Details
| ID | GooglePubSub |
|---|---|
| Provider | |
| Category | Messaging and Conferencing |
| From Version | 5.0.0 |
| Docker Image | demisto/googleapi-python3:1.0.0.115338 |
| Supported Modules | Agentix XSIAM |
README
Google Cloud Pub / Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications.
This integration was integrated and tested with Google Cloud Pub/Sub
Required Permissions
To use this integration you must have a Service Account with one of the following roles:
- Project-Owner
- Project-Editor
- Pub/Sub Admin
- Pub/Sub Editor
Known Limitations
When clicking on Reset the “last run” timestamp, messages that were recently pulled (including pulls via classification mapper) might take a few minutes before they can be fetched again.
Because the fetch ignores older messages once newer ones were fetched, it’s recommended to wait a few minutes following a reset before trying to fetch again, to prevent older messages from being dropped.
Configure GooglePubSub in Cortex
| Parameter | Description | Required |
|---|---|---|
| service_account_json | Service Account Private Key File Contents (JSON) | True |
| insecure | Trust any certificate (not secure) | False |
| proxy | Use system proxy settings | False |
| isFetch | Fetch incidents | False |
| incidentType | Incident type | False |
| default_subscription | Fetch Incidents Subscription ID | False |
| default_project | Fetch Incidents Project ID | False |
| default_max_msgs | Max Incidents Per Fetch | False |
Commands
You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.
gcp-pubsub-topics-list
Get a list of the project’s topics.
Base Command
gcp-pubsub-topics-list
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project to look in. | Optional |
| page_size | Max amount of entries to get. | Optional |
| page_token | Next page token as returned from "gcp-pubsub-topics-list" command | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubTopics.name | String | Name of the topic |
| GoogleCloudPubSub.Topics.nextPageToken | String | If not empty, indicates that there may be more topics that match the request. |
Command Example
!gcp-pubsub-topics-list project_id=dmst-integrations
Context Example
{
"GoogleCloudPubSubTopics": {
"name": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Topics for project dmst-integrations
| name |
|---|
| projects/dmst-integrations/topics/dmst-topic |
gcp-pubsub-topic-publish-message
Publish a message in a topic.
Base Command
gcp-pubsub-topic-publish-message
Input
| Argument Name | Description | Required |
|---|---|---|
| topic_id | ID of the topic e.g. "projects/{project_id}/topics/topic_id". | Required |
| data | The message data field. If this field is empty, the message must contain at least one attribute. | Optional |
| attributes | Attributes for this message. If this field is empty, the message must contain non-empty data. Input format: "key=val" pairs sepearated by ",". | Optional |
| project_id | Project ID. | Optional |
| delim_char_attributes | Set delimiter of attributes split. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubPublishedMessages.messageId | String | ID of the published message |
| GoogleCloudPubSubPublishedMessages.topic | String | Topic of the published message |
| GoogleCloudPubSubPublishedMessages.data | String | Text data of the published message. |
| GoogleCloudPubSubPublishedMessages.attributes | Unknown | The message attributes. |
Command Example
!gcp-pubsub-topic-publish-message data="42 is the answer" project_id=dmst-integrations topic_id=dmst-topic delim_char_attributes=","
Context Example
{
"GoogleCloudPubSubPublishedMessages": {
"attributes": null,
"data": "42 is the answer",
"messageId": "874663628353499",
"topic": "dmst-topic",
"delim_char_attributes": ","
}
}
Human Readable Output
Google Cloud PubSub has published the message successfully
| Data | Message Id | Topic |
|---|---|---|
| 42 is the answer | 874663628353499 | dmst-topic |
gcp-pubsub-topic-subscription-get-by-name
Get subscription details by subscription ID.
Base Command
gcp-pubsub-topic-subscription-get-by-name
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project from which the subscription is receiving messages. | Optional |
| subscription_id | ID of the subscription, without project/topic prefix. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSubscriptions.ackDeadlineSeconds | Number | The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. |
| GoogleCloudPubSubSubscriptions.expirationPolicy.ttl | String | The "time-to-live" duration for the subscription. |
| GoogleCloudPubSubSubscriptions.messageRetentionDuration | String | How long to retain unacknowledged messages in the subscription's backlog |
| GoogleCloudPubSubSubscriptions.name | String | Name of the subscription |
| GoogleCloudPubSubSubscriptions.topic | String | Name of the topic from which the subscription is receiving messages |
Command Example
!gcp-pubsub-topic-subscription-get-by-name subscription_id=test_sub_2 project_id=dmst-integrations
Context Example
{
"GoogleCloudPubSubSubscriptions": {
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "9999999999s"
},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/test_sub_2",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Subscription test_sub_2
| ackDeadlineSeconds | expirationPolicy | messageRetentionDuration | name | pushConfig | topic |
|---|---|---|---|---|---|
| 10 | ttl: 9999999999s | 604800s | projects/dmst-integrations/subscriptions/test_sub_2 | projects/dmst-integrations/topics/dmst-topic |
gcp-pubsub-topic-subscriptions-list
Get a list of subscriptions by project ID or topic ID.
Base Command
gcp-pubsub-topic-subscriptions-list
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project from which the subscription is receiving messages. | Optional |
| topic_id | ID of the topic from which the subscription is receiving messages. | Optional |
| page_size | Max number of results | Optional |
| page_token | Next page token as returned from the API. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSubscriptions.ackDeadlineSeconds | Number | The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. |
| GoogleCloudPubSubSubscriptions.expirationPolicy.ttl | String | The "time-to-live" duration for the subscription |
| GoogleCloudPubSubSubscriptions.messageRetentionDuration | String | How long to retain unacknowledged messages in the subscription's backlog |
| GoogleCloudPubSubSubscriptions.name | String | Name of the subscription |
| GoogleCloudPubSubSubscriptions.topic | String | Name of the topic from which the subscription is receiving messages. |
| GoogleCloudPubSubSubscriptions.pushConfig.pushEndpoint | String | A URL locating the endpoint to which messages should be pushed. |
| c | Unknown | If not empty, indicates that there may be more snapshot that match the request. |
Command Example
!gcp-pubsub-topic-subscriptions-list project_id=dmst-integrations
Context Example
{
"GoogleCloudPubSubSubscriptions": [
{
"ackDeadlineSeconds": 11,
"expirationPolicy": {
"ttl": "2678400s"
},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/dean-sub1",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "2678400s"
},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/dean-sub2",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "22678400s"
},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "2678400s"
},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/test_sub",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "9999999999s"
},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/test_sub_2",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587031883059",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587032827289",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587039285961",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587038878685",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587039587203",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587040075117",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587042146495",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"test": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/test_sub_1587043084505",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"doc": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/doc_sub",
"pushConfig": {},
"topic": "projects/dmst-integrations/topics/dmst-topic"
},
{
"ackDeadlineSeconds": 600,
"expirationPolicy": {},
"messageRetentionDuration": "604800s",
"name": "projects/dmst-integrations/subscriptions/gcf-function-1-us-central1-dmst-topic",
"pushConfig": {
"attributes": {
"x-goog-version": "v1"
},
"pushEndpoint": "https://d4d1290519676f29baf13a7bf18a25bf-dot-j40fd5d18d8c290e1p-tp.appspot.com/_ah/push-handlers/pubsub/projects/dmst-integrations/topics/dmst-topic?pubsub_trigger=true"
},
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
]
}
Human Readable Output
Subscriptions in project dmst-integrations
| Name | Topic | Ack Deadline Seconds | Labels |
|---|---|---|---|
| projects/dmst-integrations/subscriptions/dean-sub1 | projects/dmst-integrations/topics/dmst-topic | 11 | |
| projects/dmst-integrations/subscriptions/dean-sub2 | projects/dmst-integrations/topics/dmst-topic | 10 | |
| projects/dmst-integrations/subscriptions/test_sub_1 | projects/dmst-integrations/topics/dmst-topic | 10 | |
| projects/dmst-integrations/subscriptions/test_sub | projects/dmst-integrations/topics/dmst-topic | 10 | |
| projects/dmst-integrations/subscriptions/test_sub_2 | projects/dmst-integrations/topics/dmst-topic | 10 | |
| projects/dmst-integrations/subscriptions/test_sub_1587031883059 | projects/dmst-integrations/topics/dmst-topic | 10 | |
| projects/dmst-integrations/subscriptions/test_sub_1587032827289 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/test_sub_1587039285961 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/test_sub_1587038878685 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/test_sub_1587039587203 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/test_sub_1587040075117 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/test_sub_1587042146495 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/test_sub_1587043084505 | projects/dmst-integrations/topics/dmst-topic | 10 | test: true |
| projects/dmst-integrations/subscriptions/doc_sub | projects/dmst-integrations/topics/dmst-topic | 10 | doc: true |
| projects/dmst-integrations/subscriptions/gcf-function-1-us-central1-dmst-topic | projects/dmst-integrations/topics/dmst-topic | 600 |
gcp-pubsub-topic-messages-pull
Pull messages that were published.
Base Command
gcp-pubsub-topic-messages-pull
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | Project ID to pull messages from. | Optional |
| subscription_id | Subscription ID to pull messages from. | Required |
| max_messages | The maximum number of messages to return for this request. Must be a positive integer. | Optional |
| ack | Acknowledge the messages pulled. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubPulledMessages.data | String | Text data of the pulled message. |
| GoogleCloudPubSubPulledMessages.messageId | String | ID of the message |
| GoogleCloudPubSubPulledMessages.publishTime | Date | The time the message was published |
| GoogleCloudPubSubPulledMessages.attributes | Unknown | The message attributes. |
Command Example
!gcp-pubsub-topic-messages-pull ack=true max_messages=1 project_id=dmst-integrations subscription_id=test_sub_2
Context Example
{
"GoogleCloudPubSubPulledMessages": {
"data": "42 is the answer",
"messageId": "874662740221427",
"publishTime": "2020-04-16T13:32:41.398Z"
}
}
Human Readable Output
Google Cloud PubSub Messages
| data | messageId | publishTime |
|---|---|---|
| 42 is the answer | 874662740221427 | 2020-04-16T13:32:41.398Z |
gcp-pubsub-topic-subscription-create
Create a pull or push subscription.
Base Command
gcp-pubsub-topic-subscription-create
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project from which the subscription is receiving messages. | Optional |
| subscription_id | ID of the created subscription. | Required |
| topic_id | ID of the topic from which the subscription is receiving messages. | Required |
| push_endpoint | A URL locating the endpoint to which messages should be pushed. | Optional |
| push_attributes | Endpoint configuration attributes that can be used to control the message delivery, such as "x-goog-version", which you can use to change the format of the pushed message. Input format: "key=val" pairs sepearated by ",". | Optional |
| ack_deadline_seconds | The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. | Optional |
| retain_acked_messages | Indicates whether to retain acknowledged messages. | Optional |
| message_retention_duration | How long to retain unacknowledged messages in the subscription's backlog. A duration of seconds e.g. "4.2s" | Optional |
| labels | Input format: "key=val" pairs sepearated by ",". | Optional |
| expiration_ttl | The "time-to-live" duration for the subscription. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSubscriptions.ackDeadlineSeconds | Number | The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. |
| GoogleCloudPubSubSubscriptions.expirationPolicy.ttl | String | The "time-to-live" duration for the subscription. |
| GoogleCloudPubSubSubscriptions.messageRetentionDuration | String | How long to retain unacknowledged messages in the subscription's backlog |
| GoogleCloudPubSubSubscriptions.name | String | Name of the subscription |
| GoogleCloudPubSubSubscriptions.topic | String | Name of the topic from which the subscription is receiving messages |
| GoogleCloudPubSubSubscriptions.projectName | String | Name of the project from which the subscription is receiving messages |
| GoogleCloudPubSubSubscriptions.subscriptionName | String | Name of the newly created subscription |
| GoogleCloudPubSubSubscriptions.labels | String | An object containing a list of "key": value pairs. |
Command Example
!gcp-pubsub-topic-subscription-create expiration_ttl=86400s project_id=dmst-integrations topic_id=dmst-topic subscription_id=doc_sub_1
Context Example
{
"GoogleCloudPubSubSubscriptions": {
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/doc_sub_1",
"projectName": "dmst-integrations",
"pushConfig": {},
"subscriptionName": "doc_sub_1",
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Subscription doc_sub_1 was created successfully
| ackDeadlineSeconds | expirationPolicy | messageRetentionDuration | name | pushConfig | topic |
|---|---|---|---|---|---|
| 10 | ttl: 86400s | 86400s | projects/dmst-integrations/subscriptions/doc_sub_1 | projects/dmst-integrations/topics/dmst-topic |
gcp-pubsub-topic-create
Create a topic.
Base Command
gcp-pubsub-topic-create
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project the topic will belong to. | Optional |
| topic_id | ID of the newly created topic. | Required |
| labels | 'Input format: "key=val" pairs sepearated by ",".' | Optional |
| allowed_persistence_regions | A comma separated list of IDs of GCP regions where messages that are published to the topic may be persisted in storage. e.g. "us-east4,asia-1". https://cloud.google.com/compute/docs/regions-zones#locations |
Optional |
| kms_key_name | The full name of the Cloud KMS CryptoKey to be used to restrict access to messages published on this topic. Full name format: projects//locations//keyRings//cryptoKeys/. |
Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubTopics.name | String | Name of the topic |
| GoogleCloudPubSubTopics.labels | Unknown | An object containing a list of "key": value pairs. |
| GoogleCloudPubSubTopics.messageStoragePolicy.allowedPersistenceRegions | Unknown | A list of IDs of GCP regions where messages that are published to the topic may be persisted in storage. |
| GoogleCloudPubSubTopics.kmsKeyName | String | The resource name of the Cloud KMS CryptoKey to be used to restrict access. |
Command Example
!gcp-pubsub-topic-create project_id=dmst-integrations topic_id=dmst-doc-topic
Context Example
{
"GoogleCloudPubSubTopics": {
"name": "projects/dmst-integrations/topics/dmst-doc-topic"
}
}
Human Readable Output
Topic dmst-doc-topic was created successfully
| Name |
|---|
| projects/dmst-integrations/topics/dmst-doc-topic |
gcp-pubsub-topic-delete
Delete a topic.
Base Command
gcp-pubsub-topic-delete
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project the topic will belong to. | Optional |
| topic_id | ID of the newly created topic. | Required |
Context Output
There is no context output for this command.
Command Example
!gcp-pubsub-topic-delete project_id=dmst-integrations topic_id=dmst-doc-topic
Context Example
{}
Human Readable Output
Topic dmst-doc-topic was deleted successfully
gcp-pubsub-topic-update
Updates a topic.
Base Command
gcp-pubsub-topic-update
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project the topic belongs to. | Optional |
| topic_id | ID of the topic. | Required |
| labels | 'Input format: "key=val" pairs sepearated by ",".' | Optional |
| allowed_persistence_regions | A comma separated list of IDs of GCP regions where messages that are published to the topic may be persisted in storage. e.g. "us-east4,asia-1". https://cloud.google.com/compute/docs/regions-zones#locations |
Optional |
| kms_key_name | The full name of the Cloud KMS CryptoKey to be used to restrict access to messages published on this topic. Full name format: projects//locations//keyRings//cryptoKeys/. |
Optional |
| update_mask | Indicates which fields in the provided topic to update. A comma-separated list of fields. Example: "labels,messageStoragePolicy". |
Required |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubTopics.name | String | Name of the topic |
| GoogleCloudPubSubTopics.labels | Unknown | An object containing a list of "key": value pairs. |
| GoogleCloudPubSubTopics.messageStoragePolicy.allowedPersistenceRegions | Unknown | A list of IDs of GCP regions where messages that are published to the topic may be persisted in storage. |
| GoogleCloudPubSubTopics.kmsKeyName | String | The resource name of the Cloud KMS CryptoKey to be used to restrict access. |
Command Example
!gcp-pubsub-topic-update project_id=dmst-integrations topic_id=dmst-doc-topic labels="doc=true" update_mask=labels
Context Example
{
"GoogleCloudPubSubTopics": {
"labels": {
"doc": "true"
},
"name": "projects/dmst-integrations/topics/dmst-doc-topic"
}
}
Human Readable Output
Topic dmst-doc-topic was updated successfully
| Labels | Name |
|---|---|
| doc: true | projects/dmst-integrations/topics/dmst-doc-topic |
gcp-pubsub-topic-subscription-update
Update a subscription.
Base Command
gcp-pubsub-topic-subscription-update
Input
| Argument Name | Description | Required |
|---|---|---|
| update_mask | Indicates which fields in the provided subscription to update. A comma-separated list of fully qualified names of fields. Example: "pushConfig.pushEndpoint,ackDeadlineSeconds". |
Required |
| project_id | ID of the project from which the subscription is receiving messages. | Optional |
| subscription_id | ID of the updated subscription. | Required |
| topic_id | ID of the topic from which the subscription is receiving messages. | Required |
| push_endpoint | A URL locating the endpoint to which messages should be pushed. | Optional |
| push_attributes | Endpoint configuration attributes that can be used to control the message delivery. Input format: "key=val" pairs sepearated by ",". | Optional |
| ack_deadline_seconds | The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. | Optional |
| retain_acked_messages | Indicates whether to retain acknowledged messages. | Optional |
| message_retention_duration | How long to retain unacknowledged messages in the subscription's backlog. A duration of seconds e.g. "4.2s" | Optional |
| labels | Input format: "key=val" pairs sepearated by ",". | Optional |
| expiration_ttl | The "time-to-live" duration for the subscription. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSubscriptions.ackDeadlineSeconds | Number | The amount of time Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. |
| GoogleCloudPubSubSubscriptions.expirationPolicy.ttl | String | The "time-to-live" duration for the subscription. |
| GoogleCloudPubSubSubscriptions.messageRetentionDuration | String | How long to retain unacknowledged messages in the subscription's backlog. |
| GoogleCloudPubSubSubscriptions.name | String | Name of the subscription. |
| GoogleCloudPubSubSubscriptions.topic | String | Name of the topic from which the subscription is receiving messages. |
| GoogleCloudPubSubSubscriptions.projectName | String | Name of the project from which the subscription is receiving messages. |
| GoogleCloudPubSubSubscriptions.subscriptionName | String | Name of the subscription. |
| GoogleCloudPubSubSubscriptions.labels | String | An object containing a list of "key": value pairs. |
Command Example
!gcp-pubsub-topic-subscription-update labels="doc=true" project_id=dmst-integrations subscription_id=doc_sub_1 topic_id=dmst-topic update_mask=labels
Context Example
{
"GoogleCloudPubSubSubscriptions": {
"ackDeadlineSeconds": 10,
"expirationPolicy": {
"ttl": "86400s"
},
"labels": {
"doc": "true"
},
"messageRetentionDuration": "86400s",
"name": "projects/dmst-integrations/subscriptions/doc_sub_1",
"projectName": "dmst-integrations",
"pushConfig": {
"attributes": {
"x-goog-version": "v1"
}
},
"subscriptionName": "doc_sub_1",
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Subscription doc_sub_1 was updated successfully
| ackDeadlineSeconds | expirationPolicy | labels | messageRetentionDuration | name | pushConfig | topic |
|---|---|---|---|---|---|---|
| 10 | ttl: 86400s | doc: true | 86400s | projects/dmst-integrations/subscriptions/doc_sub_1 | attributes: {“x-goog-version”: “v1”} | projects/dmst-integrations/topics/dmst-topic |
gcp-pubsub-topic-messages-seek
Seeks a subscription to a given point in time or to a given snapshot.
Base Command
gcp-pubsub-topic-messages-seek
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project from which the subscription is receiving messages. | Optional |
| subscription_id | ID of the subscription, without project/topic prefix. | Required |
| time_string | A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2014-10-02T15:01:23.045123456Z". | Optional |
| snapshot | The snapshot to seek to. | Optional |
Context Output
There is no context output for this command.
Command Example
!gcp-pubsub-topic-messages-seek time_string="2020-04-16T13:27:55.117Z" project_id=dmst-integrations topic_id=dmst-topic subscription_id=doc_sub_1
Context Example
{}
Human Readable Output
Message seek was successful for time: 2020-04-16T13:27:55.117Z
gcp-pubsub-topic-snapshots-list
Get a list of snapshots by project ID and topic ID.
Base Command
gcp-pubsub-topic-snapshots-list
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | The ID of the project from which this snapshot is retaining messages. | Optional |
| topic_id | The ID of the topic from which this snapshot is retaining messages. | Optional |
| page_size | Max number of results | Optional |
| page_token | Next page token as returned from the API. | Optional |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSnapshots.name | String | The name of the snapshot. |
| GoogleCloudPubSubSnapshots.topic | Unknown | The name of the topic from which this snapshot is retaining messages. |
| GoogleCloudPubSubSnapshots.expireTime | Date | The snapshot is guaranteed to exist up until this time. |
| GoogleCloudPubSubSnapshots.labels | Unknown | An object containing a list of "key": value pairs. |
| GoogleCloudPubSub.Snapshots.nextPageToken | String | If not empty, indicates that there may be more snapshot that match the request. |
Command Example
!gcp-pubsub-topic-snapshots-list project_id=dmst-integrations
Context Example
{
"GoogleCloudPubSubSnapshots": {
"expireTime": "2020-04-23T13:37:26.199Z",
"labels": {
"doc": "true"
},
"name": "projects/dmst-integrations/snapshots/doc_snapshot",
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Snapshots for project dmst-integrations
| name |
|---|
| projects/dmst-integrations/snapshots/doc_snapshot |
gcp-pubsub-topic-snapshot-create
Creates a snapshot from the requested subscription. Snapshots are used in gcp-pubsub-topic-messages-seek command.
Base Command
gcp-pubsub-topic-snapshot-create
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | ID of the project from which the subscription is receiving messages. | Optional |
| subscription_id | The subscription whose backlog the snapshot retains. | Required |
| labels | Input format: "key=val" pairs sepearated by ",". | Optional |
| snapshot_id | The id of the snapshot. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSnapshots.name | String | The name of the snapshot. |
| GoogleCloudPubSubSnapshots.topic | Unknown | The name of the topic from which this snapshot is retaining messages. |
| GoogleCloudPubSubSnapshots.expireTime | Date | The snapshot is guaranteed to exist up until this time. |
| GoogleCloudPubSubSnapshots.labels | Unknown | An object containing a list of "key": value pairs |
Command Example
!gcp-pubsub-topic-snapshot-create project_id=dmst-integrations subscription_id=test_sub_2 snapshot_id=doc_snapshot
Context Example
{
"GoogleCloudPubSubSnapshots": {
"expireTime": "2020-04-23T13:37:26.199Z",
"name": "projects/dmst-integrations/snapshots/doc_snapshot",
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Snapshot doc_snapshot was created successfully
| Expire Time | Name | Topic |
|---|---|---|
| 2020-04-23T13:37:26.199Z | projects/dmst-integrations/snapshots/doc_snapshot | projects/dmst-integrations/topics/dmst-topic |
gcp-pubsub-topic-snapshot-update
Updates an existing snapshot. Snapshots are used in gcp-pubsub-topic-messages-seek command.
Base Command
gcp-pubsub-topic-snapshot-update
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | The ID of the project from which the subscription is receiving messages. | Optional |
| expire_time | The snapshot is guaranteed to exist up until this time. A newly-created snapshot expires no later than 7 days from the time of its creation. A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2020-04-01T08:01:23.045678910Z" |
Optional |
| labels | Input format: "key=val" pairs sepearated by ",". | Optional |
| snapshot_id | The id of the snapshot. | Required |
| update_mask | Indicates which fields in the provided snapshot to update. A comma-separated list of fields. Example: "labels,topic,expireTime". |
Required |
| topic_id | The ID of the topic from which this snapshot is retaining messages. | Required |
Context Output
| Path | Type | Description |
|---|---|---|
| GoogleCloudPubSubSnapshots.name | String | The name of the snapshot. |
| GoogleCloudPubSubSnapshots.topic | Unknown | The name of the topic from which this snapshot is retaining messages. |
| GoogleCloudPubSubSnapshots.expireTime | Date | The snapshot is guaranteed to exist up until this time. |
| GoogleCloudPubSubSnapshots.labels | Unknown | An object containing a list of "key": value pairs |
Command Example
!gcp-pubsub-topic-snapshot-update project_id=dmst-integrations subscription_id=test_sub_2 snapshot_id=doc_snapshot labels="doc=true" update_mask=labels topic_id=dmst-topic
Context Example
{
"GoogleCloudPubSubSnapshots": {
"expireTime": "2020-04-23T13:37:26.199Z",
"labels": {
"doc": "true"
},
"name": "projects/dmst-integrations/snapshots/doc_snapshot",
"topic": "projects/dmst-integrations/topics/dmst-topic"
}
}
Human Readable Output
Snapshot doc_snapshot was updated successfully
| Expire Time | Labels | Name | Topic |
|---|---|---|---|
| 2020-04-23T13:37:26.199Z | doc: true | projects/dmst-integrations/snapshots/doc_snapshot | projects/dmst-integrations/topics/dmst-topic |
gcp-pubsub-topic-snapshot-delete
Removes an existing snapshot.
Base Command
gcp-pubsub-topic-snapshot-delete
Input
| Argument Name | Description | Required |
|---|---|---|
| project_id | The ID of the project from which the subscription is receiving messages. | Optional |
| snapshot_id | The id of the snapshot. | Required |
Context Output
There is no context output for this command.
Command Example
!gcp-pubsub-topic-snapshot-delete project_id=dmst-integrations snapshot_id=doc_snapshot
Context Example
{}
Human Readable Output
Snapshot doc_snapshot was deleted successfully
gcp-pubsub-topic-snapshot-delete
Removes an existing snapshot.
Base Command
gcp-pubsub-topic-ack-messages
Input
| Argument Name | Description | Required |
|---|---|---|
| subscription_id | The subscription that will have the messages acked. | Required |
| ack_ids | List of comma separated ids to ACK, as received from “gcp-pubsub-topic-messages-pull” or from “fetch-incidents”. | Required |
| project_id | The project id that the messages were pulled from. | Optional |
Context Output
There is no context output for this command.
Command Example
!gcp-pubsub-topic-ack-messages ack_ids=example_ack_id subscription_id=test_sub_2
Context Example
{}
Human Readable Output
Subscription test_sub_2 had the following ids acknowledged
| ACK ID |
|---|
| example_ack_id |
Configuration parameters
isFetch— Fetch incidentsincidentType— Incident typeincidentFetchInterval— Incidents Fetch Intervalservice_account_json— Service account private key file contents (JSON).credentials—insecure— Trust any certificate (not secure)proxy— Use system proxy settingsdefault_project— Default project ID. (required)default_subscription— Fetch incidents using the subscription ID.default_max_msgs— Maximum number of incidents per fetch.ack_incidents— Acknowledge incidents.first_fetch_time— First fetch time.
Commands (16)
-
gcp-pubsub-topic-ack-messagesAcknowledge previously pulled message or messages.
-
gcp-pubsub-topic-createCreate a topic.
-
gcp-pubsub-topic-deleteDelete a topic.
-
gcp-pubsub-topic-messages-pullPull messages that were published.
-
gcp-pubsub-topic-messages-seekSeeks a subscription to a given point in time or to a given snapshot.
-
gcp-pubsub-topic-publish-messagePublish a message in a topic.
-
gcp-pubsub-topic-snapshot-createCreates a snapshot from the requested subscription. Snapshots are used in the gcp-pubsub-topic-messages-seek command.
-
gcp-pubsub-topic-snapshot-deleteRemoves an existing snapshot.
-
gcp-pubsub-topic-snapshot-updateUpdates an existing snapshot. Snapshots are used in gcp-pubsub-topic-messages-seek command.
-
gcp-pubsub-topic-snapshots-listGet a list of snapshots by project ID and topic ID.
-
gcp-pubsub-topic-subscription-createCreate a pull or push subscription.
-
gcp-pubsub-topic-subscription-get-by-nameGet subscription details by the subscription ID.
-
gcp-pubsub-topic-subscription-updateUpdate a subscription.
-
gcp-pubsub-topic-subscriptions-listGet a list of subscriptions by project ID or topic ID.
-
gcp-pubsub-topic-updateUpdates a topic.
-
gcp-pubsub-topics-listGet a list of the topics of the project.
import base64 import json import dateparser import demistomock as demisto import pytest from GooglePubSub import ( LAST_RUN_FETCHED_KEY, LAST_RUN_TIME_KEY, GoogleNameParser, attribute_pairs_to_dict, convert_datetime_to_iso_str, create_subscription_command, create_topic_command, delete_topic_command, extract_acks_and_msgs, get_publish_body, get_subscription_command, message_to_incident, publish_message_command, pull_messages_command, seek_message_command, setup_subscription_last_run, snapshot_create_command, snapshot_delete_command, snapshot_list_command, snapshot_update_command, subscriptions_list_command, topics_list_command, try_pull_unique_messages, update_subscription_command, update_topic_command, ) class TestGoogleNameParser: DFLT_PROJECT_ID = "test_project" DFLT_TOPIC_ID = "test_topic" DFLT_SUB_ID = "test_sub" DFLT_SNAPSHOT_ID = "snapshot_sub" def test_get_full_project_name(self): """ Given: - project_id When: - we want project_name Then: - GoogleNameParser should parse it in the expected format """ expected = f"projects/{self.DFLT_PROJECT_ID}" assert expected == GoogleNameParser.get_project_name(self.DFLT_PROJECT_ID) def test_get_full_topic_name(self): """ Given: - project_id - topic_id When: - we want topic_name Then: - GoogleNameParser should parse it in the expected format """ expected = f"projects/{self.DFLT_PROJECT_ID}/topics/{self.DFLT_TOPIC_ID}" assert expected == GoogleNameParser.get_topic_name(self.DFLT_PROJECT_ID, self.DFLT_TOPIC_ID) def test_get_full_subscription_project_name(self): """ Given: - project_id - subscription_id When: - we want subscription_name Then: - GoogleNameParser should parse it in the expected format """ expected = f"projects/{self.DFLT_PROJECT_ID}/subscriptions/{self.DFLT_SUB_ID}" assert expected == GoogleNameParser.get_subscription_project_name(self.DFLT_PROJECT_ID, self.DFLT_SUB_ID) def test_get_full_subscription_topic_name(self): """ Given: - project_id - topic_id - subscription_id When: - we want topic subscription_name Then: - GoogleNameParser should parse it in the expected format """ expected = f"projects/{self.DFLT_PROJECT_ID}/topics/{self.DFLT_TOPIC_ID}/subscriptions/{self.DFLT_SUB_ID}" assert expected == GoogleNameParser.get_subscription_topic_name( self.DFLT_PROJECT_ID, self.DFLT_TOPIC_ID, self.DFLT_SUB_ID ) def test_get_snapshot_project_name(self): """ Given: - project_id - snapshot_id When: - we want snapshot_name Then: - GoogleNameParser should parse it in the expected format """ expected = f"projects/{self.DFLT_PROJECT_ID}/snapshots/{self.DFLT_SNAPSHOT_ID}" assert expected == GoogleNameParser.get_snapshot_project_name(self.DFLT_PROJECT_ID, self.DFLT_SNAPSHOT_ID) class TestHelperFunctions: DECODED_B64_MESSAGE = "decoded message" ENCODED_B64_MESSAGE = str(base64.b64encode(DECODED_B64_MESSAGE.encode("utf8")))[2:-1] DATE_NO_MS = "2020-01-01T11:11:11Z" DATE_WITH_MS = "2020-01-01T11:11:11.123000Z" MOCK_MESSAGE = {"messageId": "123", "publishTime": DATE_WITH_MS} def test_convert_datetime_to_iso_str(self): """ Given: - date with ms - date without ms When: - we want the publish_time in str Then: - convert_datetime_to_iso_str should convert the dates to the same string """ datetime_no_ms = dateparser.parse(self.DATE_NO_MS) assert f"{self.DATE_NO_MS[:-1]}.000000Z" == convert_datetime_to_iso_str(datetime_no_ms) datetime_with_ms = dateparser.parse(self.DATE_WITH_MS) assert convert_datetime_to_iso_str(datetime_with_ms) == self.DATE_WITH_MS def test_message_to_incident(self): """ Given: - pulled message When: - we want to convert it to an incident Then: - message_to_incident should convert it correctly """ incident = message_to_incident(self.MOCK_MESSAGE) assert incident.get("occurred") == self.DATE_WITH_MS assert f'Google PubSub Message {self.MOCK_MESSAGE.get("messageId")}' == incident.get("name") assert json.dumps(self.MOCK_MESSAGE) == incident.get("rawJSON") class TestGetPublishBody: def test_get_publish_body__invalid(self): """ Given: - invalid message data When: - we try to create a publish body Then: - throw an exception """ # invalid message_data e_thrown = False try: get_publish_body("", message_data={"test": "val"}, delim_char=",") except AttributeError: e_thrown = True assert e_thrown # invalid message_attributes e_thrown = False try: get_publish_body(message_attributes={"test": "val"}, message_data="", delim_char=",") except AttributeError: e_thrown = True assert e_thrown def test_get_publish_body__empty(self): """ Given: - empty message data When: - we try to create a publish body Then: - return a body with no messages """ expected = {"messages": [{}]} assert expected == get_publish_body("", "", "") def test_get_publish_body__valid(self): """ Given: message with - 2 attributes - decrypted message When: - we try to create a publish body Then: return a body with - attributes dict - encrypted message """ key_1 = "t_key1" val_1 = "t_val1" key_2 = "t_key2" val_2 = "t_val2" attrs_str = f"{key_1}={val_1},{key_2}={val_2}" expected_attributes = {key_1: val_1, key_2: val_2} expected_data = TestHelperFunctions.ENCODED_B64_MESSAGE expected = {"messages": [{"data": expected_data, "attributes": expected_attributes}]} assert expected == get_publish_body( attrs_str, TestHelperFunctions.DECODED_B64_MESSAGE, delim_char=",", ) class TestAttributePairsToDict: def test_attribute_pairs_to_dict__invalid(self): """ Given: - invalid attribute pairs When: - converting attribute pairs to dict Then: - throw an error """ e_thrown = False try: attribute_pairs_to_dict({"1": "1"}) except AttributeError: e_thrown = True assert e_thrown def test_attribute_pairs_to_dict__empty(self): """ Given: - empty attribute pairs When: - converting attribute pairs to dict Then: - return attribute pairs """ assert attribute_pairs_to_dict("") == "" assert attribute_pairs_to_dict(None) is None def test_attribute_pairs_to_dict__single(self): """ Given: - single attribute pair When: - converting attribute pairs to dict Then: - return a single attribute pair dict """ expected_key = "t_key" expected_val = "t_val" expected = {expected_key: expected_val} attrs_str = f"{expected_key}={expected_val}" assert expected == attribute_pairs_to_dict(attrs_str) def test_attribute_pairs_to_dict__multi(self): """ Given: - multiple attribute pairs When: - converting attribute pairs to dict Then: - return multiple pairs in a dict """ key_1 = "t_key1" val_1 = "t_val1" key_2 = "t_key2" val_2 = "t_val2" expected = {key_1: val_1, key_2: val_2} attrs_str = f"{key_1}={val_1},{key_2}={val_2}" assert expected == attribute_pairs_to_dict(attrs_str) class TestExtractAcksAndMsgs: def test_extract_acks_and_msgs__invalid(self): """ Given: - invalid pulled messages response When: - we want to extract acks and messages Then: - return an empty array tuple """ expected = ([], []) assert expected == extract_acks_and_msgs("invalid") empty_raw_msgs = {"receivedMessages": {}} assert expected == extract_acks_and_msgs(empty_raw_msgs) def test_extract_acks_and_msgs__empty(self): """ Given: - empty pulled messages response When: - we want to extract acks and messages Then: - return empty ack list, and message list with no message """ expected = ([], []) assert expected == extract_acks_and_msgs({}) expected = ([], [{"data": ""}]) invalid_raw_msgs = {"receivedMessages": [{}]} assert expected == extract_acks_and_msgs(invalid_raw_msgs) def test_extract_acks_and_msgs__single(self): """ Given: - single pulled messages response When: - we want to extract acks and messages Then: - return ack list with ack id, and message list with decoded message """ raw_msgs = { "receivedMessages": [ { "ackId": 1, "message": {"data": TestHelperFunctions.ENCODED_B64_MESSAGE}, } ] } expected = ([1], [{"data": "decoded message", "ackId": 1}]) assert expected == extract_acks_and_msgs(raw_msgs) def test_extract_acks_and_msgs__multi(self): """ Given: - multiple pulled messages response When: - we want to extract acks and messages Then: - return ack list with multi ack id, and message list with decoded messages """ raw_msgs = { "receivedMessages": [ { "ackId": 1, "message": {"data": TestHelperFunctions.ENCODED_B64_MESSAGE}, }, {"ackId": 2, "message": {"attributes": {"q": "a"}}}, ] } expected = ( [1, 2], [ {"data": "decoded message", "ackId": 1}, {"data": "", "attributes": {"q": "a"}, "ackId": 2}, ], ) assert expected == extract_acks_and_msgs(raw_msgs) class TestCommands: class MockClient: def __init__(self): self.default_max_msgs = "1" def publish_message(self, **kwargs): return "" def pull_messages(self, **kwargs): return "" def list_project_subs(self, a, b, c): return "" def get_sub(self, **kwargs): return "" def create_subscription(self, **kwargs): return "" def update_subscription(self, **kwargs): return "" def delete_subscription(self, **kwargs): return "" def list_topic(self, **kwargs): return "" def create_topic(self, **kwargs): return "" def delete_topic(self, a): return "" def update_topic(self, **kwargs): return "" def subscription_seek_message(self, **kwargs): return "" def get_project_snapshots_list(self, **kwargs): return "" def create_snapshot(self, **kwargs): return "" def update_snapshot(self, **kwargs): return "" def delete_snapshot(self, **kwargs): return "" def ack_messages(self, a, b): return "" with open("test_data/commands_outputs.json") as f: COMMAND_OUTPUTS = json.load(f) with open("test_data/raw_responses.json") as f: RAW_RESPONSES = json.load(f) TEST_COMMANDS_LIST = [ ( "gcp-pubsub-topic-publish-message", publish_message_command, "publish_message", { "data": "42\\42", "project_id": "dmst-doc-prjct", "topic_id": "dmst-test-topic", }, ), ( "gcp-pubsub-topic-messages-pull", pull_messages_command, "pull_messages", { "ack": "true", "max_messages": "1", "project_id": "dmst-doc-prjct", "subscription_id": "test_sub_2", }, ), ( "gcp-pubsub-topic-subscriptions-list", subscriptions_list_command, "list_project_subs", {"project_id": "dmst-doc-prjct"}, ), ( "gcp-pubsub-topic-subscription-get-by-name", get_subscription_command, "get_sub", {"subscription_id": "test_sub_2", "project_id": "dmst-doc-prjct"}, ), ( "gcp-pubsub-topic-subscription-create", create_subscription_command, "create_subscription", { "expiration_ttl": "86400s", "project_id": "dmst-doc-prjct", "topic_id": "dmst-test-topic", "subscription_id": "doc_sub_11", }, ), ( "gcp-pubsub-topic-subscription-update", update_subscription_command, "update_subscription", { "labels": "doc=true", "project_id": "dmst-doc-prjct", "subscription_id": "doc_sub_11", "topic_id": "dmst-test-topic", "update_mask": "labels", }, ), ( "gcp-pubsub-topics-list", topics_list_command, "list_topic", {"project_id": "dmst-doc-prjct"}, ), ( "gcp-pubsub-topic-create", create_topic_command, "create_topic", {"project_id": "dmst-doc-prjct", "topic_id": "dmst-doc-topic11"}, ), ( "gcp-pubsub-topic-delete", delete_topic_command, "delete_subscription", {"project_id": "dmst-doc-prjct", "topic_id": "dmst-doc-topic11"}, ), ( "gcp-pubsub-topic-update", update_topic_command, "update_topic", { "project_id": "dmst-doc-prjct", "topic_id": "dmst-doc-topic11", "labels": "doc=true", "update_mask": "labels", }, ), ( "gcp-pubsub-topic-messages-seek", seek_message_command, "subscription_seek_message", { "project_id": "dmst-doc-prjct", "subscription_id": "dean-sub1", "time_string": "2020-04-10T00:00:00.123456Z", }, ), ( "gcp-pubsub-topic-snapshots-list", snapshot_list_command, "get_project_snapshots_list", {"project_id": "dmst-doc-prjct"}, ), ( "gcp-pubsub-topic-snapshot-create", snapshot_create_command, "create_snapshot", { "project_id": "dmst-doc-prjct", "subscription_id": "test_sub_2", "snapshot_id": "doc_snapshot", }, ), ( "gcp-pubsub-topic-snapshot-update", snapshot_update_command, "update_snapshot", { "project_id": "dmst-doc-prjct", "snapshot_id": "doc_snapshot", "labels": "doc=true", "update_mask": "labels", "topic_id": "dmst-test-topic", }, ), ( "gcp-pubsub-topic-snapshot-delete", snapshot_delete_command, "delete_snapshot", {"project_id": "dmst-doc-prjct", "snapshot_id": "doc_snapshot"}, ), ] @pytest.mark.parametrize("command_name,command_func,client_func,args, ", TEST_COMMANDS_LIST) def test_commands(self, command_name, command_func, client_func, args, mocker): """ Given: - command function - args - client function name to mock - expected client function result - expected command result When: - we want to execute command function with args Then: - the expected result will be the same as actual """ raw_response = self.RAW_RESPONSES[command_name] expected = self.COMMAND_OUTPUTS[command_name] client = self.MockClient() mocker.patch.object(client, client_func, return_value=raw_response) res = command_func(client, **args) assert expected == res[1] def test_try_pull_unique_messages__empty_1(self, mocker): """ Test try_pull_unique_messages with empty result Given: - previous_msg_ids = set() - last_run_time = "2020-04-25T08:36:30.242Z" - there are no messages in queue When: - trying to pull unique messages Then: - try_pull_unique_messages should be called once (verified by demisto.debug) - function should return an empty result """ client = self.MockClient() sub_name = "test_sub_2" previous_msg_ids = set() last_run_time = "2020-04-25T08:36:30.242Z" mocker.patch.object(client, "pull_messages", return_value={}) debug_mock = mocker.patch.object(demisto, "debug") ( res_msgs, res_msg_ids, res_acks, res_max_publish_time, ) = try_pull_unique_messages(client, sub_name, previous_msg_ids, last_run_time, False, retry_times=1) assert debug_mock.call_count == 0 assert res_msgs is None assert res_msg_ids is None assert res_acks is None assert res_max_publish_time is None def test_try_pull_unique_messages__empty_2(self, mocker): """ Test try_pull_unique_messages with empty result - receivedMessages is empty Given: - previous_msg_ids = set() - last_run_time = "2020-04-25T08:36:30.242Z" - there are no messages in queue When: - trying to pull unique messages Then: - try_pull_unique_messages should be called once (verified by demisto.debug) - function should return an empty result """ client = self.MockClient() sub_name = "test_sub_2" previous_msg_ids = set() last_run_time = "2020-04-25T08:36:30.242Z" mocker.patch.object(client, "pull_messages", return_value={"receivedMessages": []}) debug_mock = mocker.patch.object(demisto, "debug") ( res_msgs, res_msg_ids, res_acks, res_max_publish_time, ) = try_pull_unique_messages(client, sub_name, previous_msg_ids, last_run_time, False, retry_times=1) assert debug_mock.call_count == 0 assert res_msgs is None assert res_msg_ids is None assert res_acks == [] assert res_max_publish_time is None def test_try_pull_unique_messages__unique_first_try(self, mocker): """ Test try_pull_unique_messages with a unique result on first try Given: - previous_msg_ids = set() - last_run_time = "2020-04-25T08:36:30.242Z" - there are messages in queue When: - trying to pull unique messages Then: - try_pull_unique_messages should be called once (verified by demisto.debug) - function should return a result with the pulled message """ client = self.MockClient() sub_name = "test_sub_2" previous_msg_ids = set() last_run_time = "2020-04-25T08:36:30.242Z" unique_messages = self.RAW_RESPONSES["try_pull_unique_messages_1"] mocker.patch.object(client, "pull_messages", return_value=unique_messages) debug_mock = mocker.patch.object(demisto, "debug") ( res_msgs, res_msg_ids, res_acks, res_max_publish_time, ) = try_pull_unique_messages(client, sub_name, previous_msg_ids, last_run_time, False, retry_times=1) assert not any(call.args[0].startswith("GCP_PUBSUB_MSG") for call in debug_mock.call_args_list) assert res_msgs == [ { "ackId": "321", "data": "42", "messageId": "123", "publishTime": "2020-04-18T08:36:30.541Z", } ] assert res_msg_ids == {"123"} assert res_acks == ["321"] assert res_max_publish_time == "2020-04-18T08:36:30.541000Z" def test_try_pull_unique_messages__unique_second_try(self, mocker): """ Test try_pull_unique_messages with a non-unique result on first try and unique result second time Given: - previous_msg_ids = {'123'} - last_run_time = "2020-04-25T08:36:30.242Z" - there are messages in queue When: - trying to pull unique messages Then: - try_pull_unique_messages should be called twice (verified by demisto.debug) - function should return a result with the unique pulled message """ client = self.MockClient() sub_name = "test_sub_2" previous_msg_ids = {"123"} last_run_time = "2020-04-25T08:36:30.242Z" unique_messages_list = [ self.RAW_RESPONSES["try_pull_unique_messages_1"], self.RAW_RESPONSES["try_pull_unique_messages_2"], ] mocker.patch.object(client, "pull_messages", side_effect=unique_messages_list) debug_mock = mocker.patch.object(demisto, "debug") ( res_msgs, res_msg_ids, res_acks, res_max_publish_time, ) = try_pull_unique_messages(client, sub_name, previous_msg_ids, last_run_time, False, retry_times=1) assert len(list(filter(lambda x: x.args[0].startswith("GCP_PUBSUB_MSG"), debug_mock.call_args_list))) == 1 assert res_msgs == [ { "ackId": "654", "data": "43", "messageId": "456", "publishTime": "2020-04-19T08:36:30.541Z", } ] assert res_msg_ids == {"456"} assert res_acks == ["654"] assert res_max_publish_time == "2020-04-19T08:36:30.541000Z" def test_try_pull_unique_messages__partially_unique_first_try(self, mocker): """ Test try_pull_unique_messages with a partially unique result on first try Given: - previous_msg_ids = set() - last_run_time = "2020-04-09T08:36:30.242Z" - there are messages in queue When: - trying to pull unique messages Then: - try_pull_unique_messages should be called twice (verified by demisto.debug) - function should return a result with the unique pulled message """ client = self.MockClient() sub_name = "test_sub_2" previous_msg_ids = {"123"} last_run_time = "2020-04-09T08:36:30.242Z" unique_messages_list = [self.RAW_RESPONSES["try_pull_unique_messages_3"]] mocker.patch.object(client, "pull_messages", side_effect=unique_messages_list) debug_mock = mocker.patch.object(demisto, "debug") ( res_msgs, res_msg_ids, res_acks, res_max_publish_time, ) = try_pull_unique_messages(client, sub_name, previous_msg_ids, last_run_time, False, retry_times=1) assert not any(call.args[0].startswith("GCP_PUBSUB_MSG") for call in debug_mock.call_args_list) assert res_msgs == [ { "ackId": "654", "data": "43", "messageId": "456", "publishTime": "2020-04-19T08:36:30.541Z", } ] assert res_msg_ids == {"456"} assert res_acks == ["654"] assert res_max_publish_time == "2020-04-19T08:36:30.541000Z" def test_setup_subscription_last_run__first_run(self, mocker): """ Test setup_subscription_last_run first_run Given: - first_fetch_time = valid date str - last_run = empty When: - setting up a subscription for fetch-incidents for the first time Then: - subscription should seek previous state """ client = self.MockClient() first_fetch_time = "3 days" last_run = {} sub_name = "test_sub_2" ack_incidents = False sub_seek_mock = mocker.patch.object(client, "subscription_seek_message") last_run_fetched_ids, last_run_time = setup_subscription_last_run( client, first_fetch_time, last_run, sub_name, ack_incidents ) assert sub_seek_mock.call_count == 1 assert last_run_fetched_ids == set() assert last_run_time is not None def test_setup_subscription_last_run__not_first_run__with_acks(self, mocker): """ Test setup_subscription_last_run non-first_run Given: - last_run = previous run data - ack = True When: - setting up a subscription for fetch-incidents not for the first time Then: - subscription should not seek previous state """ client = self.MockClient() first_fetch_time = "" last_run = { LAST_RUN_TIME_KEY: "2020-04-09T08:36:30.242Z", LAST_RUN_FETCHED_KEY: ["123"], } sub_name = "test_sub_2" ack_incidents = True sub_seek_mock = mocker.patch.object(client, "subscription_seek_message") last_run_fetched_ids, last_run_time = setup_subscription_last_run( client, first_fetch_time, last_run, sub_name, ack_incidents ) assert sub_seek_mock.call_count == 0 assert last_run_fetched_ids == {"123"} assert last_run_time is not None def test_setup_subscription_last_run__not_first_run__no_acks(self, mocker): """ Test setup_subscription_last_run non-first_run Given: - last_run = previous run data - ack = False When: - setting up a subscription for fetch-incidents not for the first time Then: - subscription should seek previous state """ client = self.MockClient() first_fetch_time = "" last_run = { LAST_RUN_TIME_KEY: "2020-04-09T08:36:30.242Z", LAST_RUN_FETCHED_KEY: ["123"], } sub_name = "test_sub_2" ack_incidents = False sub_seek_mock = mocker.patch.object(client, "subscription_seek_message") last_run_fetched_ids, last_run_time = setup_subscription_last_run( client, first_fetch_time, last_run, sub_name, ack_incidents ) assert sub_seek_mock.call_count == 1 assert last_run_fetched_ids == {"123"} assert last_run_time is not None