I have a Google Cloud Storage Trigger set up on a Cloud Function with max instances of 5, to fire on the google.storage.object.finalize event of a Cloud Storage Bucket. The docs state that these events are "based on" the Cloud Pub/Sub.
Does anyone know:
Is there any way to see configuration of the topic or subscription in the console, or through the CLI?
Is there any way to get the queue depth (or equivalent?)
Is there any way to clear events?
No, No and No. When you plug Cloud Functions to Cloud Storage event, all the stuff are handle behind the scene by Google and you see nothing and you can't interact with anything.
However, you can change the notification mechanism. Instead of plugin directly your Cloud Functions on Cloud Storage Event, plug a PubSub on your Cloud Storage event.
From there, you have access to YOUR pubsub. Monitor the queue, purge it, create the subscription that you want,...
The recomended way to work with storage notifications is using Pubsub.
Legacy storage notifications still work, but with pubsub you can "peek" into the pubsub message queue and clear it if you need it.
Also, you can process pubsub events with cloud run - which is easier to develop and test (just web service), easier to deploy (just a container) and it can process several requests in parallel without having to pay more (great when you have a lot of requests together).
Where does pubsub storage notifications go?
You can see where gcloud notifications go with the gsutil command:
% gsutil notification list gs://__bucket_name__
projects/_/buckets/__bucket_name__/notificationConfigs/1
Cloud Pub/Sub topic: projects/__project_name__/topics/__topic_name__
Filters:
Event Types: OBJECT_FINALIZE
Is there any way to get the queue depth (or equivalent?)
In pubsub you can have many subsciptions to topics.
If there is no subsciption, messages get lost.
To send data to a cloud function or cloud run you setup a push subscription.
In my experience, you won't be able to see what happened because it faster that you can click: you'll find this empty 99.9999% of the time.
You can check the "queue" depht in the console (pubsub -> choose you topics -> choose the subscription).
If you need to troubleshoot this, set up a second subscription with a time to live low enough that it does not use a lot of space (you'll be billed for it).
Is there any way to clear events?
You can empty the messages from the pubsub subscription, but...
... if you're using a push notification agains a cloud function it will much faster than you can "click".
If you need it, it is on the web console (opent the pubsub subscription and click in the vertical "..." on the top right).
Related
Currently, I am moving services from AWS to GCP. Previously, I relied on an AWS S3 bucket and the inbuilt service's logic to configure event notifications to get triggered when an object with a particular prefix was inserted into my bucket. This specific event notification, which contained the prefix, would then be fed forward to trigger a lambda function.
However, now I have to leverage GCP Cloud Storage buckets and trigger a Cloud Function. My observations so far have been that I can't specify a prefix/suffix as part of my Cloud Storage service's bucket directly. Instead, I have to specify a Cloud Storage bucket to monitor during the creation of my Cloud Function. My concern with this approach is that I can't limit the bucket's object events to the three of interest to me: '_MANIFEST' '_PROCESSING' and '_PROCESSED' but rather have to pick an global event notification type of interest to me such as 'OBJECT_FINALIZE'.
There are two viable approaches I can see to this problem:
Have all the 'OBJECT_FINALIZE' event notifications trigger the Cloud Function and filter out any additional objects (those which don't contain the prefix). The issue with this approach is the unneccessary activation of the Cloud Function and the additional log files getting generated - which are of no inherent value.
Use the audit logs generated by the Cloud Storage bucket and create rules to generate events based on the watched trigger file i.e. '_MANIFEST', 'PROCESSING' and 'PROCESSED'. My concern with this approach is that I don't know how easily it will be to forward all the information about the bucket I'm interested in if I'm generating the event based on a logging rule - I am primarily interested in the information which gets forwarded by an event notification. Also, currently I have verified that the object being added to my Cloud Storage bucket is not public and I have enabled the following:
However, I tried to filter the audit logs in the GCP 'Monitoring' service (after adding a _MANIFEST object to the bucket of course) but the logs are not appearing within the 'Log Explorer'.
Any advice on how I should approach filtering the event notification of interest in GCP, when triggering my Cloud Function, would be greatly appreciated.
To achieve this, you can sink the Cloud Storage notification into PubSUb.
Then, you can create a PubSUb push subscription to your Cloud Functions (it's no longer a background functions triggered by Cloud Storage event, but and HTTP function trigger by HTTP request.
The main advantage of doing that is that you can specify a filter on PubSub push subscription that allow you to activate your Cloud Functions (or any other HTTP endpoint) only with the pattern is enforced.
I wanted to get notified if/when there is/are any VM creation in my infra on GCP.
I see a google library that can give me list of VM.
I can create a function to use this code (probably)
Schedule the above function. And check for difference.
But do storage like triggers available for Compute.
Also if there is any other solution.
You have a third solution. You can use Cloud Run instead of Cloud Functions (the migration is very easy, let me know if you have issues).
With Cloud Run, you can use the trigger (eventArc feature), a new feature (still in preview) based on the auditLog logs. It's very similar to the first solution proposed by LundinCast, but it's automatically set up by Cloud Run Trigger feature.
So, deploy your service on Cloud Run. Then configure a trigger on v1.compute.instancs.insert API, select your region or make the trigger global and that's all!! Your service will be triggered when a new instance will be created.
As you can see in my screenshot, you will be asked to activate the auditLog to be able to use this feature. Because it's built-in, it's done automatically for you!
Using Logging sink and a PubSub-triggered Cloud Function
First, export the relevant logs to a PubSub topic of your choice by creating a Logging sink. Include the logs created automatically during VM creation with the following log filter:
resource.type="gce_instance"
protoPayload.methodName="beta.compute.instances.insert"
protoPayload.methodName="compute.instances.insert"
Next, create a Cloud Function that'll trigger every time a new log is set to the PubSub topic. You can process this new message as per your needs.
Note that with this option you'll have to handle to notification yourself (for example, by sending an email). It is useful though if you want to send different notification based on some condition or if you want to perform additional actions apart from the notification.
Using a log-based metric and a Cloud Monitoring alert
You can use a Log-based metric filtering logs for Compute Engine VM creation and set an alert on that metric to get notified.
First create a counter log-based metric with a log filter similar to the one in the previous method, which will report a data point to Cloud monitoring every time a new VM instance is created.
Then go to Cloud Monitoring and create an alert based on that metric that trigger every time a metric is reported.
This option is the easiest to set up and supports various notification channels out-of-the-box.
Going along with LudninCast's answer.
Cloud Run --
Would have used it if it had not been zone issue for me. Though I conclude this from POC I did
Easy setup.
Containerised Apps. Probably more code to maintain.
Public URL for app.
Out of box support for the requirements like mine.
Cloud Function --
Sink setups for triggers can be time consuming for first timer
Easy coding and maintainance.
How would our organization log, audit, and alert on any code changes (add, change, delete) to Google Cloud Functions to survive an external audit? We've figured out how to do so on AWS (combination of CloudTrail and CloudWatch Events/Amazon EventBridge) and Azure (Audit log and Alerts under the Monitor service, although this is not as reliable as the AWS solution because some events do not seem to be picked up. Azure even has this nice new service in preview called Application Change Analysis, but it does not alert, and it goes away when a function is deleted instead of reporting that it has been deleted.)
But how do we do the same thing with Google Cloud Functions? How would we log and audit the creation/update/deletion of Cloud Functions and Cloud Function code? How would we go even further and receive an alert whenever any of those conditions occur, just like we have proven can happen with AWS and (kind of, at least) with Azure? Thank you!
You can use the Cloud Function audit logs. You can export the logs to PubSub, and then, you can do what you want on the event:
Store them in BigQuery for the history
Send an alert (email, slack message,...)
Act: for example, perform a rollback to the previous code stored in the source repository
...
All depends on your security process and what do you want to do with the events.
Is there a solution/service available on GCP in similar lines of Systems Manager?
My end goal is to run a shell script on GCP VM on specific events.
Like for AWS, via EventBridge I was able to trigger a Lambda Function and the function in turn triggered a SSM command for specific VM.
Is this possible on GCP?
There isn't a Systems Manager equivalent in GCP.
A Pub/Sub subscription from the VMs/compute units which triggers a lambda function (cloud function in GCP) is a suboptimal solution and different from what Systems Manager accomplishes..
I don't know what kind of events you have in mind that would trigger running a script but you can check out the tutorial how to run a function using pub/sub. It shows how to use scheduler based events but it's possible to use not-scheduled triggers;
Events are things that happen within your cloud environment that you might want to take action on. These might be changes to data in a database, files added to a storage system, or a new virtual machine instance being created. Currently, Cloud Functions supports events from the following providers:
HTTP
Cloud Storage
Cloud Pub/Sub
Cloud Firestore
Firebase (Realtime Database, Storage, Analytics, Auth)
Stackdriver Logging—forward log entries to a Pub/Sub topic by creating a sink. You can then trigger the function.
And here you can read on how to implement those triggers.
For example this documentation explains how to use storage based triggers in pub/sub.
If you provide more details of what exactly you want to achieve (what events have to trigger what) then I can point you to a more direct solution.
The approach depends on the exact use case you have in hand. One of the common architecture option could be using pub/sub with cloud functions. Based on messages published to Pub/Sub topics, cloud functions performing operations of our interest can be triggered/ invoked in the same cloud project as the function.
I use Cloud pub/sub and Cloud Functions.
Now, I want to publish a message to a topic which will trigger a background cloud function.
But I want to trigger my cloud function after a specific duration, like 30 seconds later.
How can I do this?
update:
Here is my architecture, is it correct?
Now, I want to publish a message to a topic which will trigger a
background cloud function. But I want to trigger my cloud function after a specific duration, like 30 seconds later.
If you setup PubSub to trigger Cloud Functions on publish events, Cloud Functions will be triggered almost immediately. There is no method to insert a delay.
You will need to implement your code as several major steps:
Setup PubSub Topic and Subscriptions. Do not trigger Cloud Functions on new messages. Messages will just sit waiting for delivery. Send messages to this topic.
Create a Cloud Function that processes PubSub subscriptions. Pull messages and process.
Use another service such as Cloud Tasks, Cloud Scheduler or App Engine Tasks to trigger your Cloud function after your desired delay.
You can use Cloud Tasks to schedule some work to happen on a delay.