I know that it is possible to send gmail messages to cloud pub/sub using watch options in gmail API. That is great! But, I have 4 different gmail accounts and I would like all them to push notifications into a single cloud pub/sub so that I can consume them with cloud functions and redistributed among slack channels (no, it cannot be done separately on each account since the billing is controlled only by one of them. It can also be external clients gmails which obviously won't pay for that.). Any option to push notifications into single cloud pub/sub?
You can configure the notifications from all four gmail accounts to be sent to the same Cloud Pub/Sub topic. From the documentation:
Using your Cloud Pub/Sub client, create the topic that the Gmail API
should send notifications to. The topic name can be any name you
choose under your project (i.e. matching projects/myproject/topics/*,
where myproject is the Project ID listed for your project in the
Google Developers Console).
You can then send watch() requests on each of the four accounts, specifying the same topicName.
Related
I am relatively new to GCP platform. I need to create a system wherein my team gets notifications whenever an email is received from there client.
I have to create a system which is publishing messages in pubsub topic filtered by email id.
example : I want to publish only those message/emails in which "To" and "From" fields has "example#gmail.com"
I have referred the online documentations but could not find the workaround.
Is this possible using GCP?
If not is there any other service via which I can achieve the same ?
We are using Gmail as the email client
Thanks
What you can use, is Pub/Sub together with Cloud Functions, which enables to access Gmail programmatically. Your specific scenario could look as following:
User sets up Gmail push notifications: every time a new message arrives at inbox, Gmail will send a notification to Cloud Pub/Sub.
Cloud Pub/Sub delivers the new message notification to Google Cloud Functions.
Upon arrival of the new message notification, a Cloud Functions instance connects to Gmail and retrieves the new message.
Check who send the message, and perform specific actions.
Before setting up a Cloud Function to automatically read your emails, you must authorize its access to Gmail. Have a look for codelab scenario and see step by step how to perform specific actions and adjust the function for your needs. Additionally, you can take a look for official documentation here.
Can I publish an event into Cloud Pub/Sub outside from GCP?
Let me clarify my query a bit. In AWS as we are able to publish events into SNS topics directly by invoking REST API via API Gateway from the non-cloud client (https://github.com/cdk-patterns/serverless/blob/master/the-big-fan/README.md), is there any such method in GCP to publish an event into Pub/Sub?
I can see there is a similar question in SO (Acces Google Pub/Sub from outside of GCP), but it is not fully answered my question I believe. Yes authentication is required and it is a cross-cutting functionality, but what is the basic technic to publish an event in Pub/Sub outside from GCP
Yes, you can publish a message into a topic, and then pull it, or even have PubSub deliver it to you through a Push subscription.
When publishing a message, or pulling it from a subscriber, you can access PubSub through the REST or RPC API. In addition, you can use one of the client libraries.
Here you can find an example of how to publish a message using the gcloud CLI tool, an example with the REST API, python and java among other programming languages.
As mentioned in the question referenced, you will need to authenticate in order to either publish a message, or pull it. You can use the quickstart as a reference on how to do so. Notice that you can follow the quickstart from any computer or VM outside GCP.
Finally, if you're using Push subscriptions to receive your messages, your endpoint will need to be a publicly accessible HTTPS address and have a valid SSL certificate signed by a certificate authority. Again, this endpoint can live outside GCP.
Let me try to explain to you what we are trying to do.
Saying it quickly: We want to give Pub/Sub Publisher (in our GoogleCloud) privileges to a GMail-API that is outside of our GoogleCloud.
What we have:
Following instructions here: https://developers.google.com/gmail/api/quickstart/ruby
We've created a project for GMail-API and the credentials in GoogleCloud, let's call it Cloud-A. (We'll not own this side in a production environment; this project and cloud will be managed by our customers' IT department.)
Next, we followed this other guide https://developers.google.com/gmail/api/guides/push
We created a Pub/Sub topic in our GoogleCloud (Cloud-B), we own this portion and it's the topic where we want to subscribe in order to listen for messages/notifications. (This topic is in a different account from the GMail-API that will be publishing messages that is Cloud-A).
So, following that last guide, it says that we need to give permissions to gmail-api-push#system.gserviceaccount.com and from my understanding what that means is that I'm giving privileges to GMail-API from Cloud-B to publish messages in Pub/Sub Cloud-B.
What I can't find out is a way to give permissions to Gmail-API from Cloud-A to publish messages in Pub/Sub Cloud-B.
To wrap up, I want to listen to Pub/Sub in Cloud-B that will receive notifications from Gmail-API in Cloud-A.
We used this https://github.com/googleapis/google-api-ruby-client/blob/master/generated/google/apis/gmail_v1/service.rb#L144 and I get an error saying that the topic doesn't exist (Probably because it is in Cloud-B and I'm configuring Cloud-A Gmail-Api)
I hope I was clear enough, we are not looking to given another project inside the same Google Cloud access to a Pub/Sub, it isn't even a service that we wrote since it is Gmail-Api and the only thing we are allowed to do is to send it the topic name we want it to publish in.
I'm not familiar with how the GMail-API publishes to Pub/Sub, but, if you have already figured out how to publish from GMail-API in project Cloud-A to a Pub/Sub topic in Cloud-A, you may try the following workarounds:
Alternative A:
Create the topic (topic-A) in project Cloud-A.
Create a pull subscription (subs-A) associated to topic-A also in project Cloud-A.
Create a service account (account-B) in project Cloud-B and grant it the Pub/Sub subscriber role for subscription subs-A.
Make your consumers (e.g. AppEngine, GKE, GCE) use service account account-B to pull messages from subs-A.
Alternative B:
Create the topic (topic-A) in project Cloud-A.
Create a push subscription (subs-A) associated to topic-A pointing to an endpoint of a service hosted in project Cloud-B (e.g. GCE, GKE, AppEngine, Cloud Function, etc.)
Alternative C:
Create the topic (topic-A) in project Cloud-A.
Create a pull or push subscription (subs-B) in project Cloud-B associated to topic-A in project Cloud-A. The user creating this subscription should have the Pub/Sub Editor role granted for topic-A.
Consume the messages from subs-B.
I am working on an assignment where our customers will sync their crm data to our systems. The sync will be ongoing process. Any best practices or suggestions on google pub/sub for sharing one of our existing (or a new one) topic that our customer will publish too and we consume? Idea here is to keep sync asynchronous.
#Alex-hong is true if your customer is on GCP. If not, you have 2 solutions:
You can generate a service account and send it the JSON key file. Define the right role on the service account (as described by Alex) and let your customer use the Service Account and to publish to the PubSub topic. This solution implies an important development by your customer (Use external JSON key, use new libraries to push messages,...)
You can deploy a Cloud Function/Cloud Run endpoint and let your customer to simply perform an HTTP request. Of course, you can secure the call with Basic Authentication or APIkey (or something like that, that you check in your Cloud Function). The function only check the security and publish to PubSub. It's often simpler and more standard for your customer.
On the last point, it could be possible to set up an ESP in place of the function, but I never tested for publishing directly to PubSub.
You can give the other party the roles/pubsub.publisher role for your specific project if you own the topic. Alternatively, if they own the topic, they can grant you the role roles/pubsub.subscriber which will let you subscribe to that topic.
For more information, see Access Control
I want to use google cloud pub sub to implement Facebook kind of notifications in blog application which has following architecture
I have a both web and android device as a client for rest services exposed.
Those rest services has been secured using OAuth2 (spring security).
When user create a blog then if someone comment on that post, the concern owner can get the notifications. Only authorized user can create a post or comment and the concern person can see the notifications
I don't want user to authorize again with Google so how I can incorporate the api's in my architecture using pull notification
You typically wouldn't use Google Cloud Pub/Sub in the client in the type of setup you describe. Cloud Pub/Sub isn't designed to scale to a number of subscriptions that are on the order of number of phones. Instead, you'd want to use Firebase Cloud Messaging. Firebase has a model that handles registration of devices to receive messages that is more in line with what you are thinking of, e.g., Set Up a Firebase Cloud Messaging Client App on Android and
Implementing a Firebase Cloud Messaging Client on Chrome
What you may use Cloud Pub/Sub for is communication between different services. For example, perhaps the service that handles blog posts publishes messages on a Cloud Pub/Sub topic that your notification service subscribes to. The notification service would process the message and send out Firebase Cloud Messaging messages to end users as appropriate.