I am trying to subscribe to Gmail for mail notifications using Google's Pub/Sub and I've gone through the documentation and tutorials provided by Google. At one point, the docs state that I need to publish the correct rights to my PubSub topic:
You need to grant publish privileges to serviceAccount:gmail-api-push#system.gserviceaccount.com. You can do this using the Cloud Pub/Sub Developer Console permissions interface following the resource-level access control instructions.
My understanding is that the publish privilege is equivalent to pubsub.topics.publish or just "PubSub Publisher". But what about the serviceAccount:gmail-api-push#system.gserviceaccount.com? I cant' find that service account anywhere and the console is not letting me create it due to character restrictions. Is there another way to get this account?
You do not need to create the gmail service account, it already exists. You are correct about “granting publish privileges” as the same as “Pub/Sub Publisher”.
Under the topic permissions interface, I would pass in the service account gmail-api-push#system.gserviceaccount.com (note: take out the leading “serviceAccount:” part) in the ‘new members’ field, and select “Pub/Sub Publisher” as the role.
I am guessing you couldn’t find the service account because you were looking up serviceAccount:gmail-api-push#system.gserviceaccount.com.
Permissions are set by Pub/Sub Topic in the Pub/Sub Topics section of the GCP Console.
Go to the Google Cloud Console.
From the Hamburget menu, go to Big Data -> Pub/Sub -> Topics.
Shortcut to this location: https://console.cloud.google.com/projectselector2/cloudpubsub/topicList
Click the checkbox for the Topic.
At the right side of the console, select "SHOW INFO PANEL".
Click Add members
Enter the service account and select the desired role. Usually Pub/Sub Publisher.
Related
How to pull the list of IAM users from google cloud along with their last activity??
Tried "gcloud projects get-iam-policy"
but it gives only list of iam users/members but not their last activity
Ok, if it's for company, you have this information in the Google Cloud Identity platform. You can log in here: https://admin.google.com
Go to users and boom
Of course you can request these values by API with the admin sdk
It works only for managed accounts. If you have unmanaged account (in gmail.com or from another company) you don't have access to this information.
EDIT 1
To track the service account activity, you can rely on the documentation. Cloud Monitoring allow you to do that. If you need to export the data to BigQuery for analytics for example, let me know I could help on that.
To know the privilege that the users have, you can rely on the Asset Inventory, and especially on the IAM search policy feature.
In my project, project A publishes a message and project B pulls it out of Google cloud. I have several B clients and want to limit them to a specific topic and subscription. I tried the IAM conditions but it does not work for pubsub and seems to be only for pubsub lite. Does anyone know how to restrict user access to a particular topic and subscription?
You can grant roles at different level:
Organization
Folder
Project
Resource
When you go to the IAM page, you grant at the project level and thus you have access to all the resources of the project.
You should be in this case where you grant a service account of the project B to pubsub role on project A. And thus the service account has access to all topics/subscriptions.
To solve this, you can only grant a service account on a topic or on a subscription (at resource level)
Go to the topic or subscription page
Tick the checkbox in front of the resource that you want
Go to the right, in the info panel, in the permission tab
Click on add member to grant a service account (or a user account) only on this resource
Alternatively, you can use the method projects.topics.setIamPolicy of the PubSub API to set permissions at the resource level.
If you don't want to call the API directly, you can also use CLI with this command
gcloud pubsub topics set-iam-policy TOPIC POLICY_FILE
My current setup in GCP includes a Logging Sink to a Pub/Sub topic which is exporting certain Healthcare API log entries to a third party logging tools. Despite having, what I believe to be, all required permissions, the activity page shows Stackdriver config error with an error message something like this
Access to the topic was denied
The specified topic does not allow the service account associated with the log sink to publish to it. Grant publish permission for the service account specified in the sink's writerIdentity field on the topic.
Now the service account in question has the following set of permissions already assigned to it
Healthcare Dataset Administrator
Healthcare FHIR Resource Editor
Healthcare FHIR Resource Reader
Healthcare FHIR Store Administrator
Service Account Admin
Service Account Key Admin
Service Account User
Workload Identity User
Logs Configuration Writer
Logs Writer
Pub/Sub Editor
Pub/Sub Publisher
According to this thread, its is a known issue but its already been past the 24 hour threshold mentioned in there so I am a bit confused.
Are there any particular permissions that I am missing here? I have tried recreating the sink once or twice to no use. Appreciate any help from the community on this one. Thanks.
EDIT:
Here is what I found on running gcloud logging sinks describe <sink name>
createTime: '2021-02-23T00:42:35.363983703Z'
destination: <PUBSUB_TOPIC>
filter: resource.labels.service="healthcare.googleapis.com"
name: <SINK_NAME>
updateTime: '2021-03-01T18:44:50.694055502Z'
writerIdentity: <SA_NAME>
Background
I have a Google Cloud project running my N applications. Each application has an exclusive IAM service account (total N service account) with minimal permissions.
Scenario
Let's imagine that one of the service accounts was leaked out. An attacker will try to take advantage of these credentials. Because he doesn't know exactly which kind of permissions this account has, we will try to make calls and see if it working for him.
Question
I want to "listen" to audit logs. Once I will see the log from kind "access denied", I will know that something is wrong with this service account.
Is this possible to write all those access denied incidents to Google Cloud Stackdriver?
How you recommend implementing it?
Thank you
Is this possible to write all those access denied incidents to Google
Cloud Stackdriver?
Most but not all Google Cloud services support this. However, access success will also be logged.
You will need to enable Data Access Audit Logs.
This could generate a massive amount of logging information.
Access logs for Org and Folder are only available via API and not the console.
Review pricing before enabling data access audit logs.
How you recommend implementing it?
This question is not suitable for Stackoverflow as it seeks recommendations and opinions. In general you will export your logs to Google Cloud Pub/Sub to be processed by a service such as Cloud Functions, Cloud Run, etc. There are also commercial services such as DataDog designed for this type of service support. Exporting logs to Google BigQuery is another popular option.
Read this article published on DataDog's website on Data Access Audit Logging and their services. I am not recommended their service, just providing a link to more information. Their article is very good.
Best practices for monitoring GCP audit logs
To understand the fields that you need to process read this document:
AuthorizationInfo
This link will also help:
Understanding audit logs
Here is one way to go about it:
Create a new cloud pubsub topic
Create a new log routing sink with destination service of cloud pubsub topic created in the previous step (set a filter to be something like
protoPayload.authenticationInfo.principalEmail="<service-account-name>#<project-name>.iam.gserviceaccount.com" AND protoPayload.authorizationInfo.granted="false" to only get messages about unsuccessful auth action for your service account)
Create a cloud function that's triggered with a new message for the pubsub topic is published; this function can do whatever you desire, like send a message to the email address, page you or anything else you can come up with in the code.
I am currently working on a GCP project, where I need to try and achieve the following.
I need to somehow restrict which GCP functions can publish to which GCP topics?
Does anybody have any ideas how I could achieve this?,
Thanks,
Tom Early
Cloud Functions runs as PROJECT_ID#appspot.gserviceaccount.com and has the permissions of that service account. Therefore if you want different functions to have different permissions, you will have to create them in separate projects. For example you could have:
Function abc in project abc-proj
Function xyz in project xyz-proj
Pub/Sub topics a-topic and z-topic in project pubsub-proj
Grant projects.topics.publish permission on topic a-topic (in pubsub-proj) to abc-proj#appspot.gserviceaccount.com
Grant projects.topics.publish permission on topic z-topic (in pubsub-proj) to xyz-proj#appspot.gserviceaccount.com
And you will then have function abc able to publish to pubsub-proj/a-topic and function xyz able to publish to pubsub-proj/z-topic, but not cross-wise.
When you wish to publish a message, you specify the topic where you will publish this message. This means the message will only be published to that particular topic and no other topics. You can have a look at documentation related to publishing messages here if you need further help with publishing a message to a particular topic.
Since I still have no reputation to do so (sorry about that), I am writing a new answer just to let everyone that might come across this know that the answer by David is also valid for Cloud Run by simply replacing the service account by the one that is handling your Cloud Run service.
According to the latest Cloud Functions doc [1], the Cloud Functions service uses the Google Cloud Functions service agent service account (service-PROJECT_NUMBER#gcf-admin-robot.iam.gserviceaccount.com).
So, if you need the additional permissions beyond the default settings. You can bind additional roles to that service account.
And make sure the service account has permission to publish to the Pub/Sub topic [2].
[1] https://cloud.google.com/functions/docs/concepts/iam#cloud_functions_service_account
[2] https://cloud.google.com/functions/docs/concepts/iam#troubleshooting_permission_errors
Hope this helps ~:)