My current setup in GCP includes a Logging Sink to a Pub/Sub topic which is exporting certain Healthcare API log entries to a third party logging tools. Despite having, what I believe to be, all required permissions, the activity page shows Stackdriver config error with an error message something like this
Access to the topic was denied
The specified topic does not allow the service account associated with the log sink to publish to it. Grant publish permission for the service account specified in the sink's writerIdentity field on the topic.
Now the service account in question has the following set of permissions already assigned to it
Healthcare Dataset Administrator
Healthcare FHIR Resource Editor
Healthcare FHIR Resource Reader
Healthcare FHIR Store Administrator
Service Account Admin
Service Account Key Admin
Service Account User
Workload Identity User
Logs Configuration Writer
Logs Writer
Pub/Sub Editor
Pub/Sub Publisher
According to this thread, its is a known issue but its already been past the 24 hour threshold mentioned in there so I am a bit confused.
Are there any particular permissions that I am missing here? I have tried recreating the sink once or twice to no use. Appreciate any help from the community on this one. Thanks.
EDIT:
Here is what I found on running gcloud logging sinks describe <sink name>
createTime: '2021-02-23T00:42:35.363983703Z'
destination: <PUBSUB_TOPIC>
filter: resource.labels.service="healthcare.googleapis.com"
name: <SINK_NAME>
updateTime: '2021-03-01T18:44:50.694055502Z'
writerIdentity: <SA_NAME>
Related
I am new to GCP pub/sub and am trying to get everything setup. I am facing a permission error when creating a subscription topic. Does anyone know what I am missing in my steps, or how to resolve? Here is what I have done so far.
Add roles, Service Account Admin and Service Account Key Admin
Created Service Account (completed the full flow)
Tried to create a subscription topic <-- this is where I am receiving an error.
Here is the error at a closer look.
UPDATE: After receiving some comments, it seems my issues are all related to roles and permissions. The problem is that I do not know which roles/permissions work.
I am able to create a subscription topic so long that I do NOT enable authentication and add a service account. Since my original post, I have added these other additional permissions: Pub/Sub Admin, Service Account Admin, Service Account Key Admin, Service Account Token Creator, Service Account User.
I'm trying to create a job in Dataflow to export to a text file the data published to a pub/sub topic. When saving the configuration I get a 'Job creation failed' message specifying 'Current user cannot act as service account ...-compute#developer.gserviceaccount.com', as shown in the attached picture.
Following Google's documentation I added the following roles to my user for this project (in addition to the owner role I already have):
Compute Viewer
Dataflow Admin
Dataflow Developer
Storage Object Admin
Service Account User
However the Controller Service Account mentioned in the message doesn't seem to exist in the list of Account Services of this project (IAM & Admin > Account Services). Is there anything I'm missing here?
Other requirements already checked:
I have the Compute Engine API already enabled
As owner I have the iam.serviceAccounts.actAs permission
Your best option is to create a custom service account in IAM and use it to build/run your job. if you're using cloud build to deploy and run your template you'll need to set your logging location.
More details at the below links:
Using custom service accounts in cloud build
Setting logging location in cloud build YAML
In my project, project A publishes a message and project B pulls it out of Google cloud. I have several B clients and want to limit them to a specific topic and subscription. I tried the IAM conditions but it does not work for pubsub and seems to be only for pubsub lite. Does anyone know how to restrict user access to a particular topic and subscription?
You can grant roles at different level:
Organization
Folder
Project
Resource
When you go to the IAM page, you grant at the project level and thus you have access to all the resources of the project.
You should be in this case where you grant a service account of the project B to pubsub role on project A. And thus the service account has access to all topics/subscriptions.
To solve this, you can only grant a service account on a topic or on a subscription (at resource level)
Go to the topic or subscription page
Tick the checkbox in front of the resource that you want
Go to the right, in the info panel, in the permission tab
Click on add member to grant a service account (or a user account) only on this resource
Alternatively, you can use the method projects.topics.setIamPolicy of the PubSub API to set permissions at the resource level.
If you don't want to call the API directly, you can also use CLI with this command
gcloud pubsub topics set-iam-policy TOPIC POLICY_FILE
Background
I have a Google Cloud project running my N applications. Each application has an exclusive IAM service account (total N service account) with minimal permissions.
Scenario
Let's imagine that one of the service accounts was leaked out. An attacker will try to take advantage of these credentials. Because he doesn't know exactly which kind of permissions this account has, we will try to make calls and see if it working for him.
Question
I want to "listen" to audit logs. Once I will see the log from kind "access denied", I will know that something is wrong with this service account.
Is this possible to write all those access denied incidents to Google Cloud Stackdriver?
How you recommend implementing it?
Thank you
Is this possible to write all those access denied incidents to Google
Cloud Stackdriver?
Most but not all Google Cloud services support this. However, access success will also be logged.
You will need to enable Data Access Audit Logs.
This could generate a massive amount of logging information.
Access logs for Org and Folder are only available via API and not the console.
Review pricing before enabling data access audit logs.
How you recommend implementing it?
This question is not suitable for Stackoverflow as it seeks recommendations and opinions. In general you will export your logs to Google Cloud Pub/Sub to be processed by a service such as Cloud Functions, Cloud Run, etc. There are also commercial services such as DataDog designed for this type of service support. Exporting logs to Google BigQuery is another popular option.
Read this article published on DataDog's website on Data Access Audit Logging and their services. I am not recommended their service, just providing a link to more information. Their article is very good.
Best practices for monitoring GCP audit logs
To understand the fields that you need to process read this document:
AuthorizationInfo
This link will also help:
Understanding audit logs
Here is one way to go about it:
Create a new cloud pubsub topic
Create a new log routing sink with destination service of cloud pubsub topic created in the previous step (set a filter to be something like
protoPayload.authenticationInfo.principalEmail="<service-account-name>#<project-name>.iam.gserviceaccount.com" AND protoPayload.authorizationInfo.granted="false" to only get messages about unsuccessful auth action for your service account)
Create a cloud function that's triggered with a new message for the pubsub topic is published; this function can do whatever you desire, like send a message to the email address, page you or anything else you can come up with in the code.
I am trying to subscribe to Gmail for mail notifications using Google's Pub/Sub and I've gone through the documentation and tutorials provided by Google. At one point, the docs state that I need to publish the correct rights to my PubSub topic:
You need to grant publish privileges to serviceAccount:gmail-api-push#system.gserviceaccount.com. You can do this using the Cloud Pub/Sub Developer Console permissions interface following the resource-level access control instructions.
My understanding is that the publish privilege is equivalent to pubsub.topics.publish or just "PubSub Publisher". But what about the serviceAccount:gmail-api-push#system.gserviceaccount.com? I cant' find that service account anywhere and the console is not letting me create it due to character restrictions. Is there another way to get this account?
You do not need to create the gmail service account, it already exists. You are correct about “granting publish privileges” as the same as “Pub/Sub Publisher”.
Under the topic permissions interface, I would pass in the service account gmail-api-push#system.gserviceaccount.com (note: take out the leading “serviceAccount:” part) in the ‘new members’ field, and select “Pub/Sub Publisher” as the role.
I am guessing you couldn’t find the service account because you were looking up serviceAccount:gmail-api-push#system.gserviceaccount.com.
Permissions are set by Pub/Sub Topic in the Pub/Sub Topics section of the GCP Console.
Go to the Google Cloud Console.
From the Hamburget menu, go to Big Data -> Pub/Sub -> Topics.
Shortcut to this location: https://console.cloud.google.com/projectselector2/cloudpubsub/topicList
Click the checkbox for the Topic.
At the right side of the console, select "SHOW INFO PANEL".
Click Add members
Enter the service account and select the desired role. Usually Pub/Sub Publisher.