It appears the Google Tasks API tasklists resource pulls the list of tasks associated with the current user. I would like to specify the user that the task list will be generated for. The plan would be to generate a list of tasks and the documents (if any) that they are associated with.
Leveraging the AdminDirectory library would allow cycling through select users to grab the tasks associated with the desired user(s).
const tasklists = Tasks.Tasklists.list();
A list of tasks associated with the account that ran the script is listed, not all the tasks in the organization.
Related
Every user on our website can schedule multiple jobs on GCP using an API. The jobs are called projects/PROJECT_ID/locations/LOCATION_ID/jobs/USER_ID-RANDOM_STRING.
Now I need to delete all jobs of a user. Assuming there are many jobs (100k+), how do I delete all jobs of that user? Simply iterating through them does not scale. Any ideas? I'd rather avoid having to store all of the job IDs separately.
TL;DR
We are looking for a way for letting a service account inherit BQ read permissions from more than one other service account. Impersonation does only work with one.
The scenario
Our company follows a data mesh approach in which our product teams are responsible for integrating their data into BigQuery. The product owner is also considered the owner of the data. So it is the product owner to decides whom to give access to the data.
Working in an analytical team, we usually combine data from multiple source systems in our BigQuery queries. Our ETL processes run on a kubernetes cluster, each process is using a separate service account. This gives us fine-grained access control as access to data is limited for each process to those very objects that they really need. This design also helps us with debugging and cost control. On the other hand, this leads to an issue on source side:
The problem
Every time we design a new process, we need to ask the data owner for allowance. They already agreed that our product team / system at wholesome level may access their data, so this authorization process is quite cumbersome and confuses the data owner.
We'd prefer to have just one "proxy" service account for each source object that holds the neccessary BQ read permissions. The processes' service accounts would then be set up to inherit the rights from the proxy service account of those BQ sources they need to access.
Using impersonation does only help if it's just one source system, but our queries often use more than one.
Using Google Groups does not help
We discussed a solution in which we setup a google group for each source system we want to read from. BigQuery Data Reader role will then be assigned to this group. In turn, service accounts that require those rights will be added to the group. However, company policy does not allow for adding service accounts to google groups. Also, google groups cannot be managed (created) by our product teams themselves, so this approach lacks flexibility.
Implementing a coarse-grained approach
One approach is to use a more coarsed-grained access control, i.e. just using one service account for all ETL processes. We could add the process name as a label to the query to cover the debugging and cost control part. However, if possible, we'd prefer an approach in which the processes can only access as little data objects as possible.
You haven't easy solution.
Data governance is in place to control the quality, the source and the access to the data. It's normal to ask them to access the data.
Special groups could have access to all the data source (after a request to the data gov team of each data mesh instance).
However, groups with service account aren't allowed.
The only solution that I see is to use a service account, authorized on all the data mesh instances, and you impersonate it to access to all the sources.
It's not the most perfect for traceabilty, but I don't see any other good solution for that.
Let's imagine a scenario where there are multiple teams with apps using AWS batch in same aws account.
Say TeamA and TeamB.
Teams use AWS SDKs to have a dashboard to manage their jobs.
I need to restrict TeamA from submitting,terminating jobs created by TeamB and vice-versa.
I'm able to restrict actions that rely on job definitions or job queue, by having teams use a prefix while creating the resources they own
and having a policy with prefix before a wildcard
But TerminateJob relies on a jobid which is dynamic. How do restrict them from terminating someother teams jobs?
I'm looking for a resource level policy or condition keys that can restrict access.
Looked around a lot not finding anything that explains how to do this.
Any advice is much appreciated.
Thanks
I have a cloud setup, with a DynamomDB table with entries - which each have a corresponding SQS queue (created by a lambda when entries are added in the DB)
A number of external systems need access to one or more entries, which means that they should read messages from the SQS queue linked to the entries in the database.
I'm looking for the smartest way to do the following:
Create database entries (based on an input JSON string or file reference etc) - this will automatically generate the needed queues
Create an IAM user
Generate ARNs for all queues generated in (1) and generate a matching set of permissions, so the user created in (2) can read/delete/purge these queues.
Output the newly created user credentials and all related items in the database.
Of course I can make a lambda and a javascript that does all of it. But I'm looking for a smarter way to do this, hopefully using the AWS CLI only?
I ended up doing like this:
Created user groups in CF
Made a shell script for creating users and assigning proper groups
Made another shell script for creating resources in the database table, and adding the created user name as a part of the data model
Made a lambda (triggered by DB events) that sets permissions on the queues related to the database entries and sets the ARN of the users in the database, as the principal.
Works so far!
I have created five separate log export sinks within Google Cloud Stackdriver. Currently they are all set to the same bucket (my-bucket) with the destination:
storage.googleapis.com/my-bucket
The bucket (my-bucket) has the following 5 folders:
iam, compute, firewall, project and storage
I would like to associate each log sink to one of those folders within my bucket, is this possible? The answer from a related question (Pointing multiple projects' log sinks to one bucket) seems to indicate that it is, however, I do not understand the "FOLDER_ID" and if this is what I need and if so where do I get that.
I have tried to update the destination manually in google cloud shell with the command
gcloud logging sinks update my-compute-log-sink storage.googleapis.com/my-bucket/compute
and get confirmation that the sink has been updated, however, running gcloud logging sinks list shows no change and now I am stuck.
This isn't currently possible.
The location you used when attempting to update the sink (when you include the directory) will only point to the bucket as a whole despite adding the additional information after it, so when you see the updated message, you are probably overwriting the current sink with the same information as before.
I've opened a feature request to see if it will be possible to implement the usage of specified bucket directory locations in Google Cloud Platform logging sinks. You can follow it here:
https://issuetracker.google.com/72036905
There is also a separate feature request which relates to the task you are trying to acehive that you can follow here:
https://issuetracker.google.com/69371200
In terms of the FOLDER_ID value, that's not related to the folder in the bucket, but relates to the Cloud Platform Resource Hierarchy as explained here. It's therefore not something related to your main issue.
The logging sink destination (for cloud storage) must be a bucket.
The folder referred to in the answer to Pointing multiple projects' log sinks to one bucket is for grouping projects. That answer links to the documentation for folders, which describes them as:
Folders are nodes in the Cloud Platform Resource Hierarchy. A folder
can contain projects, other folders, or a combination of both.
Cloud Storage does not support something like 'folders' within buckets, only buckets and objects. See https://cloud.google.com/storage/docs/key-terms