I have Datalab running on one Google Cloud project (lets call it A), I have data sitting in another project (B). I'd like to grant Datalab access to this data.
I note that Datalab uses my projects "Compute Engine default" service account - I assume I can authorize this account in my second project (B) to grant Datalab access to the data within it. Is this considered the best practice approach and are there any other considerations I should keep in mind?
The right way to do is exactly what you think.
Go to 'IAM and admin' -> 'Admin' then choose your service account and a role for this project.
Keep in mind that project has been created to create security on accessibility. Sometimes having replicated data is not a bad idea. Really depends on the need.
Related
We're adding a GCP project to be used for greenfield development, e.g. sort of a developer sandbox. My inclination is to give application/service developers full permissions in that project, to reduce friction and let them get stuff done as quickly and easily as possible.
We then have a separate beta project which we use where we prepare work for production, where application/service developers would have limited-to-no access, but the devops team could productionize things. And then, of course, we have the production project, where everything is locked down tight.
Is a sandbox like this a good idea? What permission(s) would I grant? Owner? GCP recommends not using those legacy roles...
List all of what each team is allowed to do on each env.
Translate this to a list of IAM permissions per team per env.
If there is some predefined role/s that matches exactly these permissions then use that role/s
If not, then create your own custom role/s for each team per each env.
For example, in the sandbox env:
if developers team is only allowed to create GKE clusters and deploy workloads to these GKEs then list all required permissions for such operation and find a predefined role that have permissions that only allows this operation. See here.
Or, if this is too wide and does not apply the least privilege concept for you then create your own custom role.
I personally don't recommend to restrict the IAM permission. Indeed, in a sandbox project, you want to try things, and maybe thing totally outside of the box and unexpected as usual way of working/processing. Using IAM to limit the set of allowed product restrict the creativity and protect you against (almost) nothing.
Indeed, if you want to perform security restriction it's for what? Limit the access to the service in Beta environment? Not sure... Prevent the overuse of resources in a non-production (and no profitable) environment? I think yes!
That's why, I recommend to use the Quotas to restrict the number of resources available for a project (i.e. only 10 CPUs in 1 region and not 3600 in 20 regions as by default). Like that, the app team will be able to try and experiment safely, without any restriction, but without killing your budget.
I've got project A with Storage buckets A_B1 and A_B2. Now Dataproc jobs running from project B needs to have read access to buckets A_B1 and A_B2. Is that possible somehow?
Motivation: project A is production environment with production data stored in Storage. Project B is "experimental" environment running experiment Spark jobs on production data. Goal is to obviously separate billing for production and experiment environment. Similar can be done with dev.
Indeed, the Dataproc cluster will be acting on behalf of a service account in project "B"; generally it'll be the default GCE service account, but this is also customizable to use any other service account you create inside of project B.
You can double check the service account name by getting the details of one of the VMs in your Dataproc cluster, for example by running:
gcloud compute instances describe my-dataproc-cluster-m
It might look something like <project-number>-compute#developer.gserviceaccount.com. Now, in your case if you already have data in A_B1 and A_B2 you would have to recursively edit the permissions on all the contents of those buckets to add access for your service account using something like gsutil -m acl ch -r -u -compute#developer.gserviceaccount.com:R gs://foo-bucket; while you're at it, you might also want to change the bucket's "default ACL" so that new objects also have that permission. This could get tedious to do for lots of projects, so if planning ahead, you could either:
Grant blanket GCS access into project A for project B's service account by adding the service account as a project member with a "Storage Reader" role
Update the buckets that might need to be shared in project A with read access and/or write/owners access by a new googlegroup you create to manage groupings of permissions. Then you can atomically add service accounts as members to your googlegroup without having to re-run a recursive update of all the objects in the bucket.
Short background, we're a small business but our clients are much larger businesses. We have some software they subscribe to which is deployed to AWS elastic beanstalk. Clients have their own devops teams, unlike us, and will need to manage some of the technical support. They will need access to the AWS account running the software, so they can do things like reboot the server, clear the database if they screw it up, change the EC2 instance type etc. This is OK but we want to prevent the software being downloaded outside of the AWS account.
The software is a java WAR running on Tomcat, on a single elastic beanstalk instance. We only care about limiting access to the WAR file (not the database for example).
The beanstalk application versions page appears to have no way to download the WAR file - which is good. They could SSH into the underlying EC2 instance though so presumably they could just copy the WAR out of the tomcat directory. Given the complexity of AWS there's probably other ways they could get access the WAR file too (e.g. clone the EBS volume and attach to another EC2 instance).
I assume that the machine instances available for purchase via AWS marketplace must have some form of copy protection but I've not been able to find any details on this. Also it looks like AWS only accepts marketplace vendors who are much larger than us, so marketplace option may not be open to us.
Any idea how I could prevent access to the WAR file running on elastic beanstalk while still allowing the client access to the AWS account? (Or at least make access hard).
The only solution that comes to mind for this would be, removing any EC2 SSH Key Pairs from the account, and specifically denying them access to ec2:CreateKeyPair. Really, what you need to be doing is granting them least privilege access to the account, that is, specifically granting them access only to those actions they absolutely need.
This will go a long way, but with sufficient knowledge of AWS, it's going to be an uphill battle trying to ensure that you give them enough access to do what they need, while not giving them more than you want. I'd question if a legal option (like contracts, licenses, etc) would be a better protection for this.
I am looking for a way to access AWS resources on multiple AWS accounts using AWS SDK (Java, probably irrelevant) running on an EC2 instance.
To be more precise, imagine there are two AWS accounts:
test
prod
There is a Java application running on an EC2 instance for prod account that creates for example an instance of AmazonEC2Client (from AWS Java SDK). I would like to be able to create the client instance for both test and prod accounts from there.
I am able to do this using profiles on local machine, but haven't figured out how to do this on EC2 instance.
You can do it one of two ways;
AssumeRole
Use the AWS Security Token Service to AssumeRole. The actual implementation would be up to you, but you can assign an IAM Role to the EC2 Instance, and then use the AssumeRole function in the Java SDK to switch from a production role to testing (or vice versa).
Profiles
It sounds like you've already figured out how to do this locally, but it could be replicated in EC2 (depending on the use case). That said, I would strongly advise against it and instead follow best practices and use STS:AssumeRole (above).
Realistically, it would probably be best (making some assumptions here about what you are actually trying to do) to have a separate EC2 instance for production and testing, in which case you can assign appropriate roles as needed.
Moreover, if we are talking about a larger deployment, you may want to explore Consolidated Billing which would let you isolate your production and testing environments completely by using separate AWS accounts for each (but a single billing account). To learn more about it, I would suggest taking a look at this article: https://blog.codeship.com/separate-aws-production-and-development-accounts/
I have a Google Cloud VM instance, but I overlooked setting the Storage permission to read-write when creating it.
Now further down the line, I'm looking to experiment with cloud storage, but my instance is read-only.
How can this be changed? I understand it may be possible by relaxing the storage buckets policy, but I'd prefer my instance had write access to all future project buckets.
I presume there is an option in gcloud to change the devstorage parameter?
So, you can't change the option to grant the VM permission, but I did find that you can just run
gcloud auth login
then you can sign in with your management account from the VM and you'll be able to run commands that way.
Unfortunately you can't change the scopes of the VM. You will have to create a new one to change them.
When you create a new one you can reuse the disk of the old VM, if that helps avoid the pain.
You can update the permissions of your VM instance now but only when it is shut down. Check the following documentation:
Changing the service account and access scopes for an instance
If you want to update the API permissions for the kuberntes clusters VM instance then you cannot do that unless you create a new cluster and give the API access to the nodes associated with the clusters.
I believe they have added the option to change it now, without creating another VM.
Once you have stopped the instances, click on the instances you want to change. On the top there is a Edit button, you can click on it and change any the permissions.
Hope the image helps
edit button
Once you have changed the permission to read_write and it still says Access Denied. Go in your instance SSH browser window and enter 'gcloud auth login', follow the steps and hopefully it works!
You need to stop your vm first, then click on edit and change the cloud api access scopes for storage.
You may find more information here: https://ismailyenigul.medium.com/setting-access-scope-of-google-cloud-vm-instances-c8637718f453