When a sagemaker studio domain is created. An EFS storage is associated with the domain. As the assigned users log into Sagemaker studio, a corresponding home directory is created.
Using a separate EC2 instance, I mounted the EFS storage that was created to try to see whether is it possible to look at each of the individual home domains. I noticed that each of these home directories are shown in terms of numbers (e.g 200000, 200005). Is there a specific rule on how this folders are named? Is it possible to trace the folders back to a particular user or whether this is done by design?
(currently doing exploration on my personal aws account)
Yes, if you list and describe the domain users, you'll get back the user's HomeEfsFileSystemUid value.
Here's a CLI example:
aws sagemaker describe-user-profile --domain-id d-lcn1vbt47yku --user-profile-name default-1588670743757
{
...
"UserProfileName": "default-1588670743757",
"HomeEfsFileSystemUid": "200005",
...
}
Related
In order to change some settings, there is currently a need to recreate the domain. However, I have an existing EFS that I would like to retain. Is it possible to remount this EFS directory to the new domain?
It is not currently possible to attach an existing EFS to a new domain. If you have multiple users and data, I'd recommend the steps in the Backup and recovery section here.
If it is only a handful user profiles, you can simply download the files/use S3 as an intermediate storage.
I'm trying to set up my first SageMaker Studio so my team and myself can run some post processing scripts in a shared environment but I'm having issues.
I've followed the steps in this video(https://www.youtube.com/watch?v=wiDHCWVrjCU&ab_channel=AmazonWebServices) which are:
Select Standard setup
Select AWS Identity and Access Management (IAM)
Under permissions - Create and select new execution role
Under Network and storage - Select VPC, Subnet and Security group
Hit the submit button at the bottom of the page.
In the video, he clicks submit and is taken to the control panel where he starts the next phase of adding users, however I'm greeted with this error.
Resource limit Error
I've checked my Registered domains under route 53 and it says No domains to display, I've also checked my S2 and I have no instances so I have no idea where the 2 domains being utilized are.
My dashboard, image and Notebooks are all empty so as far as I know there's nothing setup on this Sage Maker account.
Could anyone tell me how to resolve this error?
AWS Sagemaker now supports multi-domain <announced in the Re-Invent 2022, Tested in US-EAST-1 >
enter image description here
You can have maximum 1 studio domain per region, by the default limits. Though, it seems like you have two domains already provisioned. Try to delete all the domains through the AWS cli and recreate with the AWS Management Console.
Unfortunately, AWS Management Console cannot visualize more than one Studio domain.
A colleague and I have a bucket each in the same gcloud project, and are both experiencing this behavior on our respective buckets.
When I login to gcloud in local terminal and do gcloud app deploy without specifying anything, my code deploys to my bucket. If instead I do gcloud app deploy --bucket=(my bucket) a large number of files are deposited in the bucket whose names are long strings of alphanumerics. The files I want to put are compiled JS in a build folder, and these weird files seem to be all the individual JS files from the project instead. In both cases it finds the bucket fine but the first option concerns me because I worry it's only finding my bucket due to my account's permissions or something.
I'd appreciate any details anyone has on how app deploy really works because we're very confused about this. The first option appears to work but this won't do for automation and we don't want to deploy to all the buckets by accident and break everything.
gcloud app deploy uses Google Cloud Storage buckets to stage files and potentially create containers that are used by the App Engine service:
https://cloud.google.com/sdk/gcloud/reference/app/deploy#--bucket
If you don't specify a bucket using --bucket flag, defaults are used:
staging.[project-id].appspot.com
[us.]artifacts.[project-id].appspot.com
BLOBs are stored in a GCS bucket named:
[project-id].appspot.com
https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/setting-up-cloud-storage#activating_a_cloud_storage_bucket
NB If you also use Google Container Registry you may see additional buckets named *.artifacts.[project-id].appspot.com. As with the bucket used by App Engine these contain objects representing the container layers.
I have an ec2 instance that I use as sort of a staging environment for small websites and custom Wordpress websites.
What I'm trying to find out is; Can I create a bucket for /var/www/html/site1 and assign FTP access to Developer X to work on this particular site within this particular bucket?
No. Directories on your EC2 instance have no relationship with S3.*
If you want to set up permissions for files stored on your EC2 instance, you'll have to do it by making software configuration changes on that instance, just as if it were any other Linux-based server.
*: Assuming you haven't set up something weird like s3fs, which I assume isn't the case here.
Having spent a couple of days setting up and configuring a new AWS account I would like to grab an export of the account configuration across all services. I've Googled around for existing scripts, etc, but have yet to find anything that would automate this process.
Primarily this would be as a backup incase the account was corrupted in some way (including user error!) but this would also be useful to document the system.
From an account administration perspective, there are various parts of the AWS console that don't display friendly names for various resources. Being able to cross reference against offline documentation would simplify these scenarios. For example, friendly names for vpc's and subnets aren't always displayed when configuring resources to use them.
Lastly I would like to be able to use this to spot suspicious changes to the configuration as part of intrusion detection. For example, looking out for security group changes to protected resources.
To clarify, I am looking to backup the configuration of AWS resources, not the actual resources themselves. Resource backups (e.g. EC2 instances) is already covered.
The closest i've seen to that is CloudFormer.
That would create a CloudFormation template from your account's resources. Mind that this template would be only a starting point, not meant to be reproducible out-of-the-box. For example, it won't log into your instances or anything like that.
As for the intrusion detection part, see CloudTrail
Check out AWS Config: https://aws.amazon.com/config/
AWS Config records the configuration of AWS resources automatically, allowing you to query and react to configuration changes. As AWS Config stores data on S3, that is probably enough backup, but you can also sync the bucket elsewhere for paranoid redundancy.