I have a time series model and I've forecasted using SARIMAX. I'm very new to AWS. I want to deploy this on AWS Sagemaker but SARIMAX is not available directly on AWS sagemaker.
So the one method I read about was to create docker image and push it to ECR. As I don't know docker I'm searching if there are any other methods to do this. Is this the only way?
Can you please tell me if there are any alternatives of how can I do it in sagemaker? Or if I could use lambda?
Related
I am trying to trigger Sagemaker notebook or Sagemaker Studio notebook from AWS Lambda when data is available in S3 bucket. I want to know if this is possible and if yes, how?
All I want is once data is uploaded in S3, the lambda function should be able to spin up the Sagemaker notebook with a standard CPU cluster.
Here is a Jupyter plug in that you can use to do this, please note this is not managed by AWS. It is experimental software and should be used that way.
https://github.com/aws-samples/sagemaker-run-notebook
Using this extension, you can run your notebook based on an event.
I work at AWS and my opinions are my own.
I am new to AWS SageMaker and i am using this technology for building and training the machine learning models. I have now developed a docker image which contains our custom code for tensorflow. I would like to upload this custom docker image to AWS SageMaker and make use of it.
I have searched various links but could not find proper information on how to upload our own custom docker image.
Can you please suggest me the recommended links regarding the process of uploading our own docker image to AWS SageMaker?
In order to work with sagemaker, you have to push your container to ECR. The most important thing is that the container must be "adapted" to be complaint to what sagemaker requires, but everything is described here. In addition if you want to take a look to an example, here is mine.. where I use my container with TF Object Detection API in AWS Sagemaker.
I have a working setup in AWS that looks something like:
The point is that whenever a file is uploaded to S3, it will trigger a lambda that will trigger a Codebuild project. The codebuild project is then based on a docker image (Stored at ECR) and needs to run a few bash command, mainly executing python files in the docker image. That works really well actually.
The files in S3 are updated approximately once a day and each execution in codebuild takes around 4 minutes.
I got the question why I am not using fargate/SageMaker (the scripts are basicly machine learning retraining and predictions). I was just thinking about if there would be any advantages in using Fargate and/or SageMaker for this? Is it e.g. possible to use Fargate and execute bash commands inside the container when triggered?
IIUC, You're wondering the diffenerce between CodeBuild and Fargate/SageMaker
Price
Calcute the price of these three products using the links below.
Pricing Fargate
Pricing SageMaker
Pricing CodeBuild
As your said, you're using the docker image as the main training tools, so maybe the Fargateis more suitable for your scenario.
My dear people,
I am running some processing jobs in some files stored in S3, I just need regular computing power without GPU. I found I can use both Sagemaker preprocessing jobs using Sagemaker SDK; and also can do the exact same task using Fargate based AWS ECS Task using the Python-based ECS SDK. When I compare the procedure it seems like both of them are very similar which is
Build a docker image
Push the image to ECR
Configure Fargate or ECS
Run the task
Also from the pricing model, it seems really close; so I am wondering why on the same platform there are two services doing a very similar thing. Can anyone explain the motivation behind it and which service to use when if I don't need GPU?
Is there any way to load containers stored in docker hub registry in Amazon Sagemaker.
According to some documentation, it should be possible, but I have not been able to find any relevan example or guide for it.
While you can use any registry when working with Docker on a SageMaker notebook, as of this writing other SageMaker components presently only support images from Amazon ECR repositories.