I've been using a Windows 2008R2 EC2 instance for some time. As of today, it still works. I started working with the AWS API, and I was unable to start my instance using the API, the error message being "not authorized for images", specifically : An error occurred (AuthFailure) when calling the RunInstances operation: Not authorized for images: [ami-088dab1e]
That's when I learned about deprecation.
From what I read, what this means is that the AMI being used is no longer publicly available. When using the API call "describe-images", this image cannot be queried. While it apparently can still be used from the console, the API simply doesn't support it and will not start an instance using that image ID. On the console, the AMI description reads : Cannot load details for ami-088dab1e. You may not be permitted to view it.
I understand how to find a new image and I think I understand how to launch my instance using a new image. However, I have lots of custom software installed on this instance. So before I try it, I want to know if I will lose that custom software installation if I launch my existing instance with a new AMI. I'm hoping that my custom software won't change, but I'm skeptical. I don't want to fire up a brand new version of Windows and start from scratch. Mostly, I don't want to lose what I've already got.
I know this is a basic question, but I've looked all over, and I haven't yet found a straightforward answer. I was hoping y'all would know. Thanks.
I think I've found an answer here: AWS EC2 new instance from image AMI
When launching an instance from an Amazon Machine Image (AMI), the disks will contain an exact copy of the disk at the time that the AMI was created.
In other words, if I start a new instance, I'll lose my installed software. WRONG!
Launching != starting. More editing to come once I get this completely figured out.
So, given that updated Windows images are created and deprecated all the time, and the Windows OS is constantly updated by Microsoft, one must wonder how it is a static Windows image can be used with other software? It seems like far more trouble than it's worth, if you've got to constantly reinstall your software to keep your Windows system up to date.
Amazon recently came up with a solution for that, here: Patching Windows
I don't know how to do it yet, but this seems like exactly what I need in order to keep Windows up to date, and keep my installed software intact.
Related
I am trying to figure it out which one is the best practice when creating a new server on AWS EC2.
To do that I choose Tableau Server. First time, as Tableau docs recommend I did the install myself, but I would like to keep as automatic as possible everything, the idea behind that is if ec2 get destroyed how can I recover everything fast?
I am using terraform to store as a code all the AWS infrastructure, but the installation itself is not automatic yet.
To do that, I have two options, ansible (never worked before) or in this particular case Tableau has an automated install script in python, which I could add in the EC2 template launch configuration,and then using terraform I can raise it in minutes.
Which one should be the choosen why? Both seems to acomplish the final goal.
Also it raises some kind of doubs such as:
It retrieves the server up and with a full instalation of the software, but to get all users, and all the Tableau setup I have to raise anyways an snapshot, right? Is there any other tool to do that?
Then, if the manual install of the software is fast enough, why then I should use IaC to keep the install as code, instead of document the script of installation? And just keep the Infrastructure as code?
So I've recently joined in a project. They asked me to develop a particular module so I did. Now I need to integrate it with the system. They gave me AWS login credentials for the integration purpose. I'm new to AWS and I don't wanna sound dumb to them by asking where's the code. I saw that there's an EC2 instance running but I see no option to see code there. So can you please let me know where can I see the code of running EC2 instance?
Never feel dumb about asking questions on your team. It's much better to ask questions and seek clarification, rather than assume and waste your time and theirs.
So if your team is tasking you with integrating a module you've built with something running on EC2, they probably have an API of some sorta to integrate. They likely aren't expecting you to go to EC2 and view code or decompile DLLs to view source code.
However, to potentially answer your question if your EC2 instance is running some sort of application that has DLLs, you can download those and decompile them using various tools to view the actual source code. YOu would of course need the keypair to access the EC2 instances so you'd have to get that first.
I would just ask someone on your team how to integrate with the system running on EC2. They likely have the source code stored somewhere in a repository.
We have multiple windows server 2016 running, will automatic updates on. I want to change the Windows Update mode to Manual.
The problem is when instance is created, out software installation fails many times due to running updates installation. I tried to set instance metadata 'disable-agent-updates' to TRUE. But it doesn't seem to be working.
Please help on how to modify windows update mode on creation and start of instance.
The quick answer is you cannot with Google provided Windows images.
You can create a Windows VM, configure as you desire and then create a new image that you use to launch new instances. This will accomplish your goal with some effort. This is standard practice for companies that manage/control patches and updates.
Note: I do not recommend disabling Windows Update from installing security patches on startup unless you understand the implications.
I have finally arrived in the cloud to put my NLP work to the next level, but I am a bit overwhelmed with all the possibilities I have. So I am coming to you for advice.
Currently I see three possibilities:
SageMaker
Jupyter Notebooks are great
It's quick and simple
saves a lot of time spent on managing everything, you can very easily get the model into production
costs more
no version control
Cloud9
EC2(-AMI)
Well, that's where I am for now. I really like SageMaker, although I don't like the lack of version control (at least I haven't found anything for now).
Cloud9 seems just to be an IDE to an EC2 instance.. I haven't found any comparisons of Cloud9 vs SageMaker for Machine Learning. Maybe because Cloud9 is not advertised as an ML solution. But it seems to be an option.
What is your take on that question? What have I missed? What would you advise me to go for? What is your workflow and why?
I am looking for an easy work environment where I can quickly test my models, exactly. And it won't be only me working on it, it's a team effort.
Since you are working as a team I would recommend to use sagemaker with custom docker images. That way you have complete freedom over your algorithm. The docker images are stored in ecr. Here you can upload many versions of the same image and tag them to keep control of the different versions(which you build from a git repo).
Sagemaker also gives the execution role to inside the docker image. So you still have full access to other aws resources (if the execution role has the right permissions)
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb
In my opinion this is a good example to start because it shows how sagemaker is interacting with your image.
Some notes on other solutions:
The problem of every other solution you posted is you want to build and execute on the same machine. Sure you can do this but keep in mind, that gpu instances are expensive and therefore you might only switch to the cloud when the code is ready to run.
Some other notes
Jupyter Notebooks in general are not made for collaborative programming. I think they want to change this with jupyter lab but this is still in development and sagemaker only use the notebook at the moment.
EC2 is cheaper as sagemaker but you have to do more work. Especially if you want to run your model as docker images. Also with sagemaker you can easily build an endpoint for model inference which would be even more complex to realize with ec2.
Cloud 9 I never used this service and but on first glance it seems good to develop on, but the question remains if you want to do this on a gpu machine. Because you're using ec2 as instance you have the same advantage/disadvantage.
One thing I'd like to call out first is SageMaker notebook is not the only IDE environment in which you can interact with other components of SageMaker such as training and hosting. In fact you can make API calls to SageMaker training/hosting through Cloud9 or any IDEs you've installed on EC2 or even your laptop, as long as you have AWS SDK or SageMaker Python SDK installed.
Regarding the choice of the IDE, it's really up to your particular needs. SageMaker notebook is Jupyter based (now also supports JupyterLab beta), ML focused, and fully managed. Hundreds of Python packages that are commonly used in ML, as well as Tensorflow, Keras, MxNet, SageMaker Python SDK, etc., are preinstalled and automatically maintained for you. It also integrates more closely with other components of SageMaker as one can imagine.
Cloud9 is a managed IDE too but it is for general purpose rather than ML specific. If you want to use Jupyter on cloud9 it requires extra work from your side. It does not preinstall and maintain the version of common ML/DL related packages like SageMaker notebook does.
Yes, I've heard all the stories about EC2 instances being unreliable and how you need to proactively prepare for that. I've also heard stories from others about how they have never had a problem, and their instances just run and run.
Today I had a strange thing happen. I've had an Linux instance running for a couple of months, as I've been preparing to launch an e-commerce site. I've been periodically taking snapshots. I have my images on S3. I have my code in a private github repo. All things considered, I've been doing a fairly good job of protecting myself against failure. Ironically, it was while I was doing even more in this regard today that I experienced something really strange.
Since I have these snapshots, I had assumed that the best thing to do if I needed to quickly spin up a new instance (whether due to a failed instance that wouldn't come back up, or if I just needed additional capacity) would be to take a snapshot and make a volume out of it, then make an image out of that volume, and then launch a new instance using that image.
For whatever reason, every time I've tried that lately, the new instance had a kernel panic during boot, so I decided to try a different approach. I right-clicked on my RUNNING INSTANCE, and chose "Create Image." That seemed like a reasonable shortcut. Then I went to that image and launched an instance.
At almost exactly the same time, my original instance rebooted. I didn't even see it happen. I only know it did from the system log. Is this just a wild coincidence? Or did I commit a silly mistake and accidentally screw up my instance?
Fortunately, I'm just getting this new thing off the ground, so the bit of downtime didn't kill me, and I was able to very quickly get things going again. But either I totally do not understand the "Create Image" feature from the instance list, or I got really unlucky today.
"Create image" takes the following actions:
Stop EC2 instance
Snapshot EBS volume
Start EC2 instance
Register EBS snapshot as an AMI
So, yes, this would look like a reboot because it is like a reboot.
Here's an article I wrote on the difference between stop/start and simple reboot: http://alestic.com/2011/09/ec2-reboot-stop-start
Your problem sounds a lot like my problem. After some searching this page helped me: http://www.raleche.com/node/138
"The problem turned out to be the kernel. Both when creating the AMI and the instance I selected default for the kernel image.
To resolve the problem, I recreated the AMI using the same kernel image as the original instance."