This is giving me a headache.
Here's what I've done so far
Created an EC2 Virtual Server Instance, and its running
Installed the AWS CLI
Installed Docker on my EC2 Virtual Server after I SSH'd into it
So looking at the docs it tells you how to build an image. Now comes my confusion.
Question 1: So am I right by assuming that one basically have an option to a) build an image off your host or b) pull an image created by others from Docker Hub?
Question 2: If I'm right about Question #1 then what am I building an image ** off of** if I am not pulling one from docker hub? with the AWS docs here?
Question 3: then I see a whole different route I can take, using Docker Compose, so I'd use that instead of all this above? This is so confusing.
EC2 Container Registry – Now Generally Available
So again, here, it tells you to install docker on the Host. Then immediately jumps into "create an image". Create an image off what, that host's OS? I don't get it, I guess that's what it means OR I can pull an image from Docker Hub and not go this route?
Same here, it's talking about creating a docker image, what off the Host?
Or..maybe I'm not understanding what "image" means but I assume going this route, instead of pulling a Docker image from Docker Hub that I'm creating an image off my EC2 virtual Instance?
A1: No. You can't build an image off your host.
You can create an new image according to your requirement like which Operating Sytem (Ubuntu, Fedora), Stack(LAMP, LEMP) and many other things.
Or you can pull an image which will be pre-configured with all the packages like Wordpress Stack image, Magento stack image, Bitnami image which you can pull from docker-hub.
A2: As I have mentioned earlier you can build an image of any operating system you want(Ubuntu, Fedora, Debian) but not off the host.
You just need to pull image from docker-hub. e.g docker pull ubuntu will pull mininmal image of Ubuntu-14.04. And if you need specific version of Ubuntu
like Ubuntu-12.04 version e.g docker pull ubuntu:12.04 will pull minimal image of Ubuntu-12.04
A3: Docker-compose is a tool for defining and running multi-container docker applications. docker-compose conatins a compose file
in which you can configure your application services.
And finally Amazon EC2 Container Registry is little bit different thing. The Idea is the same as docker but Amazon is providing
this as a EC2 Container Service with many other functionality which docker doesn't have right now.
Hope it hepls:-)
Related
I'm having difficulty to set this up correctly, and burning through AWS server time while I try to make it work. I have segmentation code that is heavily memory intensive that I'd like to temporarily spin up an AWS server with 192GB of ram. I understand that this is possible using docker, but the instructions on pycharm are non-existent with respect to the docker instructions necessary to tie it together (it references existing code as opposed to showing how to assemble it from scratch). What would the docker run command on the server look like to enable a connection to the 2375 port?
EDIT: I am using Pycharm Professional
UPD: Checking PyCharm options I found that there is an option to use Docker Machines. This seem to be exactly what you need. With Docker Machines you can make Docker spin up an EC2 instance for you with proper security out-of-the-box. Read official documentation on how to get started here and AWS driver options to learn how to set EC2 instance type, AMI, and other options here .
Original post:
To enable this feature you have to run Docker daemon with '-H' option:
sudo dockerd -H tcp://0.0.0.0:2375
You may read more on that in the Docker docs: https://docs.docker.com/engine/reference/commandline/dockerd/ .
Beware though, for EC2 you may also need to open that port using security group https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/authorizing-access-to-an-instance.html .
I also want to add that what you want to achieve isn't good from security perspective. Exposing docker socket like that is like an invitation for bad guys to throw a party at your EC2 instance. But since you mentioned that this is temporary...
I am newbie to docker and AWS. I wanted to have container running image for Maven and Java. I was able to refer https://github.com/carlossg/docker-maven/blob/master/jdk-8/Dockerfile and could create dockerfile for the same. Through terminal, I could see new container is created , with image of Java and Maven. Upto this point it is simple , took me while to figure out though.
(1) I think this is the way you can always have Maven plus Java image , and there is no other way with lot less files/ coding. Is it right? This is just for my information. The real question is the next one.
(2) If I get image of AWS Cli, once the container starts I can login to AWS using the credentials. I know how to do it using terminal. Not a big deal. If I want to have CI/CD pipeline, where do I provide the command - docker build -t <imageName> . and command to start container. Right now I use mac terminal, but not sure how it would play out in CI/CD. I did research on here but nothing conclusive. Does it go inside .yml file?
(3) How do I send the AWS credentials while building docker image? I do not want to put into dockerfile. How do you guys do it so its safe?
As announced the Swisscom logstash buildpack is not supported any longer.
The proposed solution is to push the default docker image.
I am trying to figure out the way to attach the curator configuration without "baking" it inside the docker image. Any ideas?
thanks
There are two articles in the support forum that discuss some aspects of your question here:
https://docs.developer.swisscom.com/service-offerings/logstash-docker.html
https://docs.developer.swisscom.com/service-offerings/kibana-docker.html
They do in fact recommend:
If you wish to use configuration files instead, you can fork the official Docker image and ADD your configuration files in your own Dockerfile.
I assume that is exactly what you did not want to do, but you can pass in most of the config via environment variables as far as I understand.
If you are ok with creating a separate Docker image, you could also host the config somewhere (let's say on S3) and then dynamically retrieve it on start-up of your Docker container.
You could also build the config setup into your deployment setup, although I haven't tried this with the docker build-pack, you can "stack" multiple build-packs in CloudFoundry and pre-load your configuration files into the virtual server as part of an initial build-pack step. There is more information on how to do that here: https://docs.cloudfoundry.org/buildpacks/use-multiple-buildpacks.html
I decided to try and use Google Cloud Datalab for a small project that I'm working on rather than a Jupyter Notebook in an Anaconda environment on an AWS instance.
How can I install a package (for example OpenCV) onto the Datalab VM so that I don't have to reinstall it every time I restart my VM? Why do the packages disappear after every restart but the updated notebooks remain persistent? Any help answering these questions and clarifying how the Datalab VM works would be very helpful.
The notebooks are stored in a docker volume mount that represents a location on the persistent disk that is maintained across restarts of the VM.
The packages you install however are stored in the running container and hence lost on each restart.
You could create a custom docker image and use that instead. On the datalab create command, see the --image-name argument.
Here is an example of a Dockerfile you'll want to use:
FROM gcr.io/cloud-datalab/datalab:latest
RUN pip install opencv
Note that you'll need build the docker image using this docker file, and push the image to Google Container Registry. My memory is a bit fuzzy on this, but it is possible this image needs to be marked as public.
Hope that helps!
I have a question about asterisk, I know that I can install asterisk on EC2, but my questions is:
Its possible install AsteriskNOW on Amazon EC2? if not, why? and where its the best possible server or solution for install this
thanks
AsteriskNow is a complete distribution based on CentOS available as an ISO file. There doesn't appear to be an EC2 AMI available for it so you would have to build an image yourself.
Here's an overview of the process for Oracle Linux which boils down to:
Install AsteriskNow onto a VirtualBox or VMWare instance locally.
Configure all the EC2 specifics (This is the fiddly bit)
Export that virtual machine as a VMDK.
Copy the VMDK to S3
Import the VMDK to an EBS volume and launch on Amazon EC2.
Before you export you will have to make sure AsteriskNow has a kernel that supports EC2. In CentOS this would be the Xen kernel but I don't know if Asterisk would supply one, which means compiling. The PV-GRUB docco also covers a lot of what can and can't be used on EC2. If it doesn't work out of the box it will take some Linux smarts to figure it all out.
It will probably take a number of exports/imports to get it running. Once you have it up on EC2 you can turn that instance into an AMI to quickly create clones in the future without going through the whole export/import process.
can you not just download the ISO directly?
ubuntu#ip-172-31-14-19:~/iso$
ubuntu#ip-172-31-14-19:~/iso$ wget -v https://downloads.asterisk.org/pub/telephony/asterisk-now/AsteriskNow-1013-current-64.iso
--2017-11-17 05:52:53-- https://downloads.asterisk.org/pub/telephony/asterisk-now/AsteriskNow-1013-current-64.iso
Resolving downloads.asterisk.org (downloads.asterisk.org)... 76.164.171.238, 2001:470:e0d4::ee
Connecting to downloads.asterisk.org (downloads.asterisk.org)|76.164.171.238|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1343909888 (1.3G) [application/x-iso9660-image]
Saving to: ‘AsteriskNow-1013-current-64.iso’
AsteriskNow-1013-curr 100%[======================>] 1.25G 1.79MB/s in 9m 54s
2017-11-17 06:02:48 (2.16 MB/s) - ‘AsteriskNow-1013-current-64.iso’ saved [1343909888/1343909888]
ubuntu#ip-172-31-14-19:~/iso$
https://downloads.asterisk.org/pub/telephony/asterisk-now/