Connect a docker node running on a separate ec2 host to Jenkins - amazon-web-services

I have 2 aws ec2 instances. One instance is running Jenkins, the other is running Docker. I am trying to connect the container running on the Docker host to Jenkins as a node.
To start the container on the Docker host I ran the following:
sudo dockerd -H tcp://127.0.0.1:2376 -H unix:///var/run/docker.sock
In the cloud settings (under jenkins/configure)
Docker Host URI:
tcp://IP-ADDRESS-OF-EC2-DOCKER-HOST:2376
Docker Hostname or IP address: IP-ADDRESS-OF-EC2-DOCKER-HOST
Dockerfile:
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install openjdk-8-jdk -y
RUN mkdir -p /home/jenkins
EXPOSE 22
ec2 Docker host security open incoming ports: 2375, 2376, 4243, 22
When I hit the "Test Connection" button, I get:
Connection refused: /IP-ADDRESS-OF-EC2-DOCKER-HOST:2376
java.net.ConnectException: Connection refused
Caused: io.netty.channel.AbstractChannel$AnnotatedConnectException:
Connection refused: /IP-ADDRESS-OF-EC2-DOCKER-HOST:2376
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:632)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
On another stackoverflow post, someone recommended hitting the "Apply" button first, but that doesn't work because I get an error popup stactrace stating a java.lang.NullPointerException.

Try giving IP-ADDRESS-OF-EC2-DOCKER-HOST instead of 127.0.0.1 in below CMD
sudo dockerd -H tcp://127.0.0.1:2376 -H unix:///var/run/docker.sock
Test port is open from docker to Jenkins host using
telnet docker_host_ ip 2376

Related

Accessing Two containers on from browser

I have launch EC2 ubuntu instance and security group for this is instance allows 22,80,443 ports from 0.0.0.0/0.
Now i have installed docker on this EC2 instance.Then i have created an apache2 container and also mapped the port to access from browser using below command
sudo docker run -p 80:80 -t -i ubuntu /bin/bash
Then i create an lampstack conatiner and tried to map port using below command
sudo docker run -p 443:443 -t -i linode/lamp /bin/bash
Now docker ps gives me below
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS Name
d0751e67fd69 linode/lamp "/bin/bash" 4 min Up 4 0.0.0.0:443>443/tcp
affectionate_hamilton
0fb4e13a272a ubuntu "/bin/bash" 11 minutes 0.0.0.0:80->80/tcp
vigorous_robinson
When i take the public ip of my EC2 machine and put in browser i can see the apache page but how can i assess my Lampstack page ?
Please correct me if i have done port mapping incorrectly
You only need the LAMP container and in that one you should map the port 80:
sudo docker run -d --name lamp -t -p 80:80 linode/lamp top
Check that the container is up and running:
sudo docker ps --filter name=lamp
Now start the services:
sudo docker exec -ti lamp service apache2 start
sudo docker exec -ti lamp service mysql start
Test your setup from host:
curl http://localhost
If you want to test a connection from a different container you can start a separate ubuntu container that links to your original container "lamp":
docker run -ti --rm --link lamp --name ubuntu-box ubuntu bash
Inside the container install curl and test your connection:
apt update && apt-get install curl -y
curl http://lamp

Connection Refused on AWS Instance?

I have a linux AWS instance. I am running the following script on it:
#!/usr/bin/env bash
#This script installs java, sbt and the application
#Run this script on a new EC2 instance as the user-data script, which is run by `root` on machine start-up.
sudo yum update -y
sudo yum install -y docker
sudo service docker start
docker run repo/carrie
Everything installs and I get the below message in the logs:
REST interface bound to /0.0.0.0:8080
However when I try to actually access the port like so:
curl 0.0.0.0/8080
I get the below message:
Failed to connect to 0.0.0.0 port 8080: Connection refused
I have tried editing the inbound rules so that 8080 is open but it doesn't seem to work. Maybe because I'm editing the rules after the instance has already launched?
You have to publish the container's port to the host in the docker run command
$ docker run --help
...
-p, --publish list Publish a container's port(s) to the host
...
The last line of your script should look like this if the process in the container listens on port 80:
docker run -p 8080:80 repo/carrie
The container gets its own interface, hence host's 0.0.0.0 is not applicable.
Tell docker to bind container port 8080 out to the host:
docker run -p 8080:8080 repo/carrie

Error connecting to RDS Postgreql DB from inside Docker container

I've got an app running Flask_sqlalchemy in a Docker container.
The container wasn't running properly so I dived in and tried running the application and get the following error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
Is the server running on host "DBNAME.XXXXXXXXXX.eu-west-1.rds.amazonaws.com" (000.000.000.000) and accepting
TCP/IP connections on port 5432?
The application works fine outside the container, and I can't work out what's going on.
Could it be something to do with the AWS-RDS security groups? They're currently configured to only accept inbound connections from our office where development takes place.
EDIT:
This is my Dockerfile:
FROM ubuntu:latest
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential libpq-dev python-shapely
COPY . /src
WORKDIR /src
RUN pip install -r requirements.txt
EXPOSE 5000
CMD ["python", "application.py"]
And this is the Docker Run command I'm doing:
docker run -d -p 5000:5000 container_name
Thanks
I had exactly the same problem ;) but solved it by ensuring the following:
The AWS ELB environment configuration is Generic Docker (not preconfigured for Python).
The environment is created inside of a VPC that contains the RDS instance.
The RDS instance is listening to the right port (PostgreSQL:5432) from the VPC security group (this should already be the case).

Running iPython Notebook in Docker container on Amazon EC2 instance

How do I run and access iPython Notebook (in Docker on EC2) from the browser?
This is what I tried:
From EC2 Quick Start menu, selected Amazon Linux AMI 2015.03 on t2.micro instance.
Everything left as default, except 3 rules created for "Configure Security Group":
Type: "SSH"; Protocol: "TCP"; Port Range: "22"; Source: "Anywhere";
Type: "HTTPS"; Protocol: "TCP"; Port Range: "443"; Source: "Anywhere";
Type: "Custom TCP Rule"; Protocol: "TCP"; Port Range: "8888"; Source: "Anywhere";
After SSH'ing to instance:
$ sudo yum install -y docker ; sudo service docker start
$ sudo docker pull continuumio/miniconda # Anaconda includes iPython Notebook
$ sudo docker run -it -p 8888:8888 continuumio/miniconda ipython notebook
Then launching browser to https://ec2-xx-x-x-xxx.compute-1.amazonaws.com:8888 didn't work.
I wouldn't be too comfortable opening 443 and 8888 on the Internet for my EC2 instances. My common setup is Anaconda on an Ubuntu box.
I usually ssh port forward my ipython notebook sessions to my localhost on my macbook with this ssh command:
ssh -i myPrivateSSHKey.pem ubuntu#54.1.2.3 -L 8888:localhost:8888
Then I open Chrome and request URL:
http://127.0.0.1:8888

synching postgresql database between two EC2 instances

When running (fictitious ip addresses) on an EC2 instance that is intended to be a slave postgresql database:
ubuntu#ip-127-31-71-50:/var/lib/postgresql/9.3$ sudo -u postgres pg_basebackup -h 54.419.311.274 -D /var/lib/postgresql/9.3/main -U rep -v -P -x
The response is:
pg_basebackup: could not connect to server: could not connect to server: Connection refused
Is the server running on host "54.419.311.274" and accepting
TCP/IP connections on port 5432?
Having run this command on non_AWS instances successfully, I wonder what configuration is required under AWS to make this run properly.
User postgres exists on the master as
sudo su postgres
allows editing postgre via its main user.
postgresql.conf on master EC2 instance has
listen_addresses = 'localhost,54.419.311.274,54.86.79.188'
What am I missing?