Docker Private Registry: ping attempt failed - amazon-web-services

I'm trying to set up my private Docker Registry and I'm following the official documentation.
I have installed Docker and I'm able to run my registry on my server. But I want my registry to be more widely available.
My docker-server with the private registry is installed on an AWS-instance.
I have created my own certificate and key by using keytool:
docker run -d -p 5000:5000 --restart=always --name registry \
-v `pwd`/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
registry:2
I'm able to ping this instance by:
ping ec2-xx-xx-xx-xx.xx-west/east-1.compute.amazonaws.com
But pushing is not possible:
The push refers to a repository [ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/ubuntu] (len: 1)
unable to ping registry endpoint https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v0/
v2 ping attempt failed with error: Get https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v2/: dial tcp 10.x.x.x:5000: i/o timeout
v1 ping attempt failed with error: Get https://ec2-xx-xx-xx-xx.amazonaws.com:5000/v1/_ping: dial tcp 10.0.x.x:5000: i/o timeout
EDIT1:
After changing my aws-security group. Set port 5000 to TCP, the error changed:
unable to ping registry endpoint https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v0/
v2 ping attempt failed with error: Get https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v2/: dial tcp 10.0.x.x:5000: connection refused
v1 ping attempt failed with error: Get https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v1/_ping: dial tcp 10.0.x.x:5000: connection refused
How do I have to make my registry accessible for other aws-instances?
My docker logs are showing the following. They can't find my certificate.
level=fatal msg="open /certs/domain.crt: no such file or directory"
Do I have to put this certificate in my container itself? (and generate it with keytool by myself or using an existing)
EDIT2:
I've generated my own certificates using this documentation.
After generating the certificates I did restart my docker daemon. I did not perform the copy of domain.crt to ca.crt because the path didn't exist. Maybe I have to create it by myself?
new error:
unable to ping registry endpoint https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v0/
v2 ping attempt failed with error: Get https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v2/: dial tcp 10.0.x.x:5000: no route to host
v1 ping attempt failed with error: Get https://ec2-xx-xx-xx-xx.compute.amazonaws.com:5000/v1/_ping: dial tcp 10.0.x.x:5000: no route to host
But I still get the following in my docker logs:
level=fatal msg="open /certs/domain.crt: no such file or directory"
After trying to perform a push, there is created a new /certs folder into my existing certsfolder
EDIT3:
After finding the right directory for my certificate (/home/centos/certs/certs/*.). I get the following error:
level=fatal msg="open /certs/domain.crt: permission denied
Even if I perform a chmod -R 777 and chown -R root:root

You will need to place the certificate in this directory.
/etc/docker/certs.d/<your-domain-name>:5000/ca.crt

Related

SSH reverse port forward on EC2 aws instance

I used to have an ssh reverse port forwarding from my local computer to a remote EC2 AWS server on port 9999. (9999 for both machines.)
It used to work, but I created a new instance, and now it doesn't anymore. (Half working.) I'm not sure what I did to make it work back then... (Or something was changed.)
I have a process running on my computer on port 9999 and I want it to listen to the port 9999 of my EC2.
On my computer, curl "127.0.0.1:9999" is working.
But I want the code curl "ec2-xx-xx-xx-xx-xx.compute.amazonaws.com:9999" to work, for now it doesn't, giving me the error curl: (7) Failed to connect to ec2-xx-xx-xx-xx-xx.compute.amazonaws.com port 9999 after 59 ms: Connection refused
EC2 Security group is set to open 9999 on TCP for 0.0.0.0/0.
I create the forwarded port with the command :
ssh -R 9999:localhost:9999 -i "/home/example/XXX.pem" ubuntu#ec2-xx-xx-xx-xx-xx.compute.amazonaws.com
The connection ssh is established without errors.
Inside this ssh session I can even do curl "127.0.0.1:9999" inside and IT IS WORKING. Reaching my local computer.
But the request from the web isn't... (curl "ec2-xx-xx-xx-xx-xx.compute.amazonaws.com:9999" doesn't work...)
The path is good, if I install apache2 on port 80 curl "ec2-xx-xx-xx-xx-xx.compute.amazonaws.com:80" is working. (port 80 is added the same way to the security group)
I did sudo ufw disable, same problem.
Do you have an idea what I'm missing ?
EDIT : On the ssh -R forward session on the EC2 :
ubuntu#awsserver:~$ php -S 0.0.0.0:9999 -t .
[Wed Dec 14 16:35:11 2022] Failed to listen on 0.0.0.0:9999 (reason: Address already in use)
BUT, if I open a normal ssh session, I can run php -S 0.0.0.0:9999 -t ., the code curl "ec2-xx-xx-xx-xx-xx.compute.amazonaws.com:9999" is working everywhere as expected.
So... it is telling me that the port is already used (By the ssh -R command), but is closed when I try to connect to it... I don't get it.
The answer wasn't EC2/AWS related.
It's a security feature from SSH that I had to disable : GatewayPorts yes

Attempting to ssh tunnel to another server within the vpc to a specific port to access my api

This is an odd scenario. Essentially, within a vpc, I am attempting to create an ssh tunnel from server A to server B in which server B hosts the api at port 9000, and server A wants to be able to reverse proxy to port say 5000 which should pass the query to server B's port 9000, and return data. I have been tearing my hair out. I currently have gotten this far:
ssh 3000:localhost3000 -vvv -N -i rsa.pem serverB#serverBIP
after which I have attempted to access the port 22 on server A using a curl request but I got a response -- curl: (1) Received HTTP/0.9 when not allowed
I also tried specifying a port
ssh -vvv -N -i rsa.pem serverB#serverBIP -p3000
which quits on me entirely with the response: ssh: connect to host serverBIp port 3000: No route to host
Finally, I tried
ssh -vvv -N -i workstation_pem.pem 3000:localhost:3000 serverBUser#serverBID
which results in Could not resolve hostname 3000:localhost:3000: Name or service not known
Please advise, I am not sure what I am doing wrong. I feel like this should be simple, but I am struggling to get it to work - a simple tunnel from one instance to another at a port to port on server B where serverB has a gunicorn server running
You would need to login to ServerA and then run this command:
ssh -i key.pem -N -L 5000:serverBIP:9000 serverBUser#serverBIP
This tells the computer on which it is run (which is ServerA) to listen on port 5000 and send any incoming requests to port 9000 on ServerB.
Detailed explanation: explainshell.com - ssh -i key.pem -N -L 5000:serverBIP:9000 serverBUser#serverBIP
See also: SSH/OpenSSH/PortForwarding - Community Help Wiki

How do deal with conan server connection refused?

I have a docker container which runs as Conan server. I am trying to upload some Conan packages from another docker container to conan server. After entering username and passwd I am getting following error
Error uploading file: conanmanifest.txt,
'HTTPConnectionPool(host='localhost', port=9300): Max retries exceeded with url: /v1/files/hello/0.1/demo/testing/0/export/conanmanifest.txt?signature=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyZXNvdXJjZV9wYXRoIjoiaGVsbG8vMC4xL2RlbW8vdGVzdGluZy8wL2V4cG9ydC9jb25hbm1hbmlmZXN0LnR4dCIsInVzZXJuYW1lIjoiZGVtbyIsImZpbGVzaXplIjo1OCwiZXhwIjoxNTg1NDQyMjYyfQ.7sHncjZ7J8gV5HENMqCIwLe7b483QfrGJ2PVyolvjC4
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f99c84d2e50>:
Failed to establish a new connection: [Errno 111] Connection refused'))'
ERROR: hello/0.1#demo/testing: Upload recipe to 'my_server' failed: Execute upload again to retry upload the failed files: conanfile.py, conanmanifest.txt. [Remote: my_server]
ERROR: Errors uploading some packages
I ran the following command
docker run -t -p 9300:9999 --name mycont my_conan
then edited server.conf file accordingly
port:9300
public_port:9999
host_name: containerIP
From now on I expect to reach by curl http://localhost:9999 but getting
Failed to connect to localhost port 9999: Connection refused
PS: From my host to the server, it works perfectly fine. Bu the error is appearing when i want to upload from container to container
I solve it by copying a custom config file to docker container. Otherwise I could not override default config file.
1. Write custom config file
2. COPY or ADD to image (in Dockerfile)
3. mv custom.conf /path/to/directory/ (from entrypoint)

Ansible deployment to windows host behind bastion

I am currently successfully using Ansible to run tasks on hosts that are in a private subnet in AWS, which the below group_vars is setting up:
ansible_ssh_common_args: '-o ProxyCommand="ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -W %h:%p -q ec2-user#bastionhost#example.com"'
This is working fine.
For Windows instances not in a private subnet the following group_vars works:
---
ansible_user: "AnsibleUser"
ansible_password: "Password"
ansible_port: 5986
ansible_connection: winrm
ansible_winrm_server_cert_validation: ignore
Now, trying to get Ansible to deploy to a Windows server behind the bastion by just using the ProxyCommand won't work - which I understand.
I believe though that there is a new protocol/module I can use called psrp.
I imagine that my group_vars for my Windows hosts needs to change to something like this:
---
ansible_user: "AnsibleUser"
ansible_password: "Password"
ansible_port: 5986
ansible_connection: psrp
ansible_psrp_cert_validation: ignore
If I run with just the above changes against instances that are publicly available (and not trying to connect via a bastion), my task seems to work fine:
Using module file /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/ansible/modules/windows/win_shell.ps1
<10.100.11.14> ESTABLISH PSRP CONNECTION FOR USER: Administrator ON PORT 5986 TO 10.100.11.14
PSRP: EXEC (via pipeline wrapper)
I know there must be more changes before I can try this on a windows server behind a bastion, but ran it anyway to see what errors I get to give me clues on what to do next. Here is the result when running this on an instance behind a bastion server:
Using module file /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/ansible/modules/windows/setup.ps1
<10.100.11.14> ESTABLISH PSRP CONNECTION FOR USER: Administrator ON PORT 5986 TO 10.100.11.14
The full traceback is:
.
.
.
.
ConnectTimeout: HTTPSConnectionPool(host='10.100.11.14', port=5986): Max retries exceeded with url: /wsman (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x110bbfbd0>, 'Connection to 10.100.11.14 timed out. (connect timeout=30)'))
It seems like Ansible is ignoring my group_vars for the ProxyCommand - which I'm not sure if that's expected.
I'm also not sure on what the next steps are to enable Ansible to deploy to Windows servers behind a bastion.
What config am I missing?
The doc says, the ansible_ssh_common_args setting is appended to sftp, scp, and ssh commands. So it sounds normal to me that is not taking into account when using winrm or psrp ansible_connection.
As explained in the link provided by Pouyan in the comments, ansible_psrp_proxy variable will be used to provide proxy information.
ansible_connection: psrp
ansible_psrp_proxy=socks5h://localhost:1234
More info on the creation of the socks proxy can be found on: https://www.bloggingforlogging.com/2018/10/14/windows-host-through-ssh-bastion-on-ansible/

cf create-service-broker fails with connection refused

I'm experimenting with CF in my local bosh-lite setup.
The apps that I deploy into if work well. I am now trying to follow the steps here
https://github.com/cf-platform-eng/cf-community-workshop/blob/master/demos/service-broker-lab.adoc
to try out the custom service broker setup.
The https://github.com/mstine/haash-broker application starts and is running fine:
$ cf apps
name requested state instances memory disk urls
haash-broker started 1/1 768M 1G haash-broker.vbox.mojito, haash-broker.192.168.50.6.xip.io
I can access it from my host machine browser well:
http://haash-broker.192.168.50.6.xip.io/v2/catalog
But when I execute the
cf create-service-broker haash-broker warreng natedogg http://haash-broker.192.168.50.6.xip.io
I get
$ cf create-service-broker haash-broker warreng natedogg http://haash-broker.192.168.50.6.xip.io
Creating service broker haash-broker as admin...
FAILED
Server error, status code: 502, error code: 10001, message: The service broker could not be reached: http://haash-broker.192.168.50.6.xip.io/v2/catalog
When I log in into the CC VM:
$ bosh -e vbox -f cf ssh api/eb4cec99-bab1-4513-a980-fb92775ac2d8
I can ping the hostname:
api/eb4cec99-bab1-4513-a980-fb92775ac2d8:~$ sudo ping haash-broker.192.168.50.6.xip.io
PING haash-broker.192.168.50.6.xip.io (192.168.50.6) 56(84) bytes of data.
64 bytes from 192.168.50.6: icmp_seq=1 ttl=64 time=0.080 ms
But wget connection gets refused:
api/eb4cec99-bab1-4513-a980-fb92775ac2d8:~$ wget http://warreng:natedogg#haash-broker.192.168.50.6.xip.io/v2/catalog
--2018-04-06 04:19:05-- http://warreng:*password*#haash-broker.192.168.50.6.xip.io/v2/catalog
Resolving haash-broker.192.168.50.6.xip.io (haash-broker.192.168.50.6.xip.io)... 192.168.50.6
Connecting to haash-broker.192.168.50.6.xip.io (haash-broker.192.168.50.6.xip.io)|192.168.50.6|:80... failed: Connection refused.
The firewall permits everything on that VM (sudo iptables -L).
The hostname gets resolved properly. The ping works and the 80 port is open on the target IP, since I can reach it from my host browser.
How can that be that the wget doesn't work in such situation?
This also seems to be the reason for me failing to create a service broker cf create-service-broker
UPDATE
I've managed to to execute the cf create-service-broker command with URL of an nginx reverse proxy running outside of my bosh-lite environment. The proxy redirects to the same initial URL http://haash-broker.192.168.50.6.xip.io
and the command succeeds in this way.
But the subsequent
cf create-service-broker haash-broker warreng natedogg http://haash-broker.192.168.50.1.xip.io:9999
cf enable-service-access haash
cf create-service HaaSh basic my-hash
(where haash-broker.192.168.50.1.xip.io:9999 is my nginx proxy) fails with
Server error, status code: 502, error code: 10001, message: The service broker rejected the request to http://haash-broker.192.168.50.1.xip.io:9999/v2/service_instances/4ef19154-d238-4cb3-8003-803fba53af3f?accepts_incomplete=true. Status Code: 400 Bad Request, Body: {"timestamp":1523008856993,"error":"Bad Request","status":400,"message":""}
I can see in both nginx and broker app logs that the the request reaches the broker and it answers with 400.
Debugging now why.
Can you post the result of --server-response option used with wget? Also what happens when you try to curl the broker?
Broker requires credentials, but it is interesting if it responds with 401 or 500 on the first request that wget makes without credentials.