I'm trying to generate a certificate in my local (MacBook) environment which I can package in my Docker image and deploy into my AWS environment via Kubernetes.
I've scoured sources online for a solution to this but I'm unable to find the details I need.
From my macbook:
sudo certbot certonly -a standalone -d my.domain
Gives me this error:
Failed authorization procedure. my.domain (http-01): urn:acme:error:unauthorized ::
The client lacks sufficient authorization :: Invalid response from
http://my.domain/.well-known/acme-challenge/T8jtGQswRuMgHKIhGvb-
QD73kytTZnHfH5mK5lEZUJc: "{"timestamp":"2018-04-22T22:33:40.845+0000","status":404,
"error":"Not Found","message":"No message available","path":"/.well-kno"
Clearly, I need a way to prove that I own my own domain. How can I do this locally?
In order to verify ownership of the domain from your macbook you have these two options as stated in the certbot docs:
Use a DNS plugin - https://certbot.eff.org/docs/using.html#dns-plugins
Use the manual method - https://certbot.eff.org/docs/using.html#manual
While the standalone option does not require web server software it does require that it is run on the target web server - it is therefore not what you need to do and will result in the failure reported in your question.
Related
I currently have an AWS server set up with docker to run the Keycloak docker container. For SSL/TLS, there is an AWS loadbalancer configured to point https/443 traffic to the container and have it receive it over 8080, terminating the encryption connection on said load balancer.
When creating the container with the following command, I am able to browse to and log into the keycloak service by browsing to the server's IP address.
docker run --name keycloak -v keybase-storage -p 8080:8080 -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=TempAdminPassword jboss/keycloak However if I try to log into the server by browsing to the URL, I am redirected to the url http://default-host:8080/auth/admin/ and the browser showing a connection error page.
When trying to find a solution to this, I found how to pass java options to the container when it is first run, and using the resources from this page I used the following command to start the container(URL replaced for privacy concerns)
docker run --name keycloak -v keybase-storage -p 8080:8080 -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=TempAdminPassword -e JAVA_OPTS_APPEND="-Dkeycloak.frontendUrl=https://sso.IntendedURL.com" jboss/keycloak However this yields the same results when trying to browse to the page.
The main clue I have to go off of right now is this line near the end of the previously shown docker run command, which reads as follows:
19:23:00,039 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 67) WFLYUT0021: Registered web context: '/auth' for server 'default-server'
What I believe I need to do now is to either change the config of the docker container after it has been created(have been unable to edit files using docker exec, so this is less likely) or to pass a java option into the run command when the container is first started.
Please let me know if you have any questions or if I can provide any other information.
Thank you.
Environment information:
Operating system
Amazon Linux 2
Docker version
19.03.13-ce, build 4484c46
Keycloak version
12.0.1(WildFly Core 13.0.3.Final)
Scenario
I have AWS setup for a unifi controller, I've been to access it with https://myserverip:8443, I bypass "This connection is note sucured" and use the controller normally
Now, I need to install and SSL certificate to get the hotspot payment system going.
I have a FQDN with GoDaddy so I created a subdomain unifi.mydomain.com, that points to the elastic IP, I log on with https://unifi.mydomain.com:8443
I bought the SSL certificate from GoDaddy, added the subdomain to that certificate.
I log on my AWS with SSH, generate my csr with the following command
cd /usr/lib/unifi
sudo java -jar lib/ace.jar new_cert unifi.mydomain.dom “My Company Name” City State CC*
Then I do
cd var/lib/unifi
more unifi_certificate.csr.pem
Once I get that I copy and paste it on GoDaddy, download the cert files, go back to AWS copy the files with filezilla to /usr/lib/unifi
Then I run the following command
sudo java -jar lib/ace.jar import_cert unifi_mydomain_com.crt bundlecert.crt
They import correctly, restart unifi service and reboot EC2
When I got to any of the above address I get the following
This site can’t provide a secure connection ERR_SSL_PROTOCOL_ERROR
I've tried different browsers, incognito mode, vpn, etc, I believe it's just a matter of SSL or my server
Check your system.properties which sits in /var/lib/unifi/ open the file with vim or your text editor of choice.
Have a look at your HTTPS options, the important ones are the ciphers and protocols.
The Protocols you need are TLSv1 and potentially SSLv2Hello there should be no other SSL protocols in there.
The Ciphers you ideally want are TLS, so for example TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA
If you are having issues throw them all in, CAUTION! only use this in a demo /test environment.
unifi.https.ciphers=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,SSL_RSA_WITH_RC4_128_SHA
Remember once you have edited the system.properties you need to restart the controller.
sudo service unifi restart
Lots of help on the Unifi page
UniFi - SSL Certificate Error
UniFi - Explaining the config.properties File
UniFi - system.properties File Explanation
when running the following command:
cmd /c C:\sonar-runner-2.4\bin\sonar-runner.bat
(sonar runner is installed on the build machine)
i get the following errors:
ERROR: Sonar server 'http://localhost:9000' can not be reached
ERROR: Error during Sonar runner execution
ERROR: java.net.ConnectException: Connection refused: connect
ERROR: Caused by: Connection refused: connect
what can cause these errors?
Hi dinesh,
this is my sonar-runner.properties file:
sonar.projectKey=NDM
sonar.projectName=NDM
sonar.projectVersion=1.0
sonar.visualstudio.solution=NDM.sln
#sonar.sourceEncoding=UTF-8
sonar.web.host:sonarqube
sonar.web.port=9000
# Enable the Visual Studio bootstrapper
sonar.visualstudio.enable=true
# Unit Test Results
sonar.cs.vstest.reportsPaths=TestResults/*.trx
# Required only when using SonarQube < 4.2
sonar.language=cs
sonar.sources=.
As you can see i set the sonar.web.host:sonarqube
sonar.web.port=9000 but when i run sonar-runner.bat i still get the
ERROR: Sonar server 'http://localhost:9000' can not be reached - why is it still looking for localhost:9000
and not sonarqube:9000 as i set?
i saw that in the log of sonar-runner.bat there the following line:
INFO: Work directory: D:\sTFS\26091\Sources\NDM\Source..sonar
while my solution is in D:\sTFS\26091\Sources\NDM\Source\
could this be the problem?
thanks,
Guy
If you use SonarScanner CLI with Docker, you may have this error because the SonarScanner container can not access to the Sonar UI container.
Note that you will have the same error with a simple curl from another container:
docker run --rm byrnedo/alpine-curl 127.0.0.1:9000
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused
The solution is to connect the SonarScanner container to the same docker network of your sonar instance, for instance with --network=host:
docker run --network=host -e SONAR_HOST_URL='http://127.0.0.1:9000' --user="$(id -u):$(id -g)" -v "$PWD:/usr/src" sonarsource/sonar-scanner-cli
(other parameters of this command comes from the SonarScanner CLI documentation)
I got the same issue, and I changed to IP and it working well
Go to System References --> Network --> Advanced --> Open TCP/IP tabs --> copy the IPv4 Address.
change that IP instead localhost
Hope this can help
You should configure the sonar-runner to use your existing SonarQube server. To do so, you need to update its conf/sonar-runner.properties file and specify the SonarQube server URL, username, password, and JDBC URL as well. See https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner for details.
If you don't yet have an up and running SonarQube server, then you can launch one locally (with the default configuration) - it will bind to http://localhost:9000 and work with the default sonar-runner configuration. See https://docs.sonarqube.org/latest/setup/get-started-2-minutes/ for details on how to get started with the SonarQube server.
For others who ran into this issue in a project that is not using a sonar-runners.property file, you may find (as I did) that you need to tweak your pom.xml file, adding a sonar.host.url property.
For example, I needed to add the following line under the 'properties' element:
<sonar.host.url>https://sonar.my-internal-company-domain.net</sonar.host.url>
Where the url points to our internal sonar deployment.
For me the issue was that the maven sonar plugin was using proxy servers defined in the maven settings.xml. I was trying to access the sonarque on another (not localhost alias) and so it was trying to use the proxy server to access it. Just added my alias to nonProxyHosts in settings.xml and it is working now. I did not face this issue in maven sonar plugin 3.2, only after i upgraded it.
<proxy>
<id>proxy_id</id>
<active>true</active>
<protocol>http</protocol>
<host>your-proxy-host/host>
<port>your-proxy-host</port>
<nonProxyHosts>localhost|127.0.*|other-non-proxy-hosts</nonProxyHosts>
</proxy>enter code here
The issue occurred with me in a different way a little a while ago,
I had a docker container running normally in the main network of my host machine accessible via the browser on the normal localhost:9000. But whenever the scanner wants to connect to the server it couldn't despite being on the same network of the host.
I made sure they are, because on the docker run command I mentioned --network=bridge
So the trick was that I pointed to the actual local ip of mine instead of just writing localhost
you can know the ip of your machine by typing ipconfig on windows or ifconfig on linux
so on the scan docker run command I have pointed to the server like that -Dsonar.host.url=http://192.168.1.2:9000 where 192.168.1.2 is my local host address
That was my final docker commands to run the Server:
docker run -d --name sonarqube \
--network=bridge \
-p 9000:9000 \
-e SONAR_JDBC_USERNAME=<db username> \
-e SONAR_JDBC_PASSWORD=<db password>\
-v sonarqube_data:/opt/sonarqube/data \
-v sonarqube_extensions:/opt/sonarqube/extensions \
-v sonarqube_logs:/opt/sonarqube/logs \
sonarqube:community
and that's for the Scanner:
docker run \
--network=bridge \
-v "<local path of the project to scan>:/usr/src" sonarsource/sonar-scanner-cli \
-Dsonar.projectKey=<project key> \
-Dsonar.sources=. \
-Dsonar.host.url=http://<local-ip>:9000 \
-Dsonar.login=<token>
In the config file there is a colon instead of an equal sign after the sonar.web.host.
Is:
sonar.web.host:sonarqube
Should be
sonar.web.host=sonarqube
In sonar.properties file in conf folder I had hardcoaded ip of my machine where sobarqube was installed in property sonar.web.host=10.9 235.22 I commented this and it started working for me.
Please check if postgres(or any other database service) is running properly.
When you allow the 9000 port to firewall on your desired operating System the following error "ERROR: Sonar server 'http://localhost:9000' can not be reached" will remove successfully.In ubuntu it is just like as by typing the following command in terminal "sudo ufw allow 9000/tcp" this error will removed from the Jenkins server by clicking on build now in jenkins.
I'm using boot2docker on OSX 10.10 to try to run the WSO2 governance registry. I cannot reach the app from the host machine (OSX).
Here's my Dockerfile:
FROM dockerfile/java:openjdk-7-jdk
MAINTAINER Andrew Matthews
COPY wso2greg-4.6.0.zip /opt/
RUN unzip /opt/wso2greg-4.6.0.zip -d /opt && \
rm /opt/wso2greg-4.6.0.zip
EXPOSE 9443
CMD ["/opt/wso2greg-4.6.0/bin/wso2server.sh"]
This follows a pattern used by others on docker.io for other wso2 apps.
I built it with:
docker build -t="usmsnp/wso2greg" .
and ran it with
docker run -i -t -P usmsnp/wso2greg
everything seems to proceed nicely - I get as far as the wso2 announcement:
Mgt Console URL : https://172.17.0.27:9443/carbon/
I have the boot2docker vm IP address aliased as dockerhost in my /etc/hosts file, and when I navigate to https://dockerhost:9443/carbon/ using curl I get connection refused.
Any ideas?
UPDATE: 2014-11-28
When I deploy the image to AWS using tutum, it works. So, mission accomplished, I suppose. But I'm still confused about why it doesn't work. I understand that boot2docker requires a different IP address, but I used that.
When you use docker run -P you're telling Docker to allocate a random port on the Docker daemon host - not to use 9443.
So in this case, you'll need to run docker ports <containerid> to find out what port its been mapped to.
alternativly, you need to use docker run -p 9443:9443 ... to tell Docker to map the external 9443 to the container's 9443
I have to do distributed testing using JMeter. The objective is to have multiple remote servers in AWS controlled by one local server send a file download request to another server in AWS.
How can I set up the different servers in AWS?
How can I connect to them remotely?
Can someone provide some step by step instructions on how to do it?
I have tried several things but keep running into connectivity issues across networks.
We had a similar task and we ran into a bunch of issues as well. Here are the details of the whole process and what we did to resolve the issues we encountered. Hope it helps.
We needed to send requests from 5 servers located in various regions of the world. So we launched 5 micro instances in AWS, each in a different region. We chose the regions to be as geographically apart as possible.
Remote (server) JMeters config
Here is how we set up each instance.
Installed java:
$ sudo apt-get update
$ sudo apt-get install default-jre
Installed JMeter:
$ mkdir jmeter
$ cd jmeter;
$ wget ftp://apache.mirrors.pair.com//jmeter/binaries/apache-jmeter-2.9.tgz
$ gunzip apache-jmeter-2.9.tgz;tar xvf apache-jmeter-2.9.tar
Edited the jmeter.properties file in the /bin folder of the JMeter installation and uncomment the line containing the server.rmi.localport setting. We changed the port to 50000.
server.rmi.localport=50000
Started JMeter server. Make sure the address and the port the server reports listening to are correct.
$ cd ~/jmeter/apache-jmeter-2.9/bin
$ vi jmeter-server
Local (client) JMeter config
Then we set up JMeter to run tests remotely on these instances on our local client machine:
Ensured to use the same version of JMeter as was running on the servers. Installed Java and JMeter as described above.
Enabled remote testing by editing the jmeter.properties file that can be found in the bin folder of the JMeter installation. The parameter remote_hosts needed to be set with the public DNS of the remote servers we were connecting to.
remote_hosts=54.x.x.x,54.x.x.x,54.x.x.x,54.x.x.x,54.x.x.x
We were now able to tell our client JMeter instance to run tests on any or all of our specified remote servers.
Issues and resolutions
Here are the issues we encountered and how we resolved them:
The client failed with:
ERROR - jmeter.engine.ClientJMeterEngine: java.rmi.ConnectException: Connection - refused to host: 127.0.0.1
It was due to the server host returning the private IP address as its address because of Amazon NAT.
We fixed this by setting the parameter RMI_HOST_DEF that the /usr/local/jmeter/bin/jmeter-server script includes in starting the server:
RMI_HOST_DEF=-Djava.rmi.server.hostname=54.xx.xx.xx
Now, the AWS instance returned the server’s external IP, and we could start the test.
When the server node attempted to return the result and tried to connect to the client, the server tried to connect to the external IP address of my local machine. But it threw a connection refused error:
2013/05/16 12:23:37 ERROR - jmeter.samplers.RemoteListenerWrapper: testStarted(host) java.rmi.ConnectException: Connection refused to host: xxx.xxx.xxx.xx;
We resolved this issue by setting up reverse tunnels at the client side.
First, we edited the jmeter.properties file in the /bin folder of the JMeter installation and uncommented the line containing the client.rmi.localport setting. We changed the port to 60000:
client.rmi.localport=60000
Then we connected to each of the servers using SSH, and setup a reverse tunnel to port 60000 on the client.
$ ssh -i ~/.ssh/54-x-x-x.us-east.pem -R 60000:localhost:60000 ubuntu#54.x.x.x
We kept each of these sessions open, as the JMeter server needs to be able to deliver the test results to the client.
Then we set up the JVM_ARGS environment variable on the client, in the jmeter.sh file in the /bin folder:
export JVM_ARGS="-Djava.rmi.server.hostname=localhost"
By doing this, JMeter will tell the servers to connect to localhost:60000 for delivering their results. This ends up being tunneled back to the client.
The SSH connections to the servers kept dropping after staying idle for a little bit. To prevent that from happening, we added a parameter to each of the SSH tunnel set up directing the client to wait 60 seconds before sending a null packet to the server to keep the connection alive:
$ ssh -i ~/.ssh/54-x-x-x.us-east.pem -o ServerAliveInterval=60 -R 60000:localhost:60000 ubuntu#54.x.x.x
(.ssh/config version of all required SSH settings:
Host 54.x.x.x
HostName 54.x.x.x
Port 22
User ubuntu
ServerAliveInterval 60
RemoteForward 127.0.0.1:60000 127.0.0.1:60000
IdentityFile ~/.ssh/54-x-x-x.us-east.pem
IdentitiesOnly yes
Just use ssh 54.x.x.x after setting this up.
)
I just went though this on openstack and found the same issues... no idea why the jmeter remoting documentation only covers half the required steps. You can do it without tunnels or touching the properties files.
You need
All nodes to advertise their public IP - on AWS/OS this defaults to the private IP
Ingress rules for the RMI port which defaults to 1099 - I use this
Ingress rules for the RMI "local" port which defaults to dynamic. Below I use 4001 for the client and 4000 for servers. The port can be the same but note the properties are different.
If you are using your workstation as the client you probably still need tunnels. Above Archana Aggarwal has good tips for tunnels.
Remote servers
Set java.rmi.server.hostname and server.rmi.localport inline or in the properties file.
jmeter-server -Djava.rmi.server.hostname=publicip -Dserver.rmi.localport=4000
Sneaky server on client
You can also run one on the same machine as the client. For clarity I've set java.rmi.server.hostname but left server.rmi.localport as dynamic
jmeter-server -Djava.rmi.server.hostname=localip
Client
Set java.rmi.server.hostname and client.rmi.localport inline or in the properties file. Use -R etc like so:
jmeter -n -t Test.jmx -Rremotepublicip1,remotepublicip2 -Djava.rmi.server.hostname=clientpublicip -Dclient.rmi.localport=4001 -GmypropA=1 -GmypropB=2 -lresults.jtl
When you go for distributed testing using JMeter in AWS, I would suggest you to use docker - which will help us with jmeter test infrastructure very quickly. This way we can also ensure that same version of java and jmeter are installed in all the instances of amazon which is very important of JMeter distributed testing.
Ensure that - you set below properties and ports are open for jmeter-server. [they do not have to be 1099,50000 exactly]
server.rmi.localport=50000
server_port=1099
java.rmi.server.hostname=SERVER_IP
for client
client.rmi.localport=60000
java.rmi.server.hostname=SERVER_IP - this step is very important as the container in aws instance will have their own IP address in the docker network - so master and slave can not communicate. So we explicitly set this property
More info:
http://www.testautomationguru.com/jmeter-distributed-load-testing-using-docker-in-aws/