EC2 Security Group not connecting to my IP - amazon-web-services

Seems like a basic job, but for some reason it is not working for me. I wish to access my EC2 instances from my office IP only.
I went into my security group and added an SSH rule with source for my IP only like this -
But this does not seems to be working for me at all. I get connection denied when I try to connect via WinSCP or by using terminal.
Everything works if I change my source to Everywhere (0.0.0.0/0)
Anyone has any pointer for me please.

Login to the EC2 using the method that works and issue the command
who am i
It will say something like
ec2-user pts/0 2016-02-29 15:06 (104.66.242.192)
Use the ip address shown for you (not the one above) in the security group rule

Although "who am i" work fine. However I'd like to add two more solutions.
both are very easy.
Solution 1:
Step 1: Open security group for all IP's (0.0.0.0/0) for a while.
Step 2: Make ssh connection to your server.
Step 3: run "w" command and check the output in FROM column.
ubuntu#ip-172-31-39-228:~$ w
23:20:09 up 5 min, 1 user, load average: 0.08, 0.08, 0.04
USER TTY FROM LOGIN# IDLE JCPU PCPU WHAT
ubuntu pts/0 52.95.75.17 23:20 0.00s 0.01s 0.00s w
Step 4: Replace this IP in the security group with 0.0.0.0/0 ( like 52.95.75.17/32 ).
Solution 2:
Step 1: Open security group for all IP's (0.0.0.0/0) for a while.
Step 2: Make ssh connection to your server.
Step 3: Check the last login info on welcome message.
like :
Learn more about enabling ESM Apps service at https://ubuntu.com/esm
Last login: Thu Feb 9 23:21:42 2023 from 52.95.75.17
ubuntu#ip-172-31-39-228:~$
ubuntu#ip-172-31-39-228:~$
Step 4 ( optional ): If IP address not available in welcome message. Then run "last" command.
ubuntu#ip-172-31-39-228:~$
ubuntu#ip-172-31-39-228:~$ last
ubuntu pts/2 52.95.75.17 Thu Feb 9 23:33 still logged in
ubuntu pts/1 52.95.75.17 Thu Feb 9 23:21 still logged in
Step 5: Replace this IP in the security group with 0.0.0.0/0 ( like 52.95.75.17/32 ).
Check below screenshot for reference of above solutions:

Feel free to use my powershell script for this .
The script detects your public ip and adds it to the inbound security group rules of dedicated RDP and SSH security groups .
If these groups do not exist , the script will create them and add it to the appropriate instances .
https://github.com/manuelh2410/public/blob/1/AWSIP_Linux_Win.ps1

Related

AWS EC2: cannot get bare metal instance

I have tried several times in the last two weeks to log on to a c5.metal instance. Each time I get "Initializing" in the status checks field, but after 10 minutes it is still "Initializing" and I'm not able to log on. I have had success with c5.metal before, but not any more.
Today I also tried to get an m5.metal instance. This time the instance successfully initialized after 10 minutes but I was not able to log on with Putty. I stopped the instance, then after about 30 minutes I tried again and this time I did not get past "Initializing" in the status check field and I stopped it after 15 minutes.
I get billed for the 10 to 15 minute bare metal wait periods, even when initialization doesn't complete. I have no problems with AWS virtual instances.
Thanks for any ideas on what I can do to get the bare metal instances to work.
To reproduce your situation, I did the following:
Launched an Amazon EC2 instance in Ohio:
Instance Type: c5.metal
AMI: Ubuntu Server 18.04 LTS (HVM), SSD Volume Type
Network: In my Default VPC so that it uses a Public Subnet
Security Group: Default settings, which grants port 22 access from the Internet
Instance entered running state very quickly, Status Checks showed as Initializing
It took about 8 minutes until the status checks were showing 2/2 checks (it might have been faster, but I was testing other things in the meantime).
I was able to successfully login to the instance:
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-1065-aws x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Sat Jun 6 23:21:18 UTC 2020
System load: 0.02 Processes: 924
Usage of /: 13.7% of 7.69GB Users logged in: 0
Memory usage: 0% IP address for enp125s0: 172.31.9.77
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu#ip-172-31-9-77:~$
(Actually, I first tried to login as ec2-user and it took me a while to realize this was an Ubuntu AMI, so I connected as ubuntu).
It is possible that the slow startup is due to the Operating System or hardware checking the 192GB of RAM that is allocated to the instance.
I booted another instance using an Amazon Linux 2 AMI and it required approximately 7 minutes before I could connect.
I also noticed that the c5.metal instances did not provide anything for "Get System Log" or "Get Instance Screenshot". This might be a result of using a bare-metal instance.
I joined John Rotenstein's twitch.tv channel and he showed how he got a c5.metal instance. What I learned is that if a metal instance does not work in the region you had chosen, try launching a new instance in a different data center region. For example, I had a c5.metal instance at us-east-2a. Following John's directions, I launched an instance at us-east-2c and after about 8 minutes the instance was ready for use.

Azure VM, your credentials did not work on remote desktop

I've just had a bit of fun trying to connect to a new VM I'd created, I've found loads of posts from people with the same problem, the answer details the points I've found
(1) For me it worked with
<VMName>\Username
Password
e.g.
Windows8VM\MyUserName
SomePassword#1
(2) Some people have just needed to use a leading '\', i.e.
\Username
Password
Your credentials did not work Azure VM
(3) You can now reset the username/password from the app portal. There are powershell scripts which will also allow you to do this but that shouldn't be necessary anymore.
(4) You can also try redeploying the VM, you can do this from the app portal
(5) This blog says that "Password cannot contain the username or part of username", but that must be out of date as I tried that once I got it working and it worked fine
https://blogs.msdn.microsoft.com/narahari/2011/08/29/your-credentials-did-not-work-error-when-connecting-to-windows-azure-vms/
(6) You may find links such as the below which mention Get-AzureVM, that seems to be for classic VMs, there seem to be equivalents for the resource manager VMs such as Get-AzureRMVM
https://blogs.msdn.microsoft.com/mast/2014/03/06/enable-rdp-or-reset-password-with-the-vm-agent/
For complete novices to powershell, if you do want to go down that road here's the basics you may need. In the end I don't believe I needed this, just point 1
unInstall-Module AzureRM
Install-Module AzureRM -allowclobber
Import-Module AzureRM
Login-AzureRmAccount (this will open a window which takes you through the usual logon process)
Add-AzureAccount (not sure why you need both, but I couldn’t log on without this)
Select-AzureSubscription -SubscriptionId <the guid for your subscription>
Set-AzureRmVMAccessExtension -ResourceGroupName "<your RG name>" -VMName "Windows8VM" -Name "myVMAccess" -Location "northeurope" -username <username> -password <password>
(7) You can connect to a VM in a scale set as by default the Load Balancer will have Nat Rules mapping from port onwards 50000, i.e. just remote desktop to the IP address:port. You can also do it from a VM that isn't in the scale set. Go to the scale set's overview, click on the "virtual network/subnet", that'll give you the internal IP address. Remote desktop from the other one
Ran into similar issues. It seems to need domain by default. Here is what worked for me:
localhost\username
Other option can be vmname\username
Some more guides to help:
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/quick-create-portal#connect-to-virtual-machine
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/connect-logon
In April 2022 "Password cannot contain the username or part of username" was the issue.
During the creation of VM in Azure, everything was alright but wasn't able to connect via RDP.
Same in Nov 2022, you will be allowed to create a password that contains the user name but during login it will display the credential error. Removing the user name from the password fixed it.

KeepAlived + HAProxy gets connection refused after a while

I´ve the next scenario, 4 VM´s running Red Hat Enterprise Linux 7:
20.1.67.230 server (VIRTUAL IP) (not a host)
20.1.67.219 haproxy1 (LOAD BALANCER)
20.1.67.229 haproxy2 (LOAD BALANCER)
20.1.67.223 server1 (LOAD TO BALANCE)
20.1.67.213 server2 (LOAD TO BALANCE)
My keepalived.conf file is:
vrrp_script chk_haproxy {
script "killall -0 haproxy" # check the haproxy process
interval 2 # every 2 seconds
weight 2 # add 2 points if OK
}
vrrp_instance VI_1 {
interface enp0s3 # interface to monitor
state MASTER# MASTER on haproxy1, BACKUP on haproxy2
virtual_router_id 51
priority 101 # 101 on haproxy1, 100 on haproxy2
unicast_src_ip 20.1.67.229 # This is the IP of the interface keepalived listens on
unicast_peer { # This is the IP of the peer instance
20.1.67.219
}
virtual_ipaddress {
20.1.67.230 # virtual ip address
}
track_script {
chk_haproxy
}
}
When a execute a request to the VIRTUAL IP, for instance:
curl server:8888/info
everything is ok, but just for a while, after some requests the command returns me : connection refused
So I´ve to restart the keepalived service manually , this way:
systemctl restart keepalived.service
The whole system seems work well, VRRP messages between haproxy1 and haproxy2 are OK, it´s just like the Virtual IP is not working properly.
Can anyone point me in the right direction to diagnose and fix this problem?
It was a networking issue. There was a device on the net with same IP as the Virtual IP I had chosen.

aerospike cluster crashed after index creation

We have a cluster at AWS of 4 machines t2micro (1cpu 1gb ram 15gb ssd) and we were testing aerospike.
We used the aws marketplace AMI to install aerospike v3 community edition, and configured only the aerospike.conf file to have a namespace on the disk.
We had one namespace with two sets, totaling 18M documents, 2gb ram occupied and aprox 40gb of disk space occupied.
After the creation of an index in a 12M records set the system crashed.
Some info:
aql on the instance:
[ec2-user#ip-172-XX-XX-XXX ~]$ aql
2015-09-16 18:44:37 WARN AEROSPIKE_ERR_CLIENT Socket write error: 111
Error -1: Failed to seed cluster*
Tail of the log: (it keeps adding only lines repeated)
Sep 16 2015 19:08:26 GMT: INFO (drv_ssd): (drv_ssd.c::2406) device /opt/aerospike/data/bar.dat: used 6980578688, contig-free 5382M (5382 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 23 (0.0/s), defrag-q 0 defrag-tot 128 (0.0/s)
Sep 16 2015 19:08:46 GMT: INFO (drv_ssd): (drv_ssd.c::2406) device /opt/aerospike/data/bar.dat: used 6980578688, contig-free 5382M (5382 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 23 (0.0/s), defrag-q 0 defrag-tot 128 (0.0/s)
Sep 16 2015 19:09:06 GMT: INFO (drv_ssd): (drv_ssd.c::2406) device /opt/aerospike/data/bar.dat: used 6980578688, contig-free 5382M (5382 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 23 (0.0/s), defrag-q 0 defrag-tot 128 (0.0/s)
Sep 16 2015 19:09:26 GMT: INFO (drv_ssd): (drv_ssd.c::2406) device /opt/aerospike/data/bar.dat: used 6980578688, contig-free 5382M (5382 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 23 (0.0/s), defrag-q 0 defrag-tot 128 (0.0/s)
asmonitor:
$ asmonitor -h 54.XX.XXX.XX
request to 54.XX.XXX.XX : 3000 returned error
skipping 54.XX.XXX.XX:3000
***failed to connect to any hosts
asadm:
$ asadm -h 54.XXX.XXX.XX -p 3000
Aerospike Interactive Shell, version 0.0.10-6-gdd6fb61
Found 1 nodes
Offline: 54.207.67.238:3000
We tried restarting the instances, one of them is back but working as a standalone node, the rest are in the described state.
The instances are working, but the aerospike service is not.
There is a guide dedicated to using Aerospike on Amazon EC2 and you probably want to follow it closely to get started.
When you see a AEROSPIKE_ERR_CLIENT "Failed to seed cluster" it means that your client cannot connect to any seed node in the cluster. A seed node is the first node the client connects to, from which it learns about the cluster partition table and the other nodes. You are using aql with the default host (127.0.0.1) and port (3000) values. Try with -h and -p, or use --help for information on the flags.
There are many details you're not including, such as are these nodes all in the same Availability Zone of the same EC2 region? Did you configure your /etc/aerospike.conf with mesh configuration (that's the mode needed in Amazone EC2). Simply, can your nodes see each other? You're using what looks like public IP, but your nodes need to see each other through their local IP addresses. They have no idea what their public IP is, unless you configured it. At the same time the clients may be connecting from other AZs, so you will need to set up the access_address correctly. See this discussion forum post on the topic: https://discuss.aerospike.com/t/problems-configuring-clustering-on-aws-ec2-with-3-db-instances/1676

Enabling HA namenodes on a secure cluster in Cloudera Manager fails

I am running a CDH4.1.2 secure cluster and it works fine with the single namenode+secondarynamenode configuration, but when I try to enable High Availability (quorum based) from the Cloudera Manager interface it dies at step 10 of 16, "Starting the NameNode that will be transitioned to active mode namenode ([my namenode's hostname])".
Digging into the role log file gives the following fatal error:
Exception in namenode joinjava.lang.IllegalArgumentException: Does not contain a valid host:port authority: [my namenode's fqhn]:[my namenode's fqhn]:0 at
org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:206) at
org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:158) at
org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:147) at
org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:143) at
org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:547) at
org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:480) at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:443) at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:608) at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:589) at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1140) at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1204)
How can I resolve this?
It looks like you have two problems:
The NameNode's IP address is resolving to "my namenode's fqhn" instead of a regular hostname. Check your /etc/hosts file to fix this.
You need to configure dfs.https.port. With Cloudera Manager free edition, you must have had to add the appropriate configs to the safety valves to enable security. As part of that, you need to configure the dfs.https.port.
Given that this code path is traversed even in the non-HA mode, I'm surprised that you were able to get your secure NameNode to start up correctly before enabling HA. In case you haven't already, I recommend that you first enable security, test that all HDFS roles start up correctly and then enable HA.