I have an amazon ec2 (t2.micro) instance running. I have 2 Mean applications hosted on it. It was working fine for more than 5 months. But from last two days when more than one pm2 process starts the instance get stopped and I need to restart it again. With one pm2 it is working fine but as I start the second pm2 process it gets stopped (It becomes unresponsive and I can't login into) and again I need to reboot.
It was working fine with 3 to 4 pm2 processes but from last 2 days it is not working as expected.I have checked the CloudWatch Logs but everything is fine there.
I am using Nginx server. I have checked nginx, pm2 and mongodb logs but nothing is wrong there.
Also, I have one Lightsail instance with 1GB memory and 40GB SSD. I have hosted 5 WordPress websites there. It is also having the same issues I need to reboot the instance more than twice a day.
I don't know what is the reason behind that so, asking here for help. Is there any issue with AWS EC2 or Lightsail instances?
This is EC2 CloudWatch graph please check.
Related
I have a problem with my Amazon EC2 instance (that did not happened when I was using DigitalOcean).
I've several EC2 instances that are managed by me. My personal EC2 has about 5 Wordpress sites running on a t2.micro instance and the traffic is not high so it is working well in load speed.
Also I have another 2 instances for one of my clients, one t2.micro (running only one Wordpress site) and a t3a.micro (running 4 Wordpress sites). The issue is with all 3 instances (mine and all the 2 of my client).
I have a CloudWatch alarm to notify me by email when Error 504 happen. Since I get the alarm, the website becomes unavailable (Cloudflare shows me Error 504), but I can get into SSH or Webmin. I do service nginx status and all seems to be fine, same to service php7.2-fpm. I do pkill nginx && pkill php* and then service nginx start && service php7.2-fpm start correctly but when I try to enter to the site, the Error 504 is still there.
To test, I decided to install and configure Apache with and without PHP-FPM enabled, same problem. Instance going well and websites running fast but after X amount of hours, it becomes unaccessible via web and the only solution is rebooting...
What's the only thing that solve the issue? Well, rebooting the instance.... After it boots, the websites are available again. Please note that I moved from DigitalOcean to AWS because it is more useful but I can't understand why the problem is happening here and not there since I've a similar instance configured very similar...
In all of the instances I've a setup with:
OS: Ubuntu 18.04
Types: Two t2.micro and one t3a.micro
ELB: Enabled
Security Groups: only allow ports 80, 443 from all the sources.
Database: In a RDS, not on the same instance.
I can provide the logs of everything that you probably can ask but I review all the Nginx and PHP-fpm logs and I can't see any anomalies. Also with syslog and kern.log, but I can provide if it can helps.
Hope you can give me a hand. Thanks for your advice!
EDIT:
I already found the origin of the issue. The problem wasn't in the EC2, all my headache was because I have the RDS set with only one Security Group attached to allow access from my IP to remote management of the databases and the public IPs of the EC2 that runs Wordpress, but I figured that I also need to whitelist the private IPs of those EC2s... Really noob mistake but that was the solution.
I am running Bitnami WordPress on AWS server website working since two days but suddenly it stop showing anything and connection timeout is showing. The instance EC2 is running perfectly fine, and I have also seen IP logs, and nothing suspicious has come up.
Based on the above comments I guess the issue is with the internal web server
Make sure that the web server is running perfectly fine. And I do not mean just checking the EC2 instance state, because it is possible that the EC2 instance is running but the web server is down, causing the issue
First some insight into how my setup is:
1 ELB
4 EC2 instances
2 web servers
1 to run the migrations, queue (beanstalkd) and scheduler
1 'services' server (socket.io instance etc etc)
MySQL on RDS
Redis on Elasticache
S3 for user assets
Every day at 10:55PM, users report getting white screens and 502 Bad Gateway errors. The ELB reports that both EC2 instances are OutOfService, yet I'm SSH'd into them and fully able to use the site by bypassing the ELB. RDS and Elasticache maintenance windows aren't during this period, and the two instances aren't at load either. I can't find anything in the ELB access logs, nothing in nginx logs on the instance end, nothing in the Laravel app logs. There's nothing in the Laravel scheduler that runs at this time either.
The only thing I've found, is that in my CloudWatch metrics, the ELB latency spikes right up to about 5-10 seconds. All this results in downtime of about 5-15 minutes at the same time every day. I can't seem to find anything that is causing the issue.
I'm 100% stumped as to what could be causing this to happen. Any help is appreciated.
What probably happens is that your web servers run out of connections, ELB cannot perform health checks and takes them out of service. It's actually enough for one of the machines to experience this and be taken out of service and the other will be killed as a cascading effect.
How many connections can the web servers hold at the same time?
Do you process a particularly "heavy request" at that point in time when this happens?
Does adding more web servers solve your problem?
I have a test environment setup (along with a staging and production). The staging environment has spun down automatically in to red (health).
I would like to spin down the test server as well, is this possible without destroying the actual instance? I will be using it again in a few months, but until then I don't need it consuming resources.
Server == Instance.
Termination ==> killing a server, no chance to get it back.
Stop => Stopping a server, it can be restarted. No EC2 costs while it's stopped. ( EBS costs may apply.)
Having a single node Beanstalk environment means having 1 EC2 instance. I never tried to stop it, but you should be able to do it. ( Just check it a few minutes ago, if AWS spins up a new instance or not. In a single node setup it shouldn't.)
I have hosted a CakePHP app in EC2 using Elastic Beanstalk. Due to some performance issues I had to restart the server. Now I have lost all the files.
Did you reboot the instance or stop and start it?
If you rebooted it then it's certain your app and data are there.
If you stop and start an instance-store instance (not EBS-backed), then you will loose all your data on the ephemeral volume.
All this is well explained on the following link, take a look at it: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Storage.html.