Good afternoon people,
I created an environment in Elastic Beanstalk and uploaded a NODEjs application an api with express.
She's working fine, all right.
But the integrity of the environment is reported as serious, and this monitoring attempt appears in the logs.
----------------------------------------
/var/log/nginx/access.log
----------------------------------------
172.31.46.198 - - [03/Nov/2021:19:14:13 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.1.181 - - [03/Nov/2021:19:14:13 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.30.127 - - [03/Nov/2021:19:14:13 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.46.198 - - [03/Nov/2021:19:14:28 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.1.181 - - [03/Nov/2021:19:14:28 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.30.127 - - [03/Nov/2021:19:14:28 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.46.198 - - [03/Nov/2021:19:14:43 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.30.127 - - [03/Nov/2021:19:14:43 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.1.181 - - [03/Nov/2021:19:14:43 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.30.127 - - [03/Nov/2021:19:14:58 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.1.181 - - [03/Nov/2021:19:14:58 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.46.198 - - [03/Nov/2021:19:14:58 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
172.31.30.127 - - [03/Nov/2021:19:15:13 +0000] "GET / HTTP/1.1" 404 139 "-" "ELB-HealthChecker/2.0" "-"
Does anyone know how I can fix this, without turning off the monitoring?
Good night people,
I found the problem, I didn't have anything set in my API's root on "/", so EB tried to monitor the api state and took a 404.
I set up a HealthCheck on the root "/" and normalized the 404 errors and integrity issue in the environment.
I'm trying to adjust my WS to support ~ 20k concurrent users.
No matter what configuration I change I still get the same 6 secs avg response time / per endpoint when my tests hit 2(two)k users and various 502 / 504 errors.
WebService:
CloudFlare <--> Nginx <--> Gunicorn <--> Django/DRF <--> Memcache <---> Postgres
Here's what I tried:
Increase gunicorn workers from 4 to 10
Increase service(pod) instances from 3 to 10
Increase gunicorn worker timeout to 120
Increase Nginx proxy_pass timeout to 120
Most endpoints hit the database once every 100 seconds and the other requests get the data from memcache.
Could any one help by pointing out what kind of configuration should I be changing?
Where should I be looking for delays/bottlenecks?
Gunicorn workers clearly are timming out, which I dont undersdand since theres no logic in the WS views. It should be only getting a query from memcache and returning it.
Nginx logs:
latforms/android HTTP/1.1", upstream: "http://10.0.1.17:9090/endpoints/platforms/android", host: "myhost.co"
2018/08/13 23:43:25 [error] 8893#8893: *2809163 upstream timed out (110: Connection timed out) while connecting to upstream, client: 200.211.198.133, server: myhost.co, request: "GET /endpoints/store/products/729 HTTP/1.1", upstream: "http://10.0.1.18:9090/endpoints/store/products/729", host: "myhost.co"
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/categories/?cat_pk=13081 HTTP/1.1" 200 1718 "-" "python-requests/2.18.4" 627 80.840 [production-service-api-80] 10.0.0.112:9090, 10.0.1.13:9090, 10.0.0.113:9090 0, 0, 11150 40.000, 40.000, 0.840 504, 504, 200
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/categories/?cat_pk=13081 HTTP/1.1" 200 1718 "-" "python-requests/2.18.4" 689 80.857 [production-service-api-80] 10.0.0.112:9090, 10.0.1.12:9090, 10.0.0.113:9090 0, 0, 11150 40.000, 40.000, 0.857 504, 504, 200
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/home/ HTTP/1.1" 200 10072 "-" "python-requests/2.18.4" 670 80.580 [production-service-api-80] 10.0.1.13:9090, 10.0.1.11:9090, 10.0.0.112:9090 0, 0, 66511 40.001, 40.002, 0.577 504, 504, 200
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/products/691/ HTTP/1.1" 200 703 "-" "python-requests/2.18.4" 646 80.486 [production-service-api-80] 10.0.1.8:9090, 10.0.1.13:9090, 10.0.1.12:9090 0, 0, 1968 40.000, 40.000, 0.486 504, 504, 200
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/products/5458 HTTP/1.1" 301 0 "-" "python-requests/2.18.4" 678 80.444 [production-service-api-80] 10.0.1.13:9090, 10.0.1.12:9090, 10.0.1.17:9090 0, 0, 0 40.000, 40.002, 0.442 504, 504, 301
....
90, 10.0.1.11:9090, 10.0.1.8:9090 0, 0, 1968 40.000, 40.000, 0.584 504, 504, 200
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/products/5458/ HTTP/1.1" 200 241 "-" "python-requests/2.18.4" 647 80.709 [production-service-api-80] 10.0.0.113:9090, 10.0.1.8:9090, 10.0.0.112:9090 0, 0, 327 40.001, 40.000, 0.708 504, 504, 200
--
2018/08/13 23:43:25 [error] 8766#8766: *2809243 upstream timed out (110: Connection timed out) while connecting to upstream, client: 200.211.198.133, server: myhost.co, request: "GET /endpoints/store/categories/?cat_pk=13081 HTTP/1.1", upstream: "http://10.0.1.13:9090/endpoints/store/categories/?cat_pk=13081", host: "myhost.co"
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/products/692 HTTP/1.1" 301 0 "-" "python-requests/2.18.4" 677 80.672 [production-service-api-80] 10.0.1.17:9090, 10.0.1.10:9090, 10.0.0.113:9090 0, 0, 0 40.001, 40.001, 0.670 504, 504, 301
200.211.198.133 - [200.211.198.133] - - [13/Aug/2018:23:43:25 +0000] "GET /endpoints/store/products/4608/ HTTP/1.1" 200 553 "-" "python-requests/2.18.4" 647 80.591 [production-service-api-80] 10.0.1.11:9090, 10.0.1.17:9090, 10.0.1.8:9090 0, 0, 1090 40.000, 40.003, 0.588 504, 504, 200
Gunicorn logs:
{"asctime": "2018-08-13 23:42:55,145", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:55 +0000] \"GET /endpoints/store/products/691/ HTTP/1.1\" 200 1968 \"-\" \"python-requests/2.18.4\""}
{"asctime": "2018-08-13 23:42:55,167", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:55 +0000] \"GET /endpoints/store/products/729 HTTP/1.1\" 301 - \"-\" \"python-requests/2.18.4\""}
[2018-08-13 23:42:55 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:36)
[2018-08-13 23:42:55 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:37)
[2018-08-13 23:42:55 +0000] [382] [INFO] Booting worker with pid: 382
[2018-08-13 23:42:55 +0000] [383] [INFO] Booting worker with pid: 383
{"asctime": "2018-08-13 23:42:55,403", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:55 +0000] \"GET /endpoints/store/products/691/ HTTP/1.1\" 200 1968 \"-\" \"python-requests/2.18.4\""}
....
{"asctime": "2018-08-13 23:42:55,184", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:55 +0000] \"GET /endpoints/store/categories/?cat_pk=13081 HTTP/1.1\" 200 11150 \"-\" \"python-requests/2.18.4\""}
{"asctime": "2018-08-13 23:42:55,262", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:55 +0000] \"GET /endpoints/platforms/android HTTP/1.1\" 200 48 \"-\" \"python-requests/2.18.4\""}
{"asctime": "2018-08-13 23:42:55,439", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:55 +0000] \"GET /endpoints/platforms/android HTTP/1.1\" 200 48 \"-\" \"python-requests/2.18.4\""}
--
[2018-08-13 23:42:56 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:31)
{"asctime": "2018-08-13 23:42:56,689", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:56 +0000] \"GET /endpoints/store/products/729/ HTTP/1.1\" 200 2163 \"-\" \"python-requests/2.18.4\""}
{"asctime": "2018-08-13 23:42:56,799", "name": "gunicorn.access", "levelname": "INFO", "message": "10.0.0.13 - - [13/Aug/2018:23:42:56 +0000] \"GET /endpoints/store/products/5458/ HTTP/1.1\" 200 327 \"-\" \"python-requests/2.18.4\""}
why you not used uwsgi?
for better working do this
decrease database hit in your codes
increase worker count for gunicorn
diable info logging for gunicorn and nginx
if these configuration not worked for you you must change setup configuration or increase resource of your server.
First, I use the server environment:
sever: nginx + uwsgi + django app, docker + AWS ECS deploy
celery: rabbitmq ec2
cache: redis ec2
logging: AWS CloudWatch log + watchtower third party app
When I access ECS EC2 and check nginx access.log, the following request periodically comes in.
Why is this request coming to me? This is what keeps coming in the first time you open the server.
In addition, my ecs server's security group 80/443 ports are opened to anywhere.
nginx/access.log
54.214.101.194 - - [14/Jul/2017:03:02:12 +0000] "HEAD http://13.114.17.75:80/mysql/admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:12 +0000] "HEAD http://13.114.17.75:80/mysql/dbadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:12 +0000] "HEAD http://13.114.17.75:80/mysql/sqlmanager/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:12 +0000] "HEAD http://13.114.17.75:80/mysql/mysqlmanager/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpmyadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpMyadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpMyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpmyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpmyadmin3/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpmyadmin4/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/2phpmyadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/wp-content/plugins/portable-phpmyadmin/wp-pma-mod/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:13 +0000] "HEAD http://13.114.17.75:80/phpmy/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/phppma/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/myadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/shopdb/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/MyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/program/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/PMA/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/dbadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/pma/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:14 +0000] "HEAD http://13.114.17.75:80/db/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/mysql/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/database/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/db/phpmyadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/db/phpMyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/sqlmanager/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/mysqlmanager/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/php-myadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:15 +0000] "HEAD http://13.114.17.75:80/phpmy-admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/mysqladmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/mysql-admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/phpmyadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/phpMyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/sysadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/sqladmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/db/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/web/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:16 +0000] "HEAD http://13.114.17.75:80/admin/pMA/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/mysql/pma/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/mysql/db/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/mysql/web/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/mysql/pMA/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/sql/phpmanager/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/sql/php-myadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/sql/phpmy-admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/sql/sql/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:17 +0000] "HEAD http://13.114.17.75:80/sql/myadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/webadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/sqlweb/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/websql/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/webdb/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/sqladmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/sql-admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/phpmyadmin2/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/phpMyAdmin2/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:18 +0000] "HEAD http://13.114.17.75:80/sql/phpMyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/myadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/webadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/dbweb/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/websql/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/webdb/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/dbadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/db-admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/phpmyadmin3/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:19 +0000] "HEAD http://13.114.17.75:80/db/phpMyAdmin3/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/db/phpMyAdmin-3/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/phpmyadmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/phpMyAdmin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/db/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/web/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/pma/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/PMA/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/administrator/admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/phpMyAdmin2/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:20 +0000] "HEAD http://13.114.17.75:80/phpMyAdmin3/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/phpMyAdmin4/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/phpMyAdmin-3/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/php-my-admin/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/PMA2011/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/PMA2012/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/PMA2013/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/PMA2014/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/PMA2015/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:21 +0000] "HEAD http://13.114.17.75:80/PMA2016/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/PMA2017/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/PMA2018/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2011/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2012/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2013/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2014/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2015/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2016/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:22 +0000] "HEAD http://13.114.17.75:80/pma2017/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/pma2018/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2011/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2012/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2013/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2015/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2016/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:23 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2017/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:24 +0000] "HEAD http://13.114.17.75:80/phpmyadmin2018/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
54.214.101.194 - - [14/Jul/2017:03:02:24 +0000] "HEAD http://13.114.17.75:80/phpmanager/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 Jorgee"
95.213.177.125 - - [14/Jul/2017:03:14:35 +0000] "POST /azenv.php?auth=150000207593&a=PSCMN&i=885409785&p=80 HTTP/1.1" 404 580 "https://proxyradar.com/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)"
This is a very typical request pattern from the vulnerability scanning tool ZmEu. Long story short a hacker is running an automated tool that is trying to find a vulnerable installation of PHPMyAdmin on your system, to exploit it in order to gain root access to your system. It doesn't even matter that you don't have PHPMyAdmin on your system, they will still make requests just to test and see if you do because its cheap to do so, and if they do find something to exploit they can get into your server to steal data or use it for nefarious purposes.
Unfortunately this is just the cost of having a server on the internet, people are constantly running automated scanning tools against any server they can reach, trying to find ways to hack into it and take it over.
That's my first Django project and I'm trying to deploy it to OpenShift, but I have a problem: when I'm trying to open app's page, I see "503: Service Temporarily Unavailable" message. I thought that app wasn't started so I tried to launch 'ctl_app start' in ssh and I've got this:
python: can't open file 'app.py': [Errno 2] No such file or directory
I tried to start app this way: rhc app start customerquiz and got this:
RESULT:
customerquiz started
But it took no effect. Well, I tried to look the state of app:
$ rhc app show --state customerquiz
Cartridge python-2.7, mysql-5.1, phpmyadmin-3.4 is started
And that's what in server logs:
$ rhc tail customerquiz
==> mysql-5.1/log/mysql_error.log <==
130519 17:32:54 [Note] /usr/libexec/mysqld: Shutdown complete
130519 17:32:54 mysqld_safe mysqld from pid file /var/lib/openshift/000005//mysql-5.1/pid/mysql.pid ended
130519 17:33:04 mysqld_safe Starting mysqld daemon with databases from /var/lib/openshift/000005//mysql-5.1/data/
130519 17:33:04 InnoDB: Initializing buffer pool, size = 16.0M
130519 17:33:05 InnoDB: Completed initialization of buffer pool
130519 17:33:05 InnoDB: Started; log sequence number 0 44233
130519 17:33:05 [Note] Event Scheduler: Loaded 0 events
130519 17:33:05 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.69' socket: '/var/lib/openshift/000005//mysql-5.1/socket/mysql.sock' port: 3306 Source distribution
==> python-2.7/logs/appserver.log <==
python: can't open file 'app.py': [Errno 2] No such file or directory
==> phpmyadmin-3.4/logs/access_log-20130519-000000-EST <==
109.254.107.11 - - [19/May/2013:17:14:13 -0400] "GET /phpmyadmin/themes/pmahomme/img/error.ico HTTP/1.1" 200 5430 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/phpmyadmin.css.php?server=1&token=436aa&js_frame=right&nocache=5471357898" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
- - - [19/May/2013:17:14:13 -0400] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.15 (Red Hat) (internal dummy connection)"
109.254.107.11 - admin [19/May/2013:17:14:18 -0400] "GET /phpmyadmin/ HTTP/1.1" 200 2580 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - admin [19/May/2013:17:14:19 -0400] "GET /phpmyadmin/main.php?token=436aa HTTP/1.1" 200 35472 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - adminB [19/May/2013:17:14:21 -0400] "GET /phpmyadmin/version_check.php?&_nocache=1368998062197204658 HTTP/1.1" 200 44 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/main.php?token=436aa" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - admin [19/May/2013:17:14:19 -0400] "GET /phpmyadmin/navigation.php?token=436aa HTTP/1.1" 200 5029 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - admin [19/May/2013:17:20:49 -0400] "GET /phpmyadmin/ HTTP/1.1" 200 2580 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - admin [19/May/2013:17:20:50 -0400] "GET /phpmyadmin/navigation.php?token=436aa HTTP/1.1" 200 5029 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - admin [19/May/2013:17:20:50 -0400] "GET /phpmyadmin/main.php?token=436aa HTTP/1.1" 200 35472 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
109.254.107.11 - admin [19/May/2013:17:20:52 -0400] "GET /phpmyadmin/version_check.php?&_nocache=1368998453881950766 HTTP/1.1" 200 44 "https://customerquiz-unclelem.rhcloud.com/phpmyadmin/main.php?token=8436aa" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0"
==> phpmyadmin-3.4/logs/error_log-20130519-000000-EST <==
[Sun May 19 17:12:56 2013] [notice] Digest: done
[Sun May 19 17:12:56 2013] [notice] Apache/2.2.15 (Unix) PHP/5.3.3 configured -- resuming normal operations
[Sun May 19 17:28:00 2013] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c6,c356
[Sun May 19 17:28:00 2013] [notice] Digest: generating secret for digest authentication ...
[Sun May 19 17:28:00 2013] [notice] Digest: done
[
Sun May 19 17:28:00 2013] [notice] Apache/2.2.15 (Unix) PHP/5.3.3 configured -- resuming normal operations
[Sun May 19 17:33:17 2013] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c6,c356
[Sun May 19 17:33:17 2013] [notice] Digest: generating secret for digest authentication ...
[Sun May 19 17:33:17 2013] [notice] Digest: done
[Sun May 19 17:33:17 2013] [notice] Apache/2.2.15 (Unix) PHP/5.3.3 configured -- resuming normal operations
Does anyone have any ideas what can be wrong?
P.S. That's deployed project source code, if you need it: https://github.com/uncleLem/CustomerQuiz (I just changed database connection parameters).
P.P.S. Oh, BTW, if I open https://customerquiz-unclelem.rhcloud.com/phpmyadmin/ directly, I can see phpMyAdmin page.
We are not using mod_wsgi with these new carts. 2.7 uses gevent and 3.3 uses WebOb
Take a look at how we set up the app.py in the 2.7 cart
https://github.com/openshift/openshift-community-cartridge-python-2.7/blob/master/template/app.py
and for 3.3
https://github.com/openshift/openshift-community-cartridge-python-3.3/blob/master/template/app.py
Can you review the following blog post to see if it helps?
https://www.openshift.com/blogs/host-python-in-a-community-cart
I'm working on a web service using RESTEASY to set the response status code when get some exception.
First I tried resteasy exception mapper which works fine locally. The mapper code attached below. However, when I upload that WS into elastic beanstalk, that always return 500 (internal server error).
#Provider
public class LoadGridTileFailedExceptionMapper extends BaseExceptionMapper implements ExceptionMapper<LoadGridTileFailedException>
{
#Override
public Response toResponse(LoadGridTileFailedException e)
{
log(e.getMessage(), e);
return printMsg(e.getMessage(), DtmWebServiceReturnStatus.LOAD_GRID_TILE_FAILED_EXCEPTION_CODE);
}
}
Then I try just throw exception WebApplicationException(ex, DtmWebServiceReturnStatus.LOAD_GRID_TILE_FAILED_EXCEPTION_CODE) to get around exception mapping. The result is that I got a response status 498(LOAD_GRID_TILE_FAILED_EXCEPTION_CODE) wrapped in status code 500.
Apache Tomcat/7.0.27 - Error report HTTP Status 498 - type Status reportmessage description http.498Apache Tomcat/7.0.27
It seems that elastic beanstalk wrapped all exceptions throw out in the server side with status code 500?The question is how can I get around that feature and return the status code I set in response? Thank you.
UPDATE
Try more requests this morning and find something interesting:
Get the right return status in elastic beanstalk log snapshot
/var/log/tomcat7/localhost_access_log.txt
127.0.0.1 - - [09/Jan/2013:15:06:28 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:31 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:34 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:37 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:39 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:41 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:44 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:48 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:51 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:54 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
127.0.0.1 - - [09/Jan/2013:15:06:57 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22
/var/log/httpd/elasticbeanstalk-access_log
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:28 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:31 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:34 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:37 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:39 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:41 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:44 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:48 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:51 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:54 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
10.28.215.233 (65.167.11.254, 10.28.215.233) - - [09/Jan/2013:15:06:57 +0000] "GET /published/tile/003331330031 HTTP/1.1" 498 22 "-" "-"
However in client side, still got 500 :-(
printMsg method:
protected Response printMsg(String msg, int intStatus)
{
// Need this due to the Resteasy bug
ServiceDataCollector.processRequest(true);
ResponseBuilder builder = Response.status(intStatus);
builder.type("text/plain");
builder.entity("ERROR: " + msg);
Response rep = builder.build();
LOG.error(rep.getStatus() + ":" + rep.toString());
return rep;
}
Some one help me to work the problem out. I had the httpd deployed in my AMI before tomcat server at 80. So the load balancer will interact with httpd server, which change the status code from tomcat to 500. Disable that httpd server will solve the problem. Thx for everyone's help.