I'm trying to write a regex to detect log entries in NGinx.
Below is a list of entries that should match the expression:
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa3 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 34489 5 0.073
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa1 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 33339 5 0.091
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa4 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 21907 5 0.076
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaab HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 19671 5 0.159
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa2 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 15359 5 0.104
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa5 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 35095 5 0.084
Below is a list of entries that should not match the expression:
1.1.1.1 - - [28/Mar/2019:13:58:55 +0000] "GET /pro/p/id/63aaaaaaaaa8/4.4.4.4/YL0000000000.rom HTTP/1.1" "-" "Yealink W52P 25.81.0.10 00:15:aa:aa:aa:f9" 404 - 1 5 0.137
2.2.2.2 - - [28/Mar/2019:13:58:56 +0000] "GET /pro/p/id/67aaaaaaaaa0/4.4.4.4/T46G.rom HTTP/1.1" "-" "Yealink SIP-T46G 28.81.0.20 00:15:aa:aa:aa:eb" 404 - 1 5 0.128
3.3.3.3 - - [28/Mar/2019:13:59:00 +0000] "GET /pro/p/id/67aaaaaaa750/4.4.4.4/T46G.rom HTTP/1.1" "-" "Yealink SIP-T46G 28.81.0.20 00:15:aa:aa:aa:eb" 404 - 1 5 0.131
I am trying to exclude lines that contain one of a number of strings: Polycom, Yealink, Snom.
My current regex is as follows:
^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) - - \[\d{2}\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} \+\d{4}\] \"GET \/pro\/p((?!Polycom|Snom|Yealink).).+(?:403|404)
EDIT: added an additional requirement to this regex - need to also match the 403/404 status of these lines
However this does not work correctly and gives false positives.
try Regex: (?!.*(Polycom|Snom|Yealink))^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) - - \[(\d{2})\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} \+\d{4}\] \"GET \/pro\/p
Demo
Try this Perl solution
perl -ne ' /^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) - - \[(\d{2})\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} \+\d{4}\] \"GET \/pro\/p(?!.*(Polycom|Snom|Yealink))/ms and print ' file
with the below inputs
$ cat btong.log
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa3 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 34489 5 0.073
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa1 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 33339 5 0.091
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa4 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 21907 5 0.076
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaab HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 19671 5 0.159
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa2 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 15359 5 0.104
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa5 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 35095 5 0.084
1.1.1.1 - - [28/Mar/2019:13:58:55 +0000] "GET /pro/p/id/63aaaaaaaaa8/4.4.4.4/YL0000000000.rom HTTP/1.1" "-" "Yealink W52P 25.81.0.10 00:15:aa:aa:aa:f9" 404 - 1 5 0.137
2.2.2.2 - - [28/Mar/2019:13:58:56 +0000] "GET /pro/p/id/67aaaaaaaaa0/4.4.4.4/T46G.rom HTTP/1.1" "-" "Yealink SIP-T46G 28.81.0.20 00:15:aa:aa:aa:eb" 404 - 1 5 0.128
3.3.3.3 - - [28/Mar/2019:13:59:00 +0000] "GET /pro/p/id/67aaaaaaa750/4.4.4.4/T46G.rom HTTP/1.1" "-" "Yealink SIP-T46G 28.81.0.20 00:15:aa:aa:aa:eb" 404 - 1 5 0.131
$ perl -ne ' /^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) - - \[(\d{2})\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} \+\d{4}\] \"GET \/pro\/p(?!.*(Polycom|Snom|Yealink))/ms and print ' btong.log
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa3 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 34489 5 0.073
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa1 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 33339 5 0.091
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa4 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 21907 5 0.076
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaab HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 19671 5 0.159
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa2 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 15359 5 0.104
7.7.7.7 - - [28/Mar/2019:03:30:06 +0000] "GET /pro/p/001565a2aaa5 HTTP/1.1" "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" 404 - 35095 5 0.084
$
Related
Nginx works in Docker compose with Django, react, postgress containers
Nginx shows requests for PHP, testPHP endpoints with status code 200
1.171.112.23 - - [27/Nov/2022:09:37:21 +0000] "GET /phpMyAdmin5.2/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:21 +0000] "GET /2phpmyadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:21 +0000] "GET /wp-content/plugins/portable-phpmyadmin/wp-pma-mod/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:21 +0000] "GET /phpmyadmin4/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:22 +0000] "GET /mysql/sqlmanager/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:22 +0000] "GET /phpmyadmin2016/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:22 +0000] "GET /db/myadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:22 +0000] "GET /sql/websql/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:22 +0000] "GET /php-my-admin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:23 +0000] "GET /phpMyAdmin-5.3.0-all-languages/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:23 +0000] "GET /shopdb/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:23 +0000] "GET /administrator/db/index.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:23 +0000] "GET /sql/php-myadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:23 +0000] "GET /phpmyadmin2014/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:24 +0000] "GET /phpMyAdmin-5.1.2/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:24 +0000] "GET /phpMyAdmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:24 +0000] "GET /db/db-admin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:24 +0000] "GET /sql/phpmyadmin5/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:24 +0000] "GET /phpMyAdmin-3/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:25 +0000] "GET /administrator/phpMyAdmin/index.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:25 +0000] "GET /admin/phpMyAdmin/index.php?lang=en HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:25 +0000] "GET /admin/login/?next=/admin/phpMyAdmin/index.php%3Flang%3Den HTTP/1.1" 200 2313 "http://147.182.131.129/admin/phpMyAdmin/index.php?lang=en" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:25 +0000] "GET /db/webdb/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:25 +0000] "GET /phpmyadmin2015/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:26 +0000] "GET /mysql/admin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:26 +0000] "GET /sql/phpmanager/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:26 +0000] "GET /administrator/db/index.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:26 +0000] "GET /phpmyadmin2020/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:26 +0000] "GET /phpmyadmin5/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:27 +0000] "GET /phpmy/192.1index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:27 +0000] "GET /phpMyAdmin5.2/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:27 +0000] "GET /phpmyadmin2022/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:27 +0000] "GET /sql/sql-admin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:27 +0000] "GET /sql/sqlweb/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:28 +0000] "GET /phpmyadmin2015/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:28 +0000] "GET /phpMyAdmin-4.9.7/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:28 +0000] "GET /2phpmyadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:28 +0000] "GET /database/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:28 +0000] "GET /db/phpMyAdmin3/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:29 +0000] "GET /administratorindex.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:29 +0000] "GET /sql/phpmanager/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:29 +0000] "GET /sql/phpmanager/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:29 +0000] "GET /sql/sqlweb/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:29 +0000] "GET /db/myadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:30 +0000] "GET /administrator/web/index.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:30 +0000] "GET /wp-content/plugins/portable-phpmyadmin/wp-pma-mod/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:30 +0000] "GET /admin/pma/index.php?lang=en HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:31 +0000] "GET /_phpmyadmin_/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:31 +0000] "GET /phpmyadmin2019/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:31 +0000] "GET /phpmyadmin2016/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:31 +0000] "GET /db/phpMyAdmin-5/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:31 +0000] "GET /phpMyAdmin-5.2.0-all-languages/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:32 +0000] "GET /db/phpMyAdmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:32 +0000] "GET /phpmyadmin2012/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:32 +0000] "GET /php-myadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:32 +0000] "GET /db/phpMyAdmin-5/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:33 +0000] "GET /phpmyadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:33 +0000] "GET /admin/index.php?lang=en HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
37.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:34 +0000] "GET /db/phpMyAdmin-5/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:34 +0000] "GET /sql/sql-admin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:34 +0000] "GET /administrator/PMA/index.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:34 +0000] "GET /admin/sqladmin/index.php?lang=en HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:35 +0000] "GET /mysql/web/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:35 +0000] "GET /phpmyadmin2/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:35 +0000] "GET /sql/phpmy-admin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:35 +0000] "GET /sql/phpMyAdmin2/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:36 +0000] "GET /phpmyadmin2021/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:36 +0000] "GET /phpmyadmin2019/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:36 +0000] "GET /db/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:36 +0000] "GET /dbadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:36 +0000] "GET /sql/sqladmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:37 +0000] "GET /phpMyAdmin-latest-english/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:37 +0000] "GET /mysql/mysqlmanager/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:37 +0000] "GET /db/phpmyadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:37 +0000] "GET /administrator/db/index.php?lang=en HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:37 +0000] "GET /phpmyadmin2012/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:38 +0000] "GET /phpmyadmin4/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:38 +0000] "GET /phpmyadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:38 +0000] "GET /phpMyAdmin-5.1.1/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:38 +0000] "GET /db/phpmyadmin3/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:38 +0000] "GET /dbadmin/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
1.171.112.23 - - [27/Nov/2022:09:37:39 +0000] "GET /sql/phpmanager/index.php?lang=en HTTP/1.1" 200 557 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
197.60.121.41 - - [2
unknown requests
192.155.90.118 - - [28/Nov/2022:02:54:21 +0000] "\x16\x03\x01\x00\x85\x01\x00\x00\x81\x03\x03>\x99\xEF\xEF\xEB\xEC\xC3\x80\x02\xA9\xD7e\xEC\xE1)\xEDS\xA9\xCE\xB63\x92P\xE2\xF9db\x02{\x1F\xDF\xA2\x00\x00 \xC0/\xC00\xC0+\xC0,\xCC\xA8\xCC\xA9\xC0\x13\xC0\x09\xC0\x14\xC0" 400 150 "-" "-" "-"
51.79.29.48 - - [28/Nov/2022:02:54:24 +0000] "POST / HTTP/1.1" 405 552 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36" "-"
51.79.29.48 - - [28/Nov/2022:02:54:24 +0000] "GET /.env HTTP/1.1" 200 557 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36" "-"
185.254.196.223 - - [28/Nov/2022:02:55:11 +0000] "GET /.env HTTP/1.1" 200 557 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36" "-"
185.254.196.223 - - [28/Nov/2022:02:55:11 +0000] "POST / HTTP/1.1" 405 552 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36" "-"
172.105.89.161 - - [28/Nov/2022:03:20:23 +0000] "\xBA\xABd\xA1EZC\xDBM\x87\xEE^\xFD\xBF\x159 X\xD4>\x12\x98\xC4<\xE0\x13\xCF\x00\xAC\xA09\xD7\x90#8~\x8C\xDE\x9DReF\xBF%1Q\xE0\x9D\x06&g\xBB\x82\x95\x19\xED\x07\x14\x19ZP\x80+\x94e\xC3\xE6\x85\x06\xA4\x99\x8B\x19l\x01\xEA\x88Y\x91\x16\x95\xC4\xC8\x0EH\x02\xC7\x93g\xC14FW\x05|\xFB\xF3T\xB8\xFD\xCB\xBB)\xE3\xCE\xDD\xCD7\x9E\xEFP\x8C\xA4[V\xFD\x98\xC9l\x82\xF5\xE4\xC1d\x87X\xF7\x9B\xBF\xE8q\x12\x99&\xDB,\xF5\x87\xD7\xA8\x97j;\xE3\xEA\xA7\xB4\xB0\x02\xAD\x8DE\x9B\xAAB\x80\x0E)\xA9\xE9\xAF}\x18\x8E\xB8\x1E\x99\x04\xEF\xA8\x8C\xE8\x04\xE2\xD3\xED)1\x91\xC1\x8F\x88\x8C\x81\xF0\xDB\xA5\x88\x95H\x9BZ\xAB\xCE\xBF\xF4E%P*\x88KFY6\x9E\xE7::j\xD4\x8A\xA8V\x9A\xAA\xAB\xAF\xC3&.\xED[\x04\xC5e\x7F\x08\xBE\x8Ar\xA7\xB0\x99F\xF7\x11\xE5\xD6\x96\x8CIm+w\x1C\xFDuU\x14\x0F!x\xAC\xE8MPy\xC3\x19!2\xA0\xED\xC0}!Rw\x14\x8E\x1B\xC4\xE1\xA0\xAF+\xADKk\xC5\xE0\x5Cs\x9C\xBD\xCB" 400 150 "-" "-" "-"
84.21.172.128 - - [28/Nov/2022:03:24:05 +0000] "POST /boaform/admin/formLogin HTTP/1.1" 405 150 "http://147.182.131.129:80/admin/login.asp" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0" "-"
84.21.172.128 - - [28/Nov/2022:03:24:05 +0000] "" 400 0 "-" "-" "-"
192.241.211.240 - - [28/Nov/2022:03:35:09 +0000] "GET / HTTP/1.1" 200 557 "-" "Mozilla/5.0 zgrab/0.x" "-"
66.240.205.34 - - [28/Nov/2022:03:47:27 +0000] "145.ll|'|'|SGFjS2VkX0Q0OTkwNjI3|'|'|WIN-JNAPIER0859|'|'|JNapier|'|'|19-02-01|'|'||'|'|Win 7 Professional SP1 x64|'|'|No|'|'|0.7d|'|'|..|'|'|AA==|'|'|112.inf|'|'|SGFjS2VkDQoxOTIuMTY4LjkyLjIyMjo1NTUyDQpEZXNrdG9wDQpjbGllbnRhLmV4ZQ0KRmFsc2UNCkZhbHNlDQpUcnVlDQpGYWxzZQ==12.act|'|'|AA==" 400 150 "-" "-" "-"
Some of these request also appear in Django logs
Not Found: /portal/redlion
Not Found: /portal/redlion
Not Found: /admin.php
Not Found: /user/15751/
Not Found: /user/15761
Not Found: //script/.env
Not Found: //script/.env
Not Found: //admin-app/.env
Not Found: //admin-app/.env
Not Found: /portal/redlion
Not Found: /administrator/db/index.php
Not Found: /administratorindex.php
Not Found: /administrator/db/index.php
Not Found: /administrator/phpMyAdmin/index.php
Not Found: /administrator/db/index.php
Not Found: /administratorindex.php
Not Found: /administrator/web/index.php
Not Found: /administrator/PMA/index.php
Not Found: /administrator/db/index.php
Not Found: /administrator/db/index.php
Not Found: /administrator/db/index.php
Not Found: /admin/login.asp
Not Found: /portal/redlion
The Nginx works on the IP ADDR without a domain
nginx.conf
upstream websocket {
server asgiserver:9000;
}
error_log /var/log/nginx/nginx_error.log warn;
server {
listen 80;
server_name xxx.xxx.xxx.xx;
server_tokens off;
error_log /var/log/nginx/nginx_error.log warn;
client_max_body_size 100M;
client_body_timeout 300s;
proxy_set_header X-Forwarded-Proto https;
location ~ /.well-known {
root /var/www/certbot;
}
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html;
}
location /script {
try_files $uri #proxy_api;
}
location /auth {
try_files $uri #proxy_api;
}
location /user {
try_files $uri #proxy_api;
}
location /portal {
try_files $uri #proxy_api;
}
location /admin {
try_files $uri #proxy_api;
}
location #proxy_api {
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://backend:8000;
}
location /django_static/ {
autoindex on;
alias /app/backend/server/django_static/;
}
location /media {
autoindex on;
alias /app/backend/server/media;
}
location #proxy_websocket {
proxy_set_header Host $http_host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Forwarded-For $remote_addr;
# proxy_set_header Connection $connection_upgrade;
proxy_redirect off;
resolver 10.108.0.2;
proxy_pass http://websocket;
proxy_connect_timeout 7d;
proxy_send_timeout 7d;
proxy_read_timeout 7d;
proxy_buffers 512 256M;
proxy_buffer_size 256M;
}
}
Could the server be hacked? Especially since the rate of Ram consumption increased exaggeratedly
In reaction to the last comment in the question:
"How can I ignore them in Nginx, it's hard to define a location for each endpoint in the log above"
My comment was make under the impression that you have 1 container running Nginx and another one running Django.
The Django container might have a separate (sub)domainname. Then Nginx will only pass on requests to Django is the domainname matches, not only the IP.
In your case you have the server listening to the IP address, which gives a match for all kinds of garbage based on the IP.
My nginx server (using django) is getting hit with thousands of these types of requests per second:
199.127.61.178 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 500 186 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (Linux; U; Android 2.3.5; en-in; Micromax A87 Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"
104.238.222.87 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 400 55440 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (Linux; Android 8.0.0; SM-G950F Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Mobile Safari/537.36"
172.93.110.55 - - [09/Nov/2022:08:20:42 +0000] "GET http://tucgd.lixil-kitchen.cn/ HTTP/1.1" 400 55373 "http://tucgd.lixil-kitchen.cn" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/537.86.4"
104.243.37.94 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55205 "http://you.br-sx.com" "Mozilla/5.0 (iPhone; CPU iPhone OS 11_2 like Mac OS X) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0 Mobile/15C114 Safari/604.1"
104.238.222.9 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?nf91C2a99VqP4D43fy6uPrgt0 HTTP/1.1" 400 55722 "https://skype.gmw.cn" "Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; de-de) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5"
104.238.205.70 - - [09/Nov/2022:08:20:42 +0000] "GET http://eqksp.drtjy.com/ HTTP/1.1" 400 55224 "http://eqksp.drtjy.com" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36"
103.195.103.32 - - [09/Nov/2022:08:20:42 +0000] "GET http://eqksp.drtjy.com/ HTTP/1.1" 400 55192 "http://eqksp.drtjy.com" "Mozilla/5.0 (Windows NT 5.1; rv:40.0) Gecko/20100101 Firefox/40.0"
104.243.37.94 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55133 "http://you.br-sx.com" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0"
173.208.239.195 - - [09/Nov/2022:08:20:42 +0000] "GET http://eqksp.drtjy.com/ HTTP/1.1" 500 588 "http://eqksp.drtjy.com" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.63 Safari/537.36"
104.238.205.70 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55147 "http://you.br-sx.com" "Mozilla/5.0 (Windows NT 5.1; rv:33.0) Gecko/20100101 Firefox/33.0"
104.238.205.70 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?7n62R0Ocp5h8ymbM74co76w370m0Cv HTTP/1.1" 400 55741 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"
104.238.222.9 - - [09/Nov/2022:08:20:42 +0000] "GET http://tucgd.lixil-kitchen.cn/ HTTP/1.1" 500 588 "http://tucgd.lixil-kitchen.cn" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 400 55372 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0"
103.195.103.32 - - [09/Nov/2022:08:20:42 +0000] "GET http://tucgd.lixil-kitchen.cn/ HTTP/1.1" 400 55366 "http://tucgd.lixil-kitchen.cn" "Mozilla/5.0 (iPad; CPU OS 10_3_3 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) CriOS/61.0.3163.73 Mobile/14G60 Safari/602.1"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?DnV7mPJ19L1Li6bwt39aVP59oDDi6bJxPb8Pj0 HTTP/1.1" 400 55775 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:59.0) Gecko/20100101 Firefox/59.0"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?3as00v0ydeeRx5sXVa3wMoQ6 HTTP/1.1" 400 55676 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
104.194.8.226 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55219 "http://you.br-sx.com" "Mozilla/5.0 (Linux; Android 6.0.1; SM-G532M Build/MMB29T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 400 55434 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
104.243.37.94 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?I5RaB0sIBAt7W9i7iWueXU9104kJ HTTP/1.1" 400 55714 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
104.194.8.226 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55238 "http://you.br-sx.com" "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/14A456 Safari/602.1"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET http://px9i1.jntmzg.cn/ HTTP/1.1" 400 55245 "http://px9i1.jntmzg.cn" "Mozilla/5.0 (Linux; Android 8.0.0; FLA-LX3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.80 Mobile Safari/537.36"
...
199.127.60.99 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?s8023k1FRBDIvK6Rxw6q8h0e5S HTTP/1.1" 502 166 "https://d518b.com" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0/Nutch-1.12"
185.150.189.223 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?xCtRIk9u13N80G1J8xaWTF1GSLo80M6 HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"
104.238.221.227 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?25HwH49w9C9UqapejfQ3HQCX02EKbegWgvG4 HTTP/1.1" 502 166 "https://d518b.com" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/600.3.18 (KHTML, like Gecko) Version/8.0.3 Safari/600.3.18"
104.243.34.218 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?v8iHIV5uWx4540GvN4apQ3dG3 HTTP/1.1" 502 166 "https://d518b.com" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0"
103.195.103.32 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?4FrrkvoMux8HR162L324b2 HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; InfoPath.1)"
199.127.63.156 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?A5cAxLa81Q52KCL752QK010X3NuQ HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/5.0 (Linux; NetCast; U) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.31 SmartTV/7.5"
103.195.103.32 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?G2nga9Dw9q1Xy9bR7qBXB HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)"
They all seem to requests external URLs, (which I don't think my server needs to do for any legitimate reason), so I tried to deny them with my nginx config by denying matches to http, .com, cn, with this config:
upstream app_server {
server unix:/home/django/gunicorn.socket fail_timeout=0;
}
limit_req_zone $binary_remote_addr zone=one:10m rate=10r/s;
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
client_max_body_size 4G;
server_name _;
keepalive_timeout 5;
# Your Django project's media files - amend as required
location /media {
alias /home/django/django_project/django_project/media;
}
# your Django project's static files - amend as required
location /static {
alias /home/django/django_project/django_project/static;
}
# Proxy the static assests for the Django Admin panel
location /static/admin {
alias /usr/lib/python3/dist-packages/django/contrib/admin/static/admin/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://app_server;
limit_req zone=one;
limit_except GET HEAD POST {
deny all;
}
}
# ADDED THESE NEW LINES
location ~ http {
deny all;
}
location ~ .com {
deny all;
}
location ~ .cn {
deny all;
}
}
server {
root /usr/share/nginx/html;
index index.html index.htm;
client_max_body_size 4G;
server_name <MYURLHIDDEN>; # managed by Certbot
keepalive_timeout 5;
# Your Django project's media files - amend as required
location /media {
alias /home/django/django_project/django_project/media;
}
# your Django project's static files - amend as required
location /static {
alias /home/django/django_project/django_project/static;
}
# Proxy the static assests for the Django Admin panel
location /static/admin {
alias /usr/lib/python3/dist-packages/django/contrib/admin/static/admin/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://app_server;
limit_req zone=one;
limit_except GET HEAD POST {
deny all;
}
}
# ADDED THESE NEW LINES
location ~ http {
deny all;
}
location ~ .com {
deny all;
}
location ~ .cn {
deny all;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/<MYURLHIDDEN>/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/<MYURLHIDDEN>/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = <MYURLHIDDEN>) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80 ;
listen [::]:80 ;
server_name <MYURLHIDDEN>;
return 404; # managed by Certbot
}
But these types of requests are still hitting my access log, and my servers CPU keeps hitting 100% and site keeps going down. What am I doing wrong?
My website under Route 53 and ALB was flooded once on 12 May but seemed AWS Shield Standard version (free) didn't do anything to prevent?
Showing 1000 of 9,828,102 records matched:
2022-05-12T08:01:25.024+08:00 51.15.0.133 - - [12/May/2022:00:01:24 +0000] "GET http://www.1980mu.com:89/ HTTP/1.1" 200 8216 "-" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36" "-"
2022-05-12T08:01:25.024+08:00 51.15.0.133 - - [12/May/2022:00:01:24 +0000] "GET http://www.1980mu.com:89/ HTTP/1.1" 200 8216 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"
2022-05-12T08:01:25.024+08:00 51.15.0.133 - - [12/May/2022:00:01:24 +0000] "GET http://www.1980mu.com:89/ HTTP/1.1" 200 8216 "-" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36" "-"
2022-05-12T08:01:25.024+08:00 51.15.0.133 - - [12/May/2022:00:01:24 +0000] "GET http://www.1980mu.com:89/ HTTP/1.1" 200 8216 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3599.0 Safari/537.36" "-"
2022-05-12T08:01:25.024+08:00 51.15.0.133 - - [12/May/2022:00:01:24 +0000] "GET http://www.1980mu.com:89/ HTTP/1.1" 200 8216 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; rv:11.0) like Gecko" "-"
enter image description here
Shield standard only protects Layer 3 and 4 of your application network stack - so its expected it would allow valid web traffic through. You would need to use Shield Advanced or WAF to gain greater control and Cloud Front to provide a caching layer (all paid services) to better protect your instances.
My advice is to follow the principle of least privilege at each layer in terms of firewall ports open and what hostnames you allow. You can use rate limiting via WAF to avoid getting flooded, and using CloudFront to intercept requests and return cached responses where possible to reduce load on your instances.
I have installed airflow2.1.1 on centos7 vm in our private network. Airflow webserver is built on flask. It uses flask web authentication.
I have configured airflow.cfg and webserver_config.py to use keycloak oauth2.
I am getting below error while logging in airflow web UI with keycloak for authentication
OAUTH code in webserver_config.py
from airflow.www_rbac.security import AirflowSecurityManager
from flask_appbuilder.security.manager import AUTH_OAUTH
import os
import json
AUTH_TYPE = AUTH_OAUTH
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = "Admin"
# a mapping from the values of `userinfo["role_keys"]` to a list of FAB roles
AUTH_ROLES_MAPPING = {
"FAB_USERS": ["admin"],
"FAB_ADMINS": ["Admin"],
}
OAUTH_PROVIDERS = [
{'name':'keycloak', 'icon':'fa-user-circle', 'token_key':'access_token',
'remote_app': {
'client_id':'xxxxxx',
'client_secret':'xxxxxxxxxxxxxxxxxxx',
'api_base_url':'https://keycloak-1.dastc.stee.com:8443/auth/realms/sep',
'client_kwargs':{
'scope': 'email profile'
},
'request_token_url':None,
'access_token_url':'https://keycloak-1.dastc.stee.com:8443/auth/realms/sep/protocol/openid-connect/token',
'authorize_url':'https://keycloak-1.dastc.stee.com:8443/auth/realms/sep/protocol/openid-connect/auth'}
}
]
Error from airflow webserver
172.16.0.1 - - [26/Aug/2021:07:28:25 +0000] "GET / HTTP/1.1" 302 217 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:25 +0000] "GET /home HTTP/1.1" 302 329 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:25 +0000] "GET /login/?next=https%3A%2F%2Fexp-3.dastc.stee.com%3A8090%2Fhome HTTP/1.1" 302 1001 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:25 +0000] "GET / HTTP/1.1" 302 217 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:25 +0000] "GET /home HTTP/1.1" 302 329 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:25 +0000] "GET /login/?next=https%3A%2F%2Fexp-3.dastc.stee.com%3A8090%2Fhome HTTP/1.1" 302 1001 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:43,501] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:43 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJuZXh0IjpbImh0dHBzOi8vZXhwLTMuZGFzdGMuc3RlZS5jb206ODA5MC9ob21lIl19.gPk8CRlCWQtxpWemGEK575Q-0t_r488fczc1lDbVjsQ&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=5732a033-009f-4093-80d3-43321c5c646e.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:43 +0000] "GET /login/ HTTP/1.1" 302 871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:43,734] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:43 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=eec30213-f238-445a-b3f9-22db5091337d.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:43 +0000] "GET /login/ HTTP/1.1" 302 871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:43,972] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:43 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=384d3d06-dc9c-4478-bdea-917db0456d09.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /login/ HTTP/1.1" 302 871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:44,190] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=2561b99b-a098-4927-b8fe-5b4a4548b62f.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /login/ HTTP/1.1" 302 871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:44,431] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=a6486875-ce53-40d8-b9ed-461453a942e0.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /login/ HTTP/1.1" 302 871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:44,677] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=2710fb5a-f592-45d1-ade5-2dd7bbc3ca3e.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /login/ HTTP/1.1" 302 871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:44,920] {manager.py:1293} ERROR - OAUTH userinfo does not have username or email {}
172.16.0.1 - - [26/Aug/2021:07:28:44 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=1c7750d0-9517-4ba7-8c7c-6ff96fa4588a.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 302 221 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
[2021-08-26 07:28:46,128] {app.py:1892} ERROR - Exception on /oauth-authorized/keycloak [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.6/site-packages/flask_appbuilder/security/views.py", line 695, in oauth_authorized
resp = self.appbuilder.sm.oauth_remotes[provider].authorize_access_token()
File "/usr/local/lib/python3.6/site-packages/authlib/integrations/flask_client/remote_app.py", line 74, in authorize_access_token
params = self.retrieve_access_token_params(flask_req, request_token)
File "/usr/local/lib/python3.6/site-packages/authlib/integrations/base_client/base_app.py", line 145, in retrieve_access_token_params
params = self._retrieve_oauth2_access_token_params(request, params)
File "/usr/local/lib/python3.6/site-packages/authlib/integrations/base_client/base_app.py", line 126, in _retrieve_oauth2_access_token_params
raise MismatchingStateError()
authlib.integrations.base_client.errors.MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response.
172.16.0.1 - - [26/Aug/2021:07:28:46 +0000] "GET /oauth-authorized/keycloak?state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.e30.SqKPp4eMJlY0wyPJxubLV-L78CpXKgpfm2ilTlgecSg&session_state=c50b49a9-2878-4af6-8ce4-62da7b3a82fd&code=1c7750d0-9517-4ba7-8c7c-6ff96fa4588a.c50b49a9-2878-4af6-8ce4-62da7b3a82fd.5e50dd12-2de1-479c-906f-d5272f3ba911 HTTP/1.1" 500 2447 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
I'm currently deploying the Django & React Web on the Elastic Beanstalk, but I still get the 500 error. Is there any method to get rid of this error? I use PostgreSQL, and I grepped only 500 errors down below.
Here's my git repository:
https://github.com/ujin43255252/davidgram
172.31.1.112 (14.231.228.185) - - [18/Jul/2018:12:02:48 +0000] "GET / HTTP/1.1" 500 527 "-" "-"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:17:10 +0000] "GET /admin HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:17:10 +0000] "GET /favicon.ico HTTP/1.1" 500 527 "http://davidgram.ap-northeast-2.elasticbeanstalk.com/admin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:18:29 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:18:29 +0000] "GET /favicon.ico HTTP/1.1" 500 527 "http://davidgram.ap-northeast-2.elasticbeanstalk.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.29.137 (31.184.194.109) - - [18/Jul/2018:12:18:33 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
127.0.0.1 (-) - - [18/Jul/2018:12:25:07 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:12:25:08 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:12:25:09 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:12:25:10 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:25:57 +0000] "GET /robots.txt HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:26:00 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:12:26:00 +0000] "GET /favicon.ico HTTP/1.1" 500 527 "http://davidgram.ap-northeast-2.elasticbeanstalk.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.29.137 (85.69.223.3) - - [18/Jul/2018:12:30:18 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
172.31.29.137 (85.69.223.3) - - [18/Jul/2018:12:30:24 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
172.31.29.137 (203.189.153.192) - - [18/Jul/2018:12:38:27 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
172.31.29.137 (209.126.136.4) - - [18/Jul/2018:12:54:02 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
172.31.29.137 (164.177.41.204) - - [18/Jul/2018:12:54:15 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
172.31.1.112 (209.126.136.4) - - [18/Jul/2018:13:04:00 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:13:10:59 +0000] "GET /robots.txt HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (143.248.234.146) - - [18/Jul/2018:13:10:59 +0000] "GET /favicon.ico HTTP/1.1" 500 527 "http://davidgram.ap-northeast-2.elasticbeanstalk.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.1.112 (177.189.145.3) - - [18/Jul/2018:13:14:41 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
172.31.29.137 (47.203.88.236) - - [18/Jul/2018:13:24:38 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.28) Gecko/20120306 Firefox/3.6.28 (.NET CLR 3.5.30729)"
172.31.29.137 (31.184.194.109) - - [18/Jul/2018:13:37:56 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
172.31.29.137 (203.190.43.78) - - [18/Jul/2018:13:56:50 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
172.31.1.112 (179.55.191.177) - - [18/Jul/2018:14:38:19 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
172.31.1.112 (143.208.246.121) - - [18/Jul/2018:15:02:31 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
127.0.0.1 (-) - - [18/Jul/2018:15:19:05 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:15:19:07 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:15:19:08 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:15:19:09 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
172.31.29.137 (37.26.87.166) - - [18/Jul/2018:15:22:44 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
172.31.29.137 (70.51.79.246) - - [18/Jul/2018:15:34:30 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
172.31.29.137 (70.51.79.246) - - [18/Jul/2018:15:34:35 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
172.31.29.137 (103.255.74.132) - - [18/Jul/2018:15:34:52 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
172.31.29.137 (37.63.239.222) - - [18/Jul/2018:15:42:01 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7"
127.0.0.1 (-) - - [18/Jul/2018:15:44:02 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:15:44:04 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:15:44:05 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
127.0.0.1 (-) - - [18/Jul/2018:15:44:06 +0000] "GET / HTTP/1.1" 500 527 "-" "Python-urllib/2.7"
172.31.29.137 (143.248.234.146) - - [18/Jul/2018:15:44:26 +0000] "GET /robots.txt HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.29.137 (143.248.234.146) - - [18/Jul/2018:15:44:27 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.29.137 (143.248.234.146) - - [18/Jul/2018:15:44:28 +0000] "GET /favicon.ico HTTP/1.1" 500 527 "http://davidgram.ap-northeast-2.elasticbeanstalk.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.29.137 (143.248.234.146) - - [18/Jul/2018:15:44:29 +0000] "GET / HTTP/1.1" 500 527 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
172.31.29.137 (143.248.234.146) - - [18/Jul/2018:15:44:29 +0000] "GET /favicon.ico HTTP/1.1" 500 527 "http://davidgram.ap-northeast-2.elasticbeanstalk.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
I suspect if this is one of your first deployments, that you simply have not added the Elastic Beanstalk hostname to ALLOWED_HOSTS in your settings.py.
But without the actual error logs, that is hard to be sure of. Using the Elastic Beanstalk console, you should be able to click "request logs" and then request the last 100 lines. One of the logfiles you will get from that is the error log, that has more detailed debugging information.