Hi can someone explain me why last octet of the IP if 01 or 001 is not capched by this regex ?
(\.?)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9][0-9]?)($|\.)
Debuggex Demo
as example of the code
badOctedIPv4 := "(\\.?)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9][0-9]?)($|\\.)"
ipv4Format := badOctedIPv4
matchMe := regexp.MustCompile(ipv4Format)
return matchMe.FindString(input)
the input data looks like:
10.185.248.71 - - [09/Jan/2015:19:12:06 +0000] 808840 "GET /inventoryService/inventory/purchaseItem?userId=20253471&itemId=23434300 HTTP/1.1" 500 17 "-" "Apache-HttpClient/4.2.6 (java 1.5)"
[Thu Mar 13 19:04:13 2014] [error] [client 50.0.134.125] File does not exist: /var/www/favicon.ico
192.168.000.254 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 10 bad
092.168.000.254 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 9 bad
123.234.345.001 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 8 bad
123.234.145.001 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 7 bad
345.234.123.1 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 6 bad
092.168.72.177 - - [22/Dec/2002:23:32:14 -0400] "GET /favicon.ico HTTP/1.1" 404 1997 www.yahoo.com "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3)..." "-" 5 bad
123.234.145.001 - - 4 bad
123.234.145.01 - - 3 bad
123.234.05.100 - - 2 bad
123.234.005.100 - - 1 bad
123.234.5.100 - - Last entry
the results returned by above code only finds all bad IP octets except the last one 001 or 01
Output of the program:
❯ go run ./findInvalidIPv4.go logfile.log
[192.168.000.254] : [.000.] : 192.168.000.254 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 10 bad
[092.168.000.254] : [ 092.] : 092.168.000.254 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 9 bad
[123.234.345.001] : [.345.] : 123.234.345.001 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 8 bad
[ 345.234.123.1] : [ 345.] : 345.234.123.1 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 6 bad
[ 092.168.72.177] : [ 092.] : 092.168.72.177 - - [22/Dec/2002:23:32:14 -0400] "GET /favicon.ico HTTP/1.1" 404 1997 www.yahoo.com "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3)..." "-" 5 bad
[ 123.234.05.100] : [ .05.] : 123.234.05.100 - - 2 bad
[123.234.005.100] : [.005.] : 123.234.005.100 - - 1 bad
Output explained:
first column [...] its the full bad IP where bad octet been found
second column [...] its the bad octet ... first match is enough
third column is the full line passed to above func
Can some one point me what I am missing and why the 001 at the end is not matching the pattern ?
Thanks
Your group 3 at the end:
($|\.)
Insists on either a dot or end-of-line character appearing after the last octet. That's fine for the first three octets that are guaranteed to have a . proceed it. But it won't work for the last one.
The simple fix is to just remove it or make it optional:
(\.?)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9][0-9]?)($|\.?)
Add a whitespace for group 3:
(\.?)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9][0-9]?)(\s|$|\.)
Or just remove it:
(\.?)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9][0-9]?)
All of these have issues. So maybe this is what you really want is to match any of your 3 digit sequence with either a leading dot or a trailing dot.
\.[2-9][5-9][6-9]|\.[3-9][0-9][0-9]|\.0[0-9][0-9]|\[2-9][5-9][6-9]\.|[3-9][0-9][0-9]\.|0[0-9][0-9]\.
We start to get into regular expressions being "Write once read never again" territory.
#selbie thanks again for your help seems with all suggestions here i am getting closer to solve this, this regex
(\.|^)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9]+) seems its catching for me almost all what needed
[ 192.168.2.001] : [ .001] : 192.168.2.001 - - [28/Jul/2006:10:27:10 -0300] "GET /cgi-bin/try/ HTTP/1.0" 200 3395
[192.168.000.254] : [ .000] : 192.168.000.254 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 10 bad
[092.168.000.254] : [ 092] : 092.168.000.254 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 9 bad
[123.234.345.001] : [ .345] : 123.234.345.001 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 8 bad
[123.234.145.001] : [ .001] : 123.234.145.001 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 7 bad
[ 345.234.123.1] : [ 345] : 345.234.123.1 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 6 bad
[ 300.234.123.1] : [ 300] : 300.234.123.1 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 6 bad
[300.300.300.300] : [ 300] : 300.300.300.300 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587 6 bad
[ 092.168.72.177] : [ 092] : 092.168.72.177 - - [22/Dec/2002:23:32:14 -0400] "GET /favicon.ico HTTP/1.1" 404 1997 www.yahoo.com "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3)..." "-" 5 bad
[123.234.145.001] : [ .001] : 123.234.145.001 - - 4 bad
[ 123.234.145.01] : [ .01] : 123.234.145.01 - - 3 bad
[ 123.234.05.100] : [ .05] : 123.234.05.100 - - 2 bad
[123.234.005.100] : [ .005] : 123.234.005.100 - - 1 bad
and its skipping the good IP like 200.200.200.200 or 100.100.100.100
so we are getting closer to get that pattern working the only case now when i see is messed is when i have time string, 02:49:12 which starts the string 02 and so on as example:
[ 127.0.0.1] : [ 02] : 02:49:12 127.0.0.1 GET / 200
[ 127.0.0.1] : [ 02] : 02:49:35 127.0.0.1 GET /index.html 200
[ 127.0.0.1] : [ 03] : 03:01:06 127.0.0.1 GET /images/sponsered.gif 304
[ 127.0.0.1] : [ 03] : 03:52:36 127.0.0.1 GET /search.php 200
[ 127.0.0.1] : [ 04] : 04:17:03 127.0.0.1 GET /admin/style.css 200
[ 127.0.0.1] : [ 05] : 05:04:54 127.0.0.1 GET /favicon.ico 404
[ 127.0.0.1] : [ 05] : 05:38:07 127.0.0.1 GET /js/ads.js 200
so i am still looking for an answer what i am missing in that regular expression
================================
edit
ok this seems to do the work and its able to find the bad ip octet
(\.|^)([2-9][5-9][6-9]|[3-9][0-9][0-9]|0[0-9]+)([^:/-])
added the lat 3rd group ([^:/-]) to exclude any time format with two digits
Related
an instance was taken out of service in response to a ELB system
health check failure.
I hit the health check endpoint with my browser and it returns fine, but I'm getting the above message.
How can I debug this?
I've looked at instant settings => Get System Logs and nginx logs,
edit
nginx has
- [27/Mar/2020:05:35:42 +0000] "GET /littlehome/heartbeat/ HTTP/1.1" 200 2 2.920 2.920 "-" "ELB-HealthChecker/2.0"
- [27/Mar/2020:05:35:42 +0000] "GET /littlehome/heartbeat/ HTTP/1.1" 200 2 2.858 2.856 "-" "ELB-HealthChecker/2.0"
it returned 200 for sure..
and still aws think it received 502
{
"Target": {
"Id": "i-085e8dffe8781f876",
"Port": 80
},
"HealthCheckPort": "80",
"TargetHealth": {
"State": "unhealthy",
"Reason": "Target.ResponseCodeMismatch",
"Description": "Health checks failed with these codes: [502]"
}
},
Based on the comments, the issue was that grace period in Auto Scaling Group was too short. The solution was to increase it.
I'm using axios with Vue in my project. While under development (npm run serve), it's worked correctly. But after built, any request after 'PUT' request will have "{}" in front of http method, such as "{}GET", "{}PUT".
as I've tested, I've got this
state diagram.
// CASE 1 : OK
axios.put('someurl') // HTTP_METHOD is 'PUT'
axios.get('someurl') // HTTP_METHOD is 'PUT'
// CASE 2 : HAS CURLY BRACKET IN METHOD
axios.put('someurl', {}) // HTTP_METHOD is 'PUT'
axios.put('someurl', {}) // HTTP_METHOD is '{}PUT'
axios.post('someurl', {}) // HTTP_METHOD is '{}POST'
axios.delete('someurl', {}) // HTTP_METHOD is '{}DELETE'
axios.get('someurl', {}) // HTTP_METHOD is 'GET'
this is some logs from django
[28/May/2019 20:45:15] "GET /api/colla/comments/177/notify/ HTTP/1.1" 200 4
[28/May/2019 20:45:16] "PUT /api/colla/comments/177/notify/ HTTP/1.1" 200 5
[28/May/2019 20:45:19] "{}POST /api/colla/comments/177/notify/ HTTP/1.1" 405 43
[28/May/2019 20:45:25] "{}POST /api/colla/comments/177/notify/ HTTP/1.1" 405 43
[28/May/2019 20:45:30] "{}PUT /api/colla/comments/177/notify/ HTTP/1.1" 405 42
[28/May/2019 20:45:36] "{}DELETE /api/colla/comments/177/notify/ HTTP/1.1" 405 45
[28/May/2019 20:45:39] "DELETE /api/colla/comments/177/notify/ HTTP/1.1" 200 4
I wrote a JAX-RS web service method to run on WebLogic 12.2.1, to test how many concurrent requests it can handle. I purposely make the method take 5 minutes to execute.
#Singleton
#Path("Services")
#ApplicationPath("resources")
public class Services extends Application {
private static int count = 0;
private static synchronized int addCount(int a) {
count = count + a;
return count;
}
#GET
#Path("Ping")
public Response ping(#Context HttpServletRequest request) {
int c = addCount(1);
logger.log(INFO, "Method entered, total running requests: [{0}]", c);
try {
Thread.sleep(300000);
} catch (InterruptedException exception) {
}
c = addCount(-1);
logger.log(INFO, "Exiting method, total running requests: [{0}]", c);
return Response.ok().build();
}
}
I also wrote a stand-alone client program to send 500 concurrent requests to this service. The client uses one thread for each request.
From what I understand, WebLogic has a default maximum of 400 threads, which means that it can handle 400 requests concurrently. This figure is confirmed with my test result below. As you can see, within the first 5 minutes, starting from 10:46:31, only 400 requests were been serviced.
23/08/2016 10:46:31.393 [132] [INFO] [Services.ping] - Method entered, total running requests: [1]
23/08/2016 10:46:31.471 [204] [INFO] [Services.ping] - Method entered, total running requests: [2]
23/08/2016 10:46:31.471 [66] [INFO] [Services.ping] - Method entered, total running requests: [3]
23/08/2016 10:46:31.471 [210] [INFO] [Services.ping] - Method entered, total running requests: [4]
23/08/2016 10:46:31.471 [206] [INFO] [Services.ping] - Method entered, total running requests: [5]
23/08/2016 10:46:31.487 [207] [INFO] [Services.ping] - Method entered, total running requests: [6]
23/08/2016 10:46:31.487 [211] [INFO] [Services.ping] - Method entered, total running requests: [7]
23/08/2016 10:46:31.487 [267] [INFO] [Services.ping] - Method entered, total running requests: [8]
23/08/2016 10:46:31.487 [131] [INFO] [Services.ping] - Method entered, total running requests: [9]
23/08/2016 10:46:31.502 [65] [INFO] [Services.ping] - Method entered, total running requests: [10]
23/08/2016 10:46:31.518 [265] [INFO] [Services.ping] - Method entered, total running requests: [11]
23/08/2016 10:46:31.565 [266] [INFO] [Services.ping] - Method entered, total running requests: [12]
23/08/2016 10:46:35.690 [215] [INFO] [Services.ping] - Method entered, total running requests: [13]
23/08/2016 10:46:35.690 [269] [INFO] [Services.ping] - Method entered, total running requests: [14]
23/08/2016 10:46:35.690 [268] [INFO] [Services.ping] - Method entered, total running requests: [15]
23/08/2016 10:46:35.690 [214] [INFO] [Services.ping] - Method entered, total running requests: [16]
23/08/2016 10:46:35.690 [80] [INFO] [Services.ping] - Method entered, total running requests: [17]
23/08/2016 10:46:35.690 [79] [INFO] [Services.ping] - Method entered, total running requests: [18]
23/08/2016 10:46:35.690 [152] [INFO] [Services.ping] - Method entered, total running requests: [19]
23/08/2016 10:46:37.674 [158] [INFO] [Services.ping] - Method entered, total running requests: [20]
23/08/2016 10:46:37.674 [155] [INFO] [Services.ping] - Method entered, total running requests: [21]
23/08/2016 10:46:39.674 [163] [INFO] [Services.ping] - Method entered, total running requests: [22]
23/08/2016 10:46:39.705 [165] [INFO] [Services.ping] - Method entered, total running requests: [23]
23/08/2016 10:46:39.705 [82] [INFO] [Services.ping] - Method entered, total running requests: [24]
23/08/2016 10:46:39.705 [166] [INFO] [Services.ping] - Method entered, total running requests: [25]
23/08/2016 10:46:41.690 [84] [INFO] [Services.ping] - Method entered, total running requests: [26]
23/08/2016 10:46:41.690 [160] [INFO] [Services.ping] - Method entered, total running requests: [27]
23/08/2016 10:46:43.690 [226] [INFO] [Services.ping] - Method entered, total running requests: [28]
23/08/2016 10:46:43.705 [162] [INFO] [Services.ping] - Method entered, total running requests: [29]
....
....
23/08/2016 10:50:52.008 [445] [INFO] [Services.ping] - Method entered, total running requests: [398]
23/08/2016 10:50:52.008 [446] [INFO] [Services.ping] - Method entered, total running requests: [399]
23/08/2016 10:50:54.008 [447] [INFO] [Services.ping] - Method entered, total running requests: [400]
23/08/2016 10:51:31.397 [132] [INFO] [Services.ping] - Exiting method, total running requests: [399]
23/08/2016 10:51:31.475 [207] [INFO] [Services.ping] - Exiting method, total running requests: [398]
23/08/2016 10:51:31.475 [207] [INFO] [Services.ping] - Method entered, total running requests: [399]
....
....
But what I don't understand is how come the first 400 requests were not serviced at the same time by the service method? As you can see from the test result, the first request was serviced at 10:46:31.393, but the 400th request was serviced at 10:50:54.008, which is more than 4 minutes later.
If we look at access.log, we can see that all 500 requests were received by WebLogic between 10:46:31 and 10:46:35. So it seems that even though WebLogic received the requests with a very short period of time, it doesn't allocate a thread and call the service method that fast.
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:31 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
....
....
10.204.133.176 - - [23/Aug/2016:10:46:35 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:35 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:35 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
10.204.133.176 - - [23/Aug/2016:10:46:35 +0800] "GET /Test/Services/Ping HTTP/1.1" 200 0
EDITED
Added work manager to define a minimum of 400 threads.
weblogic.xml
<wls:work-manager>
<wls:name>HighPriorityWorkManager</wls:name>
<wls:fair-share-request-class>
<wls:name>HighPriority</wls:name>
<wls:fair-share>100</wls:fair-share>
</wls:fair-share-request-class>
<wls:min-threads-constraint>
<wls:name>MinThreadsCount</wls:name>
<wls:count>400</wls:count>
</wls:min-threads-constraint>
</wls:work-manager>
web.xml
<init-param>
<param-name>wl-dispatch-policy</param-name>
<param-value>HighPriorityWorkManager</param-value>
</init-param>
That's how weblogic scales threadpools (they are "self-tuning"), it does not start 400 Threads immediately. It's more a slow increase of threads to maximize throughput.
https://docs.oracle.com/cd/E24329_01/web.1211/e24432/self_tuned.htm#CNFGD113
I'm so confused!
I set everything up, my site was working for two days, and then suddenly today it stops working.
The only thing I changed was yesterday I was trying to serve PHP files so I installed PHP and uwsgi. It was late and I didn't realize what I was doing. It was from this website: http://uwsgi-docs.readthedocs.org/en/latest/PHP.html
# Add ppa with libphp5-embed package
sudo add-apt-repository ppa:l-mierzwa/lucid-php5
# Update to use package from ppa
sudo apt-get update
# Install needed dependencies
sudo apt-get install php5-dev libphp5-embed libonig-dev libqdbm-dev
# Compile uWSGI PHP plugin
python uwsgiconfig --plugin plugins/php
But didn't change any settings. Even after doing that, everything was still fine. However the next day, my site just doesn't load.
I tried a few things which didn't work. In my settings:
ALLOWED_HOSTS = ['*']
In my gunicorn.sh, I set TIMEOUT=60. However, when I try to access my site (lewischi.com), nothing even happens. But when I go to http://127.0.0.1:8000, I do see workers doing stuff and get a 404 error.
Using the URLconf defined in django_project.urls,
Django tried these URL patterns, in this order:
I'm not sure what's going on! nginx-error log isn't very helpful but the access log seems more useful.
From my nginx-access.log (it works, then stops working):
50.156.86.221 - - [25/Sep/2015:00:25:43 -0700] "GET /codeWindow.html
HTTP/1.1" 200 2081 "http://lewischi.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
50.156.86.221 - - [25/Sep/2015:00:25:58 -0700] "GET /test.jpg HTTP/1.1"
404 208 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
192.168.2.6 - - [25/Sep/2015:16:42:19 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
192.168.2.6 - - [25/Sep/2015:17:24:44 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
192.168.2.6 - - [25/Sep/2015:23:28:51 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
192.168.2.6 - - [25/Sep/2015:23:29:02 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
From my supervisor log file:
supervisor: couldn't exec /home/lewischi/projects/active/django_project/gunicorn.sh: ENOEXEC
supervisor: child process was not spawned
ANY HELP would be greatly appreciated!!!! I feel like I should just uninstall uwsgi. I don't want to break anything so I'm asking for advice before I go messing things up.
I'm pretty new to this so I may be overlooking something obvious. My gunicorn debug mode output:
“Starting ”djangotut” as lewischi”
[2015-09-26 17:50:28 +0000] [2316] [DEBUG] Current configuration:
proxy_protocol: False
worker_connections: 1000
statsd_host: None
max_requests_jitter: 0
post_fork: <function post_fork at 0x7faf049ec848>
pythonpath: None
enable_stdio_inheritance: False
worker_class: sync
ssl_version: 3
suppress_ragged_eofs: True
syslog: False
syslog_facility: user
when_ready: <function when_ready at 0x7faf049ec578>
pre_fork: <function pre_fork at 0x7faf049ec6e0>
cert_reqs: 0
preload_app: False
keepalive: 2
accesslog: None
group: 1000
graceful_timeout: 30
do_handshake_on_connect: False
spew: False
workers: 3
proc_name: ”djangotut”
sendfile: True
pidfile: None
umask: 0
on_reload: <function on_reload at 0x7faf049ec410>
pre_exec: <function pre_exec at 0x7faf049ecde8>
worker_tmp_dir: None
post_worker_init: <function post_worker_init at 0x7faf049ec9b0>
limit_request_fields: 100
on_exit: <function on_exit at 0x7faf049f2500>
config: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
proxy_allow_ips: ['127.0.0.1']
pre_request: <function pre_request at 0x7faf049ecf50>
post_request: <function post_request at 0x7faf049f20c8>
user: 1000
forwarded_allow_ips: ['127.0.0.1']
worker_int: <function worker_int at 0x7faf049ecb18>
threads: 1
max_requests: 1
limit_request_line: 4094
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
certfile: None
worker_exit: <function worker_exit at 0x7faf049f2230>
chdir: /home/lewischi/projects/active/django_project
paste: None
default_proc_name: django_project.wsgi:application
errorlog: -
loglevel: DEBUG
logconfig: None
syslog_addr: udp://localhost:514
syslog_prefix: None
daemon: False
ciphers: TLSv1
on_starting: <function on_starting at 0x7faf049ec2a8>
worker_abort: <function worker_abort at 0x7faf049ecc80>
bind: ['0.0.0.0:8000']
raw_env: []
reload: False
check_config: False
limit_request_field_size: 8190
nworkers_changed: <function nworkers_changed at 0x7faf049f2398>
timeout: 60
ca_certs: None
django_settings: None
tmp_upload_dir: None
keyfile: None
backlog: 2048
logger_class: gunicorn.glogging.Logger
statsd_prefix:
[2015-09-26 17:50:28 +0000] [2316] [INFO] Starting gunicorn 19.3.0
[2015-09-26 17:50:28 +0000] [2316] [DEBUG] Arbiter booted
[2015-09-26 17:50:28 +0000] [2316] [INFO] Listening at: http://0.0.0.0:8000 (2316)
[2015-09-26 17:50:28 +0000] [2316] [INFO] Using worker: sync
[2015-09-26 17:50:28 +0000] [2327] [INFO] Booting worker with pid: 2327
[2015-09-26 17:50:28 +0000] [2328] [INFO] Booting worker with pid: 2328
[2015-09-26 17:50:28 +0000] [2329] [INFO] Booting worker with pid: 2329
[2015-09-26 17:50:29 +0000] [2316] [DEBUG] 3 workers
[2015-09-26 17:50:30 +0000] [2316] [DEBUG] 3 workers
The problem is not with supervisord itself, few things to consider when dealing with Nginx, Gunicorn and Django in general:
Make sure the user running the app process(minimum 1 user non root not including users created by default for e.g: Nginx, Postgresql. Changes with the stack) has the right permissions and ownership to achieve it's goals.
When adding another app to your stack, you should first check the port it runs on by default, and change it to prevent port conflicts, keep in mind the difference between internal and external ports since you use Nginx as a proxy to Gunicorn(this is what causes most timeouts, happened to me several times at late night work), you can use Nginx as a proxy server and create many apps with different unique internal port for each app.
With the error log you provided for supervisor, it seems you're running your gunicorn.sh either with a user that doesn't have enough permissions or ownership, or executing with a wrong command.
Please provide the supervisor config file relevant to your app.
Update: seems his ip address changed.
Ah never mind. Thanks for your time.
It turned out that my ip address somehow changed which should not have happened.... Rookie mistake.
I have the access.log file with more than 1000 X-Forwarded-For log entries like the following
142.245.59.16, 67.69.175.224, 69.31.97.126 - - [22/Sep/2015:20:00:02 -0400] "GET /company-information/cs/null?path=%
157.55.39.76, 184.27.179.176, 165.254.1.175 - - [22/Sep/2015:20:00:05 -0400] "GET /metricstream/--ID__100325--/free-co-profile.xhtml
10.70.33.32 - - [22/Sep/2015:20:00:22 -0400] "GET /autodiscover/autodiscover.xml
172.30.152.90, 198.178.234.30, 184.27.120.46, 69.31.97.126 - - [22/Sep/2015:20:03:37 -0400] "GET /company-information/cs/null?path
with this log entries, I have to grep and extract them to the access_log.txt file like the following output
142.245.59.16 - - [22/Sep/2015:20:00:02 -0400] "GET /company-information/cs/null?path=%
157.55.39.76 - - [22/Sep/2015:20:00:05 -0400] "GET /metricstream/--ID__100325--/free-co-profile.xhtml
10.70.33.32 - - [22/Sep/2015:20:00:22 -0400] "GET /autodiscover/autodiscover.xml
172.30.152.90 - - [22/Sep/2015:20:03:37 -0400] "GET /company-information/, csnull ?path
which is to leaving the first ip as it is and remove the following two or more ips, i have also tired the REGEX : /\, .*?\ -/g but i don't know how to apply it in unix sed command. please help to solve this using Unix command
You can use this sed command:
sed 's/, [^-]*- -/ - -/' file.log
142.245.59.16 - - [22/Sep/2015:20:00:02 -0400] "GET /company-information/cs/null?path=%
157.55.39.76 - - [22/Sep/2015:20:00:05 -0400] "GET /metricstream/--ID__100325--/free-co-profile.xhtml
10.70.33.32 - - [22/Sep/2015:20:00:22 -0400] "GET /autodiscover/autodiscover.xml
172.30.152.90 - - [22/Sep/2015:20:03:37 -0400] "GET /company-information/cs/null?path
That way :
sed 's/\, .* -/ -/g' ./access.log