Fluentd parser for OpenWRT logs - regex

Now I'm working on collecting logs from many OenWRT routers and using for this task fluentd.
Unfortunately, I don't know why my parser isn't working.
For example, logs looks like this:
Aug 15 06:55:05 HOST123 daemon:warning Expiring: (mcpd_process_query_timer)
Aug 15 06:55:05 HOST123 daemon:warning Sending IGMPv2 query of len: 8
Aug 15 06:55:05 HOST123 daemon:warning MLD QUERY: MRT is 00:0a ifindex=15
Aug 15 06:55:05 HOST123 daemon:warning Sending Query size: 28
Aug 15 06:55:07 HOST123 daemon:notice iptv (8359): udhcpc: sending renew to 192.168.131.1
Aug 15 06:55:21 HOST123 daemon:notice data (8351): udhcpc: sending renew to 192.168.130.1
Aug 15 07:06:17 HOST123 daemon:notice voice (6262): udhcpc: sending renew to 192.168.117.1
Aug 15 07:06:17 HOST123 daemon:notice voice (6262): udhcpc: lease of 192.168.117.8 obtained, lease time 600
Aug 15 07:06:22 HOST123 daemon:notice mgmt (6263): udhcpc: sending renew to 192.168.121.1
Aug 15 07:06:22 HOST123 daemon:notice mgmt (6263): udhcpc: lease of 192.168.121.8 obtained, lease time 600
Aug 15 07:06:24 HOST123 daemon:notice iptv (6261): udhcpc: sending renew to 192.168.131.1
Aug 15 07:06:25 HOST123 daemon:notice iptv (6261): udhcpc: lease of 192.168.131.8 obtained, lease time 600
My config:
<source>
#type syslog
port 1514
tag scom
<parse>
#type syslog
expression /^(?<logtime>[^ ]) (?<host>HOST\d{4}) (?<facility>[^ ]*) (?<message>[\d\w\s\S\W]*)$/
time_key logtime
time_format "%b %d %T"
</parse>
</source>
or
<source>
#type syslog
port 1514
tag scom
expression /^\(?<time>[^ ]*) (?<host>[^ ]*) (?<facility>[^ ]*) (?<message>[\d\w\s\S\W]*)$/
time_format "%b %d %H:%M:%S"
</source>
And no one solution is working.
What's wrong?

Related

AWS API Gateway Cannot GET / when function sleeped for long time

My current stack is AWS API Gateway --> AWS Lambda --> swagger-node + swagger-express-mw + aws-serverless-express.
So my Swagger API is hosted as one node.js Lambda Function and Invoked with aws_proxy from API Gateway. This works quite good. The only thing is that when the function sleeped for too long (cold start?) I get a Cannot GET / as Output from every URL I am calling first. From 2nd Request on, it runs very fast. Any ideas on that?
I don't think that it comes from API Gateway Integration Timeout as that are 30 seconds. The slowest invocation time of the function directly via lambda is around 2,5s and when it is called more often it is normally not more than 150ms. I also increased the Time of Lambda Timeout for that function to 10s so from there should also not come an error.
Logs from Test Request via API Gateway first Invocation
Response Body
Cannot GET /hello
Response Headers
{
"x-powered-by": "Express",
"x-content-type-options": "nosniff",
"content-type": "text/html; charset=utf-8",
"content-length": "18",
"date": "Sun, 19 Feb 2017 15:00:11 GMT",
"connection": "close",
"X-Amzn-Trace-Id": "<TRACE-ID>"
}
Logs
Execution log for request test-request
Sun Feb 19 15:00:07 UTC 2017 : Starting execution for request: test-invoke-request
Sun Feb 19 15:00:07 UTC 2017 : HTTP Method: GET, Resource Path: /hello
Sun Feb 19 15:00:07 UTC 2017 : Method request path: {}
Sun Feb 19 15:00:07 UTC 2017 : Method request query string: {}
Sun Feb 19 15:00:07 UTC 2017 : Method request headers: {}
Sun Feb 19 15:00:07 UTC 2017 : Method request body before transformations:
Sun Feb 19 15:00:07 UTC 2017 : Endpoint request URI: https://lambda.eu-central-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-central-1:<ACCOUNT-ID>:function:api/invocations
Sun Feb 19 15:00:07 UTC 2017 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=**************************************************************************************************************************************************************************************************************************************************************************************************************************4b0637, X-Amz-Date=20170219T150007Z, x-amzn-apigateway-api-id=965h04axki, Accept=application/json, User-Agent=AmazonAPIGateway_965h04axki, X-Amz-Security-Token=<SECURITY-TOKEN>
Sun Feb 19 15:00:07 UTC 2017 : Endpoint request body after transformations: {"resource":"/hello","path":"/hello","httpMethod":"GET","headers":null,"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"accountId":"<ACCOUNT-ID>","resourceId":"ll6gw8","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"accountId":"<ACCOUNT-ID>","cognitoIdentityId":null,"caller":"<ACCOUNT-ID>","apiKey":"test-invoke-api-key","sourceIp":"test-invoke-source-ip","accessKey":"<ACCESS-ID>","cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":"arn:aws:iam::<ACCOUNT-ID>:root","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_102)","user":"<ACCOUNT-ID>"},"resourcePath":"/hello","httpMethod":"GET","apiId":"965h04axki"},"body":null,"isBase64Encoded":false}
Sun Feb 19 15:00:11 UTC 2017 : Endpoint response body before transformations: {"statusCode":404,"body":"Cannot GET /hello\n","headers":{"x-powered-by":"Express","x-content-type-options":"nosniff","content-type":"text/html; charset=utf-8","content-length":"18","date":"Sun, 19 Feb 2017 15:00:11 GMT","connection":"close"},"isBase64Encoded":false}
Sun Feb 19 15:00:11 UTC 2017 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=19f8554e-f6b4-11e6-8184-d3ccf0ccf643, Connection=keep-alive, Content-Length=267, Date=Sun, 19 Feb 2017 15:00:11 GMT, Content-Type=application/json}
Sun Feb 19 15:00:11 UTC 2017 : Method response body after transformations: Cannot GET /hello
Sun Feb 19 15:00:11 UTC 2017 : Method response headers: {x-powered-by=Express, x-content-type-options=nosniff, content-type=text/html; charset=utf-8, content-length=18, date=Sun, 19 Feb 2017 15:00:11 GMT, connection=close, X-Amzn-Trace-Id=Root=1-58a9b2f7-91fc7371e41d6ae9c2fbf64d}
Sun Feb 19 15:00:11 UTC 2017 : Successfully completed execution
Sun Feb 19 15:00:11 UTC 2017 : Method completed with status: 404
Logs from Test Request via API Gateway second Invocation
Response Body
"Hello, stranger!"
Response Headers
{
"x-powered-by": "Express",
"access-control-allow-origin": "*",
"content-type": "application/json; charset=utf-8",
"content-length": "18",
"etag": "W/\"12-E1p7iNXxJ4trMdmFBhlU9Q\"",
"date": "Mon, 13 Feb 2017 20:12:36 GMT",
"connection": "close",
"X-Amzn-Trace-Id": "<Trace-ID>"
}
Logs
Execution log for request test-request
Mon Feb 13 20:12:36 UTC 2017 : Starting execution for request: test-invoke-request
Mon Feb 13 20:12:36 UTC 2017 : HTTP Method: GET, Resource Path: /hello
Mon Feb 13 20:12:36 UTC 2017 : Method request path: {}
Mon Feb 13 20:12:36 UTC 2017 : Method request query string: {}
Mon Feb 13 20:12:36 UTC 2017 : Method request headers: {}
Mon Feb 13 20:12:36 UTC 2017 : Method request body before transformations:
Mon Feb 13 20:12:36 UTC 2017 : Endpoint request URI: https://lambda.eu-central-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-central-1:<LAMBDA-FUNCTION-ID>:function:api/invocations
Mon Feb 13 20:12:36 UTC 2017 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=*******************************************************************************************************************************************************************************************************************************************************************************************************************************************3e1b18, X-Amz-Date=20170213T201236Z, x-amzn-apigateway-api-id=965h04axki, X-Amz-Source-Arn=arn:aws:execute-api:eu-central-1:<ACCOUNT-ID>:965h04axki/null/GET/hello, Accept=application/json, User-Agent=AmazonAPIGateway_965h04axki, X-Amz-Security-Token=<TOKEN>
Mon Feb 13 20:12:36 UTC 2017 : Endpoint request body after transformations: {"resource":"/hello","path":"/hello","httpMethod":"GET","headers":null,"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"accountId":"<ACCOUNT-ID>","resourceId":"ll6gw8","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"accountId":"<ACCOUNT-ID>","cognitoIdentityId":null,"caller":"427402682812","apiKey":"test-invoke-api-key","sourceIp":"test-invoke-source-ip","accessKey":"<ACCESS-KEY>","cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":"arn:aws:iam::<ACCOUNT-ID>:root","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_102)","user":"<ACCOUNT-ID>"},"resourcePath":"/hello","httpMethod":"GET","apiId":"965h04axki"},"body":null,"isBase64Encoded":false}
Mon Feb 13 20:12:36 UTC 2017 : Endpoint response body before transformations: {"statusCode":200,"body":"\"Hello, stranger!\"","headers":{"x-powered-by":"Express","access-control-allow-origin":"*","content-type":"application/json; charset=utf-8","content-length":"18","etag":"W/\"12-E1p7iNXxJ4trMdmFBhlU9Q\"","date":"Mon, 13 Feb 2017 20:12:36 GMT","connection":"close"},"isBase64Encoded":false}
Mon Feb 13 20:12:36 UTC 2017 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=c3354327-f228-11e6-8c1d-ed11cc413770, Connection=keep-alive, Content-Length=315, Date=Mon, 13 Feb 2017 20:12:36 GMT, Content-Type=application/json}
Mon Feb 13 20:12:36 UTC 2017 : Method response body after transformations: "Hello, stranger!"
Mon Feb 13 20:12:36 UTC 2017 : Method response headers: {x-powered-by=Express, access-control-allow-origin=*, content-type=application/json; charset=utf-8, content-length=18, etag=W/"12-E1p7iNXxJ4trMdmFBhlU9Q", date=Mon, 13 Feb 2017 20:12:36 GMT, connection=close, X-Amzn-Trace-Id=Root=1-58a21334-8ea6c4b5944eebb873bc7d2e}
Mon Feb 13 20:12:36 UTC 2017 : Successfully completed execution
Mon Feb 13 20:12:36 UTC 2017 : Method completed with status: 200
I think the response "Cannot GET /" is coming from your Lambda function itself. Can you check API Gateway CW logs (or Test Invoke feature in console) to see what's different in the integration request and response in the first call?
I didn't see any real documentation about it (just this Medium post) but I also experienced the fact that a Lambda can be frozen until the first invocation, or in the case it's not called for a long time.
A solution is to schedule a regular invocation to wake up your lambda, with Amazon CloudWatch Events
I know that is an old question, but if you use TypeORM (or more in general, if you wrap all your Express middlewares within a .then() callback of a Promise), and you use context.callbackWaitsForEmptyEventLoop = false in your lambda handler, maybe this could help you: https://github.com/typeorm/typeorm/issues/5894
Long story short: avoid to set that flag to false, if possible, otherwise avoid to wrap the Express middlewares within the .then() callback and, for instance, initialize your db connection in the first Express middleware.

Perl regex dotall problems

I'm trying to fetch certain values from a file that I've created with a system command. The file is in order and the regex is working up until I reach a "newline". I've tried to get it to grab the other value in multiple ways, but I can't seem to figure it out. Where am I going wrong?
Here is the code
sub servicechoise2 {
my $sys_com = "Servicestatus.txt";
print "type status you would like to see status of: ";
my $service = <>;
chomp $service;
system( "systemctl status $service > $sys_com" );
open( my $fh2, "<", $sys_com );
my #services;
while ( my $line = <$fh2> ) {
if ( $line =~ /([a-z]+.service)\s-.*(running|dead)/s ) {
my %hash2 = (
"servicename" => $1,
"servicestatus" => $2
);
push( #services, \%hash2 );
}
}
return \#services;
}
and here is the file I'm parsing
sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled) Active: active (running) since Fri 2015-08-21 18:20:06 CEST; 1h 32min ago Main PID: 1297 (sshd) CGroup: /system.slice/sshd.service
└─1297 /usr/sbin/sshd -D
Aug 21 18:20:06 Thomas-PC systemd[1]: Started OpenSSH server daemon. Aug 21 18:20:07 Thomas-PC sshd[1297]: Server listening on 0.0.0.0 port
22. Aug 21 18:20:07 Thomas-PC sshd[1297]: Server listening on :: port 22.
cups.service - CUPS Printing Service Loaded: loaded (/usr/lib/systemd/system/cups.service; enabled) Active: active (running) since Fri 2015-08-21 18:20:33 CEST; 1h 32min ago Main PID: 3657 (cupsd) CGroup: /system.slice/cups.service
└─3657 /usr/sbin/cupsd -f
Aug 21 18:20:33 Thomas-PC systemd[1]: Started CUPS Printing Service.
ntpd.service - Network Time Service Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled) Active: inactive (dead)
named.service - Berkeley Internet Name Domain (DNS) Loaded: loaded (/usr/lib/systemd/system/named.service; enabled) Active: active (running) since Fri 2015-08-21 18:20:10 CEST; 1h 32min ago Process: 2477 ExecStart=/usr/sbin/named -u named $OPTIONS (code=exited, status=0/SUCCESS) Process: 1302 ExecStartPre=/usr/sbin/named-checkconf -z /etc/named.conf (code=exited, status=0/SUCCESS) Main PID: 2502 (named) CGroup: /system.slice/named.service
└─2502 /usr/sbin/named -u named
Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.biz/A/IN': 2001:503:7bbb:ffff:ffff:ffff:ffff:ff7e#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.biz/AAAA/IN': 2001:503:7bbb:ffff:ffff:ffff:ffff:ff7e#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.biz/A/IN': 2001:500:3682::12#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.biz/AAAA/IN': 2001:500:3682::12#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'ns2.isc.ultradns.net/A/IN': 2001:502:4612::e8#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.com/AAAA/IN': 2001:502:f3ff::e8#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.com/AAAA/IN': 2610:a1:1016::e8#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.co.uk/AAAA/IN': 2610:a1:1017::e8#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.co.uk/A/IN': 2610:a1:1017::e8#53 Aug 21 19:20:11 Thomas-PC named[2502]: error (network unreachable) resolving 'pdns196.ultradns.biz/A/IN': 2610:a1:1015::e8#53
postfix.service - Postfix Mail Transport Agent Loaded: loaded (/usr/lib/systemd/system/postfix.service; enabled) Active: active (running) since Fri 2015-08-21 18:20:10 CEST; 1h 32min ago Process: 1335 ExecStart=/usr/sbin/postfix start (code=exited, status=0/SUCCESS) Process: 1328 ExecStartPre=/usr/libexec/postfix/chroot-update (code=exited, status=0/SUCCESS) Process: 1298 ExecStartPre=/usr/libexec/postfix/aliasesdb (code=exited, status=0/SUCCESS) Main PID: 2531 (master) CGroup: /system.slice/postfix.service
├─2531 /usr/libexec/postfix/master -w
├─2534 pickup -l -t unix -u
└─2535 qmgr -l -t unix -u
Aug 21 18:20:06 Thomas-PC systemd[1]: Starting Postfix Mail Transport Agent... Aug 21 18:20:09 Thomas-PC postfix/postfix-script[2510]: warning: group or other writable: /etc/postfix/./main.cf Aug 21 18:20:10 Thomas-PC postfix/postfix-script[2529]: starting the Postfix mail system Aug 21 18:20:10 Thomas-PC postfix/master[2531]: daemon started -- version 2.10.1, configuration /etc/postfix Aug 21 18:20:10 Thomas-PC systemd[1]: Started Postfix Mail Transport Agent. Aug 21 18:23:08 Thomas-PC postfix/smtpd[4293]: connect from localhost[127.0.0.1] Aug 21 18:23:08 Thomas-PC postfix/smtpd[4293]: NOQUEUE: reject: RCPT from localhost[127.0.0.1]: 550 5.1.1 <a14thona#localhost>: Recipient address rejected: User unknown in local recipient table; from=<admin#localhost> to=<a14thona#localhost> proto=ESMTP helo=<localhost.localdomain> Aug 21 18:23:08 Thomas-PC postfix/smtpd[4293]: lost connection after RCPT from localhost[127.0.0.1] Aug 21 18:23:08 Thomas-PC postfix/smtpd[4293]: disconnect from localhost[127.0.0.1]
the subroutine returns this array of hashes
[
{ servicename => "sshd.service", servicestatus => "running" },
{ servicename => "cups.service", servicestatus => "running" },
{ servicename => "ntpd.service", servicestatus => "dead" },
{ servicename => "named.service", servicestatus => "running" },
{ servicename => "postfix.service", servicestatus => "running" },
]
I would try to read the answer into a var, and then process the tokens using split ( it seems there are empty lines between tokens) something like:
open(F,"<",file) || die "...";
{ local $/; $in=; } # slurp file
foreach $line ( split(/\n\n/,$in) )
{
if ( $line =~ /([a-z]+.service)\s-.*(running|dead)/s ) {
 ......
}

RegEx match IP on Mail-Header Received:

I try to fiddle a RegEx, which returns me only the Sender IP Address:
http://regexr.com?38atl
This is the RegEx I build, but cant complete:
(?<=\bReceived: from .*\[)(?:\d{1,3}\.){3}\d{1,3}
or
(?<=\bReceived: from )(.*\[)(?:\d{1,3}\.){3}\d{1,3}
So it should only match this (on lines beginning with: Received: from)
127.0.0.1
127.0.0.1
21.22.23.24
And this are a example Mail-Headers i'm search in:
To: a#domain.de
Return-Path: <t#domain.de>
X-Original-To: a#domain.de
Delivered-To: c#domain.tld
Received: from localhost (localhost [127.0.0.1])
by mail1.domain.tld (Postfix) with ESMTP id 3fT3TR72zNz8m8
for <a#domain.de>; Tue, 18 Feb 2014 14:54:35 +0100 (CET)
X-Virus-Scanned: Debian amavisd-new at mail1.domain.tld
X-Spam-Flag: YES
X-Spam-Score: 5.773
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.773 tagged_above=1 required=4.5
tests=[BAYES_05=-0.5, MISSING_MID=0.497, RCVD_IN_PBL=3.335,
RCVD_IN_RP_RNBL=1.31, RDNS_DYNAMIC=0.982, TO_NO_BRKTS_DYNIP=0.139,
T_RCVD_IN_SEMBLACK=0.01] autolearn=no
Received: from mail1.domain.tld ([127.0.0.1])
by localhost (mail1.domain.tld [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id lDJqiZjBn2t4 for <a#domain.de>;
Tue, 18 Feb 2014 14:54:34 +0100 (CET)
Received: from mail.domain.tld (pAAAAAAAA.dip0.t-ipconnect.de [21.22.23.24])
by mail1.domain.tld (Postfix) with SMTP id 3fT3TQ4Nwgz8m5
for <a#domain.de>; Tue, 18 Feb 2014 14:54:34 +0100 (CET)
Date: Tue, 18 Feb 2014 15:02:11 +0100
Sender: "From" <t#domain.de>
From: "From" <t#domain.de>
Subject: Subbbb (192.168.123.123)
Reply-To: t#domain.de
MIME-Version: 1.0
Content-type: text/plain; charset=UTF-8
Message-Id: <3fT3TR72zNz8m8#mail1.domain.tld>
Try this expression:
Received: +from[^\n]*?\[([0-9\.]+)\]
Edit:
For a PHP script try something like this (where $emailHeader contains the data you are searching):
$regex = '/Received: +from[^\\n]*?\\[([0-9\\.]+)\\]/s';
if (preg_match_all($regex, $emailHeader, $matches_out)) {
print_r($matches_out);
} else {
print('Sender IP not found');
}
The <= in the star looks funny, but other than that it seems to be working fine:
(?:\bReceived: from .*\[)((\d{1,3}\.){3}\d{1,3})(?:]\))
I believe what you're looking for is:
(?:\bReceived: from .*?\[)(?<ip>(?:\d{1,3}\.){3}\d{1,3})
the matched IP address will be in capture group named "ip".

Email header showing, dkim=hardfail (test mode) and dkim=permerror (bad sig)

I am trying to write an email client application by c++ and OpenSSL..
When I am checking and sending email through this application then Gmail and Yahoo both are showing wrong in my email header..
Gmail showing > "dkim=hardfail (test mode) header.i=#".
Yahoo showing > "dkim=permerror (bad sig)".
My Email header for Gamil :
Delivered-To: rajforever6#gmail.com
Received: by 10.112.41.134 with SMTP id f6csp80981lbl;
Tue, 26 Mar 2013 02:25:15 -0700 (PDT)
X-Received: by 10.68.59.199 with SMTP id b7mr22705642pbr.167.1364289914515;
Tue, 26 Mar 2013 02:25:14 -0700 (PDT)
Return-Path: <no_feed#korcot.info>
Received: from smtp.korcot.info ([103.18.168.250])
by mx.google.com with SMTP id ak6si16720576pbd.344.2013.03.26.02.25.08;
Tue, 26 Mar 2013 02:25:14 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning no_feed#korcot.info does not designate 103.18.168.250 as permitted sender) client-ip=103.18.168.250;
Authentication-Results: mx.google.com;
spf=softfail (google.com: domain of transitioning no_feed#korcot.info does not designate 103.18.168.250 as permitted sender) smtp.mail=no_feed#korcot.info;
dkim=hardfail (test mode) header.i=#korcot.info
Date: Tue, 26 Mar 2013 02:25:14 -0700 (PDT)
DKIM-Signature: v=1;
a=rsa-sha256;
s=mail;
d=korcot.info;
t=1364289917;
c=relaxed/relaxed;
h=from:to:subject:message-id:mime-version:content-type;
bh=Q3K/t162lFKWTLs4xBY2FR1vT5pThpI6rnqM7i6p/Vo=;
b=Tgrc2nelY/3HA0M14s4RXFvhonXQzFRtDAD/X/0ITvln6JSWjIxgQSX7WOrfAxb4K
kTXV+Bbw6DKK+2aVi2WrCYdduP5wXJpTVnjaCJQTs2+DtAo3sL1re6tz3uHOpa0PmRS
3qRzfXQqN8QczOv63UjAGO6sL+o5atU3vbRYMxg=
From: "Comfirm"<no_feed#korcot.info>
To: "Kuntal"<rajforever6#gmail.com>
Subject: Key6gg6 ?
Message-ID: <20030712040037.46341.5F8J#football.korcot.info>
MIME-Version: 1.0
Content-Type: text/html
My Email header for Yahoo :
From Comfirm Wed Mar 27 02:26:20 2013
X-Apparently-To: raja_kuntal_majumder#yahoo.com via 106.10.150.84; Tue, 26 Mar 2013 11:26:20 -0700
Return-Path: <no_feed#korcot.info>
Received-SPF: softfail (transitioning domain of korcot.info does not designate 103.18.168.135 as permitted sender)
bm5lciByZWFkeSBJcyBkaW5uZXIgcmVhZHkgSXMgZGlubmVyIHJlYWR5IElz
IGRpbm5lciByZWFkeSBJcyBkaW5uZXIgcmVhZHkgPyBKb2UuIAEwAQEBAQ--
X-YMailISG: v3YQap0WLDt5n8clVc4Zd9ENB5oCUMcfvdmbG1QujbCK18hQ
nhRHHgRhlvqfyO8lOyt.Rwjm8Y1_i35XTte6Pt.JxYeVZXQtQqLe66qLvndr
_RhG.PTZE2AWeRN.tMHi.aAIcS9WfKzOzi9i3tYtSSzZMPBMlBwFu5hBV4qI
v3KO9LCaMGeCe3W4mLsxgbkdddDi_8r8afEb7jAUKVn8i2VyRaJQGELdL5xb
.lNzJ0h_vMs5HseAhRPefe.o9W4A_oMpcairIa9OfDkC3zmZWYbyntXcxNgi
uKtvTwaqyeuUvuyIxrRGlpUy8RFxRaLOlp6s8riVCfLyu1dEE6Bpa9KFlVa0
JtRm0XZ3znl2WBzw1PnHskYZ3X2SHrzz8q7LfmUETa7idziLtkLNqdzZbvJE
nnoc3W3aYmxkgZob25oRD8Q338f9774TfJMjeqTaQLbmMflOW5GqoPYneB0v
hKqmdYUcqeZpVfcRXBVSxzS2_lK7spwOJX_wXKAH5TbiRXORouWvbkw9EHgc
yxqKoTQAqpnc8ibxcfT9cmVQ2KXYXlrl7d0VotpJ.m8I0sMY0mTviVkt6vCI
FJUYScRuGMxjOaYqyURUU5mDM_bxQXRl1WFzEhYBQKsyMOhZVtBBrPyEsCLs
NHASPNKPUS7wuB712WAI3aMqTNV.kNize8nwrm.hj9f9fb2nJwcB.IvuQvKx
Ujd9Pq68B9u80Wd8gtp5rAIy7.B_0x6JHwK14fDWgh1Yr7k1PB8A_daUJh.G
d2f4rgY0CFbg3chmWC7O5AGvODB79.pFfkl8ooZsUpYqdsj5.KX5XLQjsfud
JCsO1870Gtty31aPGvSHRPDGOTJ_lQ6v25BMaCnGS8AtvYbtjW9avY0MHTJw
rnT0_Hzca6oXayZG0CCcvQ6CMd.UnOlIt4pFEWwd9Zlo5BIJ4hd9N99qMbul
mGgNYVRBc0f7DokmkLiHSI29k34ifNyh8Z1qbeKWTRo4FXDBa.vfc9cVWqpp
DVF8FbkpuThc7QJ63cszp98TCdZy2Bq5yOMrqEdK85e141MndRChrWmjUplJ
4e_UWqNmaL17cGPzXuJc3TAD3nsdK9qh1ax.9K.jkRwSm4zgWscqosCPAgOJ
lIOo3L3QScOP9iQ6xOfuzm7dxw44Z3WBHMthoOgw_MGt488EClyDr2PiJT.f
1_BklnxGS2iuYH2TLyY69ESKE0Y4pPGmaboLASqBQkQZ6xaoFM8HDoIB20zK
KrNadXnlkMW.t6RuIcaPX3fOrKzvnKPerZ3NA1kn
X-Originating-IP: [103.18.168.135]
Authentication-Results: mta1360.mail.bf1.yahoo.com from=korcot.info; domainkeys=neutral (no sig); from=korcot.info; dkim=permerror (bad sig)
Received: from 127.0.0.1 (HELO smtp.korcot.info) (103.18.168.135)
by mta1360.mail.bf1.yahoo.com with SMTP; Tue, 26 Mar 2013 11:26:19 -0700
DKIM-Signature: v=1;
a=rsa-sha256;
s=mail;
d=korcot.info;
t=1364322363;
c=relaxed/simple;
h=from:to:subject:mime-version:content-type;
bh=Q3K/t162lFKWTLs4xBY2FR1vT5pThpI6rnqM7i6p/Vo=;
b=THj4wE+EOSKThXGIUPBgcTlWTICIDFikgEvlQVpBotViQTNW+k/dpr58kVZ1HAght
KTdACtbEO2m3A17vxrY1kEQazhwu7pbWIW5TwBfUtIU9SbFBL88AQz2fxxi6BHqEfNh
WYeGG0mEskG9J2JFeNVG0WaaNJ9gTh5t/4nqvVc=
From: "Comfirm" <no_feed#korcot.info>
To: "Kuntal" <raja_kuntal_majumder#yahoo.com>
Subject: Key6gg6 ?
MIME-Version: 1.0
Content-Type: text/html
Content-Length: 287
My TXT records in DNS(godaddy.com) :
# >>>> v=spf1 a mx mx:smtp.korcot.info ip4:103.18.168.250 ~all
mail._domainkey >>>> k=rsa; t=y; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCjKsxH3Hb8UrNZ+Aar7GfJ30dcbS19i+89JA8SSlDEhsFfTAR9UZIolxKtD3HgPzhWGfcoLkQg28NS/Y3tT/WTvfXxryDyFRIbKJox6VbVmmvSmATsik1l2YQSkTTpeHX6nzxgzRwDVvmLfZBZBbO3n+y0YNTucOUsfNFYCL/UAwIDAQAB
_domainkey >>>> t=y; o=~;
Really I don't have knowledge why Gmail and Yahoo showing "dkim=hardfail (test mode) header.i=#" and "dkim=permerror (bad sig)", both header should be "dkim=pass (ok)"..
So please give me advice.
Please tell me what should I do to solve this problem..
Thanks...
You didn't mention whether you are using a standard library to calculate the DKIM signature, or have done your own. I strongly recommend OpenDKIM as a good library to use if you aren't already.
Another place to test is to send a message to the mail reflector at sa-test#sendmail.net. It may give you more diagnostic information.
You should check the header fields that are signed (From, To, Subject, Mime-Version, and Content-Type) between what you sent originally and what was received. Some mailers "helpfully" rewrite these header fields, breaking the signature in the process. For example, it could have signed no_feed#smtp.korcot.info and subsequently changed it to no_feed#korcot.info.

How a device registers itself after adding a pass

When I add a pass in a device, I see on the console like this:
Mar 26 14:32:36 CamMobs-iPod4 passd[7128] <Warning>: Card has more than 10 locations. Capping.
Mar 26 14:32:38 CamMobs-iPod4 MobileSafari[7115] <Warning>: Warning: Attempt to dismiss from view controller <BrowserRootViewController: 0x1ed546a0> while a presentation or dismiss is in progress!
Mar 26 14:32:39 CamMobs-iPod4 backboardd[52] <Warning>: CoreAnimation: updates deferred for too long
Mar 26 14:32:39 CamMobs-iPod4 locationd[41] <Notice>: Location icon should now be in state 'Active'
Mar 26 14:32:50 CamMobs-iPod4 locationd[41] <Notice>: Location icon should now be in state 'Inactive'
Mar 26 14:32:52 CamMobs-iPod4 profiled[7122] <Notice>: (Note ) profiled: Idled.
Mar 26 14:32:52 CamMobs-iPod4 profiled[7122] <Notice>: (Note ) profiled: Service stopping.
Mar 26 14:33:31 CamMobs-iPod4 locationd[41] <Warning>: Launch Services: Registering unknown app identifier com.apple.PassKit failed
Mar 26 14:33:31 CamMobs-iPod4 locationd[41] <Warning>: Launch Services: Unable to find app identifier com.apple.PassKit
Mar 26 14:33:33 CamMobs-iPod4 configd[50] <Notice>: network changed: v4(en0:192.168.1.109) DNS Proxy
Mar 26 14:33:53 CamMobs-iPod4 backboardd[52] <Notice>: Posting 'com.apple.iokit.hid.displayStatus' notifyState=0
.....
Why it tries to register to com.apple.PassKit ?
In my server, I use some codes in index.php like the following:
<?php
// Transfer Request URL into array
$request = explode("/", substr(#$_SERVER['REQUEST_URI'], 1));
//$request = explode("/", substr(#$_SERVER['REQUEST_URI'], 1));
print_r($_SERVER['REQUEST_URI']);
if (strtoupper($_SERVER['REQUEST_METHOD']) === "POST"
&& isset($_SERVER['HTTP_AUTHORIZATION'])
&& strpos($_SERVER['HTTP_AUTHORIZATION'], 'ApplePass') === 0
&& $request[2] === "devices"
&& $request[4] === "registrations") {
$auth_key = str_replace('ApplePass ', '', $_SERVER['HTTP_AUTHORIZATION']);
$device_id = $request[3];
$pass_id = $request[5];
$serial = $request[6];
echo $request[3];
//$device_id = $_POST[''];
echo $device_id;
echo $pass_id;
echo $serial ;
// Catch the JSON post and decode it
$dt = #file_get_contents('php://input');
// $dt = #file_get_contents('php://input');
//$device_token = json_decode($dt);
//$device_token = $device_token->pushToken;
$pushtoken=json_decode($dt)->pushToken;
if (!$device_token) die('No Token Found'); // Token wasn't found
$dbhost = 'localhost:8889';
$dbuser = 'root';
$dbpass = 'root';
$dbname = 'passesdb';
$conn = mysql_connect($dbhost, $dbuser, $dbpass)
or die ('Error connecting to mysql'.mysql_error());
mysql_select_db($dbname);
mysql_query("SET NAMES UTF8");
mysql_query($sql,$conn);
$table = 'Devices';
$sql = mysql_query("insert into Devices values('$device_id','$pushtoken')");
mysql_query($sql);
exit;
}
?>
This is what a successful registration looks like in the console:
1. Mar 26 17:00:03 iPhone5 passd[6262] <Warning>: Generating POST request with URL <https:/afr.passk.it/v1/devices/7864dc8fdcfe739273cf7362a0db2b35/registrations/pass.it.passk.developer3/1wqdDAqHydkRURA9YCjbq>
2. Mar 26 17:00:03 iPhone5 passd[6262] <Warning>: Request contains header field <Authorization: ApplePass 5cdddad65324384efa39575a4cf22424>
3. Mar 26 17:00:03 iPhone5 passd[6262] <Warning>: Request contains body dictionary {
pushToken = 0bbe54794500332b789a3ddb69827386d5c9aad1cb035c9f2725761d419950b2;
}
4. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Register task (for device 7864dc8fdcfe739273cf7362a0db2b35, pass type pass.it.passk.developer3, serial number 1wqdDAqHydkRURA9YCjbq; with web service url https://afr.passk.it/) got response with code 201
5. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Generating GET request with URL <https:/afr.passk.it/v1/devices/7864dc8fdcfe739273cf7362a0db2b35/registrations/pass.it.passk.developer3?passesUpdatedSince=1364287618>
6. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Generating GET request with URL <https:/afr.passk.it/v1/passes/pass.it.passk.developer3/1wqdDAqHydkRURA9YCjbq>
7. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Request contains header field <If-Modified-Since: Tue, 26 Mar 2013 07:35:33 GMT>
8. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Request contains header field <Authorization: ApplePass 5cdddad65324384efa39575a4cf22424>
9. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Get serial #s task (for device 7864dc8fdcfe739273cf7362a0db2b35, pass type pass.it.passk.developer3, last updated 1364287618; with web service url https://afr.passk.it/) got response with code 204
10. Mar 26 17:00:04 iPhone5 passd[6262] <Warning>: Get serial numbers task completed with update tag (null), serial numbers (null)
11. Mar 26 17:00:05 iPhone5 passd[6262] <Warning>: Get pass task (pass type pass.it.passk.developer3, serial number 1wqdDAqHydkRURA9YCjbq, if-modified-since Tue, 26 Mar 2013 07:35:33 GMT; with web service url https://afr.passk.it/) got response with code 304
What you have posted above is only the last line of this process, (where passd has picked up that you have more than 10 locations in your pass.json).
If you examine the above you can see the flow of events that your web service needs to respond to:
Row 1: Device sends a POST request to:
https://webserviceURL/v1/devices/{deviceLibraryIdentifier}/registrations/{passTypeIdentifier}/{serialNumber}`
Row 2: POST request is sent with the header field:
Authorization: ApplePass {authenticationToken}
Row 3: POST body contains the JSON Dictionary:
{
pushToken = {pushToken};
}
Provided your rewrite rule is correct, your PHP code should analyse the URL and capture the deviceLibraryIdentifier and pushToken and store it in the database, linked to the pass record containing the serialNumber, authenticationToken and passTypeIdentifier.
Then Row 4: Your web service responds to the device with a 201 code to indicate that the registration was successful.
On Row 5: The device then generates a GET request to your web service to check if there is a newer versions pof passes for the same certificate:
https:/webserviceURL/v1/devices/{deviceLibraryIdentifier}/registrations/{passTypeIdentifier}?passesUpdatedSince={lastUpdateTag}
On Row 6, 7 and 8: The device generates a GET request to your web service to check if there is a newer version of this specific pass. It provides an If-Modified-Since header containing the date provided in the header of the last downloaded .pkpass bundle (Row 7), and provides another Authorization header containing Applepass {authenticationToken} (Row 8) so that your web service can validate the request against by checking the database record for the serialNumber.
https:/webserviceURL/v1/passes/{passTypeIdentifier}/{serialNumber}
Header: If-Modified-Since: {last modified date}
Header: Authorization: ApplePass {authenticationToken}
On Row 9: the web service responds with a 204 response, indicating that there are no serialNumbers for the passTypeIdentifier that require updating. Row 10 confirms this.
Finally, on Row 11, the device receives a 304 response from your web service, confirming that the pass it has just installed is the latest version of the pass.
The above outlines precisely what your web service will receive and the responses it needs to provide to successfully register a device.