I have the below procmail recipe that for years has faithfully worked (passed senders email and subject) to the perl script and saved the attachments.
FROM=""
:0
* ^FROM:.*
* ^From:[ ]*\/[^ ].*
{
FROM=$MATCH
}
#grab the subject
SUBJECT=""
:0
* ^FROM:.*
* ^Subject:[ ]*\/[^ ].*
{
SUBJECT=$MATCH
}
:0 ifw # rip & save attachements
|ripmime -i - -d /home/carrdocs/.pmdir/attachments/ &&\
/usr/bin/perl -e 'require "/home/carrdocs/.pmdir/test_carr_subject.pl"; rename_att($ENV{SUBJECT},$ENV{FROM},$ENV{MESSAGE}); exit(0)'
:0 A
filed
I am trying to modify the recipe to also send the contents of the email's body to the perl script as a scalar variable. Accordingly, I added:
:0b w
MESSAGE=| cat
just before the line (with one line of space between):
:0 ifw
This results in the program sometimes working as hoped to and other times failing to pass the variables and save the attachments with the error:
procmail: Program failure (-11) of "ripmime -i - -d /home/carrdocs/.pmdir/attachments/ &&\
/usr/bin/perl -e 'require "/home/carrdocs/.pmdir/test_carr_subject.pl"; rename_att($ENV{SUBJECT},$ENV{FROM},$ENV{MESSAGE}); exit(0)'"
Does anyone know how I can correctly pass the body's contents as a scalar variable to the perl script?
This probably happens when MESSAGE is longer than LINEBUF (or even actually when it is only slightly shorter, so that the entire ripmime command line ends up exceeding LINEBUF).
Check in the log for a message like this immediately before the failure:
procmail: Assigning "MESSAGE="
procmail: Executing "cat"
procmail: Exceeded LINEBUF
procmail: Assigning "PROCMAIL_OVERFLOW=yes"
The assignment of MESSAGE is fine, but attempting to use its value in a subsequent recipe will fail because the string is longer than Procmail can accommodate.
So, TL;DR; don't try to put things which are longer than a handful of bytes into variables. (For the record, the default value of LINEBUF on my Debian test box is 2048 bytes.)
Probably for other reasons too, refactor your recipe so that your Perl script receives the message on standard input instead. Without exact knowledge of what ripmime does or how it's supposed to interface with your Perl script (looks to me like actually combining the two is completely crazy here, but perhaps I'm missing something), this is tentative at best, but something like
:0w # rip & save attachements
|ripmime -i - -d /home/carrdocs/.pmdir/attachments/
:0Afw
| /usr/bin/perl -e 'require "/home/carrdocs/.pmdir/test_carr_subject.pl"; \
($body = join("", <>)) =~ s/^.*?\r?\n\r?\n//s; \
} rename_att($ENV{SUBJECT},$ENV{FROM},$body))'
I also took out the exit(0) which seemed entirely superfluous; and the i flag is now no longer topical for either of these recipes.
I guess you could easily inline the extraction of the Subject: and From: headers into Perl as well, with some benefits e.g. around RFC2047 decoding, assuming of course you know how to do this correctly in Perl.
Related
I need to go another server and perform a word count. Based on the count variable I will perform a if else logic.
However i am unable to do a word count and further unable to compare the variable value in if condition.
Error:
wc: cannot open the file v.txt
Script:
#!/bin/bash
ssh u1#s1 "cd ~/path1/ | fgrep-f abc.csv xyz.csv > par.csv | a=$(wc -l par.csv)| if ["$a" == "0"];
then echo "success"
fi"
First, although the wc program is named for 'word count', wc -l actually counts lines not words. I assume that is what you want even though it isn't what you said.
A shell pipline one | two | three runs things in parallel with (only) their stdout and stdin connected; thus your command runs one subshell that changes directory to ~/path1 and immediately exits with no effect on anything else, and at the same time tries to run fgrep-f (see below) in a different subshell which has not changed the directory and thus probably can't find any file, and in a third subshell does the assignment a= (see below) which also immediately exits so it cannot be used for anything.
You want to do things sequentially:
ssh u#h 'cd path1; fgrep -f abc.csv xyz.csv >par.csv; a=$(wc -l par.csv); if [ "$a" == "0" ] ...'
# you _might_ want to use && instead of ; so that if one command fails
# the subsequent ones aren't attempted (and possibly go further wrong)
Note several other important changes I made:
the command you give ssh to send the remote must be in singlequotes ' not doublequotes " if it contains any dollar as yours does (or backtick); with " the $(wc ...) is done in the local shell before sending the command to the remote
you don't need ~/ in ~/path1 because ssh (or really sshd) always starts in your home directory
there is no common command or program fgrep-f; I assume you meant the program fgrep with the flag -f, which must be separated by a space. Also fgrep although traditional is not standard (POSIX); grep -F is preferred
you must have a space after [ and before ]
However, this won't do what you probably want. The value of $a will be something like 0 par.csv or 1 par.csv or 999 par.csv; it will never equal 0 so your "success" branch will never happen. In addition there's no need to do these in separate commands: if your actual goal is to check that there are no occurrences in xyz.csv of the (exact/non-regexp) strings in abc.csv both in path1, you can just do
ssh u#h 'if ! grep -qFf path1/abc.csv path1/xyz.csv; then echo success; fi'
# _this_ case would work with " instead of ' but easier to be consistent
grep (always) sets its exit status to indicate whether it found anything or not; flag -q tells it not to output any matches. So grep -q ... just sets the status to true if it matched and false otherwise; using ! inverts this so that if grep does not match anything, the then clause is executed.
If you want the line count for something else as well, you can do it with a pipe
'a=$( fgrep -Ff path1/abc.csv path1/xyz.csv | wc -l ); if [ $a == 0 ] ...'
Not only does this avoid the temp file, when the input to wc is stdin (here the pipe) and not a named file, it outputs only the number and no filename -- 999 rather than 999 par.csv -- thus making the comparison work right.
When a Qt application that uses QSharedMemory crashes, some memory handles are left stuck in the system.
The "recommended" way to get rid of them is to
if(memory.attach(QSharedMemory::ReadWrite))
memory.detach();
bool created = memory.create(dataSize, QSharedMemory::ReadWrite);
In theory the above code should work like this:
We attach to a left over piece of sh...ared memory, detach from it, it detects that we are the last living user and gracefully goes down.
Except... that is not what happens in a lot of cases. What I actually see happening, a lot, is this:
// fails with memory.error() = SharedMemoryError::NotFound
memory.attach(QSharedMemory::ReadWrite);
// fails with "segment already exists" .. wait, what?! (see above)
bool created = memory.create(dataSize, QSharedMemory::ReadWrite);
The only somewhat working way I've found for me to work around this is to write a pid file on application startup containing the pid of the currently running app.
The next time the same app is run it picks up this file and does
//QProcess::make sure that PID is not reused by another app at the moment
//the output of the command below should be empty
ps -p $previouspid -o comm=
//QProcess::(runs this script, reads output)
ipcs -m -p | grep $user | grep $previouspid | sed "s/ / /g" | cut -f1 -d " "
//QProcess::(passes the result of the previous script to clean up stuff)
ipcrm -m $1
Now, I can see the problems with such approach myself, but it is the only thing that works
The question is: can someone explain to me what exactly is the deal with not so not existing memory in the first piece of code above and how to deal with it properly?
I'm trying to work through a simple hackerank problem and I'm stuck.
When executing grep on the command line, I have no issues
grep -i \"the[^a-z0-9]\" $filename <<< "from fairest creatures we desire increase"
result: (returns nothing)
grep -i \"the[^a-z0-9]\" $filename <<< "the clock struck twice"
result: the(highlighted) clock struck twice
I read this article :
Can I open bash from a popen() stream?
I have tried prepending the grep command with "exec bash -c" and "/bin/bash -c" to no avail.
Can anyone tell me why I'm getting this error and how to fix it?
sh: 1: Syntax error: redirection unexpected
f is null
My code:
#include <iostream>
#include <vector>
#include <algorithm>
#include <cstdio>
#include<cmath>
using namespace std;
int main(){
/* Enter your code here. Read input from STDIN. Print output to STDOUT */
int MAX_BUFFER =2048;
string output;
char fileInput [MAX_BUFFER];
// string input(istreambuf_iterator<char>(cin), {});
string input =" From fairest creatures we desire increase ,\
That thereby beauty's rose might never die,\
But as the riper should by time decease,\
His tender heir might bear his memory:\
But thou contracted to thine own bright eyes,\
Feed'st thy light's flame with self-substantial fuel,\
Making a famine where abundance lies,\
Thy self thy foe, to thy sweet self too cruel:\
Thou that art now the world's fresh ornament,\
And only herald to the gaudy spring,\
Within thine own bud buriest thy content,\
And tender churl mak'st waste in niggarding:\
Pity the world, or else this glutton be,\
To eat the world's due, by the grave and thee.\
When forty winters shall besiege thy brow,\
And dig deep trenches in thy beauty's field,\
Thy youth's proud livery so gazed on now,\
Will be a tattered weed of small worth held:\
Then being asked, where all thy beauty lies,\
Where all the treasure of thy lusty days;\
To say within thine own deep sunken eyes,\
Were an all-eating shame, and thriftless praise.\
How much more praise deserved thy beauty's use,\
If thou couldst answer 'This fair child of mine\
Shall sum my count, and make my old excuse";
string cmd_string = "/bin/bash -c grep -i \"the[^a-z0-9]\" $filename <<< "+input;
//cout<<cmd_string.c_str();
FILE *f=popen(cmd_string.c_str(),"r");
while (!feof(f))
{
if (fgets(fileInput, MAX_BUFFER, f) != NULL)
{
cout<<" line 1"<<fileInput[0];
output+=fileInput;
}
else
{
cout<<"f is null"<<endl;
}
}
pclose(f);
cout<< output;
return 0;
}
The normal (and almost universal) convention for handling command-line flags which take arguments is that the single argument following the command-line flag itself is the associated argument. Subsequent arguments are either other flags or positional arguments.
That's certainly true of bash. If you want to use the -c command-line flag to execute a bash command, the entire command needs to be the argument to the -c flag, and it needs to be a single argument. Subsequent arguments are assigned to $0, $1, etc. before the command is executed.
So if you type:
bash -c echo foo
you will see a blank line. You could do this:
bash -c 'echo $0' foo
which will print foo "as expected" (if your expectations are well-honed), but it would be normal to do this:
bash -c 'echo foo'
Similarly, if you execute
/bin/bash -c grep -i \"the[^a-z0-9]\" ...
you are asking bash to execute the command grep with no arguments; the -i and subsequent arguments are assigned to $0, etc., but since those are never referenced by the command, that is somewhat pointless.
What you meant was
/bin/bash -c 'grep -i "the[^a-z0-9]" ... '
(You cannot backslash escape anything inside single-quoted strings, so the backslashes are removed. I don't see how they could have worked in the original command-line either; there are no double quotes to match in the target string.)
Obviously, it is vital that the here-string (<<<"...poem...") be part of the -c argument to /bin/bash and not part of the command-line interpreted by popen. popen uses sh to execute the provided command-line, and it is highly likely that sh does not recognize the "here-string" syntax; it's a bash extension. Because <<< is not special in sh, << is interpreted normally as a here-doc, which must be followed by a delimiter string. It cannot be followed by a < redirection; hence the "unexpected redirection" error. When you make the word following <<< very long, you apparently trigger a "filename too long" error before the erroneous redirection has been parsed. That's just a guess; I didn't reproduce the problem.
Quoting is going to be annoying for you. As mentioned, there is no problem with " inside single-quoted strings, but you will have to backslash the double-quotes to comply with C string literal syntax. Unfortunately, the string literal contains single quotes, and the first of those will be parsed as closing the single-quote in bash -c '. And there is no way to backslash-escape anything inside bash single-quoted strings, not even a single quote.
The only thing that will work will be to represent each ' with the sequence '\'' (which will need to be '\\'' inside the C string). In that sequence, the first and last ' close and reopen the single-quoted string, and the \' in the middle is a backslash-escaped '.
Three final notes:
$filename has no effect in your commands, since $filename has never been defined. If it were defined, it would create a variety of errors; since it is not defined and the expansion is not quoted, it simply disappears from the command-line, but it would be a lot easier not to put it there in the first place.
C strings can be continued from line to line using a backslashed newline, as in your sample code, but that does not insert a newline into the string. So the input provided to grep will be a single line. I don't think that's what you meant.
You do have to quote the here-string; the syntax of here-strings is <<< followed by a single "word"; if you don't put quotes around it, you'll end up with only the part up to the first space being treated as input.
I think your input is not quoted.
string cmd_string = "/bin/bash -c grep -i \"the[^a-z0-9]\" $filename <<< "+ \" + input + \";
I don't do much shell scripting but I want to essentially do this:
run the command "grunt check" about 30 times (the process takes 60 seconds).
Do a regex on the output of that command for "Some random error message. Failed." Where "Failed" is the thing I'm searching for but I want to capture the whole sentence.
Write the associated line to a file.
#!/bin/bash
COUNTER=0
while [ $COUNTER -lt 30 ]; do
command grunt check
// ERROR = regex(/\/Failed./)
// WRITE ERROR TO FILE
let COUNTER=COUNTER+1
done
for ((cr=0; cr<30; cr++))
do
grunt check | grep Failed
done > outfile.txt
counter=0
while [ $counter -lt 30 ]; do
grunt check | grep Failed
let counter=counter+1
done > some file
The above uses a pipeline to capture the output of the grunt command and sent it to grep. grep searches through the output and prints any lines that contain the word Failed. Any such lines are then sent to a file named somefile.
As a minor point, I have converted COUNTER to lower case. This is because the system uses upper case environment variables. If you make a practice of using lower case ones then you won't accidentally overwrite one. (In this particular case, there is no system variable named COUNTER, so you are safe.)
Another method for counting to 30:
You might find this simpler:
for counter in {1..30}; do
grunt check | grep Failed
done > somefile
The {1..30} notation provides the numbers from one to thirty. It is a bash feature so don't try to use it on a bare-bones POSIX shell.
To get more context
If you would like to see more context around the error message, grep offers several options to help. To see both the line matching "Failed" and the line before, use -B:
for counter in {1..30}; do
grunt check | grep -B 1 Failed
done >somefile
Similarly, -A can be used to display lines after the match. -C will display lines both before and after the match.
I have captured some application data in wireshark, (FIX messages) , and I am wondering if there is a way to export just the tcp data layer of each packet into a separate file, one row/line per packet.
I thought there used to be a function called tcp.data but this doesn't seem to exist in the version of wireshark I'm using (v1.10.1).
I had been planning on adding an extra column in Wireshark preferences, and setting it to type "custom" then putting tcp.data into the field. Then exporting this column to a text file.
Thanks in advance for any ideas or suggestions.
PS. the other idea i had was to write a script to parse the capture file and effectively chop off the leading headers in front of the tcp layer data, but this will take some time - hopefully a way exists already to do this within wireshark itself.
UPDATE 1:
Extending Isaac's solution, I have come up with the following, however this is actually printing the entire tcp segment, not just the data from within the segment. I've also tried tcp.segment_data but this also results in the same issue where more than the tcp data payload is getting outputted. Unfortunately, at the moment the best option looks like manually parsing the pcap file. Does anyone else have any suggestions, or perhaps spot what I've got wrong in the tshark command syntax?
tshark -r capture_chopped.pcap -c4 -2 -R "(tcp.len > 0)" -T fields -d tcp.port==2634,echo -e tcp.segment -w - > output.2
UPDATE 2 - ISSUE RESOLVED:
I found that every option with tshark didn't provide the entire info I needed, so I went with creating my own Perl script to fish out the FIX message strings from the pcap file directly. The Perl script is included below in case it is helpful to anyone else in a similar situation with a PCAP file.
You don't need a script, you can use the built-in wireshark tool called tshark. It is usually located at: c:\Program Files\Wireshark if you installed wireshark in the default folder.
Then you use the following command line and it will do the trick:
tshark -r c:\captures\your_file.cap -R "(tcp.len > 0)" -T fields -d tcp.port=3868,echo -e echo.data
Few things to note about the above:
It filters tcp packets that have no payload, remove it if you want to identify the empty ones
It assumes you know the protocol port that your file contain which is usually a reasonable assumption. In the above 3868, replace it with the protocol you are using.
Then redirect the output to a file and you are done.
In the end I found that creating a perl script to parse the PCAP file was sufficient for my needs - mainly due the other options of using tshark would have still needed some extra cleanup / manipulation (eg. converting the HEX into binary) so the script seemed like the best approach to me.
The perl script is included here in case it is useful for anyone else.
use strict;
use warnings;
my $in_file = shift || die "must give input pcap file as argument here. \n";
my $out_file = 'fix_output.txt';
open (my $out_fh, ">", $out_file) || die "error opening output file: [$out_file]. $! \n";
open (my $in_fh , "<", $in_file) || die "error opening input file: [$in_file]. $! \n";
my $pcap_blob = do {
local $/ = undef;
<$in_fh>;
};
my #FIX_strings = $pcap_blob =~ m/(8=FIX\.4\.0.*10=\d\d\d)/g;
print "found " . scalar #FIX_strings . " FIX messages\n";
for (#FIX_strings){
print {$out_fh} $_ , "\n";
}
print "finished writing FIX messages to $out_file\n";
close $out_fh;
close $in_fh;
print "all done\n";
exit;