Following is my code
oldname= open('oldportname.txt').read().split('\n')
newname =open('newportname.txt').read().split('\n')
find_replace = dict(zip(oldname, newname))
with open('data.txt') as data:
with open('data_newname.txt', 'w') as new_data:
for line in data:
for key in find_replace:
if key in line:
line = line.replace(key, find_replace[key])
new_data.write(line)
I have three files,
data.txt
AD_WEB_TCP9389
AOL
Bittorrent
BOOTPC
UDP_67
BOOTPS
UDP_68
EPM_TCP135
Oldportname(find):
AD_WEB_TCP9389
AOL
Bittorrent
BOOTPC
UDP_67
BOOTPS
UDP_68
EPM_TCP135
newname(Replace):
AD-WEB_TCP_9389
AOL
BITTORRENT
BOOTPS_UDP_67
BOOTPS_UDP_67
BOOTPC_UDP_68
BOOTPC_UDP_68
EPMAP_TCP_135
I am getting weird output
AD-WEB_TCP_9389(Correct)
AOL (Correct)
BITTORRENT (Correct)
BOOTPC_BOOTPC_UDP_68_BOOTPC_BOOTPC_UDP_68_UDP_67 (WRONG)
BOOTPC_BOOTPC_UDP_68_UDP_67 (WRONG)
BOOTPC_BOOTPC_UDP_68 (WRONG)
BOOTPC_UDP_68
EPMAP_EPMAP_TCP_135 (WRON)
Related
After my company purchased new servers I'm doing a top-down upgrade of the server room. since all the hardware is changing I'm not able to use bare-metal cloning tool to migrate. Using the newusers command from Debian I am able to create in bulk all the users from the old server. For the /etc/shadow file, you can copy the second column from your shadow.sync (from the old server) file into the second column of the associated account in the new system. This will transfer the passwords for your accounts to the new system. However I'm not sure how to do this programmatically using awk (or something else I can integrate into my shell script I already have setup).
shadow.sync contains the following (users & passwords changed for security reasons) This is the file to be copied INTO the current shadow file which looks almost identical except the data in the second column has the INCORECT values.
An in-depth explanation of the fields for the /etc/shadow file can be found here
user1:$6$HiwQEKYDgT$xYU9F3Wv0jFWHmZxN60nFMkTqWn87RRIOvx7Epp57rOmdHN9plJgjhC.jRVVNc1.HUaqSpX/ZcCEFSn6RmQQA0:17531::0:99999:7:::
user2:$6$oOuwJtrIKk$THLsfDppLI8QVw9xEOAaIoZ90Mcz3xGukVdyWGJJqygsavtXvtJ8X9ECc0CfuGzHp0pHNSAqdZY9TAzF5YKLc.:17531::0:99999:7:::
user3:$6$IEHAyRsokQ$e5K3RicE.PUAej8IxG9GnF/SUl1NQ57pqzUVuAzsP8.89SNhuaKE1W7kG5P4hbzV23Bb2zWHx353t.e9ERSVy.:17531::0:99999:7:::
user4:$6$lFOIUQvxdb$W5ITiH/Y021xw1vo8uw6ZtIOmfKjnNnC/SttQjN85MHtLbFeQ2Th5kfAIijXC81CRG4T0kJQ3rzRNRSyQHjyb1:17531::0:99999:7:::
user5:$6$RZbtYxWiwE$lnP8.tTbs0JbLZg5FsmPR8QvrJARbcRuJi2nYm1okwjfkWPkj212mBPjVF1BTo2hVCxLGSw64Cp6DgXheacSx.:17531::0:99999:7:::
Essentially i need to match column 1 (username) between the sync file and the shadow file and copy column 2 from the sync file over-top of the same column on the actual shadow file. Doing this by hand would be terrible as I have 90 servers that I'm migrating with over 900 users total.
Random shadow.sync file for demonstration was generated using:
#!/usr/bin/env python
import random, string, crypt, datetime
userList = ['user1','user2','user3','user4','user5']
dateNow = (datetime.datetime.utcnow() - datetime.datetime(1970,1,1)).days
for user in userList:
randomsalt = ''.join(random.sample(string.ascii_letters,10))
randompass = ''.join(random.sample(string.ascii_letters,10))
print("%s:%s:%s::0:99999:7:::" % (user, crypt.crypt(randompass, "$6$"+randomsalt), dateNow))
Please note this python script was ONLY for demonstration and not for actual production data. As users are added to the server the /etc/shadow file is generated with the password presented on the command line. The Original data (from shadow.sync) needs to be "Merged" with the data in /etc/shadow after the newusers command is run (which essentially sets every password to the letter x)
#!/usr/bin/env python
with open('/etc/shadow','rb') as file:
for line in file:
TargetLine = line.rstrip().split(":")
with open('shadow.sync','rb') as shadow:
for row in shadow:
SyncLine = row.rstrip().split(":")
if TargetLine[0] == SyncLine[0]:
TargetLine[1] = SyncLine[1]
break
print "NEW MODIFIED LINE: %s" % ":".join(TargetLine)
This will open the /etc/shadow file and loop through the lines. For each line on the /etc/shadow file we loop through the shadow.sync file once a match for the usernames TargetLine[0] == SyncLine[0] the password field is modified and the loop is broken.
If a match is NOT found (username in /etc/shadow but NOT in the shadow.sync file) the if block on the inner loop falls through and the line is left untouched the results are handled on the final print statement. As this answers the question I will leave the data output and file manipulation up to the user.
use Data::Dumper;
# we only need to process the sync file once -
# and store what we find in a hash (dictionary)
open $fh1, '<', 'shadow.sync.txt';
while (<$fh1>)
{
m/^([^:]+):(.*)$/;
$hash->{$1} = $2;
}
close $fh1;
# this shows us what we found & stored
print Dumper $hash;
# now we'll process the shadow file which needs updating -
# here we output a side-by-side comarison of what the passwords
# currently are & what they will be updated to (from the hash)
open $fh2, '<', 'shadow.txt';
open $fh3, '>', 'shadow.UPDATED.txt';
while (<$fh2>)
{
m/^([^:]+):(.*)$/;
printf ( "%s => %s\n", $1, $2 );
printf ( "%s => %s\n\n", $1, $hash->{$1} );
printf $fh3 ( "%s:%s\n", $1, $hash->{$1} );
}
close $fh3;
close $fh2;
Sample output:
$VAR1 = {
'user5' => '$6$RZbtYxWiwE$lnP8w64Cp6DgXheacSx.:17531::0:99999:7:::',
'user1' => '$6$HiwVVNc1.HUaqSpX/ZcCEFSn6RmQQA0:17531::0:99999:7:::',
'user4' => '$6$lFOIUQv1CRG4T0kJQ3rzRNRSyQHjyb1:17531::0:99999:7:::',
'user3' => '$6$P8.89SNhu23Bb2zWHx353t.e9ERSVy.:17531::0:99999:7:::',
'user2' => '$6$Cc0CfuGzHp0pHNSAqdZY9TAzF5YKLc.:17531::0:99999:7:::'
};
user1 => $6$RANDOM1RANDOM1RANDOM1RANDOM1:17531::0:99999:7:::
user1 => $6$HiwVVNc1.HUaqSpX/ZcCEFSn6RmQQA0:17531::0:99999:7:::
user2 => $6$RANDOM2RANDOM2RANDOM2RANDOM2:17531::0:99999:7:::
user2 => $6$Cc0CfuGzHp0pHNSAqdZY9TAzF5YKLc.:17531::0:99999:7:::
user3 => $6$RANDOM3RANDOM3RANDOM3RANDOM3:17531::0:99999:7:::
user3 => $6$P8.89SNhu23Bb2zWHx353t.e9ERSVy.:17531::0:99999:7:::
user4 => $6$RANDOM4RANDOM4RANDOM4RANDOM4:17531::0:99999:7:::
user4 => $6$lFOIUQv1CRG4T0kJQ3rzRNRSyQHjyb1:17531::0:99999:7:::
user5 => $6$RANDOM5RANDOM5RANDOM5RANDOM5:17531::0:99999:7:::
user5 => $6$RZbtYxWiwE$lnP8w64Cp6DgXheacSx.:17531::0:99999:7:::
Input file:File name='sample1.txt'
lion is a good friend (Host=lion) (Port=animal) and tiger is
(Host=Tiger)(Port=an)
burger is a food (Host=Burger)(Port=Food)
I have data as shown in the above txt file.I want to collect the host and port in each line from a txt file and place them in new txt file
Required Outfile:
lion:animal
Tiger:an
Burger:Food
Code used till nw:
cat sample1.txt | perl -ne 'print "$1=$2\n" if(/Host=([\w.]*.'-'*[\w.]*.).*Port=(\d+)/)' > sample2.txt
sed 's|[()]||g' sample2.txt > sample3.txt
Obtained output:
lion:animal
Burger:Food
Not getting the Tiger and an:
Problem : I am not able to get the host and port in same line which is present more then once..i some line it have only one host and port value..in other line there are more than one host and port value..pls help me to slove this ..thank you ..:)
If you want to have all entries in the order of appearance, you can use something like that:
perl -nle 's{Host=([^)]*).*?Port=([^)]*)}{print "$1:$2"}ge' < sample1.txt > sample3.txt
I have implemented a script to parse ERF file to get the DNS records from the packets. The script works under Linux but DOES NOT work under Windows.
I have tried to simplify it and read only two packets from the file and the result was totally wrong.
Here is the output for the first two packets:
rlen 1232
wlen 1213
ts: (5822080496043415499L,)
rec len: 1232
protocol: 6 rem 1180
tcp
src port: 59626
remaining length based on the erf header 1160 remaining length based in IP total length 1155
----------------------
rlen 44076
wlen 13638
ts: (246640611164160L,)
rec len: 44076
protocol: 9 rem 44024
----------------------
for the first packet the output is correct, but for the second one everything is wrong. What I did was reading the record length from the ERF header to keep track of the packet boundaries. When I printed the payload of the tcp I found that the erf header of the next packet was in the payload of tcp. This problem didn't occur when I ran the code under linux.
Can anyone tell me what I'm doing wrong?
Here is my code:
if __name__ == '__main__':
argv= sys.argv
outputFile=''
inputFile=''
dnsPacketCounter=0
ethH = {}
ipHeader = {}
ipH = {}
totalPackets=0
if len(argv) ==1:
print 'erfParser.py -i <inputfile> -o <outputfile>'
sys.exit(0)
elif len(argv) == 2:
if argv[1] == '-h':
print 'erfParser.py -i <inputfile> -o <outputfile>'
sys.exit(0)
elif len(argv) == 5:
if argv[1] == '-i':
inputFile = argv[2].strip()
elif argv[3] == '-i':
inputFile = argv[4].strip()
if argv[1] == '-o':
outputFile = argv[2].strip()
elif argv[3] == '-o':
outputFile= argv[4].strip()
else:
# Open the trace file
print 'erfParser.py -i <inputfile> -o <outputfile>'
sys.exit(0)
try:
packets = open(inputFile , 'r+')
except IOError:
print 'The file: ',inputFile,' not found.'
sys.exit(0)
try:
outFile=open(outputFile+'.txt', 'w+')
except IOError:
print 'The file: ',outputFile,' can not be opened.'
sys.exit(0)
ts=packets.read(8)
i=0
while ts:
erf={}
hdr = packets.read(8)
#print ts.encode('hex')
totalPackets=totalPackets+1
erf= getERFHeader(ts,hdr)
print 'rlen',erf['rlen']
print 'wlen',erf['wlen']
print 'ts: ',erf['ts']
remainingLength=erf['rlen']- 16
print 'rec len: ',erf['rlen']
if erf['type'] == 0x07:
ext=packets.read(8)
remainingLength=remainingLength- 8
pad=packets.read(2) # pad
remainingLength=remainingLength- 2
ethH= packets.read(14) # ethernet header `16 bytes
remainingLength=remainingLength- 14
ipHeader= packets.read(20) #ip header length is 20 bytes
remainingLength=remainingLength- 20
ipH= getIPHeader(ipHeader)
print 'protocol: ',ipH['protocol'],' rem ',remainingLength
if ipH['protocol'] ==TCP:
print 'tcp'
hdr = packets.read(20)
remainingLength=remainingLength- 20
tcpHeader=getTCPHeader(hdr)
tcpPayload= packets.read(remainingLength)
print 'src port: ',tcpHeader['srcPort']
# print 'tcp payload in hex: ',tcpPayload.encode('hex')
print 'remaining length based on the erf header',remainingLength,'remaining length based in IP total length' ,ipH['totalL']-40
print '----------------------'
ts=packets.read(8)
i=i+1
if i==2:
break;
pass
Can anyone tell me what I'm doing wrong?
Yes, I can tell you that you're opening the file in text mode rather than binary mode:
packets = open(inputFile , 'r+')
To quote the Python documentation for open():
Modes 'r+', 'w+' and 'a+' open the file for updating (reading and writing); note that 'w+' truncates the file. Append 'b' to the mode to open the file in binary mode, on systems that differentiate between binary and text files; on systems that don’t have this distinction, adding the 'b' has no effect.
UN*Xes such as Linux are "systems that don't have this distinction", because the Python open() is modeled after the UN*X version of the "standard I/O library", in which lines end with a \n. On Windows, lines end with \r\n, and opens in the "standard I/O library" can either:
open in text mode, in which the \r\n at the ends of lines, when read, are shown to the program as a \n, and a \n, when written to a line, is written as \r\n, so that programs written for UN*X can work on Windows without having to worry about the end-of-line sequence;
open in binary mode, in which case a read gives you exactly the bytes that are in the file, and a write puts the bytes given to it into the file;
so it's a system that "[differentiates] between binary and text files", at least in some of the I/O libraries. (At the lowest level of I/O, namely the CreateFile(), ReadFile(), and WriteFile() calls, Windows makes no such distinction - it treats files as sequences of raw bytes, with no "open as text" option, just as UN*X systems do with open(), read(), and write() - but at all levels of I/O intended to be somewhat UN*X-compatible, they provide a text vs. binary option.)
ERF files are binary files, so you need to open with 'rb+' or 'r+b', not 'r+'. That will make no difference on UN*Xes such as Linux, but will give you raw binary data on Windows.
(Actually, just 'rb' will suffice - if you don't plan to write to the file you're reading, the + isn't necessary, and creates the risk of accidentally overwriting the file.)
I am trying to extract CDS and corresponding amino acid sequences from GenBank file using BioPerl. The script is shown below:
while (my $seq_object = $in->next_seq){
for my $feat_object ($seq_object->get_SeqFeatures) {
if ($feat_object ->primary_tag eq "CDS") {
# warn("all tags are ", join ("," , $feat_object->get_all_tags),"\n");
if ($feat_object->has_tag ("protein_id")){
my ($protein_id) = $feat_object->get_tag_values('protein_id');
my ($pseq) = $feat_object->get_tag_values('translation') ;
my ($pepseq) = Bio::Seq->new(-id => $protein_id , -description => $seq_object -> accession_number,
-seq => $pseq);
$out->write_seq($pepseq);
}
}
}
}
I am getting error message as:
Filehandle GEN1 opened only for input at /Library/Perl/5.12/Bio/Root/IO.pm line 533, line 148.
Kindly help me to solve this issue.
Thanks in advance
I'll add this as an answer since it is likely the source of the error. When creating a Bio::SeqIO object for output, you must follow the normal Perl rules for open and specify the file is for output. So, try the following:
my $out = Bio::SeqIO->new( -file => ">Oct_test.fasta", -format => 'fasta');
This is a really easy thing to forget and the error message could be a bit more descriptive.
Hey, my Qt C++ program has a part where it needs to send the first 128 characters or so of the output of a bash command to an email address. The output from the tty is captured in a text box in my gui called textEdit_displayOutput and put into my message I built using the Message Builder ( the object m_vmMessage ) Here is the relevant code snippet:
m_vmMessage.getTextPart()->setCharset( vmime::charsets::US_ASCII );
m_vmMessage.getTextPart()->setText( vmime::create < vmime::stringContentHandler > ( ui->textEdit_displayOutput->toPlainText().toStdString() ) );
vmime::ref < vmime::message > msg = m_vmMessage.construct();
vmime::utility::outputStreamAdapter out( std::cout );
msg->generate( out );
Giving bash 'ls /' and a newline makes vmime give terminal output like this:
ls /=0Abin etc=09 initrd.img.old mnt=09 sbin=09 tmp=09 vmlinuz.o=
ld=0Aboot farts=09 lib=09=09 opt=09 selinux usr=0Acdrom home=09 =
lost+found=09 proc srv=09 var=0Adev initrd.img media=09 root =
Whereas it should look more like this:
ls /
bin etc initrd.img.old mnt sbin tmp vmlinuz.old
boot farts lib opt selinux usr
cdrom home lost+found proc srv var
dev initrd.img media root sys vmlinuz
18:22>
Output seems to be truncated around 'root', nothing after it is displayed.
How do I encode and piece together the email properly? Does vmime just display it like that on purpose and the actual content of the email is complete and properly formatted?
Thanks!
=0A is a line feed (LF) character.
=09 is a horizontal tab (HT).
I think this is just MIME's way of encoding your non-printing (control) characters.