Need advise how to make the third list from a couple of lists.
The first one is like this (about 20000 rows):
LIST1.
field1 field2 field3 field4 field5 field6 field7
---------------------------------------------------------------------------
1167 28669 001f.ce5d.cb4d Gi0/0/1.10 1 Vi2.156 PTA
848 32350 c83a.350d.f368 Gi0/0/1.10 1 Vi2.601 PTA
1771 43465 c46e.1f7a.4763 Gi0/0/1.10 1 Vi2.959 PTA
1390 24116 dc9f.db01.c6e8 Gi0/0/1.10 1 Vi2.32 PTA
712 23579 d850.e6d5.cb1c Gi0/0/1.10 1 Vi2.436 PTA
1239 28354 2828.5dd4.bc65 Gi0/0/1.10 1 Vi2.78 PTA
204 27816 e03f.491d.9978 Gi0/0/1.10 1 Vi2.341 PTA
383 28368 60e3.278c.7199 Gi0/0/1.10 1 Vi2.114 PTA
671 54657 c46e.1f81.a3d3 Gi0/0/1.10 1 Vi2.224 PTA
The second one is like this (about 20000 rows):
LIST2
field1 field2 field3 field4 field5
---------------------------------------------------------------------
Vi2.1 0001799 PPPoE 00:00:08 10.100.146.30
Vi2.2 0010129 PPPoE 00:00:08 10.100.148.108
Vi2.4 0010173 PPPoE 00:00:08 10.100.150.56
Vi2.5 0011093 PPPoE 00:00:08 10.100.146.143
Vi2.6 0003301 PPPoE 00:43:48 10.100.150.107
Vi2.7 0010101 PPPoE 00:00:08 10.100.147.133
Vi2.8 0001859 PPPoE 00:00:08 10.100.145.223
Vi2.9 0010049 PPPoE 06:45:08 10.100.147.138
Vi2.10 0003515 PPPoE 00:00:28 10.100.146.173
Vi2.11 0001747 PPPoE 00:00:18 10.100.146.37
Vi2.12 0011060 PPPoE 04:40:28 10.100.149.165
Vi2.13 0001335 PPPoE 00:00:08 10.239.152.165
Vi2.14 0010154 PPPoE 00:00:08 10.100.148.68
I need to create the third list, and the order is needed in such the way:
field6(list1) Field2(list1) field3(list1) field2(list2) field5(list2)
By the way. Field6 from list1 is the same as field1 in list 2.
I do understand, that I need to take every row from list1, make it a list of fields and after that take the field 6 and go to list2 and search that value in the list2.
And after that gather all needed fields into a new row. Anybody, I'm very,very new to parsing, please give me a couple of examples how to deal with this (I think typical) task!
Clarifying.
I'm receiving that rows via python 3 telnetlib, like this:
import telnetlib
HOST = '2.22.22.22'
password = "user"
user = "user"
tn = telnetlib.Telnet(HOST)
tn.read_until(b"Username: ")
tn.write(user.encode('ascii') + b"\n")
tn.read_until(b"Password: ")
tn.write(password.encode('ascii') + b"\n")
tn.write(b"term len 0 \n")
tn.write(b"show pppoe session | exclude 7878.7878.7878 \n")
tn.write(b"\n exit\n")
mystring = tn.read_all().decode('ascii').replace('\r\n', '\n')
print(mystring)
temp_list = mystring.splitlines()
print(temp_list)
mylist = ["\n".join(s for s in temp_list if 'PTA' in s and 'Vi2' in s)]
You might want to hack numpy. loadtxt method loads tabular data from the text file and transforms it to array (not native python).
With this handy method, you can easily achieve the result.
>>> import numpy as np
>>> t1 = "/home/ziya/Projects/python/test/list1.txt"
>>> t2 = "/home/ziya/Projects/python/test/list1.txt"
>>> d1 = np.loadtxt(t1, skiprows=2, dtype='str')
>>> d2 = np.loadtxt(t2, skiprows=2, dtype='str')
>>> d1_field1 = [i[0] for i in d1]
>>> d2_field1 = [i[1] for i in d1]
>>> d3_field1 = [i[2] for i in d1]
>>> d4_field2 = [i[3] for i in d2]
>>> d5_field2 = [i[4] for i in d2]
>>> d6_field2 = [i[5] for i in d2]
>>> new_list = []
>>> new_list.append(d1_field1)
>>> new_list.append(d2_field1)
>>> new_list.append(d3_field1)
>>> new_list.append(d4_field2)
>>> new_list.append(d5_field2)
>>> new_list.append(d6_field2)
>>> new_list
[['1167', '848', '1771', '1390', '712', '1239', '204', '383', '671'], ['28669', '32350', '43465', '24116', '23579', '28354', '27816', '28368', '54657'], ['001f.ce5d.cb4d', 'c83a.350d.f368', 'c46e.1f7a.4763', 'dc9f.db01.c6e8', 'd850.e6d5.cb1c', '2828.5dd4.bc65', 'e03f.491d.9978', '60e3.278c.7199', 'c46e.1f81.a3d3'], ['Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10', 'Gi0/0/1.10'], ['1', '1', '1', '1', '1', '1', '1', '1', '1'], ['Vi2.156', 'Vi2.601', 'Vi2.959', 'Vi2.32', 'Vi2.436', 'Vi2.78', 'Vi2.341', 'Vi2.114', 'Vi2.224']]
>>> np.array(new_list).transpose()
array([['1167', '28669', '001f.ce5d.cb4d', 'Gi0/0/1.10', '1', 'Vi2.156'],
['848', '32350', 'c83a.350d.f368', 'Gi0/0/1.10', '1', 'Vi2.601'],
['1771', '43465', 'c46e.1f7a.4763', 'Gi0/0/1.10', '1', 'Vi2.959'],
['1390', '24116', 'dc9f.db01.c6e8', 'Gi0/0/1.10', '1', 'Vi2.32'],
['712', '23579', 'd850.e6d5.cb1c', 'Gi0/0/1.10', '1', 'Vi2.436'],
['1239', '28354', '2828.5dd4.bc65', 'Gi0/0/1.10', '1', 'Vi2.78'],
['204', '27816', 'e03f.491d.9978', 'Gi0/0/1.10', '1', 'Vi2.341'],
['383', '28368', '60e3.278c.7199', 'Gi0/0/1.10', '1', 'Vi2.114'],
['671', '54657', 'c46e.1f81.a3d3', 'Gi0/0/1.10', '1', 'Vi2.224']],
dtype='|S14')
Related
I'm trying to get duos and trios, but seems that my list is constructed little bit differently. Is there solution for this. I have exported lottery number to csv and after some modification imported back here is the code
import csv
import pandas as pd
from itertools import combinations
from collections import Counter
results = []
with open('numerot.csv', newline='') as inputfile:
for row in csv.reader(inputfile):
results.append(row[0])
results=[i.replace(';',',') for i in results]
print(results)
duos = Counter()
trios = Counter()
for draw in results:
duos.update(combinations(draw, 2))
trios.update(combinations(draw, 3))
print('Top 5 duos')
for x in duos.most_common(5):
print(f'{x[0]}: {x[1]}')
print()
print('Top 5 trios')
for x in trios.most_common(5):
print(f'{x[0]}: {x[1]}')
and result.
['1;3;7;17;34;35;39;33', '2;4;6;30;32;33;35;11', '9;12;13;25;26;36;40;29', '2;7;11;22;30;33;35;23', '7;12;13;17;18;24;31;10', '2;5;8;11;15;19;29;38', '7;12;14;17;21;28;35;19', '19;20;23;27;29;32;34;8', '4;17;22;25;27;29;30;1', '1;7;15;19;20;35;38;32', '6;16;17;21;23;30;36;40', '1;6;8;9;15;30;40;17', '9;15;17;18;19;22;30;11', '2;6;13;14;29;30;38;40', '1;13;24;29;31;37;39;32', '8;10;15;19;32;38;40;34', '7;16;19;26;27;30;31;14', '4;5;7;9;19;24;37;40', '13;18;19;27;30;33;40;32', '2;7;12;17;27;38;39;25', '3;6;9;14;16;25;33;2', '3;9;24;25;30;32;34;13', '9;22;28;31;32;34;40;20', '3;4;14;19;20;28;33;7', '17;21;25;27;28;37;40;1', '1;7;16;23;25;32;33;9', '2;18;24;31;33;38;39;23', '15;18;22;25;29;32;34;23', '19;20;22;32;35;36;39;4', '2;8;22;28;32;34;39;14', '9;11;12;13;24;28;34;5', '8;22;27;28;31;34;39;30', '4;8;25;28;29;31;38;21', '4;15;18;23;35;36;40;22', '3;6;12;24;27;28;30;19', '2;7;12;24;26;37;39;29', '1;5;10;12;17;21;22;40', '4;5;10;14;25;26;29;19', '4;5;9;10;15;31;40;1', '1;16;24;27;28;36;40;38', '2;17;19;20;21;30;38;32', '10;12;13;20;24;33;36;8', '3;8;23;28;36;38;39;30', '1;10;13;30;34;35;39;6', '4;7;10;15;16;29;36;23', '7;12;15;22;27;28;31;20', '5;10;19;21;26;32;38;6', '4;13;16;17;18;21;26;20', '4;6;10;15;27;29;40;36', '4;15;26;31;33;34;38;19', '9;13;16;24;25;35;36;10', '1;13;19;28;30;33;35;7', '2;3;7;13;14;22;27;32', '5;6;7;14;27;37;38;40', '10;13;17;19;21;25;30;31']
Top 5 duos
(';', ';'): 1155
(';', '3'): 732
('1', ';'): 603
(';', '2'): 561
('2', ';'): 524
Top 5 trios
(';', ';', ';'): 1925
(';', ';', '3'): 1621
(';', '2', ';'): 1310
('1', ';', ';'): 1276
I have tried to change csv file and import with pandas but no success
I have a list of lists in the following format:
[['a'],['1'],['2'],['3'], ['b'],['4'],['5'],['6']]
My desired output is:
[['a', '1'], ['a', '2'], ['a','3'],['b', '4'],['b', '5'],['b', '6']]
or even better would be:
{'a':['1','2','3'], 'b':['4','5','6']}
Essentially, the "number values" are never the same size (think that a could include 1 2 and 3, and b could include 4 5 6 7 and 8, etc)
What would be the easiest way of doing this? Using regex?
Thanks
You can use a for loop and check if the element is a digit or not:
d = {}
for i in lst:
if not i[0].isdigit(): # Check if element is a digit. If not, add a key with the current value of i[0]
d[i[0]] = []
current = i[0]
else:
d[current].append(i[0])
Output:
>>> d
{'a': ['1', '2', '3'], 'b': ['4', '5', '6']}
This is assuming everything in the list is a string
df = pd.read_csv("data.csv", index_col = 'endDate', parse_dates = True)
endDate in csv
2016-05-06 15:01:01 -0400
endDate index for data frame in pandas after import
2016-05-06 19:01:01
How do I ignore the -0400 offset in the csv?
I think the best thing here is to not parse initially so you can strip the timezone information away and then convert:
In [14]:
t="""date
2016-05-06 15:01:01 -0400"""
df = pd.read_csv(io.StringIO(t),index_col=[0])
df.index = df.index.str.rsplit().str[0:-1].str.join(' ')
df.index = pd.to_datetime(df.index)
df
Out[14]:
Empty DataFrame
Columns: []
Index: [2016-05-06 15:01:01]
So here the dates import as str so we now remove the timezone using str.rsplit and then join again but without the timezone info. We can then use to_datetime to convert to datetime
i would do it this way:
filename = '/path/to/file.csv'
df = pd.read_csv(filename, index_col='endDate', parse_dates=['endDate'],
date_parser=lambda x: pd.to_datetime(x.rsplit(' ', 1)[0]))
Test:
data = """\
endDate,val
2016-05-06 15:01:01 -0400,11
2016-05-06 20:20:20 -0100,12
"""
df = pd.read_csv(io.StringIO(data), index_col='endDate', parse_dates=['endDate'],
date_parser=lambda x: pd.to_datetime(x.rsplit(' ', 1)[0]))
Output:
In [119]: df = pd.read_csv(io.StringIO(data), index_col='endDate', parse_dates=['endDate'],
.....: date_parser=lambda x: pd.to_datetime(x.rsplit(' ', 1)[0]))
In [120]: df
Out[120]:
val
endDate
2016-05-06 15:01:01 11
2016-05-06 20:20:20 12
You can use a list comprehension together with str.split().
df = pd.DataFrame({'endDate': ['2016-05-06 15:01:01 -0400', '2016-05-06 16:01:01 -0400']})
df['endDate'] = pd.to_datetime([date + " " + time
for date, time, _ in df.endDate.str.split()])
>>> df
endDate
0 2016-05-06 15:01:01
1 2016-05-06 16:01:01
I'm trying to let the dictionary called
theInventory = {}
to sort the following items, so i can add price or view the books by authors and etc.
It's my homework problem. so i need to use dictionary and multi-value using 2D list
Let the some text file called database.txt contains
last name/first name/quantity/price
Shakespeare,William,Romeo And Juliet,5,5.99
Shakespeare,William,Macbeth,3,7.99
Dickens,Charles,Hard Times,7,27.00
Austin,Jane,Sense And Sensibility,2,4.95
Is it possible to do the following?
inFile = (database.txt, "r")
For line in inFile:
aLine = []
aLine = line.split(",") # dont know how to split by ,
theInventory[aLine[0] + ", " + aLine[1]] = list[list[aLine[3], int(aLine[4]), float(aLine[5])]]
inFile.close()
the result will be like
>print (theInventory)
>> {"Shakespeare, William": [["Romeo And Juliet", 5, 5.99], ["Macbeth", 3, 7.99]], "Dickens, Charles": [["Hard Times", 7, 27.00]], "Austin, Jane": [["Sense And Sensibility", 2, 4.95]]}
so that you can modify quantity and price of the certain book.
or even add books to dictionary.
the file looks like a CSV file. Python has a module named as csv which enables the user to read/write rcsv files.
Below is the example code as given in the python document.
import csv
with open('database.txt', 'rb') as f:
reader = csv.reader(f, delimiter=',', quoting=csv.QUOTE_NONE)
for row in reader:
print row
output:
['Shakespeare', 'William', 'Romeo And Juliet', '5', '5.99']
['Shakespeare', 'William', 'Macbeth', '3', '7.99']
['Dickens', 'Charles', 'Hard Times', '7', '27.00']
['Austin', 'Jane', 'Sense And Sensibility', '2', '4.95']
>>>
more information about the csv module can be found here:https://docs.python.org/2/library/csv.html
edit: this is something very basic, but might given you an idea
import csv
dicto = {}
name = ''
#dicto[name] = []
with open('database.txt', 'rb') as f:
reader = csv.reader(f, delimiter=',', quoting=csv.QUOTE_NONE)
for row in reader:
name = row[0] + ',' + row[1] # frames the name
if name in dicto.keys(): # checks if the name exists in the dictionary , if yes just append the book
books = dicto.get(name)
books.append(row[2:])
else: # if no entry of author is made, create new book list
books = []
books.append(row[2:])
dicto[name] =books # update the list of books
print dicto
output:
{'Dickens,Charles': [['Hard Times', '7', '27.00']], 'Shakespeare,William': [['Romeo And Juliet', '5', '5.99'], ['Macbeth', '3', '7.99']], 'Austin,Jane': [['Sense And Sensibility', '2', '4.95']]}
How to unpack a list, which is nested inside another list.
Practically to transform this:
l=['aa','bb',['ccc','ddd'],'ee']
to
l=['aa','bb','ccc','ddd','ee']
See this thread and e. g. the solution of elqott
>>> from compiler.ast import flatten
>>> l = ['1','2',['3','4'],'5']
>>> flatten(l)
Following your edit
['1', '2', '3', '4', '5']
>>> l = ['aa','bb',['ccc','ddd'],'ee']
>>> flatten(l)
['aa', 'bb', 'ccc', 'ddd', 'ee']