List Comparison in Python with Diff as output - python-2.7

The below script i have to compare Test1 vs Test2.Test1 and Test2 data is mentioned in the bottom .I tried to make it a generic one so that it will work for different devices also.The below script i have to compare Test1 vs Test2.Test1 and Test2 data is mentioned in the bottom .I tried to make it a generic one so that it will work for different devices also
import re
data_cleaned = {}
current_key = ''
action_flag = False
data_group = []
if_found_vlan = True
output = open('./output.txt','r').read()
switch_red = re.findall(r'(\w*-RED\d{0,1})', output)[0]
switch_blue = re.findall(r'(\w*-BLUE\d{0,1})', output)[0]
for line in open('./output.txt'):
m = re.match(r'(\w*-RED\d{0,1}|\w*-BLUE\d{0,1})# sh run vlan \d+', line)
if m:
if not if_found_vlan:
data_cleaned[current_key].append([])
if_found_vlan = False
current_key = m.group(1)
if not data_cleaned.has_key(current_key):
data_cleaned[current_key] = []
continue
mm = re.match(r'vlan \d+', line)
if mm:
if_found_vlan = True
action_flag = True
data_group = []
if action_flag and '' == line.strip():
action_flag = False
data_cleaned[current_key].append(data_group)
if action_flag:
data_group.append(line.replace('\r', '').replace('\n', ''))
if not if_found_vlan:
data_cleaned[current_key].append([])
#print ("+++++++++++++++++ The missing configuration ++++++++++++++\n")
print switch_blue + "#" + " has below missing VLAN config\n "
p = [item for index, item in enumerate(data_cleaned[switch_blue]) if [] != [it for it in item if it not in data_cleaned[switch_red][index]]]
print('\n'.join(['\n'.join(item) for item in p]))
print ("+++++++++++++++++++++++++++++++\n")
print switch_red + "#" + " has below missing VLAN config\n "
q = [item for index, item in enumerate(data_cleaned[switch_red]) if [] != [it for it in item if it not in data_cleaned[switch_blue][index]]]
print('\n'.join(['\n'.join(item) for item in q]))

Update with your raw output, I think use a one-dimensional list to represent your output is not a good way for further handling.
When we handle a data, we first need to clean the data & setup a model easy to handle for further program processing, so I use a dict with a two-dimensional list inside it to model your output, and finally easier to process.
import re
data_cleaned = {}
current_key = ''
action_flag = False
data_group = []
if_found_vlan = True
for line in open('./output.txt'):
m = re.match(r'(Test\d+)# sh run vlan \d+', line)
if m:
if not if_found_vlan:
data_cleaned[current_key].append([])
if_found_vlan = False
current_key = m.group(1)
if not data_cleaned.has_key(current_key):
data_cleaned[current_key] = []
continue
mm = re.match(r'vlan \d+', line)
if mm:
if_found_vlan = True
action_flag = True
data_group = []
if action_flag and '' == line.strip():
action_flag = False
data_cleaned[current_key].append(data_group)
if action_flag:
data_group.append(line.replace('\r', '').replace('\n', ''))
if not if_found_vlan:
data_cleaned[current_key].append([])
print ("+++++++++++++++++ The missing configuration is++++++++++++++\n")
p = [item for index, item in enumerate(data_cleaned['Test2']) if [] != [it for it in item if it not in data_cleaned['Test1'][index]]]
print('\n'.join(['\n'.join(item) for item in p]))
print ("+++++++++++++++++ The missing configuration is++++++++++++++\n")
q = [item for index, item in enumerate(data_cleaned['Test1']) if [] != [it for it in item if it not in data_cleaned['Test2'][index]]]
print('\n'.join(['\n'.join(item) for item in q]))

Related

Group Items together dictionary while loop

I am having trouble storing the ID to keys, like a sub (parent-child) kind of thing. I spent hours on it and could not figure a way to accomplish this. What output I am expecting is at the end of this post. Any help would be great.
import sys
import collections
dict = collections.OrderedDict()
dict["A.1"] = {"parent_child":0}
dict["A.1.1"] = {"parent_child":1}
dict["A.1.1.1"] = {"parent_child":2}
dict["A.1.1.2"] = {"parent_child":2}
dict["A.1.1.3"] = {"parent_child":2}
dict["A.1.2"] = {"parent_child":1}
dict["A.1.2.1"] = {"parent_child":2}
dict["A.1.2.2"] = {"parent_child":2}
dict["A.1.2.2.1"] = {"parent_child":3}
dict["A.1.2.2.2"] = {"parent_child":3}
dict["A.1.2.3"] = {"parent_child":2}
dict["A.1.3"] = {"parent_child":1}
dict["A.1.4"] = {"parent_child":1}
print(dict)
new_dict = {}
p = 0 # previous index
i = 0 # current
n = 1 # next index
current_PC = 0 # current parent_child
next_PC = 0 # next parent_child
previous_id = ""
current_id = ""
next_id = ""
change_current = True
change = True
lst = []
while(True):
if change_current:
current_id = dict.keys()[i]
current_PC = dict.values()[i]["parent_child"]
change_current = False
try:
next_id = dict.keys()[n]
next_PC = dict.values()[n]["parent_child"]
except:
pass # it will go out of index
print("KEY {0}".format(current_id))
if next_PC > current_PC:
if next_PC - current_PC == 1:
lst.append(next_PC)
next_PC += 1
print("next_PC: {0}".format(next_PC))
if next_PC == current_PC:
new_dict[current_id] = lst
lst = []
break
print(new_dict)
Trying to make output looks like this (at in similar way), the new_dict should look like:
new_dict["A.1"] = ["A.1.1", "A.1.2", "A.1.3", "A.1.4"]
new_dict["A.1.1"] = ["A.1.1.1", "A.1.1.2", "A.1.1.3"]
new_dict["A.1.1.1"] = []
new_dict["A.1.1.2"] = []
new_dict["A.1.1.3"] = []
new_dict["A.1.2"] = ["A.1.2.1", "A.1.2.2", "A.1.2.3"]
new_dict["A.1.2.1"] = []
new_dict["A.1.2.2"] = ["A.1.2.2.1", "A.1.2.2.2"]
new_dict["A.1.2.2.1"] = []
new_dict["A.1.2.2.2"] = []
new_dict["A.1.2.3"] = []
new_dict["A.1.3"] = []
new_dict["A.1.4"] = []
This gives you the output you are asking for. Since i did not see a {"parent_child":...} in you desired output i did not proceed with anything else.
options = ["A.1","A.1.1","A.1.1.1","A.1.1.2","A.1.1.3","A.1.2","A.1.2.1","A.1.2.2","A.1.2.2.1","A.1.2.2.2","A.1.2.3","A.1.3","A.1.4"]
new_dict = {}
for i, key in enumerate(options):
new_dict[key] = []
ls = []
for j, opt in enumerate(options):
if (key in opt) and (len(opt)-len(key)==2):
new_dict[key].append(opt)
print(new_dict)
EDIT
Using the comment of #Ranbir Aulakh
options = ["A.1","A.1.1","A.1.1.1","A.1.1.2","A.1.1.3","A.1.2","A.1.2.1","A.1.2.2","A.1.2.2.1","A.1.2.2.2","A.1.2.3","A.1.3","A.1.4"]
new_dict = {}
for i, key in enumerate(options):
new_dict[key] = []
ls = []
for j, opt in enumerate(options):
if (key in opt) and (len(opt.split("."))-len(key.split("."))==1):#(len(opt)-len(key)==2):
new_dict[key].append(opt)
print(new_dict)

Code not recognizing attribute in SOAP response while attribute is being printed

I am working with ExactTarget FUEL SDK to retrieve data from the SalesForce Marketing Cloud. More specifically I working on calling "Unsub Events"(https://github.com/salesforce-marketingcloud/FuelSDK-Python/blob/master/objsamples/sample_unsubevent.py#L15) but the structure of the SOAP response has couple of deeper nested dictionary objects, which I need to iterate over and place into dataframes. Here is what the response looks like and I need to place each of the variables into seperate dataframe.
(UnsubEvent){
Client =
(ClientID){
ID = 11111111
}
PartnerKey = None
CreatedDate = 2016-07-13 13:37:46.000663
ModifiedDate = 2016-07-13 13:37:46.000663
ID = 11111111
ObjectID = "11111111"
SendID = 11111111
SubscriberKey = "aaa#aaa.com"
EventDate = 2016-07-13 13:37:46.000663
EventType = "Unsubscribe"
TriggeredSendDefinitionObjectID = None
BatchID = 1
List =
(List){
PartnerKey = None
ID = 11111111
ObjectID = None
Type = "aaaa"
ListClassification = "aaa"
}
IsMasterUnsubscribed = False
}]
I have successfully placed all variables in data frames except one "ListClassification". I am getting the error "List instance has no attribute 'ListClassification', my question is why is this happening if I can see the attribute in the response? and is there a fix for the issue?
My Code:
import ET_Client
import pandas as pd
try:
debug = False
stubObj = ET_Client.ET_Client(False, debug)
print '>>>UnsubEvents'
getUnsubEvent = ET_Client.ET_UnsubEvent()
getUnsubEvent.auth_stub = stubObj
getResponse3 = getUnsubEvent.get()
ResponseResultsUnsubEvent = getResponse3.results
#print ResponseResultsUnsubEvent
ClientIDUnsubEvents = []
partner_keys3 = []
created_dates3 = []
modified_date3 = []
ID3 = []
ObjectID3 = []
SendID3 = []
SubscriberKey3 = []
EventDate3 = []
EventType3 = []
TriggeredSendDefinitionObjectID3 = []
BatchID3 = []
IsMasterUnsubscribed = []
ListPartnerKey = []
ListID = []
ListObjectID = []
ListType = []
ListClassification = []
for UnsubEvent in ResponseResultsUnsubEvent:
ClientIDUnsubEvents.append(str(UnsubEvent['Client']['ID']))
partner_keys3.append(UnsubEvent['PartnerKey'])
created_dates3.append(UnsubEvent['CreatedDate'])
modified_date3.append(UnsubEvent['ModifiedDate'])
ID3.append(UnsubEvent['ID'])
ObjectID3.append(UnsubEvent['ObjectID'])
SendID3.append(UnsubEvent['SendID'])
SubscriberKey3.append(UnsubEvent['SubscriberKey'])
EventDate3.append(UnsubEvent['EventDate'])
EventType3.append(UnsubEvent['EventType'])
TriggeredSendDefinitionObjectID3.append(UnsubEvent['TriggeredSendDefinitionObjectID'])
BatchID3.append(UnsubEvent['BatchID'])
IsMasterUnsubscribed.append(UnsubEvent['IsMasterUnsubscribed'])
ListPartnerKey.append(str(UnsubEvent['List']['PartnerKey']))
ListID.append(str(UnsubEvent['List']['ID']))
ListObjectID.append(str(UnsubEvent['List']['ObjectID']))
ListType.append(str(UnsubEvent['List']['Type']))
ListClassification.append(str(UnsubEvent['List']['ListClassification']))
df3 = pd.DataFrame({'ListPartnerKey':ListPartnerKey,'ListID':ListID,'ListObjectID':ListObjectID,'ListType':ListType,
'ClientID':ClientIDUnsubEvents,'PartnerKey':partner_keys3,'CreatedDate':created_dates3,
'ModifiedDate':modified_date3,'ID':ID3,'ObjectID':ObjectID3,'SendID':SendID3,'SubscriberKey':SubscriberKey3,
'EventDate':EventDate3,'EventType':EventType3,'TriggeredSendDefinitionObjectID':TriggeredSendDefinitionObjectID3,
'BatchID':BatchID3,'ListClassification':ListClassification,'IsMasterUnsubscribed':IsMasterUnsubscribed})
print df3
Literally all other attributes are going into the dataframe but not sure why "ListClassification" is not being picked up on.
Thank you in advance for you help!

Appending individual lists created from a list comprehension using values from input()

I created a list comprehension to provide me the following:
listoflists = [[] for i in range(252*5)]
I then simplified the list in variable newlists to contain only the number of lists in range(weeks) which is a dynamic variable.
I want to append each individual list in the following loop for a specified range with the append process moving through each list after its reached a specified length. The values are generated from an input function. For instance, if the first list in newlists exceeds a length of 5 I want the values following the 5th loop to then append to the next list and so on. The code I currently have is:
p = 0
singlist = []
listoflists = [[] for i in range(252*5)]
newlists= [listoflists[i] for i in range(weeks)]
while p<(int(people)*weeks): #fix appending process
for i in range(int(people)*weeks):
weekly =input("Put your hours: ")
singlist.append(int(weekly))
p += 1
if weekly.isalpha() == True:
print("Not a valid amount of time")
for i in range(0,weeks):
while len(newlists[i])<int(people):
newlists[i].append(singlist[i])
This code however appends the same values to all lists in range weeks. What is the most efficient way to fix this? Thank you!
if singlist = [10,15,20,25]
desire output for newlists is: [[10,15],[20,25]]
How I've structured the program:
import sys
import numpy as np
import pandas as pd
from datetime import tzinfo,timedelta,datetime
import matplotlib.pyplot as plt
import itertools as it
from itertools import count,islice
team = []
y = 0
while y == 0:
try:
people = input("How many people are on your engagement? ")
if people.isdigit() == True:
y += 1
except:
print("Not a number try again")
z= 0
while z<int(people):
for i in range(int(people)):
names = input("Name: ")
if names.isalpha() == False:
team.append(names)
z+=1
elif names.isdigit() == True:
print("Not a name try again")
ties = [] # fix looping for more than one person
e = 0
while e<int(people):
for i in range(int(people)):
title = input("What is their title: ")
if title.isdigit() == True:
print("Not a title try again")
else:
ties.append(title)
e+=1
values = [] #fix looping for more than one person
t= 0
while t <int(people):
for i in range(int(people)):
charge = input("How much are you charging for them: ")
if charge.isalpha() == True:
print("Not a valid rate")
else:
values.append(int(charge))
t +=1
weeks = int(input("How many weeks are you including: "))
days = []
x = 0
while x<weeks: #include a parameter for dates of a 7 day difference to only be permitted
try:
for i in range(int(weeks)):
dates = input("Input the dates (mm/dd/yy): ")
dt_start = datetime.strptime(dates,'%m/%d/%y')
days.append(dates)
x+=1
except:
print("Incorrect format")
p = 0
singlist = []
listoflists = [[] for i in range(252*5)]
newlists= [listoflists[i] for i in range(weeks)]
while p<(int(people)*weeks): #fix appending process
for i in range(int(people)*weeks):
weekly =input("Put your hours: ")
singlist.append(int(weekly))
p += 1
if weekly.isalpha() == True:
print("Not a valid amount of time")
def func(items,n):
items = iter(items)
for i in it.count():
out = it.islice(items,weeks*i,weeks*i+n)
if not out:
break
output = list(func(singlist,weeks))
# items = [1,2,3,...n]
# output = [[1,2],[3,4],..], n = 2 elements each
items_ = iter(items)
outiter = iter(lambda: [next(items_) for i in range(n)],[])
outlist = list(outiter)
You can do the same thing using while loop in place of count() and [a:b] slice operation on list instead of islice(). But using iterators is very efficient.

IndexError: Python list index out of range

I have an empty list, (r) and declared first element as r[0] = a
import time, urllib.request,random
def getDictionary():
word_site = "http://svnweb.freebsd.org/csrg/share/dict/words?view=co&content-type=text/plain"
response = urllib.request.urlopen(word_site)
txt = response.read()
return txt.splitlines()
def getWordsList(listOfWords, sample):
word = ""
randWords = []
for i in range(0,sample):
while(len(word) <=2):
word = random.choice(listOfWords).decode('utf-8')
randWords.append(word)
word = ""
return randWords
start = True
noOfWords = 25
words = getDictionary()
wordsList = getWordsList(words, noOfWords)
start = True
print ("\nINSTRUCTIONS\nWhen the coundown gets to zero, type the word in lowercase letters!\n That's the only rule!")
name = input("What is your name? ")
name = name.split(" ")
input("Press enter when ready...")
while start == True:
print("Game will start in: ")
print ("3 seconds")
time.sleep(1)
print ("2 seconds")
time.sleep(1)
print ("1 seconds")
time.sleep(1)
times = []
k = list()
r = list()
for i in range(25):
startTime = time.time()
userWord = input(str(i+1) + ". " + wordsList[i].capitalize() + " " )
k.append(wordsList[i].capitalize())
if (userWord.lower() == wordsList[i].lower()):
endTime = time.time()
times.append(endTime - startTime)
r[i] = str(endTime - startTime)
else:
times.append("Wrong Word")
r[i] = ("Wrong Word")
Above is where I am having a problem.
for i in range(25):
startTime = time.time()
print (str(i+1) + ". " + str(k[i]) + ": " + str(times[i]) )
a = 0
for i in range(25):
a = a+i
for i in range(25):
if r[i] == "Wrong Word":
r = r.pop(i)
b = (a/len(r))
c = round(b, 2)
print (c)
start = False
here is my error:
r[i] = "Wrong Word"
IndexError: list assignment index out of range
The pop() method removes an element from the list and returnes it (see an example). What I think is happening is that at some point the condition of the if statment resolves to true. Next, after calling r.pop(i) r is replaced by its i-th element. It's probpably a string so calling its (i+1)-th element later can result in Index out of range error.
In other words, something like this is happening:
r = ["a", "foo", "bar", "baz"]
for i in range(4):
if r[i] == "a": # for i=0 this gives "a" == "a"
r = r.pop(i) # later,this results in r = "a"
next loop iteration with i = 1 will result in "a"[1] which will result in Index out of range.
All in all instead of:
for i in range(25):
if r[i] == "Wrong Word":
r = r.pop(i)
you could just write:
r = [item for item in r if item != "Wrong word"]
which would be also more pythonic solution.

Django Filter Loop OR

Does anyone know how I get Django Filter build an OR statement? I'm not sure if I have to use the Q object or not, I thought I need some type of OR pipe but this doesn't seem right:
filter_mfr_method = request.GET.getlist('filter_mfr_method')
for m in filter_mfr_method:
designs = designs.filter(Q(mfr_method = m) | m if m else '')
# Or should I do it this way?
#designs = designs.filter(mfr_method = m | m if m else '')
I would like this to be:
SELECT * FROM table WHERE mfr_method = 1 OR mfr_method = 2 OR mfr_method = 3
EDIT: Here is what Worked
filter_mfr_method = request.GET.getlist('filter_mfr_method')
list = []
for m in filter_mfr_method:
list.append(Q(mfr_method = m))
designs = designs.filter(reduce(operator.or_, list))
What about:
import operator
filter_mfr_method = request.GET.getlist('filter_mfr_method')
filter_params = reduce(operator.or_, filter_mfr_method, Q())
designs = designs.filter(filter_params)
Something I used before:
qry = None
for val in request.GET.getlist('filter_mfr_method'):
v = {'mfr_method': val}
q = Q(**v)
if qry:
qry = qry | q
else:
qry = q
designs = designs.filter(qry)
That is taken from one of my query builders.