I am trying to create a sequence of similar dictionaries to further store them in a tuple. I tried two approaches, using and not using a for loop
Without for loop
dic0 = {'modo': lambda x: x[0]}
dic1 = {'modo': lambda x: x[1]}
lst = []
lst.append(dic0)
lst.append(dic1)
tup = tuple(lst)
dic0 = tup[0]
dic1 = tup[1]
f0 = dic0['modo']
f1 = dic1['modo']
x = np.array([0,1])
print (f0(x) , f1(x)) # 0 , 1
With a for loop
lst = []
for j in range(0,2):
dic = {}
dic = {'modo': lambda x: x[j]}
lst.insert(j,dic)
tup = tuple(lst)
dic0 = tup[0]
dic1 = tup[1]
f0 = dic0['modo']
f1 = dic1['modo']
x = np.array([0,1])
print (f0(x) , f1(x)) # 1 , 1
I really don't understand why I am getting different results. It seems that the last dictionary I insert overwrite the previous ones, but I don't know why (the append method does not work neither).
Any help would be really welcomed
This is happening due to how scoping works in this case. Try putting j = 0 above the final print statement and you'll see what happens.
Also, you might try
from operator import itemgetter
lst = [{'modo': itemgetter(j)} for j in range(2)]
You have accidentally created what is know as a closure. The lambda functions in your second (loop-based) example include a reference to a variable j. That variable is actually the loop variable used to iterate your loop. So the lambda call actually produces code with a reference to "some variable named 'j' that I didn't define, but it's around here somewhere."
This is called "closing over" or "enclosing" the variable j, because even when the loop is finished, there will be this lambda function you wrote that references the variable j. And so it will never get garbage-collected until you release the references to the lambda function(s).
You get the same value (1, 1) printed because j stops iterating over the range(0,2) with j=1, and nothing changes that. So when your lambda functions ask for x[j], they're asking for the present value of j, then getting the present value of x[j]. In both functions, the present value of j is 1.
You could work around this by creating a make_lambda function that takes an index number as a parameter. Or you could do what #DavisYoshida suggested, and use someone else's code to create the appropriate closure for you.
Related
I have data in two directories and i'm using for loop to read the files from both the folders.
path_to_files = '/home/Desktop/computed_2d/'
path_to_files1 = '/home/Desktop/computed_1d/'
for filen in [x for x in os.listdir(path_to_files) if '.ares' in x]:
df = pd.read_table(path_to_files+filen, skiprows=0, usecols=(0,1,2,3,4,8),names=['wave','num','stlines','fwhm','EWs','MeasredWave'],delimiter=r'\s+')
for filen1 in [x for x in os.listdir(path_to_files1) if '.ares' in x]:
df1 = pd.read_table(path_to_files1+filen1, skiprows=0, usecols=(0,1,2,3,4,8),names=['wave','num','stlines','fwhm','EWs','MeasredWave'],delimiter=r'\s+')
print(filen,filen1)
Now what's happening is like when tried to print the filenames then it kept printing the names forever. So, its basically taking the first iteration from first loop then print it with all the iteration of the second loop.I don't understand why is it happening.
But what i want to do is, i want to print the first iteration of first loop with the first iteration of second for loop
As the file names are same in both the folders.So when i do the print, then desired result should look like something like this:
(txt_1.txt,txt_1.txt)
(txt_2.txt,txt_2.txt)
(txt_3.txt,txt_3.txt)
(txt_4.txt,txt_4.txt)
Where i'm making the mistake??
If I understand your question correctly, you seem to want to print pairs of files from path_to_files and path_to_files1. Since you are nesting a for loop, for every iteration of the nested for loop, filen is not going to change.
I think you might want something more like this:
path_to_files = '/home/Desktop/computed_2d/'
path_to_files1 = '/home/Desktop/computed_1d/'
filelistn = [x for x in os.listdir(path_to_files) if '.ares' in x]
filelist1 = [x for x in os.listdir(path_to_files1) if '.ares' in x]
for filen, filen1 in zip(filelistn, filelist1):
df = pd.read_table(path_to_files+filen, skiprows=0, usecols=(0,1,2,3,4,8),names=['wave','num','stlines','fwhm','EWs','MeasredWave'],delimiter=r'\s+')
df1 = pd.read_table(path_to_files1+filen1, skiprows=0, usecols=(0,1,2,3,4,8),names=['wave','num','stlines','fwhm','EWs','MeasredWave'],delimiter=r'\s+')
print(filen,filen1)
For a sample input of:
filelistn = ['a.ar', 'b.ar']
filelist1 = ['c.ar', 'd.ar']
I get the following output:
('a.ar', 'c.ar')
('b.ar', 'd.ar')
I have data in the form of list of lists where I am trying to match the demand and supply such that each demand matches uniquely to one supply item.
dmndId_w_freq = [['a',4,[1,2,3,4]],['b',6,[5,6,7,8,3,4]],['c',7,[6,5,7,9,8,3,4]],['d',8,[1,6,3,4,5,6,7,10]]]
num_sims = 1
for sim_count in range(num_sims):
dmndID_used_flag = {}
splID_used_flag = {}
dmndID_splId_one_match = {}
for i in dmndId_w_freq:
if i[0] not in dmndID_used_flag.keys():
for j in i[2]:
#print j
#print "CLICK TO CONTINUE..."
#raw_input()
if j in splID_used_flag.keys():
i[2].remove(j)
dmndID_splId_one_match[i[0]] = i[2][0]
splID_used_flag[i[2][0]] = 1
dmndID_used_flag[i[0]] = 1
print
print dmndID_splId_one_match
#print splID_used_flag
#print dmndID_used_flag
#raw_input()
I would expect the output dmndID_splId_one_match to be something like {'a':1,'b':5,'c':6,'d':3}.
But I end up getting {'a':1,'b':5,'c':6,'d':6}
So I am NOT getting a unique match as the supply item 6 is getting mapped to demands 'c' as well as demand 'd'.
I tried to debug the code by looping through it and it seems that the problem is in the for-loop
for j in i[2]:
The code is not going through all the elements of i[2]. It does not happen with the 'a' and 'b' part. but starts happening with the 'c' part. It goes over 6 and 5 but skips 7 (the third element of the list [6,5,7,9,8,3,4]). Similarly, in the 'd' part it skips the 2nd element 6 in the list [1,6,3,4,5,6,7,10]. And that is why the mapping is happening twice, since I am not able to remove it.
What am I doing wrong that it is not executing the for-loop as expected?
Is there a better way to write this code?
I figured out the problem in the for loop. I am looping through i[2] and then trying to modify it within the loop. this makes it unstable.
I want to create an indexing set of tuples, i mean if i do:
LINEAS_DOWNSTREAM_BARRA[1] I want to see [(1,3),(1,2),(1,4)].
My code is:
m=ConcreteModel()
m.BARRAS = Set()
m.LINEAS_DOWNSTREAM_BARRA = Set(dimen = 2)
m.LINEAS_DOWNSTREAM_BARRA = Set(m.BARRAS, initialize=lambda m, i:
set(tuple(z) for z in m.LINEAS if (i == z[0])))
And the problem is:
ValueError: The value=(1, 2) is a tuple for
set=LINEAS_DOWNSTREAM_BARRA, which has dimen=1
Thanks!!
You should declare the Set m.LINEAS_DOWNSTREAM_BARRA on a single line. Also, make sure that your lambda function is returning a list of tuples
m.LINEAS_DOWNSTREAM_BARRA = Set(m.BARRAS, dimen=2, initialize=your_lambda_fcn)
I am trying to extract particular lines from txt output file. The lines I am interested in are few lines above and few below the key_string that I am using to search through the results. The key string is the same for each results.
fi = open('Inputfile.txt')
fo = open('Outputfile.txt', 'a')
lines = fi.readlines()
filtered_list=[]
for item in lines:
if item.startswith("key string"):
filtered_list.append(lines[lines.index(item)-2])
filtered_list.append(lines[lines.index(item)+6])
filtered_list.append(lines[lines.index(item)+10])
filtered_list.append(lines[lines.index(item)+11])
fo.writelines(filtered_list)
fi.close()
fo.close()
The output file contains the right lines for the first record, but multiplied for every record available. How can I update the indexing so it can read every individual record? I've tried to find the solution but as a novice programmer I was struggling to use enumerate() function or collections package.
First of all, it would probably help if you said what exactly goes wrong with your code (a stack trace, it doesn't work at all, etc). Anyway, here's some thoughts. You can try to divide your problem into subproblems to make it easier to work with. In this case, let's separate finding the relevant lines from collecting them.
First, let's find the indexes of all the relevant lines.
key = "key string"
relevant = []
for i, item in enumerate(lines):
if item.startswith(key):
relevant.append(item)
enumerate is actually quite simple. It takes a list, and returns a sequence of (index, item) pairs. So, enumerate(['a', 'b', 'c']) returns [(0, 'a'), (1, 'b'), (2, 'c')].
What I had written above can be achieved with a list comprehension:
relevant = [i for (i, item) in enumerate(lines) if item.startswith(key)]
So, we have the indexes of the relevant lines. Now, let's collected them. You are interested in the line 2 lines before it and 6 and 10 and 11 lines after it. If your first lines contains the key, then you have a problem – you don't really want lines[-1] – that's the last item! Also, you need to handle the situation in which your offset would take you past the end of the list: otherwise Python will raise an IndexError.
out = []
for r in relevant:
for offset in -2, 6, 10, 11:
index = r + offset
if 0 < index < len(lines):
out.append(lines[index])
You could also catch the IndexError, but that won't save us much typing, as we have to handle negative indexes anyway.
The whole program would look like this:
key = "key string"
with open('Inputfile.txt') as fi:
lines = fi.readlines()
relevant = [i for (i, item) in enumerate(lines) if item.startswith(key)]
out = []
for r in relevant:
for offset in -2, 6, 10, 11:
index = r + offset
if 0 < index < len(lines):
out.append(lines[index])
with open('Outputfile.txt', 'a') as fi:
fi.writelines(out)
To get rid of duplicates you can cast list to set; example:
x=['a','b','a']
y=set(x)
print(y)
will result in:
['a','b']
I used the split() function to convert string to a list time = time.split() and this is how my output looks like :
[u'1472120400.107']
[u'1472120399.999']
[u'1472120399.334']
[u'1472120397.633']
[u'1472120397.261']
[u'1472120394.328']
[u'1472120393.762']
[u'1472120393.737']
Then I tried accessing the contents of the list using print time[1] which gives the index out of range error (cause only a single value is stored in one list). I checked questions posted by other people and used print len(time). This is the output for that:
1
[u'1472120400.107']
1
[u'1472120399.999']
1
[u'1472120399.334']
1
[u'1472120397.633']
1
[u'1472120397.261']
1
[u'1472120394.328']
1
[u'1472120393.762']
1
[u'1472120393.737']
I do this entire thing inside a for loop because I get logs dynamically and have to extract out just the time.
This is part of my code:
line_collect = lines.collect() #spark function
for line in line_collect :
a = re.search(rx1,line)
time = a.group()
time = time.split()
#print time[1] #index out of range error which is why I wrote another for below
for k in time :
time1 = time[k]#trying to put those individual list values into one variable but get type error
print len(time1)
I get the following error :
time1 = time[k]
TypeError: list indices must be integers, not unicode
Can someone tell me how to read each of those single list values into just one list so I can access each of them using a single index[value]. I'm new to python.
My required output:
time =['1472120400.107','1472120399.999','1472120399.334','1472120397.633','1472120397.261','1472120394.328','1472120393.762','1472120393.737']
so that i can use time[1] to give 1472120399.999 as result.
Update: I misunderstood what you wanted. You have the correct output already and it's a string. The reason you have a u before the string is because it's a unicode string that has 16 bits. u is a python flag to distinguish it from a normal string. Printing it to the screen will give you the correct string. Use it normally as you would any other string.
time = [u'1472120400.107'] # One element just to show
for k in time:
print(k)
Looping over a list using a for loop will give you one value at a time, not the index itself. Consider using enumerate:
for k, value in enumerate(time):
time1 = value # Or time1 = time[k]
print(time1)
Or just getting the value itself:
for k in time:
time1 = k
print(time1)
--
Also, Python is zero based language, so to get the first element out of a list you probably want to use time[0].
Thanks for your help. I finally got the code right:
newlst = []
for line in line_collect :
a = re.search(rx1,line)
time = a.group()
newlst.append(float(time))
print newlst
This will put the whole list values into one list.
Output:
[1472120400.107, 1472120399.999, 1472120399.334, 1472120397.633,
1472120397.261, 1472120394.328, 1472120393.762, 1472120393.737]