I come with a short question. I've wrote this part of code, where from an array of my data, values are saved when the condition below is fulfilled. Right now I'd like to narrow a little bit this condition and write a function, which will save only data above some value.
For e.g.
res >= 100.0.
res = [j for i, j, k in zip(lst, lst[1:], lst[2:]) if i < j and j > k]
with open("S:\Doc\Python\Data\Codes\BxHPF.txt", 'w') as Peak1:
Peak1.write(str(res))
Peak1.close()
Add the additional condition to the list comprehension.
res = [j for i, j, k in zip(lst, lst[1:], lst[2:]) if i < j and j > k and j >= 100.0]
Related
Following is the code:
for (i = n-1; i>0; i--)
for (j=0; j<i; j++)
if (arr[i] < arr[i+1])
{
temp = arr[i];
arr[i] = arr[i+1];
arr[i+1] = temp;
}
I could find for the outer for outer for loop will execute n times and the inner for loop will be executed i+i-1+i-2+....+1 i(i+1)/2=(i^2+i)/2 and the if condition will be checked for (i-1)*i/2=(i^2-i)/2 times but I am confused for the statements in if condition and also correct me if I am wrong for the my above calculations too.
for n = 5, the values of i and j encountered when the if statement is executed can be listed as follows:
(4,0) (4,1) (4,2) (4,3)
(3,0) (3,1) (3,2)
(2,0) (2,1)
(1,0)
I arranged the items like that on purpose because they form a triangle.
####
###
##
#
To count how many items are in this triangle we can mirror the triangle and count each item twice. There are two ways to do it neatly depending on whether you place the mirrored items to the right or below.
####o
###oo
##ooo
#oooo
####
###o
##oo
#ooo
oooo
Either way, by inspecting width times height, this can easily be seen to be a rectangle of either n * (n-1) or (n-1) * n items (equal area in both cases). And since we counted each element twice, we can divide by two, and use (n-1) * n / 2 as the formula for the number of items.
Therefore your if condition will be computed exactly (n-1) * n / 2 times.
You also correctly expanded this expression to be ((n*n) - (n)) / 2 which is also equal to (n^2 - n) / 2.
But a few things were bad...
You said (i-1)*i/2, using i instead of n. That's not right.
Your code appears to intend to compute a Bubble sort. And the index for the if condition and its block should be j (not i) throughout. (You were comparing arr[i] to arr[i+1] for the same value of i repeatedly in the inner loop. The actual swap would be evaluated at most once for a given value of i, in that case, or not at all, depending on the values of arr.)
The code you intended was likely:
for (i = n-1; i>0; i--)
for (j=0; j<i; j++)
if (arr[j] < arr[j+1])
{
temp = arr[j];
arr[j] = arr[j+1];
arr[j+1] = temp;
}
I was trying to solve a question and I got into a few obstacles that I failed to solve, starting off here is the question: Codeforces - 817D
Now I tried to brute force it, using a basic get min and max for each segment of the array I could generate and then keeping track of them I subtract them and add them together to get the final imbalance, this looked good but it gave me a time limit exceeded cause brute forcing n*(n+1)/2 subsegments of the array given n is 10^6 , so I just failed to go around it and after like a couple of hours of not getting any new ideas I decided to see a solution that I could not understand anything in to be honest :/ , here is the solution:
#include <bits/stdc++.h>
#define ll long long
int a[1000000], l[1000000], r[1000000];
int main(void) {
int i, j, n;
scanf("%d",&n);
for(i = 0; i < n; i++) scanf("%d",&a[i]);
ll ans = 0;
for(j = 0; j < 2; j++) {
vector<pair<int,int>> v;
v.push_back({-1,INF});
for(i = 0; i < n; i++) {
while (v.back().second <= a[i]) v.pop_back();
l[i] = v.back().first;
v.push_back({i,a[i]});
}
v.clear();
v.push_back({n,INF});
for(i = n-1; i >= 0; i--) {
while (v.back().second < a[i]) v.pop_back();
r[i] = v.back().first;
v.push_back({i,a[i]});
}
for(i = 0; i < n; i++) ans += (ll) a[i] * (i-l[i]) * (r[i]-i);
for(i = 0; i < n; i++) a[i] *= -1;
}
cout << ans;
}
I tried tracing it but I keep wondering why was the vector used , the only idea I got is he wanted to use the vector as a stack since they both act the same(Almost) but then the fact that I don't even know why we needed a stack here and this equation ans += (ll) a[i] * (i-l[i]) * (r[i]-i); is really confusing me because I don't get where did it come from.
Well thats a beast of a calculation. I must confess, that i don't understand it completely either. The problem with the brute force solution is, that you have to calculate values or all over again.
In a slightly modified example, you calculate the following values for an input of 2, 4, 1 (i reordered it by "distance")
[2, *, *] (from index 0 to index 0), imbalance value is 0; i_min = 0, i_max = 0
[*, 4, *] (from index 1 to index 1), imbalance value is 0; i_min = 1, i_max = 1
[*, *, 1] (from index 2 to index 2), imbalance value is 0; i_min = 2, i_max = 2
[2, 4, *] (from index 0 to index 1), imbalance value is 2; i_min = 0, i_max = 1
[*, 4, 1] (from index 1 to index 2), imbalance value is 3; i_min = 2, i_max = 1
[2, 4, 1] (from index 0 to index 2), imbalance value is 3; i_min = 2, i_max = 1
where i_min and i_max are the indices of the element with the minimum and maximum value.
For a better visual understanding, i wrote the complete array, but hid the unused values with *
So in the last case [2, 4, 1], brute-force looks for the minimum value over all values, which is not necessary, because you already calculated the values for a sub-space of the problem, by calculating [2,4] and [4,1]. But comparing only the values is not enough, you also need to keep track of the indices of the minimum and maximum element, because those can be reused in the next step, when calculating [2, 4, 1].
The idead behind this is a concept called dynamic programming, where results from a calculation are stored to be used again. As often, you have to choose between speed and memory consumption.
So to come back to your question, here is what i understood :
the arrays l and r are used to store the indices of the greatest number left or right of the current one
vector v is used to find the last number (and it's index) that is greater than the current one (a[i]). It keeps track of rising number series, e.g. for the input 5,3,4 at first the 5 is stored, then the 3 and when the 4 comes, the 3 is popped but the index of 5 is needed (to be stored in l[2])
then there is this fancy calculation (ans += (ll) a[i] * (i-l[i]) * (r[i]-i)). The stored indices of the maximum (and in the second run the minimum) elements are calculated together with the value a[i] which does not make much sense for me by now, but seems to work (sorry).
at last, all values in the array a are multiplied by -1, which means, the old maximums are now the minimums, and the calculation is done again (2nd run of the outer for-loop over j)
This last step (multiply a by -1) and the outer for-loop over j are not necessary but it's an elegant way to reuse the code.
Hope this helps a bit.
When I execute this, there is no response. Why or what might be causing this problem to occur?
Here is the code that I have tried.
#Frequent Words
name = raw_input("Enter file:")
if len(name) < 1 :
name = "dataset_2_10.txt"
handle = open(name,"r")
Text = handle.read()
k = raw_input("Enter k:")
k = int(k)
def PatternCount(Text,Pattern):
count = 0
i = 0
while i < len(Text)-len(Pattern):
if Text[i:i+len(Pattern)] == Pattern :
count = count + 1
i = i + 1
return count
FrequentPatterns = list()
i = 0
Count = list()
while i < len(Text)-k:
Pattern = Text[i:i+k]
Count.append(PatternCount(Text,Pattern))
i = i + 1
maxCount = max(Count)
#print maxCount
j = 0
while j < len(Text)-k:
if Count[j] == maxCount:
FrequentPatterns.append(Text[j:j+k])
print Count[j]
j = j + 1
print FrequentPatterns
Any help would be much appreciated.
You do not get any output, because you run into an infinite loop every time you provide a k larger than one.
This is because in the second while loop you only increment j if the k-mer has been found. But since the condition if Count[j] == maxCount: is not always satisfied for k > 1, you do not reach the j = j + 1 and j stays the same. (You can see that, if you add a print statement to your loop.)
To fix this move the j = j + 1 from the if block (where it is now) to the loop body, like this:
j = 0
while j < len(Text)-k:
if Count[j] == maxCount:
FrequentPatterns.append(Text[j:j+k])
print Count[j]
j = j + 1
Now j is incremented every time, regardless if the k-mer was found or not.
That said, in python there is a better way to do this. You can use a for-loop to iterate over all indices like you do now, but without the danger of infinite loops. Consider this:
i = 0
while i < 10:
print i
i = i + 1
for j in range(10):
print(j)
The second loop yields the same result and is much less prone to failure. You generate a list of the numbers [0, .., 9] with range(10) and then use each item from the list.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Finding Hardy Ramanujan Numbers
I need to find the lowest natural number x where
x = k^3 + l^3 = i^3 + j^3
and (k, l, i, j) must all be different.
I tried the following four for loops, but I couldn't get it to the right solution because of infinitely increasing variables...
for (int i=0;;i++)
for (int j=i+1;;j++)
for (int k=j+1;;k++)
for (int l=k+1;;i++)
compare(i,j,k,l);
You need to reframe how you're thinking about the problem.
It's really saying this: what's the smallest natural number expressible as the sum of two cubes in two different ways?
The problem statement calls that number x, and the pairs of cubes are (i, j) and (k, l).
Restated in this way, it's not nearly so bad. Here's a hint in pseudocode:
function count_num_cubic_pairs(n):
cubic_pairs = []
for i..n:
first_cube = i * i * i
remainder = n - first_cube
if remainder is a cube and (first_cube, remainder) not in cubic_pairs:
cubic_pairs.add((first_cube, remainder))
return length(cubic_pairs)
The tough part will be testing whether remainder is a cube - floating point errors will complicate that a lot. That's the real meat of this problem - have fun with it.
One easy way to make your code work is to limit the domain of your variables, and then expand it a bit at a time.
As mazayus mentioned, you're keeping each variable strictly greater than the previous ones, so you never any variation that could possibly be correct.
Something like this may work (pseudocode) but it's horribly inefficient:
for max in [100, 200, 300, ...]
for i in [0..max]
for j in [0..max]
for k in [0..max]
for l in [0..max]
if (i equals k or l, or j equals k or l) continue
if (i^3 + j^3 equals k^3 + l^3)
return answer
int i = 1
int j = 3
int k = 2
int l = 4
do {
do {
do {
do {
compare(i, j ,k l);
i++;
} while (i < k);
k++;
} while (k < j);
j++;
} while(j < l);
l++;
} while(l < 100);
Something like this tries every combination of numbers without dups (up to values of 100), with i < k < j < l.
Your loops assume i<j<k<l, which is not necessarily true. (It might be that j>k.) Once you get the right assumptions, you can reorder you loops so the first item is biggest and so the other loops are limited.
Here's an example with the i>j, i>k>l,
for (int i=1;;i++)
for (int j=1;j<i;j++)
for (int k=1;k<i;k++)
for (int l=1;l<k;i++)
compare(i,j,k,l);
Once you get that working, try eliminating the fourth loop by checking if the cube root of i*i*i+j*j*j-k*k*k is a natural number. Then try finding a smarter starting value for k.
for (int i = 0; i < 5000; i++)
for (int j = 0; j < 5000; j++)
{
for (int ii = 0; ii < 20; ii++)
for (int jj = 0; jj < 20; jj++)
{
int num = matBigger[i+ii][j+jj];
// Extract range from this.
int low = num & 0xff;
int high = num >> 8;
if (low < matSmaller[ii][jj] && matSmaller[ii][jj] > high)
// match found
}
}
The machine is x86_64, 32kb L1 cahce, 256 Kb L2 cache.
Any pointers on how can I possibly optimize this code?
EDIT Some background to the original problem : Fastest way to Find a m x n submatrix in M X N matrix
First thing I'd try is to move the ii and jj loops outside the i and j loops. That way you're using the same elements of matSmaller for 25 million iterations of the i and j loops, meaning that you (or the compiler if you're lucky) can hoist the access to them outside those loops:
for (int ii = 0; ii < 20; ii++)
for (int jj = 0; jj < 20; jj++)
int smaller = matSmaller[ii][jj];
for (int i = 0; i < 5000; i++)
for (int j = 0; j < 5000; j++) {
int num = matBigger[i+ii][j+jj];
int low = num & 0xff;
if (low < smaller && smaller > (num >> 8)) {
// match found
}
}
This might be faster (thanks to less access to the matSmaller array), or it might be slower (because I've changed the pattern of access to the matBigger array, and it's possible that I've made it less cache-friendly). A similar alternative would be to move the ii loop outside i and j and hoist matSmaller[ii], but leave the jj loop inside. The rule of thumb is that it's more cache-friendly to increment the last index of a multi-dimensional array in your inner loops, than earlier indexes. So we're "happier" to modify jj and j than we are to modify ii and i.
Second thing I'd try - what's the type of matBigger? Looks like the values in it are only 16 bits, so try it both as int and as (u)int16_t. The former might be faster because aligned int access is fast. The latter might be faster because more of the array fits in cache at any one time.
There are some higher-level things you could consider with some early analysis of smaller: for example if it's 0 then you needn't examine matBigger for that value of ii and jj, because num & 0xff < 0 is always false.
To do better than "guess things and see whether they're faster or not" you need to know for starters which line is hottest, which means you need a profiler.
Some basic advice:
Profile it, so you can learn where the hot-spots are.
Think about cache locality, and the addresses resulting from your loop order.
Use more const in the innermost scope, to hint more to the compiler.
Try breaking it up so you don't compute high if the low test is failing.
Try maintaining the offset into matBigger and matSmaller explicitly, to the innermost stepping into a simple increment.
Best thing ist to understand what the code is supposed to do, then check whether another algorithm exists for this problem.
Apart from that:
if you are just interested if a matching entry exists, make sure to break out of all 3 loops at the position of // match found.
make sure the data is stored in an optimal way. It all depends on your problem, but i.e. it could be more efficient to have just one array of size 5000*5000*20 and overload operator()(int,int,int) for accessing elements.
What are matSmaller and matBigger?
Try changing them to matBigger[i+ii * COL_COUNT + j+jj]
I agree with Steve about rearranging your loops to have the higher count as the inner loop. Since your code is only doing loads and compares, I believe a significant portion of the time is used for pointer arithmetic. Try an experiment to change Steve's answer into this:
for (int ii = 0; ii < 20; ii++)
{
for (int jj = 0; jj < 20; jj++)
{
int smaller = matSmaller[ii][jj];
for (int i = 0; i < 5000; i++)
{
int *pI = &matBigger[i+ii][jj];
for (int j = 0; j < 5000; j++)
{
int num = *pI++;
int low = num & 0xff;
if (low < smaller && smaller > (num >> 8)) {
// match found
} // for j
} // for i
} // for jj
} // for ii
Even in 64-bit mode, the C compiler doesn't necessarily do a great job of keeping everything in register. By changing the array access to be a simple pointer increment, you'll make the compiler's job easier to produce efficient code.
Edit: I just noticed #unwind suggested basically the same thing. Another issue to consider is the statistics of your comparison. Is the low or high comparison more probable? Arrange the conditional statement so that the less probable test is first.
Looks like there is a lot of repetition here. One optimization is to reduce the amount of duplicate effort. Using pen and paper, I'm showing the matBigger "i" index iterating as:
[0 + 0], [0 + 1], [0 + 2], ..., [0 + 19],
[1 + 0], [1 + 1], ..., [1 + 18], [1 + 19]
[2 + 0], ..., [2 + 17], [2 + 18], [2 + 19]
As you can see there are locations that are accessed many times.
Also, multiplying the iteration counts indicate that the inner content is accessed: 20 * 20 * 5000 * 5000, or 10000000000 (10E+9) times. That's a lot!
So rather than trying to speed up the execution of 10E9 instructions (such as execution (pipeline) cache or data cache optimization), try reducing the number of iterations.
The code is searcing the matrix for a number that is within a range: larger than a minimal value and less than the maximum range value.
Based on this, try a different approach:
Find and remember all coordinates where the search value is greater
than the low value. Let us call these anchor points.
For each anchor point, find the coordinates of the first value after
the anchor point that is outside the range.
The objective is to reduce the number of duplicate accesses. Anchor points allow for a one pass scan and allow other decisions such as finding a range or determining an MxN matrix that contains the anchor value.
Another idea is to create new data structures containing the matBigger and matSmaller that are more optimized for searching.
For example, create a {value, coordinate list} entry for each unique value in matSmaller:
Value coordinate list
26 -> (2,3), (6,5), ..., (1007, 75)
31 -> (4,7), (2634, 5), ...
Now you can use this data structure to find values in matSmaller and immediately know their locations. So you could search matBigger for each unique value in this data structure. This again reduces the number of access to the matrices.