Related
Question:
Given a string S S.length() <= 5.10^5 and an integer K K <= S.length(). For each removal, you can:
Remove the first character of the string
Remove the second character of the string
Remove the last character of the string
Remove the second last character of the string
How can I do exactly K removals such that the final string has minimum lexicographical order?
Example:
S= "abacaaba", K = 2
Remove the second character of the string
Remove the second last character of the string
The final string: "aacaaa" which is the smallest lexicographical possible.
P/S:
I've tried for many days but can't figure out an efficience way to solve this problem. But I think there's something to do with dynamic programming.
All together, these ideas should lead to a linear-time algorithm.
If K ≤ N−4, the final string has at least four characters. Its two-character prefix is the least two-character subsequence of the (K+2)-character prefix of the initial string. Compute this prefix and the possible positions of its second character. This can be accomplished in O(K) time by scanning through first K+2 characters, maintaining the least character so far and the least two-character subsequence so far.
Now that we know the two-character prefix, we just have to determine the best suffix. For a prefix that required J deletions to set up, the final string continues with the next N−4 − K characters that we can't touch, followed by the least two-character subsequence of the (K+2 − J)-character suffix of the initial string. We can compute the least two-character subsequence of each of the relevant suffixes using the scanning algorithm described previously. The one tricky part is comparing the untouchable middles efficiently. This can be accomplished with some difficulty using a suffix array with longest common prefixes.
If K > N−4, just return the least (N−K)-character subsequence.
Interesting task!
Update: step 5 incorrect. Here is correct one:
All combinations with length M, which consist of 3'th and 4'th remove operations are equal to this class of operations:
Zero or more 3 after that zero or more 4, like this regexp: (3)(4)
You can prove it:
43 pair is equal to 33
343 pair equal to 443.
Moreover 34...43 is equal to 44...43.
So you can pick rightmost 3 and with rule 3 make it the only one 3. And with rule 4 make transform all left 4 to 3.
any ->rule3-> 4...434...4 -> rule1-> 3...34...4
It leads to O(K^3) complexity of step 6 brute force.
Original answer
There are some ideas and solution that works nice in common
[More short word is smaller in lexicographical order] Wrong, as #n. 1.8e9-where's-my-share m. mentinoed. All possible results will be equal length (Length-K), because we should use all of them.
Lexicographical order means: for semi-length words we match symbols from left to right until it equal. Result of word comparison is result of first different char comparison result. So minimization of i'th symbol is more important than minimization (i+j)'th symbol for all positive j.
So most important is first symbol minimization. Only first removal operation can influence on it. By first removal operation we try to place at first place minimal possible value (It will be minimal value from first K symbols). If there is some positions with minimal letter - we will pick leftmost one (we don't want to delete extra symbols and lost correct answer).
Now most important is second letter. So we want to minimize it too. We will make it like in 3'th step of algorithm. But, we use 2'nd remove operation and if we had some variants as minimal - we save all of them as candidates.
All combinations with length M, which consist of 3'th and 4'th remove operations are equal to only 2 combinations:
all operations are 4'th: 44...44
all operations are 4'th but the last one is 3: 44...43.
So for every candidate we can check only two possibilities.
Brute force all candidates with both possibilities. Find minimal.
In common case this algorithm work's well. But in worst case it's weak. There is counterpoint: Maxlength string with same letter. Then we have K candidates and algorithm complexity will be O(K^2) - it's not good for this task.
For deal with it i think we can choose right candidate at 6'th step of algorithm:
6*. For two candidates - compare their suffix - letters after it. Candidate with smaller letter at same tail position (tail position counts from this candidate head position) is better for our purposes.
7*. Compare two possibilities form 5'th algorithm step and choose minimal.
Problem of this (*) approach - i cannot get a rigid proof that it's better solution. Most hard part, when one candidate is a prefix of another - we compare it letter by letter until smallest doesn't end. for example in string abcabcabc...abc with candidate at first and fourth position.
Here is a (hopefully) mostly complete solution in Python (sorry, I'm even less versed in C++). The idea is I believe the same or very similar to David Eisenstat's, who's answer helped me think more about handling the middles. The comparisons for the middle section use O(1) lookups with O(n log n) preprocessing based on the suffix array construction referenced and linked in the code (David's suggestion is to use O(n) preprocessing and O(1) lookups but I haven't had time to get into O(1) RMQs or Ukkonen's; I'm also charmed by the referenced CP suffix array algorithm). The code includes testing comparing with brute-force but is incomplete in that it does not handle the case where there are only the prefix and suffix and no middle, which should be much simpler to handle anyway. There are probably ways to make the code more succinct and organised but I haven't had time yet to consider it more carefully.
Since we can remove the first, second, penultimate or last characters; the first two letters of the solution will be chosen from two letters (a subsequence) remaining after k or less deletions:
xxxAxxxxxxxB...
Once we've committed to character A by deleting some first characters, we are left only with a choice for B, based on how many deletions we make of the second character. Clearly, we'd like the lowest available character for A, which we may have more than one instance of, and the lowest choice for B after that, for which we also may have more than once instance of.
The suffix is composed similarly, but we need to store the best suffix for each k - num_deletions already chosen for the prefix. Then the final candidate is the lowest two-character-prefix + middle + two-character-suffix, where the middle is fixed by the distribution of deletions in each candidate. We can compare middles using a suffix array or tree with additional information.
Python
def log2(n):
i = -1
while(n):
i += 1
n >>= 1
return i
# https://cp-algorithms.com/string/suffix-array.html
def sort_cyclic_shifts(s):
n = len(s)
alphabet = 256
cs = []
p = [0] * n
c = [0] * n
cnt = [0] * max(alphabet, n + 1)
for i in range(n):
cnt[ord(s[i])] += 1
for i in range(1, alphabet):
cnt[i] += cnt[i-1]
for i in range(n):
cnt[ord(s[i])] -= 1
p[cnt[ord(s[i])]] = i
c[p[0]] = 0
classes = 1
for i in range(1, n):
if s[p[i]] != s[p[i-1]]:
classes += 1
c[p[i]] = classes - 1
cs.append(c[:])
pn = [0] * n
cn = [0] * n
h = 0
while (1 << h) < n:
for i in range(n):
pn[i] = p[i] - (1 << h)
if pn[i] < 0:
pn[i] += n
for i in range(0, classes):
cnt[i] = 0
for i in range(n):
cnt[c[pn[i]]] += 1
for i in range(1, classes):
cnt[i] += cnt[i-1]
for i in range(n-1, -1, -1):
cnt[c[pn[i]]] -= 1
p[cnt[c[pn[i]]]] = pn[i]
cn[p[0]] = 0
classes = 1
for i in range(i, n):
cur = c[p[i]], c[(p[i] + (1 << h)) % n]
prev = c[p[i-1]], c[(p[i-1] + (1 << h)) % n]
if cur != prev:
classes += 1
cn[p[i]] = classes - 1
c = cn
cs.append(c[:])
h += 1
return p, cs
# https://cp-algorithms.com/string/suffix-array.html
def suffix_array_construction(s):
s += "$"
sorted_shifts, cs = sort_cyclic_shifts(s)
return sorted_shifts[1:], cs
# https://cp-algorithms.com/string/suffix-array.html
def compare(i, j, l, k, n, c):
a = c[k][i], c[k][(i+l-(1 << k))%n]
b = c[k][j], c[k][(j+l-(1 << k))%n]
if a == b:
return 0
elif a < b:
return -1
return 1
## MAIN FUNCTION
def f(s, k):
debug = 0
n = len(s)
# Best prefix
best_first = s[k]
best_second = s[k+1]
first_idxs = [k]
second_idxs = [k + 1]
for i in range(k - 1, -1, -1):
if s[i] <= best_first:
best_first = s[i]
# We only need one leftmost index
first_idxs = [i]
for i in range(k, first_idxs[0], -1):
if (s[i] < best_second):
best_second = s[i]
second_idxs = [i]
elif s[i] == best_second:
second_idxs.append(i)
second_idxs = list(reversed(second_idxs))
# Best suffix
# For each of l deletions,
# we can place the last
# character anywhere ahead
# of the penultimate.
last_idxs = {(n - 2): [n - 1]}
best_last = s[n - 1]
for l in range(2, k + 2):
idx = n - l
if s[idx] < best_last:
best_last = s[idx]
last_idxs[n - 1 - l] = [idx]
else:
last_idxs[n - 1 - l] = last_idxs[n - l]
p, cs = suffix_array_construction(s)
second_idx = 0
if debug:
print(first_idxs, second_idxs, last_idxs)
while first_idxs[0] >= second_idxs[second_idx]:
second_idx += 1
prefix_end = second_idxs[second_idx]
num_deleted = prefix_end - 1
remaining = k - num_deleted
suffix_start = n - remaining - 2
best = (prefix_end + 1, suffix_start - 1)
while second_idx < len(second_idxs):
prefix_end = second_idxs[second_idx]
num_deleted = prefix_end - 1
remaining = k - num_deleted
suffix_start = n - remaining - 2
len_candidate_middle = suffix_start - 1 - prefix_end
# The prefixes are all equal.
# We need to compare the middle
# and suffix.
# compare(i, j, l, k, n, c)
len_best_middle = best[1] - best[0] + 1
l = min(len_candidate_middle, len_best_middle)
# Compare middles
comp = compare(best[0], prefix_end + 1, l, log2(l), n + 1, cs)
# Candidate is better
if comp == 1:
best = (prefix_end + 1, suffix_start - 1)
elif comp == 0:
# Compare suffix of candidate with
# substring at the comparable position
# of best.
[last_idx] = last_idxs[suffix_start]
candidate_suffix = s[suffix_start] + s[last_idx]
if len_candidate_middle < len_best_middle:
# One character of best's suffix
if len_candidate_middle + 1 == len_best_middle:
to_compare = s[best[1]] + s[best[1] + 1]
# None of best's suffix
else:
idx = best[0] + len_candidate_middle
to_compare = s[idx] + s[idx + 1]
# If the candidate suffix is equal
# to best's equivalent, the candidate
# wins since it's shorter.
if candidate_suffix <= to_compare:
best = (prefix_end + 1, suffix_start - 1)
elif len_candidate_middle == len_best_middle:
idx = best[1] + 1
to_compare = s[idx] + s[last_idxs[idx][0]]
if candidate_suffix < to_compare:
best = (prefix_end + 1, suffix_start - 1)
# len_best_middle < len_candidate_middle
else:
# One character of candidate's suffix
if len_best_middle + 1 == len_candidate_middle:
to_compare = s[suffix_start - 1] + s[suffix_start]
# None of candidates's suffix
else:
idx = prefix_end + 1 + len_best_middle
to_compare = s[idx] + s[idx + 1]
if candidate_suffix < to_compare:
best = (prefix_end + 1, suffix_start - 1)
second_idx += 1
prefix = s[first_idxs[0]] + s[second_idxs[second_idx-1]]
middle = s[best[0]:best[1] + 1]
suffix = s[best[1] + 1] + s[last_idxs[best[1] + 1][0]]
return prefix + middle + suffix
def brute_force(s, k):
best = s + "z"
stack = [(s, k)]
while stack:
_s, _k = stack.pop()
if _k == 0:
best = min(best, _s)
continue
stack.append((_s[1:], _k - 1))
stack.append((_s[0] + _s[2:], _k - 1))
stack.append((_s[0:len(_s)-1], _k - 1))
stack.append((_s[0:len(_s)-2] + _s[-1], _k - 1))
return best
# 01234567
#s = "abacaaba"
#k = 2
# Test
import random
n = 12
num_tests = 500
for _ in range(num_tests):
s = "".join([chr(97 + random.randint(0, 25)) for i in range(n)])
k = random.randint(1, n - 5)
#print(s, k)
_f = f(s, k)
brute = brute_force(s, k)
if brute != _f:
print("MISMATCH!")
print(s, k)
print(_f)
print(brute)
break
print("Done.")
A non-solution failing for abcbaa, K = 2
(Kudos to גלעד ברקן for sharing a test.)
find the minimum in the first K+1 elements at lowest 0-based index l1
drop a prefix of length l1 (remove 1st l1 times) - done if l1 = K
find the minimum in the elements from 1 to 1 + K - l1, inclusive at l2
"delete" elements from 1 to l2 (if any)(remove 2nd l2 times)
starting with l3 = 0 and while K - l1 - l2 - l3 > 0,
remove the larger of the last two elements and increase l3
I will post this answer even though there is an accepted answer because I feel like all the other answers are more complex than they need to be. Below is a O(NK) algorithm that solves this, which can "easily" be made into an O(N) algorithm if a suffix tree is used to do the comparisons of the "middle" portions.
#!/usr/bin/python
def lex_kr(x,K,k_r):
"""
Get a lexicographically comparable subset of `x` for a given value of
`k_r`.
"""
N = len(x)
assert k_r > 0 and k_r < K # check for corner cases
k_l = K - k_r
v_l = min(x[:k_l+1])
v_r = min(x[-k_r-1:])
lex = [v_l]
lex += x[k_l+1:N-k_r-1]
lex += [v_r]
return lex
def lex_corner(x,K):
"""
Get the two lexicographically comparable subsets of `x` for corner cases
when `k_r=0` and `k_r=K`.
"""
N = len(x)
k_l = K
v_l = min(x[:k_l+1])
lex0 = [v_l]
lex0 += x[k_l+1:]
k_r = K
v_r = min(x[-k_r-1:])
lex1 = x[:N-k_r-1]
lex1 += [v_r]
return lex0,lex1
def min_lex(x,K):
subsets = [ lex_kr(x,K,k_r) for k_r in range(1,K) ]
subsets += lex_corner(x,K) # append the two corner cases
return min(subsets)
if __name__ == '__main__':
S = [ x for x in 'abacaaba' ]
K = 2
print(min_lex(S,K))
which prints ['a', 'a', 'c', 'a', 'a', 'a'].
The comparisons of the min of the left and right (prefix & suffix) of the arrays can obviously be pre-computed in O(N) time in the function lex_kr.
The middle portion (i.e. x[k_l+1:N-k_r-1]) needs a clever trick to compare lexographically against all other middle portions efficiently. This can be done in O(1) per comparison using a suffix-tree/array as described in other answers (https://cp-algorithms.com/string/suffix-array.html) or a suffix automaton (https://cp-algorithms.com/string/suffix-automaton.html) with the latter being more complex but more efficient. Once implemented, this would yield an O(N) algorithm with less special cases to be checked than other answers.
I am looking for a least time-complex algorithm that would solve a variant of the perfect sum problem (initially: finding all variable size subset combinations from an array [*] of integers of size n that sum to a specific number x) where the subset combination size is of a fixed size k and return the possible combinations without direct and also indirect (when there's a combination containing the exact same elements from another in another order) duplicates.
I'm aware this problem is NP-hard, so I am not expecting a perfect general solution but something that could at least run in a reasonable time in my case, with n close to 1000 and k around 10
Things I have tried so far:
Finding a combination, then doing successive modifications on it and its modifications
Let's assume I have an array such as:
s = [1,2,3,3,4,5,6,9]
So I have n = 8, and I'd like x = 10 for k = 3
I found thanks to some obscure method (bruteforce?) a subset [3,3,4]
From this subset I'm finding other possible combinations by taking two elements out of it and replacing them with other elements that sum the same, i.e. (3, 3) can be replaced by (1, 5) since both got the same sum and the replacing numbers are not already in use. So I obtain another subset [1,5,4], then I repeat the process for all the obtained subsets... indefinitely?
The main issue as suggested here is that it's hard to determine when it's done and this method is rather chaotic. I imagined some variants of this method but they really are work in progress
Iterating through the set to list all k long combinations that sum to x
Pretty self explanatory. This is a naive method that do not work well in my case since I have a pretty large n and a k that is not small enough to avoid a catastrophically big number of combinations (the magnitude of the number of combinations is 10^27!)
I experimented several mechanism related to setting an area of research instead of stupidly iterating through all possibilities, but it's rather complicated and still work in progress
What would you suggest? (Snippets can be in any language, but I prefer C++)
[*] To clear the doubt about whether or not the base collection can contain duplicates, I used the term "array" instead of "set" to be more precise. The collection can contain duplicate integers in my case and quite much, with 70 different integers for 1000 elements (counts rounded), for example
With reasonable sum limit this problem might be solved using extension of dynamic programming approach for subset sum problem or coin change problem with predetermined number of coins. Note that we can count all variants in pseudopolynomial time O(x*n), but output size might grow exponentially, so generation of all variants might be a problem.
Make 3d array, list or vector with outer dimension x-1 for example: A[][][]. Every element A[p] of this list contains list of possible subsets with sum p.
We can walk through all elements (call current element item) of initial "set" (I noticed repeating elements in your example, so it is not true set).
Now scan A[] list from the last entry to the beginning. (This trick helps to avoid repeating usage of the same item).
If A[i - item] contains subsets with size < k, we can add all these subsets to A[i] appending item.
After full scan A[x] will contain subsets of size k and less, having sum x, and we can filter only those of size k
Example of output of my quick-made Delphi program for the next data:
Lst := [1,2,3,3,4,5,6,7];
k := 3;
sum := 10;
3 3 4
2 3 5 //distinct 3's
2 3 5
1 4 5
1 3 6
1 3 6 //distinct 3's
1 2 7
To exclude variants with distinct repeated elements (if needed), we can use non-first occurence only for subsets already containing the first occurence of item (so 3 3 4 will be valid while the second 2 3 5 won't be generated)
I literally translate my Delphi code into C++ (weird, I think :)
int main()
{
vector<vector<vector<int>>> A;
vector<int> Lst = { 1, 2, 3, 3, 4, 5, 6, 7 };
int k = 3;
int sum = 10;
A.push_back({ {0} }); //fictive array to make non-empty variant
for (int i = 0; i < sum; i++)
A.push_back({{}});
for (int item : Lst) {
for (int i = sum; i >= item; i--) {
for (int j = 0; j < A[i - item].size(); j++)
if (A[i - item][j].size() < k + 1 &&
A[i - item][j].size() > 0) {
vector<int> t = A[i - item][j];
t.push_back(item);
A[i].push_back(t); //add new variant including current item
}
}
}
//output needed variants
for (int i = 0; i < A[sum].size(); i++)
if (A[sum][i].size() == k + 1) {
for (int j = 1; j < A[sum][i].size(); j++) //excluding fictive 0
cout << A[sum][i][j] << " ";
cout << endl;
}
}
Here is a complete solution in Python. Translation to C++ is left to the reader.
Like the usual subset sum, generation of the doubly linked summary of the solutions is pseudo-polynomial. It is O(count_values * distinct_sums * depths_of_sums). However actually iterating through them can be exponential. But using generators the way I did avoids using a lot of memory to generate that list, even if it can take a long time to run.
from collections import namedtuple
# This is a doubly linked list.
# (value, tail) will be one group of solutions. (next_answer) is another.
SumPath = namedtuple('SumPath', 'value tail next_answer')
def fixed_sum_paths (array, target, count):
# First find counts of values to handle duplications.
value_repeats = {}
for value in array:
if value in value_repeats:
value_repeats[value] += 1
else:
value_repeats[value] = 1
# paths[depth][x] will be all subsets of size depth that sum to x.
paths = [{} for i in range(count+1)]
# First we add the empty set.
paths[0][0] = SumPath(value=None, tail=None, next_answer=None)
# Now we start adding values to it.
for value, repeats in value_repeats.items():
# Reversed depth avoids seeing paths we will find using this value.
for depth in reversed(range(len(paths))):
for result, path in paths[depth].items():
for i in range(1, repeats+1):
if count < i + depth:
# Do not fill in too deep.
break
result += value
if result in paths[depth+i]:
path = SumPath(
value=value,
tail=path,
next_answer=paths[depth+i][result]
)
else:
path = SumPath(
value=value,
tail=path,
next_answer=None
)
paths[depth+i][result] = path
# Subtle bug fix, a path for value, value
# should not lead to value, other_value because
# we already inserted that first.
path = SumPath(
value=value,
tail=path.tail,
next_answer=None
)
return paths[count][target]
def path_iter(paths):
if paths.value is None:
# We are the tail
yield []
else:
while paths is not None:
value = paths.value
for answer in path_iter(paths.tail):
answer.append(value)
yield answer
paths = paths.next_answer
def fixed_sums (array, target, count):
paths = fixed_sum_paths(array, target, count)
return path_iter(paths)
for path in fixed_sums([1,2,3,3,4,5,6,9], 10, 3):
print(path)
Incidentally for your example, here are the solutions:
[1, 3, 6]
[1, 4, 5]
[2, 3, 5]
[3, 3, 4]
You should first sort the so called array. Secondly, you should determine if the problem is actually solvable, to save time... So what you do is you take the last k elements and see if the sum of those is larger or equal to the x value, if it is smaller, you are done it is not possible to do something like that.... If it is actually equal yes you are also done there is no other permutations.... O(n) feels nice doesn't it?? If it is larger, than you got a lot of work to do..... You need to store all the permutations in an seperate array.... Then you go ahead and replace the smallest of the k numbers with the smallest element in the array.... If this is still larger than x then you do it for the second and third and so on until you get something smaller than x. Once you reach a point where you have the sum smaller than x, you can go ahead and start to increase the value of the last position you stopped at until you hit x.... Once you hit x that is your combination.... Then you can go ahead and get the previous element so if you had 1,1,5, 6 in your thingy, you can go ahead and grab the 1 as well, add it to your smallest element, 5 to get 6, next you check, can you write this number 6 as a combination of two values, you stop once you hit the value.... Then you can repeat for the others as well.... You problem can be solved in O(n!) time in the worst case.... I would not suggest that you 10^27 combinations, meaning you have more than 10^27 elements, mhmmm bad idea do you even have that much space??? That's like 3bits for the header and 8 bits for each integer you would need 9.8765*10^25 terabytes just to store that clossal array, more memory than a supercomputer, you should worry about whether your computer can even store this monster rather than if you can solve the problem, that many combinations even if you find a quadratic solution it would crash your computer, and you know what quadratic is a long way off from O(n!)...
A brute force method using recursion might look like this...
For example, given variables set, x, k, the following pseudo code might work:
setSumStructure find(int[] set, int x, int k, int setIdx)
{
int sz = set.length - setIdx;
if (sz < x) return null;
if (sz == x) check sum of set[setIdx] -> set[set.size] == k. if it does, return the set together with the sum, else return null;
for (int i = setIdx; i < set.size - (k - 1); i++)
filter(find (set, x - set[i], k - 1, i + 1));
return filteredSets;
}
I have an array of integers and need to apply a variant of the subset sum algorithm on it, except that instead of finding a set of integers whose sum is 0 I am trying to find a set of integers whose sum is n. I am unclear as to how to adapt one of the standard subset sum algorithms to this variant and was hoping for any insight into the problem.
This is subset sum problem, which is NP-Complete (there is no known efficient solution to NP-Complete problems), but if your numbers are relatively small integers - there is an efficient pseudo polynomial solution to it that follows the recurrence:
D(x,i) = false x<0
D(0,i) = true
D(x,0) = false x != 0
D(x,i) = D(x,i-1) OR D(x-arr[i],i-1)
Later, you need to step back on your choices, see where you decided to "reduce" (take the element), and where you decided not to "reduce" (not take the element), on the generated matrix.
This thread and this thread discuss how to get the elements for similar problems.
Here is a python code (taken from the thread I linked to) that does the trick.
If you are not familiar with python - read it as pseudo code, it's pretty easy to understand python!.
arr = [1,2,4,5]
n = len(arr)
SUM = 6
#pre processing:
D = [[True] * (n+1)]
for x in range(1,SUM+1):
D.append([False]*(n+1))
#DP solution to populate D:
for x in range(1,SUM+1):
for i in range(1,n+1):
D[x][i] = D[x][i-1]
if x >= arr[i-1]:
D[x][i] = D[x][i] or D[x-arr[i-1]][i-1]
print D
#get a random solution:
if D[SUM][n] == False:
print 'no solution'
else:
sol = []
x = SUM
i = n
while x != 0:
possibleVals = []
if D[x][i-1] == True:
possibleVals.append(x)
if x >= arr[i-1] and D[x-arr[i-1]][i-1] == True:
possibleVals.append(x-arr[i-1])
#by here possibleVals contains 1/2 solutions, depending on how many choices we have.
#chose randomly one of them
from random import randint
r = possibleVals[randint(0,len(possibleVals)-1)]
#if decided to add element:
if r != x:
sol.append(x-r)
#modify i and x accordingly
x = r
i = i-1
print sol
You can solve this by using dynamic programming.
Lets assume that:
N - is the sum that required (your first input).
M - is the number of summands available (your second input).
a1...aM - are the summands available.
f[x] is true when you can reach the sum of x, and false otherwise
Now the solution:
Initially f[0] = true and f[1..N] = false - we can reach only the sum of zero without taking any summand.
Now you can iterate over all ai, where i in [1..M], and with each of them perform next operation:
f[x + ai] = f[x + ai] || f[x], for each x in [M..ai] - the order of processing is relevant!
Finally you output f[N].
This solution has the complexity of O(N*M), so it is not very useful when you either have large input numbers or large number of summands.
Some time ago I used the (blazing fast) primesieve in python that I found here: Fastest way to list all primes below N
To be precise, this implementation:
def primes2(n):
""" Input n>=6, Returns a list of primes, 2 <= p < n """
n, correction = n-n%6+6, 2-(n%6>1)
sieve = [True] * (n/3)
for i in xrange(1,int(n**0.5)/3+1):
if sieve[i]:
k=3*i+1|1
sieve[ k*k/3 ::2*k] = [False] * ((n/6-k*k/6-1)/k+1)
sieve[k*(k-2*(i&1)+4)/3::2*k] = [False] * ((n/6-k*(k-2*(i&1)+4)/6-1)/k+1)
return [2,3] + [3*i+1|1 for i in xrange(1,n/3-correction) if sieve[i]]
Now I can slightly grasp the idea of the optimizing by automaticly skipping multiples of 2, 3 and so on, but when it comes to porting this algorithm to C++ I get stuck (I have a good understanding of python and a reasonable/bad understanding of C++, but good enough for rock 'n roll).
What I currently have rolled myself is this (isqrt() is just a simple integer square root function):
template <class T>
void primesbelow(T N, std::vector<T> &primes) {
T sievemax = (N-3 + (1-(N % 2))) / 2;
T i;
T sievemaxroot = isqrt(sievemax) + 1;
boost::dynamic_bitset<> sieve(sievemax);
sieve.set();
primes.push_back(2);
for (i = 0; i <= sievemaxroot; i++) {
if (sieve[i]) {
primes.push_back(2*i+3);
for (T j = 3*i+3; j <= sievemax; j += 2*i+3) sieve[j] = 0; // filter multiples
}
}
for (; i <= sievemax; i++) {
if (sieve[i]) primes.push_back(2*i+3);
}
}
This implementation is decent and automatically skips multiples of 2, but if I could port the Python implementation I think it could be much faster (50%-30% or so).
To compare the results (in the hope this question will be successfully answered), the current execution time with N=100000000, g++ -O3 on a Q6600 Ubuntu 10.10 is 1230ms.
Now I would love some help with either understanding what the above Python implementation does or that you would port it for me (not as helpful though).
EDIT
Some extra information about what I find difficult.
I have trouble with the techniques used like the correction variable and in general how it comes together. A link to a site explaining different Eratosthenes optimizations (apart from the simple sites that say "well you just skip multiples of 2, 3 and 5" and then get slam you with a 1000 line C file) would be awesome.
I don't think I would have issues with a 100% direct and literal port, but since after all this is for learning that would be utterly useless.
EDIT
After looking at the code in the original numpy version, it actually is pretty easy to implement and with some thinking not too hard to understand. This is the C++ version I came up with. I'm posting it here in full version to help further readers in case they need a pretty efficient primesieve that is not two million lines of code. This primesieve does all primes under 100000000 in about 415 ms on the same machine as above. That's a 3x speedup, better then I expected!
#include <vector>
#include <boost/dynamic_bitset.hpp>
// http://vault.embedded.com/98/9802fe2.htm - integer square root
unsigned short isqrt(unsigned long a) {
unsigned long rem = 0;
unsigned long root = 0;
for (short i = 0; i < 16; i++) {
root <<= 1;
rem = ((rem << 2) + (a >> 30));
a <<= 2;
root++;
if (root <= rem) {
rem -= root;
root++;
} else root--;
}
return static_cast<unsigned short> (root >> 1);
}
// https://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
// https://stackoverflow.com/questions/5293238/porting-optimized-sieve-of-eratosthenes-from-python-to-c/5293492
template <class T>
void primesbelow(T N, std::vector<T> &primes) {
T i, j, k, l, sievemax, sievemaxroot;
sievemax = N/3;
if ((N % 6) == 2) sievemax++;
sievemaxroot = isqrt(N)/3;
boost::dynamic_bitset<> sieve(sievemax);
sieve.set();
primes.push_back(2);
primes.push_back(3);
for (i = 1; i <= sievemaxroot; i++) {
if (sieve[i]) {
k = (3*i + 1) | 1;
l = (4*k-2*k*(i&1)) / 3;
for (j = k*k/3; j < sievemax; j += 2*k) {
sieve[j] = 0;
sieve[j+l] = 0;
}
primes.push_back(k);
}
}
for (i = sievemaxroot + 1; i < sievemax; i++) {
if (sieve[i]) primes.push_back((3*i+1)|1);
}
}
I'll try to explain as much as I can. The sieve array has an unusual indexing scheme; it stores a bit for each number that is congruent to 1 or 5 mod 6. Thus, a number 6*k + 1 will be stored in position 2*k and k*6 + 5 will be stored in position 2*k + 1. The 3*i+1|1 operation is the inverse of that: it takes numbers of the form 2*n and converts them into 6*n + 1, and takes 2*n + 1 and converts it into 6*n + 5 (the +1|1 thing converts 0 to 1 and 3 to 5). The main loop iterates k through all numbers with that property, starting with 5 (when i is 1); i is the corresponding index into sieve for the number k. The first slice update to sieve then clears all bits in the sieve with indexes of the form k*k/3 + 2*m*k (for m a natural number); the corresponding numbers for those indexes start at k^2 and increase by 6*k at each step. The second slice update starts at index k*(k-2*(i&1)+4)/3 (number k * (k+4) for k congruent to 1 mod 6 and k * (k+2) otherwise) and similarly increases the number by 6*k at each step.
Here's another attempt at an explanation: let candidates be the set of all numbers that are at least 5 and are congruent to either 1 or 5 mod 6. If you multiply two elements in that set, you get another element in the set. Let succ(k) for some k in candidates be the next element (in numerical order) in candidates that is larger than k. In that case, the inner loop of the sieve is basically (using normal indexing for sieve):
for k in candidates:
for (l = k; ; l += 6) sieve[k * l] = False
for (l = succ(k); ; l += 6) sieve[k * l] = False
Because of the limitations on which elements are stored in sieve, that is the same as:
for k in candidates:
for l in candidates where l >= k:
sieve[k * l] = False
which will remove all multiples of k in candidates (other than k itself) from the sieve at some point (either when the current k was used as l earlier or when it is used as k now).
Piggy-Backing onto Howard Hinnant's response, Howard, you don't have to test numbers in the set of all natural numbers not divisible by 2, 3 or 5 for primality, per se. You need simply multiply each number in the array (except 1, which self-eliminates) times itself and every subsequent number in the array. These overlapping products will give you all the non-primes in the array up to whatever point you extend the deterministic-multiplicative process. Thus the first non-prime in the array will be 7 squared, or 49. The 2nd, 7 times 11, or 77, etc. A full explanation here: http://www.primesdemystified.com
As an aside, you can "approximate" prime numbers. Call the approximate prime P. Here are a few formulas:
P = 2*k+1 // not divisible by 2
P = 6*k + {1, 5} // not divisible 2, 3
P = 30*k + {1, 7, 11, 13, 17, 19, 23, 29} // not divisble by 2, 3, 5
The properties of the set of numbers found by these formulas is that P may not be prime, however all primes are in the set P. I.e. if you only test numbers in the set P for prime, you won't miss any.
You can reformulate these formulas to:
P = X*k + {-i, -j, -k, k, j, i}
if that is more convenient for you.
Here is some code that uses this technique with a formula for P not divisible by 2, 3, 5, 7.
This link may represent the extent to which this technique can be practically leveraged.
I've been trying to solve a problem in combinations. I have a matrix 6X6 i'm trying to find all combinations of length 8 in the matrix.
I have to move from neighbor to neighbor form each row,column position and i wrote a recursive program which generates the combination but the problem is it generates a lot of duplicates as well and hence is inefficient. I would like to know how could i eliminate calculating duplicates and save time.
int a={{1,2,3,4,5,6},
{8,9,1,2,3,4},
{5,6,7,8,9,1},
{2,3,4,5,6,7},
{8,9,1,2,3,4},
{5,6,7,8,9,1},
}
void genSeq(int row,int col,int length,int combi)
{
if(length==8)
{
printf("%d\n",combi);
return;
}
combi = (combi * 10) + a[row][col];
if((row-1)>=0)
genSeq(row-1,col,length+1,combi);
if((col-1)>=0)
genSeq(row,col-1,length+1,combi);
if((row+1)<6)
genSeq(row+1,col,length+1,combi);
if((col+1)<6)
genSeq(row,col+1,length+1,combi);
if((row+1)<6&&(col+1)<6)
genSeq(row+1,col+1,length+1,combi);
if((row-1)>=0&&(col+1)<6)
genSeq(row-1,col+1,length+1,combi);
if((row+1)<6&&(row-1)>=0)
genSeq(row+1,col-1,length+1,combi);
if((row-1)>=0&&(col-1)>=0)
genSeq(row-1,col-1,length+1,combi);
}
I was also thinking of writing a dynamic program basically recursion with memorization. Is it a better choice?? if yes than I'm not clear how to implement it in recursion. Have i really hit a dead end with approach???
Thankyou
Edit
Eg result
12121212,12121218,12121219,12121211,12121213.
the restrictions are that you have to move to your neighbor from any point, you have to start for each position in the matrix i.e each row,col. you can move one step at a time, i.e right, left, up, down and the both diagonal positions. Check the if conditions.
i.e
if your in (0,0) you can move to either (1,0) or (1,1) or (0,1) i.e three neighbors.
if your in (2,2) you can move to eight neighbors.
so on...
To eliminate duplicates you can covert 8 digit sequences into 8-digit integers and put them in a hashtable.
Memoization might be a good idea. You can memoize for each cell in the matrix all possible combinations of length 2-7 that can be achieved from it. Going backwards: first generate for each cell all sequences of 2 digits. Then based on that of 3 digits etc.
UPDATE: code in Python
# original matrix
lst = [
[1,2,3,4,5,6],
[8,9,1,2,3,4],
[5,6,7,8,9,1],
[2,3,4,5,6,7],
[8,9,1,2,3,4],
[5,6,7,8,9,1]]
# working matrtix; wrk[i][j] contains a set of all possible paths of length k which can end in lst[i][j]
wrk = [[set() for i in range(6)] for j in range(6)]
# for the first (0rh) iteration initialize with single step paths
for i in range(0, 6):
for j in range(0, 6):
wrk[i][j].add(lst[i][j])
# run iterations 1 through 7
for k in range(1,8):
# create new emtpy wrk matrix for the next iteration
nw = [[set() for i in range(6)] for j in range(6)]
for i in range(0, 6):
for j in range(0, 6):
# the next gen. wrk[i][j] is going to be based on the current wrk paths of its neighbors
ns = set()
if i > 0:
for p in wrk[i-1][j]:
ns.add(10**k * lst[i][j] + p)
if i < 5:
for p in wrk[i+1][j]:
ns.add(10**k * lst[i][j] + p)
if j > 0:
for p in wrk[i][j-1]:
ns.add(10**k * lst[i][j] + p)
if j < 5:
for p in wrk[i][j+1]:
ns.add(10**k * lst[i][j] + p)
nw[i][j] = ns
wrk = nw
# now build final set to eliminate duplicates
result = set()
for i in range(0, 6):
for j in range(0, 6):
result |= wrk[i][j]
print len(result)
print result
There are LOTS of ways to do this. Going through every combination is a perfectly reasonable first approach. It all depends on your requirements. If your matrix is small, and this operation isn't time sensitive, then there's no problem.
I'm not really an algorithms guy, but I'm sure there are really clever ways of doing this that someone will post after me.
Also, in Java when using CamelCase, method names should start with a lowercase character.
int a={{1,2,3,4,5,6},
{8,9,1,2,3,4},
{5,6,7,8,9,1},
{2,3,4,5,6,7},
{8,9,1,2,3,4},
{5,6,7,8,9,1},
}
By length you mean summation of combination of matrix elements resulting 8. i.e., elements to sum up 8 with in row itself and with the other row elements. From row 1 = { {2,6}, {3,5}, } and now row 1 elements with row 2 and so on. Is that what you are expecting ?
You can think about your matrix like it is one-dimension array - no matter here ("place" the rows one by one). For one-dimension array you can write a function like (assuming you should print the combinations)
f(i, n) prints all combinations of length n using elements a[i] ... a[last].
It should skip some elements from a[i] to a[i + k] (for all possible k), print a[k] and make a recursive call f(i + k + 1, n - 1).