Mathematica: converting output from Cluster[] - list

Imagine a data set like this:
{{{1,2},{3,4}},{{8,8},{3,7},{5,2}}}.
Note that at the top level this list has {{1,2},{3,4}} as the first element and {{8,8},{3,7},{5,2}} as the second.
Using that fact, the desired output would be:
{{1,2,1},{3,4,1},{8,8,2},{3,7,2},{5,2,2}}
I have already tried using Map[].
This arose because I was using cluster analysis which gave me a list, rather than an indexing of various clusters. I did not find an option in Cluster[] to do this directly.


In[1]:= v = {{{1, 2}, {3, 4}}, {{8, 8}, {3, 7}, {5, 2}}};
Flatten[Table[Map[Join[#, {i}] &, v[[i]]], {i, 1, Length[v]}], 1]
Out[1]= {{1, 2, 1}, {3, 4, 1}, {8, 8, 2}, {3, 7, 2}, {5, 2, 2}}


This is how I would go about the conversion, using the steps as they naturally come to mind.
v = {{{1, 2}, {3, 4}}, {{8, 8}, {3, 7}, {5, 2}}};
Note the result obtained using MapIndexed :-
MapIndexed[{#1, First[#2]} &, v]
{{{{1, 2}, {3, 4}}, 1}, {{{8, 8}, {3, 7}, {5, 2}}, 2}}
To append the part specs (1 & 2) to the subelements I would use MapThread. This requires multiple part specs, e.g. {2, 2, 2} for element 2 :-
MapThread[Append, {{{8, 8}, {3, 7}, {5, 2}}, {2, 2, 2}}]
{{8, 8, 2}, {3, 7, 2}, {5, 2, 2}}
So the MapIndexed expression is modified to produce the necessary part specs :-
MapIndexed[{#1, ConstantArray[First[#2], Length[#1]]} &, v]
{{{{1, 2}, {3, 4}}, {1, 1}}, {{{8, 8}, {3, 7}, {5, 2}}, {2, 2, 2}}}
Now MapThread can be used in the MapIndexed expression :-
MapIndexed[MapThread[Append, {#1, ConstantArray[First[#2], Length[#1]]}] &, v]
{{1, 2, 1}, {3, 4, 1}}, {{8, 8, 2}, {3, 7, 2}, {5, 2, 2}}}
Finally, the first list level is flattened :-
Flatten[MapIndexed[MapThread[Append,
{#1, ConstantArray[First[#2], Length[#1]]}] &, v], 1]
{{1, 2, 1}, {3, 4, 1}, {8, 8, 2}, {3, 7, 2}, {5, 2, 2}}

Related

C++ Unordered map initializer map not initializing properly

I'm relatively new to hash maps. I have the following code in my program:
std::unordered_map<int, int> XY ({
{0, 0}, {0, 3}, {0, 6},
{3, 0}, {3, 3}, {3, 6},
{6, 0}, {6, 3}, {6, 6}
});
For some reason, the map only contains the first three pairs ({0, 0}, {0, 3}, and {0, 6}). Even when I cout the bucket count, it outputs 11. Yet, there's still only three in my map.
How do I fix this? It seems unreasonable to do a bunch of .insert()'s.

Manipulating Tables (or lists) in Mathematcia

For scientific purposes (code research) I am using Mathematica to enumerate all periodic sequences of some linear recurrences. As an example the command
Table[{Mod[i * 2^n + j * 4^n + k * 6^n, 7] },{i, 0, 5}, {j, 0, 5}, {k, 0, 5}, {n, 0, 5}]
allows to enumerate all 216 distinct periodic sequences of linear recurrent sequences in a finite field of order 7 (or mod 7) with characteristic polynomial whose roots are 2,4 and 6. The first five sequences it produces are:
{0, 0, 0, 0, 0, 0}, {1, 6, 1, 6, 1, 6}, {2, 5, 2, 5, 2, 5}, {3, 4, 3, 4, 3, 4}, {4, 3, 4, 3, 4, 3}, …
I have two questions:
i) The first sequence is obtained when i=0,j=0,k=0; the second when i=0,j=0,k=1, the third when i=0,j=0,k=2, etc. Is there a way to join these numbers with the sequence they generate so that I will get to know these parameters and therefore to be able to, later (if needed), generate a particular sequence? That is I would like that the output would be something like this
{{0, 0, 0, 0, 0, 0}, {0, 0, 0}}, {{1, 6, 1, 6, 1, 6}, {0, 0, 1}}, {{2, 5, 2, 5, 2, 5}, {0, 0, 2}} , etc.
ii) In the example above (3rd order linear recurrence, and mod 7) the number of sequences obtained (216) is manageable by hand; but this number grows very quickly when the linear recurrence has order higher than 3 and the field has characteristic higher than 7. And, in those cases, most of the sequences that are produced are of no interest to me. I think that I could throw away at least 99% of the sequences that do not interest me if I could add an instruction that would read the output (the sequences obtained) and would say “I only want the sequences such that the products of its elements is 216 (say)”: from the five sequences above only {1, 6, 1, 6, 1, 6} would remain because 1x6x1x6x1x6=216$; or “I only want the sequences such that the products of its elements is 216 or 1000 (say)” from the five sequences above {1, 6, 1, 6, 1, 6} and {2, 5, 2, 5, 2, 5} would remain because 1x6x1x6x1x6 = 216 and 2x5x2x5x2x 5=1000.
Is it possible to do this? I tried some list and tables manipulation, but had no success.
Thank you in advance.

here is the first part..
Flatten[Table[{Table[Mod[i*2^n + j*4^n + k*6^n, 7], {n, 0, 5}], {i, j,
k}}, {i, 0, 5}, {j, 0, 5}, {k, 0, 5}], 2]
{{{0, 0, 0, 0, 0, 0}, {0, 0, 0}}, {{1, 6, 1, 6, 1, 6}, {0, 0,
1}}, {{2, 5, 2, 5, 2, 5}, {0, 0, 2}}, {{3, 4, 3, 4, 3, 4}, {0, 0,
3}},...
better way:
{Table[Mod[#.{2, 4, 6}^n, 7], {n, 0, 5}],#} & /# Tuples[Range[0, 5], {3}]
example finding cases with a specified product:
Reap[Do[
s = Table[Mod[i*2^n + j*4^n + k*6^n, 7], {n, 0, 5}];
If[Times ## s == 81, Sow[{s, {i, j, k}}]],
{i, 0, 5}, {j, 0, 5}, {k, 0, 5}]][[2, 1]]
{{{3, 3, 1, 3, 3, 1}, {1, 2, 0}}, {{3, 1, 3, 3, 1, 3}, {2, 1,
0}}, {{1, 3, 3, 1, 3, 3}, {4, 4, 0}}}

Mathematica: adding a list of lists elementwise

I have a list of lists of numbers. I add them into one list by adding all of the first elements together, all of the second elements together, etc. For example, if my list were { {1,2,3}, {1,2,3}, {1,2,3,4} } I would want to end up with {3,6,9,4}. How do I do this in Mathematica?

a = {{1, 2, 3}, {1, 2, 3}, {1, 2, 3, 4}};
Total#PadRight#a
{3, 6, 9, 4}

Among its many useful features, Flatten will transpose a 'ragged' array (see here for a nice explanation, or check out the 'applications' subsection of the documentation on Flatten)
Total /# Flatten[#, {{2}}] &#{{1, 2, 3}, {1, 2, 3}, {1, 2, 3, 4}}
{3, 6, 9, 4}

If all the rows were the same length then adding the rows would do this.
So make all the rows the same length by appending zeros and then add them.
lists = {{1, 2, 3}, {1, 2, 3}, {1, 2, 3, 4}};
max = Max[Length /# lists]; min = Min[Length /# lists];
zeros = Table[0, {max - min}];
Plus ## Map[Take[Join[#, zeros], max] &, lists]

2D int array in C++

So I want to initialize an int 2d array very quickly, but I can't figure out how to do it. I've done a few searches and none of them say how to initialize a 2D array, except to do:
int [SOME_CONSTANT][ANOTHER_CONSTANT] = {{0}};
Basically, I've got 8 vertices, and I'm listing the 4 vertices of each face of a cube in an array. I've tried this:
int[6][4] sides = {{0, 1, 2, 3}, {4, 5, 6, 7}, {0, 4, 7, 3}, {7, 6, 2, 3}, {5, 1, 2, 6}, {0, 1, 5, 4}};
But that tells me that there's an error with 'sides', and that it expected a semi-colon. Is there any way to initialize an array quickly like this?
Thanks!

You have the [][] on the wrong side. Try this:
int sides[6][4] = {{0, 1, 2, 3}, {4, 5, 6, 7}, {0, 4, 7, 3}, {7, 6, 2, 3}, {5, 1, 2, 6}, {0, 1, 5, 4}};
Keep in mind that what you really have is:
int **sides
(A pointer to a pointer of ints). It's sides that has the dimensions, not the int. Therefore, you could also do:
int x, y[2], z[3][4], ...;

I think You meant to say
int sides[6][4] = {{0, 1, 2, 3}, {4, 5, 6, 7}, {0, 4, 7, 3}, {7, 6, 2, 3}, {5, 1, 2, 6}, {0, 1, 5, 4}};

int array[n][m] behaves just like int array[n * m].
In fact, array[i][j] = array[m * i + j] for all i, j.
So int array[2][3] = {1, 2, 3, 4, 5, 6}; is a valid declaration and, for example,
array[1][1] = array[3 * 1 + 1] = array[4] = 5.

int sides[6][4] = {{0, 1, 2, 3}, {4, 5, 6, 7}, {0, 4, 7, 3}, {7, 6, 2, 3}, {5, 1, 2, 6}, {0, 1, 5, 4}};
I'm not a regular c++ programmer but I looks like int sides[6][4] seems to compile while int[6][4] sides fails. Languages like C# lets you have the [][] on either sides but apparently c++ doesn't.

int sides[6][4] = ... should do the trick. This sounds like you may be coming from a Java (or other language) background so I do recommend a C++ book The Definitive C++ Book Guide and List for more details.

Yes, the intended type of sides is int[6][4], but C++ has confusing syntax sometimes. The way to declare said array is:
int sides[6][4] = {/*stuff*/};
You run into this with function pointers too, but even worse:
int (*myfuncptr)(int); //creates a function pointer called myfuncptr
With function pointers though, you can do this:
typedef int (*func_ptr_type)(int);
func_ptr_type myfuncptr;
Unfortunately, there's no corresponding magic trick for arrays.

i would make a array outside of function and just assign it it to your local. this will very likely invoke memcpy or just inline memory copying loop
this is the fastest you can get

Optimally picking one element from each list

I came across an old problem that you Mathematica/StackOverflow folks will probably like and that seems valuable to have on StackOverflow for posterity.
Suppose you have a list of lists and you want to pick one element from each and put them in a new list so that the number of elements that are identical to their next neighbor is maximized.
In other words, for the resulting list l, minimize Length#Split[l].
In yet other words, we want the list with the fewest interruptions of identical contiguous elements.
For example:
pick[{ {1,2,3}, {2,3}, {1}, {1,3,4}, {4,1} }]
--> { 2, 2, 1, 1, 1 }
(Or {3,3,1,1,1} is equally good.)
Here's a preposterously brute force solution:
pick[x_] := argMax[-Length#Split[#]&, Tuples[x]]
where argMax is as described here:
posmax: like argmax but gives the position(s) of the element x for which f[x] is maximal
Can you come up with something better?
The legendary Carl Woll nailed this for me and I'll reveal his solution in a week.

Not an answer, but a comparison of the methods proposed here. I generated test sets with a variable number of subsets this number varying from 5 to 100. Each test set was generated with this code
Table[RandomSample[Range[10], RandomInteger[{1, 7}]], {rl}]
with rl the number of subsets involved.
For every test set that was generated this way I had all the algorithms do their thing. I did this 10 times (with the same test set) with the algorithms operating in a random order so as to level out order effects and the effects of random background processes on my laptop. This results in mean timing for the given data set. The above line was used 20 times for each rl length, from which a mean (of means) and a standard deviation were calculated.
The results are below (horizontally the number of subsets and vertically the mean AbsoluteTiming):
It seems that Mr.Wizard is the (not so clear) winner. Congrats!
Update
As requested by Timo here the timings as a function of the number of distinct subset elements that can be chosen from as well as the maximum number of elements in each subset. The data sets are generated for a fixed number of subsets (50) according to this line of code:
lst = Table[RandomSample[Range[ch], RandomInteger[{1, ch}]], {50}];
I also increased the number of datasets I tried for each value from 20 to 40.
Here for 5 subsets:

I'll toss this into the ring. I am not certain it always gives an optimal solution, but it appears to work on the same logic as some other answers given, and it is fast.
f#{} := (Sow[m]; m = {i, 1})
f#x_ := m = {x, m[[2]] + 1}
findruns[lst_] :=
Reap[m = {{}, 0}; f[m[[1]] ⋂ i] ~Do~ {i, lst}; Sow#m][[2, 1, 2 ;;]]
findruns gives run-length-encoded output, including parallel answers. If output as strictly specified is required, use:
Flatten[First[#]~ConstantArray~#2 & ### #] &
Here is a variation using Fold. It is faster on some set shapes, but a little slower on others.
f2[{}, m_, i_] := (Sow[m]; {i, 1})
f2[x_, m_, _] := {x, m[[2]] + 1}
findruns2[lst_] :=
Reap[Sow#Fold[f2[#[[1]] ⋂ #2, ##] &, {{}, 0}, lst]][[2, 1, 2 ;;]]

This is my take on it, and does pretty much the same thing as Sjoerd, just in a less amount of code.
LongestRuns[list_List] :=
Block[{gr, f = Intersection},
ReplaceRepeated[
list, {a___gr, Longest[e__List] /; f[e] =!= {}, b___} :> {a,
gr[e], b}] /.
gr[e__] :> ConstantArray[First[f[e]], Length[{e}]]]
Some gallery:
In[497]:= LongestRuns[{{1, 2, 3}, {2, 3}, {1}, {1, 3, 4}, {4, 1}}]
Out[497]= {{2, 2}, {1, 1, 1}}
In[498]:= LongestRuns[{{3, 10, 6}, {8, 2, 10, 5, 9, 3, 6}, {3, 7, 10,
2, 8, 5, 9}, {6, 9, 1, 8, 3, 10}, {1}, {2, 9, 4}, {9, 5, 2, 6, 8,
7}, {6, 9, 4, 5}}]
Out[498]= {{3, 3, 3, 3}, {1}, {9, 9, 9}}
In[499]:= pickPath[{{3, 10, 6}, {8, 2, 10, 5, 9, 3, 6}, {3, 7, 10, 2,
8, 5, 9}, {6, 9, 1, 8, 3, 10}, {1}, {2, 9, 4}, {9, 5, 2, 6, 8,
7}, {6, 9, 4, 5}}]
Out[499]= {{10, 10, 10, 10}, {{1}, {9, 9, 9}}}
In[500]:= LongestRuns[{{2, 8}, {4, 2}, {3}, {9, 4, 6, 8, 2}, {5}, {8,
10, 6, 2, 3}, {9, 4, 6, 3, 10, 1}, {9}}]
Out[500]= {{2, 2}, {3}, {2}, {5}, {3, 3}, {9}}
In[501]:= LongestRuns[{{4, 6, 18, 15}, {1, 20, 16, 7, 14, 2, 9}, {12,
3, 15}, {17, 6, 13, 10, 3, 19}, {1, 15, 2, 19}, {5, 17, 3, 6,
14}, {5, 17, 9}, {15, 9, 19, 13, 8, 20}, {18, 13, 5}, {11, 5, 1,
12, 2}, {10, 4, 7}, {1, 2, 14, 9, 12, 3}, {9, 5, 19, 8}, {14, 1, 3,
4, 9}, {11, 13, 5, 1}, {16, 3, 7, 12, 14, 9}, {7, 4, 17, 18,
6}, {17, 19, 9}, {7, 15, 3, 12}, {19, 12, 5, 14, 8}, {1, 10, 12,
8}, {18, 16, 14, 19}, {2, 7, 10}, {19, 2, 5, 3}, {16, 17, 3}, {16,
2, 6, 20, 1, 3}, {12, 18, 11, 19, 17}, {12, 16, 9, 20, 4}, {19, 20,
10, 12, 9, 11}, {10, 12, 6, 19, 17, 5}}]
Out[501]= {{4}, {1}, {3, 3}, {1}, {5, 5}, {13, 13}, {1}, {4}, {9, 9,
9}, {1}, {7, 7}, {9}, {12, 12, 12}, {14}, {2, 2}, {3, 3}, {12, 12,
12, 12}}
EDIT given that Sjoerd's Dreeves's brute force approach fails on large samples due to inability to generate all Tuples at once, here is another brute force approach:
bfBestPick[e_List] := Block[{splits, gr, f = Intersection},
splits[{}] = {{}};
splits[list_List] :=
ReplaceList[
list, {a___gr, el__List /; f[el] =!= {},
b___} :> (Join[{a, gr[el]}, #] & /# splits[{b}])];
Module[{sp =
Cases[splits[
e] //. {seq__gr,
re__List} :> (Join[{seq}, #] & /# {re}), {__gr}, Infinity]},
sp[[First#Ordering[Length /# sp, 1]]] /.
gr[args__] :> ConstantArray[First[f[args]], Length[{args}]]]]
This brute-force-best-pick might generate different splitting, but it is length that matters according to the original question.
test = {{4, 6, 18, 15}, {1, 20, 16, 7, 14, 2, 9}, {12, 3, 15}, {17, 6,
13, 10, 3, 19}, {1, 15, 2, 19}, {5, 17, 3, 6, 14}, {5, 17,
9}, {15, 9, 19, 13, 8, 20}, {18, 13, 5}, {11, 5, 1, 12, 2}, {10,
4, 7}, {1, 2, 14, 9, 12, 3}, {9, 5, 19, 8}, {14, 1, 3, 4, 9}, {11,
13, 5, 1}, {16, 3, 7, 12, 14, 9}, {7, 4, 17, 18, 6}, {17, 19,
9}, {7, 15, 3, 12}, {19, 12, 5, 14, 8}, {1, 10, 12, 8}, {18, 16,
14, 19}, {2, 7, 10}, {19, 2, 5, 3}, {16, 17, 3}, {16, 2, 6, 20, 1,
3}, {12, 18, 11, 19, 17}, {12, 16, 9, 20, 4}, {19, 20, 10, 12, 9,
11}, {10, 12, 6, 19, 17, 5}};
pick fails on this example.
In[637]:= Length[bfBestPick[test]] // Timing
Out[637]= {58.407, 17}
In[638]:= Length[LongestRuns[test]] // Timing
Out[638]= {0., 17}
In[639]:=
Length[Cases[pickPath[test], {__Integer}, Infinity]] // Timing
Out[639]= {0., 17}
I am posting this in case somebody might want to search for counterexamples that the code like pickPath or LongestRuns does indeed generate a sequence with smallest number of interruptions.

Here's a go at it...
runsByN: For each number, show whether it appears or not in each sublist
list= {{4, 2, 7, 5, 1, 9, 10}, {10, 1, 8, 3, 2, 7}, {9, 2, 7, 3, 6, 4, 5}, {10, 3, 6, 4, 8, 7}, {7}, {3, 1, 8, 2, 4, 7, 10, 6}, {7, 6}, {10, 2, 8, 5, 6, 9, 7, 3}, {1, 4, 8}, {5, 6, 1}, {3, 2, 1}, {10,6, 4}, {10, 7, 3}, {10, 2, 4}, {1, 3, 5, 9, 7, 4, 2, 8}, {7, 1, 3}, {5, 7, 1, 10, 2, 3, 6, 8}, {10, 8, 3, 6, 9, 4, 5, 7}, {3, 10, 5}, {1}, {7, 9, 1, 6, 2, 4}, {9, 7, 6, 2}, {5, 6, 9, 7}, {1, 5}, {1,9, 7, 5, 4}, {5, 4, 9, 3, 1, 7, 6, 8}, {6}, {10}, {6}, {7, 9}};
runsByN = Transpose[Table[If[MemberQ[#, n], n, 0], {n, Max[list]}] & /# list]
Out = {{1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0,1, 1, 1, 0, 0, 0, 0}, {2, 2, 2, 0, 0, 2, 0, 2, 0, 0, 2, 0, 0, 2, 2,0, 2, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 3, 3, 3, 0, 3, 0,3, 0, 0, 3, 0, 3, 0, 3, 3, 3, 3, 3, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0,0}, {4, 0, 4, 4, 0, 4, 0, 0, 4, 0, 0, 4, 0, 4, 4, 0, 0, 4, 0, 0, 4, 0, 0, 0, 4, 4, 0, 0, 0, 0}, {5, 0, 5, 0, 0, 0, 0, 5, 0, 5, 0, 0, 0, 0, 5, 0, 5, 5, 5, 0, 0, 0, 5, 5, 5, 5, 0, 0, 0, 0}, {0, 0, 6, 6, 0, 6, 6, 6, 0, 6, 0, 6, 0, 0, 0, 0, 6, 6, 0, 0, 6, 6, 6, 0, 0, 6, 6, 0,6, 0}, {7, 7, 7, 7, 7, 7, 7, 7, 0, 0, 0, 0, 7, 0, 7, 7, 7, 7, 0, 0, 7, 7, 7, 0, 7, 7, 0, 0, 0, 7}, {0, 8, 0, 8, 0, 8, 0, 8, 8, 0, 0, 0, 0, 0, 8, 0, 8, 8, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0}, {9, 0, 9, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 9, 0, 0, 9, 0, 0, 9, 9, 9, 0, 9, 9, 0, 0, 0, 9}, {10, 10, 0, 10, 0, 10, 0, 10, 0, 0, 0, 10, 10, 10, 0, 0, 10, 10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0}};
runsByN is list transposed, with zeros inserted to represent missing numbers. It shows the sublists in which 1, 2, 3, and 4 appeared.
myPick: Picking numbers that constitute an optimal path
myPick recursively builds a list of the longest runs. It doesn't look for all optimal solutions, but rather the first solution of minimal length.
myPick[{}, c_] := Flatten[c]
myPick[l_, c_: {}] :=
Module[{r = Length /# (l /. {x___, 0, ___} :> {x}), m}, m = Max[r];
myPick[Cases[(Drop[#, m]) & /# l, Except[{}]],
Append[c, Table[Position[r, m, 1, 1][[1, 1]], {m}]]]]
choices = myPick[runsByN]
(* Out= {7, 7, 7, 7, 7, 7, 7, 7, 1, 1, 1, 10, 10, 10, 3, 3, 3, 3, 3, 1, 1, 6, 6, 1, 1, 1, 6, 10, 6, 7} *)
Thanks to Mr.Wizard for suggesting the use of a replacement rule as an efficient alternative to TakeWhile.
Epilog:Visualizing the solution path
runsPlot[choices1_, runsN_] :=
Module[{runs = {First[#], Length[#]} & /# Split[choices1], myArrow,
m = Max[runsN]},
myArrow[runs1_] :=
Module[{data1 = Reverse#First[runs1], data2 = Reverse[runs1[[2]]],
deltaX},
deltaX := data2[[1]] - 1;
myA[{}, _, out_] := out;
myA[inL_, deltaX_, outL_] :=
Module[{data3 = outL[[-1, 1, 2]]},
myA[Drop[inL, 1], inL[[1, 2]] - 1,
Append[outL, Arrow[{{First[data3] + deltaX,
data3[[2]]}, {First[data3] + deltaX + 1, inL[[1, 1]]}}]]]];
myA[Drop[runs1, 2], deltaX, {Thickness[.005],
Arrow[{data1, {First[data1] + 1, data2[[2]]}}]}]];
ListPlot[runsN,
Epilog -> myArrow[runs],
PlotStyle -> PointSize[Large],
Frame -> True,
PlotRange -> {{1, Length[choices1]}, {1, m}},
FrameTicks -> {All, Range[m]},
PlotRangePadding -> .5,
FrameLabel -> {"Sublist", "Number", "Sublist", "Number"},
GridLines :> {FoldList[Plus, 0, Length /# Split[choices1]], None}
]];
runsPlot[choices, runsByN]
The chart below represents the data from list.
Each plotted point corresponds to a number and the sublist in which it occurred.

So here is my "one liner" with improvements by Mr.Wizard:
pickPath[lst_List] :=
Module[{M = Fold[{#2, #} &, {{}}, Reverse#lst]},
Reap[While[M != {{}},
Do[Sow##[[-2,1]], {Length## - 1}] &#
NestWhileList[# ⋂ First[M = Last#M] &, M[[1]], # != {} &]
]][[2, 1]]
]
It basically uses intersection repeatedly on consecutive lists until it comes up empty, and then does it again and again. In a humongous torture test case with
M = Table[RandomSample[Range[1000], RandomInteger[{1, 200}]], {1000}];
I get Timing[] consistently around 0.032 on my 2GHz Core 2 Duo.
Below this point is my first attempt, which I'll leave for your perusal.
For a given list of lists of elements M we count the different elements and the number of lists, list the different elements in canonical order, and construct a matrix K[i,j] detailing the presence of element i in list j:
elements = Length#(Union ## M);
lists = Length#M;
eList = Union ## M;
positions = Flatten#Table[{i, Sequence ## First#Position[eList, M[[i,j]]} -> 1,
{i, lists},
{j, Length#M[[i]]}];
K = Transpose#Normal#SparseArray#positions;
The problem is now equivalent to traversing this matrix from left to right, by only stepping on 1's, and changing rows as few times as possible.
To achieve this I Sort the rows, take the one with the most consecutive 1's at the start, keep track of what element I picked, Drop that many columns from K and repeat:
R = {};
While[Length#K[[1]] > 0,
len = LengthWhile[K[[row = Last#Ordering#K]], # == 1 &];
Do[AppendTo[R, eList[[row]]], {len}];
K = Drop[#, len] & /# K;
]
This has an AbsoluteTiming of approximately three times that of Sjoerd's approach.

My solution is based on the observation that 'greed is good' here. If I have the choice between interrupting a chain and beginning a new, potentially long chain, picking the new one to continue doesn't do me any good. The new chain gets longer with the same amount as the old chain gets shorter.
So, what the algorithm basically does is starting at the first sublist and for each of its members finding the number of additional sublists that have the same member and choosing the sublist member that has the most neighboring twins. This process then continues at the sublist at the end of this first chain and so on.
So combining this in a recursive algorithm we end up with:
pickPath[lst_] :=
Module[{lengthChoices, bestElement},
lengthChoices =
LengthWhile[lst, Function[{lstMember}, MemberQ[lstMember, #]]] & /#First[lst];
bestElement = Ordering[lengthChoices][[-1]];
If[ Length[lst] == lengthChoices[[bestElement]],
ConstantArray[lst[[1, bestElement]], lengthChoices[[bestElement]]],
{
ConstantArray[lst[[1, bestElement]], lengthChoices[[bestElement]]],
pickPath[lst[[lengthChoices[[bestElement]] + 1 ;; -1]]]
}
]
]
Test
In[12]:= lst =
Table[RandomSample[Range[10], RandomInteger[{1, 7}]], {8}]
Out[12]= {{3, 10, 6}, {8, 2, 10, 5, 9, 3, 6}, {3, 7, 10, 2, 8, 5,
9}, {6, 9, 1, 8, 3, 10}, {1}, {2, 9, 4}, {9, 5, 2, 6, 8, 7}, {6, 9,
4, 5}}
In[13]:= pickPath[lst] // Flatten // AbsoluteTiming
Out[13]= {0.0020001, {10, 10, 10, 10, 1, 9, 9, 9}}
Dreeves' Brute Force approach
argMax[f_, dom_List] :=
Module[{g}, g[e___] := g[e] = f[e];(*memoize*) dom[[Ordering[g /# dom, -1]]]]
pick[x_] := argMax[-Length#Split[#] &, Tuples[x]]
In[14]:= pick[lst] // AbsoluteTiming
Out[14]= {0.7340420, {{10, 10, 10, 10, 1, 9, 9, 9}}}
The first time I used a slightly longer test list. The brute force approach brought my computer to a virtual standstill, claiming all the memory it had. Pretty bad. I had to restart after 10 minutes. Restarting took me another quarter, due to the PC becoming extremely non-responsive.

Could use integer linear programming. Here is code for that.
bestPick[lists_] := Module[
{picks, span, diffs, v, dv, vars, diffvars, fvars,
c1, c2, c3, c4, constraints, obj, res},
span = Max[lists] - Min[lists];
vars = MapIndexed[v[Sequence ## #2] &, lists, {2}];
picks = Total[vars*lists, {2}];
diffs = Differences[picks];
diffvars = Array[dv, Length[diffs]];
fvars = Flatten[{vars, diffvars}];
c1 = Map[Total[#] == 1 &, vars];
c2 = Map[0 <= # <= 1 &, fvars];
c3 = Thread[span*diffvars >= diffs];
c4 = Thread[span*diffvars >= -diffs];
constraints = Join[c1, c2, c3, c4];
obj = Total[diffvars];
res = Minimize[{obj, constraints}, fvars, Integers];
{res[[1]], Flatten[vars*lists /. res[[2]] /. 0 :> Sequence[]]}
]
Your example:
lists = {{1, 2, 3}, {2, 3}, {1}, {1, 3, 4}, {4, 1}}
bestPick[lists]
Out[88]= {1, {2, 2, 1, 1, 1}}
For larger problems Minimize might run into trouble since it uses exact methods for solving relaxed LPs. In which case you might need to switch to NMinimize, and change the domain argument to a constraint of the form Element[fvars,Integers].
Daniel Lichtblau

A week is up! Here is the fabled solution from Carl Woll. (I tried to get him to post it himself. Carl, if you come across this and want to take official credit, just paste it in as a separate answer and I'll delete this one!)
pick[data_] := Module[{common,tmp},
common = {};
tmp = Reverse[If[(common = Intersection[common,#])=={}, common = #, common]& /#
data];
common = .;
Reverse[If[MemberQ[#, common], common, common = First[#]]& /# tmp]]
Still quoting Carl:
Basically, you start at the beginning, and find the element which gives you
the longest string of common elements. Once the string can no longer be
extended, start a new string. It seems to me that this algorithm ought to
give you a correct answer (there are many correct answers).