Related
I have a list of custom data objects which track an increasing total value on a daily basis using one field total. Another field in the custom data type is the value new. Using a csv file I have read in the values for date and total and am trying to calculate and set the values for new from these values.
data item = Item{
date :: Day,
total :: Int,
new :: Int
}
Before
date
total
new
01/01/2021
0
0
02/01/2021
2
0
03/01/2021
6
0
04/01/2021
15
0
After
date
total
new
01/01/2021
0
0
02/01/2021
2
2
03/01/2021
6
4
04/01/2021
15
9
My understanding is that in haskell I should be trying to avoid the use of for loops which iterate over a list until the final row is reached, for example using a loop control which terminates upon reaching a value equal to the length of the list.
Instead I have tried to create a function which assigns the value of new which can used with map to update each item in the list. My problem is that such a function requires access to both the item being updated, as well as the previous item's value for total and I'm unsure how to implement this in haskell.
--Set daily values by mapping single update function across list
calcNew:: [Item] -> Int -> [Item]
calcNew items = map updateOneItem items
-- takes an item and a value to fill the new field
updateOneItem :: Item -> Int -> Item
updateOneItem item x = Item date item total item x
Is it possible to populate that value while using map? If not, is a recursive solution required?
We can do this by zipping the input list with itself, shifted by one step.
Assuming you have a list of items already populated with total values, which you want to update to contain the correct new values (building an updated copy of course),
type Day = Int
data Item = Item{ -- data Item, NB
date :: Day,
total :: Int,
new :: Int
} deriving Show
calcNews :: [Item] -> [Item]
calcNews [] = []
calcNews totalsOK#(t:ts) = t : zipWith f ts totalsOK
where
f this prev = this{ new = total this - total prev }
This gives us
> calcNews [Item 1 0 0, Item 2 2 0, Item 3 5 0, Item 4 10 0]
[Item {date = 1, total = 0, new = 0},Item {date = 2, total = 2, new = 2},
Item {date = 3, total = 5,new = 3},Item {date = 4, total = 10, new = 5}]
Of course zipWith f x y == map (\(a,b) -> f a b) $ zip x y, as we saw in your previous question, so zipWith is like a binary map.
Sometimes (though not here) we might need access to the previously calculated value as well, to calculate the next value. To arrange for that we can create the result by zipping the input with the shifted version of the result itself:
calcNews2 :: [Item] -> [Item]
calcNews2 [] = []
calcNews2 (t:totalsOK) = newsOK
where
newsOK = t : zipWith f totalsOK newsOK
f tot nw = tot{ new = total tot - total nw }
Brand new to haskell and I need to print out the data contained on a seperate row for each individual item
Unsure on how to
type ItemDescr = String
type ItemYear = Int
type ItemPrice = Int
type ItemSold = Int
type ItemSales = Int
type Item = (ItemRegion,ItemDescr,ItemYear,ItemPrice,ItemSold,ItemSales)
type ListItems = [Item]
rownumber x
| x == 1 = ("Scotland","Desktop",2017,900,25,22500)
| x == 2 = ("England","Laptop",2017,1100,75,82500)
| x == 3 = ("Wales","Printer",2017,120,15,1800)
| x == 4 = ("England","Printer",2017,120,60,7200)
| x == 5 = ("England","Desktop",2017,900,50,45000)
| x == 6 = ("Wales","Desktop",2017,900,20,18000)
| x == 7 = ("Scotland","Printer",2017,25,25,3000)
showall
--print??
So for example on each individual line
show
"Scotland","Desktop",2017,900,25,22500
followed by the next record
Tip 1:
Store the data like this
items = [("Scotland","Desktop",2017,900,25,22500),
("England","Laptop",2017,1100,75,82500),
("Wales","Printer",2017,120,15,1800),
("England","Printer",2017,120,60,7200),
("England","Desktop",2017,900,50,45000),
("Wales","Desktop",2017,900,20,18000),
("Scotland","Printer",2017,25,25,3000)]
Tip 2:
Implement this function
toString :: Item -> String
toString = undefined -- do this yourselves
Tip 3:
Try to combine the following functions
unlines, already in the Prelude
toString, you just wrote it
map, does not need any explanation
putStrLn, not even sure if this is a real function, but you need it anyway.
($), you can do without this one, but it will give you bonus points
I'm trying to code the fast Non Dominated Sorting algorithm (NDS) of Deb used in NSGA2 in immutable way using Scala.
But the problem seems more difficult than i think, so i simplify here the problem to make a MWE.
Imagine a population of Seq[A], and each A element is decoratedA with a list which contains pointers to other elements of the population Seq[A].
A function evalA(a:decoratedA) take the list of linkedA it contains, and decrement value of each.
Next i take a subset list decoratedAPopulation of population A, and call evalA on each. I have a problem, because between each iteration on element on this subset list decoratedAPopulation, i need to update my population of A with the new decoratedA and the new updated linkedA it contain ...
More problematic, each element of population need an update of 'linkedA' to replace the linked element if it change ...
Hum as you can see, it seem complicated to maintain all linked list synchronized in this way. I propose another solution bottom, which probably need recursion to return after each EvalA a new Population with element replaced.
How can i do that correctly in an immutable way ?
It's easy to code in a mutable way, but i don't find a good way to do this in an immutable way, do you have a path or an idea to do that ?
object test extends App{
case class A(value:Int) {def decrement()= new A(value - 1)}
case class decoratedA(oneAdecorated:A, listOfLinkedA:Seq[A])
// We start algorithm loop with A element with value = 0
val population = Seq(new A(0), new A(0), new A(8), new A(1))
val decoratedApopulation = Seq(new decoratedA(population(1),Seq(population(2),population(3))),
new decoratedA(population(2),Seq(population(1),population(3))))
def evalA(a:decoratedA) = {
val newListOfLinked = a.listOfLinkedA.map{e => e.decrement()
new decoratedA(a.oneAdecorated,newListOfLinked)}
}
def run()= {
//decoratedApopulation.map{
// ?
//}
}
}
Update 1:
About the input / output of the initial algorithm.
The first part of Deb algorithm (Step 1 to Step 3) analyse a list of Individual, and compute for each A : (a) domination count, the number of A which dominate me (the value attribute of A) (b) a list of A i dominate (listOfLinkedA).
So it return a Population of decoratedA totally initialized, and for the entry of Step 4 (my problem) i take the first non dominated front, cf. the subset of elements of decoratedA with A value = 0.
My problem start here, with a list of decoratedA with A value = 0; and i search the next front into this list by computing each listOfLinkedA of each of this A
At each iteration between step 4 to step 6, i need to compute a new B subset list of decoratedA with A value = 0. For each , i decrementing first the domination count attribute of each element into listOfLinkedA, then i filter to get the element equal to 0. A the end of step 6, B is saved to a list List[Seq[DecoratedA]], then i restart to step 4 with B, and compute a new C, etc.
Something like that in my code, i call explore() for each element of B, with Q equal at the end to new subset of decoratedA with value (fitness here) = 0 :
case class PopulationElement(popElement:Seq[Double]){
implicit def poptodouble():Seq[Double] = {
popElement
}
}
class SolutionElement(values: PopulationElement, fitness:Double, dominates: Seq[SolutionElement]) {
def decrement()= if (fitness == 0) this else new SolutionElement(values,fitness - 1, dominates)
def explore(Q:Seq[SolutionElement]):(SolutionElement, Seq[SolutionElement])={
// return all dominates elements with fitness - 1
val newSolutionSet = dominates.map{_.decrement()}
val filteredSolution:Seq[SolutionElement] = newSolutionSet.filter{s => s.fitness == 0.0}.diff{Q}
filteredSolution
}
}
A the end of algorithm, i have a final list of seq of decoratedA List[Seq[DecoratedA]] which contain all my fronts computed.
Update 2
A sample of value extracted from this example.
I take only the pareto front (red) and the {f,h,l} next front with dominated count = 1.
case class p(x: Int, y: Int)
val a = A(p(3.5, 1.0),0)
val b = A(p(3.0, 1.5),0)
val c = A(p(2.0, 2.0),0)
val d = A(p(1.0, 3.0),0)
val e = A(p(0.5, 4.0),0)
val f = A(p(0.5, 4.5),1)
val h = A(p(1.5, 3.5),1)
val l = A(p(4.5, 1.0),1)
case class A(XY:p, value:Int) {def decrement()= new A(XY, value - 1)}
case class ARoot(node:A, children:Seq[A])
val population = Seq(
ARoot(a,Seq(f,h,l),
ARoot(b,Seq(f,h,l)),
ARoot(c,Seq(f,h,l)),
ARoot(d,Seq(f,h,l)),
ARoot(e,Seq(f,h,l)),
ARoot(f,Nil),
ARoot(h,Nil),
ARoot(l,Nil))
Algorithm return List(List(a,b,c,d,e), List(f,h,l))
Update 3
After 2 hour, and some pattern matching problems (Ahum...) i'm comming back with complete example which compute automaticaly the dominated counter, and the children of each ARoot.
But i have the same problem, my children list computation is not totally correct, because each element A is possibly a shared member of another ARoot children list, so i need to think about your answer to modify it :/ At this time i only compute children list of Seq[p], and i need list of seq[A]
case class p(x: Double, y: Double){
def toSeq():Seq[Double] = Seq(x,y)
}
case class A(XY:p, dominatedCounter:Int) {def decrement()= new A(XY, dominatedCounter - 1)}
case class ARoot(node:A, children:Seq[A])
case class ARootRaw(node:A, children:Seq[p])
object test_stackoverflow extends App {
val a = new p(3.5, 1.0)
val b = new p(3.0, 1.5)
val c = new p(2.0, 2.0)
val d = new p(1.0, 3.0)
val e = new p(0.5, 4.0)
val f = new p(0.5, 4.5)
val g = new p(1.5, 4.5)
val h = new p(1.5, 3.5)
val i = new p(2.0, 3.5)
val j = new p(2.5, 3.0)
val k = new p(3.5, 2.0)
val l = new p(4.5, 1.0)
val m = new p(4.5, 2.5)
val n = new p(4.0, 4.0)
val o = new p(3.0, 4.0)
val p = new p(5.0, 4.5)
def isStriclyDominated(p1: p, p2: p): Boolean = {
(p1.toSeq zip p2.toSeq).exists { case (g1, g2) => g1 < g2 }
}
def sortedByRank(population: Seq[p]) = {
def paretoRanking(values: Set[p]) = {
//comment from #dk14: I suppose order of values isn't matter here, otherwise use SortedSet
values.map { v1 =>
val t = (values - v1).filter(isStriclyDominated(v1, _)).toSeq
val a = new A(v1, values.size - t.size - 1)
val root = new ARootRaw(a, t)
println("Root value ", root)
root
}
}
val listOfARootRaw = paretoRanking(population.toSet)
//From #dk14: Here is convertion from Seq[p] to Seq[A]
val dominations: Map[p, Int] = listOfARootRaw.map(a => a.node.XY -> a.node.dominatedCounter) //From #dk14: It's a map with dominatedCounter for each point
val listOfARoot = listOfARootRaw.map(raw => ARoot(raw.node, raw.children.map(p => A(p, dominations.getOrElse(p, 0)))))
listOfARoot.groupBy(_.node.dominatedCounter)
}
//Get the first front, a subset of ARoot, and start the step 4
println(sortedByRank(Seq(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p)).head)
}
Talking about your problem with distinguishing fronts (after update 2):
val (left,right) = population.partition(_.node.value == 0)
List(left, right.map(_.copy(node = node.copy(value = node.value - 1))))
No need for mutating anything here. copy will copy everything but fields you specified with new values. Talking about the code, the new copy will be linked to the same list of children, but new value = value - 1.
P.S. I have a feeling you may actually want to do something like this:
case class A(id: String, level: Int)
val a = A("a", 1)
val b = A("b", 2)
val c = A("c", 2)
val d = A("d", 3)
clusterize(List(a,b,c,d)) === List(List(a), List(b,c), List(d))
It's simple to implement:
def clusterize(list: List[A]) =
list.groupBy(_.level).toList.sortBy(_._1).map(_._2)
Test:
scala> clusterize(List(A("a", 1), A("b", 2), A("c", 2), A("d", 3)))
res2: List[List[A]] = List(List(A(a,1)), List(A(b,2), A(c,2)), List(A(d,3)))
P.S.2. Please consider better naming conventions, like here.
Talking about "mutating" elements in some complex structure:
The idea of "immutable mutating" some shared (between parts of a structure) value is to separate your "mutation" from the structure. Or simply saying, divide and conquerror:
calculate changes in advance
apply them
The code:
case class A(v: Int)
case class AA(a: A, seq: Seq[A]) //decoratedA
def update(input: Seq[AA]) = {
//shows how to decrement each value wherever it is:
val stats = input.map(_.a).groupBy(identity).mapValues(_.size) //domination count for each A
def upd(a: A) = A(a.v - stats.getOrElse(a, 0)) //apply decrement
input.map(aa => aa.copy(aa = aa.seq.map(upd))) //traverse and "update" original structure
}
So, I've introduced new Map[A, Int] structure, that shows how to modify the original one. This approach is based on highly simplified version of Applicative Functor concept. In general case, it should be Map[A, A => A] or even Map[K, A => B] or even Map[K, Zipper[A] => B] as applicative functor (input <*> map). *Zipper (see 1, 2) actually could give you information about current element's context.
Notes:
I assumed that As with same value are same; that's default behaviour for case classess, otherwise you need to provide some additional id's (or redefine hashCode/equals).
If you need more levels - like AA(AA(AA(...)))) - just make stats and upd recursive, if dеcrement's weight depends on nesting level - just add nesting level as parameter to your recursive function.
If decrement depends on parent node (like decrement only A(3)'s, which belongs to A(3)) - add parent node(s) as part of stats's key and analise it during upd.
If there is some dependency between stats calculation (how much to decrement) of let's say input(1) from input(0) - you should use foldLeft with partial stats as accumulator: val stats = input.foldLeft(Map[A, Int]())((partialStats, elem) => partialStats ++ analize(partialStats, elem))
Btw, it takes O(N) here (linear memory and cpu usage)
Example:
scala> val population = Seq(A(3), A(6), A(8), A(3))
population: Seq[A] = List(A(3), A(6), A(8), A(3))
scala> val input = Seq(AA(population(1),Seq(population(2),population(3))), AA(population(2),Seq(population(1),population(3))))
input: Seq[AA] = List(AA(A(6),List(A(8), A(3))), AA(A(8),List(A(6), A(3))))
scala> update(input)
res34: Seq[AA] = List(AA(A(5),List(A(7), A(3))), AA(A(7),List(A(5), A(3))))
I have a list of items and for each item I am computing a value. Computing this value is a bit computationally intensive so I want to minimise it as much as possible.
The algorithm I need to implement is this:
I have a value X
For each item
a. compute the value for it, if it is < 0 ignore it completely
b. if (value > 0) && (value < X)
return pair (item, value)
Return all (item, value) pairs in a List (that have the value > 0), ideally sorted by value
To make it a bit clearer, step 3 only happens if none of the items have a value less than X. In step 2, when we encounter the first item that is less than X we should not compute the rest and just return that item (we can obviously return it in a Set() by itself to match the return type).
The code I have at the moment is as follows:
val itemValMap = items.foldLeft(Map[Item, Int)]()) {
(map : Map[Item, Int], key : Item) =>
val value = computeValue(item)
if ( value >= 0 ) //we filter out negative ones
map + (key -> value)
else
map
}
val bestItem = itemValMap.minBy(_._2)
if (bestItem._2 < bestX)
{
List(bestItem)
}
else
{
itemValMap.toList.sortBy(_._2)
}
However, what this code is doing is computing all the values in the list and choosing the best one, rather than stopping as a 'better' one is found. I suspect I have to use Streams in some way to achieve this?
OK, I'm not sure how your whole setup looks like, but I tried to prepare a minimal example that would mirror your situation.
Here it is then:
object StreamTest {
case class Item(value : Int)
def createItems() = List(Item(0),Item(3),Item(30),Item(8),Item(8),Item(4),Item(54),Item(-1),Item(23),Item(131))
def computeValue(i : Item) = { Thread.sleep(3000); i.value * 2 - 2 }
def process(minValue : Int)(items : Seq[Item]) = {
val stream = Stream(items: _*).map(item => item -> computeValue(item)).filter(tuple => tuple._2 >= 0)
stream.find(tuple => tuple._2 < minValue).map(List(_)).getOrElse(stream.sortBy(_._2).toList)
}
}
Each calculation takes 3 seconds. Now let's see how it works:
val items = StreamTest.createItems()
val result = StreamTest.process(2)(items)
result.foreach(r => println("Original: " + r._1 + " , calculated: " + r._2))
Gives:
[info] Running Main
Original: Item(3) , calculated: 4
Original: Item(4) , calculated: 6
Original: Item(8) , calculated: 14
Original: Item(8) , calculated: 14
Original: Item(23) , calculated: 44
Original: Item(30) , calculated: 58
Original: Item(54) , calculated: 106
Original: Item(131) , calculated: 260
[success] Total time: 31 s, completed 2013-11-21 15:57:54
Since there's no value smaller than 2, we got a list ordered by the calculated value. Notice that two pairs are missing, because calculated values are smaller than 0 and got filtered out.
OK, now let's try with a different minimum cut-off point:
val result = StreamTest.process(5)(items)
Which gives:
[info] Running Main
Original: Item(3) , calculated: 4
[success] Total time: 7 s, completed 2013-11-21 15:55:20
Good, it returned a list with only one item, the first value (second item in the original list) that was smaller than 'minimal' value and was not smaller than 0.
I hope that the example above is easily adaptable to your needs...
A simple way to avoid the computation of unneeded values is to make your collection lazy by using the view method:
val weigthedItems = items.view.map{ i => i -> computeValue(i) }.filter(_._2 >= 0 )
weigthedItems.find(_._2 < X).map(List(_)).getOrElse(weigthedItems.sortBy(_._2))
By example here is a test in the REPL:
scala> :paste
// Entering paste mode (ctrl-D to finish)
type Item = String
def computeValue( item: Item ): Int = {
println("Computing " + item)
item.toInt
}
val items = List[Item]("13", "1", "5", "-7", "12", "3", "-1", "15")
val X = 10
val weigthedItems = items.view.map{ i => i -> computeValue(i) }.filter(_._2 >= 0 )
weigthedItems.find(_._2 < X).map(List(_)).getOrElse(weigthedItems.sortBy(_._2))
// Exiting paste mode, now interpreting.
Computing 13
Computing 1
defined type alias Item
computeValue: (item: Item)Int
items: List[String] = List(13, 1, 5, -7, 12, 3, -1, 15)
X: Int = 10
weigthedItems: scala.collection.SeqView[(String, Int),Seq[_]] = SeqViewM(...)
res27: Seq[(String, Int)] = List((1,1))
As you can see computeValue was only called up to the first value < X (that is, up to 1)
i trying to assign a row number and a Set-number for List, but Set Number containing wrong number of rows in one set.
var objx = new List<x>();
var i = 0;
var r = 1;
objY.ForEach(x => objx .Add(new x
{
RowNumber = ++i,
DatabaseID= x.QuestionID,
SetID= i == 5 ? r++ : i % 5 == 0 ? r += 1 : r
}));
for Above code like objY Contains 23 rows, and i want to break 23 rows in 5-5 set.
so above code will give the sequence like[Consider only RowNumber]
[1 2 3 4 5][6 7 8 9][ 10 11 12 13 14 ].......
its a valid as by the logic
and if i change the logic for Setid as
SetID= i % 5 == 0 ? r += 1 : r
Result Will come Like
[1 2 3 4 ][5 6 7 8 9][10 11 12 13 14].
Again correct output of code
but expected for set of 5.
[1 2 3 4 5][ 6 7 8 9 10].........
What i missing.............
i should have taken my Maths class very Serious.
I think you want something like this:
var objX = objY.Select((x, i) => new { ObjX = x, Index = i })
.GroupBy(x => x.Index / 5)
.Select((g, i) =>
g.Select(x => new objx
{
RowNumber = x.Index + 1
DatabaseID = x.ObjX.QuestionID,
SetID = i + 1
}).ToList())
.ToList();
Note that i'm grouping by x.Index / 5 to ensure that every group has 5 items.
Here's a demo.
Update
it will be very helpful,if you can explain your logic
Where should i start? I'm using Linq methods to select and group the original list to create a new List<List<ObjX>> where every inner list has maximum 5 elements(less in the last if the total-count is not dividable by 5).
Enumerable.Select enables to project something from the input sequence to create something new. This method is comparable to a variable in a loop. In this case i project an anonymous type with the original object and the index of it in the list(Select has an overload that incorporates the index). I create this anonymous type to simply the query and because i need the index later in the GroupBy``.
Enumerable.GroupBy enables to group the elements in a sequence by a specified key. This key can be anything which is derivable from the element. Here i'm using the index two build groups of a maximum size of 5:
.GroupBy(x => x.Index / 5)
That works because integer division in C# (or C) results always in an int, where the remainder is truncated(unlike VB.NET btw), so 3/4 results in 0. You can use this fact to build groups of the specified size.
Then i use Select on the groups to create the inner lists, again by using the index-overload to be able to set the SetId of the group:
.Select((g, i) =>
g.Select(x => new objx
{
RowNumber = x.Index + 1
DatabaseID = x.ObjX.QuestionID,
SetID = i + 1
}).ToList())
The last step is using ToList on the IEnumerable<List<ObjX>> to create the final List<List<ObX>>. That also "materializes" the query. Have a look at deferred execution and especially Jon Skeets blog to learn more.