I'm looking for an idiomatic constraint satisfaction solver that can maximize or minimize a goal function, rather than producing a list of matching solutions.
To be precise, I'm more interested in minimization (e.g., of gasoline consumption on a route that must also satisfy other constraints), but the question is:
I'm currently looking at core.logic and am under the impression that the module will not do either max or min. As far as I understand, that functionality would usually be specifically denoted as CLP. Core.logic does mention CLP(FD) (https://github.com/clojure/core.logic/wiki/Features), but looking at the description, I keep having serious doubts.
Could someone please comment on this - as my going through the entire Reasoned Schemer book would be too much, I guess?
The original question is not very specific about what kind optimization problem it wants to solve. Usually, the type of optimization problem will determine which solver is adequate.
However, many optimization problems that arise in engineering consist of objective functions with linear and quadratic terms, linear constraints, real and integer variables. The ojAlgo library is great for those problems. For example, if we want to minimize f(x, y) = x + 2y with x and y being integers, and three linear constraints x >= 2, y >= 3 and x + y >= 14.2, here is how to solve this problem using ojAlgo in Clojure:
(ns playground.ojalgo
(:import [org.ojalgo.optimisation ExpressionsBasedModel]))
(defn demo []
(let [model (ExpressionsBasedModel.)
;; Declare variables
;; X has lower limit 2
x (-> (.addVariable model)
(.lower 2)
(.integer true))
;; Y has lower limit 3
y (-> (.addVariable model)
(.lower 3)
(.integer true))]
;; Objective function: Minimize x + 2y
(-> (.addExpression model)
(.set x 1.0)
(.set y 2.0)
(.weight 1.0)) ;; <-- Terms in the objective function
;; need a weight.
;; Linear constraint: x + y >= 14.2
(-> (.addExpression model)
(.set x 1.0)
(.set y 1.0)
(.lower 14.2))
(let [result (.minimise model)]
(println "Result" result)
(println "Objective function value: " (.getValue result))
(println "Optimal X value: " (.getValue x))
(println "Optimal Y value: " (.getValue y)))))
Which will display
Result #object[org.ojalgo.optimisation.Optimisation$Result 0xb755873 OPTIMAL 18.0 # { 12, 3 }]
Objective function value: 18.0
Optimal X value: 12M
Optimal Y value: 3M
Chances are your problem (vaguely worded as "gasoline consumption on a route that must also satisfy other constraints") can be expressed in a format that this solver can handle. But with no more details, it is hard to provide a more accurate answer than this.
To use ojAlgo, you need to add the [org.ojalgo/ojalgo "48.1.0"] dependency to your Leiningen project.clj.
Related
I am working through the Armstrong Numbers exercise on Exercism's Clojure track. An armstrong number is a number equal to the sum of its digits raised to the power of the number of digits. 153 is an Armstrong number, because: 153 = 1^3 + 5^3 + 3^3 = 1 + 125 + 27 = 153. 154 is not an Armstrong number, because: 154 != 1^3 + 5^3 + 4^3 = 1 + 125 + 64 = 190.
The test file for this exercise will call the armstrong? function, pass in a number, and expects true if the number is an Armstrong number. I have already solved the problem with this code:
(ns armstrong-numbers)
(defn pow [a b]
(reduce * 1 (repeat b a)))
(defn armstrong? [num]
(->> num
(iterate #(quot % 10))
(take-while pos?)
(map #(mod % 10))
((fn [sq]
(map #(pow % (count sq))
sq)))
(apply +)
(= num)))
but now I am trying to refactor the code. This is what I would like the code to look like:
(ns armstrong-numbers
(:require [swiss.arrows :refer :all]))
(defn pow [a b]
(reduce * 1 (repeat b a)))
(defn armstrong? [num]
(-<>> num
(iterate #(quot % 10))
(take-while pos?)
(map #(mod % 10))
(map #(pow % (count <>))
<>)
(apply +)
(= num)))
A link to the package required above: https://github.com/rplevy/swiss-arrows.
In the first code section, I create an implicit function within the thread-last macro because the sequence returned from the map form is needed in two different places in the second map form. That implicit function works just fine, but I just wanted to make the code sleeker. But when I test the second code block, I get the following error: java.lang.RuntimeException: Unable to resolve symbol: <> in this context.
I get this error whether I use #(), partial , or fn inside the second map form. I have figured out that because all of the preceding are macros (or a special form in fns case), they cannot resolve <> because it's only meaningful to the -<>> macro, which is called at a later point in macroexpansion. But why do #(), partial, and fn attempt to resolve that character at all? As far as I can see, they have no reason to know what the symbol is, or what it's purpose is. All they need to do is return that symbol rearranged into the proper s-expressions. So why does clojure attempt to resolve this (<>) symbol?
The <> symbol is only valid in the topmost level of a clause (plus literals for set, map, vector directly therein). -<> and -<>> do not establish bindings (as in let) for <>, but do code insertion at macro expansion time.
This code insertion is done only at toplevel, because making it work deeper is not only much more complex (you need a so-called code walker), but also raises interesting questions regarding the nesting of arrow forms. This complexity is likely not worth it, for such a simple effect.
If you want a real binding, you can use as-> (from clojure.core).
The documentation for -<>> is quite clear that it doesn't behave the way you wish it did:
"the 'diamond spear': top-level insertion of x in place of single
positional '<>' symbol within the threaded form if present, otherwise
mostly behave as the thread-last macro. Also works with hash literals
and vectors."
It performs replacement:
Of a single symbol
At the top level of the threaded form
Your example wishing to use it for multiple symbols, nested within subforms, will thus not work.
You have made a mistake leaving off the <> symbol in most of your forms in the failing case. Here is a working version (using the similar it-> macro in place of the Swiss Arrows). I also cleaned up the pow a bit.
(defn pow-int [a b] (Math/round (Math/pow a b)))
(defn armstrong? [num]
(it-> num
(iterate #(quot % 10) it)
(take-while pos? it)
(map #(mod % 10) it)
(map #(pow-int % (count it)) it)
(apply + it)
(= it num)))
(armstrong? 153) => true
(armstrong? 154) => false
You can find the docs on it-> here.
If you leave off the (collection) arg to a function like map, it returns a transducer; from the docs:
(map f)(map f coll)(map f c1 c2)(map f c1 c2 c3)(map f c1 c2 c3 & colls)
Returns a lazy sequence consisting of the result of applying f to
the set of first items of each coll, followed by applying f to the
set of second items in each coll, until any one of the colls is
exhausted. Any remaining items in other colls are ignored. Function
f should accept number-of-colls arguments. Returns a transducer when
no collection is provided.
And, always refer to the Clojure CheatSheet!
I have 2 csv files with around 22K records in file F1, and 50K records in file F2, both containing company name, and address information. I need to do a fuzzy-match on name, address, and phone. Each record in F1 needs to be fuzzy-matched against each record in F2. I have made a third file R3 which is a csv containing the rules for fuzzy-matching which column from F1 to which column with F2, with a fuzzy-tolerance-level. I am trying to do it with for loop, this way -
(for [j f1-rows
h f2-rows
r r3-rows
:while (match-row j h r)]
(merge j h))
(defn match-row [j h rules]
(every?
identity
(map (fn [rule]
(<= (fuzzy/jaccard
((keyword (first rule)) j)
((keyword (second rule)) h))
((nth rule 2))))
rules)))
f1-rows and f2-rows are collections of map. Rules is a collection of sequences containing column name from f1, f2, and the tolerance level. The code is running and functioning as expected. But my problem is, it is taking around 2 hours to execute. I read up on how transducers help improving performance by eliminating intermediate chunks, but I am not able to visualize how I would apply that in my case. Any thoughts on how I can make this better/faster ?
:while vs :when
Your use of :while in this case doesn't seem to agree with your stated problem. Your for-expression will keep going while match-row is true, and stop altogether at the first false result. :when will iterate through all the combinations and include only ones where match-row is true in the resulting lazy-seq. The difference is explained here.
For example:
(for [i (range 10)
j (range 10)
:while (= i j)]
[i j]) ;=> ([0 0])
(for [i (range 10)
j (range 10)
:when (= i j)]
[i j]) ;=> ([0 0] [1 1] [2 2] [3 3] [4 4] [5 5] [6 6] [7 7] [8 8] [9 9])
It's really strange that your code kept running for 2 hours, because that means that during those two hours every invocation of (match-row j h r) returned true, and only the last one returned false. I would check the result again to see if it really makes sense.
How long should it take?
Let's first do some back-of-the-napkin math. If you want to compare every one of 22k records with every one of 55k records, you're gonna be doing 22k * 55k comparisons, there's no way around that.
22k * 55k = 1,210,000,000
That's a big number!
What's the cost of a comparison?
From a half-a-minute glance at wikipedia, jaccard is something about sets.
The following will do to get a ballpark estimate of the cost, though it's probably very much on the low end.
(time (clojure.set/difference (set "foo") (set "bar")))
That takes about a tenth of a millisecond on my computer.
(/ (* 22e3 55e3) ;; Number of comparisons.
10 ; milliseconds
1000 ;seconds
60 ;minutes
60) ;hours
;=> 33.611111111111114
That's 33 and a half hours. And that's with a low-end estimate of the individual cost, and not counting the fact that you want to compare name, address and phone on each one (?). So that' 33 hours if every comparison fails at the first row, and 99 hours if they all get to the last row.
Before any micro-optimizations, you need to work on the algorithm, by finding some clever way of not needing to do more than a billion comparisons. If you want help on that, you need to at least supply some sample data.
Nitpicks
The indentation of the anon fn inside match-row is confusing. I'd use an editor that indents automatically, and stick to that 99% of the time, because lisp programmers read nesting by the indentation, like python. There are some slight differences between editors/autoindenters, but they're all consistent with the nesting.
(defn match-row [j h rules]
(every?
identity
(map (fn [rule]
(<= (fuzzy/jaccard
((keyword (first rule)) j)
((keyword (second rule)) h))
((nth rule 2))))
rules)))
Also,match-row needs to be defined before it is used (which it probably is in your actual code, seeing as it compiles).
22k x 50k is over 1 billion combinations. Multiply by 3 fuzzy rules & you're at 3 billion calculations. Most of which are wasted.
The only way to speed it up is to do some pre-sorting or other pre-trimming of all the combinations. For example, only do the fuzzy calc if zipcodes are identical. If you waste time trying to match people from N.Y. and Florida, you are throwing away 99% or more of your work
I'm trying to make music with the overtone library of clojure. To produce interesting sounds, additive synthesis is useful, that means that I take sinus oscillators of several frequencies and simply add them. In overtone, to create a synthesizer, which realizes that, I can write:
(definst myinst [freq 100]
(* 0.2 (reduce + (map sin-osc [101.0 100 99.0]))))
To be a bit more reusable, I write a function, which takes a frequency and returns a list with all these frequencies, you can see in the code snippet:
(defn split-freq [freq]
(apply vector (map #(* freq %) [1.01 1 0.99])))
When executing (split-freq 100), the REPL gives me the following:
[101.0 100 99.0]
which is exactly the same as the input, I provided the map function above. In fact, I copied the result. Now, I try this:
(definst myinst [freq 100]
(* 0.2 (reduce + (map sin-osc (split-freq freq)))))
Unfortunately, the REPL tells me, that I'm doing wrong:
CompilerException java.lang.ClassCastException:
overtone.sc.machinery.ugen.sc_ugen.ControlProxy cannot be cast to java.lang.Number,
compiling:(form-init1807789563491679853.clj:1)
But this code works fine:
(definst myinst [freq 100]
(* 0.2 (reduce + (map sin-osc (apply vector (map #(* freq %) [1.01 1 0.99]))))))
although, I simply put in the function definition.
I think I have a principal lack of understanding. I thought, that if one of those version work, the others must work, too.
My questions are:
Why does my 'improved' version does not work?
Why does the first snippet work?
How can I circumvent this behavior?
Clojure 1.3.0
Overtone 0.8
(use 'overtone.core)
ups, you are right, i haven't understand the noisesmith comment. It's true that you can't use this value "freq" in your function directly because this value is not bound at the time that your function need it (before the macro execution)
So here is a workaround to this problem, also using the let form, it is not so elegant as to make your own macro but works
(defn myinst [inst-name]
(let [the-freq 100]
(definst inst-name [freq the-freq] (* 0.2 (reduce + (map sin-osc (split-freq the-freq)))))))
and you can execute as a normal function
((myinst "my-instrument"))
I hope that helps you
Juan
Previous comment, now not valid response
maybe you are making a mistake with your argument value declaration in your improved version
That's your code
(definst myinst [freq 100]
(* 0.2 (reduce + (map sin-osc (split-freq freq)))))
And, as in clojure you can't assign default value to argument function, your improved version would work if, by example, you use a let form to assign the default value
(definst myinst []
(let [freq 100]
(* 0.2 (reduce + (map sin-osc (split-freq freq))))))
So I've been trying to teach myself clojure and core.logic and decided to try and implement a solution to the sum product problem (http://en.wikipedia.org/wiki/Impossible_Puzzle).
So far I've implemented the first two steps.
(ns sum-product-problem.core
(:refer-clojure :exclude [==])
(:use clojure.core.logic)
(:require [clojure.core.logic.fd :as fd]))
(def non-unique-factors
(tabled [product]
(fresh [p q x y]
(fd/in p q x y (fd/interval 2 99))
(fd/>= q p)
(fd/* p q product)
(fd/>= y x)
(fd/* x y product)
(fd/distinct [p x]))))
(defn inscruitable-summands [sum x]
(fresh [y product]
(conde
[(fd/> (* 2 x) sum)]
[(fd/in y (fd/interval 2 99))
(fd/+ x y sum)
(fd/* x y product)
(non-unique-factors product)
(inscruitable-summands sum (+ x 1))])))
(defn solution []
(run* [q]
(fd/in q (fd/interval 17 17))
(inscruitable-summands q 2)))
This seems to give the correct answer when the domain is limited to a single number in (solution) but if I extend that domain it immediately stops working.
The results when searching over a single member domain also contains the answer multiple times. It seems to be caused by each recursion of inscruitable-summands but I'm not really sure why.
Finally, I was hoping someone could take a quick look at my non-unique-factors function. It feels like a bit of a bodge and I was wondering if anyone could suggest a better alternative?
Thanks for the help,
Dean
I don't believe this puzzle is solvable with CLP(FD) facilities alone (though someone better versed in Prolog may be able to answer this one), solutions I've seen in Prolog require some form of subquery facility like like setof.
How do I write modulus syntax in programming language clojure?
For example the symbols money %= money_value.
There are two functions in Clojure you might want to try: mod and rem. They work the same for positive numbers, but differently for negative numbers. Here's an example from the docs:
(mod -10 3) ; => 2
(rem -10 3) ; => -1
Update:
If you really want to convert your code to Clojure, you need to realize that an idiomatic Clojure solution probably won't look anything like your JavaScript solution. Here's a nice solution that I think does roughly what you want:
(defn change [amount]
(zipmap [:quarters :dimes :nickels :pennies]
(reduce (fn [acc x]
(let [amt (peek acc)]
(conj (pop acc)
(quot amt x)
(rem amt x))))
[amount] [25 10 5])))
(change 142)
; => {:pennies 2, :nickels 1, :dimes 1, :quarters 5}
You can look up any of the functions you don't recognize on ClojureDocs. If you just don't understand the style, then you probably need some more experience programming with higher-order functions. I think 4Clojure is a good place to start.