After solving my error with values-list and being able to run my program until the end, I found that my diagonal check seems to have a logic error. My input is as follows:
(THREAT? '(1 3) '((1 0)(2 4)(3 0)(4 0)(5 0)(6 0)(7 0)(8 0)))
The first argument being a board space that we are testing is ok or not to place a queen and the second argument is the state of the board, y values 1-8 determine the column positions of a piece and a 0 value would indicate that x value row would hold no piece. My code is as follows:
(defun diagonal(point1 point2)
(= (abs (- ( car point1 ) ( car point2 )))
(abs (- ( cadr point1 ) ( cadr point2 ))))
)
(defun THREAT?(x y)
; Checks threat on the vertical
(when (not (eq (values-list (cdr (nth (- (car x) 1 ) y )) ) '0 ) )
(return-from THREAT? t)
)
(loop for i from 0 to (list-length y)
; Checks threat on the horizontal
when (eq (values-list ( cdr x )) (values-list (cdr (nth i y))) )
do (return-from THREAT? t)
; With the help of the diagonal function checks along the diagonal
when (diagonal x (nth i y) )
do (return-from THREAT? t)
)
)
If my understanding is correct my program should loop through every single element of y. It will pass the x and the current y pair to the diagonal function. The diagonal function will minus the two and absolute value them and check if they are equal (if they are diagonal then they should be ex. (1 2) and (2 3) are diagonal and therefore |1 - 2| = 1 and |2 - 3| = 1). The diagonal function should return true if these numbers are equivalent. The corresponding when statement should only activate when it receives a true from the diagonal function and yet it seems to always return true, even when I give the program a completely blank board. How do I fix diagonal to correctly determine a threat on the board? Any and all help is greatly appreciated!
I have rewritten your code to better Lisp style.
much better naming.
procedures with useful names make comments redundant
individual procedures are better testable
got rid of the VALUES-LIST nonsense
get rid of all CAR, CDR, CADR. Use FIRST and SECOND.
introduced accessors for x and y components of a point
got rid of the strange control flow with RETURN-FROM, replaced it with a simple OR
actually directly iterate over a list, instead of using NTH all the time
EQ is not for comparing number equality, use = instead
don't place parentheses alone on a line.
indent and format the code correctly
don't put spaces between parentheses
put a space between an atom and an open parenthesis
Code:
(defun get-x (point)
(first point))
(defun get-y (point)
(second point))
(defun diagonal? (point1 point2)
(= (abs (- (get-x point1) (get-x point2)))
(abs (- (get-y point1) (get-y point2)))))
(defun vertical? (point)
(not (zerop (get-y point))))
(defun horizontal? (point1 point2)
(= (get-y point1)
(get-y point2)))
(defun threat? (point list-of-columns)
(or (vertical? (nth (1- (get-x point)) list-of-columns))
(loop for point2 in list-of-columns
when (or (horizontal? point point2)
(diagonal? point point2))
return t)))
Example
Now we can trace the three threat predicates:
? (trace vertical? diagonal? horizontal?)
NIL
Now you can call your example:
? (threat? '(1 3) '((1 0) (2 4) (3 0) (4 0) (5 0) (6 0) (7 0) (8 0)))
0> Calling (VERTICAL? (1 0))
<0 VERTICAL? returned NIL
0> Calling (HORIZONTAL? (1 3) (1 0))
<0 HORIZONTAL? returned NIL
0> Calling (DIAGONAL? (1 3) (1 0))
<0 DIAGONAL? returned NIL
0> Calling (HORIZONTAL? (1 3) (2 4))
<0 HORIZONTAL? returned NIL
0> Calling (DIAGONAL? (1 3) (2 4))
<0 DIAGONAL? returned T
T
This should help, so that you can better debug your code... Look at the trace output.
A version which does not use empty column descriptions
(defun get-x (point)
(first point))
(defun get-y (point)
(second point))
(defun diagonal? (point1 point2)
(= (abs (- (get-x point1) (get-x point2)))
(abs (- (get-y point1) (get-y point2)))))
(defun vertical? (point list-of-columns)
(let ((point2 (find (get-x point) list-of-columns :key #'get-x)))
(and point2 (not (zerop (get-y point2))))))
(defun horizontal? (point1 point2)
(= (get-y point1)
(get-y point2)))
(defun threat? (point list-of-columns)
(or (vertical? point list-of-columns)
(loop for point2 in list-of-columns
when (or (horizontal? point point2)
(diagonal? point point2))
return t)))
(defun print-board (board)
(format t "~%+-+-+-+-+-+-+-+-+")
(dotimes (y 8)
(terpri)
(dotimes (x 8)
(format t "|~a" (if (member (list x y) board :test #'equal) "x" " ")))
(format t "|~%+-+-+-+-+-+-+-+-+")))
Example:
CL-USER 138 > (threat? '(1 2) '((2 4)))
NIL
CL-USER 139 > (print-board '((1 2) (2 4)))
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| |x| | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | |x| | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
NIL
Another example:
CL-USER 140 > (threat? '(1 2) '((2 4) (4 5)))
T
CL-USER 141 > (print-board '((1 2) (2 4) (4 5)))
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| |x| | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | |x| | | | | |
+-+-+-+-+-+-+-+-+
| | | | |x| | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
| | | | | | | | |
+-+-+-+-+-+-+-+-+
NIL
Related
Config
Tested under clojure 1.10.3 and openjdk 17.0.1
Problem
Below is the slightly revised version of the memoized Fibonacci, and the general techniques refer to
wiki memoization.
(def fib
(memoize #(condp = %
0 (bigdec 0)
1 1
(+ (fib (dec %)) (fib (- % 2))))))
(fib 225) ; line 7
I had thought that the above memoized Fibonacci in FP like Clojure would act in spirit as the equivalent to the imperative DP e.g. in Python below,
def fib(n):
dp = [0, 1] + [0] * n
for i in range(2, n + 1):
dp[i] = dp[i - 1] + dp[i - 2]
return dp[n]
Question 1
Why did I actually get the following error when Fibonacci number was raised to 225 in my case?
Syntax error (StackOverflowError) compiling at 7:1
I also tried on the drop-in replacement memo on core.memoize and got the same error when Fibonacci number was raised to 110.
Tracing
Below I added tracing the recursive outlooks of the naive Fibonacci vs. the memoized Fibonacci,
(ns fib.core)
(defn naive-fib [n]
(condp = n
0 (bigdec 0)
1 1
(+ (naive-fib (dec n)) (naive-fib (- n 2)))))
(def memo-fib
(memoize #(condp = %
0 (bigdec 0)
1 1
(+ (memo-fib (dec %)) (memo-fib (- % 2))))))
(in-ns 'user)
(require '[clojure.tools.trace :refer [trace-ns]])
(trace-ns 'fib.core)
(fib.core/naive-fib 5)
(println)
(fib.core/memo-fib 5)
Overlapping sub-problems from the naive Fibonacci were clearly eliminated by the memoized Fibonacci. Nothing seemed suspicious to cause StackOverflowError at the first sight, the depth of the stack frames for the memoized Fibonacci was strictly linear to the input number n, and the width was limited to 1.
TRACE t427: (fib.core/naive-fib 5)
TRACE t428: | (fib.core/naive-fib 4)
TRACE t429: | | (fib.core/naive-fib 3)
TRACE t430: | | | (fib.core/naive-fib 2)
TRACE t431: | | | | (fib.core/naive-fib 1)
TRACE t431: | | | | => 1
TRACE t432: | | | | (fib.core/naive-fib 0)
TRACE t432: | | | | => 0M
TRACE t430: | | | => 1M
TRACE t433: | | | (fib.core/naive-fib 1)
TRACE t433: | | | => 1
TRACE t429: | | => 2M
TRACE t434: | | (fib.core/naive-fib 2)
TRACE t435: | | | (fib.core/naive-fib 1)
TRACE t435: | | | => 1
TRACE t436: | | | (fib.core/naive-fib 0)
TRACE t436: | | | => 0M
TRACE t434: | | => 1M
TRACE t428: | => 3M
TRACE t437: | (fib.core/naive-fib 3)
TRACE t438: | | (fib.core/naive-fib 2)
TRACE t439: | | | (fib.core/naive-fib 1)
TRACE t439: | | | => 1
TRACE t440: | | | (fib.core/naive-fib 0)
TRACE t440: | | | => 0M
TRACE t438: | | => 1M
TRACE t441: | | (fib.core/naive-fib 1)
TRACE t441: | | => 1
TRACE t437: | => 2M
TRACE t427: => 5M
TRACE t446: (fib.core/memo-fib 5)
TRACE t447: | (fib.core/memo-fib 4)
TRACE t448: | | (fib.core/memo-fib 3)
TRACE t449: | | | (fib.core/memo-fib 2)
TRACE t450: | | | | (fib.core/memo-fib 1)
TRACE t450: | | | | => 1
TRACE t451: | | | | (fib.core/memo-fib 0)
TRACE t451: | | | | => 0M
TRACE t449: | | | => 1M
TRACE t452: | | | (fib.core/memo-fib 1)
TRACE t452: | | | => 1
TRACE t448: | | => 2M
TRACE t453: | | (fib.core/memo-fib 2)
TRACE t453: | | => 1M
TRACE t447: | => 3M
TRACE t454: | (fib.core/memo-fib 3)
TRACE t454: | => 2M
TRACE t446: => 5M
Question 2
Why could Clojure assert at compile-time that the depth of merely 225 stack frames for the memoized Fibonacci in my case could potentially explode the whole JVM stack thus cease to run the recursion altogether? From the source code on memoize below, I could see that an empty hashmap was initiated to cache the arguments and the returns for the memoized Fibonacci. Did the said hashmap cause the assertion of StackOverflowError? Why?
(defn memoize
"Returns a memoized version of a referentially transparent function. The
memoized version of the function keeps a cache of the mapping from arguments
to results and, when calls with the same arguments are repeated often, has
higher performance at the expense of higher memory use."
{:added "1.0"
:static true}
[f]
(let [mem (atom {})]
(fn [& args]
(if-let [e (find #mem args)]
(val e)
(let [ret (apply f args)]
(swap! mem assoc args ret)
ret)))))
Others - (for the sake of completeness but unrelated to the OP)
We can take advantage of loop recur to achieve something like TCO, or the laziness of iterate implementation.
(ns fib.core)
(defn tail-fib [n]
(loop [n n
x (bigdec 0)
y 1]
(condp = n
0 0
1 y
(recur (dec n) y (+ x y)))))
(defn lazy-fib [n]
(->> n
(nth (iterate (fn [[x y]]
[y (+ x y)])
[(bigdec 0) 1]))
first))
(in-ns 'user)
(require '[clojure.tools.trace :refer [trace-ns]])
(trace-ns 'fib.core)
(fib.core/tail-fib 2000)
(println)
(fib.core/lazy-fib 2000)
The tracing tells that they don't make any recursive call longer in effect.
TRACE t471: (fib.core/tail-fib 2000)
TRACE t471: => 4224696333392304878706725602341482782579852840250681098010280137314308584370130707224123599639141511088446087538909603607640194711643596029271983312598737326253555802606991585915229492453904998722256795316982874482472992263901833716778060607011615497886719879858311468870876264597369086722884023654422295243347964480139515349562972087652656069529806499841977448720155612802665404554171717881930324025204312082516817125M
TRACE t476: (fib.core/lazy-fib 2000)
TRACE t476: => 4224696333392304878706725602341482782579852840250681098010280137314308584370130707224123599639141511088446087538909603607640194711643596029271983312598737326253555802606991585915229492453904998722256795316982874482472992263901833716778060607011615497886719879858311468870876264597369086722884023654422295243347964480139515349562972087652656069529806499841977448720155612802665404554171717881930324025204312082516817125M
Memoization does not affect the stack when the cache is empty. Your Clojure code is not equivalent to the Python code because the Clojure version is recursive while the Python version is iterative.
Memoize returns function, so you have to use def:
(def fib
(memoize #(condp = %
0 (bigdec 0)
1 1
(+ (fib (dec %)) (fib (- % 2))))))
(fib 225)
=> 47068900554068939361891195233676009091941690850M
Not sure why on a fresh repl
(def fib
(memoize #(condp = %
0 (bigdec 0)
1 1
(+ (fib (dec %)) (fib (- % 2))))))
(fib 136) gives me an overflow
(fib 135) works fine
then when I do (fib 136) again I git no error.
Doing, from a fresh repl,
(map fib (range 1 10000))
works fine
I've carefully investigated the problem and finally figured it out, hopefully.
At first, let's look at a comparable yet handy recursive implementation on memoized Fibonacci in Python, before we jump into study on it in Clojure.
def memo_fib(n):
def fib(n):
print(f"ALL: {n}")
if n not in dp:
print(f"NEW: {n}")
if n == 0:
dp[0] = 0
elif n == 1:
dp[1] = 1
else:
dp[n - 1], dp[n - 2] = fib(n - 1), fib(n - 2)
dp[n] = dp[n - 1] + dp[n - 2]
return dp[n]
dp = {}
return fib(n)
memo_fib(5)
We add two print clauses to trace the recursive executions on memoized Fibonacci. ALL denotes what actually enter the spawned call stacks even if they are already memoized, while NEW denotes what proportionally show up only if they have yet to memoize. Obviously, the total ALL steps are greater than the NEW ones, which is also evident on the output below,
ALL: 5
NEW: 5
ALL: 4
NEW: 4
ALL: 3
NEW: 3
ALL: 2
NEW: 2
ALL: 1
NEW: 1
ALL: 0
NEW: 0
ALL: 1
ALL: 2
ALL: 3
Let's also compare it to naive Fibonacci in Python below,
def naive_fib(n):
print(f"NAIVE: {n}")
if n == 0:
return 0
if n == 1:
return 1
return naive_fib(n - 1) + naive_fib(n - 2)
naive_fib(5)
We can see below that memoized Fibonacci does reduce the recursive steps thus improve the stack more efficiently over naive Fibonacci, which invalidates the explanation #Eugene Pakhomov posted:
Memoization does not affect the stack when the cache is empty.
NAIVE: 5
NAIVE: 4
NAIVE: 3
NAIVE: 2
NAIVE: 1
NAIVE: 0
NAIVE: 1
NAIVE: 2
NAIVE: 1
NAIVE: 0
NAIVE: 3
NAIVE: 2
NAIVE: 1
NAIVE: 0
NAIVE: 1
Back to memoized Fibonacci in Clojure, what we have been able to trace in the OP are just the NEW steps. To peek at the ALL steps, we have to trace memoize function itself inside out. Please help if you know how to do it conveniently.
Now it's time to conclude that memoize can help reduce repetitive recursive calls but it cannot entirely prevent recursive calls from spawning stack frames again and again for some of overlapping sub-problems. StackOverflowError emerges to surprise you if those sub-problems grow rapidly.
In the clojure documentation it says:
Usage: (reduced x)
Wraps x in a way such that a reduce will terminate with the value x
I am trying to return from a reduce function with a boolean and a vector values.
(def bp (reduce (fn [[balanced stack] singlenum]
(def stack2 (conj stack singlenum))
(println stack2)
(if (= 2 singlenum)
(reduced [false stack2])
)
[balanced stack2]
)
[true (vector)] [1 2 3 4]
))
bp evaluates as [true [1 2 3 4]], I was expecting [false [1 2]]. The reduced did not terminate the reduce function. I was attempting to terminate the reduce function with a specific values.
You have the correct logic there. I just revised your usage of if and def.
if - I moved [balanced stack2] to the else part. Otherwise reduced will never be detected.
def - the def inside fn should be replaced with let
(def bp (reduce (fn [[balanced stack] singlenum]
(let [stack2 (conj stack singlenum)]
(println stack2)
(if (= 2 singlenum)
(reduced [false stack2])
[balanced stack2])))
[true (vector)]
[1 2 3 4]))
| | | | | stack=> []
| | | | | singlenum=> 1
| | | | (conj stack singlenum)=> [1]
| | | | stack2=> [1]
[1]
| | | (println stack2)=> nil
| | | | | singlenum=> 1
| | | | (= 2 singlenum)=> false
| | | | | balanced=> true
| | | | | stack2=> [1]
| | | (if (= 2 singlenum) (reduced #) [balanced stack2])=> [true [1]]
| | (let [stack2 #] (println stack2) (if # # #))=> [true [1]]
| | | | | stack=> [1]
| | | | | singlenum=> 2
| | | | (conj stack singlenum)=> [1 2]
| | | | stack2=> [1 2]
[1 2]
| | | (println stack2)=> nil
| | | | | singlenum=> 2
| | | | (= 2 singlenum)=> true
| | | | | | stack2=> [1 2]
| | | | (reduced [false stack2])=> #reduced[{:status :ready, :val [false [1 2]]} 0x5fbdbb78]
| | | (if (= 2 singlenum) (reduced #) [balanced stack2])=> #reduced[{:status :ready, :val [false [1 2]]} 0x5fbdbb78]
| | (let [stack2 #] (println stack2) (if # # #))=> #reduced[{:status :ready, :val [false [1 2]]} 0x5fbdbb78]
(reduce (fn # #) [true #] [1 2 3 4])=> [false [1 2]]
I am using DrRacket to learn Scheme, The language subset is racket (#lang racket). I have the following list/expression:
(cons (cons (cons 'a 'b) (cons 'c 'd)) (cons (cons 'e 'f) (cons 'g 'h)))
When I press enter I get the following value printed out:
'(((a . b) c . d) (e . f) g . h)
I would expect the result to be:
'((( a . b) (c . d)) ((e . f) ( g . h)))
Can anyone explain what rules scheme uses to decide to print the value the way it does?
Attached is also a screen of the terminal:
cons creates a pair, which is also a list. So, when you later cons something else to a list, you extend this list.
(cons 'a 'b)
==> (a . b)
(cons 'c '(a . b))
==> (c a . b)
(cons 'd '(c a . b))
==> (d c a . b)
And this is what happens when you cons new pair to another pair/list:
(cons 'something '(c . d))
==> (something c . d)
(cons '(a . b) (c . d))
==> ((a . b) c . d)
Sometimes it's helpful to draw pairs as boxes to see what is going on:
(cons 'a 'b) (cons 'c 'd)
+---+---+ +---+---+
| | |--- b | | |--- d
+---+---+ +---+---+
| |
a c
(cons (cons 'a 'b) (cons 'c 'd))
+---+---+ +---+---+
| | |--- | | |--- d
+---+---+ +---+---+
| |
| c
+---+---+
| | |--- b
+---+---+
|
a
To get ((a . b) (c . d)) you have to:
(cons (cons 'a 'b) (cons (cons 'c 'd) '()))
==> ((a . b) (c . d))
+---+---+ +---+---+
| | |------- | | |--- NIL/()
+---+---+ +---+---+
| |
| |
+---+---+ +---+---+
| | |--- b | | |--- d
+---+---+ +---+---+
| |
a c
The result of your expression is:
'((( a . b) . (c . d)) . ((e . f) . ( g . h)))
and not
'((( a . b) (c . d)) ((e . f) ( g . h)))
as you are expecting.
However it is not printed in that way, for the rules of printing of lisp languages. See for instance this page (it is for Common Lisp, but the printing rules are the same for Scheme):
Notice that LISP prints linked lists a special way: it omits some of
the periods and parentheses. The rule is: if the cdr of a cons is nil,
LISP doesn't bother to print the period or the nil; and if the cdr of
cons A is cons B, then LISP doesn't bother to print the period for cons
A or the parentheses for cons B. So:
(cons 4 nil)
(4)
(cons 4 (cons 5 6))
(4 5 . 6)
(cons 4 (cons 5 (cons 6 nil)))
(4 5 6)
Suppose we define a relation matches via the following table:
|-----+-----+-----+-----+-----|
| | *A* | *B* | *C* | *D* |
|-----+-----+-----+-----+-----|
| *A* | | X | X | |
|-----+-----+-----+-----+-----|
| *B* | X | | | X |
|-----+-----+-----+-----+-----|
| *C* | X | | | |
|-----+-----+-----+-----+-----|
| *D* | | X | | |
|-----+-----+-----+-----+-----|
By this, it is meant that (in pseduo-code)
(matches A) ;=> (B C)
(matches B) ;=> (A D)
(matches C) ;=> (C)
(matches D) ;=> (B)
In core.logic, I think I would know how to make individualized functions which could approximate the behavior of matches:
(defn matches-A
(run* [q]
(membero q [B C]))) ;=> (B C)
...and so forth for matches-B and matches-C.
Question: How could I generalize, i.e., matches-A to a single function matches as above? In particular, I'd be interested in making it so that you could run queries like (matches "not A"), (matches "B and C") and (matches "C or D") (in pseudo-code) to get results like (A D), (A), and (A B) respectively. Is this possible?
NOTE: I'm using clojurescript instead of clojure. I'm not sure if that would impact the answer at all.
You can use conde to solve this task:
(ns qradv.welcome
(:require [cljs.core.logic :as l]))
;; |-----+-----+-----+-----+-----|
;; | | *A* | *B* | *C* | *D* |
;; |-----+-----+-----+-----+-----|
;; | *A* | | X | X | |
;; |-----+-----+-----+-----+-----|
;; | *B* | X | | | X |
;; |-----+-----+-----+-----+-----|
;; | *C* | X | | | |
;; |-----+-----+-----+-----+-----|
;; | *D* | | X | | |
;; |-----+-----+-----+-----+-----|
(defn matches [x]
(l/run* [y]
(l/conde
[(l/== x "A") (l/membero y ["B" "C"])]
[(l/== x "B") (l/membero y ["A" "D"])]
[(l/== x "C") (l/membero y ["C"])]
[(l/== x "D") (l/membero y ["B"])])))
(prn (matches "A"))
(prn (matches "B"))
(prn (matches "C"))
(prn (matches "D"))
Output:
("B" "C")
("A" "D")
("C")
("B")
While learning clojure, I was very surprised to find out that these two objects are different types:
(list? `(inc)) ;; true
(list? `(inc 1)) ;; false
In theory, I understand why the second form returns false, that object
is actually a clojure.lang.Cons. In practice, though, I don't
understand why that is happening.
Why does the reader read `(inc) different from `(inc 1)? What is happening under the hood?
When the reader encounters a syntax-quoted form, that turns out to be a collection, it will iterate over each element and call syntax-quote recursively. The result is consed, beginning with nil.
So it comes down to the question why the following holds:
> (list? (cons 'inc nil))
true
> (list? (cons 'inc (cons 1 nil)))
false
This seems to be a matter of definition.
list? is actually a function of very limited usefulness. In fact I have yet to see Clojure code that used list? without it being at best a poor choice, more often the cause of a bug.
If you want to know if something is "listy", seq? is a great choice.
in action:
user=> (pprint/print-table (for [item [[] () `(a) `(a b) (seq [1])]]
{'item (pr-str item)
'seq? (seq? item)
'list? (list? item)
'type (type item)}))
| item | seq? | list? | type |
|-----------------+-------+-------+------------------------------------------------|
| [] | false | false | class clojure.lang.PersistentVector |
| () | true | true | class clojure.lang.PersistentList$EmptyList |
| (user/a) | true | true | class clojure.lang.PersistentList |
| (user/a user/b) | true | false | class clojure.lang.Cons |
| (1) | true | false | class clojure.lang.PersistentVector$ChunkedSeq |