MKL library error when using Neanderthal in Clojure - clojure

I'm trying to use the Neanderthal library in Clojure but I keep getting an error when executing any rand-normal! and rand-uniform!. I've installed the Intel MKL library, executed the mklvars.bat as mklvars intel64 file to set the environment variables, but then I checked the env. variables and since it didn't add anything new, I've added them manually. What can I do to make it to work?
I'm using Windows 10 64 bits.
This is my code:
(ns testing-grounds.neanderthal.core
(:gen-class)
(:require [uncomplicate.neanderthal
[native :refer [dv dge fv fge native-float]]
[core :refer [submatrix native dot mm ge]]
[math :refer [sqrt log sin pi sqr]]
[random :refer [rand-normal! rand-uniform! rng-state]]]))
(def x (dv 1 2 3))
;; => #'testing-grounds.neanderthal.core/x
(def y (dv 10 20 30))
;; => #'testing-grounds.neanderthal.core/y
(dot x y)
;; => 140.0
(def a (dge 3 2 [1 2 3 4 5 6]))
;; => #'testing-grounds.neanderthal.core/a
(def b (dge 2 3 [10 20 30 40 50 60]))
;; => #'testing-grounds.neanderthal.core/b
(mm a b)
;; => #RealGEMatrix[double, mxn:3x3, layout:column, offset:0]
;; ▥ ↓ ↓ ↓ ┓
;; → 90.00 190.00 290.00
;; → 120.00 260.00 400.00
;; → 150.00 330.00 510.00
;; ┗ ┛
(def x
"Define an empty float vector"
(fv 5))
;; => #'testing-grounds.neanderthal.core/x
(rand-uniform! x)
; Error
(def a (fge 3 2))
;; => #'testing-grounds.neanderthal.core/a
(rand-normal! 33 2.5 a)
; Error
The project.clj
(defproject testing-grounds "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "LATEST"]
[uncomplicate/neanderthal "LATEST"]
[incanter "LATEST"]
[org.clojure/tools.nrepl "LATEST"]
[nrepl "LATEST"]
[cider/cider-nrepl "LATEST"]
]
:plugins [[cider/cider-nrepl "LATEST"]]
:java-cmd "C:/Users/user/Downloads/jdk-12.0.1/bin/java"
:jvm-opts ["--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED"]
:main ^:skip-aot testing-grounds.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})
The error:
Show: Project-Only All
Hide: Clojure Java REPL Tooling Duplicates (0 frames hidden)
1. Unhandled clojure.lang.ExceptionInfo
MKL error.
{:error-code -1140}
mkl.clj: 823 uncomplicate.neanderthal.internal.host.mkl.FloatVectorEngine/rand_uniform
random.clj: 42 uncomplicate.neanderthal.random/rand-uniform!
random.clj: 37 uncomplicate.neanderthal.random/rand-uniform!
REPL: 28 testing-grounds.neanderthal.core/eval33584
REPL: 28 testing-grounds.neanderthal.core/eval33584
Compiler.java: 7177 clojure.lang.Compiler/eval
Compiler.java: 7132 clojure.lang.Compiler/eval
core.clj: 3214 clojure.core/eval
core.clj: 3210 clojure.core/eval
main.clj: 437 clojure.main/repl/read-eval-print/fn
main.clj: 437 clojure.main/repl/read-eval-print
main.clj: 458 clojure.main/repl/fn
main.clj: 458 clojure.main/repl
main.clj: 368 clojure.main/repl
RestFn.java: 1523 clojure.lang.RestFn/invoke
interruptible_eval.clj: 79 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 55 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 142 nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
AFn.java: 22 clojure.lang.AFn/run
session.clj: 171 nrepl.middleware.session/session-exec/main-loop/fn
session.clj: 170 nrepl.middleware.session/session-exec/main-loop
AFn.java: 22 clojure.lang.AFn/run
Thread.java: 835 java.lang.Thread/run
Edit: I've tried this script on another computer and it worked. It must be something wrong with my computer.

Intel MKL error code -1140 is VSL_RNG_ERROR_ARS5_NOT_SUPPORTED.
You can look up error codes by grepping the header files in your installation. On my macOS computer it was this file: /opt/intel/oneapi/mkl/2021.1.1/include/mkl_vsl_defines.h

ARS-5 is available only on microarchitectures with AES-NI instructions, and in case if the application would be launched on microarchitecture which doesn’t support AES-NI instructions, then VSL RNG routines will return VSL_RNG_ERROR_ARS5_NOT_SUPPORTED status code.

Related

Datomic hides parts of its history when query is about all attributes

I have a datomic query:
(pprint (d/q '[:find ?tx ?attr ?val ?op :in $ ?e :where [?e ?attr ?val ?tx ?op]] (d/history (d/db db/conn)) 17592186766274))
that yields:
#{[13194140264981 354 6 true]
[13194140265425 352 "OK" true]
[13194140255168 348 17592186593957 true]
[13194140265425 353 true true]
[13194140255168 311 #uuid "6086bec6-7a11-4238-9ce3-67e6427d7b07" true]
[13194140255168 346 17592186491729 true]
[13194140265424 351 #uuid "60871d86-cdc2-4ddf-b091-98117c3948af" true]
[13194140265425 349 #inst "2021-04-26T20:29:34.566-00:00" true]
[13194140255168 347 "{:driver \"6144c7bd-8e71-49c3-8eef-2e9d157f3e11\", :order \"c89d32d2-92da-4cde-827d-da6406877c28\"}\n" true]
[13194140265424 350 #inst "2021-04-26T20:29:34.472-00:00" true]
[13194140255168 356 #inst "2021-04-26T18:23:18.140-00:00" true]}
Please note that attribute 354 is shown to be changed in history once only (to value 6).
Then, if we ask for this particular attribute...
(pprint (d/q '[:find ?tx ?val ?op :in $ ?e :where [?e 354 ?val ?tx ?op]] (d/history (d/db db/conn)) 17592186766274))
... we suddenly find that it has been modified multiple times:
#{[13194140264479 1 false]
[13194140264516 2 false]
[13194140264585 4 true]
[13194140264981 6 true]
[13194140264721 4 false]
[13194140264461 1 true]
[13194140264479 2 true]
[13194140264721 5 true]
[13194140264516 3 true]
[13194140264585 3 false]
[13194140264981 5 false]}
Furthermore, if we ask the original question agains the current database instead of history, we get the same result!
(pprint (d/q '[:find ?tx ?attr ?val ?op :in $ ?e :where [?e ?attr ?val ?tx ?op]] (d/db db/conn) 17592186766274))
gives:
#{[13194140264981 354 6 true]
[13194140265425 352 "OK" true]
[13194140255168 348 17592186593957 true]
[13194140265425 353 true true]
[13194140255168 311 #uuid "6086bec6-7a11-4238-9ce3-67e6427d7b07" true]
[13194140255168 346 17592186491729 true]
[13194140265424 351 #uuid "60871d86-cdc2-4ddf-b091-98117c3948af" true]
[13194140265425 349 #inst "2021-04-26T20:29:34.566-00:00" true]
[13194140255168 347 "{:driver \"6144c7bd-8e71-49c3-8eef-2e9d157f3e11\", :order \"c89d32d2-92da-4cde-827d-da6406877c28\"}\n" true]
[13194140265424 350 #inst "2021-04-26T20:29:34.472-00:00" true]
[13194140255168 356 #inst "2021-04-26T18:23:18.140-00:00" true]}
It looks as if the question for all attribute changes for an entity was too general, but no error is returned and wrong answer is presented. Why?
Ok, I know what has happened. All the attributes for this entity had :db/noHistory set to true. It is still quite mysterious why the changes still show when we query for a particular attribute.

Error running a Hello World app using spacemacs and clojure

(ns myname.myapp
(:gen-class))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "Hello, World!"))
Running "evaluate last sexpression" with the marker on (println "Hello, World!") caused the following error:
Show: Project-Only All
Hide: Clojure Java REPL Tooling Duplicates (15 frames hidden)
2 Unhandled clojure.lang.Compiler$CompilerException
Error compiling src/myname/myapp.clj at (0:0)
#:clojure.error{:phase :compile-syntax-check,
:line 0,
:column 0,
:source "/Users/*/dev/myapp/src/myname/myapp.clj"}
Compiler.java: 6808 clojure.lang.Compiler/analyze
Compiler.java: 6745 clojure.lang.Compiler/analyze
Compiler.java: 7181 clojure.lang.Compiler/eval
Compiler.java: 7132 clojure.lang.Compiler/eval
core.clj: 3216 clojure.core/eval
core.clj: 3212 clojure.core/eval
interruptible_eval.clj: 91 nrepl.middleware.interruptible-eval/evaluate/fn
main.clj: 437 clojure.main/repl/read-eval-print/fn
main.clj: 437 clojure.main/repl/read-eval-print
main.clj: 458 clojure.main/repl/fn
main.clj: 458 clojure.main/repl
main.clj: 368 clojure.main/repl
RestFn.java: 137 clojure.lang.RestFn/applyTo
core.clj: 665 clojure.core/apply
core.clj: 660 clojure.core/apply
regrow.clj: 20 refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
RestFn.java: 1523 clojure.lang.RestFn/invoke
interruptible_eval.clj: 84 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 56 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 155 nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
AFn.java: 22 clojure.lang.AFn/run
session.clj: 190 nrepl.middleware.session/session-exec/main-loop/fn
session.clj: 189 nrepl.middleware.session/session-exec/main-loop
AFn.java: 22 clojure.lang.AFn/run
Thread.java: 832 java.lang.Thread/run
1 Caused by java.lang.RuntimeException
Unable to resolve symbol: World! in this context
Util.java: 221 clojure.lang.Util/runtimeException
Compiler.java: 7414 clojure.lang.Compiler/resolveIn
Compiler.java: 7358 clojure.lang.Compiler/resolve
Compiler.java: 7319 clojure.lang.Compiler/analyzeSymbol
Compiler.java: 6768 clojure.lang.Compiler/analyze
Compiler.java: 6745 clojure.lang.Compiler/analyze
Compiler.java: 7181 clojure.lang.Compiler/eval
Compiler.java: 7132 clojure.lang.Compiler/eval
core.clj: 3216 clojure.core/eval
core.clj: 3212 clojure.core/eval
interruptible_eval.clj: 91 nrepl.middleware.interruptible-eval/evaluate/fn
main.clj: 437 clojure.main/repl/read-eval-print/fn
main.clj: 437 clojure.main/repl/read-eval-print
main.clj: 458 clojure.main/repl/fn
main.clj: 458 clojure.main/repl
main.clj: 368 clojure.main/repl
RestFn.java: 137 clojure.lang.RestFn/applyTo
core.clj: 665 clojure.core/apply
core.clj: 660 clojure.core/apply
regrow.clj: 20 refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
RestFn.java: 1523 clojure.lang.RestFn/invoke
interruptible_eval.clj: 84 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 56 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 155 nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
AFn.java: 22 clojure.lang.AFn/run
session.clj: 190 nrepl.middleware.session/session-exec/main-loop/fn
session.clj: 189 nrepl.middleware.session/session-exec/main-loop
AFn.java: 22 clojure.lang.AFn/run
Thread.java: 832 java.lang.Thread/run
How do I resolve this?
My guess is that you haven't placed your cursor right after the expression you wanted to evaluate. "Evaluate last sexp" should be invoked when your cursor is right after the form you want to eval.
Here's a more visual explanation. For the command to work your cursor (represented as |) should be here:
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "Hello, World!")|)
or here:
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "Hello, World!"))|
You can also try "eval defun" ("defun" means top-level form in CIDER's lingo) or "eval buffer". Those command are easier to work with for people new to Clojure/Lisp.
Something has gone wrong with your evaluation of the form. This error message:
1 Caused by java.lang.RuntimeException
Unable to resolve symbol: World! in this context
indicates that the compiler is seeing the chars World! without the surrounding double-quotes, so it then looks for a nonexistent variable of that name.
It may be easier to start off just using lein run and lein test to run/test your code. You can also clone this sample project which includes the lein-test-refresh plugin, which is even better than a repl IMHO.

Why does this Clojure prime generator raises a StackOverflowError? [duplicate]

This question already has answers here:
Recursive function causing a stack overflow
(2 answers)
Closed 3 years ago.
I am just learning Clojure and, as usual when lerning new programming languages, one of the first things I tried is implementing the Sieve of Eratosthenes.
I came up with the following solution:
(defn primes
"Calculate all primes up to the given number"
[n]
(loop
[
result []
numbers (range 2 (inc n))
]
(if (empty? numbers)
result
(let [[next & rest] numbers]
(recur (conj result next) (filter (fn [n] (not= 0 (mod n next))) rest)))
)
)
)
It works fine and quite fast for small numbers but for large inputs a StackOverflowError is raised with a suspiciously short stacktrace, eg.:
(primes 100000)
Execution error (StackOverflowError) at (REPL:1).
null
(pst)
StackOverflowError
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:51)
clojure.lang.RT.seq (RT.java:531)
clojure.core/seq--5387 (core.clj:137)
clojure.core/filter/fn--5878 (core.clj:2809)
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:51)
clojure.lang.RT.seq (RT.java:531)
clojure.core/seq--5387 (core.clj:137)
clojure.core/filter/fn--5878 (core.clj:2809)
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:51)
=> nil
I was under the impression that recur implements tail recursion if it is evaluated last in a loop form and my first question is if this is really the case here. My second question is why the stack trace is so short for a StackOverflowError. I also have problems interpreting the stacktrace, ie. what line corresponds to what form.
I am only interested in better or more Clojure-like solutions if they provide insights for these questions, since otherwise I would like to find them by myself. Thank you!
Slightly modified, with comments to describe what is happening on each line, this is your function:
(defn primes
"Calculate all primes up to the given number"
[n]
;; `loop` is not lazy, it runs until it produces a result:
(loop [result []
;; a lazy sequence implemented by clojure.lang.LongRange:
numbers (range 2 (inc n))]
(if (not (nil? (seq numbers)))
result
(let [current (first numbers)
remaining (rest numbers)]
(recur
;; `conj` on a vector returns a vector (non-lazy):
(conj result current)
;; `filter` on a lazy sequence returns a new lazy sequence:
(filter (fn [n] (not= 0 (mod n next)))
remaining))))))
The key is that filter at the end.
Most lazy sequence operations such as filter work by wrapping one lazy sequence in another. On each iteration of the loop, filter adds another layer of lazy sequence, like this:
(filter (fn [n] (not= 0 (mod n 5))) ; returns a LazySeq
(filter (fn [n] (not= 0 (mod n 4))) ; returns a LazySeq
(filter (fn [n] (not= 0 (mod n 3))) ; returns a LazySeq
(filter (fn [n] (not= 0 (mod n 2))) ; returns a LazySeq
remaining))))
The LazySeq objects stack up, each one holding a reference to the previous.
With most lazy sequences, the wrapping doesn't matter because they automatically "unwrap" as soon as you request a value. That happens in LazySeq.seq.
This is one case where it does matter, because your loop builds up so many layers of lazy sequence objects that the nested calls to LazySeq.seq and .sval overflow the maximum stack size allowed by the JVM. That's what you see in the stacktrace.
(This also has memory implications, since a reference to the start of the sequence prevents any of the others from being garbage-collected, what Clojure programmers call "holding on to the head" of the sequence.)
The more general issue with this function is mixing lazy (filter) and non-lazy (loop) operations. That's often a source of problems, so Clojure programmers learn to avoid it out of habit.
As Alan suggests, you can avoid the problem by using only non-lazy operations, such as filterv instead of filter, which forces the lazy sequence into a vector.
Almost any style of lazy evaluation can exhibit some variation of this problem. I described it in Clojure don'ts: concat. For another example see foldr versus foldl in Haskell.
Even without laziness, deeply-nested object trees can cause a StackOverflow, for examples in Java I found xstream#88 or circe#1074.
Here is a version that works:
(ns tst.demo.core
(:use tupelo.core tupelo.test))
(defn primes
"Calculate all primes up to the given number"
[n]
(loop [result []
numbers (range 2 (inc n))]
(if (empty? numbers)
result
(let [[new-prime & candidate-primes] numbers]
(recur
(conj result new-prime)
(filterv (fn [n] (not= 0 (mod n new-prime)))
candidate-primes))) )))
(dotest
(spyx (primes 99999))
)
with result:
-------------------------------
Clojure 1.10.1 Java 13
-------------------------------
Testing tst.demo.core
(primes 99999) => [2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61
67 71 73 79 83 89 97 101 103 107 109 113 127 131 137 139 149 151 157 163
167 173 179 181 191 193 197 199 211 223 227 229 233 239 241 251 257 263
269 271 277 281 283 293 307 311 313 317 331 337 347 349 353 359 367 373
379 383 389 397 401 409 419 421 431 433 439 443 449 457 461 463 467 479
487 491 499 503 509 521 523 541 547 557 563 569 571 577 587 593 599 601
...<snip>...
99401 99409 99431 99439 99469 99487 99497 99523 99527 99529 99551 99559
99563 99571 99577 99581 99607 99611 99623 99643 99661 99667 99679 99689
99707 99709 99713 99719 99721 99733 99761 99767 99787 99793 99809 99817
99823 99829 99833 99839 99859 99871 99877 99881 99901 99907 99923 99929
99961 99971 99989 99991]
I renames your variables a bit to make them clearer. If you look closely, you'll see the only substantive difference is the change from the lazy filter to the eager filterv.
Before this change, it worked for a N of 9999 but failed for 99999.
I'm not sure about the implementation of the lazy filter function, but that is clearly the problem.
Strange (& unpredictable) problems like this reinforce my dislike of excessive laziness in Clojure code. It appears you have crashed into a variant of the Clojure Don'ts: Concat problem. In this instance, you code looks like:
(filter ...
(filter ...
(filter ...
(filter ...
...<many, many more>... ))))
Lazy sequences are implemented as nested function calls. Since the last loop that finds prime 99991 is dependent on the first call that finds prime 2, the earlier lazy sequences (and their nested function calls on the stack) cannot be released and you eventually blow the stack.
On my computer, a simple recursive implementation of factorial(N) blows up around N=4400. The above found 9592 primes, so it the specific cause seems to be a bit more complex than 1 stack frame per prime.
Possibly N=32 chunking could play a part.
In order to avoid bugs due to unnecessary laziness, you may be interested in replacing concat with glue, and replacing for with forv. You can also see the full API docs.

Unittest in Clojure doesn't compile when I test my own functions

I am new to Clojure and now I'm trying to use some unittesting.
I have a sample project with this structure:
core.clj in src/hello contains
(ns hello.core
(:gen-class))
(defn side-eq [x]
(if (< 0 (count x))
(= (.charAt x 0) (.charAt x (- (count x) 1)))
false))
core.clj in test/hello contains
(ns hello.core
(:use clojure.test)
(:require [hello.core :refer :all])
(:gen-class))
(use 'clojure.test)
(deftest side-eq-tests (
is (= false (side-eq "aws"))))
(run-tests)
when I execute tests, it throws
java.lang.RuntimeException: Unable to resolve symbol: side-eq in this context
When I test something like
is (= 1 1)
then everything works fine.
What is going on?
You should not have multiple files with the same namespace. Rename your tests to something else. The idiomatic name here would be hello.core-test In test/hello/core_test.clj
A variation which I prefer is to begin all testing namespacese with the tst.* prefix, which avoids the hyphen-underscore conversion & confusion (e.g. demo.core-test vs demo.core_test.clj. So your files look like so:
> d **/*.clj
-rwxrwxr-x 1 alan alan 1024 Jan 5 19:00 project.clj*
-rwxrwxr-x 1 alan alan 84 Jan 5 15:29 src/demo/core.clj*
-rwxrwxr-x 1 alan alan 248 Jan 7 12:42 test/tst/demo/core.clj*
and the code looks like:
(ns demo.core)
(defn -main []
(println "main - enter")
)
and
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test)
(:require
[tupelo.misc :as tm] ))
....

parse doesn't accept all month names abbreviation

For a given custom-formatter, parse refuses certain month values.
(require '[clj-time.core :as t]
'[clj-time.format :as f])
(let [custom-formatter (f/formatter "dd MMM yyyy")]
(f/parse custom-formatter "27 mar 2010")
(f/parse custom-formatter "27 dec 2010"))
The first parsereturns the expected result, #object[org.joda.time.DateTime 0x62a9d176 "2010-03-27T00:00:00.000Z"], while the second returns Actual: java.lang.IllegalArgumentException: Invalid format: "27 dec 2010" is malformed at "dec 2010".
I can't make sense of this behavior. What can possibly causes this issue?
Thanks for help.
update
Here's my project.cljfile
:dependencies [[org.clojure/clojure "1.7.0"]
[twitter-api "0.7.8"]
[cheshire "5.6.1"]
[clojure-csv/clojure-csv "2.0.1"]
[org.postgresql/postgresql "9.4-1206-jdbc42"]
[com.jolbox/bonecp "0.8.0.RELEASE"]
[org.clojure/java.jdbc "0.6.1"]
[java-jdbc/dsl "0.1.0"]
[org.slf4j/slf4j-nop "1.7.21"]
[clj-time "0.11.0"]
]
You likely need to specify a Locale for your formatter. You can check to see your default Locale with:
user=> (java.util.Locale/getDefault)
#object[java.util.Locale 0x4713d9a5 "en_US"]
See the results for different locales (including the error you are seeing):
user=> (require '[clj-time.core :as t]
#_=> '[clj-time.format :as f])
nil
user=> (let [custom-formatter (f/with-locale
(f/formatter "dd MMM yyyy")
java.util.Locale/ENGLISH)]
(f/parse custom-formatter "27 mar 2010"))
#object[org.joda.time.DateTime 0x3cfceae6 "2010-03-27T00:00:00.000Z"]
user=> (let [custom-formatter (f/with-locale
(f/formatter "dd MMM yyyy")
java.util.Locale/ITALY)]
(f/parse custom-formatter "27 mar 2010"))
#object[org.joda.time.DateTime 0x4f5e9955 "2010-03-27T00:00:00.000Z"]
user=> (let [custom-formatter (f/with-locale
(f/formatter "dd MMM yyyy")
java.util.Locale/CHINA)]
(f/parse custom-formatter "27 mar 2010"))
IllegalArgumentException Invalid format: "27 mar 2010" is malformed at "mar 2010" org.joda.time.format.DateTimeFormatter.parseDateTime (DateTimeFormatter.java:899)