How do I intitailze my database so my (def db ...) will not attempt to initialize the database when running my unit-tests (since it will not be a database there then).
I mock the the sql and dml in functions (with-redefs) but all examples i have found so far simply define the database directly in the namespace (eg not wrapped in a function).
I strongly recommend you do not put your database in a Var. It's a very tempting, but quite harmful, form of environment coupling.
The recommended way is to pass the database your database as an argument of whatever function needs access to it in its body:
;; BAD
(def db ...)
(defn find-user [user-id]
(run-query find-user-query db user-id))
;; GOOD
(defn find-user [db user-id]
(run-query find-user-query db user-id))
It's not as tedious as you'd think, and the benefits are great (for testing, repl-driven development, etc.).
NOTE:
In the early years of the Clojure community, people have been using dynamic Vars to avoid having to add a parameter all the time.
(def ^:dynamic db nil)
(binding [db ...]
(find-user user-id))
Then we learned to stop doing that :) complecting your logic with its environment is just a bad foundation for your programs.
Related
In my clojure Luminus/Compojure app I have this in routes.clj:
(def page-size 12)
(def images-path "/public/images/.....")
I need to move them to a config of some sort. Where is the best place? I'd like something simple and not to use any additional library on top on the ones that I'm already using which come with Luminus.
Luminus uses it's config library for configuration. You can put your configuration variables into appropriate config.edn files (per environment). Config values are available as a map stored in config.core/env. You can see an example in your <app>.core namespace:
(defn http-port [port]
;;default production port is set in
;;env/prod/resources/config.edn
(parse-port (or port (env :port))))
Ask yourself this question:
Would I ever want multiple deployments of my application where this setting differs?
If the answer to that question is "yes" then the configuration should be dictated by the environment running you application either by an edn file, Environ or other means.
If not... then you are talking about something I would categorize as an application constant, which helps avoiding magic numbers. In some situations it can improve readability by placing these in specific namespaces like so.
Constants namespace:
(ns my.app.constants)
(def page-size 12)
(def images-path "/public/images/.....")
Application:
(ns my.app.core
(:require [my.app.constants :as const)
;; now access the application constant like this
;; const/page-size
(defn image-url [image-name]
(str const/images-path "/" image-name))
I'm trying to truncate my dataomic database between tests. I see a lot of questions on google about excision and how to delete data, but none about just wiping a database clean.
I'm doing something along the following using core.test:
(with-redefs [db/uri "datomic:free://localhost:4334/test_om_asyn101_next"
db/conn (d/connect db/uri)]
(run-tests 'rtest.core-test))
I've been wiping out the DB by changing the URI and creating anew, but getting tired of doing that! Thanks!
There is a delete-database fn. See day of datomic tutorial for an example about how to create a new in memory db for each test.
Also, yeller has a nice example of how to use datomic's "what if" functionality to do unit testing.
I've been using (delete-database) as dAni suggests, with a clojure.test fixture:
(ns test-util)
(defn datomic-rollback-fixture [test-fn]
;; setup
(run-migrations)
;; run the tests
(test-fn)
;; clean up
(let [datomic-uri (get-in config/config [:datomic :uri])]
(when (string/starts-with? datomic-uri "datomic:mem:")
(d/delete-database datomic-uri))))
The (run-migrations) function loads our schema definitions from an EDN file, which we have defined via conformity. The bit about only destroying a datomic:mem database is because I'm paranoid about deleting databases out of production by accident.
Then in the test functions, we've got:
(use-fixtures :each test-util/datomic-rollback-fixture)
(deftest my-test ...)
Seems to work fine for me so far and is plenty fast enough with the memory database.
Being new to clojure, I would like some advice on implementing the repository pattern*.
In an OO language, I would create a repository interface, a test and at least one db impl. I would instantiate them during bootstrap, pass the objects around using DI or get them through a service locator. I'm guessing it's done completely different in clojure?
1) What is a good way of grouping the functions in the repo? Protocol, "free" functions in a namespace?
2) Where do I instantiate the repository back-end, i.e. allocate resources like db-connections etc.? Do I instantiate an implementation of the repository protocol and assign it to an atom, or in the case of free functions, redefine them?
*) A repository is a back-end abstraction for persistence and typically supports a CRUD style range of operations.
EDIT: Here is the approach I'm currently using. A protocol to group functions. A test and a "real" record implementing it. Then an atom to register the repo with.
(defprotocol GardenRepo
"Repository of Gardens. Supports CRUD style operations."
(list-gardens [repo] "Get a seq of all gardens in the repo.")
(get-garden [repo id] "Get a specific garden by it's id.")
...)
(let [repo (atom nil)]
(defn get-garden-repo [] #locator)
(defn set-garden-repo [new-repo] (reset! repo new-repo)))
1) Group functions by shared subproblems. In our ORM we have a namespace for db interaction, a separate namespace for each target db, a namespace for model construction and query operations, a namespace for field definition, a separate namespace describing each implementation of the field protocol (ie. int, string, text, slug, collection).
2) Use a function that returns all the functions used, each implicitly using resources defined by the passed in config, ie.:
(defn make-repository
[config]
(let [db (initialize-db config)
cleanup #(do-cleanup db)
store (fn [key val] (store-data key val db))
retrieve (fn [key] (retrieve-data key db))]
{:db db ;; this is optional, can be very useful during initial development
:cleanup cleanup
:store store
:retrieve retrieve}))
This could of course create an instance of a record if access of the functions is a performance bottleneck, and the record could implement a protocol if you want to define multiple implementations of the same functionality (different db drivers that may need different setup etc.). The user of the library decides how and where to bind these functions you return, as appropriate for their own design.
An example of how a client could use this repository:
(def repo (make-repository config))
(def cleanup-repo (:cleanup repo))
(def store-to-repo (:store repo))
(def retrieve-from-repo (:retrieve repo))
(defn store-item
[data]
(let [processed (process data)
key (generate-key data)]
(try (store-to-repo key data)
(catch Exception e
(cleanup-repo)))))
I've got the following problem:
I do understand that the concept of functional programming languages do not allow immutable variables, so classic variable assignment and usage like
var foo = bar();
after that in another function...
var baz = funnything(foo);
is not possible (correct me if it is, I'm still trying to get the whole idea and concepts behind functional programming).
I'm trying to play around a bit with Clojure and come to a point where I stuck:
I want to get a webpage and save the DOM into a variable for later usage.
The program starts, a prompt is spawning and the user can enter a URL which he want to fetch.
After the DOM of the website is fetched he can navigate through the DOM by for example getting the title, or all links or whatever.
The Problem:
Because I can't use a construct like
; More or less pseudo-code
(def page nil)
(get-url [url]
(page (browser/get url)))
(get-title
(println (html/select page [:title])))
I have to use the following construct:
(get-title [url]
(println (html/select (browser/get url) [:title])))
Due to this construct everytime I want to access an element of the site via get-title, get-links or whatever I have to download the whole webpage whenever the user is entering one of those methods on the command prompt.
Do I miss something here? Are there classical assigments in Clojure? Am I doing the complete thing wrong and should study a whole lot more functional programming concepts before getting hands-on and learning-by-doing?
You are misunderstanding assignment in clojure.
(page (browser/get url))
is not equivalent to: page = browser/get(url), it is equivalent to page(browser/get(url)).
I think you don't really need a global variable, just composable functions. Your example could look something like this:
(defn get-title [url]
(let [page (browser/get url)]
(println (html/select page [:title]))))
That and using memoize, like Joost mentioned, if you want to cache the return values of broser/get.
You can easily cache the value of (browser/get url) so you don't have to refetch the page every time you want to examine a property of it. See clojure.core/memoize. But that will only solve part of the problem.
In your particular case, your program requires some sort of global (or session scoped), mutable, concept of "the current page" or at least, "the current url", since that is what you're expecting your user to enter and modify. Clojure provides multiple constructs for managing mutable state.
Since you're only dealing with one page at a time, this is probably best modeled by using atoms. See http://clojure.org/atoms and http://clojuredocs.org/clojure_core/clojure.core/swap!
I'd like to run a session about Clojure. Could you recommend a problem which can be elegantly solved using functional programming with Clojure? Can you point to resources which cover this topic?
A lot of arguments for using Clojure seem to be related to its handling of concurrency, but I will not touch that issue here.
I will list some problems that I am forced to deal week in, week out with Java and how I have or would solve them in Clojure.
Immutability
In Java achieving immutability is very very hard. Besides following strict coding practices you will have to choose your frameworks and libraries very carefully. Also as a side-effect you will either write a lot of code to make a clean and usable API, or just force the client to deal with it.
final Person person = personDao.getById(id);
// I would like to "change" the person's email, but no setters... :(
In Clojure you model your data based on immutable data structures so all of your objects are immutable by default and due to this Clojure offers powerful functions which operate on these structures.
(let [person (get-by-id person-dao id)
person-with-email (assoc person :email email)]
; Use person-with-email...
Conversions
In Java you have a domain class Person with fields id, name, email, socialSecurityNumber and others. You are creating a web service for retrieving the names and emails of all the persons in your database. You do not want to expose your domain so you create a class PersonDto containing name and email. That was easy, so now you need a function to map the data from Person to PersonDto. Probably something like this:
public class PersonPopulator {
public PersonDto toPersonDto(Person person) {
return new PersonDto(person.getName(), person.getEmail());
}
public List<PersonDto> toPersonDtos(List<Person> persons) {
List<PersonDto> personDtos = new ArrayList<PersonDto>();
for (Person person : persons) {
personDtos.add(toPersonDto(person));
}
return personDtos;
}
}
Okay well that wasn't so bad, but what if you want to put more data in the DTO? Well the constructor code in toPersonDto will grow a bit, no worries. What if there are two different use cases, one as above and another where we want to send just the emails? Well we could leave the name null (bad idea) or create a new DTO, perhaps PersonWithEmailDto. So we would create a new class, a few new methods for populating the data... you probably see where this is going?
Clojure, a dynamically typed language with immutable data structures, allows me to do this:
(defn person-with-fields [person & fields]
(reduce #(assoc %1 %2 (get person %2)) {} fields))
(person-with-fields {:id 1
:name "John Doe"
:email "john.doe#gmail.com"
:ssn "1234567890"} :name :email)
; -> {:email "john.doe#gmail.com", :name "John Doe"}
And to manipulate a list of persons:
(map #(person-with-fields % :name :email) persons)
Also adding ad hoc data to a person would be easy:
(assoc person :tweets tweets)
And this would break nothing. In Java if your objects are immutable they probably don't have setters, so you would have to write a lot of boilerplate just to modify one field (new Person(oldPerson.getName(), oldPerson.getEmail(), tweets)), or create a totally new class. Mutable objects offer a nice API (oldPerson.setTweets(tweets)), but are difficult to test and understand.
Testing
A lot of Java code is based on some state even when there is no need for it. This means you can either mock this state, which usually means additional boilerplate and gets harder if you have not created your code with testability in mind. On the other hand tests without mocks are usually slow and depend on database access or time or something else that will surely fail on you from time to time.
While coding Clojure I have noticed that I do not actually need state that much. Pretty much the only situations are when I am retrieving something from the "outside", be it a database or some web service.
Summary
My code is a pipe, from one end I get some data, this data is then changed in the pipe via filtering, transforming or joining it with some other data until it reaches the end of the pipe. Inside the pipe there is no need to really change the data, but a lot of cases where powerful functions and immutable data structures are useful and this is why Clojure works wonders with code like this.
The classic concurrency problem "The Sleeping Barber" might be good.
Here is a couple of examples
http://www.bestinclass.dk/index.clj/2009/09/scala-vs-clojure-round-2-concurrency.html
https://github.com/bitsai/clojure-actors/blob/master/sleeping_barber.clj
A couple of months ago I run into the same problem and we decided to solve 'the Mona Lisa' problem in Clojure. The result was this presentation. Basically we showed that Clojure is extremely cool for solving problems for which 'code as data' gives an elegant solution. For example in genetic algorithms.