I've got the following problem:
I do understand that the concept of functional programming languages do not allow immutable variables, so classic variable assignment and usage like
var foo = bar();
after that in another function...
var baz = funnything(foo);
is not possible (correct me if it is, I'm still trying to get the whole idea and concepts behind functional programming).
I'm trying to play around a bit with Clojure and come to a point where I stuck:
I want to get a webpage and save the DOM into a variable for later usage.
The program starts, a prompt is spawning and the user can enter a URL which he want to fetch.
After the DOM of the website is fetched he can navigate through the DOM by for example getting the title, or all links or whatever.
The Problem:
Because I can't use a construct like
; More or less pseudo-code
(def page nil)
(get-url [url]
(page (browser/get url)))
(get-title
(println (html/select page [:title])))
I have to use the following construct:
(get-title [url]
(println (html/select (browser/get url) [:title])))
Due to this construct everytime I want to access an element of the site via get-title, get-links or whatever I have to download the whole webpage whenever the user is entering one of those methods on the command prompt.
Do I miss something here? Are there classical assigments in Clojure? Am I doing the complete thing wrong and should study a whole lot more functional programming concepts before getting hands-on and learning-by-doing?
You are misunderstanding assignment in clojure.
(page (browser/get url))
is not equivalent to: page = browser/get(url), it is equivalent to page(browser/get(url)).
I think you don't really need a global variable, just composable functions. Your example could look something like this:
(defn get-title [url]
(let [page (browser/get url)]
(println (html/select page [:title]))))
That and using memoize, like Joost mentioned, if you want to cache the return values of broser/get.
You can easily cache the value of (browser/get url) so you don't have to refetch the page every time you want to examine a property of it. See clojure.core/memoize. But that will only solve part of the problem.
In your particular case, your program requires some sort of global (or session scoped), mutable, concept of "the current page" or at least, "the current url", since that is what you're expecting your user to enter and modify. Clojure provides multiple constructs for managing mutable state.
Since you're only dealing with one page at a time, this is probably best modeled by using atoms. See http://clojure.org/atoms and http://clojuredocs.org/clojure_core/clojure.core/swap!
Related
Context
I am writing an app using Clojure and following the functional programming paradigm. In this app I have two HTTP endpoints: /rank and /invite. In /rank the app ranks a list of costumers based on their scores. In /invite the app receives an invite from one costumer to another and this should yield changes on the scores of some costumers.
Problem
Data from costumers are kept in a single vector of maps called record.
Putting aside referential transparency for a moment, record should be a shared resource between the endpoints, one reads it and uses it in a ranking function to respond an HTTP request and the other reads it and updates scores in it.
Now, with functional programming in mind, record cannot be updated, so the /invite endpoint should read it and return a new record', problem is, /rank endpoint is setup to use record, but when a new record' is generated it should use it instead of the original one.
My ideas for solving this
I understand that in this context the whole app cannot be fully, functionally speaking, pure. It reads initial input from file and receive requests from the external environment, all of which renders functions that deal with these parts not referentially transparent. And almost every program will have these small portions of non-functional code, but in an attempt of trying not to add more of these non-functional functions to the app and this app being just to exercise some functional programming, I am not persisting record to a database or something, because if this were the case, problem would be solved as I could just call a function to update record on the database.
My best idea so far is: The endpoints are created with a routes function from Compojure, so in the /invite endpoint I should process the new record' vector, and recreate the /rank endpoint supplying it with record'. This part of recreating /rank is what I am struggling at, I am trying to call routes again and define again all the endpoints in the hope that it will override the original call of routes, but as expected, as I believe Compojure follows functional programming, once created, the routes are immutable and a new call of routes won't override anything, it will just create new routes that will be left in the void, not attached to the HTTP requests.
So, is it possible to do what I want with pure functional code? Or it is unavoidable to break referential transparency and I should persist record to a file or database to update it?
PS.: I don't know if this is relevant, but I am new to Clojure and to any sort of web interaction.
Clojure (in contrast to Haskell) is not pure and has its own constructs to manage changes to shared state. It doesn't isolate impureness using a type system (like IO monad in Haskell) but promotes using pure functions and managing the state with different types of references (atom, agent, ref) defining a clear semantics how and when the state is changed.
For your scenario Clojure's atom would be the simplest solution. It provides a clear contract on how its state is managed.
I would create a var holding your record within an atom:
(def record (atom [])) ;; initial record is empty
Then in your rank endpoint you could use its value using deref or with # as a syntactic sugar:
(GET "/rank" []
(calculate-rank #record))
Whereas your invite endpoint could update your record value atomically using swap!:
(POST "/invite/:id" [id]
(invite id)
(swap! record calculate-new-rank id)
(create-response))
Your calculate-new-rank function would look like that:
(defn calculate-new-rank [current-record id]
;; do some calculations
;; create a new record value and return it
(let [new-record ...]
new-record))
Your function will be called with the current version of the data stored in the atom and other optional parameters and the result of your function will be installed as the new value of your atom.
This may actually be a bit of an XY-problem, so I'll try to explain what the goal is first.
I'm building a ClojureScript application which is composed of a set of Reagent components. It provides a user interface where you can dynamically add or remove UI elements. These UI elements (components) have a certain type. For example a Markdown component is-a Text component. Whenever the user is presented with the option to add Text we list all the components that match the type+descendants (in this case Markdown, there could be others).
The way I've coded it up is as follows.
Each component is in its own namespace, this namespace contains a builder function that returns the new component. At the root of the namespace it also calls (derive ::type ::parent)
now in some different namespace we require and enumerate all of these components in a map like:
(ns app.components
(:require
[app.gui.markdown :as markdown]
[app.gui.study-list :as study-list]))
(def all
{markdown/t markdown/builder
study-list/t study-list/builder})
Where the /t refers to the namespace-qualified keyword which was used to define the hierarchy. We use the all map to provide the data for the menu's that face the user (which components can be added, filtered by type).
Now, as you can imagine, this isn't pretty. Since it must now maintain such a (potentially) long list of all the types in the hierarchy manually.
Rather I would do something like (def all (components-of (descendants ::root))) but I'm unsure how to tackle this, since I think it would require finding vars by name (not supported in ClojureScript).
So my question is: how do you maintain a map or list of namespaces + vars (dynamically) in ClojureScript?
You cannot do this dynamically, but as far as I can tell for your problem there isn't much need. ClojureScript macros can reflect back into the compiler - you could easily use the cljs.analyzer.api namespace to figure out which vars you need (through var metadata or whatever) and automatically emit the runtime info map you want. This is in fact very similar to how cljs.test/run-tests works. It uses the compiler to filter out all vars in all namespaces with :test metadata attached and it generates the code to test each one. It's worth examining cljs.test in detail to see how this can be done.
EDIT: is what I want basically load-string?
Question
In Clojure, if I do:
(require :reload 'foo.bar)
then Clojure looks for src/foo/bar.clj, and reloads it.
Now, I want to do something like this:
(reload-from-string 'foo.bar STR)
the semantics of this would be: reload namespace 'foo.bar, but instead of compiling src/foo/bar.clj, compile STR instead.
How do I define reload-from-string?
Context
I need to hot reload code on a server that is running an Clojure application. I don't want to have to continuously shuffle files back & forth to the server (either via scp, sftp, or fuse/sshfs) in order to reload. Thus, I would prefer to just pass it a string.
Thanks!
You can use read-string and then eval. Keep in mind the risks though. An advantage of splitting them up is you can whitelist what is present in the resulting list before evaling it.
You probably want to bind *read-eval* to false also.
I've just started learning Clojure and I'm attempting to write something to pull some URL's from a web page into memory with some additional meta data around each URL.
I can get the URLs but it's storing them somewhere that I'm having an issue with. I've figured out I need a vector with nested maps, which I can then add new records to with assoc-in, however since I don't know the URL's I'm not sure how I go about defining my data structure initially.
So for example:
(def *comics*
[{:name "Penny-Arcade"
:url "http://www.penny-arcade.com/comic/"
:working 0
}
{:name "We The Robots"
:url "http://www.wetherobots.com/"
:working 0
}])
I'm just not sure how I start the above data structure with no data in it, then add it say initially from a command line arg, then the rest from the website.
If someone can suggest a better way that the above to store the data, please feel free.
I take it you want to modify your *comics* var from some command line argument - and then modify it even more while "working" on the elements in it.
I would suggest you don't do that.
There doesn't seem to be any reason you can't take the comics urls from the command line and pass them as arguments to a function that does the processing and returns whatever you want from those urls. Doing it that way - that is; functionally, without mutating vars - will definitely be easier to implement in clojure, easier to parallellize and just all out more idiomatic and fun.
I'd like to run a session about Clojure. Could you recommend a problem which can be elegantly solved using functional programming with Clojure? Can you point to resources which cover this topic?
A lot of arguments for using Clojure seem to be related to its handling of concurrency, but I will not touch that issue here.
I will list some problems that I am forced to deal week in, week out with Java and how I have or would solve them in Clojure.
Immutability
In Java achieving immutability is very very hard. Besides following strict coding practices you will have to choose your frameworks and libraries very carefully. Also as a side-effect you will either write a lot of code to make a clean and usable API, or just force the client to deal with it.
final Person person = personDao.getById(id);
// I would like to "change" the person's email, but no setters... :(
In Clojure you model your data based on immutable data structures so all of your objects are immutable by default and due to this Clojure offers powerful functions which operate on these structures.
(let [person (get-by-id person-dao id)
person-with-email (assoc person :email email)]
; Use person-with-email...
Conversions
In Java you have a domain class Person with fields id, name, email, socialSecurityNumber and others. You are creating a web service for retrieving the names and emails of all the persons in your database. You do not want to expose your domain so you create a class PersonDto containing name and email. That was easy, so now you need a function to map the data from Person to PersonDto. Probably something like this:
public class PersonPopulator {
public PersonDto toPersonDto(Person person) {
return new PersonDto(person.getName(), person.getEmail());
}
public List<PersonDto> toPersonDtos(List<Person> persons) {
List<PersonDto> personDtos = new ArrayList<PersonDto>();
for (Person person : persons) {
personDtos.add(toPersonDto(person));
}
return personDtos;
}
}
Okay well that wasn't so bad, but what if you want to put more data in the DTO? Well the constructor code in toPersonDto will grow a bit, no worries. What if there are two different use cases, one as above and another where we want to send just the emails? Well we could leave the name null (bad idea) or create a new DTO, perhaps PersonWithEmailDto. So we would create a new class, a few new methods for populating the data... you probably see where this is going?
Clojure, a dynamically typed language with immutable data structures, allows me to do this:
(defn person-with-fields [person & fields]
(reduce #(assoc %1 %2 (get person %2)) {} fields))
(person-with-fields {:id 1
:name "John Doe"
:email "john.doe#gmail.com"
:ssn "1234567890"} :name :email)
; -> {:email "john.doe#gmail.com", :name "John Doe"}
And to manipulate a list of persons:
(map #(person-with-fields % :name :email) persons)
Also adding ad hoc data to a person would be easy:
(assoc person :tweets tweets)
And this would break nothing. In Java if your objects are immutable they probably don't have setters, so you would have to write a lot of boilerplate just to modify one field (new Person(oldPerson.getName(), oldPerson.getEmail(), tweets)), or create a totally new class. Mutable objects offer a nice API (oldPerson.setTweets(tweets)), but are difficult to test and understand.
Testing
A lot of Java code is based on some state even when there is no need for it. This means you can either mock this state, which usually means additional boilerplate and gets harder if you have not created your code with testability in mind. On the other hand tests without mocks are usually slow and depend on database access or time or something else that will surely fail on you from time to time.
While coding Clojure I have noticed that I do not actually need state that much. Pretty much the only situations are when I am retrieving something from the "outside", be it a database or some web service.
Summary
My code is a pipe, from one end I get some data, this data is then changed in the pipe via filtering, transforming or joining it with some other data until it reaches the end of the pipe. Inside the pipe there is no need to really change the data, but a lot of cases where powerful functions and immutable data structures are useful and this is why Clojure works wonders with code like this.
The classic concurrency problem "The Sleeping Barber" might be good.
Here is a couple of examples
http://www.bestinclass.dk/index.clj/2009/09/scala-vs-clojure-round-2-concurrency.html
https://github.com/bitsai/clojure-actors/blob/master/sleeping_barber.clj
A couple of months ago I run into the same problem and we decided to solve 'the Mona Lisa' problem in Clojure. The result was this presentation. Basically we showed that Clojure is extremely cool for solving problems for which 'code as data' gives an elegant solution. For example in genetic algorithms.