I've just started learning Clojure and I'm attempting to write something to pull some URL's from a web page into memory with some additional meta data around each URL.
I can get the URLs but it's storing them somewhere that I'm having an issue with. I've figured out I need a vector with nested maps, which I can then add new records to with assoc-in, however since I don't know the URL's I'm not sure how I go about defining my data structure initially.
So for example:
(def *comics*
[{:name "Penny-Arcade"
:url "http://www.penny-arcade.com/comic/"
:working 0
}
{:name "We The Robots"
:url "http://www.wetherobots.com/"
:working 0
}])
I'm just not sure how I start the above data structure with no data in it, then add it say initially from a command line arg, then the rest from the website.
If someone can suggest a better way that the above to store the data, please feel free.
I take it you want to modify your *comics* var from some command line argument - and then modify it even more while "working" on the elements in it.
I would suggest you don't do that.
There doesn't seem to be any reason you can't take the comics urls from the command line and pass them as arguments to a function that does the processing and returns whatever you want from those urls. Doing it that way - that is; functionally, without mutating vars - will definitely be easier to implement in clojure, easier to parallellize and just all out more idiomatic and fun.
Related
I'm writing a simple SPA game in ClojureScript and I want to save the game state, which is a map, to a cookie and read it back.
I'm successfully saving the cookie, though it turns my map to a string.
When I read it back, I get Error: No protocol method IAssociative.-assoc defined for type string:(then it outputs the map)
What's the simplest and best way to use maps with cookies?
I have tried https://github.com/reagent-project/reagent-utils and https://github.com/Quantisan/cljs-cookies, but I get similar issues.
I think you're looking for "cljs.core/read-string cljs.core/pr-str" functions. You need to serialize your Clojure objects to string and read it back to store it in a cookie. Functions above will be able to serialize all standard Clojure(Script) data structures for you. And then if you have something custom, this function can come to rescue: https://cljs.github.io/api/cljs.reader/register-tag-parserBANG.
I'm trying to wrap my head around the difference between a "collection" and a "store" in REST. From what I've read so far,
a collection is:
"a server-managed directory of resources"
and a store is a:
"client-managed resource repository"
I found this post: How "store" REST archetype isn't creating a new resource and a new URI?
but it didn't really help me clarify the difference. I mean, I understand one is controlled by the server and the other by the client... but can someone give me a concrete example of what a store might be in a real world application?
I *think it's something like this:
GET http://myrestapplication.com/widgets/{widget_id} -- retrieves a widget from db
POST http://myrestapplication.com/widgets/{widget_id} -- creates a new widget from db
PUT http://myrestapplication.com/widgets/{widget_id},[list of updated parms & their vals] -- update widget
PUT http://myrestapplication.com/users/johndoe/mywishlist/{widget_id} -- updates john doe's profile to add a widget that already exists in the database... but links to it as his favorite one or one that he wants to buy
Is this correct?
if so, could the last PUT also be expressed as a POST somehow?
EDIT 1
I found an online link to the book i'm reading... where it makes the distinction between the two:
https://books.google.ca/books?id=4lZcsRwXo6MC&pg=PA16&lpg=PA16&dq=A+store+is+a+client-managed+resource+repository.+A+store+resource+lets+an+API+client:+put+resources+in,+get+them+back+out,+and+decide+when+to+delete+them&source=bl&ots=F4CkbFkweL&sig=H6eKZMPR_jQdeBZkBL1h6hVkK_E&hl=en&sa=X&ei=BB-vVJX6HYWvyQTByYHIAg&ved=0CB0Q6AEwAA#v=onepage&q=A%20store%20is%20a%20client-managed%20resource%20repository.%20A%20store%20resource%20lets%20an%20API%20client%3A%20put%20resources%20in%2C%20get%20them%20back%20out%2C%20and%20decide%20when%20to%20delete%20them&f=false
REST uses http verbs to manipulate resources. Full-Stop. That's it. To build some classes of browser-based application developers sometimes use local storage (a store), but that has absolutely nothing to do with REST (in fact, it's the opposite). Collections are a special consideration in REST-based API design because the REST principles place considerable constraints on how they are represented in the results of your queries -- special consideration also because there are no standards on how these things should be represented and access if you're using anything other than html as a resource type.
Edit:
REST suggests that when we ask for a resource we receive that resource and only that resource and that things referenced by that resource are returned as links, not as data. This mimics the http standard by which we return the requested page and links to other pages rather than embedding linked pages. So, our resources should return links to related resources, not the resources themselves.
So, what about collections?
Let's use as an example a college management system that has Course objects each of which contains a huge lists of Students.
When I GET the course I don't want to have the collection of students returned as an embedded list, because that could be huge and because my user might not be interested. Instead, I want to know that the course has a students collection and I want to be able to query that collection separately (when I need to) and I want to be able to page it on demand. For this to work, the course needs to link to the students collection URL (maybe with an appropriate type so that my code knows how to handle the link). Then, I want to use the given collection's url to request a paged list of resources. In this example, the collection's url could be something like: course/1/students, with the convention that I can add paging info to the search string to constrain the results with something like course/1/students?page=1&count=10. Embedding the students collection into the course resource would be a violation of REST. I would not be returning a course, I'd be returning course-and-students.
I've got the following problem:
I do understand that the concept of functional programming languages do not allow immutable variables, so classic variable assignment and usage like
var foo = bar();
after that in another function...
var baz = funnything(foo);
is not possible (correct me if it is, I'm still trying to get the whole idea and concepts behind functional programming).
I'm trying to play around a bit with Clojure and come to a point where I stuck:
I want to get a webpage and save the DOM into a variable for later usage.
The program starts, a prompt is spawning and the user can enter a URL which he want to fetch.
After the DOM of the website is fetched he can navigate through the DOM by for example getting the title, or all links or whatever.
The Problem:
Because I can't use a construct like
; More or less pseudo-code
(def page nil)
(get-url [url]
(page (browser/get url)))
(get-title
(println (html/select page [:title])))
I have to use the following construct:
(get-title [url]
(println (html/select (browser/get url) [:title])))
Due to this construct everytime I want to access an element of the site via get-title, get-links or whatever I have to download the whole webpage whenever the user is entering one of those methods on the command prompt.
Do I miss something here? Are there classical assigments in Clojure? Am I doing the complete thing wrong and should study a whole lot more functional programming concepts before getting hands-on and learning-by-doing?
You are misunderstanding assignment in clojure.
(page (browser/get url))
is not equivalent to: page = browser/get(url), it is equivalent to page(browser/get(url)).
I think you don't really need a global variable, just composable functions. Your example could look something like this:
(defn get-title [url]
(let [page (browser/get url)]
(println (html/select page [:title]))))
That and using memoize, like Joost mentioned, if you want to cache the return values of broser/get.
You can easily cache the value of (browser/get url) so you don't have to refetch the page every time you want to examine a property of it. See clojure.core/memoize. But that will only solve part of the problem.
In your particular case, your program requires some sort of global (or session scoped), mutable, concept of "the current page" or at least, "the current url", since that is what you're expecting your user to enter and modify. Clojure provides multiple constructs for managing mutable state.
Since you're only dealing with one page at a time, this is probably best modeled by using atoms. See http://clojure.org/atoms and http://clojuredocs.org/clojure_core/clojure.core/swap!
EDIT: is what I want basically load-string?
Question
In Clojure, if I do:
(require :reload 'foo.bar)
then Clojure looks for src/foo/bar.clj, and reloads it.
Now, I want to do something like this:
(reload-from-string 'foo.bar STR)
the semantics of this would be: reload namespace 'foo.bar, but instead of compiling src/foo/bar.clj, compile STR instead.
How do I define reload-from-string?
Context
I need to hot reload code on a server that is running an Clojure application. I don't want to have to continuously shuffle files back & forth to the server (either via scp, sftp, or fuse/sshfs) in order to reload. Thus, I would prefer to just pass it a string.
Thanks!
You can use read-string and then eval. Keep in mind the risks though. An advantage of splitting them up is you can whitelist what is present in the resulting list before evaling it.
You probably want to bind *read-eval* to false also.
I'd like to run a session about Clojure. Could you recommend a problem which can be elegantly solved using functional programming with Clojure? Can you point to resources which cover this topic?
A lot of arguments for using Clojure seem to be related to its handling of concurrency, but I will not touch that issue here.
I will list some problems that I am forced to deal week in, week out with Java and how I have or would solve them in Clojure.
Immutability
In Java achieving immutability is very very hard. Besides following strict coding practices you will have to choose your frameworks and libraries very carefully. Also as a side-effect you will either write a lot of code to make a clean and usable API, or just force the client to deal with it.
final Person person = personDao.getById(id);
// I would like to "change" the person's email, but no setters... :(
In Clojure you model your data based on immutable data structures so all of your objects are immutable by default and due to this Clojure offers powerful functions which operate on these structures.
(let [person (get-by-id person-dao id)
person-with-email (assoc person :email email)]
; Use person-with-email...
Conversions
In Java you have a domain class Person with fields id, name, email, socialSecurityNumber and others. You are creating a web service for retrieving the names and emails of all the persons in your database. You do not want to expose your domain so you create a class PersonDto containing name and email. That was easy, so now you need a function to map the data from Person to PersonDto. Probably something like this:
public class PersonPopulator {
public PersonDto toPersonDto(Person person) {
return new PersonDto(person.getName(), person.getEmail());
}
public List<PersonDto> toPersonDtos(List<Person> persons) {
List<PersonDto> personDtos = new ArrayList<PersonDto>();
for (Person person : persons) {
personDtos.add(toPersonDto(person));
}
return personDtos;
}
}
Okay well that wasn't so bad, but what if you want to put more data in the DTO? Well the constructor code in toPersonDto will grow a bit, no worries. What if there are two different use cases, one as above and another where we want to send just the emails? Well we could leave the name null (bad idea) or create a new DTO, perhaps PersonWithEmailDto. So we would create a new class, a few new methods for populating the data... you probably see where this is going?
Clojure, a dynamically typed language with immutable data structures, allows me to do this:
(defn person-with-fields [person & fields]
(reduce #(assoc %1 %2 (get person %2)) {} fields))
(person-with-fields {:id 1
:name "John Doe"
:email "john.doe#gmail.com"
:ssn "1234567890"} :name :email)
; -> {:email "john.doe#gmail.com", :name "John Doe"}
And to manipulate a list of persons:
(map #(person-with-fields % :name :email) persons)
Also adding ad hoc data to a person would be easy:
(assoc person :tweets tweets)
And this would break nothing. In Java if your objects are immutable they probably don't have setters, so you would have to write a lot of boilerplate just to modify one field (new Person(oldPerson.getName(), oldPerson.getEmail(), tweets)), or create a totally new class. Mutable objects offer a nice API (oldPerson.setTweets(tweets)), but are difficult to test and understand.
Testing
A lot of Java code is based on some state even when there is no need for it. This means you can either mock this state, which usually means additional boilerplate and gets harder if you have not created your code with testability in mind. On the other hand tests without mocks are usually slow and depend on database access or time or something else that will surely fail on you from time to time.
While coding Clojure I have noticed that I do not actually need state that much. Pretty much the only situations are when I am retrieving something from the "outside", be it a database or some web service.
Summary
My code is a pipe, from one end I get some data, this data is then changed in the pipe via filtering, transforming or joining it with some other data until it reaches the end of the pipe. Inside the pipe there is no need to really change the data, but a lot of cases where powerful functions and immutable data structures are useful and this is why Clojure works wonders with code like this.
The classic concurrency problem "The Sleeping Barber" might be good.
Here is a couple of examples
http://www.bestinclass.dk/index.clj/2009/09/scala-vs-clojure-round-2-concurrency.html
https://github.com/bitsai/clojure-actors/blob/master/sleeping_barber.clj
A couple of months ago I run into the same problem and we decided to solve 'the Mona Lisa' problem in Clojure. The result was this presentation. Basically we showed that Clojure is extremely cool for solving problems for which 'code as data' gives an elegant solution. For example in genetic algorithms.