I've had a problem, now it's gone, I know what fixed it but I have no idea why. I'm pretty new to clojure, but it seems to me that some namespace names will just not work, and I don't mean obvious ones that are already taken by another symbol.
I have a rather simple project to implement a scheduler, and it has three namespaces: schedule.core, schedule.jdate, and schedule.validation.
The validation namespace worked, but for the jdate namespace I always received errors like this:
CompilerException java.lang.Exception: No namespace: schedule.jdate,
compiling:(schedule/core.clj:1:1)
The errors varied slightly, depending on the way I was currently trying to solve the issue, but basically always meant the same: There was no namespace with that name, in one instance the error even said that there was no jdate__init (which I could believe) AND no jdate.clj, although the file was right there in the folder.
My namespace declarations and inclusions looked like this:
src/schedule/core.clj:
(ns schedule.core
(:gen-class)
(:require schedule.jdate)
(:refer schedule.jdate)
(:require schedule.validation)
(:refer schedule.validation))
src/schedule/jdate.clj:
(ns schedule.jdate
(:gen-class)
(:import java.util.Date)
(:import java.util.Calendar))
src/schedule/validation.clj:
(ns schedule.validation
(:gen-class))
As I said, the validation namespace worked without trouble.
Eventually I ran out of ideas and started to try things that couldn't possibly have any relation, among other things I renamed the jdate namespace to scheduledate. There was no change in the behavior.
Some more time later, I got to the "I'm not even trying anymore" stage, in which I renamed the jdate namespace to "murks" (a swiss word conveying the meaning of "I need a larger hammer for this to work") and... it worked.
I thought there must be something somewhere else in the code I changed inadvertantly that suddenly made it work. So I changed the name back to jdate. It stopped working immediately.
In the end, I'm somewhat confused. Could anybody here tell me a) why the first two names didn't work and the third one did, and b) by extension what names are save to use for namespaces, because I'm not looking forward to the feeling of playing minesweeper when naming them in the future.
EDIT: It seems that the problem stemmed from multiple conspiring sources. For one, the structure of my code required a two-way dependency of two namespaces. Since I was not very familiar with how Clojure works, I assumed a c-style include hierarchy, where files included later in the same tree had all the includes up to that point at their disposal. I am somewhat surprised at the nature of the received errors, but then again, I should know by now that Clojure error messages are almost indecipherable and tend to be highly misleading.
The inconsistent behavior seems to have originated from me frantically trying to reload different namespaces without closing the REPL in-between... i.e. the results of the REPL-operations did not always reflect the state of the code.
Oh well, that was a rather tedious lesson in "Clojure really hates circular includes", but I think it's one I've learned now.
Related
So here is my problem,
I have a file sample.clj in namespace abc.xyzin which i am calling db init method directly before doing any CRUD operation.
(db/init some-config)
When i require this namespace abc.xyz in a test namespace cdf.qpr as (:require [abc.xyz :refer :all]), db is already being initialized with some-config but i want to initialize it with some-other-config.
How can i require namespace preventing call of db/init
Without changing the code in namespace abc.xyz, in particular changing the line (db/init some-config), what you ask is not possible.
A more productive approach would be to ask "How do other people handle this situation, without having a top level expression like (db/init some-config) in their namespaces? I suspect most put calls like that not as top level expressions in their namespaces, but inside some init function for their entire application.
My Problem
I'm currently writing a REST-API which is supposed to take JSON requests and work with an intern library we use. The main usage will be to either run the server with a web interface or to possibly use another language to work with the API since Clojure isn't common elsewhere.
In order to achieve this, the JSON request contains data and a functionname, which is run with resolve, since I'm supposed to make it so that we don't have to change the API each time a function is added/removed.
Now the actual question is: How can I make sure the function I run combined with it's argument dosen't destroy the whole thing?
So, what did I try already?
Now, I've actually told only half the truth until now: I don't use resolve, I use ns-resolve. My first intuition was to create a seperate file which will load in all namespaces from the library, there's nothing malicious you could do with those. The problem is, I want only those functions and I'm not aware of any way to remove clojure.core functions. I could do a blacklist for those but whitelisting would be a whole lot easier. Not to mention I could never find all core functions I actually should be blacklisting.
The oher thing is the input.
Again I've got a basic idea which is to sanitize the input to replace all sort of brackets just to make sure the input isn't other clojure code which would just bypass the namespace restriction from above. But would this actually be enough? I've got not much experience in breaking things.
Another concern I've heard is that some functions could run the input as argument long before intended. The server works with ring and its JSON extension.
JSON should only give strings, numbers, booleans and nil as atomic data types. I conclude each possible malicious input should be a string at my end - besides resolve, is there any function which could have the side effect of running such input?
Since they are string: Is there even a concern to be had with the data at all?
I would strongly advise to use a whitelisting approach for functions, and not to evaluate anything else.
You could maybe add a metadata flag to the exposed functions that you check where you resolve them.
Everything else should just be data; don't evaluate it.
Probably you want to look into the following:
How to determine public functions from a given namespace. This will give you a list of the valid functions names that your API can accept as part of the input. Here's a sample:
user=> (ns-publics (symbol "clojure.string"))
{ends-with? #'clojure.string/ends-with?, capitalize #'clojure.string/capitalize, reverse #'clojure.string/reverse, join #'clojure.string/join, replace-first #'clojure.string/replace-first, starts-with? #'clojure.string/starts-with?, escape #'clojure.string/escape, last-index-of #'clojure.string/last-index-of, re-quote-replacement #'clojure.string/re-quote-replacement, includes? #'clojure.string/includes?, replace #'clojure.string/replace, split-lines #'clojure.string/split-lines, lower-case #'clojure.string/lower-case, trim-newline #'clojure.string/trim-newline, upper-case #'clojure.string/upper-case, split #'clojure.string/split, trimr #'clojure.string/trimr, index-of #'clojure.string/index-of, trim #'clojure.string/trim, triml #'clojure.string/triml, blank? #'clojure.string/blank?}
You probably want to use the keys from the map above (in the namespace that applies to your use case) to validate the input, because you can "escape" the ns-resolve namespace if you fully qualify the function name:
user=> ((ns-resolve (symbol "clojure.string") (symbol "reverse")) "hello")
"olleh"
user=> ((ns-resolve (symbol "clojure.string") (symbol "clojure.core/reverse")) "hello")
(\o \l \l \e \h) ;; Called Clojure's own reverse, probably you don't want to allow this
Now, with that being said, I'm going to offer you some free advice:
I'm supposed to make it so that we don't have to change the API each time a function is added/removed
If you have watched some of Rich Hickey's talks you'll know that API changes are a sensible topic. In general you should think carefully before adding new functions or thinking of deleting any, because it sounds like your team is willing to cut corners on getting clients of the API together on the same page.
Unless your clients can discover dynamically what functions are available (maybe you'll expose some API?), it sounds like you will be open to receiving requests you cannot fulfill because the functions have changed or could be removed.
This question already has answers here:
Why is "using namespace std;" considered bad practice?
(41 answers)
Closed 8 years ago.
using namespace std;
So far in my computer science courses, this is all we have been told to do. Not only that, but it's all that we have been allowed to do, otherwise we get penalized on our code. I understand, through looking at code posted online, that you can use ::std or std:: to accomplish the same thing.
My question is generall why? Obviously for the sake of learners and simplicity using the global declaration is simpler, but what are the draw backs? Is it more realistic to expect ::std in a real world application? And I guess to add on to this, what is the logic/concept behind the using declaration? None of this has been explained during my courses, and I'd like to get a better grasp on it.
As a general question: if I haven't been taught this content, vectors, templates, classes, or error handling does it seem like I'm missing a lot of essential C++ functionality?
Thanks in advance!
This is really one of those things that you can discuss over beer for hours and hours, and still not have an answer everyone is happy with.
If you are nearly always using std:: functionality, then adding using namespace std; at the beginning of the file is not that bad an idea. On the other hand, if you are using things from more than one namespace (e.g. writing a compiler using llvm:: and also using std::, it may get confusing as to which parts are part of llvm and which parts are std:: - so in my compilter project, I don't have a single file with using namespace ...; - instead I write out llvm:: and std:: as needed. There are several functions (unwisely, perhaps) called Type(), and some places use something->Type()->Type() to get the type-thing that I need... Yes, it confuses even me a little at times...
I also have many things that look like Constants::ConstDecl and Token::RightParen, so that I can quickly see "what is what". All of these COULD be made shorter and "simpler", but I prefer to see where things belong most of the time.
Being more verbose helps making it easier to see where things belong - but it makes for more typing and more reading, so it is a balance.
I'd say that generally, you do not declare the use of std globally. I guess if you're making a simple application, that would suffice. However, when you work in a large organization you often times have different namespaces that are used, and those might have overlapping objects. If you have a function in std, and in a namespace that you created, and then call "using namespace std" AND "using namespace yournamespace", you'll get unwanted results when calling that function. When you prefix every call with the namespace, it keeps it clearer and doesn't give issues with overlap.
general why?
Naming things is one of the more difficult aspects of software development. Beginners simply have little idea of how their name choices might generate ambiguities later on.
In particular, our software jargon often has preferred terms for certain issues. These preferences can cause unrelated class instances to be developed with the same (or similar) symbol with similar meanings.
Some of my often used symbols include init(), exec(), load(), store(), and I use timeStampGet() lots of places. I also use open(), close(), send()/recv() or write()/read().
So, I could rename init() in each of the 3 name spaces, and 5 objects into which I have added it, but it is much simpler to specify which one I want.
I find exec() in 2 name spaces and 12 objects. And there are 3 different timeStampGet() methods that I use. Whether namespaces or functions or class methods, these symbols make sense to me.
Furthermore, I find the 5 char "std::" namespace-as-prefix completely natural, and much preferable to the global "using namespace std". I suppose this comes with practice.
One more item - any where a larger name space or class name becomes tiresome, I sometimes add a typedef short name ... here are some examples from production code:
typedef ALARM_HISTORY ALM_HST;
typedef MONITOR_ITEM MI
typedef BACKUP_CONTROL BC;
On one team, we agreed to use well defined 'full' names, which, occasionally, became tiresome because of the length. Later in the project, we agreed that typedefs (for short class or namespace names) could be used when they were simple and added no confusion.
Personally I hate 'using' declarations. To me they make code unreadable and you break namespaces. I've spent 20 years as a maintenance programmer and I hate anything that make the code harder to read - in my mind using is as useless as throw specifications.
What is more readable 6months - a year - 10 years down the line
UDP::Socket sock
sock.send(data)
TCP::Socket sock2
sock2.send(data)
vs
using UDP;
using TCP;
sock.send(data)
sock2.end(data)
I also don't like namespace aliases
using namespace po = boost::program_options;
Now you are making the next programmer work harder with an extra level of indirection looking up what po is compared to boost::program_options. Same goes for those horrible typedef of
typedef long QUADWORD;
What size is a quadword - how long is a long 4 bytes? 8 bytes maybe 17 bytes on my OS
My last take is if you cannot type then don't be a programmer - a saved keystroke != good maintainable code
I'm familiar with packages from e.g. Java and Lisp, but what I'm seeing in other people's code is some apparent idioms like calling the entry point '-main' and using a backtick for the namespace name in (in-ns `foo), that kind of thing. I don't see these spelled out anywhere in docs or tutorials. Is there some resource which explains such conventions around structuring programs?
Edit:
I think I must have picked up the backtick thing from this answer: Splitting a Clojure namespace over multiple files, which I stumbled across while trying to make sense of (defn -main ...). Given the author, I took it as best practice. However, now that I poke around in the Clojure sources he cites, I see that only the regular quote is used. I guess most likely it's a typo.
(Narrowed the scope of the question title accordingly)
The default for gen-class is to use - as the prefix for method names of that class. Which is why -main is the default entry point for java -cp clojure.jar yourclass
Backticks qualify their argument with the current namespace, so (in-ns `foo) is the same as (in-ns 'current-namespace/foo) and I don't think that's particularly idiomatic. The idiomatic way is to put each namespace in its own file with (ns ...) at the top, and use or require them as needed.
Would it be possible to override the 'require' command so that it will try to download a certain resource if it was not found on the local machine. For example:
(require 'examples.introduction)
; if not found => download from the net
; (url: http://media.pragprog.com/titles/shcloj/code/examples/introduction.clj)
You could override the require function and of course the overriden variant could download stuff if the namespace it was asked for is not available on the classpath. Overriding the way :require works in (ns ...) forms is, AFAIK, impossible for now due to the way in which ns is handled.
Note that such a 'downloading require' wouldn't be very helpful if you wanted to place new paths on the classpath (including new jars), as classpath injection doesn't work reliably in Clojure (due to JVM issues). There is clojure.core/add-classpath... but it's been marked as deprecated since forever now, its use is strongly discouraged, there are no guarantees that it will work for you and this situation isn't likely to change anytime soon. On the other hand, if you wanted to put new source files in a directory which was already present on the classpath, then that should work fine.
In case you do want to play around with overriding require, if you have a foo namespace, you could do
(ns foo
(:refer-clojure :exclude [require])
; other stuff; any :requires here will work as usual!
)
Then define your own require, using clojure.core/require when appropriate:
(defn require [ns-symbol]
(do-stuff-to-obtain-the-namespace))
clojure.contrib.find-namespaces namespace might be helpful in finding out what's available on the classpath. (Or you could use the the-ns function and see if it throws an exception after an initial attempt at requiring the namespace through clojure.core/require.)
Note that the binding approach which might come to mind first ((binding [require ...] ...)) will not work, since require normally resolves to a Var interned in the clojure.core namespace and Vars from namespaces whose names start with clojure are currently directly linked by the compiler (meaning no actual Var lookup is performed at runtime, so rebinding of those Vars has no effect on the code).
The (:refer-clojure :exclude [require]) in the ns form for your namespace prevents require from resolving to clojure.core/require and leaves you free to define a Var of that name in your own namespace. As mentioned above, that doesn't prevent the clojure.core/require Var from being accessible if you type out the fully qualified the symbol.
Actually, (add-classpath "http://foo/bar/baz/src/") or (add-classpath "http://www.foo.com/bar.jar"), will allow requiring remote stuff.
Michał's warnings do apply though: Only use this for toying at the repl ...