I am trying to set up multiple logging streams for riemann. I have decided to use the simple Clojure file write function to set up additional streams. However, the file is written to as soon as riemann is reloaded but results in a null pointer exception when the relevant event is called and the file needs to be written to.
(where (and (or (tagged "source1") (host "shubham"))
(not (= (:installation_id event) "default")))
(smap (fn [event] (prepare-influx-event event {:time-unit :nanoseconds}))
influx
)
(let [wrtr (io/writer "/var/log/riemann/test.txt" :append true)]
(.write wrtr "Listen please1\n")
(.close wrtr))
;;(spit "/var/log/riemann/test.txt" "Listen please2\n" :append true)
)
the riemann config contains a Clojure expression that is run when riemann starts. The result of running this expsreassion, that is the thing that evaluating it returns, is then used to process all the events. Riemann's config file is a function, which returns a function that will do the actual work. (insert yo'dog memes here)
In this case, when riemann loads that last expression it, while it's getting ready to run, it will open the file, write to it, close it. then it will take the result of closing it and treat that as the function to handle events.
the result of closing a file is null, so it will later try to "run" that null as a function and a NPE results.
you likely want an smap around the let, or just remove the close as you want riemann to keep this file open the whole time it runs. depending on what your going for you may consider making the call to smap above be the last expression inside the let.
below this line is purely my opinion, and not part of the answer:
you almost certainly want to use a proper logging-thing to log stuff to disk on the riemann host. These hosts tend to run for a long time uninterrupted (i tend to leave them a couple years) and if you have it writing to a single file, that file will eventually encounter physics (full disk, size limit etc.) and then if you ssh in and delete it, no space will be freed because riemann will still have the file open. This somehow seems to always happen during some event where you'd really rather monitoring was working. Using a thing with log-rotation is generally a good idea.
Related
I'd like to run an interactive CLI program from within Clojure (e.g., vim) and be able to interact with it.
In bash and other programming languages, I can do that with
vim > `tty`
I tried to do the same in Clojure:
(require '[clojure.java.shell :as shell])
(shell/sh "vim > `tty`")
but it just opens vim without giving me tty.
Background: I'm developing a Clojure CLI tool which parses emails and lets a user edit the parsed data before saving them on the disk. It works the following way:
Read a file with email content and parse it. Each email is stored as a separate file.
Show a user the parsed data and let the user edit the data in vim. Internally I create a temporary file with the parsed data, but I don't mind doing it another way if that would solve my issue.
After a user finished editing the parsed data (they might decide to keep it as it is) append the data to a file on a disk. So all parsed data are saved to the same file.
Go to 1st step if there are any files with emails left.
This code relies on Clojure Java interop to make use of Java's ProcessBuilder class.
(defn -main
[]
;use doseq instead of for because for is lazily evaluated
(doseq [i [1 2 3]]
;extract current directory from system variable
(let [file-name (str "test" i ".txt")
working-directory (trim-newline (:out (sh "printenv" "PWD")))]
(spit file-name "")
;this is where fun begins. We use ProcessBuilder to forward commands to terminal
;we pass a list of commands and their arguments to its constructor
(let [process-builder (java.lang.ProcessBuilder. (list "vim" (str working-directory "/" file-name)))
;inherit is a configuration constant
inherit (java.lang.ProcessBuilder$Redirect/INHERIT)]
;we configure input, output and error redirection
(.redirectOutput process-builder inherit)
(.redirectError process-builder inherit)
(.redirectInput process-builder inherit)
;waitFor used to block execution until vim is closed
(.waitFor (.start process-builder))
)
;additional processing here
)
)
;not necessary but script tends to hang for around 30 seconds at end of its execution
;so this command is used to terminate it instantly
(System/exit 0)
)
I am new to Emacs functions. Today is my first attempt to create a function.
I know that count-matches will tell me how many times a regex appears in the rest of the buffer, but most of the time I need to count from the beginning of the buffer. So I tried this:
(defun count-matches-for-whole-buffer (text-to-count)
"Opens the ~/.emacs.d/init.el file"
(interactive "sText-to-count:")
(beginning-of-buffer)
(count-matches text-to-count))
I put this in ~/.emacs.d/init.el and then do "eval-buffer" on that buffer.
So now I have access to this function. And if I run it, it will ask me for text to search for.
But the function only gets as far as this line:
beginning-of-buffer
I never get the count. Why is that?
Two things.
You should use (goto-char (point-min)) instead of beginning-of-buffer.
count-matches will not display messages when called from lisp code unless you provide a parameter indicating so.
Try this code:
(defun count-matches-for-whole-buffer (text-to-count)
(interactive "sText-to-count:")
(count-matches text-to-count (point-min) (point-max) t))
Is there a way to get a dump of all the source code I have entered into a repl session. I have created a bunch of functions using (defn ...) but did it 'on the fly' without entering them in a text file (IDE) first.
Is there a convenience way to get the source back out of the repl session?
I note that:
(dir user)
will give me a printed list of type:
user.proxy$java.lang.Object
so I can't appear to get that printed list into a Seq for mapping a function like 'source' over. And even if I could then:
(source my-defined-fn)
returns "source not found"...even though I personally entered it in to the repl session.
Any way of doing this? Thanks.
Sorry, but I suspect the answer is no :-/
The best you get is scrolling up in the repl buffer to where you defined it. The source function works by looking in the var's metadata for the file and line number where the functions code is (or was last time it was evaluated), opening the file, and printing the lines. It looks like this:
...
(when-let [filepath (:file (meta v))]
(when-let [strm (.getResourceAsStream (RT/baseLoader) filepath)]
(with-open [rdr (LineNumberReader. (InputStreamReader. strm))]
(dotimes [_ (dec (:line (meta v)))] (.readLine rdr))
...
Not including the full source in the metadata was done on purpose to save memory in the normal case, though it does make it less convenient here.
I started to mess with Ypsilon, which is a C++ implementation of Scheme.
It conforms R6RS, features fast garbage collector, supports multi-core CPUs and Unicode but has a LACK of documentation, C++ code examples and comments in the code!
Authors provide it as a standalone console application.
My goal is to use it as a scripting engine in an image processing application.
The source code is well structured, but the structure is unfamiliar.
I spent two weeks penetrating it, and here's what I've found out:
All communication with outer world is done via C++ structures called
ports, they correspond to Scheme ports.
Virtual machine has 3 ports: IN, OUT and ERROR.
Ports can be std-ports (via console), socket-ports,
bytevector-ports, named-file-ports and custom-ports.
Each custom port must provide a filled structure called handlers.
Handlers is a vector containing 6 elements: 1st one is a boolean
(whether
port is textual), and other five are function pointers (onRead, onWrite, onSetPos, onGetPos, onClose).
As far as I understand, I need to implement 3 custom ports (IN, OUT and ERROR).
But for now I can't figure out, what are the input parameters of each function (onRead, onWrite, onSetPos, onGetPos, onClose) in handlers.
Unfortunately, there is neither example of implementing a custom port no example of following stuff:
C++ to Scheme function bindings (provided examples are a bunch of
.scm-files, still unclear what to do on the C++ side).
Compiling and
running bytecode (via bytevector-ports? But how to compile text to
bytecode?).
Summarizing, if anyone provides a C++ example of any scenario mentioned above, it would significantly save my time.
Thanks in advance!
Okay, from what I can read of the source code, here's how the various handlers get called (this is all unofficial, based purely on source code inspection):
Read handler: (lambda (bv off len)): takes a bytevector (which your handler will put the read data into), an offset (fixnum), and a length (fixnum). You should read in up to len bytes, placing those bytes into bv starting at off. Return the number of bytes actually read in (as a fixnum).
Write handler: (lambda (bv off len)): takes a bytevector (which contains the data to write), an offset (fixnum), and a length (fixnum). Grab up to len bytes from bv, starting at off, and write them out. Return the number of bytes actually written (as a fixnum).
Get position handler: (lambda (pos)) (called in text mode only): Allows you to store some data for pos so that a future call to the set position handler with the same pos value will reset the position back to the current position. Return value ignored.
Set position handler: (lambda (pos)): Move the current position to the value of pos. Return value ignored.
Close handler: (lambda ()): Close the port. Return value ignored.
To answer another question you had, about compiling and running "bytecode":
To compile an expression, use compile. This returns a code object.
There is no publicly-exported approach to run this code object. Internally, the code uses run-vmi, but you can't access this from outside code.
Internally, the only place where compiled code is loaded and used is in its auto-compile-cache system.
Have a look at heap/boot/eval.scm for details. (Again, this is not an official response, but based purely on personal experimentation and source code inspection.)
I've got a series of cpp source file and I want to write another program to JUDGE if they can run correctly (give input and compare their output with standart output) . so how to:
call/spawn another program, and give a file to be its standard input
limit the time and memory of the child process (maybe setrlimit thing? is there any examples?)
donot let the process to read/write any file
use a file to be its standard output
compare the output with the standard output.
I think the 2nd and 3rd are the core part of this prob. Is there any way to do this?
ps. system is Linux
To do this right, you probably want to spawn the child program with fork, not system.
This allows you to do a few things. First of all, you can set up some pipes to the parent process so the parent can supply the input to the child, and capture the output from the child to compare to the expected result.
Second, it will let you call seteuid (or one of its close relatives like setreuid) to set the child process to run under a (very) limited user account, to prevent it from writing to files. When fork returns in the parent, you'll want to call setrlimit to limit the child's CPU usage.
Just to be clear: rather than directing the child's output to a file, then comparing that to the expected output, I'd capture the child's output directly via a pipe to the parent. From there the parent can write the data to a file if desired, but can also compare the output directly to what's expected, without going through a file.
std::string command = "/bin/local/app < my_input.txt > my_output_file.txt 2> my_error_file.txt";
int rv = std::system( command.c_str() );
1) The system function from the STL allows you to execute a program (basically as if invoked from a shell). Note that this approach is inherenly insecure, so only use it in a trusted environment.
2) You will need to use threads to be able to achieve this. There are a number of thread libraries available for C++, but I cannot give you recommendation.
[After edit in OP's post]
3) This one is harder. You either have to write a wrapper that monitors read/write access to files or do some Linux/Unix privilege magic to prevent it from accessing files.
4) You can redirect the output of a program (that it thinks goes to the standard output) by adding > outFile.txt after the way you would normally invoke the program (see 1)) -- e.g. otherapp > out.txt
5) You could run diff on the saved file (from 3)) to the "golden standard"/expected output captured in another file. Or use some other method that better fits your need (for example you don't care about certain formatting as long as the "content" is there). -- This part is really dependent on your needs. diff does a basic comparing job well.