Is there a way I can make emacs compile and run my code in an external console window with the command M-x compile (what I mean by external console window is that I want it to run my code in a new console window like it does when I run my C++ code in visual studio).
I want emacs to automatically open up a new console window and run the executable after M-x compile like this:
Normally with Emacs, when you want some obscure (i.e. some simple, or special to your use case) automation, you just write a command, which does what you want.
Here is such an example. Following Elisp code calls a function and-run, when compilation is finished (through using hook compilation-finish-functions).
(defun and-run (&rest _)
"run after comilation, an not elaborated example function"
(interactive)
(let ((exe "./test.out")
(prevent-closing (concat "echo \"Press Enter to continue\"" ";"
"read"))
(terminal "xterm -e"))
(when (file-exists-p exe)
(let ((my-command (concat exe ";" prevent-closing)))
(shell-command (format "%s %s" terminal (shell-quote-argument my-command)))))))
(add-hook 'compilation-finish-functions #'and-run)
Since I'm no Windows user you have to make some small adaptions to this function, in order to be able to run it on Windows.
Note: there is also a compilation-start-hook, if you e.g. want to compare file-modification times of your exe.
When running a clojure core.clj as Clojure Application will the following method gets executed and "Test" gets printed?
(defn -main
[& args]
(println "Test")
I know a few of my friends who use Counterclockwise plugin on Eclipse.
Here is a link to their docs. It also has a step by step way to Create your first project
I am wondering how one goes about deploying a Common Lisp web application written in, say, Hunchentoot, Wookie, Woo, or even Clack.
That is, suppose I write an app that contains some files, packages, etc. Typically when I am working locally, I simply run a command in REPL that starts the server and then visit it using localhost:8000 or something like that.
However, I am a bit puzzled as to what the process is for deploying an app to a production server like AWS EC2. In what form should I deploy the Lisp code? Are there different options? What happens if the server needs to be restarted or is experiencing problems?
I've figured something out lately by building self-contained executables for web apps and I wrote about it on lisp-journey/web-dev (shipping and deployment sections), as well as for the building part on the Common Lisp Cookbook/scripting#for-web-apps.
I copy the interesting parts here, there's a bit more on each resource. Edits are welcome, primarily on those resources thanks !
edit july 2019: I contributed a page on the Cookbook: https://lispcookbook.github.io/cl-cookbook/web.html
edit: see also a list of tools and platforms that provide professional CL support: https://github.com/CodyReichert/awesome-cl#deployment
(edited) How to run the web app as a script
I explain below how to build and run executables, but we can of course run the application as a script. In a lisp file, say run.lisp, ensure:
to load your project's asd file: (load "my-project.asd")
to load its dependencies: (ql:quickload :my-project)
to call its main function: (my-project:start) (given start is an exported symbol, otherwise ::start).
In doing so, the application starts and gives you back a Lisp REPL. You can interact with the running application. You can update it and even install new Quicklisp libraries as it runs.
How to build a self-contained executable
See also https://github.com/CodyReichert/awesome-cl#interfaces-to-other-package-managers for bindings to Homebrew and Debian packages.
With SBCL
How to build (self-contained) executables is implementation-specific (see
below Buildapp and Rowsell). With SBCL, as says
its documentation,
it is a matter of:
(sb-ext:save-lisp-and-die #P"path/name-of-executable" :toplevel #'my-app:main-function :executable t)
sb-ext is an SBCL extension to run external processes. See other
SBCL extensions
(many of them are made implementation-portable in other libraries).
:executable t tells to build an executable instead of an
image. We could build an image to save the state of our current
Lisp image, to come back working with it later. Specially useful if
we made a lot of work that is computing intensive.
If you try to run this in Slime, you'll get an error about threads running:
Cannot save core with multiple threads running.
Run the command from a simple SBCL repl.
I suppose your project has Quicklisp dependencies. You must then:
ensure Quicklisp is installed and loaded at Lisp startup (you
completed Quicklisp installation)
load the project's .asd
install dependencies
build the executable.
That gives:
(load "my-app.asd")
(ql:quickload :my-app)
(sb-ext:save-lisp-and-die #p"my-app-binary" :toplevel #'my-app:main :executable t)
From the command line, or from a Makefile, use --load and --eval:
build:
sbcl --non-interactive \
--load my-app.asd \
--eval '(ql:quickload :my-app)' \
--eval "(sb-ext:save-lisp-and-die #p\"my-app\" :toplevel #my-app:main :executable t)"
With ASDF
Now that we'seen the basics, we need a portable method. Since its
version 3.1, ASDF allows to do that. It introduces the make command,
that reads parameters from the .asd. Add this to your .asd declaration:
:build-operation "program-op" ;; leave as is
:build-pathname "<binary-name>"
:entry-point "<my-system:main-function>"
and call asdf:make :my-system.
So, in a Makefile:
LISP ?= sbcl
build:
$(LISP) --non-interactive \
--load my-app.asd \
--eval '(ql:quickload :my-app)' \
--eval '(asdf:make :my-system)'
With Roswell or Buildapp
Roswell, an implementation manager and much
more, also has the ros build command, that should work for many
implementations.
We can also make our app installable with Roswell by a ros install my-app. See its documentation.
We'll finish with a word on
Buildapp, a battle-tested and
still popular "application for SBCL or CCL that configures and saves
an executable Common Lisp image".
Many applications use it (for example,
pgloader), it is available on
Debian: apt install buildapp, but you shouldn't need it now with asdf:make or Roswell.
For web apps
We can similarly build a self-contained executable for our web-app. It
would thus contain a web server and would be able to run on the
command line:
$ ./my-web-app
Hunchentoot server is started.
Listening on localhost:9003.
Note that this runs the production webserver, not a development one,
so we can run the binary on our VPS right away and access the app from
outside.
We have one thing to take care of, it is to find and put the thread of
the running web server on the foreground. In our main function, we
can do something like this:
(defun main ()
(start-app :port 9003) ;; our start-app, for example clack:clack-up
;; let the webserver run.
;; warning: hardcoded "hunchentoot".
(handler-case (bt:join-thread (find-if (lambda (th)
(search "hunchentoot" (bt:thread-name th)))
(bt:all-threads)))
;; Catch a user's C-c
(#+sbcl sb-sys:interactive-interrupt
#+ccl ccl:interrupt-signal-condition
#+clisp system::simple-interrupt-condition
#+ecl ext:interactive-interrupt
#+allegro excl:interrupt-signal
() (progn
(format *error-output* "Aborting.~&")
(clack:stop *server*)
(uiop:quit)))
(error (c) (format t "Woops, an unknown error occured:~&~a~&" c))))
We used the bordeaux-threads library ((ql:quickload "bordeaux-threads"), alias bt) and uiop, which is part of ASDF so
already loaded, in order to exit in a portable way (uiop:quit, with
an optional return code, instead of sb-ext:quit).
Parsing command line arguments
see the Cookbook here. TLDR; use uiop:command-line-arguments to get a list of the arguments. To parse them for real, there are libraries.
Deployment
Straightforward with an executable. The web app is visible from the outside right away.
On Heroku
See this buildpack.
Daemonizing, restarting in case of crashes, handling logs
See how to do that on your system.
Most GNU/Linux distros now come with Systemd.
Examples search result:
https://seanmcgary.com/posts/deploying-nodejs-applications-with-systemd/
It is as simple as writing a configuration file:
# /etc/systemd/system/my-app.service
[Unit]
Description=stupid simple example
[Service]
WorkingDirectory=/path/to/your/app
ExecStart=/usr/local/bin/sthg sthg
Type=simple
Restart=always
RestartSec=10
running a command to start it:
sudo systemctl start my-app.service
a command to check its status:
systemctl status my-app.service
and Systemd can handle logging (we write to stdout or stderr, it writes logs):
journalctl -f -u my-app.service
and it handles crashes and restarts the app:
Restart=always
and it can start the app after a reboot:
[Install]
WantedBy=basic.target
to enable it:
sudo systemctl enable my-app.service
Debugging SBCL error: ensure_space: failed to allocate n bytes
If you get this error with SBCL on your server:
mmap: wanted 1040384 bytes at 0x20000000, actually mapped at 0x715fa2145000
ensure_space: failed to allocate 1040384 bytes at 0x20000000
(hint: Try "ulimit -a"; maybe you should increase memory limits.)
then disable ASLR:
sudo bash -c "echo 0 > /proc/sys/kernel/randomize_va_space"
Connecting to a remote Swank server
Little example here: http://cvberry.com/tech_writings/howtos/remotely_modifying_a_running_program_using_swank.html.
Demo project here: https://lisp-journey.gitlab.io/blog/i-realized-that-to-live-reload-my-web-app-is-easy-and-convenient/
It defines a simple function that prints forever:
;; a little common lisp swank demo
;; while this program is running, you can connect to it from another terminal or machine
;; and change the definition of doprint to print something else out!
;; (ql:quickload :swank)
;; (ql:quickload :bordeaux-threads)
(require :swank)
(require :bordeaux-threads)
(defparameter *counter* 0)
(defun dostuff ()
(format t "hello world ~a!~%" *counter*))
(defun runner ()
(bt:make-thread (lambda ()
(swank:create-server :port 4006)))
(format t "we are past go!~%")
(loop while t do
(sleep 5)
(dostuff)
(incf *counter*)
))
(runner)
On our server, we run it with
sbcl --load demo.lisp
we do port forwarding on our development machine:
ssh -L4006:127.0.0.1:4006 username#example.com
this will securely forward port 4006 on the server at example.com to
our local computer's port 4006 (swanks accepts connections from
localhost).
We connect to the running swank with M-x slime-connect, typing in
port 4006.
We can write new code:
(defun dostuff ()
(format t "goodbye world ~a!~%" *counter*))
(setf *counter* 0)
and eval it as usual with M-x slime-eval-region for instance. The output should change.
There are more pointers on CV Berry's page.
Hot reload
Example with Quickutil. See notes on lisp-journey.
It has to be run on the server (a simple fabfile command can call this
through ssh). Beforehand, a fab update has run git pull on the
server, so new code is present but not running. It connects to the
local swank server, loads the new code, stops and starts the app in a
row.
Continuous Integration, continuous delivery of executables, Docker
See https://lispcookbook.github.io/cl-cookbook/testing.html#continuous-integration
To run a lisp image in production, you can generate a fasl file from your lisp code with:
(compile-file "app.lisp")
run your generated .fas file by invoking sbcl.
sbcl --noinform \
--load app.fas \
--eval "(defun main (argv) (declare (ignore argv)) (hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4242)))"
I found a blog with a solution that I have adapted to my needs for a production system on a linux box. Unfortunately I cannot find the reference to that blog anymore, so that I can just show you my solution, which is for CCL (while the original solution was for SBCL), which with I am more familiar. Here is the program that starts the system:
(require 'swank)
(require 'hunchentoot)
(defparameter *httpd-port* 9090) ; The port Hunchentoot will be listening on
(defparameter *shutdown-port* 6700) ; The port CCL will be listening for shutdown
; this port is the same used in /etc/init.d/hunchentoot
(defparameter *swank-port* 5016) ; The port used for remote interaction with slime
;; Start the Swank server
(defparameter *swank-server*
(swank:create-server :port *swank-port* :dont-close t))
(require 'YOUR-PACKAGE)
(YOUR-PACKAGE:YOUR-STARTING-FUNCTION)
(princ "Hunchentoot started on port ")
(princ *httpd-port*)
(terpri)
(let* ((socket (make-socket :connect :passive :local-host "127.0.0.1" :local-port *shutdown-port* :reuse-address t))
(stream (accept-connection socket)))
(close stream)
(close socket))
(print "Stopping Hunchentoot...")
(YOUR-PACKAGE:YOUR-STOPPING-FUNCTION)
(dolist (proc (all-processes))
(unless (equal proc *current-process*)
(process-kill proc)))
(sleep 1)
(quit)
The idea is that you can connect to the running system with slime, by specifying the port used by swank. I used it a couple of times, for instance to change a database link on the fly, and was quite impressed by the power of such possibility.
The running system can be terminated by:
telnet 127.0.0.1 6700
and initiated by something like:
nohup ccl -l initcclserver.lisp >& server.out &
In a previous version of the script I found the SBCL-specific parts, so if you use it you could modify the script.
For accepting the terminating connections:
(sb-bsd-sockets:socket-bind socket #(127 0 0 1) *shutdown-port*)
(sb-bsd-sockets:socket-listen socket 1)
(multiple-value-bind (client-socket addr port)
(sb-bsd-sockets:socket-accept socket)
(sb-bsd-sockets:socket-close client-socket)
(sb-bsd-sockets:socket-close socket)))
To close the system:
(dolist (thread (sb-thread:list-all-threads))
(unless (equal sb-thread:*current-thread* thread)
(sb-thread:terminate-thread thread)))
(sleep 1)
(sb-ext:quit)
Hope this can help.
I am building an application for the raspberry pi and use pi4j as a dependency for Software PWM on GPIO. I'd like to test my code on my local machine though, so I would like to compile my code without the pi4j dependency and skip method calls to the library.
Example code:
(ns led-server.model
(:require [clojure.tools.logging :as log])
(:import [com.pi4j.wiringpi SoftPwm Gpio])) ;; pi4j dependency, only compiles on rPi
(defn- soft-pwm-write [pin value]
(let [ival (Math/round (double (* value RANGE)))]
(SoftPwm/softPwmWrite pin ival) ;; call to pi4j. This is what I want to skip
(log/info "pin" pin "set to" ival))
)
pi4j requires the wiringPi C library, which is only available on the raspberry pi (which makes sense). For testing on my dev machine it would be sufficient to see the log printout. I don't want to comment out the :import and method calls for testing, I would like a more elegant solution.
Apart from the question if conditional compilation is the best approach here, it is not difficult to compile files conditionally with leiningen: put the files you want to compile conditionally in a folder different than src/clj, and define it as a source folder in a profile:
:profiles {
:native {:source-paths ["src/native/clj/"]}
:mock {:source-paths ["src/mock/clj/"]}
Then, run leininig with profile:
lein with-profiles +mock repl
See more at https://github.com/technomancy/leiningen/blob/master/doc/PROFILES.md
In your situation, you could define a protocol, provide 2 implementations and make sure you load only the classes relevant to your environment.
I found while adding warnings to code that writes macros that the body of a file was being executed twice during compilation. Is there a reason for this? Is this specific to leiningen? I can not reproduce this with (compile ...).
Simplified version:
(ns foo.core
(:require foo.bar))
;; empty
(ns foo.bar)
(println "hello")
$ lein compile :all
Compiling foo.core
hello
Compiling foo.bar
hello
Further testing shows that the namespace is reloaded on top of itself during compile:
(ns foo.bar)
(declare wasd)
(println wasd)
(def wasd 2)
$ lein clean
$ lein compile :all
Compiling foo.core
#<Unbound Unbound: #'foo.bar/wasd>
Compiling foo.bar
2
In a more complicated case I have this happening during compile, and then once every time when run or started a repl from lein. I am not sure why. This is all with clojure 1.6 and leiningen 2.5.0.
Leiningen knows nothing about the structure of your project in terms of how namespaces relate to each-other. Consequently, when compiling a project, lein simply boots a JVM and forces each namespace to be loaded one at a time. This means that, as you noticed, namespaces will be reloaded causing the double evaluation behaviour you are noticing.
In comparison, (clojure.core/compile) simply loads the targeted resource with clojure.core/*compile-files* bound. This will cause the targeted resource and all the resources it requires to be loaded and compiled to class-files. However, it will not traverse your entire project structure compiling all resources as Leiningen's compile operation does.
The reason why you're seeing println output during compile time is because println is called during namespace evaluation. You should have a -main fn or some other entry point for your program that calls on println.
(defn -main [& _]
(println "Won't see on compile.")
(println "lein run -- is printing from -main"))
I think your project is throwing an Unbound Exception because you're trying to reference the wasd variable with println before a value is assigned to it. The variable was declared, but nothing was assigned to it before the println fn tried to get the value.