I'm trying to set up dynamic loading of OCaml modules using Dynlink. I wrote a very simple test program, but it's not working.
This test program consists of two modules, Plug and Ext. Plug is the "main" module. It knows the filename of Ext and loads it with Dynlink.loadfile. Ext references Plug, and calls a function in Plug that informs Plug about one of its functions so that it can be called. After Plug loads Ext, it calls the function that Ext is supposed to have registered.
The Ext module loads successfully if I write it so that it doesn't execute any code. However, if I include the part that registers its extension function with Plug, then I get an error: The module 'Plug' is not yet initialized.
I don't understand how Plug could not be initialized yet, since it is already executing. (Actually I don't know what it means for an OCaml module to be initialized.)
I've reduced the code down to pretty much the least necessary to reproduce the problem. I am using OCaml 4.01.0 on Linux.
Here's the main module:
(* plug.ml *)
type xfn = string -> unit
let dummy_func str =
print_endline ("Dummy: " ^ str)
let ext_func : xfn ref = ref dummy_func
let register func =
ext_func := func
let call () =
(!ext_func) "calling"
(* load extension *)
let () =
try
print_endline "Loading ext.";
Dynlink.loadfile "ext.cmo";
print_endline "Loaded ext.";
()
with
| Dynlink.Error e ->
print_endline (Dynlink.error_message e);
print_endline "Calling registered func.";
call ()
This is the extension file:
(* ext.ml *)
open Plug
let myext str =
print_endline ("Ext: " ^ str)
let () = Plug.register myext
I used this shell script to compile the executable:
#!/bin/sh
ocamlc -c plug.ml && \
ocamlc -c ext.ml && \
ocamlc -o plug dynlink.cma plug.cmo ext.cmo \
|| exit 1
This is the output I get:
Loading ext.
error while linking ext.cmo.
The module `Plug' is not yet initialized
Calling registered func.
Dummy: calling
It looks like I was not on the ball with my search engine skills in this case.
Some more digging turned up an archived email chain that states that the referenced module (Plug in this case) must have been completely evaluated before the dynamically loaded module can call into it.
The solution is to separate the code that loads the extension module and the code that the extension module calls into two separate modules (so now there will be three modules in total).
Related
tl;dr I'm trying to make a source transformation binary using AST_mapper and ppx_driver. I can't figure out how to get the example in the AST_mapper docs to be used by ppx_driver. Are there any good examples of how to use Ppx_driver.register_transformation_using_ocaml_current_ast?
I'm trying to port the example AST_mapper in the docs to be compatible with ppx_driver. Specifically, I'm looking to create a binary that takes source as input, transforms the source using this test mapper, and then outputs the transformed source. Unfortunately, the default main provided by Ast_mapper only accepts Ocaml AST as input (and presumably produces it as output). This is undesirable, because I don't want to have to run this through ocamlc with -dsource to get my output.
Here's my best stab at porting this:
test_mapper.ml
open Asttypes
open Parsetree
open Ast_mapper
let test_mapper argv =
{ default_mapper with
expr = fun mapper expr ->
Pprintast.expression Format.std_formatter expr;
match expr with
| { pexp_desc = Pexp_extension ({ txt = "test" }, PStr [])} ->
Ast_helper.Exp.constant (Ast_helper.Const.int 42)
| other -> default_mapper.expr mapper other; }
let test_transformation ast =
let mapper = (test_mapper ast) in
mapper.structure mapper ast
let () =
Ppx_driver.register_transformation_using_ocaml_current_ast
~impl:test_transformation
"test_transformation"
A few things to note:
The example from the docs didn't work out of the box (before introducing ppx_driver): Const_int 42 had to be replaced with Ast_helper.Const.int 42
For some reason test_mapper is Parsetree.structure -> mapper. (It is unclear to me why a recursive transformation needs the structure to create the mapper, but no matter.) But, this type isn't what Ppx_driver.register_transformation_using_ocaml_current_ast expects. So I wrote a sloppy wrapper test_transformation to make the typechecker happy (that is loosely based off how Ast_mapper.apply_lazy appears to apply a mapper to an AST so in theory it should work)
Unfortunately, after compiling this into a binary:
ocamlfind ocamlc -predicates ppx_driver -o test_mapper test_mapper.ml -linkpkg -package ppx_driver.runner
And running it on a sample file:
sample.ml
let x _ = [%test]
with the following:
./test_mapper sample.ml
I don't see any transformation occur (the sample file is regurgitated verbatim). What's more, the logging Pprintast.expression that I left in the code doesn't print anything, which suggests to me that my mapper never visits anything.
All of the examples I've been able to find in the wild are open sourced by Jane Street (who wrote ppx_*) and seem to either not register their transformation (perhaps there's some magic detection going on that's going over my head) or if they do they use Ppx_driver.register_transformation ~rules which uses Ppx_core.ContextFree (which doesn't seem to be complete and won't work for my real use case--but for the purposes of this question, I'm trying to keep things generally applicable).
Are there any good examples of how to do this properly? Why doesn't ppx_driver use my transformation?
If you want to make a standalone rewriter with a single module, you need to add
let () = Ppx_driver.standalone ()
to run the rewriter and links with ppx_driver and not ppx_driver.runner: ppx_driver.runner runs the driver as soon as it is loaded, therefore before your transformation is registered. Note also that you should at least specify a specific Ast version and uses Ppx_driver.register_transformation rather than Ppx_driver.register_transformation_using_current_ocaml_ast otherwise there is little point in using ppx_driver rather than doing the parsing by hand with compiler-libs and the Pparse module.
I'm using menhir to generate a parser and right now, the parser.mli file that it generated from my parser.mly file looks like this:
(* The type of tokens. *)
type token =
(* ... huge ADT definition goes here ... *)
(* This exception is raised by the monolithic API functions. *)
exception Error
(* The monolithic API. *)
val start: (Lexing.lexbuf -> token) -> Lexing.lexbuf -> Types.ast
Is there a way to include more stuff in my parser's interface? In particular, I would like to be able to also export the datatype for my AST (which is currently in a separate Types module) and some functions that work with the token datatype (for example, a function to convert them back to strings).
I tried putting some Ocaml code after the %% in parser.mly but while that code shows up in parser.ml none of the functions I declared appear in parser.mli.
I'm making a sudoku game with in OCaml with the Graphics library.
I'm trying to add a timer at the top so that the player can see how long he's taking to solve the sudoku. I've looked around and the Lwt library seems to be what I'm looking for.
To draw the timer I wrote a function draw_time, the content of the function is too long and not important, but it has this kind of structure:
let rec hello () =
Lwt.bind (Lwt_unix.sleep 14400.)
(fun () -> print_endline "Hello, world !"; hello ())
Lwt.async (hello)
I've tested it and the function does work as it's supposed to.
The main loop of my game looks like this:
let main_loop gs f_init f_end f_key f_mouse =
f_init;
let timer = draw_time () in
try
while true do
try
if gs.completed <> true then
let event = Graphics.wait_next_event event_types in
if event.Graphics.keypressed then
f_key event.Graphics.key
else if event.Graphics.button then
f_mouse event.Graphics.mouse_x event.Graphics.mouse_y
with
| _ -> raise End
done
with
| End -> f_end ()
;;
This doesn't seem to work. The program only executes draw_time when I close the window (and the program) and not when the window is open and the timer is supposed to be drawn.
What am I doing wrong exactly?
In order to run Lwt program, you need to start the main loop, that will process your threads,
let () = Lwt_main.run (Lwt_io.printl "Hello, world!")
Lwt has code that will integrate Lwt main loop into the Glib. But, as far as I know, there is no such integration for Graphics module.
You may find async_graphics interesting, it is an integration of the Async library, that is quite near to Lwt with Graphics module. It was even mentioned in Real World OCaml
I have implemented a pretty-printer for a module. Currently I start up utop, load the dependencies then do #install_printer pp_custom;; where pp_custom is the pretty-printer.
I would like to automate this so that I can have it in a way that is similar to the lacaml library where the pretty-printer for the matrix is "installed" by default.
How would I go about to doing that?
In short, you need to run #install_printer directive whenever you load your library in the top. I'm using the following code to evaluate code in the toplevel:
open Core_kernel.Std
open Or_error
let eval_exn str =
let lexbuf = Lexing.from_string str in
let phrase = !Toploop.parse_toplevel_phrase lexbuf in
Toploop.execute_phrase false Format.err_formatter phrase
let eval str = try_with (fun () -> eval_exn str)
It depends on Core_kernel, but you can get rid of this easily, by just using eval_exn instead of eval (the last one wraps a possible exception into the Or_error monad). Once you've got the eval function, it can be used to load your printers:
let () = eval (sprintf "#install_printer %s;;" printer)
where printer is the name of the pretty printing function (usually qualified with a module name). Usually, one put such code into the separate library, named library.top, where library is the name of your library.
For further automation, you can require all types, that you want to be auto-printable in toplevel, to register itself in a central registry, and then call all registered printers, instead of enumerating them by hand. To see all this working at once, you can take a look at the BAP library. It has a sublibrary called bap.top that automatically installs all printers. Each type that is desired to be printable implements Printable signature, using Printable.Make functor, that not only derives many printing functions from the basic definition, but also registers the generated pretty-printers in Core_kernel's Pretty_printer registry (you can use your own registry, this is just a set of strings, nothing more). When bap.top library is loaded into the toplevel (using require or load directive), it enumerates all registered printers and install them.
I am trying to achieve something similar to eval() in OCaml.
I have a string and I want to get an OCaml function out of it.
Currently I am doing the following:
I dump the string to new.ml and compile the file:
Compile.implementation Format.std_formatter "new.ml" "New"
Then I try to dynlink the file:
Dynlink.loadfile "new.cmo";
But if I try to do New.foo it fails. I am not sure why I cannot access the module New after Dynlinking. Am I missing something?
Thanks!
The comment of Dynlink.loadfile says:
No facilities are provided to
access value names defined by the unit. Therefore, the unit
must register itself its entry points with the main program,
e.g. by modifying tables of functions.
The loader program cannot access values of dyn-loaded module without any hint, since it has no idea what values are defined in it just from the .cmo file. The dynamic loaded module must register its entry point to some state defined in the loader program.
Here is such a minimal example. First, the module for the entry points:
(* entry.ml *)
let f : (unit -> unit) ref = ref (fun () -> assert false)
The loader program:
(* loader.ml *)
let () =
Dynlink.loadfile "plugin.cmo";
!Entry.f ()
The plugin to be dyn-loaded:
(* plugin.ml *)
let () = Entry.f := (fun () -> prerr_endline "hello world")
Here, Plugin registers its function to Entry.f which is statically linked to Loader, so that Loader can access the function.
They must be compiled as follows:
$ ocamlc -o loader.exe dynlink.cma entry.ml loader.ml
$ ocamlc -c plugin.ml
Execution of loader.exe should demonstrate how the dyn loading works:
$ ./loader.exe
hello world
Note that Entry and Loader must be different modules. Otherwise, you get Uninitialized_global exception at dyn-loading Plugin. Dyn-loaded modules can only access values in "already initialized modules", and the loader module thinks itself not yet initialized when Dynlink.loadfile is called, since the entire evaluation of the module is not finished yet.
Entry.f is the simplest state to have only one entry point. To dyn-load many values, you may want to have more complicated data structure, such as (string, (unit -> unit)) list ref or (string, (unit -> unit)) Hashtbl.t.