What is the best module for HttpRequest in OCaml - ocaml

I wish to use OCaml to access the Yahoo Finance API. Essentially, it will be just a bunch of HTTP requests to get quotes from Yahoo Finance.
Which module I should use?
I wish to have async HTTP requests.

There are possibilities using lwt:
ocsigen has a quite complete and a bit complex implementation
cohttp is a bit simpler but lacks some usefull parts
using opam to install:
$ opam install ocsigenserver cohttp
For instance in a toplevel:
try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with _ -> ();;
#use "topfind";;
#thread;;
#require "ocsigenserver";;
open Lwt
(* a simple function to access the content of the response *)
let content = function
| { Ocsigen_http_frame.frame_content = Some v } ->
Ocsigen_stream.string_of_stream 100000 (Ocsigen_stream.get v)
| _ -> return ""
(* launch both requests in parallel *)
let t = Lwt_list.map_p Ocsigen_http_client.get_url
[ "http://ocsigen.org/";
"http://stackoverflow.com/" ]
(* maps the result through the content function *)
let t2 = t >>= Lwt_list.map_p content
(* launch the event loop *)
let result = Lwt_main.run t2
and using cohttp:
try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with _ -> ();;
#use "topfind";;
#require "cohttp.lwt";;
open Lwt
(* a simple function to access the content of the response *)
let content = function
| Some (_, body) -> Cohttp_lwt_unix.Body.string_of_body body
| _ -> return ""
(* launch both requests in parallel *)
let t = Lwt_list.map_p Cohttp_lwt_unix.Client.get
(List.map Uri.of_string
[ "http://example.org/";
"http://example2.org/" ])
(* maps the result through the content function *)
let t2 = t >>= Lwt_list.map_p content
(* launch the event loop *)
let v = Lwt_main.run t2
Notice that an implementation of cohttp for jane street async library is also available

Just for the record, there is also ocurl with curl multi API support.

Related

Unbound value "string_of_sexp"

It works quite well in utop with #require "ppx_jane" but
I added (preprocess (pps ppx_jane)) in my dune file which looks like this:
(library
(preprocess (pps ppx_jane))
(name raftml)
(modules raft rpc types)
(libraries
core
core_unix
proto
grpc
grpc-lwt
ocaml-protoc
lwt
lwt.unix
h2
h2-lwt-unix
domainslib
yojson
ppx_jane
ppx_sexp_conv
ppx_deriving_yojson
ppx_deriving
ppx_deriving_yojson.runtime))
And my types are like this:
type log = {
mutable command: string;
mutable term: int32;
mutable index: int32
} [##deriving sexp]
I call sexp_of_log in my code like this:
let persist () = Out_channel.write_all "file_name" ~data:(Sexp.to_string (sexp_of_log { command = "hello"; term = (10l); index = (24l); }))
And there's an error when I run dune build: Unbound value "string_of_sexp"
I solved this by adding:
open Sexplib.Std
At the top of the file that has the record types deriving sexp.

How to define custom exception printers using Janestreet Core?

By default, a Failure exception is printed as such:
# print_endline (Printexc.to_string (Failure "uh\noh"));;
Failure("uh\noh")
For improved readability, we want to print the argument of Failure as is because we understand it should be human-readable. In the OCaml standard library, we would initialize an application with the following:
# Printexc.register_printer (function
| Failure s -> Some ("Failure: " ^ s)
| _ -> None
);;
The new behavior of Printexc.to_string would be:
# print_endline (Printexc.to_string (Failure "uh\noh"));;
Failure: uh
oh
Great. Now if we use the core_kernel library, first we can see that printing an exception is slightly different but not better to a human reader:
#require "core_kernel";;
# print_endline (Printexc.to_string (Failure "uh\noh"));;
(Failure "uh\
\noh")
Perhaps we can override this? Let's try.
# Printexc.register_printer (function
| Failure s -> Some ("Failure: " ^ s)
| _ -> None
);;
# print_endline (Printexc.to_string (Failure "uh\noh"));;
Failure: uh
oh
This works, but it's not using the printer that's part of Core_kernel. If we use it, we still get the same unreadable result:
# print_endline (Core_kernel.Exn.to_string (Failure "uh\noh"));;
(Failure "uh\
\noh")
And Core_kernel.Exn doesn't offer a register_printer function. So effectively, it looks like Core_kernel.Exn makes sure that we don't define custom exception printers. Is there another way or should we just not use Core_kernel.Exn then if we want to show human-readable error messages?
Edit: For context, our original problem is to print nested error messages nicely. For example, we would like to read something like the following:
Uncaught exception: Failure:
Uncaught exception in subprocess 1234: Failure:
something happened
trace line 1
trace line 2
trace line 1
trace line 2
trace line 3
where we use indentation for quoting and escaping, rather than double-quotes and backslash escape sequences.
Base.Exn (for which Core_kernel.Exn is an alias) prints errors as human readable s-expressions using Sexplib0 printer and converters. It is possible to add custom sexp converter for exceptions with Sexplib0.Exn_converter.add.
However, if you don't intend to print exceptions as s-expressions, I see indeed no reason to use the printer of Base.Exn.
EDIT: Since the problem seems to be the printing of S-expressions, one solution might be to use Base.Exn.sexp_of and then plug-in a custom S-expression printers that does not escape strings, nor print parenthesis:
let pp_sep ppf () = Format.fprintf ppf "# "
let pp_sexp ppf = function
| Atom s -> Format.pp_print_string ppf s
| List l ->
Format.fprintf ppf "#[<v 2> %a#]"
(Format.pp_print_list ~pp_sep pp_sexp) l
let to_string exn = Format.asprintf "%a" pp_sexp (Exn.sexp_of_t exn)

Global CLI flag in OCaml Core.Command

I am currently creating a CLI application in OCaml and using Core.Command, the CLI parser included in core (v0.10), to parse the command line.
I want to have a global flag that can be used for any subcommand (like the --paginate or --git-dir flags in git for example).
For instance, I want a -debug flag so that the two following commands are valid
my-cli -debug hello world
my-cli -debug goodbye world
However, I could not find a way to do this with the Core.Command API.
Here is a simplified version what I currently have.
open Core
let initialize_logger debug =
Logs.set_reporter (Logs_fmt.reporter ());
let log_level = if debug then Logs.Debug else Logs.Info in
Logs.set_level (Some log_level)
let some_func_with_logging () =
Logs.debug (fun m -> m "the flag debug was passed!")
let hello name =
some_func_with_logging ();
Printf.printf "Hello %s!\n" name
let goodbye name =
some_func_with_logging ();
Printf.printf "Goodbye %s!\n" name
let hello_command =
let open Command.Let_syntax in
Command.basic
~summary:"says hello"
[%map_open
let name = anon ("name" %: string)
and debug = flag "debug" no_arg ~doc:"debug" in
fun () ->
initialize_logger debug;
hello name
]
let goodbye_command =
let open Command.Let_syntax in
Command.basic
~summary:"says goodbye"
[%map_open
let name = anon ("name" %: string)
and debug = flag "debug" no_arg ~doc:"debug" in
fun () ->
initialize_logger debug;
goodbye name
]
let main_command =
Command.group ~summary:"a cool CLI tool"
[ ("hello", hello_command);
("goodbye", goodbye_command);
]
let () = Command.run main_command
There are two main issues here:
the debug flag as well as the call to initialize_logger is duplicated in every subcommand
the debug flag needs to be passed after the subcommand when invoking the command: my-cli hello world -debug instead of my-cli -debug hello world
Is there a clean way to handle this with Core.Command API?

ocaml-core equivalent of Unix.create_process

I'd like to port the following command from Unix library to Jane Street's Core.Std.Unix library.
Unix.create_process exec args Unix.stdin Unix.stdout Unix.stderr
That is, I have an executable exec and arguments args and want to run the process using the same in/out/error channels as the current process.
I can get close with Core.Std.Unix.create_process ~exec:exec ~args:args, but can't figure out how to connect the stdin,stdout,stderr returned by this function from core with the file descriptors used by the current process.
You can dup2 the returned descriptors to your current descriptors, but I'm not sure that this would work.
open Core.Std
let redirect ( p : Unix.Process_info.t ) : unit =
let open Unix.Process_info in
List.iter ~f:(fun (src,dst) -> Unix.dup2 ~src ~dst) [
p.stdin, Unix.stdin;
p.stdout, Unix.stdout;
p.stderr, Unix.stderr
]
let exec prog args : unit =
let p = Unix.create_process ~prog ~args in
redirect p
But there is another solution, that maybe applicable. Consider using just Unix.system, it should work just out of box.
I'm not sure if I understand your question correctly, but if you just want to use the same channels as the current process, take a look at fork_exec in Core.Std.Unix:
val fork_exec : prog:string ->
args:string list ->
?use_path:bool ->
?env:string list ->
unit -> Core_kernel.Std.Pid.t
According to the docs,
fork_exec ~prog ~args ?use_path ?env () forks and execs prog with args in the child process, returning the child pid to the parent.

Module case name confusion

I made the mistake of updating software and now I can't run any OUnit tests.
I think I've managed to boil the problem down to a simple REPL session.
$ ocaml -I /opt/local/lib/ocaml/site-lib/oUnit
OCaml version 4.01.0
# Ounit.assert_equal;;
Error: Wrong file naming: /opt/local/lib/ocaml/site-lib/oUnit/ounit.cmi
contains the compiled interface for
Ounit when OUnit was expected
# OUnit.assert_equal;;
Error: Reference to undefined global `OUnit'
Any ideas what I'm doing wrong?
I'm running this on a Mac laptop which has the default case-insensitive/case-preserving file-system, but dinking with the case of the include path doesn't help.
My larger problem manifests thus:
ocamlbuild \
-libs \
nums,str,unix,oUnit,graph \
-cflags \
-g,-w,+a-4,-warn-error,+a-4,-I,/opt/local/lib/ocaml/site-lib/oUnit,-I,/opt/local/lib/ocaml/site-lib/ocamlgraph \
-lflags \
-g,-I,/opt/local/lib/ocaml/site-lib/oUnit,-I,/opt/local/lib/ocaml/site-lib/ocamlgraph \
./AllTests.native
Error: No implementations provided for the following modules:
OUnitCore referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitLoggerStd referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitUtils referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitConf referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitAssert referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitBracket referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitTest referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitRunner referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitChooser referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitLogger referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2),
/opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit)
OUnitTestData referenced from /opt/local/lib/ocaml/site-lib/oUnit/oUnit.cmxa(OUnit2)
Command exited with code 2.
Compilation unsuccessful after building 646 targets (645 cached) in 00:00:02.
Your first error is probably because your OS does not distinguish cases in filenames but OCaml does:
$ ocamlobjinfo `ocamlfind query oUnit`/oUnit.cmi
File /mnt/home/jun/.opam/system/lib/oUnit/oUnit.cmi
Unit name: OUnit
Interfaces imported:
cb146e345f0b34fc8ad4b5afe69d1f20 OUnit
...
You can see that the module is called "OUnit" not "Ounit".
The second error is clearly because the library is not loaded into the REPL yet. The REPL's knows the existence of the function, but has no code loaded. It is accessible if you load the library and those on which it depends:
ocaml -I `ocamlfind query oUnit` unix.cma oUnitAdvanced.cma oUnit.cma
OCaml version 4.01.0
# OUnit.assert_equal;;
- : ?cmp:('a -> 'a -> bool) ->
?printer:('a -> string) ->
?pp_diff:(Format.formatter -> 'a * 'a -> unit) ->
?msg:string -> 'a -> 'a -> unit
= <fun>
I solved the problem by changing my ocamlbuild invocation to use -use-ocamlfind which seems to work around all the path confusion.
I also uninstalled all OCaml modules and instead installed OPAM and then installed OUnit and others via OPAM, and re-logged in so that my environment is pristine.
The build command that works for me now is
ocamlbuild \
-use-ocamlfind \
-libs graph \
-pkgs str,unix,ounit \
-cflags -g,-w,+a-4,-warn-error,+a-4,-I,/opt/local/lib/ocaml/site-lib/ocamlgraph \
-lflags -g,-I,/opt/local/lib/ocaml/site-lib/ocamlgraph \
./AllTests.native
which is generated by my updated build script
(* Usage: ocaml make.ml [<ModuleToBuild>.{byte,native}] *)
#load "unix.cma";;
let args = match Array.to_list Sys.argv with
| [] -> failwith "no program on argv"
| _::[] -> ["AllTests.byte"]
| _::argv -> argv
let library_paths = [
(* TODO: Figure out why `opam install ocamlgraph` fails *)
"-I"; "/opt/local/lib/ocaml/site-lib/ocamlgraph";
]
(* If the -p flag is first in the argument list, then replace it
with a host of flags that enable profiling via ocamlprof. *)
let args, c_opt_flags, l_opt_flags =
match args with
| "-p"::rest ->
(* Treat the targets as debug and profile targets. *)
"-tags"::"debug,profile"
(* Use profiling versions of the compilers which instrument
various flow control constructs with entry counters. *)
::"-ocamlc"::"ocamlcp"
::"-ocamlopt"::"ocamloptp"
::rest,
(* Instrument all available points. *)
["-P"; "a"],
(* Link with a wrapper that dumps profiling data if program
exits noramlly. *)
["-p"]
| _ -> args, [], []
let standard_flags = [
"-use-ocamlfind";
(* TODO: Figure out why `opam install ocamlgraph` fails *)
"-libs"; "graph";
"-pkgs"; "str,unix,ounit"; (* TODO: graph *)
"-cflags"; (String.concat "," (
[
"-g";
"-w"; "+a-4";
"-warn-error"; "+a-4";
]
# c_opt_flags
# library_paths));
"-lflags"; (String.concat "," (
[
"-g";
]
# l_opt_flags
# library_paths));
]
let build_command = "ocamlbuild"
let argv = build_command::(standard_flags # args)
let _ = begin
Printf.printf "%s\n" (String.concat " \\\n\t" argv);
flush stdout;
Unix.execvp build_command (Array.of_list argv);
end;;