Module dependency cycle - ocaml

I have:
Module 1:
provides type Module1.type1, its constructor, and some functions that accept and return type1
Module 2:
open Module1
open Module3
provides type Module2.type2, also has functions that accept type1 and type3 as params
Module 3:
open Module1
open Module2
provides type Module3.type3, and its constructor that depends on type1
provides functions that accept and return types type1, type2 and type3
question
as a result I obviously get dependency cycle: src/Module3.cmj -> src/Module2.cmj -> src/Module3.cmj error by compiler. Something that is trivially achievable in TypeScript/JS with individual import, is not possible in Reason. How to get around this?
I don't really want to change the architecture of my program, just to facilitate shortcomings of compiler/module system.

The simplest way to handle your problem is indeed recursive modules. I don't advise you to use them, as recursive modules can make your code harder to read, compile and can in the most complex cases break your code at run time. Not to mention if you use side-effects in your module definitions (please don't).
I will use the OCaml syntax, you should be able to easily translate to Reason.
If you want to go with that anyway, here is the quick and dirty solution, using recursive module and functors.
The quick and dirty solution
1) Create a module myModTypes that will indicate the expected types of module2 and module3. It should look like:
module type Module2type = sig ... end
module type Module3type = sig ... end
with ... being the expected signatures of your modules (if you already have interface files written, just copy/paste them here, if you don't write those, they are important)
2) Put module2 and module3 within functors expecting the other module
For example, the code of module2 should now look like
module MakeModule2(Module3 : MyModTypes.Module3type) = struct
(* the code of module2 *)
end
The code of module3 will be in the same way, just swap 2 and 3 in the added lines.
3) Create a module makemodules2and3 with that code (translated to Reason):
module rec Module2 : MyModTypes.Module2type = Module2.MakeModule2(Module3)
and Module3 : MyModTypes.Module3type = Module3.MakeModule3(Module2)
Note that recursive module definitions always expect a module type.
4) Subsequent uses of Module2 and Module3 should now open Makemodules2and3 before being able to use them.
The right solution
You have to change the architecture of your program. Slightly.
As the OP said, there are no cycle of dependency in the functions, and that's a relief. Just split module2 and module3 into two new modules each. One with the functions that only depend on module1 and their own module, one with the "next step" functions.
This is a better way to approach how you declare your modules: they should be one with the types they define. Ideally, you have a module for each type, plus one additional module for each interaction between types.

Looks like Module1 doesn't depend on the other two modules. You can keep it as is. But since the other two are mutually recursive, you can express that using the recursive module syntax. This does have a requirement though that you declare the module signatures at the point of definition, since Reason needs to know what to expect. For example:
/* Impl.re */
module rec Module2: {
type type2;
let make: (Module1.type1, Module3.type3) => type2;
} = {
... actual implementation of Module2 ...
} and Module3: {
type type3;
let make: Module1.type1 => type3;
let foo: Module1.type1;
let bar: Module2.type2 => type3;
} = {
... actual implementation of Module3 ...
};
This is the general shape you'd use, you can adapt it to your needs.
If you don't want your users to have to do Impl.Module2.... to access the recursive modules, you can even expose them as file modules using include:
/* Module2.re */
include Impl.Module2;
And you can even annotate the implementation modules (Impl.Module2 and 3) with a compile-time warning to let users know not to use those ones:
/* Impl.re */
[#deprecated "Please use Module2 instead of Impl.Module2"]
module Module2: {...} = {...};

Related

How to unit test functions whose interfaces are defined within submodules

It seems to me that one of the nice features of submodules is that you can create a helper function in a submodule at very little cost to the programmer; you don't trigger a compilation cascade, you don't clutter the namespace or your documentation, and it is immediately clear where that function can and can't be used. They're like nicer versions of private functions.
However, functions in submodules cannot be used. While this is working as intended, it also appears that this prevents the function from being unit tested. Both unit test frameworks that I'm aware of, pFUnit and fruit, require use syntax to operate.
There are some (imho somewhat inelegant) workarounds for the same problem with private functions discussed in How to access private variables in fortran modules?, but none of those solutions appear to work for functions in submodules, at least not without negating all of the benefits of putting those functions in submodules in the first place.
So are there any solutions to this problem?
The primary purpose of introducing the submodule concept was to avoid long recompilation cascades when only a minor non-interface-breaking change had been introduced to a module file. This is done by separating the interface of the procedures and putting them in the parent module and keeping the implementation in the submodule.
Submodules are themselves permitted to have submodules, which is useful for very large
programs. The number of levels of submodules that I have seen typically does not exceed two (that is, a module with submodules that themselves have submodules) but there is no limit. Each module or submodule is the root of a tree whose other nodes are its descendants and have access to it by host association. No other submodules have such access, which is helpful for developing parts of large modules independently. Furthermore, there is no mechanism for accessing anything declared in a submodule from elsewhere – it is effectively private, as you said.
Therefore, in summary, if you need anything from any submodule at any level to be accessed by any other parts of the program, it must be declared in the original parent module of the submodule. There is no other way, as far as I am aware to access anything in any submodule whose interface or declaration is not given in the original parent module.
Once you put the procedure interfaces and variable/type declarations in the parent module, you can use them anywhere in your program, even though the procedure implementations could be buried in submodules of the parent module.
I have little experience with submodules (so not sure if this is useful), but just to extend my comment above...
!! parent_mod.f90 (public things)
module parent_mod
implicit none
type myint_t
integer :: n = 100
contains
procedure :: add, show
endtype
interface
module subroutine add( x )
class(myint_t) :: x
end
module subroutine show( x )
class(myint_t) :: x
end
endinterface
end
!! parent_helper.f90 (private things to be used by parent_impl.f90
!! and possibly by unit tests)
module parent_helper
use parent_mod, only: myint_t
implicit none
contains
subroutine debug_show( x )
type(myint_t) :: x
print *, "debug: x = ", x
end
end
!! parent_impl.f90 (implementation)
submodule (parent_mod) parent_impl
implicit none
contains
module procedure add
x% n = x% n + 1
end
module procedure show
use parent_helper, only: debug_show
call debug_show( x )
end
end
!! main.f90
program main
use parent_mod, only: myint_t
implicit none
type(myint_t) :: a
call a% add()
call a% show() !! 101
call a% add()
call a% show() !! 102
block
use parent_helper
call debug_show( a ) !! 102
endblock
end
!! build
$ gfortran-10 -fcheck=all -Wall -Wextra parent_mod.f90 parent_helper.f90 parent_impl.f90 main.f90
Does this possibly help avoid recompilation of parent_mod.f90 (even when parent_helper or parent_impl are modified)? (And I noticed that the module name "parent" has no meaning here... XD)

Resolve library conflict in SML/NJ Compilation Manager

I'm using SML/NJ 110.79, which includes support for new structures defined by the Successor ML project. Among others, the Fn structure.
As it happens, I already had an identically named structure in one of my personal project with utilities, which worked fine before 110.79.
With 110.79, for this .cm file:
group is
$/basis.cm
$SMACKAGE/sml-extras/v0.1.0/sources.sml.cm
I get the following error, though:
sources.cm:3.3-3.45 Error: structure Fn imported from
$SMLNJ-BASIS/(basis.cm):basis-common.cm#155252(fn.sml) and also from
$SMACKAGE/sml-extras/v0.1.0/(sources.sml.cm):src/fn.sml
Does anyone know how to resolve this conflict through the Compilation Manager. Ideally, my Fn structure will be able to "extend" the standard Fn by just open-ing it, but projects using the sml-extras library, will not see the standard Fn structure, only my extended version.
Is this possible? Do I need to wrap/re-export the whole basis.cm library in my sml-extras.cm project?
I managed to solve this by using what I believe is called an administrative library in the CM manual, §2.9.
What that means precisely is to create an auxiliary .cm file that wraps the basis library and re-exports only the symbols we're interested in.
sources.cm
This is the main project file.
library
structure Main
is
(* Let's say this library redefines the Fn and Ref structures *)
(* and the REF signature. *)
$SMACKAGE/sml-extras/v0.1.0/sources.sml.cm
(* This excludes out Fn, Ref and REF from the Basis library, but *)
(* imports anything else. *)
amended-basis.cm
main.sml
amended-basis.cm
This file imports $/basis.cm and then re-exports all of it except Fn, Ref and REF.
group
library($/basis.cm) - (
structure Fn
structure Ref
signature REF
)
is
$/basis.cm
main.sml
structure Main =
struct
open Fn (* sml-extras's Fn *)
end
The solution is based on the set calculus described in the CM manual, §4 and on the EBNF grammar from Appendix A.
Another solution would have been to change sml-extras to re-export the whole $/basis.cm, while shadowing the conflicting symbols. However, in the interest of modularity I decided to go with the solution detailed above.

D: finding all functions with certain attribute

Is it currently possible to scan/query/iterate all functions (or classes) with some attribute across modules?
For example:
source/packageA/something.d:
#sillyWalk(10)
void doSomething()
{
}
source/packageB/anotherThing.d:
#sillyWalk(50)
void anotherThing()
{
}
source/main.d:
void main()
{
for (func; /* All #sillWalk ... */) {
...
}
}
Believe it or not, but yes, it kinda is... though it is REALLY hacky and has a lot of holes. Code: http://arsdnet.net/d-walk/
Running that will print:
Processing: module main
Processing: module object
Processing: module c
Processing: module attr
test2() has sillyWalk
main() has sillyWalk
You'll want to take a quick look at c.d, b.d, and main.d to see the usage. The onEach function in main.d processes each hit the helper function finds, here just printing the name. In the main function, you'll see a crazy looking mixin(__MODULE__) - this is a hacky trick to get a reference to the current module as a starting point for our iteration.
Also notice that the main.d file has a module project.main; line up top - if the module name was just main as it is automatically without that declaration, the mixin hack would confuse the module for the function main. This code is really brittle!
Now, direct your attention to attr.d: http://arsdnet.net/d-walk/attr.d
module attr;
struct sillyWalk { int i; }
enum isSillyWalk(alias T) = is(typeof(T) == sillyWalk);
import std.typetuple;
alias hasSillyWalk(alias what) = anySatisfy!(isSillyWalk, __traits(getAttributes, what));
enum hasSillyWalk(what) = false;
alias helper(alias T) = T;
alias helper(T) = T;
void allWithSillyWalk(alias a, alias onEach)() {
pragma(msg, "Processing: " ~ a.stringof);
foreach(memberName; __traits(allMembers, a)) {
// guards against errors from trying to access private stuff etc.
static if(__traits(compiles, __traits(getMember, a, memberName))) {
alias member = helper!(__traits(getMember, a, memberName));
// pragma(msg, "looking at " ~ memberName);
import std.string;
static if(!is(typeof(member)) && member.stringof.startsWith("module ")) {
enum mn = member.stringof["module ".length .. $];
mixin("import " ~ mn ~ ";");
allWithSillyWalk!(mixin(mn), onEach);
}
static if(hasSillyWalk!(member)) {
onEach!member;
}
}
}
}
First, we have the attribute definition and some helpers to detect its presence. If you've used UDAs before, nothing really new here - just scanning the attributes tuple for the type we're interested in.
The helper templates are a trick to abbreviate repeated calls to __traits(getMember) - it just aliases it to a nicer name while avoiding a silly parse error in the compiler.
Finally, we have the meat of the walker. It loops over allMembers, D's compile time reflection's workhorse (if you aren't familiar with this, take a gander at the sample chapter of my D Cookbook https://www.packtpub.com/application-development/d-cookbook - the "Free Sample" link is the chapter on compile time reflection)
Next, the first static if just makes sure we can actually get the member we want to get. Without that, it would throw errors on trying to get private members of the automatically imported object module.
The end of the function is simple too - it just calls our onEach thing on each element. But the middle is where the magic is: if it detects a module (sooo hacky btw but only way I know to do it) import in the walk, it imports it here, gaining access to it via the mixin(module) trick used at the top level... thus recursing through the program's import graph.
If you play around, you'll see it actually kinda works. (Compile all those files together on the command line btw for best results: dmd main.d attr.d b.d c.d)
But it also has a number of limitations:
Going into class/struct members is possible, but not implemented here. Pretty straightforward though: if the member is a class, just descend into it recursively too.
It is liable to break if a module shares a name with a member, such as the example with main mentioned above. Work around by using unique module names with some package dots too, should be ok.
It will not descend into function-local imports, meaning it is possible to use a function in the program that will not be picked up by this trick. I'm not aware of any solution to this in D today, not even if you're willing to use every hack in the language.
Adding code with UDAs is always tricky, but doubly so here because the onEach is a function with its on scope. You could perhaps build up a global associative array of delegates into handlers for the things though: void delegate()[string] handlers; /* ... */ handlers[memberName] = &localHandlerForThis; kind of thing for runtime access to the information.
I betcha it will fail to compile on more complex stuff too, I just slapped this together now as a toy proof of concept.
Most D code, instead of trying to walk the import tree like this, just demands that you mixin UdaHandler!T; in the individual aggregate or module where it is used, e.g. mixin RegisterSerializableClass!MyClass; after each one. Maybe not super DRY, but way more reliable.
edit:
There's another bug I didn't notice when writing the answer originally: the "module b.d;" didn't actually get picked up. Renaming it to "module b;" works, but not when it includes the package.
ooooh cuz it is considered "package mod" in stringof.... which has no members. Maybe if the compiler just called it "module foo.bar" instead of "package foo" we'd be in business though. (of course this isn't practical for application writers... which kinda ruins the trick's usefulness at this time)

Functors in OCaml: triple code duplication necessary?

I'd like to clarify one point: currently it seems to me that triple signature duplication is necessary while declaring a functor, provided we export it in the .mli file. Here is an example:
Suppose we have a functor Make, which produces a module A parametrized by SigA (simplest example I could think of). Consequently, in the .mli file we have:
module type A = sig
type a
val identity : a -> a
end
module type SigA = sig
type a
end
module Make (MA:SigA) :
A with type a := MA.a
Now I understand that we have to write an implementation in the .ml file:
module Make (MA:SigA) = struct
type a = MA.a
let identity obj = obj
end
So far so good, right? No! Turns out we have to copy the declaration of A and SigA verbatim into the .ml file:
module type A = sig
type a
val identity : a -> a
end
module type SigA = sig
type a
end
module Make (MA:SigA) = struct
type a = MA.a
let identity obj = obj
end
While I (vaguely) understand the rationale behind copying SigA (after all, it is mentioned in the source code), copying A definition seems like a completely pointless exercise to me.
I've had a brief look through the Core codebase, and they just seem to either duplicate it for small modules and for larger once they export it to the separate .mli, which is used both from .ml and .mli.
So is it just a state of affairs? Is everyone fine with copying the module signature THREE times (once in the .mli file, two times in the .ml file: declaration and the definition!!)
Currently I'm considering just ditching .mli files altogether and restricting the modules export using signatures in the .ml files.
EDIT: yes I know that I can avoid this problem by declaring the interface for A inline inside Make in the .mli file. However this doesn't help me if I want to use that interface from outside of that module.
That's because a pair of ML and MLI file acts like a structure and a corresponding signature it is matched against.
The usual way to avoid writing out the module type twice is to define it in a separate ML file. For example,
(* sig.ml *)
module type A = sig
type a
end
module type B = sig
type b
val identity : b -> b
end
(* make.mli *)
module Make (A : Sig.A) : Sig.B with type b = A.a
(* make.ml *)
module Make (A : Sig.A) =
struct
type b = A.a
let identity x = x
end
It is fine to leave out an MLI file in the case where it does not hide anything, like for the Sig module above.
In other cases, writing out the signature separately from the implementation is a feature, and not really duplication -- it defines the export of a module, and usually, that is a small subset of what's in the implementation.

Is there a way to print user-defined datatypes in ocaml?

I can't use print_endline because it requires a string, and I don't (think) I have any way to convert my very simple user-defined datatypes to strings. How can I check the values of variables of these datatypes?
In many cases, it's not hard to write your own string_of_ conversion routine. That's a simple alternative that doesn't require any extra libraries or non-standard OCaml extensions. For the courses I teach that use OCaml, this is often the simplest mechanism for students.
(It would be nice if there were support for a generic conversion to strings though; perhaps the OCaml deriving stuff will catch on.)
There's nothing in the base language that does this for you. There is a project named OCaml Deriving (named after a feature of Haskell) that can automatically derive print functions from type declarations. I haven't used it, but it sounds excellent.
http://code.google.com/p/deriving/
Once you have a function for printing your type (derived or not), you can install it in the ocaml top-level. This can be handy, as the built-in top-level printing sometimes doesn't do quite what you want. To do this, use the #install-printer directive, described in Chapter 9 of the OCaml Manual.
There are third-party library functions like dump in OCaml Batteries Included or OCaml Extlib, that will generically convert any value to a string using all the runtime information it can get. But this won't be able to recover all information; for example, constructor names are lost and become just integers, so it will not look exactly the way you want. You will basically have to write your own conversion functions, or use some tool that will write them for you.
Along the lines of previous answers, ppx_sexp is a PPX for generating printers from type definitions. Here's an example of how to use it while using jbuilder as your build system, and using Base and Stdio as your stdlib.
First, the jbuild file which specifies how to do the build:
(jbuild_version 1)
(executables
((names (w))
(libraries (base stdio))
(preprocess (pps (ppx_jane ppx_driver.runner)))
))
And here's the code.
open Base
open Stdio
type t = { a: int; b: float * float }
[##deriving sexp]
let () =
let x = { a = 3; b = (4.5,5.6) } in
[%sexp (x : t)] |> Sexp.to_string_hum |> print_endline
And when you run it you get this output:
((a 3) (b (4.5 5.6)))
S-expression converters are present throughout Base and all the related libraries (Stdio, Core_kernel, Core, Async, Incremental, etc.), and so you can pretty much count on being able to serialize any data structure you encounter there, as well as anything you define on your own.