OCaml Forge

Detail: [#1305] Separating subpackage in subdirectories while installing

Feature Requests: Browse | Download .csv | Monitor

[#1305] Separating subpackage in subdirectories while installing

2013-08-09 14:35
Submitted by:
Jacques-Pascal Deplaix (jpdeplaix)
Assigned to:
Sylvain Le Gall (gildor-admin)
Due in version:
Operating System:
Separating subpackage in subdirectories while installing

Detailed description
Admitting that there is a package named foo and a sub-package foo.bar.
In both packages there is a module named Test.

So when installing, there will be a conflict between both packages since both packages will be installed in the same directory.

So, my proposal is to install sub-packages in subdirectories.

Note: This would be very useful, for example, when installing libraries depending on eliom.


Date: 2014-11-26 22:43
Sender: Petter Urkedal

Sorry, I read this thread from bottom to top, and though there was some confusion towards the "end" (actually beginning), so my previous post is rather redundant.
Date: 2014-11-26 22:32
Sender: Petter Urkedal

I've hit this issue myself as I'm trying to distribute an Eliom library, though I realize I'm a bit late joining this discussion. Commenting on the latest response, I think Jacques-Pascal's intention was not to link the two libraries, but picking one or the other depending on the target.

My suggestion would be to add a "FindlibDirectory" directive for library sections. This would add a "directory" setting in the package and cause all .cmi, .cmx, and .mli files to be installed in that subdirectory. ocamlfind should then add the right include options, thus:
- Only interfaces for sub-packages linked against is visible, allowing earlier detection of missing BuildDepends.
- It's easy to figure out which subpackage to link against by inspecting the directory. (E.g. it wasn't obvious to me that Lwt_main is provided by lwt.unix.)
- Conflicting sub-packages are possible, which was the original point.

Here is a somewhat more worked example for the Eliom use-case:

Library "ocsidant-shared"
Path: src/shared
FindlibContainers: ocsidant
FindlibName: shared
shared/Ocsidant_text_utils # Same version usable for both native and JS.

Library "ocsidant-server"
Path: src/server
FindlibContainers: ocsidant
FindlibName: server
BuildDepends: ocsidant-shared
server/Ocsidant_database, # Only avaliable on the server side.
server/Ocsidant_widgets # Server-side version not using DOM/JS.

Library "ocsidant-client"
Path: src/client
FindlibContainers: ocsidant
FindlibName: client
BuildDepends: ocsidant-shared
client/Ocsidant_dom_utils, # Only avaliable to JS clients.
client/Ocsidant_widgets # Client-side version depending on DOM/JS.

Eliom makes essential use of same-named modules with not only different implementations, but also slightly differing interfaces. E.g. a `t client_value` on the server side is just a handle, but becomes a real `t` object on the client side. However, what is needed can be understood as a "multilib" feature, i.e. different incompatible packages for different targets/purposes.
Date: 2014-03-12 16:57
Sender: Gabriel Radanne

Just to make everything clear, and as jpdeplaix pointed already, we would need this not only for eliom libraries, but also in the case of tyxml (which is a fairly "standard" ocaml library, by most standard).

We want to provide a "simple to use" package foo which contains the module Foo, but also an "internal" package foo_bis for advanced users, which contains the module Foo_bis. Of course, Foo depends on Foo_bis, so Foo_bis is listed in the internal modules. Currently, since everything is installed in the same directory, Foo_bis is accessible when loading the foo package.

All of this is easily visible when looking at the tyxml _oasis file : https://github.com/ocsigen/tyxml/blob/master/_oasis
Date: 2014-03-07 20:42
Sender: Jacques-Pascal Deplaix

Oh ! Good use case ! Thanks for sharing :)
Date: 2014-03-07 20:06
Sender: Philippe Veber

I think there may be another, less eliom-centered use case:

Library lib_a
Path: src/a
Pack: true
Modules: M1, M2
FindlibName: a
FindlibParent: lib

Library lib_b
Path: src/b
Pack: true
Modules: M1,M2
FindlibName: b
FindlibParent: lib

This works well thanks to packing, apart from mli files, that are copied in the same directory (and so the mli files are overwritten).
Date: 2014-03-07 17:26
Sender: Sylvain Le Gall

s/TestInner.cmo/TestInner.cmx in fact, because anyway TestInner.cmi will only be installed in test/inner and TestInner.cmo should not be installed at all.

In the very specific case of this kind of library, you will not have conflict because we use exactly the same interface across module. So this should be fine...
Date: 2014-03-07 16:56
Sender: Jacques-Pascal Deplaix

No, this is wanted. The module TestInner shouldn't be accessed through test.
So yes TestInner.cmo should be installed in 2 directories.
Date: 2014-03-07 16:02
Sender: Sylvain Le Gall

the library test should depends on test.inner.

With the proposal describe in this bug, you'll have to install TestInner.cmo in 2 directories!
Date: 2014-03-07 15:06
Sender: Jacques-Pascal Deplaix

@Drup has another use-case for this feature:
Consider the following _oasis (part):

Library test
Path: src
Modules: Test
InnerModules: TestInner

Library inner
FindlibParent: test
Path: src
Modules: TestInner

then the module TestInner will be installed and is accessible through test AND test.inner.
Date: 2014-02-25 00:46
Sender: Sylvain Le Gall

Nothing done so far.
Date: 2014-02-24 23:56
Sender: Jacques-Pascal Deplaix

What are the news on this one ?
Date: 2013-08-20 16:09
Sender: Sylvain Le Gall

I would like to speed up the next release, so it won't be in 0.4.0 but since it will be hidden behind a feature, you can have it in 0.4.X (X > 1).

Release 0.4.0 main feature will be to introduce a field 'ExperimentalFeatures', e.g:

Name: foobar
Version: 0.1
ExperimentalFeatures: subpackage-in-subdir, toplevel

Library foo

This way I can release big changes hidden behind feature and promote features to official release later.
Date: 2013-08-20 15:56
Sender: Jacques-Pascal Deplaix

Great ! :)

Does it will be ready for the next release ?
Date: 2013-08-20 15:50
Sender: Sylvain Le Gall

I see the use and it somehow makes sense. I will provide something to fix that but it will come as a feature (something that you need to turn on) until I am fine with it.
Date: 2013-08-20 15:44
Sender: Jacques-Pascal Deplaix

Ok, I see. Thanks.

Otherwise, did you understand the use case described by Gabriel Radanne ?
Date: 2013-08-19 21:13
Sender: Sylvain Le Gall

Library "foo.bar" (your proposal)
will create

Library "bar" (2nd option)
FindlibParent: foo
will create

I think 2nd option is cleaner in term of filename, but this is IMHO.
Date: 2013-08-19 20:46
Sender: Jacques-Pascal Deplaix

You mean, with my proposal ? Because without, it is a sub-package.
Date: 2013-08-19 19:15
Sender: Sylvain Le Gall

But bar will not be a subpackage of foo, it will live in 2 different directories...
Date: 2013-08-19 18:35
Sender: Jacques-Pascal Deplaix

I don't get it :/ The findlib name doesn't change. It still foo.bar in both cases.
Date: 2013-08-14 23:02
Sender: Sylvain Le Gall

"foo.bar" is a correct filename, but it will translate to a strange findlib name...
Date: 2013-08-14 21:12
Sender: Jacques-Pascal Deplaix

I have a little question off topic: why « Library "foo.bar" » is not good ? The cm{a,xa,xs} name doesn't count, no ?
Date: 2013-08-09 21:25
Sender: Gabriel Radanne

There is some reasons for this :
- the code for a given identifier can be different in the server and the client. (It's quite often the case for low level stuff, like the Html5 module). With some trickery, you can even have different types and still use it in a transparent way in the client/server in eliom.
- The compilation process is not the same for the client and the server, there is probably details in the linking and all that stuff. A simple aspect of that is that you want to ship native version for the server but not for the client (since it will not work with js_of_ocaml).
- The code can be actually shared, but the same code will produce different .cm[foo]
Date: 2013-08-09 21:13
Sender: Sylvain Le Gall

So I perfectly see the use case now. However I am not sure why eliom doesn't ship eliom.common that contains Html5 and eliom.server and eliom.client that depends on eliom.common. I would say it look likes a better way to share code.

But maybe you can explain me the precise reason.

The feature request is doable but I need to figure out a little bit how valuable it is.
Date: 2013-08-09 20:17
Sender: Gabriel Radanne

I have a precise use case for this : shipping library for eliom.
For most eliom library, you have shared code between the client and the server. You would like to be able to define a server and a client package that contains the same modules and eliom will load them according to a configure file.
This is how the eliom "standard" library is shipped : there is an "eliom.server" and an "eliom.client" ocamlfind library that contains both the Html5 module (for example).

Currently, we have an ugly makefile that define an empty ocamlfind library and manually copy each file in different subdirectories. This is completely hideous and I tried to avoid it when I did the build system for https://github.com/ocsigen/eliom-widgets. The compilation part worked quite nicely, thanks to jpdeplaix's plugin for ocamlbuild, but not the installation part because of this precise bug.
Because of this, we had to switch back to the ugly makefile and the whole build part is far more painful than with oasis (I did not do it).

Is this use case convincing enough ? :D
Date: 2013-08-09 15:21
Sender: Sylvain Le Gall

Library "foo.bar" is not possible (because it will end up by being called foo.bar.cma).

Library bar
FindlibParent: foo

is better.

So the case you describe here will not be valid, from the general OCaml point of view, because having two modules Test will generate a lot of problems for compiling. So anything which will embed foo and foo.bar and that will try to use Test.a_function will don't know if it should look into foo/Test.cm* or foo/bar/Test.cm*.

But I think your precise use case, is that you want to be able to dynamically load them in ocsigen. Since this kind of object doesn't require module name, it should be ok (it doesn't require module name because it use side effect while loading the file dynamically).

This is a very specific corner case, although it is doable to generate a META file that will allow doing that.

I would however recommend you to not use the same name, because this is a known general problem of OCaml.
Date: 2013-08-09 15:05
Sender: Jacques-Pascal Deplaix

I'm missing your point either, But, yes, I have:

OASISFormat: 0.3
Name: test
Version: 1.0
Synopsis: Test
Authors: Test
License: MIT
BuildTools: ocamlbuild
Plugins: DevFiles (0.3), META (0.3)

Library foo
Path: foo
Modules: Test

Library "foo.bar"
Path: bar
Modules: Test
Date: 2013-08-09 14:49
Sender: Sylvain Le Gall

Unfortunately, even if it solves the installation problem, you won't be able to compile with foo and foo.bar at the same time. Because OCaml compiler will not pay attention to paths in this case.

But maybe I am missing your point. Do you have a precise example for this ?

Attached Files:


Field Old Value Date By
assigned_tonone2014-06-16 09:41gildor-admin