Detail: [#1262] parallel running tests

Patches: Browse | Download .csv | Monitor

[#1262] parallel running tests

Date:
2013-03-03 12:12 Priority:
3

State:
Open

Submitted by:
Thomas Wickham (mackwic) Assigned to:
Sylvain Le Gall (gildor-admin)

Resolution:
Accepted Version:
None

Component:
None

Summary:
parallel running tests

Detailed description
Hello, I work this morning on the multi-threading of OUnit2, feature-request #1095 I've patched OUnitCore and _oasis. Seems to works but I don't know if adding `threads` in the BuildDepends section is the correct way to add thread support, does it add the `-thread` flag ? However, tests shows that OUnit is running in parallel in the processus point of view. I choose the thread approach because in OCaml, processes will be a far more heavy solution for... what benefits ? As threads are system threads, they are preemptive, are they ? (The doc says so, I've always learn so, but you don't feel so in the FR...). Timeouts can be managed with threads also, just add a watchdog for each thread test. Moreover, the cost of process creation comparing to the running time of a little OUnit test is a bit unfair. That's why threads seems to be the Right Thing. I used a thread pool to minimize the cost of thread creation, keeping in mind that tests should be small. Maybe the pool size should be adjusted, It's quiet easy. Please review my patch and help me test, I'm far from confident with oasis. I only give you oUnitCore.ml, I think you'll know better than me how to change _oasis. ;-) Attached: patch in whatsnew and diff format

Followup

Message
Date: 2013-03-23 00:01 Sender: Sylvain Le Gall I have patched OUnit to use your changes. http://darcs.ocamlcore.org/cgi-bin/darcsweb.cgi?r=ounit/ounit;a=commit;h=20130322234849-4d692-faf5ac417afc820a5164127cd599f5e98c47ff6d.gz
Date: 2013-03-16 18:08 Sender: Thomas Wickham Hello Sylvain, sorry for the latency, I was busy last week. I have worked a bit on OUnit : https://gist.github.com/mackwic/5177470 There is a new file: oUnitWorker, basically a simple process that loop reading on a socket. In oUnitCore.ml, run_all_test has been split in 3: run_all_test_{seq, threaded, process}. I didn't see any regression with sequential and threaded runners. The by-process runner is not terminated, as I'm not very confident with sockets in Ocaml and I lack of time to polish it. You can see the documented TODO to fill. It ``only'' miss the select & read/write part of the main process. If you have already worked on a piece of code, use yours, that's really not a problem. ;-)
Date: 2013-03-08 13:56 Sender: Sylvain Le Gall I plan to release oUnit next week or th week following. If you have made any progress on this patches, keep me updated so that we don't do the wor twice. Although don't hesitate to send me an incomplete patch, just leave TODO item inside it.
Date: 2013-03-05 22:12 Sender: Thomas Wickham So be it. I'll work on that very soon.
Date: 2013-03-05 22:07 Sender: Sylvain Le Gall fork + marshalling + select. You fork X process, and you send to each process a test to run (since they have been forked, they share the same definition). And then you select to wait for one of the forked process to send back the result -- and assign a new one. (your last solution).
Date: 2013-03-05 21:09 Sender: Thomas Wickham Damn ! I was sure this issue was gone with Ocaml 4. You should specify it here: http://ocamlunix.forge.ocamlcore.org/threads.html Ok, so what do we do ? Do we keep that ? Or are we playing with myocamlbuild.ml in order to register tests as binary targets ? I the latter case, I think we should give the right to the dev to choose for each test if it should be in its own process or not, by example in introducing a new test_case operator (which one ?). We can also make a pool of process and play with marshalling/S-exp and IPC to transfer our tests (do the Marshalling keep closures ?). An advice ? I personally have a preference for the last solution, but only because it's seems far more fun.
Date: 2013-03-03 19:55 Sender: Sylvain Le Gall Thanks for the patch. Unfortunately, thread in OCaml are not what you expect they are (that is a long standing issue in OCaml). If your read the doc about threads: http://caml.inria.fr/pub/docs/manual-ocaml/manual039.html The threads library is implemented by time-sharing on a single processor. It will not take advantage of multi-processor machines. Using this library will therefore never make programs run faster. However, many programs are easier to write when structured as several communicating processes. That is not 100% clear, but I can give you an explanation. The Threads library should be used for concurrency and not parallelism. Whenever a program run it acquires a "big lock" that prevents any other OCaml code to run. The only exception is when entering special part of the OCaml runtime that release the lock (like file reading). So Yes, processus creation is heavy but is the only to get true parallelism in OCaml.

Attached Files:

Attachments:

oUnitCore.ml.diff.patch

oUnitCore.ml.whatsnew.patch

Changes:

Field	Old Value	Date	By
Resolution	Awaiting Response	2013-03-22 12:22	gildor-admin
assigned_to	none	2013-03-03 19:55	gildor-admin
File Added	208: oUnitCore.ml.whatsnew.patch	2013-03-03 12:12	mackwic
File Added	207: oUnitCore.ml.diff.patch	2013-03-03 12:12	mackwic