Rpc_parallelSourceA type-safe parallel library built on top of Async_rpc.
module Worker = Rpc_parallel.Make (T : Worker_spec)The Worker module can be used to spawn new workers, either locally or remotely, and run functions on these workers. T specifies which functions can be run on a Worker.t as well as the implementations for these functions. In addition, T specifies worker states and connection states. See README for more details
A parallel map/reduce library. See examples/add_numbers.ml and examples/number_stats.ml for examples.
include module type of struct include Parallel endmodule Worker = Make(T)
val start_app :
?rpc_max_message_size:int ->
?rpc_handshake_timeout:Core.Time.Span.t ->
?rpc_heartbeat_config:Async.Rpc.Connection.Heartbeat_config.t ->
Async.Command.t ->
unitstart_app command should be called from the top-level in order to start the parallel application. This function will parse certain environment variables and determine whether to start as a master or a worker.
rpc_max_message_size, rpc_handshake_timeout, rpc_heartbeat_config can be used to alter the rpc defaults. These rpc settings will be used for all connections. This can be useful if you have long async jobs.
Use State.get to query whether the current process has been initialized as an rpc parallel master (start_app or init_master_exn has been called). We return a State.t rather than a bool so that you can require evidence at the type level. If you want to certify, as a precondition, for some function that start_app was used, require a State.t as an argument. If you don't need the State.t anymore, just pattern match on it.
If you want more direct control over your executable, you can use the Expert module instead of start_app. If you use Expert, you are responsible for starting the master and worker rpc servers. worker_command_args will be the arguments sent to each spawned worker. Running your executable with these args must follow a code path that calls worker_init_before_async_exn and then start_worker_server_exn. An easy way to do this is to use worker_command.
Rpc_parallel.Managed is a wrapper around Rpc_parallel that attempts to make manage connections for you, but it ended up being too magical to reason about, so you should prefer to use the plain Rpc_parallel interface.