package faraday
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=d2679553d813341e6737f67d68577279ebaa92647e4e95dd9b2cd1360de35119
md5=1e6d8f5950d099c6ad9ae0e960fe17a9
doc/faraday/Faraday/index.html
Module Faraday
Source
Serialization primitives built for speed an memory-efficiency.
Faraday is a library for writing fast and memory-efficient serializers. Its core type and related operation gives the user fine-grained control over copying and allocation behavior while serializing user-defined types, and presents the output in a form that makes it possible to use vectorized write operations, such as the writev
system call, or any other platform or application-specific output APIs.
A Faraday serializer manages an internal buffer and a queue of output buffers. The output bufferes may be a sub range of the serializer's internal buffer or one that is user-provided. Buffered writes such as write_string
, write_char
, write_bigstring
, etc., copy the source bytes into the serializer's internal buffer. Unbuffered writes such as schedule_string
, schedule_bigstring
, etc., on the other hand perform no copying. Instead, they enqueue the source bytes into the serializer's write queue directly.
The type of a serializer.
Constructors
create len
creates a serializer with a fixed-length internal buffer of length len
. See the Buffered writes section for details about what happens when len
is not large enough to support a write.
of_bigstring buf
creates a serializer, using buf
as its internal buffer. The serializer takes ownership of buf
until the serializer has been closed and flushed of all output.
Buffered Writes
A serializer manages an internal buffer for coalescing small writes. The size of this buffer is determined when the serializer is created. If the buffer does not contain sufficient space to service a caller's buffered write, the serializer will allocate a new buffer of the sufficient size and use it for the current and subsequent writes. The old buffer will be garbage collected once all of its contents have been flush
ed.
write_string t ?off ?len str
copies str
into the serializer's internal buffer.
write_bytes t ?off ?len bytes
copies bytes
into the serializer's internal buffer. It is safe to modify bytes
after this call returns.
write_bigstring t ?off ?len bigstring
copies bigstring
into the serializer's internal buffer. It is safe to modify bigstring
after this call returns.
val write_gen :
t ->
length:('a -> int) ->
blit:('a -> src_off:int -> bigstring -> dst_off:int -> len:int -> unit) ->
?off:int ->
?len:int ->
'a ->
unit
write_gen t ~length ~blit ?off ?len x
copies x
into the serializer's internal buffer using the provided length
and blit
operations. See Bigstring.blit
for documentation of the arguments.
write_char t char
copies char
into the serializer's internal buffer.
write_uint8 t n
copies the lower 8 bits of n
into the serializer's internal buffer.
Unbuffered Writes
Unbuffered writes do not involve copying bytes to the serializers internal buffer.
schedule_bigstring t ?off ?len bigstring
schedules bigstring
to be written the next time the serializer surfaces writes to the user. bigstring
is not copied in this process, so bigstring
should only be modified after t
has been flush
ed.
Querying A Serializer's State
free_bytes_in_buffer t
returns the free space, in bytes, of the serializer's write buffer. If a write_*
call has a length that exceeds this value, the serializer will allocate a new buffer that will replace the serializer's internal buffer for that and subsequent calls.
has_pending_output t
is true
if t
's output queue is non-empty. It may be the case that t
's queued output is being serviced by some other thread of control, but has not yet completed.
pending_bytes t
is the size of the next write, in bytes, that t
will surface to the caller as a `Writev
.
Control Operations
yield t
causes t
to delay surfacing writes to the user, instead returning a `Yield
. This gives the serializer an opportunity to collect additional writes before sending them to the underlying device, which will increase the write batch size.
As one example, code may want to call this function if it's about to release the OCaml lock and perform a blocking system call, but would like to batch output across that system call. To hint to the thread of control that is performing the writes on behalf of the serializer, the code might call yield t
before releasing the lock.
flush t f
registers f
to be called when all prior writes have been successfully completed. If t
has no pending writes, then f
will be called immediately. If yield
was recently called on t
, then the effect of the yield
will be ignored so that client code has an opportunity to write pending output, regardless of how it handles `Yield
operations.
close t
closes t
. All subsequent write calls will raise, and any pending or subsequent yield
calls will be ignored. If the serializer has any pending writes, user code will have an opportunity to service them before it receives the Close
operation. Flush callbacks will continue to be invoked while output is shift
ed out of t
as needed.
is_closed t
is true
if close
has been called on t
and false
otherwise. A closed t
may still have pending output.
shift t n
removes the first n
bytes in t
's write queue. Any flush callbacks registered with t
within this span of the write queue will be called.
drain t
removes all pending writes from t
, returning the number of bytes that were enqueued to be written and freeing any scheduled buffers in the process.
Running
Low-level operations for runing a serializer. For production use-cases, consider the Async and Lwt support that this library includes before attempting to use this these operations directly.
A view into iovec.buffer
starting at iovec.off
and with length iovec.len
.
The type of operations that the serialier may wish to perform.
`Writev iovecs
: Write the bytes iniovecs
s reporting the actual number of bytes written by callingshift
. You must accurately report the number of bytes written. Failure to do so will result in the same bytes being surfaced in a`Writev
operation multiple times.`Yield
: Yield to other threads of control, waiting for additional output before procedding. The method for achieving this is application-specific, but once complete, the caller can proceed with serialization by simply making another call tooperation
orserialize
.`Close
: Serialization is complete. No further output will generated. The action to take as a result, if any, is application-specific.
val serialize :
t ->
(bigstring iovec list -> [ `Ok of int | `Closed ]) ->
[ `Yield | `Close ]
serialize t writev
sufaces the next operation of t
to the caller, handling a `Writev
operation with writev
function and performing an additional bookkeeping on the caller's behalf. In the event that writev
indicates a partial write, serialize
will call yield
on the serializer rather than attempting successive writev
calls.
Convenience Functions
These functions are included for testing, debugging, and general development. They are not the suggested way of driving a serializer in a production setting.
serialize_to_string t
runs t
, collecting the output into a string and returning it. serialzie_to_string t
immediately closes t
and ignores any calls to yield
on t
.