package irmin-bench

  1. Overview
  2. Docs
Irmin benchmarking suite

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (3)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

irmin-3.11.0.tbz
sha256=09996fbcc2c43e117a9bd8e9028c635e81cccb264d5e02d425ab8b06bbacdbdb
sha512=0391a6bf7b94a1edd50a3a8df9e58961739fa78d7d689d61f56bc87144483bad2ee539df595c33d9d52c29b3458da5dddf3a73b5eb85e49c4667c26d2cd46be1

doc/irmin-bench.traces/Irmin_traces/Trace_definitions/Stat_trace/index.html

Module Trace_definitions.Stat_traceSource

Trace of a tezos node run, or a replay run.

May be summarised to a JSON file.

Implicitly Auto-Upgradable File Format

The stat trace has these two properties:

  • It supports extensions, in order to change or add new stats in the future.
  • Old trace files from old versions are still readable.

There are multiple reasons for wanting compatibility with old versions:

  • Because one of the goal of the benchmarks is to assess the evolution of performances across distant versions of irmin, we need stability in order to avoid recomputing everything every time.
  • When those traces will be produced by Tezos nodes, we have no control over the version of those traces.

For this system to work, the "decoding shape" of a version of the stat trace shouldn't ever change (once fixed). The way the trace is built for a version should be stable too.

To modify something in the definition or the collection: append a new version.

Sourcemodule V0 : sig ... end
Sourcemodule V1 : sig ... end
Sourcemodule Latest = V1
include module type of struct include Latest end
include module type of struct include V0 end
Sourcetype float32 = int32
Sourceval float32_t : int32 Repr.t
Sourcetype tree = V0.tree = {
  1. contents_hash : int;
  2. contents_find : int;
  3. contents_add : int;
  4. node_hash : int;
  5. node_mem : int;
  6. node_add : int;
  7. node_find : int;
  8. node_val_v : int;
  9. node_val_find : int;
  10. node_val_list : int;
}

Stats extracted from Irmin.Tree.counters ().

Sourceval tree_t : tree Repr.t
Sourcetype index = V0.index = {
  1. bytes_read : int;
  2. nb_reads : int;
  3. bytes_written : int;
  4. nb_writes : int;
  5. nb_merge : int;
  6. new_merge_durations : float list;
}

Stats extracted from Index.Stats.get ().

new_merge_durations is not just a mirror of Index.Stats.merge_durations, it only contains the new entries since the last time it was recorded. This list is always empty when in the header.

Sourceval index_t : index Repr.t
Sourcetype gc = V0.gc = {
  1. minor_words : float;
  2. promoted_words : float;
  3. major_words : float;
  4. minor_collections : int;
  5. major_collections : int;
  6. heap_words : int;
  7. compactions : int;
  8. top_heap_words : int;
  9. stack_size : int;
}

Stats extracted from Gc.quick_stat ().

Sourceval gc_t : gc Repr.t
Sourcetype disk = V0.disk = {
  1. index_data : int64;
  2. index_log : int64;
  3. index_log_async : int64;
  4. store_dict : int64;
  5. store_pack : int64;
}

Stats extracted from filesystem. Requires the path to the irmin store.

Sourceval disk_t : disk Repr.t
Sourcetype 'pack_stats bag_of_stats_base = 'pack_stats V0.bag_of_stats_base = {
  1. pack : 'pack_stats;
  2. tree : tree;
  3. index : index;
  4. gc : gc;
  5. disk : disk;
  6. timestamp_wall : float;
  7. timestamp_cpu : float;
}

Melting pot of stats, recorded before and after every commits.

They are necessary in order to compute any throughput analytics.

Sourceval bag_of_stats_base_t : 'a Repr.t -> 'a bag_of_stats_base Repr.t
Sourcetype store_before = V0.store_before = {
  1. nodes : int;
  2. leafs : int;
  3. skips : int;
  4. depth : int;
  5. width : int;
}

Stats computed from the tree value passed to the commit operation, before the commit, when the tree still carries the modifications brought by the previous operations.

Sourceval store_before_t : store_before Repr.t
Sourcetype watched_node = [
  1. | `Contracts_index
  2. | `Big_maps_index
  3. | `Rolls_index
  4. | `Rolls_owner_current
  5. | `Commitments
  6. | `Contracts_index_ed25519
  7. | `Contracts_index_originated
]
Sourceval watched_node_t : [ `Big_maps_index | `Commitments | `Contracts_index | `Contracts_index_ed25519 | `Contracts_index_originated | `Rolls_index | `Rolls_owner_current ] Repr.t
Sourceval min_watched_node : int
Sourceval max_watched_node : int
Sourceval watched_node_to_enum : [< `Big_maps_index | `Commitments | `Contracts_index | `Contracts_index_ed25519 | `Contracts_index_originated | `Rolls_index | `Rolls_owner_current ] -> int
Sourceval watched_node_of_enum : int -> [> `Big_maps_index | `Commitments | `Contracts_index | `Contracts_index_ed25519 | `Contracts_index_originated | `Rolls_index | `Rolls_owner_current ] option
Sourcetype store_after = V0.store_after = {
  1. watched_nodes_length : int list;
}

Stats computed on the tree value passed to the commit operation, after the commit, when the inode has been reconstructed and that Tree.length is now innexpensive to perform.

Sourceval store_after_t : store_after Repr.t
Sourcetype 'pack_stats commit_base = 'pack_stats V0.commit_base = {
  1. duration : float32;
  2. before : 'pack_stats bag_of_stats_base;
  3. after : 'pack_stats bag_of_stats_base;
  4. store_before : store_before;
  5. store_after : store_after;
}
Sourceval commit_base_t : 'a Repr.t -> 'a commit_base Repr.t
Sourcetype 'pack_stats row_base = [
  1. | `Add of float32
  2. | `Remove of float32
  3. | `Find of float32
  4. | `Mem of float32
  5. | `Mem_tree of float32
  6. | `Checkout of float32
  7. | `Copy of float32
  8. | `Commit of 'pack_stats commit_base
]
Sourceval row_base_t : 'a Repr.t -> [ `Add of int32 | `Checkout of int32 | `Commit of 'a commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] Repr.t
Sourcetype setup_play = unit

Informations gathered from the tezos node.

Noting so far. Any ideas?

Sourceval setup_play_t : unit Repr.t
Sourcetype setup_replay = V0.setup_replay = {
  1. path_conversion : [ `None | `V1 | `V0_and_v1 | `V0 ];
  2. artefacts_dir : string;
}

Informations gathered from the tree.exe parameters.

Sourceval setup_replay_t : setup_replay Repr.t
Sourcetype config = V0.config = {
  1. inode_config : int * int * int;
  2. store_type : [ `Pack | `Pack_layered | `Pack_mem ];
  3. setup : [ `Play of setup_play | `Replay of setup_replay ];
}
Sourceval config_t : config Repr.t
Sourcetype 'pack_stats header_base = 'pack_stats V0.header_base = {
  1. config : config;
  2. hostname : string;
  3. timeofday : float;
  4. word_size : int;
  5. initial_stats : 'pack_stats bag_of_stats_base;
}
Sourceval header_base_t : 'a Repr.t -> 'a header_base Repr.t
Sourceval version : int
Sourcetype finds = V1.finds = {
  1. total : int;
  2. from_staging : int;
  3. from_lru : int;
  4. from_pack_direct : int;
  5. from_pack_indexed : int;
}

Stats extracted from Irmin_pack.Stats.get ().

Sourceval finds_t : finds Repr.t
Sourcetype pack = V1.pack = {
  1. finds : finds;
  2. appended_hashes : int;
  3. appended_offsets : int;
  4. inode_add : int;
  5. inode_remove : int;
  6. inode_of_seq : int;
  7. inode_of_raw : int;
  8. inode_rec_add : int;
  9. inode_rec_remove : int;
  10. inode_to_binv : int;
  11. inode_decode_bin : int;
  12. inode_encode_bin : int;
}

Stats extracted from Irmin_pack.Stats.get ().

Sourceval pack_t : pack Repr.t
Sourcetype commit = pack commit_base
Sourcetype bag_of_stats = pack bag_of_stats_base
Sourceval bag_of_stats_t : pack bag_of_stats_base Repr.t
Sourcetype row = pack row_base
Sourceval row_t : [ `Add of int32 | `Checkout of int32 | `Commit of pack commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] Repr.t
Sourcetype header = pack header_base
Sourceval v1pack_of_v0pack : V0.pack -> pack
Sourceval v1bos_of_v0bos : V0.bag_of_stats -> bag_of_stats
Sourceval v1commit_of_v0commit : V0.commit -> commit
Sourceval v1row_of_v0row : V0.row -> row
Sourceval v1header_of_v0header : V0.header -> header
Sourceval watched_nodes : watched_node list
Sourceval step_list_per_watched_node : (watched_node * string list) list
Sourceval path_per_watched_node : (watched_node * string) list
include sig ... end
Sourceval decode_i32 : int32 Repr.decode_bin
Sourceval encode_i32 : int32 Repr.encode_bin
Sourceval encode_lrow : [ `Add of int32 | `Checkout of int32 | `Commit of V1.pack V0.commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] Repr.encode_bin
Sourceval magic : Irmin_traces__Trace_common.Magic.t
Sourceval read_with_prefix_exn : (string -> int ref -> 'a) -> in_channel -> 'a
Sourceval decoded_seq_of_encoded_chan_with_prefixes : 'a Repr.ty -> in_channel -> unit -> 'a Seq.node
Sourceval open_reader : string -> V1.pack V0.header_base * (unit -> [ `Add of int32 | `Checkout of int32 | `Commit of V1.pack V0.commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] Seq.node)
Sourcetype writer = {
  1. path : string;
  2. channel : out_channel;
  3. buffer : Buffer.t;
}
Sourceval create_file : string -> V1.pack V0.header_base -> writer
Sourceval append_row : writer -> [ `Add of int32 | `Checkout of int32 | `Commit of V1.pack V0.commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] -> unit
Sourceval flush : writer -> unit
Sourceval close : writer -> unit
Sourceval remove : writer -> unit
OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct