package irmin-bench

  1. Overview
  2. Docs
Irmin benchmarking suite

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (3)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

irmin-3.11.0.tbz
sha256=09996fbcc2c43e117a9bd8e9028c635e81cccb264d5e02d425ab8b06bbacdbdb
sha512=0391a6bf7b94a1edd50a3a8df9e58961739fa78d7d689d61f56bc87144483bad2ee539df595c33d9d52c29b3458da5dddf3a73b5eb85e49c4667c26d2cd46be1

doc/irmin-bench.traces/Irmin_traces/Trace_definitions/Stat_trace/V0/index.html

Module Stat_trace.V0Source

Sourceval version : int
Sourcetype float32 = int32
Sourceval float32_t : int32 Repr.t
Sourcetype pack = {
  1. finds : int;
  2. cache_misses : int;
  3. appended_hashes : int;
  4. appended_offsets : int;
}

Stats extracted from Irmin_pack.Stats.get ().

Sourceval pack_t : pack Repr.t
Sourcetype tree = {
  1. contents_hash : int;
  2. contents_find : int;
  3. contents_add : int;
  4. node_hash : int;
  5. node_mem : int;
  6. node_add : int;
  7. node_find : int;
  8. node_val_v : int;
  9. node_val_find : int;
  10. node_val_list : int;
}

Stats extracted from Irmin.Tree.counters ().

Sourceval tree_t : tree Repr.t
Sourcetype index = {
  1. bytes_read : int;
  2. nb_reads : int;
  3. bytes_written : int;
  4. nb_writes : int;
  5. nb_merge : int;
  6. new_merge_durations : float list;
}

Stats extracted from Index.Stats.get ().

new_merge_durations is not just a mirror of Index.Stats.merge_durations, it only contains the new entries since the last time it was recorded. This list is always empty when in the header.

Sourceval index_t : index Repr.t
Sourcetype gc = {
  1. minor_words : float;
  2. promoted_words : float;
  3. major_words : float;
  4. minor_collections : int;
  5. major_collections : int;
  6. heap_words : int;
  7. compactions : int;
  8. top_heap_words : int;
  9. stack_size : int;
}

Stats extracted from Gc.quick_stat ().

Sourceval gc_t : gc Repr.t
Sourcetype disk = {
  1. index_data : int64;
  2. index_log : int64;
  3. index_log_async : int64;
  4. store_dict : int64;
  5. store_pack : int64;
}

Stats extracted from filesystem. Requires the path to the irmin store.

Sourceval disk_t : disk Repr.t
Sourcetype 'pack_stats bag_of_stats_base = {
  1. pack : 'pack_stats;
  2. tree : tree;
  3. index : index;
  4. gc : gc;
  5. disk : disk;
  6. timestamp_wall : float;
  7. timestamp_cpu : float;
}

Melting pot of stats, recorded before and after every commits.

They are necessary in order to compute any throughput analytics.

Sourceval bag_of_stats_base_t : 'a Repr.t -> 'a bag_of_stats_base Repr.t
Sourcetype store_before = {
  1. nodes : int;
  2. leafs : int;
  3. skips : int;
  4. depth : int;
  5. width : int;
}

Stats computed from the tree value passed to the commit operation, before the commit, when the tree still carries the modifications brought by the previous operations.

Sourceval store_before_t : store_before Repr.t
Sourcetype watched_node = [
  1. | `Contracts_index
  2. | `Big_maps_index
  3. | `Rolls_index
  4. | `Rolls_owner_current
  5. | `Commitments
  6. | `Contracts_index_ed25519
  7. | `Contracts_index_originated
]
Sourceval watched_node_t : [ `Big_maps_index | `Commitments | `Contracts_index | `Contracts_index_ed25519 | `Contracts_index_originated | `Rolls_index | `Rolls_owner_current ] Repr.t
Sourceval min_watched_node : int
Sourceval max_watched_node : int
Sourceval watched_node_to_enum : [< `Big_maps_index | `Commitments | `Contracts_index | `Contracts_index_ed25519 | `Contracts_index_originated | `Rolls_index | `Rolls_owner_current ] -> int
Sourceval watched_node_of_enum : int -> [> `Big_maps_index | `Commitments | `Contracts_index | `Contracts_index_ed25519 | `Contracts_index_originated | `Rolls_index | `Rolls_owner_current ] option
Sourcetype store_after = {
  1. watched_nodes_length : int list;
}

Stats computed on the tree value passed to the commit operation, after the commit, when the inode has been reconstructed and that Tree.length is now innexpensive to perform.

Sourceval store_after_t : store_after Repr.t
Sourcetype 'pack_stats commit_base = {
  1. duration : float32;
  2. before : 'pack_stats bag_of_stats_base;
  3. after : 'pack_stats bag_of_stats_base;
  4. store_before : store_before;
  5. store_after : store_after;
}
Sourceval commit_base_t : 'a Repr.t -> 'a commit_base Repr.t
Sourcetype 'pack_stats row_base = [
  1. | `Add of float32
  2. | `Remove of float32
  3. | `Find of float32
  4. | `Mem of float32
  5. | `Mem_tree of float32
  6. | `Checkout of float32
  7. | `Copy of float32
  8. | `Commit of 'pack_stats commit_base
]
Sourceval row_base_t : 'a Repr.t -> [ `Add of int32 | `Checkout of int32 | `Commit of 'a commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] Repr.t
Sourcetype row = pack row_base

Stats gathered while running an operation.

Operation durations

For each operation we record its wall time length using a float32, a float16 would be suitable too (it has >3 digits of precision).

Time and disk performance considerations

On commit we record a lot of things, thankfuly the frequency is low: ~1/600. 599 small operations weigh ~3600 bytes, 1 commit weighs ~300 bytes. The trace reaches 1GB after ~250k commits.

Sourceval row_t : [ `Add of int32 | `Checkout of int32 | `Commit of pack commit_base | `Copy of int32 | `Find of int32 | `Mem of int32 | `Mem_tree of int32 | `Remove of int32 ] Repr.t
Sourcetype setup_play = unit

Informations gathered from the tezos node.

Noting so far. Any ideas?

Sourceval setup_play_t : unit Repr.t
Sourcetype setup_replay = {
  1. path_conversion : [ `None | `V1 | `V0_and_v1 | `V0 ];
  2. artefacts_dir : string;
}

Informations gathered from the tree.exe parameters.

Sourceval setup_replay_t : setup_replay Repr.t
Sourcetype config = {
  1. inode_config : int * int * int;
  2. store_type : [ `Pack | `Pack_layered | `Pack_mem ];
  3. setup : [ `Play of setup_play | `Replay of setup_replay ];
}
Sourceval config_t : config Repr.t
Sourcetype 'pack_stats header_base = {
  1. config : config;
  2. hostname : string;
  3. timeofday : float;
  4. word_size : int;
  5. initial_stats : 'pack_stats bag_of_stats_base;
}
Sourceval header_base_t : 'a Repr.t -> 'a header_base Repr.t
Sourcetype header = pack header_base

File header.

Timestamps

stats.timestamp_wall and stats.timestamp_cpu are the starting points of the trace, they are to be substracted from their counterpart in commit to compute time spans.

timeofday is the date and time at which the stats started to be accumulated.

stats.timestamp_wall may originate from Mtime_clock.now.

stats.timestamp_cpu may originate from Sys.time.

It would be great to be able to record the library/sources versions.

Sourcetype commit = pack commit_base
Sourcetype bag_of_stats = pack bag_of_stats_base
Sourceval bag_of_stats_t : pack bag_of_stats_base Repr.t
OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct