Module Eris_lwt

Encoding for Robust Immutable Storage (ERIS)

ERIS is an encoding of arbitrary content into a set of uniformly sized, encrypted and content-addressed blocks as well as a short identifier that can be encoded as an URN. The content can be reassembled from the blocks only with this identifier. The encoding is defined independent of any storage and transport layer or any specific application

Basics

val spec_version : string

Version of the ERIS specification implemented.

Block Size

ERIS uses two block sizes which must be specified when encoding content.

type block_size = [
  1. | `Large
    (*

    Block size of 32 KiB (32768 bytes). Use this if the content to be encoded is larger than 16 KiB.

    *)
  2. | `Small
    (*

    Block size of 1 KiB (1024 bytes). Use this if the content to be encoded is smaller than 16 KiB.

    *)
]

Read Capability

module Read_capability : sig ... end

A read capability contains the necessary information to decode some encoded content from blocks. A read capability can be encoded as a binary string or as an URN.

Blocks and References

type ref = string

Type of a reference to a block. A reference to a block is the Blake2b-256 hash of the block.

type block = string

Type of a block. Length of a block is either 1024 bytes (for block size `Small) or 32678 bytes (for block size `Large).

Encoding

Inputs to the encoding process are:

The output is a read capability and a set of blocks.

val null_convergence_secret : string

The null convergence-secret (32 bytes of zeroes). This may be used if the known attacks against convergent encryption are well understood and the advantages of deterministic identifiers and de-duplication outweigh.

module Encoder : sig ... end

This module provides a streaming encoder that can be used to efficiently encode content piece-wise. The memory used is proportional to the level of the encoded content. The level is related logarithmicly to the size of the content.

val encode_string : convergence_secret:string -> block_size:block_size -> string -> (ref * block) list * Read_capability.t

encode_string ~convergence_secret ~block_size content encodes content using the provided convergence-secret and block size and returns a list of blocks as well as a read capability.

Decoding

Content can be decoded given the read capability and access to blocks.

module Decoder : sig ... end

This module provides a random-access decoder that can be used to efficiently decode pieces of content at given positions.

val decode_string : block_get:(ref -> block Lwt.t) -> Read_capability.t -> string Lwt.t

decode_string ~block_get read_capability returns the entire decoded content as string.