mtbl_writer(3) [debian man page]

MTBL_WRITER(3)															    MTBL_WRITER(3)

NAME

       mtbl_writer - create an MTBL file

SYNOPSIS

       #include <mtbl.h>

       Writer objects:

       struct mtbl_writer *
       mtbl_writer_init(const char *fname, const struct mtbl_writer_options *wopt);

       struct mtbl_writer *
       mtbl_writer_init_fd(int fd, const struct mtbl_writer_options *wopt);

       void
       mtbl_writer_destroy(struct mtbl_writer **w);

       mtbl_res
       mtbl_writer_add(struct mtbl_writer *w,
	       const uint8_t *key, size_t len_key,
	       const uint8_t *val, size_t len_val);

       Writer options:

       struct mtbl_writer_options *
       mtbl_writer_options_init(void);

       void
       mtbl_writer_options_destroy(struct mtbl_writer_options **wopt);

       void
       mtbl_writer_options_set_compression(
	       struct mtbl_writer_options *wopt,
	       mtbl_compression_type compression_type);

       void
       mtbl_writer_options_set_block_size(
	       struct mtbl_writer_options *wopt,
	       size_t block_size);

       void
       mtbl_writer_options_set_block_restart_interval(
	       struct mtbl_writer_options *wopt,
	       size_t block_restart_interval);

DESCRIPTION

       MTBL files are written to disk by creating an mtbl_writer object, calling mtbl_writer_add() for each key-value entry, and then calling
       mtbl_writer_destroy().

       mtbl_writer_add() copies key-value pairs from the caller into the mtbl_writer object. Keys are specified as a pointer to a buffer, key, and
       the length of that buffer, len_key. Values are specified as a pointer to a buffer, val, and the length of that buffer, len_val.

       Keys must be in sorted, lexicographical byte order. The same key may not be added to an mtbl_writer more than once. If the input entries
       are not sorted or may contain duplicate keys, then the mtbl_sorter(3) interface should be used instead.

       mtbl_writer objects may be created by calling mtbl_writer_init() with an fname argument specifying a filename to be created. The filename
       must not already exist on the filesystem. Or, mtbl_writer_init_fd() may be called with an fd argument specifying an open, writable file
       descriptor. No data may have been written to the file descriptor.

       If the wopt parameter to mtbl_writer_init() or mtbl_writer_init_fd() is non-NULL, the parameters specified in the mtbl_writer_options
       object will be configured into the mtbl_writer object.

   Writer options
       compression
	   Specifies the compression algorithm to use on data blocks. Possible values are MTBL_COMPRESSION_NONE, MTBL_COMPRESSION_SNAPPY, or
	   MTBL_COMPRESSION_ZLIB (the default).

       block_size
	   The maximum size of uncompressed data blocks, specified in bytes. The default is 8 kilobytes.

       block_restart_interval
	   How frequently to restart intra-block key prefix compression. The default is every 16 keys.

RETURN VALUE

       mtbl_writer_init() and mtbl_writer_init_fd() return NULL on failure, and non-NULL on success.

       mtbl_writer_add() returns mtbl_res_success if the key-value entry was successfully copied into the mtbl_writer object, and mtbl_res_failure
       if not, for instance if there has been a key-ordering violation.

								    05/29/2012							    MTBL_WRITER(3)

MTBL_MERGER(3)															    MTBL_MERGER(3)

NAME

       mtbl_merger - merge multiple MTBL data sources into a single output

SYNOPSIS

       #include <mtbl.h>

       Merger objects:

       struct mtbl_merger *
       mtbl_merger_init(const struct mtbl_merger_options *mopt);

       void
       mtbl_merger_destroy(struct mtbl_merger **m);

       void
       mtbl_merger_add_source(struct mtbl_merger *m, const struct mtbl_source *s);

       const struct mtbl_source *
       mtbl_merger_source(struct mtbl_merger *m);

       Merger options:

       struct mtbl_merger_options *
       mtbl_merger_options_init(void);

       void
       mtbl_merger_options_destroy(struct mtbl_merger_options **mopt);

       void
       mtbl_merger_options_set_merge_func(
	       struct mtbl_merger_options *mopt,
	       mtbl_merge_func fp,
	       void *clos);

       typedef void
       (*mtbl_merge_func)(void *clos,
	       const uint8_t *key, size_t len_key,
	       const uint8_t *val0, size_t len_val0,
	       const uint8_t *val1, size_t len_val1,
	       uint8_t **merged_val, size_t *len_merged_val);

DESCRIPTION

       Multiple MTBL data sources may be merged together using the mtbl_merger interface, which reads key-value entries from one or more sources
       and provides these entries in sorted order. The sorted entries may be consumed via the mtbl_source(3) and mtbl_iter(3) interfaces.

       Because the MTBL format does not allow duplicate keys, the caller must provide a function which will accept a key and two conflicting
       values for that key and return a replacement value. This function may be called multiple times for the same key if more than two sources
       are being merged.

       mtbl_merger objects are created with the mtbl_merger_init() function, which requires a non-NULL mopt argument which has been configured
       with a merge function fp.

       One or more mtbl_reader objects must be provided as input to the mtbl_merger object by calling mtbl_merger_add_source(). After the desired
       sources have been configured, mtbl_merger_source() should be called in order to consume the merged output via the mtbl_source(3) interface.

   Merger options
       merge_func
	   This option specifies a merge function callback, consisting of a function pointer fp and a pointer to user data clos which will be
	   passed as the first argument to fp. The merge function callback will be used during iteration over the mtbl_merger object to merge
	   entries with duplicate keys in the input sources.

	   The remaining arguments to the merge function are:

	   key -- pointer to the key for which there exist duplicate values.

	   len_key -- length of the key.

	   val0 -- pointer to the first value.

	   len_val_0 -- length of the first value.

	   val1 -- pointer to the second value.

	   len_val_1 -- length of the second value.

	   merged_val -- pointer to where the callee should place its merged value.

	   len_merged_val -- pointer to where the callee should place the length of its merged value.

	   merged_val must be allocated with the system allocator, and the mtbl_merger interface takes responsibility for free()ing the value once
	   it is no longer needed.

	   The callee may provide an empty value as the merged value, in which case merged_val must still contain an allocated, non-NULL value and
	   len_merged_val must contain the value 0.

	   The callee may indicate an error by returning NULL in the merged_val argument, which will abort iteration over the mtbl_merger object.

RETURN VALUE

       If the merge function callback is unable to provide a merged value (that is, it fails to return a non-NULL value in its merged_val
       argument), the merge process will be aborted, and any iterators over the mtbl_merger object (via the mtbl_source(3) interface) will return
       mtbl_res_failure.

								    05/29/2012							    MTBL_MERGER(3)