mtbl_fileset(3) [debian man page]

MTBL_FILESET(3) 														   MTBL_FILESET(3)

NAME

       mtbl_fileset - automatic multiple MTBL data file merger

SYNOPSIS

       #include <mtbl.h>

       Fileset objects:

       struct mtbl_fileset *
       mtbl_fileset_init(const char *fname, const struct mtbl_fileset_options *fopt);

       void
       mtbl_fileset_destroy(struct mtbl_fileset **f);

       void
       mtbl_fileset_reload(struct mtbl_fileset *f);

       const struct mtbl_source *
       mtbl_fileset_source(struct mtbl_fileset *f);

       Fileset options:

       struct mtbl_fileset_options *
       mtbl_fileset_options_init(void);

       void
       mtbl_fileset_options_destroy(struct mtbl_fileset_options **fopt);

       void
       mtbl_fileset_options_set_merge_func(
	       struct mtbl_fileset_options *fopt,
	       mtbl_merge_func fp,
	       void *clos);

       void
       mtbl_fileset_options_set_reload_frequency(
	       struct mtbl_fileset_options *fopt,
	       uint32_t reload_frequency);

DESCRIPTION

       The mtbl_fileset is a convenience interface for automatically maintaining a merged view of a set of MTBL data files. The merged entries may
       be consumed via the mtbl_source(3) and mtbl_iter(3) interfaces.

       mtbl_fileset objects are initialized from a "setfile", which specifies a list of filenames of MTBL data files, one per line. Internally, an
       mtbl_reader object is initialized from each filename and added to an mtbl_merger object. The setfile is watched for changes and the
       addition or removal of filenames from the setfile will result in the corresponding addition or removal of mtbl_reader objects.

       Because the MTBL format does not allow duplicate keys, the caller must provide a function which will accept a key and two conflicting
       values for that key and return a replacement value. This function may be called multiple times for the same key if the same key is inserted
       more than twice. See mtbl_merger(3) for further details about the merge function.

       mtbl_fileset objects are created with the mtbl_fileset_init() function, which requires the path to a "setfile", fname, and a non-NULL fopt
       argument which has been configured with a merge function fp. mtbl_fileset_source() should then be called in order to consume output via the
       mtbl_source(3) interface.

       Accesses via the mtbl_source(3) interface will implicitly check for updates to the setfile. However, it may be necessary to explicitly call
       the mtbl_fileset_reload() function in order to check for updates, especially if files are being removed from the setfile.

   Fileset options
       merge_func
	   See mtbl_merger(3). An mtbl_merger object is used internally for the external sort.

       reload_interval
	   Specifies the interval between checks for updates to the setfile, in seconds. Defaults to 60 seconds.

								    05/29/2012							   MTBL_FILESET(3)

MTBL_WRITER(3)															    MTBL_WRITER(3)

NAME

       mtbl_writer - create an MTBL file

SYNOPSIS

       #include <mtbl.h>

       Writer objects:

       struct mtbl_writer *
       mtbl_writer_init(const char *fname, const struct mtbl_writer_options *wopt);

       struct mtbl_writer *
       mtbl_writer_init_fd(int fd, const struct mtbl_writer_options *wopt);

       void
       mtbl_writer_destroy(struct mtbl_writer **w);

       mtbl_res
       mtbl_writer_add(struct mtbl_writer *w,
	       const uint8_t *key, size_t len_key,
	       const uint8_t *val, size_t len_val);

       Writer options:

       struct mtbl_writer_options *
       mtbl_writer_options_init(void);

       void
       mtbl_writer_options_destroy(struct mtbl_writer_options **wopt);

       void
       mtbl_writer_options_set_compression(
	       struct mtbl_writer_options *wopt,
	       mtbl_compression_type compression_type);

       void
       mtbl_writer_options_set_block_size(
	       struct mtbl_writer_options *wopt,
	       size_t block_size);

       void
       mtbl_writer_options_set_block_restart_interval(
	       struct mtbl_writer_options *wopt,
	       size_t block_restart_interval);

DESCRIPTION

       MTBL files are written to disk by creating an mtbl_writer object, calling mtbl_writer_add() for each key-value entry, and then calling
       mtbl_writer_destroy().

       mtbl_writer_add() copies key-value pairs from the caller into the mtbl_writer object. Keys are specified as a pointer to a buffer, key, and
       the length of that buffer, len_key. Values are specified as a pointer to a buffer, val, and the length of that buffer, len_val.

       Keys must be in sorted, lexicographical byte order. The same key may not be added to an mtbl_writer more than once. If the input entries
       are not sorted or may contain duplicate keys, then the mtbl_sorter(3) interface should be used instead.

       mtbl_writer objects may be created by calling mtbl_writer_init() with an fname argument specifying a filename to be created. The filename
       must not already exist on the filesystem. Or, mtbl_writer_init_fd() may be called with an fd argument specifying an open, writable file
       descriptor. No data may have been written to the file descriptor.

       If the wopt parameter to mtbl_writer_init() or mtbl_writer_init_fd() is non-NULL, the parameters specified in the mtbl_writer_options
       object will be configured into the mtbl_writer object.

   Writer options
       compression
	   Specifies the compression algorithm to use on data blocks. Possible values are MTBL_COMPRESSION_NONE, MTBL_COMPRESSION_SNAPPY, or
	   MTBL_COMPRESSION_ZLIB (the default).

       block_size
	   The maximum size of uncompressed data blocks, specified in bytes. The default is 8 kilobytes.

       block_restart_interval
	   How frequently to restart intra-block key prefix compression. The default is every 16 keys.

RETURN VALUE

       mtbl_writer_init() and mtbl_writer_init_fd() return NULL on failure, and non-NULL on success.

       mtbl_writer_add() returns mtbl_res_success if the key-value entry was successfully copied into the mtbl_writer object, and mtbl_res_failure
       if not, for instance if there has been a key-ordering violation.

								    05/29/2012							    MTBL_WRITER(3)