Blosc1 API

This is the classic API from Blosc1 with 32-bit limited containers.

Main API

void blosc2_init(void)

Initialize the Blosc library environment.

You must call this previous to any other Blosc call, unless you want Blosc to be used simultaneously in a multi-threaded environment, in which case you can use the blosc2_compress_ctx blosc2_decompress_ctx pair.

See also

blosc2_destroy

void blosc2_destroy(void)

Destroy the Blosc library environment.

You must call this after to you are done with all the Blosc calls, unless you have not used blosc2_init() before.

See also

blosc2_init

int blosc1_compress(int clevel, int doshuffle, size_t typesize, size_t nbytes, const void *src, void *dest, size_t destsize)

Compress a block of data in the src buffer and returns the size of compressed block.

Remark

Compression is memory safe and guaranteed not to write dest more than what is specified in destsize. There is not a minimum for src buffer size nbytes. Equivalent to blosc2_compress.

Environment variables

This function honors different environment variables to control internal parameters without the need of doing that programmatically. Here are the ones supported:

• BLOSC_CLEVEL=(INTEGER): This will overwrite the clevel parameter before the compression process starts.

• BLOSC_SHUFFLE=[NOSHUFFLE | SHUFFLE | BITSHUFFLE]: This will overwrite the doshuffle parameter before the compression process starts.

• BLOSC_DELTA=(1|0): This will call blosc2_set_delta() before the compression process starts.

• BLOSC_TYPESIZE=(INTEGER): This will overwrite the typesize parameter before the compression process starts.

• BLOSC_COMPRESSOR=[BLOSCLZ | LZ4 | LZ4HC | ZLIB | ZSTD]: This will call blosc1_set_compressor before the compression process starts.

• BLOSC_NTHREADS=(INTEGER): This will call blosc2_set_nthreads before the compression process starts.

• BLOSC_SPLITMODE=(ALWAYS | NEVER | AUTO | FORWARD_COMPAT): This will call blosc1_set_splitmode() before the compression process starts.

• BLOSC_BLOCKSIZE=(INTEGER): This will call blosc1_set_blocksize before the compression process starts. NOTE: The blocksize is a critical parameter with important restrictions in the allowed values, so use this with care.

• BLOSC_NOLOCK=(ANY VALUE): This will call blosc2_compress_ctx under the hood, with the compressor, blocksize and numinternalthreads parameters set to the same as the last calls to blosc1_set_compressor, blosc1_set_blocksize and blosc2_set_nthreads. BLOSC_CLEVEL, BLOSC_SHUFFLE, BLOSC_DELTA and BLOSC_TYPESIZE environment vars will also be honored.

See also

blosc1_decompress

Parameters

• clevel – The desired compression level and must be a number between 0 (no compression) and 9 (maximum compression).

• doshuffle – Specifies whether the shuffle compression preconditioner should be applied or not. BLOSC_NOFILTER means not applying filters, BLOSC_SHUFFLE means applying shuffle at a byte level and BLOSC_BITSHUFFLE at a bit level (slower but may achieve better compression).

• typesize – Is the number of bytes for the atomic type in binary src buffer. This is mainly useful for the shuffle preconditioner. For implementation reasons, only a 1 < typesize < 256 will allow the shuffle filter to work. When typesize is not in this range, shuffle will be silently disabled.

• nbytes – The number of bytes to compress in the src buffer.

• src – The buffer containing the data to compress.

• dest – The buffer where the compressed data will be put, must have at least the size of destsize.

• destsize – The size of the dest buffer. Blosc guarantees that if you set destsize to, at least, (nbytes + BLOSC2_MAX_OVERHEAD), the compression will always succeed.

Returns

The number of bytes compressed. If src buffer cannot be compressed into destsize, the return value is zero and you should discard the contents of the dest buffer. A negative return value means that an internal error happened. This should never happen. If you see this, please report it back together with the buffer data causing this and compression settings.

Warning

The src buffer and the dest buffer can not overlap.

int blosc1_decompress(const void *src, void *dest, size_t destsize)

Decompress a block of compressed data in src, put the result in dest and returns the size of the decompressed block.

Remark

Decompression is memory safe and guaranteed not to write the dest buffer more than what is specified in destsize. Similar to blosc2_decompress.

Remark

In case you want to keep under control the number of bytes read from source, you can call blosc1_cbuffer_sizes first to check whether the nbytes (i.e. the number of bytes to be read from src buffer by this function) in the compressed buffer is ok with you.

Environment variables

This function honors different environment variables to control internal parameters without the need of doing that programmatically. Here are the ones supported:

• BLOSC_NTHREADS=(INTEGER): This will call blosc2_set_nthreads before the proper decompression process starts.

• BLOSC_NOLOCK=(ANY VALUE): This will call blosc2_decompress_ctx under the hood, with the numinternalthreads parameter set to the same value as the last call to blosc2_set_nthreads.

See also

blosc1_compress

Parameters

• src – The buffer to be decompressed.

• dest – The buffer where the decompressed data will be put.

• destsize – The size of the dest buffer.

Returns

The number of bytes decompressed. If an error occurs, e.g. the compressed data is corrupted or the output buffer is not large enough, then a negative value will be returned instead.

Warning

The src buffer and the dest buffer can not overlap.

int blosc1_getitem(const void *src, int start, int nitems, void *dest)

Get nitems (of typesize size) in src buffer starting in start.

The items are returned in dest buffer, which has to have enough space for storing all items.

Remark

The function blosc2_getitem is a more complete and secure version.

Parameters

• src – The compressed buffer from data will be decompressed.

• start – The position of the first item (of typesize size) from where data will be retrieved.

• nitems – The number of items (of typesize size) that will be retrieved.

• dest – The buffer where the decompressed data retrieved will be put.

Returns

The number of bytes copied to dest or a negative value if some error happens.

int16_t blosc2_get_nthreads(void)

Returns the current number of threads that are used for compression/decompression.

int16_t blosc2_set_nthreads(int16_t nthreads)

Initialize a pool of threads for compression/decompression.

If nthreads is 1, then the serial version is chosen and a possible previous existing pool is ended. If this is not called, nthreads is set to 1 internally.

Parameters

• nthreads – The number of threads to use.

Returns

The previous number of threads.

typedef void (*blosc_threads_callback)(void *callback_data, void (*dojob)(void*), int numjobs, size_t jobdata_elsize, void *jobdata)

Pointer to a callback function that executes dojob(jobdata + i*jobdata_elsize) for i = 0 to numjobs-1, possibly in parallel threads (but not returning until all dojob calls have returned).

This allows the caller to provide a custom threading backend as an alternative to the default Blosc-managed threads. callback_data is passed through from blosc2_set_threads_callback.

void blosc2_set_threads_callback(blosc_threads_callback callback, void *callback_data)

Set the threading backend for parallel compression/decompression to use callback to execute work instead of using the Blosc-managed threads.

The callback_data argument is passed through to the callback.

See also

Blosc/c-blosc2#81

Parameters

• callback – the callback to use. Passing NULL uses the default Blosc threading backend.

• callback_data – the callback data.

Warning

This function is not thread-safe and should be called before any other Blosc function: it affects all Blosc contexts.

const char *blosc1_get_compressor(void)

Get the current compressor that is used for compression.

Returns

The string identifying the compressor being used.

int blosc1_set_compressor(const char *compname)

Select the compressor to be used.

The supported ones are “blosclz”, “lz4”, “lz4hc”, “zlib” and “ztsd”. If this function is not called, then “blosclz” will be used.

Parameters

• compname – The name identifier of the compressor to be set.

Returns

The code for the compressor (>=0). In case the compressor is not recognized, or there is not support for it in this build, it returns a -1.

void blosc2_set_delta(int dodelta)

Select the delta coding filter to be used.

This call should always succeed.

Parameters

• dodelta – A value >0 will activate the delta filter. If 0, it will be de-activated

int blosc1_get_blocksize(void)

Get the internal blocksize to be used during compression.

0 means that an automatic blocksize is computed internally.

Returns

The size in bytes of the internal block size.

void blosc1_set_blocksize(size_t blocksize)

Force the use of a specific blocksize.

If 0, an automatic blocksize will be used (the default).

Warning

The blocksize is a critical parameter with important restrictions in the allowed values, so use this with care.

void blosc1_set_splitmode(int splitmode)

Set the split mode.

BLOSC_FORWARD_COMPAT offers reasonably forward compatibility, BLOSC_AUTO_SPLIT is for nearly optimal results (based on heuristics), BLOSC_NEVER_SPLIT and BLOSC_ALWAYS_SPLIT are for the user experimenting when trying to get best compression ratios and/or speed.

If not called, the default mode is BLOSC_FORWARD_COMPAT_SPLIT.

This function should always succeed.

Parameters

• splitmode – It can take the next values: BLOSC_FORWARD_COMPAT_SPLIT BLOSC_AUTO_SPLIT BLOSC_NEVER_SPLIT BLOSC_ALWAYS_SPLIT

int blosc2_free_resources(void)

Free possible memory temporaries and thread resources.

Use this when you are not going to use Blosc for a long while.

Returns

A 0 if succeeds, in case of problems releasing the resources, it returns a negative number.

Compressed buffer information

void blosc1_cbuffer_sizes(const void *cbuffer, size_t *nbytes, size_t *cbytes, size_t *blocksize)

Get information about a compressed buffer, namely the number of uncompressed bytes (nbytes) and compressed (cbytes).

It also returns the blocksize (which is used internally for doing the compression by blocks).

You only need to pass the first BLOSC_MIN_HEADER_LENGTH bytes of a compressed buffer for this call to work.

Remark

Equivalent to function blosc2_cbuffer_sizes.

This function should always succeed.

Parameters

• cbuffer – The buffer of compressed data.

• nbytes – The pointer where the number of uncompressed bytes will be put.

• cbytes – The pointer where the number of compressed bytes will be put.

• blocksize – The pointer where the block size will be put.

void blosc1_cbuffer_metainfo(const void *cbuffer, size_t *typesize, int *flags)

Get information about a compressed buffer, namely the type size (typesize), as well as some internal flags.

You can use the BLOSC_DOSHUFFLE, BLOSC_DOBITSHUFFLE, BLOSC_DODELTA and BLOSC_MEMCPYED symbols for extracting the interesting bits (e.g. flags & BLOSC_DOSHUFFLE says whether the buffer is byte-shuffled or not).

This function should always succeed.

Parameters

• cbuffer – The buffer of compressed data.

• typesize – The pointer where the type size will be put.

• flags – The pointer of the integer where the additional info is encoded. The flags is a set of bits, where the currently used ones are:

  • bit 0: whether the shuffle filter has been applied or not

  • bit 1: whether the internal buffer is a pure memcpy or not

  • bit 2: whether the bitshuffle filter has been applied or not

  • bit 3: whether the delta coding filter has been applied or not

void blosc2_cbuffer_versions(const void *cbuffer, int *version, int *versionlz)

Get information about a compressed buffer, namely the internal Blosc format version (version) and the format for the internal Lempel-Ziv compressor used (versionlz).

This function should always succeed.

Parameters

• cbuffer – The buffer of compressed data.

• version – The pointer where the Blosc format version will be put.

• versionlz – The pointer where the Lempel-Ziv version will be put.

const char *blosc2_cbuffer_complib(const void *cbuffer)

Get the compressor library/format used in a compressed buffer.

This function should always succeed.

Parameters

• cbuffer – The buffer of compressed data.

Returns

The string identifying the compressor library/format used.

int blosc1_cbuffer_validate(const void *cbuffer, size_t cbytes, size_t *nbytes)

Checks that the compressed buffer starting at cbuffer of length cbytes may contain valid blosc compressed data, and that it is safe to call blosc1_decompress/blosc1_getitem.

On success, returns 0 and sets nbytes to the size of the uncompressed data. This does not guarantee that the decompression function won’t return an error, but does guarantee that it is safe to attempt decompression.

Parameters

• cbuffer – The buffer of compressed data.

• cbytes – The number of compressed bytes.

• nbytes – The pointer where the number of uncompressed bytes will be put.

Returns

On failure, returns negative value.

Utility functions

int blosc2_compcode_to_compname(int compcode, const char **compname)

Get the compressor name associated with the compressor code.

Parameters

• compcode – The code identifying the compressor

• compname – The pointer to a string where the compressor name will be put.

Returns

The compressor code. If the compressor code is not recognized, or there is not support for it in this build, -1 is returned.

int blosc2_compname_to_compcode(const char *compname)

Get the compressor code associated with the compressor name.

Parameters

• compname – The string containing the compressor name.

Returns

The compressor code. If the compressor name is not recognized, or there is not support for it in this build, -1 is returned instead.

const char *blosc2_list_compressors(void)

Get a list of compressors supported in the current build.

This function does not leak, so you should not free() the returned list.

This function should always succeed.

Returns

The comma separated string with the list of compressor names supported.

const char *blosc2_get_version_string(void)

Get the version of Blosc in string format.

Returns

The string with the current Blosc version. Useful for dynamic libraries.

int blosc2_get_complib_info(const char *compname, char **complib, char **version)

Get info from compression libraries included in the current build.

Parameters

• compname – The compressor name that you want info from.

• complib – The pointer to a string where the compression library name, if available, will be put.

• version – The pointer to a string where the compression library version, if available, will be put.

Returns

The code for the compression library (>=0). If it is not supported, this function returns -1.

Warning

You are in charge of the complib and version strings, you should free() them so as to avoid leaks.