C2Array¶

This is a class for remote arrays. This kind of array can also work as operand on a LazyExpr, LazyUDF or reduction.

class blosc2.C2Array(path: str, /, urlbase: str | None = None, auth_token: str | None = None)[source]¶

Attributes:

blocks: The blocks of the remote array
blocksize: The block size (in bytes) for the remote container.
cbytes: The number of compressed bytes of the remote array
chunks: The chunks of the remote array
cparams: The compression parameters of the remote array
cratio: The compression ratio of the remote array
device: Hardware device the array data resides on.
dtype: The dtype of the remote array
info: Print information about this remote array.
info_items: A list of tuples with the information about the remote array.
nbytes: The number of bytes of the remote array
ndim: Get the number of dimensions of the Operand.
shape: The shape of the remote array
vlmeta: The variable-length metadata f the remote array

Methods

`get_chunk`(nchunk)	Get the compressed unidimensional chunk of a C2Array.
`item`()	Copy an element of an array to a standard Python scalar and return it.
`slice`(slice_)	Get a slice of the array (returning blosc2 NDArray array).
`to_device`(device)	Copy the array from the device on which it currently resides to the specified device.
`where`([value1, value2])	Select `value1` or `value2` values based on `True`/`False` for `self`.

Special Methods:

`__init__`(path, /[, urlbase, auth_token])	Create an instance of a remote NDArray.
`__getitem__`(slice_)	Get a slice of the array (returning NumPy array).

Constructor¶

__init__(path: str, /, urlbase: str | None = None, auth_token: str | None = None)[source]¶

Create an instance of a remote NDArray.

Remote NDArrays can be accessed via HTTP from a Caterva2 server (e.g., https://cat2.cloud). More information about Caterva2 at: https://ironarray.io/caterva2.

Parameters:

path¶ (str) – The path to the remote NDArray file (root + file path) as a posix path.
urlbase¶ (str) – The base URL (slash-terminated) of the subscriber to query.
auth_token¶ (str) – An optional token to authorize requests via HTTP. Currently, it will be sent as an HTTP cookie.

Returns:

out

Return type:

C2Array

Examples

>>> import blosc2
>>> urlbase = "https://cat2.cloud/demo"
>>> path = "@public/examples/dir1/ds-3d.b2nd"
>>> remote_array = blosc2.C2Array(path, urlbase=urlbase)
>>> remote_array.shape
(3, 4, 5)
>>> remote_array.chunks
(2, 3, 4)
>>> remote_array.blocks
(2, 2, 2)
>>> remote_array.dtype
dtype('float32')

Utility Methods¶

__getitem__(slice_: int | slice | Sequence[slice]) → np.ndarray[source]¶

Get a slice of the array (returning NumPy array).

Parameters:: slice_¶ (int, slice, tuple of ints and slices, or None) – The slice to fetch.
Returns:: out – A numpy.ndarray containing the data slice.
Return type:: numpy.ndarray

Examples

>>> import blosc2
>>> urlbase = "https://cat2.cloud/demo"
>>> path = "@public/examples/dir1/ds-2d.b2nd"
>>> remote_array = blosc2.C2Array(path, urlbase=urlbase)
>>> data_slice = remote_array[3:5, 1:4]
>>> data_slice.shape
(2, 3)
>>> data_slice[:]
array([[61, 62, 63],
       [81, 82, 83]], dtype=uint16)

get_chunk(nchunk: int) → bytes[source]¶

Get the compressed unidimensional chunk of a C2Array.

Parameters:: nchunk¶ (int) – The index of the unidimensional chunk to retrieve.
Returns:: out – The requested compressed chunk.
Return type:: bytes

Examples

>>> import numpy as np
>>> import blosc2
>>> urlbase = "https://cat2.cloud/demo"
>>> path = "@public/examples/dir1/ds-3d.b2nd"
>>> a = blosc2.C2Array(path, urlbase)
>>>  # Get the compressed chunk from array 'a' for index 0
>>> compressed_chunk = a.get_chunk(0)
>>> f"Size of chunk {0} from a: {len(compressed_chunk)} bytes"
Size of chunk 0 from a: 160 bytes
>>> # Decompress the chunk and convert it to a NumPy array
>>> decompressed_chunk = blosc2.decompress(compressed_chunk)
>>> np.frombuffer(decompressed_chunk, dtype=a.dtype)
array([ 0.,  1.,  5.,  6., 20., 21., 25., 26.,  2.,  3.,  7.,  8., 22.,
       23., 27., 28., 10., 11.,  0.,  0., 30., 31.,  0.,  0., 12., 13.,
        0.,  0., 32., 33.,  0.,  0.], dtype=float32)

slice(slice_: int | slice | Sequence[slice]) → blosc2.NDArray[source]¶

Get a slice of the array (returning blosc2 NDArray array).

Parameters:: slice_¶ (int, slice, tuple of ints and slices, or None) – The slice to fetch.
Returns:: out – A blosc2.NDArray containing the data slice.
Return type:: blosc2.NDArray

Examples

>>> import blosc2
>>> urlbase = "https://cat2.cloud/demo"
>>> path = "@public/examples/dir1/ds-2d.b2nd"
>>> remote_array = blosc2.C2Array(path, urlbase=urlbase)
>>> data_slice = remote_array.slice((slice(3,5), slice(1,4)))
>>> data_slice.shape
(2, 3)
>>> type(data_slice)
blosc2.ndarray.NDArray

property blocks: tuple[int]¶: The blocks of the remote array

property blocksize: int¶: The block size (in bytes) for the remote container.

property cbytes: int¶: The number of compressed bytes of the remote array

property chunks: tuple[int]¶: The chunks of the remote array

property cparams: CParams¶: The compression parameters of the remote array

property cratio: float¶: The compression ratio of the remote array

property dtype: dtype¶: The dtype of the remote array

property info: InfoReporter¶: Print information about this remote array.

property info_items: list¶: A list of tuples with the information about the remote array. Each tuple contains the name of the attribute and its value.

property nbytes: int¶: The number of bytes of the remote array

property shape: tuple[int]¶: The shape of the remote array

property vlmeta: dict¶: The variable-length metadata f the remote array

URLPath class¶

class blosc2.URLPath(path: str, /, urlbase: str | None = None, auth_token: str | None = None)[source]¶

__init__(path, /[, urlbase, auth_token])

Create an instance of a remote data file (aka C2Array) urlpath.

__init__(path: str, /, urlbase: str | None = None, auth_token: str | None = None)[source]¶

Create an instance of a remote data file (aka C2Array) urlpath. This is meant to be used in the blosc2.open() function.

The parameters are the same as for the C2Array.__init__().

Context managers¶

blosc2.c2context(*, urlbase: str | None = None, username: str | None = None, password: str | None = None, auth_token: str | None = None) → None[source]¶

Context manager that sets parameters in Caterva2 subscriber requests.

A parameter not specified or set to None will inherit the value from the previous context manager, defaulting to an environment variable (see below) if supported by that parameter. Parameters set to an empty string will not be used in requests (without a default either).

If the subscriber requires authorization for requests, you can either provide an auth_token (which you should have obtained previously from the subscriber), or both username and password to obtain the token by logging in to the subscriber. The token will be reused until it is explicitly reset or requested again in a later context manager invocation.

Please note that this manager is reentrant but not safe for concurrent use.

Parameters:

urlbase¶ (str | None) – The base URL to be used when a C2Array instance does not have a subscriber URL base set. If not specified, it defaults to the value of the BLOSC_C2URLBASE environment variable.
username¶ (str | None) – The username for logging in to the subscriber to obtain an authorization token. If not specified, it defaults to the value of the BLOSC_C2USERNAME environment variable.
password¶ (str | None) – The password for logging in to the subscriber to obtain an authorization token. If not specified, it defaults to the value of the BLOSC_C2PASSWORD environment variable.
auth_token¶ (str | None) – The authorization token to be used when a C2Array instance does not have an authorization token set.

Yields:

out (None)