DevitoComp provider interface

DevitoComp is the provider-neutral compression ABI used by DevitoPRO. The public contract is include/devitocomp.h; a provider supplies a shared library named libdevitocomp.so that exports the functions declared there.

The bundled dummy provider supports both host and CUDA execution. It performs a synchronous host copy or an asynchronous CUDA device-to-device copy. It deliberately uses an eight-byte host sidecar record so that DevitoPRO exercises providers whose decompressor requires per-snapshot state in addition to the compressed buffer.

Configuring the error tolerance

Pass a DevitoComp configuration as the compression argument of a TimeFunction:

from devitopro import DevitoComp, TimeFunction

usave = TimeFunction(..., compression=DevitoComp(error=0.01))

error is a non-negative relative error tolerance. It is fixed when the compression plan is created and cannot be changed through Operator.apply. Use compression='devitocomp' or DevitoComp() to let the provider select its default tolerance.

Provider layout

By default DevitoPRO uses the bundled header and dummy library. To select a private provider, set DEVITOCOMP_ROOT to a directory containing:

DEVITOCOMP_ROOT/
├── include/devitocomp.h
└── lib/libdevitocomp.so

The private header must implement the same ABI version as the public header. devitocomp_plan_create receives an optional provider-defined options string; DevitoPRO passes the relative error in DevitoComp(error=...) as a decimal string, or NULL when the error is omitted.

Tensor dimensions and strides use slowest-to-fastest order, while strides are measured in elements. The devitocomp_execution_t descriptor determines how the provider may access compression and decompression pointers:

  • DEVITOCOMP_EXECUTION_HOST uses host pointers, an optional thread count, and no stream.
  • DEVITOCOMP_EXECUTION_CUDA and DEVITOCOMP_EXECUTION_HIP use pointers accessible from the active device and an optional native stream. The thread count is ignored.

Per-snapshot sidecars always refer to aligned host storage and must be ready when the provider call returns. Device work may remain queued on the supplied stream, while host work must be complete when the call returns. Providers may support any subset of the execution kinds and return DEVITOCOMP_UNSUPPORTED for the others.

Providers report the required compressed-payload alignment through devitocomp_limits_t.payload_alignment. DevitoPRO aligns the start of every stored payload while preserving the compressed_bytes returned by the provider. The alignment must be a nonzero power of two supported by DevitoPRO’s active allocator.

Provider-specific types, configuration, and library calls belong entirely behind this interface. The shared header and DevitoPRO integration must remain independent of any provider implementation.

Dummy provider

From devitopro_ext/devitocomp/dummy:

make host       # build the host provider and test
make cuda       # build the CUDA provider and test
make test-host  # run the host round-trip test
make test-cuda  # run the CUDA round-trip test

The dummy accepts contiguous float32 rank-2 and rank-3 tensors. It performs no quantization and reports a maximum compressed size equal to the raw tensor size. It requests eight-byte payload alignment so the integration tests cover aligned packing even though its copy operation does not require it. The host build supports host execution; the CUDA build supports both host and CUDA execution.

Validating a private provider

The DevitoPRO compressed-function and composite-function tests are the primary provider acceptance suite. They exercise the ABI through real lowering, layered host/device storage, disk serialization, pickling, and MPI rather than through an isolated provider-only harness.

After installing a private header and library under DEVITOCOMP_ROOT, run:

pytest tests/test_compressed_funcs.py -k devitocomp
pytest tests/test_composite_funcs.py -k devitocomp

Run both commands in every execution environment supported by the provider. In particular, test a CUDA provider with the CUDA backend and a host provider with the CPU backend. The repository’s standard Docker test wrapper may be used when DEVITOCOMP_ROOT is visible at the same path inside its container.

The bundled dummy is lossless, so the tests use exact numerical tolerances by default. For a lossy private provider, set DEVITOCOMP_RTOL to the relative tolerance promised by its configured error mode before starting pytest. A provider with a lossless or test mode should be validated in that mode first; then repeat the suite using the intended production settings and tolerance.

Back to top