Serialization

Data streaming ensures that snapshots not fitting in host or device memory are seamlessly stored to disk – this is what we refer to as “serialization”.

While this is all automated, DevitoPRO offers some APIs to control serialization.

Custom I/O directory

Use the serialization kwarg when creating a TimeFunction to store the snapshots in a directory of choice.

#NBVAL_IGNORE_OUTPUT

from devito import Eq, Grid, TimeFunction, Operator
from devito.tools import make_tempdir
from devitopro import *
from devitopro.types.enriched import Disk

nt = 5
grid = Grid(shape=(4, 4))
time = grid.time_dim

dumpdir = make_tempdir('aaa')

# `serialization=dumpdir`
usave = TimeFunction(name='usave', grid=grid, save=nt, layers=Disk, serialization=dumpdir)

# Forward, dumping everything to disk inside `dumpdir`
summary = Operator(Eq(usave, time))(time_M=nt-1)

Operator `Kernel` ran in 0.01 s

Let’s now see how we can fetch the snapshots using a different TimeFunction.

#NBVAL_IGNORE_OUTPUT

v = TimeFunction(name='v', grid=grid)
vsave = TimeFunction(name='vsave', grid=grid, save=nt)

fetchdir = usave._fnbase

# NOTE: currently the `fetchdir` does not coincide with the `dumpdir`
# This might change in the near future, thus enabling a much neater `serialization=dumpdir`
usave1 = TimeFunction(name='usave1', grid=grid, save=nt, layers=Disk, serialization=fetchdir)

# print(fetchdir)

eqns = [Eq(v, v.backward - 1),
        Eq(vsave, usave1)]

summary = Operator(eqns)(time_M=nt-1, time_m=0)

# print(vsave.data)

Operator `Kernel` ran in 0.01 s