from devitopro import Float16, Float8Data quantization in DevitoPRO
While single precision is the general consensus for numerical computations in seismic modeling and inversion to obtain accurate results, many applications requires a large number of model parameters that don’t necessarily require as much precision due to their smoothness and small value range. In these cases, quantization can be used to reduce the memory footprint and computational cost of the model parameters.
We provide in DevitoPRO a simple quantization type FloatX that allows you to specify the number of bits to use for a given Function. We also provide two standard Float16 and Float8 for 16-bit and 8-bit quantization, respectively. These types quantize a fixed value interval with an offset and scale factor with the method known as compression by scaling and offset (CSO).
We are considering a simple case of a Function whose values are in the range of [1.5, 4.5]. The quantization will map these values to a smaller range of integers, which can be stored in less bits.
# Float16 type
d16 = Float16(1.5, 4.5)
d8 = Float8(1.5, 4.5)# The datatype of the "uncompressed" value
d8.dcmptype, d16.dcmptype(numpy.float32, numpy.float32)# The datatype of the quantized value
d8.nptype, d16.nptype(numpy.uint8, numpy.uint16)We can now create Function with these quantization types
from devito import Grid
from devitopro import Function
grid=Grid((10, 10))
f8 = Function(name='f8', dtype=d8, grid=grid)
f16 = Function(name='f16', dtype=d16, grid=grid)# Check the data types of the created Functions. This will show the quantized types.
f8.dtype, f16.dtype(numpy.uint8, numpy.uint16)# Assign data
f8.data[:] = 2.0
f16.data[:] = 3.0# The data will be the quantized values
f8.data[0], f16.data[0](Data([42, 42, 42, 42, 42, 42, 42, 42, 42, 42], dtype=uint8),
Data([32767, 32767, 32767, 32767, 32767, 32767, 32767, 32767, 32767,
32767], dtype=uint16))# The actual value is accessible through the `decompress` property
# We can see that since the values dont exactly fit the quantized interval, there is a small error, but we do get the expected values.
f8.data.decompress[0], f16.data.decompress[0](array([1.9941176, 1.9941176, 1.9941176, 1.9941176, 1.9941176, 1.9941176,
1.9941176, 1.9941176, 1.9941176, 1.9941176], dtype=float32),
array([2.999977, 2.999977, 2.999977, 2.999977, 2.999977, 2.999977,
2.999977, 2.999977, 2.999977, 2.999977], dtype=float32))Let’s now make a small operator to show that it gets lowered into valid code
from devito import Eq
u = Function(name='u', grid=grid, space_order=2)
eq = Eq(u, f8 + f16)
print(eq)Eq(u(x, y), f16(x, y) + f8(x, y))#NBVAL_IGNORE_OUTPUT
from IPython.display import Code
from devito import Operator
from devitopro import *
op = Operator(eq)
Code(str(op), language='C')/* NON-COMMERCIAL, NON-EXCLUSIVE, NON-TRANSFERABLE SOFTWARE LICENSE
Copyright (c) 2021-25, Devito Codes Ltd, Riverside House,
1-5 Como Street, Romford, Essex, RM7 7DN, United Kingdom.
Definitions:
* "Devito Codes": Refers to Devito Codes Ltd., a company incorporated and operating under the laws of England and Wales, and encompasses all its divisions, affiliates, and subsidiaries involved in the development and licensing of software products.
* "Software": Includes all software products developed by Devito Codes, such as Devito PRO, and any other software titles offered by Devito Codes, along with their associated materials, documentation, and software generated by their respective compilers or runtime environments.
* "Licensee": Denotes any individual, company, or entity that has legally acquired a copy or license of the Software under the terms specified in an Agreement with Devito Codes. This term also encompasses all employees, contractors, and agents acting within the scope of their engagement by the Licensee.
* "Agreement": Refers to the legally binding license agreement executed between the Licensee and Devito Codes, which outlines the terms and conditions under which the Licensee is authorized to use the Software. This Agreement may be specific to a single software product or may encompass multiple products offered by Devito Codes.
* "Protected Data/Software": Encompasses all data, software, algorithms, source and object code, documentation, proprietary methodologies, and intellectual property owned or developed by Devito Codes. This includes, without limitation, all versions and iterations of Devito PRO software, materials, and any output generated by Devito Codes' compilers or software, as well as any other proprietary software and tools developed by Devito Codes.
* "Unauthorized Use": Constitutes any action involving the Protected Data/Software that is not expressly permitted under the terms of the Agreement. This includes, but is not limited to, copying, modifying, distributing, selling, leasing, reverse engineering, decompiling, or incorporating the Protected Data/Software into any external system or software, including data retrieval systems, databases, generative AI models, machine learning models, Retrieval Augmented Generation (RAG) applications, or any form of technology or AI system, without the explicit written consent of Devito Codes.
* "Non-Commercial Use" refers to activities not intended for or directed towards commercial advantage or monetary compensation. This includes, but is not limited to, academic research, educational purposes, and personal projects.
Devito Codes provides the Software on the following terms:
1. Scope of License: The Licensee is hereby granted a non-commercial, non-exclusive, non-transferable license without the right to sublicense the Protected Data/Software for the purpose of internal evaluation, testing, and development within the Licensee's organization. This license explicitly excludes any commercial applications, product development for sale, or external consulting services.
2. Prohibition on Unauthorized Use: The Licensee shall not use, adapt, modify, or incorporate the Protected Data/Software into any software or hardware that is not explicitly authorized by this Agreement. This includes prohibitions against the use of the Protected Data/Software in the development, training, or operation of generative AI, machine learning models, RAG applications, or any form of AI that utilizes external data sources for augmentation without the prior written consent of Devito Codes.
3. Distribution and External Use: The Licensee agrees not to distribute, share, lease, or otherwise make the Protected Data/Software available to any third party, including through cloud services, Software as a Service (SaaS), or any form of redistribution mechanism. The Licensee also agrees not to use the Protected Data/Software to provide any service to external parties that involves the processing of data or computation using the Protected Data/Software.
4. The Licensee is prohibited from reverse engineering, decompiling, disassembling, or otherwise attempting to discover the source code of the Protected Data/Software. This prohibition extends to any form of reverse engineering or hacking intended to access or create unauthorized derivatives of the Software.
5. The Software is experimental and licensed "as is." The license of the Software does not include technical support.
6. The Licensee agrees that it will not license or sell the Software or any other Software, information, or data that incorporate any part of the Software, including derivative works ("Derivatives"), to any other parties.
7. Devito Codes will consider all requests for a commercial license but shall be under no obligation to grant such a license
8. Licensee agrees that any person within the Licensee utilizing the Software will be advised of, and is subject to, the conditions in the License.
9. THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. Devito Codes Ltd expressly disclaims all warranties, whether express, implied, statutory, or otherwise, with respect to the software provided hereunder, including any warranty of reliability, accuracy, or the absence of errors or defects, and the implied warranty arising from course of performance, course of dealing, or usage of trade. Devito Codes Ltd shall not be liable for any direct, indirect, incidental, special, consequential, or exemplary damages, including but not limited to, damages for loss of profits, goodwill, use, data, or other intangible losses (even if Devito Codes Ltd has been advised of the possibility of such damages), resulting from the use or the inability to use the Software. The Licensee acknowledges and agrees that the entire risk arising out of the use or performance of the Software remains with the Licensee. Notwithstanding any damages that the Licensee might incur for any reason whatsoever (including, without limitation, all damages referenced above and all direct or general damages), the entire liability of Devito Codes Ltd and any of its suppliers under any provision of this License and the Licensee's exclusive remedy for all of the foregoing shall be limited to the amount actually paid by the Licensee for the Software. The foregoing limitations, exclusions, and disclaimers shall apply to the maximum extent permitted by applicable law, even if any remedy fails its essential purpose.
10. All rights, title, and interest in and to all data, information, and inventions that result from the Software by the Licensee shall vest in and belong to the Licensee. Licensee agrees to provide Devito Codes with copies of publications referencing the Software and acknowledge Devito Codes in those publications.
11. The Licensee agrees to notify Devito Codes in writing of any known or suspected distribution or use of the Software not in compliance with the License requirements and to enforce the terms of the License.
12. The Licensee agrees to indemnify and hold harmless Devito Codes against any claims or damages arising from the Licensee's use of the Software, exceeding the scope of this License. This includes any third-party claims related to the Software's use.
13. Intellectual Property Rights: All intellectual property rights in the Protected Data/Software and any derivatives thereof shall remain with Devito Codes. The Licensee acknowledges that no ownership rights are transferred by this Agreement.
14. Compliance with Future Technologies: The Licensee commits to continuously ensuring that the use of the Software, including in the context of emerging technologies and computational techniques, adheres to the terms outlined in this License. Devito Codes will provide guidelines for compliance as new technologies develop, facilitating ongoing lawful use of the Software.
15. Notification of Unauthorized Use: The Licensee shall promptly notify Devito Codes of any unauthorized use, copying, or distribution of the Protected Data/Software, or any breach of this License, of which it becomes aware.
16. Where a clause in this License and the Agreement conflict, the clause in the Agreement will take precedence.
*/
#define _POSIX_C_SOURCE 200809L
#define START(S) struct timeval start_ ## S , end_ ## S ; gettimeofday(&start_ ## S , NULL);
#define STOP(S,T) gettimeofday(&end_ ## S, NULL); T->S += (double)(end_ ## S .tv_sec-start_ ## S.tv_sec)+(double)(end_ ## S .tv_usec-start_ ## S .tv_usec)/1000000;
#define f16L0(x,y) f16[(x)*y_stride0 + (y)]
#define f8L0(x,y) f8[(x)*y_stride0 + (y)]
#define uL0(x,y) u[(x)*y_stride1 + (y)]
#include "stdlib.h"
#include "math.h"
#include "sys/time.h"
#include "omp.h"
struct dataobj
{
void *restrict data;
int * size;
unsigned long nbytes;
unsigned long * npsize;
unsigned long * dsize;
int * hsize;
int * hofs;
int * oofs;
void * dmap;
} ;
struct profiler
{
double section0;
} ;
int Kernel(struct dataobj *restrict f16_vec, const float f160, const float f161, struct dataobj *restrict f8_vec, const float f80, const float f81, struct dataobj *restrict u_vec, const int x_M, const int x_m, const int y_M, const int y_m, const int nthreads, struct profiler * timers)
{
unsigned short *f16 __attribute__ ((aligned (64))) = (unsigned short *) f16_vec->data;
unsigned char *f8 __attribute__ ((aligned (64))) = (unsigned char *) f8_vec->data;
float *u __attribute__ ((aligned (64))) = (float *) u_vec->data;
const int y_fsz0 = f8_vec->size[1];
const int y_fsz1 = u_vec->size[1];
const int y_stride0 = y_fsz0;
const int y_stride1 = y_fsz1;
START(section0)
#pragma omp parallel num_threads(nthreads)
{
#pragma omp for schedule(static,1)
for (int x = x_m; x <= x_M; x += 1)
{
#pragma omp simd aligned(f16,f8,u:16)
for (int y = y_m; y <= y_M; y += 1)
{
uL0(x + 2, y + 2) = f160 + f80 + (1.0F/65535.0F)*(-f160 + f161)*f16L0(x + 1, y + 1) + (1.0F/255.0F)*(-f80 + f81)*f8L0(x + 1, y + 1);
}
}
}
STOP(section0,timers)
return 0;
}
We can see as expected that the quantized functions have been lowered into an offset and scale expression that will use the correct decompress value while minimizing the memory footprint only storing 8/16 bits instead of 32.
Back to top