Utilizing Low-precision datatypes in PyTorch and Beyond

Опубликовано 10 ноября 2025, 18:00

This talk shows how to use emerging low-precision formats in PyTorch to boost performance on AMD GPUs while preserving accuracy. We introduce scaling that stores high-precision tensors as low-precision values plus scales and covers per-tensor, per-row, and microscaling (MX) tiling with power-of-two E8M0 scales. Driss reviews current PyTorch support for FP8 and FP4 and demonstrate the new scaled_mm API, which captures your quantization recipe and automatically selects optimized GEMM kernels.

Find the resources you need to develop using AMD products: amd.com/en/developer.html

***

© 2025 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, EPYC, ROCm, and AMD Instinct and combinations thereof are trademarks of Advanced Micro Devices, Inc.