Optimizing GPU Programs from Java Using Babylon and Hat

Summary

As we have mentioned, HAT is a programming toolkit that enables Java developers to target heterogeneous computing systems from Java. To do so, HAT offers a set of programming abstractions on top of the lower-level foreign programming models for GPUs: ND-Range API that defines GPU’s global thread configuration and block thread configurations. This is a similar concept to OpenCL, CUDA and SYCL in which programmers can define how thread blocks are structured. This makes HAT programs scalable and flexible when running on multiple hardware accelerators from different architectures and vendors. A Kernel-Context Layer: this abstraction represents the code to be offloaded to the GPUs. These are Java methods that express the work to be done per thread (as in a GPU thread), similar to CUDA and OpenCL. To do so, HAT also exposes a KernelContext object, which gives access to Java developers to GPU’s global and local thread-ids, block partitions, and barriers. Compute-Context Layer: this is a higher-level programming abstraction on top of the kernel context API to allow developers to compose a graph of compute kernels and their data dependencies. For example, if we want to launch multiple programs on a GPU, we can group all invocation under the same compute-context layer. Interface Mapper for global and device memory: good memory handling is as important as good compute handling when it comes to achieving performance on GPUs from managed runtime systems. HAT exposes an API on top of the Panama FFM Memory Segments and an API for programming different device types from Java. With these APIs, developers can express their own data types and map them efficiently to different memory regions of the GPUs. Let’s take a simple example to see all these concepts in practice. We are going to express vector multiplication for float arrays. Let’s start with the Java code: public void vectorMul(float[] a, float[] b, float[] c) { for (int i = 0; i < a.length; i++) { c[i] = a[i] * b[i]; } } This ...