Products / Data accelerators

Spark RAPIDS GPU acceleration

Keep the whole Spark query on the GPU, UDFs included.

Talk to us about Spark RAPIDS spark-jitudf
At a glance● same answers
In placekernels run over the cuDF buffers RAPIDS already holds
0UDF rewrites: plain Java and Scala functions
Fallbackto the CPU path when a UDF cannot run exactly on the GPU
The problem

Spark jobs that fall off the GPU

Spark RAPIDS moves SQL to NVIDIA GPUs, but when its UDF compiler declines a Java or Scala UDF, that stage drops back to the CPU. Data is copied off the GPU and back again, and much of the speed-up is lost.

What Beestream does

A drop-in GPU path.

Beestream compiles the UDFs that RAPIDS declines into GPU kernels through TornadoVM. The kernels work on the cuDF buffers in place, so the whole query stays on the GPU.

What you get
  • Works with your existing Spark RAPIDS deployment
  • Reports why RAPIDS declined a UDF, then compiles it to a GPU kernel
  • No copies back to the CPU between stages
  • Same results as the CPU path, or it does not run on the GPU
Scalaunchanged
val score = udf((price: Double, qty: Int) =>
  math.log1p(price * qty) / 7.0)

df.withColumn("score", score($"price", $"qty"))
// declined by RAPIDS → compiled by Beestream
The Beestream platform

Start with one product. Add the rest on the same GPUs.

Spark RAPIDS is one part of the Beestream platform. Once it is in, the other products run on the same GPUs, under the same support, with no new stack to adopt.

7.3×Apache Iceberg GPU accelerationfaster Iceberg compaction 19×JVector GPU accelerationfaster vector index builds 1.4×jitLLMllama.cpp prompt speed 2.4×jitNNfaster than PyTorch, ResNet-50 13×jitCastfaster forecasts
Contact

Tell us what is costing you CPU time. We'll show you the GPU path.

LinkedIn/company/beestream-ai X@beestream_ai Bluesky@beestream.ai GitHubgithub.com/beestream-ai