Cloudera, the only company bringing AI to data anywhere, today announced native GPU acceleration for Apache Spark 4.1 in Cloudera Data Engineering, enabled by the NVIDIA CUDA-X library, cuDF. The NVIDIA cuDF plug-in for Apache Spark will support the just-announced Cloudera Anywhere Cloud, which is designed to enable organizations to accelerate Spark workloads without rewriting PySpark or SQL code.
This helps data teams prepare AI-ready data faster while reducing cloud infrastructure costs across hybrid environments.
As organizations expand AI initiatives, the speed of data preparation has become a critical challenge. Large-scale Spark workloads often take hours to complete, delaying analytics and AI applications while driving up cloud compute costs. By embedding GPU acceleration directly into Cloudera Data Engineering, organizations can dramatically reduce processing times using existing Spark applications, without changing code or operational workflows.
Apache Spark powers many of today's enterprise data pipelines. With native GPU acceleration built into Cloudera Data Engineering, organizations can improve performance while maintaining security and governance required for production workloads. Leveraging NVIDIA cuDF for Spark workloads, Cloudera will provide up to 4x workload acceleration on NVIDIA GPUs compared to traditional CPU infrastructure.
Together, Cloudera Data Engineering accelerated by NVIDIA CUDA-X libraries will deliver zero-code GPU acceleration for Apache Spark 4.1 workloads, faster ETL and data preparation, lower cloud infrastructure costs through shorter runtimes, built-in deployment with no manual driver configuration, enterprise security and governance through Cloudera Unified Data Fabric, and consistent performance across environments.
"For many organizations, AI isn't limited by models. It's limited by how quickly they can turn raw data into trusted, usable insights," said Leo Brunnick, Chief Product Officer at Cloudera. "Accelerating Spark inside Cloudera Data Engineering helps remove that bottleneck, allowing customers to move from data preparation to analytics and AI faster while keeping governance, security, and operational consistency at the center."
"The fastest path to accelerating AI deployments is the one that aligns with how enterprises already operate today," said Pat Lee, vice president, Strategic Enterprise Partnerships at NVIDIA. "With NVIDIA AI infrastructure and CUDA-X libraries now native to Cloudera Data Engineering, enterprises can lower costs and dramatically speed up Apache Spark pipelines without changing a single line of PySpark or SQL code."
About Cloudera
Cloudera is the only hybrid data and AI platform company that large organizations trust to bring AI to their data anywhere it lives. Unlike other providers, Cloudera delivers a consistent cloud experience that converges public clouds, on-prem data centers, sovereign clouds, and the edge, leveraging a proven open-source foundation.
TechIntelPro empowers B2B technology decision-makers with the latest industry trends, deep insights, leadership perspectives and actionable intelligence to drive both organizational and professional milestones, and thrive on the tech pulse.
