Databricks today announced a partnership with Google Cloud to offer its “data lakehouse” on the tech giant’s public cloud for the first time, meaning Databricks’ data platform is now available across each of the three markets leading clouds.
The integration gives Databricks customers more choice on where to run the popular analytics platform and, for those already invested in the Google Cloud ecosystem, access to Databricks “lakehouse” model for the first time.
From today customers will be able to deploy Databricks in a fully containerised Google Cloud environment, integrating with Google’s data warehouse and its managed Kubernetes service.
Databricks ANZ country manager Bede Hackney says customers have requested Google Cloud access for a long time but it was important not to simply port over a version from another cloud.
“This was jointly developed and optimised to be delivered in the Google environment leveraging for the first time ever containers in the Google Kubernetes Engine,” Hackney tells Which-50.
“It was super important that it wasn’t just a generic offering that we ported from one cloud to another. It was really important that we went back and took the time to genuinely optimise for the Google environment.
“And that’s why you see this is the first time that we’re offering a Databricks rich platform that’s based on containerisation.”
Founded in 2013 by the original creators of Apache Spark, a popular analytics engine for large-scale data processing, Databricks has soared to a $US28 billion valuation, attracting funding from tech giants and financial service firms in the latest round.
The company helps organisations find the “gold” in massive amounts of data from various sources, through its cloud platform built around Spark. It does so with a “lakehouse” model, aiming to combine the low cost of data lakes with the management tools of a data warehouse.
In Australia Databricks customers include Coles, AGL, Atlassian, oOH!Media, JB Hi-Fi, UNSW, the Australian Securities and Investments Commission, the Department of Education Skills and Employment, Transport for New South Wales, Healthdirect, Sportsbet and ZipCo.
Hackney says the Databricks platform essentially breaks down data silos and creates a single source of truth for data scientists, data engineers and business analysts, ultimately lowering the time to value of data driven projects.
While businesses are already seeing value from data projects, most are still at the “start line” in terms of operationalising them, Hackney says.
“I think data teams in the past have been a little bit guilty of offering great promise, but potentially taking too long to deliver on that promises,” he tells Which-50.
“Many a data team has been formed and it’s taken them a year or two to output the first business value based on an AI.”
Google Cloud deal
Under the partnership announced today, organisations can use Databricks to create a lakehouse capable of running on the Google Cloud network.
Databricks is already available on market leader AWS and its closest rival Microsoft Azure.
Databricks says the Goolge partnership enables customers to deploy Databricks in a fully containerised Google Cloud environment, integrating with Google BigQuery and the Google Kubernetes Engine.
GoogleCloud offers its own data lake and warehouse services but – like its new partner – making Dataabricks available on Google Cloud is about giving customers choice, according to Rhody Burton, the tech giant’s head of cloud partnerships.
“We really want to meet the customer where they are and give them the choice of what tools they want to use.”
Burton says Google also leapt at the opportunity to offer Databricks in a new, containerised way.
“It helps Google Cloud differentiate, and offers significant value to customers as increasingly containers and Kubernetes becomes the de facto standard for enterprise workloads.
“So we see that as a really incredible opportunity for us in the market.”