Solving the World’s Toughest Problems with the Growing Open Source Ecosystem...
We started Databricks in 2013 in a tiny little office in Berkeley with the belief that data has the potential to solve the world’s toughest problems. We entered 2020 as a global organization with over...
View ArticleFine-Grained Time Series Forecasting At Scale With Facebook Prophet And...
Try this time series forecasting notebook in Databricks Advances in time series forecasting are enabling retailers to generate more reliable demand forecasts. The challenge now is to produce these...
View ArticleOn-Demand Webinar: Geospatial Analytics and AI in the Public Sector
We recently hosted a live webinar — Geospatial Analytics and AI in Public Sector — during which we covered top geospatial analysis use cases in the Public Sector along with live demos showcasing how to...
View ArticleQuery Delta Lake Tables from Presto and Athena, Improved Operations...
We are excited to announce the release of Delta Lake 0.5.0, which introduces Presto/Athena support and improved concurrency. The key features in this release are: Support for other processing engines...
View ArticleWhat Is a Data Lakehouse?
Over the past few years at Databricks, we’ve seen a new data management paradigm that emerged independently across many customers and use cases: the data lakehouse. In this post we describe this new...
View ArticleAutomating Digital Pathology Image Analysis with Machine Learning on Databricks
With technological advancements in imaging and the availability of new efficient computational tools, digital pathology has taken center stage in both research and diagnostic settings. Whole Slide...
View ArticleBuilding Reliable Data Pipelines for Machine Learning Webinar Recap
This is a guest blog from Ryan Fox Squire | Product & Data Science at SafeGraph At SafeGraph we are big fans of Databricks. We use Databricks every day for ad hoc analysis, prototyping, and many of...
View ArticleDatabricks Named A Leader in Gartner Magic Quadrant for Data Science and...
Gartner has released its 2020 Data Science and Machine Learning Platforms Magic Quadrant, and we are excited to announce that Databricks has been recognized as a Leader. Gartner evaluated 17 vendors...
View ArticleHow to Display Model Metrics in Dashboards using the MLflow Search API
Machine learning engineers and data scientists frequently train models to optimize a loss function. With optimization methods like gradient descent, we iteratively improve upon our loss, eventually...
View ArticleActionable Insight for Engineers and Scientists at Big Data Scale with...
Today, Databricks announced that it is launching a new partnership with MathWorks, the leading developer of mathematical computing software, MATLAB, and Simulink products that are used by engineers and...
View ArticleCelebrating Black History Month | Black @ Databricks
Celebrating the launch of our two newest ERG Groups:Black at Databricks and Latinx at our 2020 Company Retreat With the start of Black History Month, Databricks launched our newest Employee Resource...
View ArticleOn-Demand Webinar: Granular Demand Forecasting At Scale
We recently hosted a live webinar — How Starbucks Forecasts Demand at Scale with Facebook Prophet and Databricks — During this webinar we learnt why Demand Forecasting is critical to Retail/ CPG firms...
View ArticleNew Data Ingestion Network for Databricks: The Partner Ecosystem for...
Organizations have a wealth of information siloed in various sources, and pulling this data together for BI, reporting and machine learning applications is one of the biggest obstacles to realizing...
View ArticleIntroducing Databricks Ingest: Easy and Efficient Data Ingestion from...
We are excited to introduce a new feature – Auto Loader – and a set of partner integrations, in a public preview, that allows Databricks users to incrementally ingest data into Delta Lake from a...
View ArticleCheck out the killer lineup of keynotes at Spark + AI Summit 2020
The Spark + AI Summit is already the world’s largest data and machine learning conference bringing together engineers, scientists, developers, analysts and leaders from around the world. This year is...
View ArticleSecurely Accessing Azure Data Sources from Azure Databricks
Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLFlow , Koalas and Apache Spark, Azure Databricks is a...
View ArticleI Joined Databricks to Make Data Science a Little Less Scary
Big data and AI has always struck me as useful, but slightly scary. For example, it’s useful when Waze uses big data to help me outsmart a traffic jam. On the other hand, big data’s ad-targeting is so...
View ArticleData Quality Monitoring on Streaming Data Using Spark Streaming and Delta Lake
Try this notebook to reproduce the steps outlined below In the era of accelerating everything, streaming data is no longer an outlier- instead, it is becoming the norm. We often no longer hear...
View ArticleA Look into the Mid Market Sales Team
At Databricks, we are passionate about helping data teams solve the world’s toughest problems. Databricks helps organizations innovate faster and tackle challenges like treating chronic disease through...
View ArticleConnect 90+ Data Sources to Your Data Lake with Azure Databricks and Azure...
Data lakes enable organizations to consistently deliver value and insight through secure and timely access to a wide variety of data sources. The first step on that journey is to orchestrate and...
View Article