Microsoft unveils Fabric analytics program, OneLake data lake to span cloud providers

Responding to the increasing size of data analysis projects, but also to its customers desire to trim their budgets on cloud spend, Microsoft on Tuesday unveiled Fabric, what it bills as an “an end-to-end, unified analytics platform that brings together all the data and analytics tools that organizations need.”

The announcement comes as part of the software giant’s three-day Build conference for developers that kicks off today in Seattle.

Also: All the major AI announcements from Microsoft Build 2023

The Fabric workload essentially combines multiple applications, including the company’s main analytics application, PowerBI, with its tools for large, enterprise-class analytics across multiple data stores, Synapse, its Azure OpenAI artificial intelligence service developed with the startup OpenAI, and wraps it all in a Microsoft calls unified compute and storage. 

“Customers can purchase a single pool of compute that powers all Fabric workloads,” says Microsoft. “The universal compute capacities significantly reduce costs, as any unused compute capacity in one workload can be utilized by any of the workloads.”

Also: Microsoft announces new Azure tools to help developers

A new data lake is offered, called OneLake, which is a parallel to the OneDrive used for file upload across Microsoft applications. 

OneLake, claims Microsoft, allows developers to tap into a pool of storage regardless of whose cloud computing environment they are running workloads. 

“A key capability of OneLake is “Shortcuts.” OneLake allows easy sharing of data between users and applications without having to move and duplicate information unnecessarily. Shortcuts allow OneLake to virtualize data lake storage in ADLS Gen2, AWS S3, and Google Storage (coming soon), enabling developers to compose and analyze data across clouds.”

That, says Microsoft, can eliminate “data silos,” from setting up storage in each cloud environment. 

Also: Bolstered Dev Box leads developer delights at Microsoft Build 2023

That promise of saving on compute and storage is clearly a nod to what Microsoft CEO Satya Nadella has told Wall Street is customers’ attempts to “optimize” their cloud usage, meaning, to lower their monthly bill as they tighten their belts to deal with economic uncertainty. 

The company on April 24th reported its Intelligent Cloud business products saw growth slow, but still come in above Microsoft’s forecast. As Nadella remarked on the call with Wall Street analysts, “We incent our people to help our customers with optimization, because we believe, in the long run, that’s the best way to secure the loyalty and long-term contracts with customers, when they know that they can count on a cloud provider like us to help them continuously optimize their workload.”

Also: Microsoft CEO Nadella: ‘Expect us to incorporate AI in every layer of the stack’

At the same time, Fabric is clearly a response to new-style database and analytics vendors that have given competition to the traditional relational database on top of which Microsoft builds it business tools. Snowflake and Data Bricks have developed multi-billion-dollar businesses based on data lakes and data warehouses that span many cloud computing services. 

That presents a challenge to a vendor such as Microsoft to show it can be larger than its own relational database and tools.

Microsoft has integrated the Copilot program it developed with OpenAI to give natural-language interfaces into PowerBI and the rest.

Also: Microsoft adding more AI smarts to Windows 11 via Copilot and Dev Home

“With Copilot in Microsoft Fabric in every data experience, users can use conversational language to create dataflows and data pipelines, generate code and entire functions, build machine learning models, or visualize results,” said Microsoft. 

“Customers can even create their own conversational language experiences that combine Azure Open AI Service models and their data and publish them as plug-ins.”

Microsoft detailed several components in Fabric, many of which are in a preview form. Here are descriptions — as provided by Microsoft — of the Fabric components:

• Data Factory provides 150+ connectors to cloud and on-prem data sources, drag and drop experiences for data transformation, and the ability to orchestrate data pipelines. Currently in Public Preview. 
• Synapse Data Engineering enables great authoring experiences for Spark, instant start with live-pools, and the ability to collaborate. Currently in Public Preview. 
• Synapse Data Science provides an end-to-end workflow for data scientists to build sophisticated AI models, collaborate easily, and train, deploy, and manage machine learning models. Currently in Public Preview. 
• Synapse Data Warehousing provides a converged lake house and data warehouse experience with industry-leading SQL performance on open data formats. Currently in Public Preview.
• Synapse Real-Time Analytics enables developers to work with data streaming in from IoT devices, telemetry, logs, etc. and analyze massive volumes of semi-structured data with high performance and low latency. Currently in Public Preview. 
• Power BI in Microsoft Fabric provides industry-leading visualization and AI driven analytics that enable business analysts and business users to gain insights from data. The Power BI experience is also deeply integrated into Microsoft 365, providing relevant insights where business users already work. 
• Data Activator provides real-time detection and monitoring of data and can trigger notifications and actions when it finds specified patterns in data — all in a no-code experience. Currently in Private Preview.