This article sets out to give a high-level overview of Microsoft Fabric by discussing two main topics:

1. What is Microsoft Fabric, who is it for, and what services does it provide?

2. A high-level comparison of the seven different services Microsoft Fabric offers to their standalone Azure counterparts.

Overview of Microsoft Fabric

Microsoft Fabric is a complete analytics platform enabling data professionals and businesses to collaborate on projects in a single, integrated environment. It provides integrated services, allowing users to ingest, store, process, and analyse data in a single environment.

In addition to a simple, shared user experience, Microsoft Fabric is a unified software-as-a-service (SaaS) offering, with all your data stored in OneLake. The OneLake architecture facilitates collaboration between data team members and saves time by eliminating the need to move and copy data between different systems and teams. Like how Office applications are pre-wired to use your organisational OneDrive, all the compute workloads in Microsoft Fabric are preconfigured to work with OneLake and the Delta Lake open table format. Microsoft Fabric's data warehousing, data engineering (Lakehouses and Notebooks), data integration (pipelines and dataflows), real-time analytics, and Power BI all use OneLake as their native store without extra configuration.

OneLake is built on top of Azure Data Lake Storage (ADLS), and data can be stored in any format, including Delta, Parquet, CSV, JSON, and more. This means that all the compute engines in Microsoft Fabric automatically store their data in OneLake. Data stored in OneLake is then directly accessible by all the compute engines without needing to be moved or copied. For tabular data, the analytical engines in Microsoft Fabric will write the data to Delta tables, and all the engines interact with the them seamlessly.

Who is Microsoft Fabric for?

Traditionally, the data engineer and data analyst role separation meant that an extra conversation was needed to ensure that the engineer curated a perfect data model to help the analyst display data effectively and insightfully for the business.

With Microsoft Fabric, data professionals work together in the same SaaS product to better understand and identify the needs of each other and the business. Further, data analysts now have greater context and the ability to transform data further upstream with data factory. Whether you're a data engineer looking to simplify your data model curation or expanding your knowledge with data science techniques, Microsoft Fabric provides a complete experience to serve your organisation.

For data analysts, who may have had to perform extensive downstream data transformations before creating Power BI reports, you can now see the lineage and connect with data more directly with Direct Lake mode. Data scientists now have an easier way to integrate native data science techniques and then use Power BI's interactive reporting to provide data-informed insights in a new way. Because Microsoft Fabric is a SaaS platform, it allows you to provision and run any workload or job without needing pre-approval or planning quickly and easily. This means you can scale resources up or down as needed and be more agile and responsive to changing business needs.

Lastly, Microsoft Fabric is bringing the low-to-no-code concept, functionality, and approach that has successfully empowered many users on the Power Platform to its own unified data and analytics SaaS offering. While it maintains scale and integrity for data science, data warehousing, data ingestion and preparation, and analytics, it also offers many ways to visually represent code that previously blocked many from going further.

Services Offered by Microsoft Fabric

Microsoft Fabric offers analytics experiences designed to accomplish specific tasks and work together seamlessly.

Synapse Data Engineering: Microsoft Fabric implements its version of Apache Spark, which encourages large-scale data processing and analytics. Just as data engineers can set up clusters in Azure Databricks, an administrator in Microsoft Microsoft Fabric can manage settings for the Spark Cluster in the workspace's data engineering/science section. Furthermore, Microsoft Fabric includes many of the most used Python libraries typically available via PySpark. Still, if users need to install other libraries, they can do so on the Library management page in the workspace settings. Spark in Microsoft Fabric also enables data analysts to use SQL expressions to query and manipulate data via a Spark library named Spark SQL. The Spark runtime can use the catalog to seamlessly integrate code written in any Spark-supported language with SQL expressions that may be more natural to some data analysts or developers.

As with Azure Databricks, Notebooks are also available in Microsoft Fabric, and this is how Microsoft Fabric users can use Spark to explore and analyse data interactively. Microsoft Fabric users can also set up spark jobs to run a script on demand or based on a schedule, as in Azure Databricks Notebooks. Synapse Data Warehouse: Microsoft Fabric's data warehouse is a modern version of the traditional data warehouse. It centralises and organises data from different departments, systems, and databases into a unified view for analysis and reporting purposes. Microsoft Fabric's data warehouse provides full SQL semantics, including inserting, updating, and deleting data in the tables. Microsoft Fabric's data warehouse is unique because it's built on the Lakehouse, which is stored in Delta format and can be queried using SQL. It's designed for the whole data team, not just data engineers.

A few ways to ingest data into a Microsoft Fabric Data Warehouse include Pipelines, Dataflows (Gne2), cross-database querying, and the COPY INTO command. After ingestion, the data becomes available for analysis by multiple business groups, who can use features such as cross-database querying and sharing to access it. Once the data is ingested into the Microsoft Fabric data warehouse, there are two ways to query the data. The Visual query editor provides a no-code, drag-and-drop experience to create your queries. If you're comfortable with T-SQL, you may prefer to use the SQL query editor to write your queries. In both cases, you can create tables, views, and stored procedures to query data in the data warehouse and Lakehouse. Microsoft Fabric data warehouses also allow users to create data models, build relationships, create measures, and visualise data, all of which are key features of Azure Synapse Analytics.

Synapse Data Science: When you perform data science in Microsoft Fabric, there are three key features to help you manage your work: notebooks, experiments, and models. You use Notebooks to write and run code. Within a notebook, you can run one or more Experiments. An experiment allows you to track your workloads, like training a machine learning model. Finally, you can save a machine learning model as a Model in Microsoft Fabric. Readers interested in learning more about Synapse Data Science in Microsoft Fabric can find more information here.

Synapse Real-Time Analytics: Data analysts, engineers or scientists who are familiar with the use of Kusto Query Language (KQL) for querying IoT data and log analytics data in real-time will be pleased to know that Microsoft Fabric also provides an end-to-end streaming solution for high-speed data analysis via its Synapse Real-Time Analytics service. Some benefits for users using KQL in Microsoft Fabric include the following:

• Enables data exploration and analysis efficiency by allowing users to work with multiple data sources and visualise the results in various ways.

• Supports reproducible analyses by allowing users to create notebooks with Kusto kernel that can capture code, results, and context on the analysis.

• Enriches DevOps flow by allowing users to add KQL files and KQL notebook files to their Git repositories and CI/CD pipelines.

• Provides guidance and helps users build search queries from scratch using the KQL editor, which quickly identifies potential errors and displays hints about resolving issues.

• Allows users to filter, present, and aggregate their data using various operators and functions that are easy to read and author.

Data Factory: Microsoft Fabric includes Data Factory capabilities, including the ability to create pipelines that orchestrate data ingestion and transformation tasks. If you're already familiar with Azure Data Factory (ADF), then data pipelines in Microsoft Fabric will be immediately familiar. They use the same architecture of connected activities to define a process that includes multiple data processing tasks and control flow logic. You can run pipelines interactively in the Microsoft Fabric user interface or schedule them to run automatically.

Core pipeline concepts in ADF, such as data transformation, control flow activities, parameters, and the ability to run and monitor these pipelines, also exist in Microsoft Fabric pipelines. Furthermore, users of Microsoft Fabric also have most of the data source and destination connectors available in ADF. However, there are a few differences that ADF users might notice when using Data Factory in Microsoft Fabric. These differences are summarised in the the Microsoft Fabric documentation. You can see a snapshot of the table below:

Source: Getting from Azure Data Factory to Data Factory in Microsoft Fabric (as of 15th September 2023)

Data Activator: Data is only valuable when users can act on it. This means you need to generate insights from your data and then transform those insights into jobs to be done. Here is where Data Activator becomes useful. Data Activator is a no-code Microsoft Fabric experience that empowers the business analyst to drive actions automatically from your data. Data Activator drives actions through a 3-step process:

• Connect to your data: Data Activator can connect to a wide range of data sources in Microsoft Fabric, from Power BI datasets, Event streams, and more. Once Data Activator is connected to your data, it continually monitors it for actionable patterns.

• Detect actionable conditions: Data Activator gives you a single place to define actionable patterns in your data. These can range from simple thresholds (such as a value being exceeded) to more complex patterns over time (such as a value trending down).

• Trigger actions: When Data Activator detects an actionable pattern, it triggers an action. That action can be an email or a Teams alert to the relevant person in your organisation. It can also trigger an automatic process via a Power Automate flow or an action in one of your organisation's line-of-business apps.

Power BI: This is the same market-leading business intelligence service that has received updates on a monthly cadence since it launched in July 2015. Within Microsoft Fabric, it's been extended with some new capabilities, the most notable one being Direct Lake. Direct Lake allows Power BI models to directly query Delta Lake tables in OneLake by loading them into memory on-the-fly. It's expected that by the time Microsoft Fabric reaches its general availability milestone, this feature will have equivalent or better performance than import mode (i.e. loading the data into memory upfront).

Conclusion

Microsoft Fabric provides a comprehensive data analytics solution by unifying all these experiences on a single platform. By adopting it, your organisation might be able to reduce the costs of integration overhead and complexity, thereby allowing data and analytics users to concentrate more on solving business problems and addressing business opportunities.

Furthermore, there have been announcements recently that Microsoft will add its Azure OpenAI Service to Microsoft Fabric and will soon integrate GPT-powered Copilot into Microsoft Fabric. This will be a massive benefit to users as with Copilot in Microsoft Fabric; users will be able to use conversational language to create dataflows and data pipelines, generate code and entire functions, build machine learning models, or visualise results, which in turn will help to speed up the work done by data professionals significantly. However, it is essential to bear in mind that many of the features of Microsoft Fabric are still in preview, so some of these features might change or never make it to general availability, depending on feedback received by Microsoft.

Follow Us and Stay Up to Date

Get notified when we post something new by following us on X and LinkedIn.