Data Factory Pipelines in Fabric

Data Factory in Fabric provides robust data orchestration capabilities for building enterprise data workflows.

What is Data Factory in Fabric?

Visual pipeline builder for: - Data movement - Transformation orchestration - Workflow automation - Error handling

Key Components

Pipelines Container for activities: - Sequences of operations - Conditional logic - Looping and iteration

Activities Individual operations: - Copy data - Run notebooks - Execute dataflows - Call procedures

Triggers Pipeline execution: - Scheduled - Event-based - Manual

Common Activities

Copy Activity Move data between sources: - 100+ connectors - Efficient transfer - Mapping support

Notebook Activity Run Spark notebooks: - Pass parameters - Return values - Error handling

Dataflow Activity Execute dataflows: - Power Query transformations - Staged output

Stored Procedure Call database procedures: - SQL transformations - Existing logic reuse

Pipeline Patterns

Sequential Activities run in order: - Load staging → Transform → Load facts

Parallel Independent activities simultaneously: - Load multiple sources together

Conditional Branch based on outcomes: - If success → continue - If failure → alert

Looping Iterate over collections: - Process each file in folder - Load each table in list

Error Handling

Retry policies
Failure activities
Alert on errors
Logging and monitoring

Frequently Asked Questions

Is Data Factory in Fabric the same as Azure Data Factory?

They share similar concepts and interface but have differences. Fabric Data Factory is integrated with the Fabric platform, uses capacity billing, and has native OneLake integration. ADF remains a separate Azure service.

Can I migrate ADF pipelines to Fabric?

Some migration is possible by exporting and importing pipeline definitions. However, connections and Fabric-specific features may require reconfiguration. Microsoft provides migration guidance and tools.