Dataflows Gen2 in Microsoft Fabric
Next-generation dataflows with improved performance and OneLake integration.
Dataflows Gen2 represents the evolution of Power BI dataflows, offering significant performance improvements and native Fabric integration.
What Are Dataflows Gen2?
Dataflows Gen2 use the same Power Query interface as Gen1 but run on the Fabric compute engine with output directly to OneLake storage.
Key Improvements Over Gen1
Performance - Faster execution with optimized engine - Better handling of large datasets - Improved refresh reliability
OneLake Integration - Output directly to Lakehouse tables - Delta format for versioning - Query with SQL or Spark
Fabric Native - Unified billing with capacity - Consistent governance - Works with all Fabric workloads
Creating Dataflows Gen2
Step 1: Choose Destination In a Fabric workspace, create new Dataflow Gen2 and specify your Lakehouse destination.
Step 2: Get Data Connect to sources using Power Query: - Databases - Files - APIs - Other Fabric items
Step 3: Transform Apply transformations: - Filter and sort - Merge and append - Pivot and unpivot - Custom columns
Step 4: Load Configure destination settings: - Table name - Load mode (replace, append) - Partitioning
Migration from Gen1
Existing Gen1 dataflows can be migrated to Gen2. Key considerations: - Review destination configuration - Update dependent items - Test refresh performance - Update scheduling
Best Practices
- Use for reusable data preparation
- Keep transformations modular
- Leverage query folding where possible
- Monitor refresh performance
Frequently Asked Questions
Should I migrate from Dataflows Gen1 to Gen2?
Yes, Gen2 offers significant performance improvements, OneLake integration, and better Fabric compatibility. Plan your migration to take advantage of these benefits.
Can Dataflows Gen2 replace traditional ETL tools?
For many scenarios, yes. Gen2 dataflows handle common ETL patterns well. For complex enterprise requirements, you might combine with Data Factory pipelines for orchestration.