Ravindra BagaleCourses & study guides Track your progress

Chapter 41

41.1 Dataflows, Pipelines, Lineage and Usage Metrics

By Ravindra Bagale · Power BI deep series

Chala mitrano — stop copying the same Sales clean-up into five .pbix files. One dataflow, Dev, then Test, then Prod pipelines, lineage, and usage metrics keep MH Ops honest.

Fictional sample data for learning. All company names and brands belong to their owners. We use the names only so students can understand. Numbers on this page are made up.
After this lesson you can
  • Explain why a dataflow beats copy-paste Power Query.
  • Describe Dev, then Test, then Prod deployment pipelines.
  • Use lineage view before a breaking column change.
  • Read usage metrics to find unused reports.
  • Walk a Friday Prod promotion checklist.
  • Give a senior interview answer on governance.

Start with a problem (fictional). Priya Deshmukh (Pune, fictional MH Ops) found five .pbix files each transforming the same dairy Sales CSV. At 07:00 IST five refreshes fought the file share. When she renamed CityKey, three Apps went blank. She needs one Sales_Clean dataflow, a pipeline, lineage checks, and usage metrics before the next Friday cutover.

Sample layout — fictional data: dataflow to model to Dev Test Prod App
Sample layout — fictional data (not a real Service screenshot)

1. Why a dataflow beats five copy-paste queries

Why: Five .pbix files each load the same messy Sales CSV. One rename breaks all five, and five refreshes hit the file at 07:00 IST.

What: A dataflow is a cloud Power Query that stores cleansed entities once. Many semantic models connect to that dataflow as a source.

How — build and reuse one entity:

Vertical steps (one under another)

  1. In the workspace choose New, then Dataflow (or Dataflow Gen2 where Fabric is available).
  2. Build the same Power Query steps you would in Desktop — rename, types, merge CityDim.
  3. Save and refresh the dataflow on a schedule (example: 06:30 IST).
  4. In Desktop: Get data, then Power BI dataflows, then pick the entity, then build the model on top.
  5. Publish models that share the same clean Sales table.

Tiny example: Nashik dairy (fictional): Sales_Clean dataflow feeds Ops App, Finance App, and a small Analyze-in-Excel model — ₹ amounts stay consistent.

Ravindra Bagale's Tip

Friends, one dataflow refresh, then model refreshes that only pull the clean entity — fewer 07:00 fights.

2. Deployment pipelines Dev, then Test, then Prod

Why: Editing in Prod while city managers are in a Monday standup is how you ship a blank card by accident.

What: A deployment pipeline links three workspaces (Dev, Test, Prod) and copies content forward after review.

How — promote safely:

Vertical steps (one under another)

  1. Create (or ask capacity admin for) a pipeline with three stages.
  2. Assign your Dev, Test, and Prod workspaces.
  3. Build in Dev — dataflow, model, report, App draft.
  4. Deploy to Test; run UAT with sample UPN (city manager of Pune).
  5. Deploy to Prod only after Test sign-off; rematch gateways and parameters if asked.

Tiny example: MH Ops pipeline (fictional): Dev breaks safely; Test has yesterday’s sample; Prod App goes live Friday 18:00 IST.

Ravindra Bagale's Tip

Friends, keep Prod parameters (gateway, folder path) different from Dev — pipelines can store stage rules.

3. Lineage view — who breaks if I change this?

Why: Deleting CityKey from a dataflow can blank every city card overnight.

What: Lineage view draws source, then dataflow, then model, then report, then dashboard/App so you see dependents.

How — check before you break:

Vertical steps (one under another)

  1. In the workspace open Lineage view (or item, then View lineage).
  2. Click the dataflow or model you plan to change.
  3. List every report and App downstream.
  4. Message owners before a breaking rename.
  5. After change, refresh and spot-check one visual per dependent report.

Tiny example: Lineage shows Sales_Clean, then three models, then two Apps. Priya postpones the column drop until Friday after notice.

Ravindra Bagale's Tip

Friends, lineage is a conversation tool — send the screenshot to owners before you drop a column.

4. Usage metrics — what people actually open

Why: A pretty report that nobody opens still costs refresh and confusion.

What: Usage metrics show views, viewers, and popular reports for the workspace (capacity/Pro rules apply).

How — read and act:

Vertical steps (one under another)

  1. Workspace, then … View usage metrics report (admin may need to enable).
  2. Sort by views last 30 days.
  3. Flag reports with near-zero views for archive or training.
  4. Check which city managers never opened the App after go-live.
  5. Act: train, simplify, or retire — do not keep zombie reports.

Tiny example: After go-live, Kolhapur manager (fictional) never opened the App — Training books a 20-minute call, not another email blast.

Ravindra Bagale's Tip

Friends, usage is a conversation with managers — not a blame chart.

5. Worked example — MH Ops promotion Friday

Why: You need a calm Friday cutover, not a Monday surprise.

What: Combine dataflow refresh, pipeline deploy, and a short lineage check.

How — run the cutover:

Vertical steps (one under another)

  1. Confirm Sales_Clean dataflow green at 06:30 IST in Prod.
  2. Deploy Test, then Prod in the pipeline after Test UAT email.
  3. Remap gateway if the dialog asks; save.
  4. Open lineage — confirm App points at the new model version.
  5. Smoke-test one city (Nashik) with Test as role / App link.

Tiny example: Sign-off mail (fictional): “Prod App updated 18:10 IST; usage check Monday.”

Ravindra Bagale's Tip

Friends, never deploy during the 09:00 IST standup — pick a quiet window and tell Teams first.

6. Interview answers that sound senior

Why: Interviewers ask how you avoid copy-paste ETL and risky Prod edits.

What: Say: dataflows for shared clean data; pipelines for Dev/Test/Prod; lineage before breaking changes; usage to prune.

How — practise the 60-second answer:

Vertical steps (one under another)

  1. Explain one shared entity and two consuming models.
  2. Name the three pipeline stages.
  3. Mention gateway/parameter rules between stages.
  4. Say you check lineage before dropping a column.
  5. Mention usage metrics for adoption.

Tiny example: Good line: “We refresh Sales_Clean once, promote with a pipeline, and refuse silent Prod edits.”

Ravindra Bagale's Tip

Friends, name a real failure you prevented (blank App) — stories beat buzzwords.

Quiz (5 questions)

1. A dataflow mainly helps you:

2. Deployment pipelines typically move content:

3. Lineage view is for:

4. Usage metrics help you:

5. Best time to deploy to Prod in the story:

Practice tasks (12) — hints only, no full answers

  1. Task. List three problems of five copy-paste Sales queries.
    Hint: section 1.
  2. Task. Write steps to connect Desktop to a dataflow entity.
    Hint: Get data, then dataflows.
  3. Task. Name the three pipeline stages and one UAT check.
    Hint: section 2.
  4. Task. Why remap gateway after Prod deploy?
    Hint: paths/credentials differ.
  5. Task. Using lineage, list what you check before dropping CityKey.
    Hint: section 3.
  6. Task. Draft a one-line mail to owners before a breaking rename.
    Hint: lineage dependents.
  7. Task. What does a near-zero-view report suggest?
    Hint: section 4.
  8. Task. Interview: dataflow vs copying Power Query into every .pbix.
    Hint: shared entity.
  9. Task. Schedule idea: dataflow 06:30, model 06:45, App open 07:05 IST — why this order?
    Hint: order matters.
  10. Task. Why not edit visuals only in Prod?
    Hint: pipelines / Dev.
  11. Task. MH Ops Friday cutover — five bullet checklist.
    Hint: section 5.
  12. Task. How does usage support training decisions?
    Hint: who never opened App.

Ravi Sir's cards (Excel card style)

Standalone cards — no tip or photo sections. QR points to this lesson. Download PNG to revise offline.

Interview Q&A: How do you avoid copy-paste cleaning and risky Prod edits?
Download interview-card PNG
Comparison Card: Five copied queries vs one dataflow
Download comparison-card PNG
Revision Card: Friday cutover checklist
Download revision-card PNG

Think like a professional

In a real tenant, rights and sharing are controlled by Fabric / Power BI admins. Always follow your company policy. This lesson uses fictional people and numbers so you can practise the ideas safely.