Incremental Refresh in Power BI: Setup, Limits and Pitfalls

Incremental refresh lets Power BI reload only your recent data instead of the whole table. You split a large table by date, set a refresh window, and the service handles the rest. Done well, refreshes get shorter and more reliable. Done poorly, nothing improves, and sometimes data gets duplicated.

This guide covers how the feature works, the setup steps in Power BI Desktop, the limits that catch teams off guard, and the checks we run before we call a refresh design finished.

What incremental refresh does

A normal scheduled refresh reloads every row in every table. That works for small models. For a fact table with years of orders, claims or ledger entries, it wastes time, memory and source capacity on data that has not changed.

With incremental refresh, Power BI partitions a table by date or time. Only the recent refresh window is reprocessed. Older data stays in historical partitions. After you publish and run the first refresh, the service creates and manages those partitions for you.

The pattern is a rolling window:

• New partitions enter the refresh period.
• Older partitions move into history.
• Historical partitions may be merged.
• Data outside the archive period is removed.

You do not maintain partitions by hand. That is the main difference from classic partitioning in tabular models, where an administrator defines and processes partitions directly. If you searched for Power BI partitions, this is the managed version. Advanced control comes through the XMLA endpoint, covered below.

When it is worth using

Incremental refresh fits large fact or transaction tables that have a usable date or date/time column and a predictable pattern of new or updated records. Small, static dimension tables usually do not need it.

Typical candidates by team:

Skip it when the table is small, when there is no reliable date column, or when old rows change in ways you cannot tie to a date. In those cases the policy adds complexity without a payoff.

Licensing and limits to know first

Incremental refresh is supported for Power BI Pro, Premium, Premium per user and Embedded models, according to Microsoft's overview. The real-time DirectQuery partition is narrower. It works only on Premium capacity, Premium per user and Embedded models, not ordinary Pro workspaces.

Refresh time limits still apply. Without XMLA, the documented limit is two hours for Pro models and five hours for models in Premium capacity. XMLA refresh operations in Premium capacity have no time limit.

Memory matters too. Microsoft notes that a refresh can require as much as twice the model's memory, because the service keeps a snapshot until the refresh completes. That explains a common surprise: a model that fits in capacity can still fail to refresh.

For refresh calls through the REST API, the enhanced refresh documentation covers large partitioned models. The default timeout is five hours, and the total duration, retries included, cannot exceed 24 hours.

Incremental refresh reduces how much data is processed. It does not remove these limits, and it does not fix a slow source.

How to set it up in Power BI Desktop

The steps below follow Microsoft's configuration guidance.

Step 1: Create the RangeStart and RangeEnd parameters

Open Transform data to get to Power Query. Create two parameters with the Date/Time type. Name them RangeStart and RangeEnd. The names are case sensitive, so spell them exactly. Give each a default value. These values only control how much data loads while you build the report in Desktop.

Step 2: Filter the fact table

Filter your date column using the two parameters. Use one inclusive boundary and one exclusive boundary:

• OrderDate >= RangeStart
• OrderDate < RangeEnd

Do not make both boundaries inclusive. A row that sits exactly on a boundary can land in two partitions, and you end up with duplicate data. The filter column should be a Date/Time column. If your source stores dates as integers or text, you need a way to convert them without breaking the push to the source.

Step 3: Check that the filter folds

Query folding means Power Query translates your steps into a query the source runs. When the RangeStart and RangeEnd filter folds, the source returns only the rows for each partition. When it does not, Power BI may retrieve the full table and filter it locally. That removes the benefit and can exhaust service or gateway resources.

In Power Query, right-click the filter step and look for the View Native Query option. If it is greyed out, folding stopped at or before that step. Move the date filter as early as possible, and remove or reorder steps that break folding. Microsoft's post on incremental refresh and query folding explains why the two are tied together.

Step 4: Close and apply

Load the filtered table into the model. Desktop will now hold only the data between your default parameter values.

Step 5: Define the policy

Right-click the table, choose Incremental refresh, and set:

1. Archive period. How far back to keep data, such as 5 years.
2. Refresh period. How recent a window to reload each time, such as 7 days or 1 month.

Applying the policy in Desktop does not load any data. Nothing happens until you publish.

Step 6: Publish and run the first refresh

Publish to the Power BI Service. The initial historical load happens when the service performs the first manual or scheduled refresh. Expect that first run to be the longest, since it builds all the historical partitions. After publication, the service overrides the parameter values according to your policy.

If the first refresh fails, Microsoft's troubleshooting page separates the two phases of an implementation: configuration in Desktop, then the initial refresh in the service. Knowing which phase failed narrows the search.

Choosing the refresh window

The smallest window is not always the best one. The right window covers the period in which your data can still change. Ask these questions about each table:

• How late can a record arrive? A claim or invoice may be posted days after the event date.
• How long do corrections go on? Finance teams often adjust prior periods during close.
• Which date does the column hold: the event date or the last modified date?

If corrections reach back 30 days and you refresh only 3 days, the model will quietly disagree with the source. Pick the window from how the business corrects its data, then test it against a known set of corrected records.

Change detection can help here. It lets Power BI refresh a partition only when a tracking column, such as a last updated timestamp, shows a newer value. It has a firm limit: it does not detect hard-deleted rows. It catches a soft delete only when the source keeps the row and updates the tracking column to mark it as deleted. If your healthcare, finance, inventory or fundraising data needs removals reflected, plan a separate deletion strategy.

Adding near real-time data

On Premium-capable workspaces, you can select the option to get the latest data in real time with DirectQuery in the policy. This creates a DirectQuery partition for the newest slice, so queries can read changes that happened after the imported refresh period.

This is a hybrid model, not a replacement for scheduled refresh. Imported partitions still refresh on schedule. Only the newest slice uses DirectQuery. There is a trade-off: DirectQuery makes report queries depend on the source system at the moment someone opens the report. Before you promise real-time decision support to executives, test:

• How many people will query at once during peak hours
• How fast the source answers the DirectQuery slice
• What happens to the report when the source is down or busy

Freshness and speed are separate problems. A report can be current and slow at the same time.

Mistakes that cost teams the most time

Both boundaries inclusive

This produces duplicate rows. Check the filter before you publish.

A filter that does not fold

The policy looks correct, the refresh still takes hours, and the gateway struggles. The cause is almost always a step that stopped folding. Fix the source query first.

Republishing over live partitions

Publishing a changed PBIX file after production partitions exist is not a harmless metadata update. A republish can remove the existing partitions and data and trigger a new initialization. For models in Premium capacity, XMLA-based deployment supports metadata-only changes and partition management through tools, scripts and APIs, so you can keep partitions when that matters.

Treating the archive period as only a performance setting

Data outside the archive window is removed from the semantic model. That changes what users can analyze. For healthcare, banking, finance and non-profit teams, set the archive period together with approved retention and audit requirements, not after.

Assuming it solved performance

Measure before and after. Track refresh duration, source query duration, gateway load, failed refreshes and data latency. If refresh time did not drop, look at source indexing, gateway sizing, capacity and model design.

A pre-launch checklist

1. The fact table has a reliable Date/Time column.
2. RangeStart and RangeEnd are Date/Time parameters with exact names.
3. One boundary is inclusive, one is exclusive.
4. View Native Query is available on the filter step.
5. The refresh window covers late and corrected records.
6. The archive period matches retention policy.
7. A deletion strategy exists where removed rows must disappear.
8. The workspace license supports any real-time partition you plan to use.
9. A deployment method is chosen that will not wipe partitions.
10. Baseline refresh metrics are recorded for comparison.

Our view

We see the same pattern in many refresh projects. A team enables incremental refresh, the policy looks right, and the refresh is still slow. The policy was never the bottleneck. The source query, the gateway or the model design was.

Our approach is to diagnose the data flow first. We look at where the time goes, from the source query to the gateway to the service. Then we set the refresh window around how the business actually corrects its data, not around the smallest number that works in a test. We also ask about retention before we set the archive period. Many teams do not realize that the policy decides what history users can see.

Last, we plan deployment from day one. If the model will sit in Premium capacity, we choose a deployment path that keeps partitions. Reloading years of history because of a small report change is a cost that is easy to avoid. An attractive dashboard on top of a slow or unreliable refresh does not help the people who rely on it. The data flow has to be right first.

Get help with your refresh design

If your refreshes run long, fail on capacity, or return numbers that do not match the source, we can review the model and the data flow. Our Power BI consulting work covers refresh design, performance tuning and dashboard automation. If you are planning a larger build, see our BI and dashboard development service, or start with a free assessment.

If the real problem is retention rules, ownership or inconsistent definitions across sources, our data strategy and governance service is the better starting point. You can also read how we approach a first dashboard in Power BI for FP&A, or our Tableau to Power BI migration guide.

Frequently asked questions

What are the limitations of incremental refresh in Power BI?

It needs a Date/Time column to partition on and works best on large fact tables. The real-time DirectQuery partition requires Premium capacity, Premium per user or Embedded. Refresh time limits, memory needs and source speed still apply. Change detection also cannot see rows that are hard deleted at the source.

Does incremental refresh load data when I click Apply in Desktop?

No. Applying the policy only saves the rules. The historical load happens in the Power BI Service during the first manual or scheduled refresh after you publish. Desktop loads only the data allowed by your RangeStart and RangeEnd default values.

How do I use incremental refresh in Power Query?

Create Date/Time parameters named RangeStart and RangeEnd, then filter your date column with them. Use one inclusive and one exclusive boundary. Confirm the filter folds to the source, then close and apply and configure the policy on the table.

Can I change the model after incremental refresh is live?

Be careful. Republishing a changed PBIX file can remove existing partitions and data and trigger a new initialization. On Premium capacity, XMLA-compatible deployment methods support certain metadata-only changes while keeping partitions.

Is incremental refresh the same as real-time data?

No. Incremental refresh reloads a recent window on a schedule. Real-time data is an optional DirectQuery partition for the newest slice, available on Premium-capable workspaces. It adds freshness but also makes report queries depend on the source at viewing time.