Combine data sets with APPEND in Power Query You can use this simple technique to consolidate data from different sources into a single data set to make analysis easier This is useful if you can't combine the data at source or if you dont have the facility https://www.selectdistinct.co.uk/2023/09/27/append-data-in-power-query/ #PowerQuery #Append #PowerBI

1. SQL Tips
and Timesavers
Combine data with
APPEND in Power Query

2. You may need to combine or consolidate data from
multiple sources, which may not be possible to do
before it is loaded into Power Query
Here is a simplified example combining sales data
from two different sources using the APPEND
function

3. Assuming we have two
different sales data sources
We want to combine Sales A
with the data in Sales B
(Notice that Store ID 5 appears
in both)
Sales A Sales B

4. We already have two similar
tables from two different
sources already loaded into
Power BI

5. Go to the table view in Power
BI desktop and select the first
table Right click 'Edit query' option
on the data pane for the first
table

6. The interface now opens up a Power Query
editor window and we can see the data in the
first table

7. On the Home tab, under
the Append Queries
button, select the drop
down and choose
'Append Queries as New'
Combine the tables using the APPEND QUERIES option
Now select the two
tables in the Append
dialogue box that pops
up

8. Here is the new table after the data
is appended
From here you can rename the
table, apply any other
transformations you may need and
use it as you would any other table

9. To demonstrate this, go back to
Sales A and add a column
We added a custom column called
Source and populated it with
"SalesA"
if we now look back at the
Appended table, the extra column is
automatically added, and the SalesB
data is shown as NULL values in this
new column
If the columns are not identical what happens?

10. Go into SalesB and add a
custom column called
‘Company’ and populate
it with "SalesB"
What happens if the columns have different names?
Now, if we look back at the
appended table, another
new column is added, with
NULL values in the
"SalesA"data
You can see that if column names don’t exactly match then the append will treat them as separate columns
To correct it go back into the one of the source tables and change the field name to match

11. You may actually need to remove duplicates
To do this, select all of the columns you want
to be included in your duplicate checking.
Note that we have only selected the first three
columns, as the Source Column would have
two different values for Store ID 5
Then select 'Remove Rows' on the Home
Ribbon, and select Remove Duplicates from
the drop-down options
How to remove duplicates in append queries?

12. The key things to remember when appending data in Power Query is to
make sure that the field names and data types are the same before
running the append
It does not matter if the columns are in a different order as the final
column order is derived from the first table
You can append more than two tables, which is a very similar process

13. For more Tips, Tricks and
Timesavers, visit our website
Business Analytics Blog – Select Distinct
Credit: simon.harrison@selectdistinct.co.uk