Use Azure Data Factory to find duplicate records (two source files)

I've read many sources online that illustrate how to create a flow that detects record duplicates. Some of these duplicate detection patterns can be quite complex, most are old patterns.…

Configuring Dynamics 365 CE Data Export Service

The steps below outline the actions needed to build the Dynamics 365 Data Export Service. What I found was that documentation on this subject is sparse, but also, blog posts…

Stop Azure Data Factory Data Flows outputting to multiple files

I came across a recent issue where I was trying to overwrite a single in my Data Flow. However, Data Factory started to output several intermediary files instead.

Filtering in Azure Data Factory

Microsoft have now made it easier to filter records from a delimited source. We all know it's possible to filter SQL data sources, but now filtering in Azure Data Factory…

Create a Dynamics 365 CE Trial Instance with your Azure Student Subscription

I signed-up to an Azure Student Account for an Azure Developers course that I was studying during the evenings. One thing that I noticed immediately was that I could not…

Get started with Azure Analysis Services

I was looking for some documentation online that would help me run Azure Analysis Services on an Azure SQL DB using Visual Studio 2019. Documentation is sparse so I thought…

Table is required for Copy Activity

I've seen a lot of people manually create tables in Azure SQL when doing a data migration from a source system to SQL DB as the Sink. For those that…

Integration testing with Data Factory, Data Lake, Dynamics 365 Finance & Operation

Just for fun, I decided to play around with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS) and Microsoft Dynamics 365 Finance & Operation VM. I wanted to see…

Designing a Modern Data Warehouse – Available Options

I recently spent some time with Microsoft to look at the various options for building our big data pipelines. Here are the final options that exist for anyone considering doing…

Transfer an Azure Subscription between tenants

Here's a quick tip for those that have subscriptions between tenants, that want to manage or transfer a Subscription between Azure tenants.