Developers trying to use ADF for ETL & have already experienced its inability to transform data.
In this session we’ll go beyond the Azure Data Factory copy activity normally presented using the limited portal wizard. Extract and load are never the hard parts of the pipeline. It is the ability to transform, manipulate and clean data that normally requires more effort. Sadly, this task doesn’t come so naturally to Azure Data Factory as an orchestration tool so we need to rely on its custom activities to break out the C# or VB to perform such tasks. Using Visual Studio, we’ll look at how to do exactly that and see what’s involved in Azure to utilise this pipeline extensibility feature. What handles the compute for the compiled .Net code and how can does this get deployed by ADF. With real world use cases we’ll learn how to fight back against those poorly formed CSV files and what we could to if Excel files are our only data source. Plus, lots of useful tips and tricks along the way when working with this emerging technology.
Why I Want to Present This Session:
Azure Data Factory has been poorly marketed and often we only get talks showing the basics. This talk will give people an understanding of how to use ADF in the real world and how to deal with those frustrating limitations.
Latest posts by Paul Andrew (see all)
- Working with Azure Data Factory & Creating Custom Activities - March 21, 2017