Easily Install Task Factory on an Azure-SSIS IR in Azure Data Factory
When Microsoft first introduced how to install third-party components, such as SentryOne Task Factory, on an Azure-SSIS Integration Runtime (IR) in Azure Data Factory (ADF), I thought, "This could be a bit difficult for some users."
At the time, you had to write a PowerShell script, upload it to an Azure storage blob, grant access rights to the storage blob, and execute another PowerShell script from your machine to install Task Factory. And, if you got any of those steps wrong, you wouldn't know the installation process had failed for 20-30 minutes. So, it was possible you'd have to attempt the installation process more than once to install Task Factory on an Azure-SSIS IR.
The Process Is Much Easier Now
Microsoft recently made updates to the third-party component installation process, enabling you to easily install Task Factory on ADF as part of the Azure-SSIS IR express custom setup process. Microsoft wants you to move to the cloud and use ADF, and they figure the easiest way to get you there is by using a lift and shift approach for your SQL Server Integration Services (SSIS) packages.
Lift and shift is a cloud migration strategy where you move apps, files, databases, or processes to the cloud with very little friction. From an SSIS and Task Factory standpoint, lift and shift to an Azure-SSIS IR is relatively simple and involves creating an Azure-SSIS IR, selecting Task Factory as a licensed component during the express custom setup process, and deploying your SSIS packages. (Of course, this doesn't include the process of moving your databases/data sources to the cloud, which will add a layer of complexity to your overall lift and shift process.)
That said, let's look at how easy it is to move your SSIS packages to an Azure-SSIS IR that includes Task Factory components.
The Step-By-Step Process
First, create your Azure-SSIS IR by clicking the Configure SSIS Integration icon on the ADF overview page.
On the 'Integration runtime setup' screen, configure the general settings of your Azure-SSIS IR.
You will also select the subscription, location, and the Azure SQL Database server that will contain your SSIS catalog database. If you want the wizard to create the SSIS catalog, select the 'Create SSIS catalog' checkbox. If you already have an SSIS catalog on the SQL Server defined, clear this checkbox.
On the 'Advanced settings' screen, select the 'Customize your Azure-SSIS Integration Runtime with additional system configurations/components installations' checkbox. Then, click '+ New' below 'Express custom setup.'
In the 'Add express custom setup' window, select 'Install licensed component' and then select 'SentryOne Task Factory.'
You can then enter your Task Factory ADF license key in the 'License key' field.
And that's it! Every time you start your Azure-SSIS IR, each node will have Task Factory automatically installed and activated without any further steps required from you.
Dozens of high-performing SSIS components that help you save time managing ETL tasks.
Deploying Packages with Task Factory Components to the Azure-SSIS IR
Deploying packages that contain Task Factory components to an Azure-SSIS IR is like deploying packages to a local SSIS catalog.
To deploy your packages, simply follow these steps. (Note that the screenshots below are using Visual Studio 2017 with SQL Server Data Tools (SSDT) installed.)
First, right-click your project and click ‘Deploy.’
Choose 'SSIS in Azure Data Factory' as your deployment target. (If you don't see the 'SSIS in Azure Data Factory' option, you'll need to upgrade Visual Studio 2017 and SSDT to the latest version.)
Enter the authentication information you entered in the 'Catalog database server endpoint' field when you configured your Azure-SSIS IR settings. Click 'Connect' after you enter the details to enable the Browse button for the path.
Click 'Browse' and choose which folder and project you want to deploy the packages.
Review your settings to ensure they are correct, and then click 'Deploy.' Your packages are now deployed to your Azure-SSIS IR.
Executing Packages with Task Factory Components within an Azure-SSIS IR
Now that your packages are deployed to the Azure-SSIS IR and Task Factory is installed, you can now execute those packages. Here are two options you can use to execute them.
The first option is to use the same process as executing a package on a local SSIS catalog, which means connecting to the server you deployed your packages to from within SSMS and executing them.
The second package execution option is to add an Execute SSIS Package activity to your Azure Data Factory pipeline and configure the package to execute. Then, click 'Debug' in the Pipeline design service to execute the package.
See? It's easy to execute SSIS packages in an Azure-SSIS IR!
A Streamlined Lift and Shift Process for SSIS and Task Factory
The lift and shift process for SSIS and Task Factory has been simplified by Microsoft and SentryOne. We worked together to ensure a smooth transition from local package storage and execution to a more dynamic and flexible cloud solution.
If you have questions about how to use Task Factory during your lift and shift process, don't hesitate to reach out to our world-class SentryOne Support team. They're happy to help you ensure a smooth transition.
Chris started with PragmaticWorks (now SentryOne) in 2009 and has worked on several projects, including Task Factory, Workbench, and DOC xPress. Now the lead developer for the Task Factory project, he spends his days deep in SSIS and ADF, creating solutions to make the lives of Microsoft Data Professionals more enjoyable. Plus, he makes beer.