#Microsoft #Azure #DataFactory #MappingDataFlows instructions for migrating off private preview to ADF V2

2. • V1
• V2 – GA version
• V2 w/ Data Flow (preview) – upgrading
customers to the GA version

3. • GA version of ADF Dataflow will support
• Managed Databricksinstances
• New Dataflow dataset types
• CSV, Parquet,DB, DW

5. Preview (Old) GA (New) Instances
{
"name":"[concat(parameters('factoryName'),
’/…')]",
"type":"Microsoft.DataFactory/factories/pip
elines",
"apiVersion":"2018-07-01-preview",
"properties":{
"activities":[
{
…
}
{
"name":"[concat(parameters('factoryName'), ’/…'
)]",
"type":"Microsoft.DataFactory/factories/pipelin
es",
"apiVersion":"2018-06-01",
"properties":{
"activities":[
{
…
}
1 change per entity
(pipeline, dataset,
dataflow)

6. Preview (Old) GA (New) Instances
{
"name":"[concat(parameters('factoryName'),
'/MovieDemo')]",
"type":"Microsoft.DataFactory/factories/dat
aflows",
"apiVersion":"2018-07-01-preview",
"properties":{
"sources":[
{ }
],
"sinks":[
{ }
],
"transformations":[
{ }
],
"parameters":{ }
},
"dependsOn":[ ]
}
{
"name":"[concat(parameters('factoryName'), '/Mo
vieDemo')]",
"type":"Microsoft.DataFactory/factories/dataflo
ws",
"apiVersion":"2018-06-01",
"properties":{
"type":"MappingDataFlow",
"typeProperties":{
"sources":[
{ … }
],
"sinks":[
{ … }
],
"transformations":[
{ … }
],
"parameters":{ }
}
},
"dependsOn":[ ]
}
1 change per dataflow
entity

7. Preview (Old) GA (New) Instances
{
"name":"[concat(parameters('factoryName'), ’/Mo
vieDemo')]",
"type":"Microsoft.DataFactory/factories/pipelin
es",
"apiVersion":"2018-07-01-preview",
"properties":{
"activities":[
{
"name":“MovieDemoDataflow",
"type":"ExecuteDataFlow",
"dependsOn":[
],
"policy":{ },
"userProperties":[ ],
"typeProperties":{
"dataflow":{ },
"linkedService":{
"referenceName":“My_Databricks",
"type":"LinkedServiceReference"
},
"staging":{ }
}
}
],
"annotations":[ ]
},
"dependsOn":[ ]
}
{
"name":"[concat(parameters('factoryName'),
’/MovieDemo')]",
"type":"Microsoft.DataFactory/factories/pi
pelines",
"apiVersion":"2018-06-01",
"properties":{
"activities":[
{
"name":“MovieDemoDataflow",
"type":"ExecuteDataFlow",
"dependsOn":[
],
"policy":{…},
"userProperties":[ ],
"typeProperties":{
"dataflow":{… },
"staging":{ },
"compute":{
"computeType": "General",
"coreCounts": 4
}
}
}
],
"annotations":[ ]
},
"dependsOn":[ ]
}
1 change per dataflow
activity in pipeline
Azure IR/Compute

8. 5) Create new Dataflow datasetsas needed