c# - How do you automate the creation of a dataset within the .NET SDK for Azure Data Factory? -

i using microsoft azure data factory .net sdk in order automate dataset creation large number of tables.

a method within .net console application provides me ability create input , output datasets, based on specified table name:

createinputdataset(string table_name, datafactorymanagementclient client) {     client.datasets.createorupdate(resourcegroupname, datafactoryname,         new datasetcreateorupdateparameters()         {             dataset = new dataset()             {                 properties = new datasetproperties()                 {                     structure = new list<dataelement>()                     {                         //todo: autogenerate columns , types                         new dataelement() {name = "name", type = "string" },                         new dataelement() {name = "date", type = "datetime" }                     }             }... 

currently, dataset creation accomplished through stored procedure on either source sql server or target sql data warehouse. stored procedure specifies table name , looks information_schema in order generate valid columns , types each adf dataset. manually copy result portal.azure.com.

we have on 600 datasets, need utilize .net sdk automated copy adf.

how 1 create datasets automatically, while taking account each dataset's structure (i.e. columns , types) differ?

the way i've been able accomplish writing stored procedure generate column names , types on both source and target. such stored procedure should call information_schema , information_schema.columns in order generate each column , type inputted table.

once procedure adequately outputs 2 columns (name, type) programmatically call procedure , save follow:

list<dataelement> inputparams = new list<dataelement>();  sqlconnection connect = new sqlconnection(<connection_string>); sqlcommand cmd = new sqlcommand("putil_gendfautomate", connect);  cmd.commandtype = commandtype.storedprocedure; cmd.parameters.add(new sqlparameter("@tablename", <table_name>)); using (var reader = cmd.executereader()) {     if (reader.hasrows)     {         while (reader.read())         {             var name = reader.getstring(0);             var type = reader.getstring(1);              inputparams.add(new dataelement             {                 name = name,                 type = type             });         }         reader.close();     } } 

then, upon creation of input/output dataset, use variable inputparams follow:

new datasetcreateorupdateparameters() {     dataset = new dataset()     {         properties = new datasetproperties()         {             structure = inputparams  //etc.