File Encryption And Decryption Using ADF
DESCRIPTION: Files will be placed on the SFTP server. They should be staged in blob storage and from the blob, they will be loaded into the Azure SQL database. After loading into SQL DB, they should be archived and loaded as encrypted files in the Archive folder . DATA FLOW: SOLUTION: Entire pipeline in ADF STEP1: From the SFTP server files will be loaded into staging storage i.e., blob storage, and the files in it are as follows: STEP2: As a next step, multiple files with the same structure in blob storage will be loaded into the Azure SQL DB table. The data in SQL DB is as follows: STEP3: Using the “Get Metadata” activity, extract the list of files from blob storage which is staging storage - as ‘child items’. STEP4: Using parallel processing in the “For Each” activity iterate through child items that were extracted from the above step. STEP5: Using “web activity”, passing the filename as a parameter, Azure Function is called with URL concatenated wit...