Course Overview

This course, Data Factory and Data Lake will enable you to master the techniques of performing Big Data Engineering on Microsoft Cloud Services.

In this course, you'll be introduced to Azure Data Lake, the U-SQL language, query using the powerful U-SQL language, built straight into the Azure Data Lake, how to throw your files into the Data Lake and query them directly without needing to load them into a database

Finally, you'll learn about how Azure Data Lakes offers the best of both worlds, with support for unstructured files and structured databases. By the end of this course, you’ll know what a Data Lake is, how to populate it, query it, and develop for it using Visual Studio, how to connect power bi with data lake and using data factory

WHAT YOU'LL LEARN / COURSE OBJECTIVE:

Understand what is Azure Data lake and its architecture
How to copy data from azure SQL data lake
How to load data from data lake to PowerBI
Create Azure Data Lake account and Data Lake Storage
ETL process by creating a Pipeline in Data Factory.
Load Data from Data Lake to Power BI.
Using Data Factory for automating the movement of data and transformation of data.
Design and Implement Complex Event Processing using Azure Stream Analytics.
Design and Implement Analytics by using Azure Data Lake.
Design and Implement Azure SQL Data Warehouse Solutions
Design and Implement Cloud-Based Integration by using Azure Data Factory.
Manage and Maintain Azure SQL Data Warehouse, Azure Data Lake, Azure Data Factory, and Azure Stream Analytics.

PRE-REQUISITES:

Knowledge on data engineering issues with Azure SQL Data Warehouse, Azure Data Lake, Azure Data Factory, and Azure Stream Analytics.

Training Syllabus

Overview of Azure Data Lake

What is Data Lake?
What is Azure Data Lake?
Data Lake Architecture?

Working With Azure Data Lake

Create Azure Data Lake account and Data Lake Storage
Explore Data Lake Analytics
Explore Data Lake Store
Uploading Sample File
Using Azure Portal
Using Storage Explorer

Working With Azure Data Lake Job

Create simple Job
Using Simple U-SQL Query
Understanding AU
Overview of Job Graph

Working With U-SQL

Introduction to U-SQL
Extractors and Outputters
Azure Data Lake Schema on Read
Rowset
U-SQL Datatypes:
Using C# inline Functions
Process data From Multiple Files and Filter it
Using Visual Studio for Executing U-SQL Script

Working With U-SQL Catalog

U-SQL Catalog
Working With Table Valued Function
Create Database, Schema and Table
Insert Data
Query Table
Working With View
Working With Stored Procedure

Using C# to extend U-SQL Capabilities

Using Inline Function and operators
Using Code-Behind Class
Using Custom Assembly
Using Visual Studio to create and register assembly.

Understanding U-SQL JOB.

U-SQL JOB Architecture
Understanding Job Graph
Understanding Job Heat Map
Monitoring and Optimizing U-SQL Jobs

Introduction to Data Factory

What is Data Factory?
Data Factory Key Components
- Pipeline and Activity
- Linked Service
- Data Set
- Integration Runtime

Provision Required Azure Resources

Create Resource Group
Create Storage Account
Provision SQL Server and Create Database
Provision Data Factory

Working With Copy Activity

- Understanding Data Factory UI
- Copy Data from Blob Storage to SQL Database (Use Copy Data Wizard)
- Copy data from storage account to storage account
  - Create Linked service
  - Create Dataset
  - Create Pipeline
- Integration Service
- Copy Data from on-premise SQL Server to Blob Storage

Working with Activities

Understanding Lookup Activity
Understanding For Each Activity
Filter Activity
Get Metadata Activity

Azure Lift and Shift

Provisioning Azure - SSIS Integration Runtime
Execute SSIS Packages from Azure
Execute SSIS Packages from SSISDB

Triggers, Monitoring Pipeline

Debug Pipeline
Trigger pipeline manually
Monitor pipeline
Trigger pipeline on schedule

Azure Courses

About Azure Online Training at Deccansoft AZ-103: Microsoft Azure Administrator AZ-203: Developing Solutions for Microsoft Azure AZ-300: Microsoft Azure Architect Technologies

Fees and timings

Azure Administrator Suite: AZ-104 (AZ-100 + AZ-101):
Rs. 17,500/- + GST or USD 300/-
Azure Suite Plus (AZ-104 + AZ-204 + AZ-305 + AZ-400 + DataFactory & DataLake)
Rs. 22500/- + GST or USD 375/-
Azure Devops(AZ-400)
Rs. 12000/- + or USD 200/-
DataFactory & DataLake
Rs. 5000/- + or USD 90/-
To stay updated on all information related to Azure Batches and Free Webinars, Join our WhatsApp group.