Azure Data Engineer (ADF, Databricks, Spark SQL)
A
Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!
📍 toronto, on, Canada
Job Description
Scope of Work
Responsible for designing, developing, and maintaining Azure-based data pipelines using ADF and Databricks, including ETL/ELT workflows, Spark SQL processing, and Azure SQL integration, while ensuring data quality, performance optimization, and reliable data delivery.Works on monitoring, troubleshooting, and resolving pipeline and cluster issues, and collaborates with DevOps teams using Azure DevOps, Git, CI/CD, and monitoring tools like Azure Monitor and Log Analytics to support stable and scalable data operations.Required SkillsAzure Data Factory (ADF)Azure DatabricksAzure SQLSpark SQLUnderstanding ofETL / ELT conceptsData ingestion patternsData warehousingExperience inTroubleshooting pipeline failuresCluster issuesPerformance bottlenecksFamiliarity withAzure DevOpsGit version controlCI/CD pipelinesKnowledge of monitoring toolsAzure MonitorLog AnalyticsDatabricks logs#J-18808-Ljbffr
Responsible for designing, developing, and maintaining Azure-based data pipelines using ADF and Databricks, including ETL/ELT workflows, Spark SQL processing, and Azure SQL integration, while ensuring data quality, performance optimization, and reliable data delivery.Works on monitoring, troubleshooting, and resolving pipeline and cluster issues, and collaborates with DevOps teams using Azure DevOps, Git, CI/CD, and monitoring tools like Azure Monitor and Log Analytics to support stable and scalable data operations.Required SkillsAzure Data Factory (ADF)Azure DatabricksAzure SQLSpark SQLUnderstanding ofETL / ELT conceptsData ingestion patternsData warehousingExperience inTroubleshooting pipeline failuresCluster issuesPerformance bottlenecksFamiliarity withAzure DevOpsGit version controlCI/CD pipelinesKnowledge of monitoring toolsAzure MonitorLog AnalyticsDatabricks logs#J-18808-Ljbffr