Data Pipeline Engineer
Global Connect Technologies
📍 jalisco, jalisco, Mexico
Job Description
Job DescriptionWe are seeking an experienced Senior Data Pipeline Engineer to design, build, and operate secure, reliable, and cost-efficient data pipelines supporting our Android connected and infotainment experiences.
You will lead Python-based ETL/ELT development, Airflow orchestration, and data platform operations across cloud environments, working closely with Android, backend, and product teams.ResponsibilitiesBuild and maintain Airflow 2.X DAGs (TaskFlow, dynamic DAGs, deferrable operators, providers).
Develop robust Python ETL/ELT ingesting from APIs, storage, message buses, and databases.Operate Airflow on Azure/Kubernetes; support blue/green and canary DAG releases.Implement data quality testing, monitoring, observability, and lineage.Design scalable batch and streaming pipelines with strong schema management.Manage SQL and blob data stores (partitioning, clustering, retention).
Enforce security best practices, IAM, RBAC, secrets management, and data contracts....