Big Data Pipeline Developer

Dear Professionals

Greetings from Tata consultancy Services,

Job Title AWS Data Engineer with Pyspark

Experiernce: 5-10 Years

Location: Only Pune/ Hyderabad

Mode of Work : Work from Office

Mode of Interview: Virtual

Key Responsibilities

• Data Pipeline Development: Design, build, and optimize scalable batch and near-real-time ETL/ELT pipelines using PySpark and AWS data services.
• Cloud Ingestion & Processing: Process massive datasets from diverse sources including structured databases, APIs, and raw flat files.
• Workload Migration: Migrate legacy on-premises workloads (such as Ab Initio, Informatica, or SSIS) into modern cloud-based data lakes.
• Pipeline Orchestration: Develop, monitor, and schedule complex workflows using Apache Airflow or AWS Step Functions.
• Performance Tuning: Debug and optimize Spark co...