Senior/Lead Data Software Engineer (Python, Spark, Azure)

Epam

📍 Poland, Poland, Poland

Full-time Computer Occupations

Job Description

We are seeking a Senior/Lead Data Software Engineer to join our team working on a scalable, ML-ready platform that enhances portfolio model development and deployment with advanced data governance and AI capabilities.

You will play a key role in migrating from an IaaS Big Data platform to Azure-native Databricks, optimizing data workflows and improving data quality. Join us to contribute to innovative solutions that boost client services and regulatory compliance.

Responsibilities
  • Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques
  • Manage and process 12 TB of data efficiently across platforms
  • Tune machine learning models for Azure environments using Java Spark and Delta tables
  • Update and maintain libraries to address security vulnerabilities
  • Develop and maintain ETL/ELT pipelines using PySpark and related technologies
  • Collaborate with cross-functional teams to integrate GenAI capabilities i...
  • Apply for this Position