Job Description
We are looking for a highly skilled Senior Databricks Engineer to contribute to the engineering, modernization, and continuous evolution of data processing platform on Databricks on AWS. While supporting the transition from the legacy Cloudera Hadoop platform to Databricks on AWS, this role will continue to play a key part in enhancing performance, simplifying pipelines, and delivering new capabilities on the Databricks platform over the long term.
The ideal candidate is a strong hands‑on Spark engineer with solid design experience, capable of contributing to architectural decisions while leading complex implementation and optimization efforts.
**Responsibilities:**
**1. Platform Engineering & Modernization**
+ Refactor and modernize existing Spark pipelines to Databricks native architectures
+ Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
+ Enhance and extend existing processing logic using optimized Spark (JavaSpark...
The ideal candidate is a strong hands‑on Spark engineer with solid design experience, capable of contributing to architectural decisions while leading complex implementation and optimization efforts.
**Responsibilities:**
**1. Platform Engineering & Modernization**
+ Refactor and modernize existing Spark pipelines to Databricks native architectures
+ Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
+ Enhance and extend existing processing logic using optimized Spark (JavaSpark...