Job Description
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design appropriate solutions.
- Work closely with IT and infrastructure teams to deploy and maintain data storage and processing systems, including databases, data warehouses, and cloud infrastructure.
- Design, build, and maintain robust, scalable data pipelines to ingest, process, and transform data from various sources, including internal databases and external APIs.
- Optimize data pipelines for performance, scalability, and reliability, ensuring timely and accurate delivery of data.
- Implement data validation and quality checks to identify and rectify any issues in data pipelines, ensuring data integrity.
- Explore and evaluate new technologies, tools, and frameworks to improve data engineering processes and capabilities.
- Perform data modeling and schema design to support analytical and reporting requirements.
- Design and buil...