Design and implement data pipelines to ingest, process, and store large datasets from various sources, ensuring data quality and reliability.

Develop and maintain data models, schemas, and ETL processes for data warehousing and analytics solutions, optimizing for performance and scalability.

Collaborate with data scientists and analysts to understand their data requirements and provide them with clean, transformed data for analysis and model building.

Implement data governance policies and procedures to ensure data security, privacy, and compliance with regulatory requirements.

Monitor and troubleshoot data pipeline performance, identifying and resolving bottlenecks to ensure timely and accurate data delivery.

Automate data integration and data quality checks to improve efficiency and reduce manual effort.

Evaluate and recommend new data technologies and tools to improve data processing and storage capabilities.

Create and maintain documentation for data pipelines, data models, and ETL processes.

Provide training and support to data users on how to access and use data effectively.

Stay up-to-date with the latest trends and technologies in data engineering and big data.

Architect robust data pipelines for efficient data ingestion, transformation, and loading into data warehouses.

Develop and maintain ETL processes using tools like Apache Spark, Informatica, or Python to ensure data quality.

Manage and optimize relational (SQL) and NoSQL databases (e.g., MongoDB, Cassandra) for performance and scalability.

Utilize cloud-based data services (AWS, Azure, GCP) for storage, processing, and analytics to build scalable solutions.

Design and implement data models that support business requirements and analytical needs, ensuring data integrity.

Implement data quality checks and monitoring systems to identify and resolve data inconsistencies and errors.

Process large datasets using big data technologies like Hadoop, Spark, and Kafka for real-time and batch processing.

Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.

Optimize data pipelines and database queries for performance, ensuring efficient data processing and retrieval.

Work with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.