We are seeking an experienced Data Engineer to join our team for a long-term project.
If you are passionate about data, skilled in modern data engineering tools, and have a strong background in designing, building, and optimizing data pipelines, we want to hear from you!
Key Responsibilities:
- Configure and integrate data sources.
- Organize and maintain a data catalog to support various teams.
- Manage and optimize data warehouses for business intelligence purposes.
- Collaborate closely with development and analytics teams to support data-driven decision-making.
Required Skills and Qualifications:
-
Apache Airflow:
- Proven experience in corporate environments configuring and managing data pipelines.
- Developing DAGs in Python for ETL automation.
- Optimizing and monitoring DAG performance.
-
ETL/ELT Development:
- Hands-on experience designing and implementing ETL/ELT processes based on business requirements.
- Strong understanding of data transformation and integration principles.
-
Data Catalogs:
- Experience working with and managing data catalogs.
-
DBT (Data Build Tool):
- Developing and maintaining data models using DBT.
- Integrating DBT into data pipelines for data transformation.
- Knowledge of version control for data models.
-
Streaming Data (Kafka, Spark Streaming):
- Experience with Kafka: configuration, optimization, and scaling.
-
SQL Expertise:
- Writing complex queries and optimizing the performance of existing queries.
-
Relational Databases:
- Hands-on experience with PostgreSQL, MSSQL, or similar databases.
Nice to Have:
- Familiarity with cloud platforms like Yandex.Cloud.
- Version control with Git, including CI/CD setup for data processes (e.g., Airflow, DBT).
What We’re Looking For in a Candidate:
- Proven experience with data transformations using DBT.
- Experience with data quality testing and validation.
- Strong skills in setting up CI/CD pipelines for data processes.
