Data Engineer • Mid-Level

Who you are

You have a strong background in data engineering with experience in designing and building data pipelines — you've worked with large-scale data systems and understand the intricacies of data ingestion and transformation. Your proficiency in Python allows you to develop robust batch data pipelines that efficiently load data from various sources into data lakes. You are familiar with AWS services, particularly AWS Glue and EMR, and have a solid understanding of data modeling in environments like Hive and Athena.

You are comfortable operating workflows in Airflow, managing dependencies, scheduling tasks, and ensuring that data quality checks are in place. Your experience includes implementing validation checks to maintain data integrity and using monitoring tools to surface issues quickly. You thrive in collaborative environments and are eager to contribute to a team that values innovative ideas and solutions.

Desirable

Experience with machine learning models and their integration into data pipelines is a plus — you understand how to prepare data for ML workloads and can model datasets to support various use cases. Familiarity with Elasticsearch for indexing and querying data will enhance your contributions to the team.

What you'll do

As a Data Engineer at Qloo, you will be responsible for designing, building, and maintaining data pipelines that facilitate the movement of data from external vendors and internal systems into our S3-based data lake. You will work closely with cross-functional teams to ensure that data is accurately modeled and efficiently accessible for analytics and machine learning applications. Your role will involve optimizing data workflows and ensuring that data quality is maintained throughout the pipeline.

You will implement and operate workflows in Airflow, managing task dependencies and scheduling to ensure timely data processing. Your responsibilities will include building robust data quality checks, such as schema validation and anomaly detection, to ensure that data issues are identified and addressed promptly. You will also be involved in performance tuning of data jobs to optimize costs and improve efficiency.

Collaboration is key in this role, as you will partner with data scientists and analysts to understand their data needs and ensure that the data infrastructure supports their requirements. You will have the opportunity to shape a modern cloud-based data platform that powers real products and machine learning experiences, contributing to the overall success of Qloo's data initiatives.

What we offer

At Qloo, we provide a comprehensive benefits package that includes health insurance, a retirement plan, and paid time off. We foster a collaborative, low-ego work environment where your ideas are valued and your contributions are visible. You will have flexible work arrangements, including remote and hybrid options, allowing for a healthy work-life balance. Join us in shaping a modern data platform that drives insights and recommendations across various industries.