Data Engineer • Senior

Who you are

You have a strong background in data engineering with experience in ingesting and processing healthcare data from various sources. Your expertise in dbt and PySpark allows you to design efficient data pipelines that ensure data quality and accessibility. You understand the importance of collaboration and can work effectively with cross-functional teams to identify data requirements and improve existing processes.

You are skilled in metadata-driven design, enabling you to account for schema drift and implement robust error handling strategies. Your focus on scalability and efficiency drives you to optimize data ingestion systems, making them adaptable to changing data sources and formats. You are proactive in identifying areas for improvement and are committed to leveraging technology for social good.

What you'll do

As a Senior Data Engineer at 10Pearls, you will be responsible for ingesting a variety of healthcare data, ensuring it is converted into a common data model. You will work on consolidating and streamlining redundant data pipelines to enhance scalability and performance. Your role will involve designing and implementing a metadata-driven approach for data ingestion, which includes robust error handling strategies to ensure data integrity.

Collaboration is key in this role, as you will work closely with cross-functional teams to understand their data needs and identify opportunities for process improvements. You will focus on making the data ingestion system more scalable and efficient, adapting to new data sources and formats as they arise. Leveraging technologies like dbt and PySpark, you will enhance data processing capabilities and improve overall data workflows.

What we offer

At 10Pearls, we foster a culture of innovation and are committed to harnessing technology for social good. You will have the opportunity to work in a collaborative environment where your contributions directly impact the success of our projects. We provide internal training and access to online courses to support your professional development. Join us in our mission to help businesses transform and scale through digital technology.