Lead Data Platform Engineer

Onsite Dallas, Texas, United States

pythonsqlkubernetesterraformazure programming

greenhouse

Match & cover letter

Create a profile to see your match and get a cover letter for this role.

Create profile and match

Job description

You will design and build a next-generation risk analysis platform in Dallas. This role involves working with a small, senior team to create data architectures and pipelines.

Details

Dallas, Texas, on-site
5+ years of data engineering experience, 2+ years with Spark development
Skills: Python, SQL, Kubernetes, Terraform, Azure

The work

Design and implement Bronze, Silver, and Gold Delta table schemas
Build Fabric Spark notebooks for feature pipelines
Create the canonical MERGE DAG for data ingestion
Establish idempotent Delta write patterns using delta-rs