Senior Data Engineer Backend
Sign up free to see how well your resume matches this role.
What you'll do
- Build and run ingestion pipelines from EHR systems into Databricks: landing, validation, normalization, and the silver and gold tables that analytics and the product read.
- Onboard new clinics, which usually means understanding a new export format and writing the orchestration to fetch it on a schedule.
- Maintain change-data-capture from our MongoDB application database into the analytics layer and keep the two provably consistent.
- Build de-identified data exports for research partners, with controls that keep real identifiers from ever leaving.
- Own data quality: freshness checks, reconciliation against source, and alerts that fire before anyone downstream notices.
- Investigate data incidents to the root cause and backfill safely without breaking downstream readers.
- Write orchestration code in TypeScript alongside the backend team.
What they're looking for
- Five or more years of experience building and operating production data pipelines.
- Strong command of SQL and Python. You write both in production, not occasionally.
- Hands-on experience with Spark or a comparable engine, and with Delta Lake or an equivalent table format.
- Working knowledge of idempotency, deduplication keys, event ordering, and schema evolution in real pipelines.
- Experience with change-data-capture or event-stream processing from an operational database.
- Willingness to write TypeScript for orchestration code.
- Understanding of how sensitive data leaks in practice, including through file names, object keys, and logs.
- Comfort with ambiguous, unglamorous problems, such as working out why one clinic’s spreadsheet has a different header this week.
Nice to have
- Familiarity with Databricks and Unity Catalog, including governance and access controls.
- Experience with Temporal or another durable-execution engine.
- Hands-on experience with MongoDB.
- Familiarity with healthcare data, including claims, CPT and ICD-10 codes, and HIPAA de-identification rules.
- Prior experience with TypeScript or Node.js.
- Experience with infrastructure as code (Terraform) and CI/CD for data pipelines.
Summarised by NextRaise from the employer’s description, which follows in full below.
Full description from employer
Flagler Health is building the clinical operating system for modern musculoskeletal care.
We partner with MSK provider groups and specialty clinics to help them grow, operate more efficiently, and deliver better longitudinal care across patient acquisition, clinical workflows, and ongoing patient engagement. Our platform sits at the intersection of care delivery and clinic operations, helping providers capture more value across the full patient lifecycle.
We’ve recently raised our Series B and are entering our next phase of growth.
The Role:
Flagler Health works with outpatient clinics to run care management programs and identify patients who would benefit from them. That work depends on clinical and claims data pulled from dozens of different EHR systems, each exporting it differently: SFTP drops, portal downloads, vendor APIs, bulk exports. Formats drift, files arrive late or twice, and everything is protected health information.
As a Senior Data Engineer, you will own the pipelines that bring this data in, keep it correct, and get it back out to the product and to research partners. You will make messy input dependable, and you will be the one who notices when it quietly stops being dependable. This is a hands-on role on a small team: you write and operate production code.
What You Will Do
Build and run ingestion pipelines from EHR systems into Databricks: landing, validation, normalization, and the silver and gold tables that analytics and the product read.
Onboard new clinics, which usually means understanding a new export format and writing the orchestration to fetch it on a schedule.
Maintain change-data-capture from our MongoDB application database into the analytics layer and keep the two provably consistent.
Build de-identified data exports for research partners, with controls that keep real identifiers from ever leaving.
Own data quality: freshness checks, reconciliation against source, and alerts that fire before anyone downstream notices.
Investigate data incidents to the root cause and backfill safely without breaking downstream readers.
Write orchestration code in TypeScript alongside the backend team.
Required Qualifications
Five or more years of experience building and operating production data pipelines.
Strong command of SQL and Python. You write both in production, not occasionally.
Hands-on experience with Spark or a comparable engine, and with Delta Lake or an equivalent table format.
Working knowledge of idempotency, deduplication keys, event ordering, and schema evolution in real pipelines.
Experience with change-data-capture or event-stream processing from an operational database.
Willingness to write TypeScript for orchestration code.
Understanding of how sensitive data leaks in practice, including through file names, object keys, and logs.
Comfort with ambiguous, unglamorous problems
Preferred Qualifications:
Familiarity with Databricks and Unity Catalog, including governance and access controls.
Experience with Temporal or another durable-execution engine.
Hands-on experience with MongoDB.
Familiarity with healthcare data, including claims, CPT and ICD-10 codes, and HIPAA de-identification rules.
Prior experience with TypeScript or Node.js.
Experience with infrastructure as code (Terraform) and CI/CD for data pipelines.
Work Environment:
Distributed remote team with hybrid offices in NYC and Vancouver.
Hybrid. Regular and consistent on-site presence is required for this position.
We have video on for our weekly team syncs and pair-coding/tech discussions. Camera is required to be on during meetings.
Our Values:
This is what you can expect of your teammates at Flagler:
Owner: We own our work like founders. We don't wait to be asked, and we pick up the problem nobody else has.
Builder: We like making things from scratch. We don't need a playbook to start, and we'd rather ship a rough first version than wait on a perfect spec.
Fast: We make quick decisions. We favor progress and respect process.
Transparent: We communicate directly. We value merit and honesty, and we challenge assumptions, including our own.
Curious: We explore new approaches and ask "why" often.
Company
Company facts come from this company's own listings. We only show what the postings themselves carry.