Lead Azure Data Engineer
Sign up free to see how well your resume matches this role.
What you'll do
- Create and maintain data sources inside our data lake on Azure
- Assemble and analyze large, complex data sets that meet both functional and non-functional business requirements
- Identify, design, and implement internal process improvements: automating manual processes; optimizing data delivery; re-designing infrastructure for greater scalability.
- Work with stakeholders including the Executive, Tech, Product, Data and Design teams to optimize data driven decisions
- Keep our data separated and secure by understanding and applying best practice access policies
- Create tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
- Ensure data integrity with thorough data quality test coverage and data validation
- Improve, create and implement CI/CD processes and tooling inside the data lake
What they're looking for
- Join immediately
- Databricks
- Databricks hands on experience (esp. Pipelines PySpark & SQL, Delta Lake, Unity Catalog, DLT*)
- Ingesting & transforming data using Databricks - Min 3 years continuous hands-on
- Advanced SQL and Python knowledge
- Data Lake housing and Modelling experience (e.g. medallion architecture & query performance optimization)
- Experience with backend systems ERP, CRM (e.g. SAP, Oracle, Salesforce)
- Exposure of working with Azure (ADLS, ADF)
- Good Communication skills
Summarised by NextRaise from the employer’s description, which follows in full below.
Full description from employer
- Job Title: Lead Data Engineer
- Location: Marathahalli, Bangalore
- Job Mode: Hybrid
Key Responsibilities:
· Create and maintain data sources inside our data lake on Azure
· Assemble and analyze large, complex data sets that meet both functional and non-functional business requirements
· Identify, design, and implement internal process improvements: automating manual processes; optimizing data delivery; re-designing infrastructure for greater scalability.
· Work with stakeholders including the Executive, Tech, Product, Data and Design teams to optimize data driven decisions
· Keep our data separated and secure by understanding and applying best practice access policies
· Create tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
· Ensure data integrity with thorough data quality test coverage and data validation
· Improve, create and implement CI/CD processes and tooling inside the data lake
Required Qualifications
- Join immediately
- Databricks
- Databricks hands on experience (esp. Pipelines PySpark & SQL, Delta Lake, Unity Catalog, DLT*)
- Ingesting & transforming data using Databricks - Min 3 years continuous hands-on
- Advanced SQL and Python knowledge
- Data Lake housing and Modelling experience (e.g. medallion architecture & query performance optimization)
- Experience with backend systems ERP, CRM (e.g. SAP, Oracle, Salesforce)
- Exposure of working with Azure (ADLS, ADF)
- Good Communication skills
Company
Company facts come from this company's own listings. We only show what the postings themselves carry.