Job description:
* building solutions against delta lakes and relational databases
* work on Azure Databricks writing code in Python and SQL, with PySpark
* perform analysis, design, implementation and testing
* extend the functionality of the consistency check engine for the sub-ledger data and any other tenants
* Given the large data volume (datasets with 10s of billion rows, 200+ columns), you also need to dive deeply into the spark performance optimization
About the customer:
For our renowned client in the Insurance industry, we are looking for an experienced Senior Application Engineer for a major transformation project.
Requirements:
* Bachelor's or master's degree in computer science or electrical engineering
* Expert software engineering skills. Proven experience developing complex software systems as a software engineer
* Strong core Python software engineering experience * Have a good understanding and significant experience with projects using relational data models and SQL
* Spark experience is a must.
* Good work ethic and self-starter. Not content to be "task driven".
* You look out for issues, opportunities and address them when you can, raise them when you can't.
* Good analytical skills and the ability to understand complexity and break it down into smaller, achievable steps.
* Highly proficient in English (written and verbal)
jid50ef43dja jit0936ja jiy25ja