Senior Cloud Engineer for Databricks Platform
Join to apply for the Senior Cloud Engineer for Databricks Platform role at Callista.
Role
For our client, a Swiss insurance company, we are seeking a highly skilled Senior Cloud Engineer for Databricks. You will be responsible for designing, implementing, and maintaining their data platform infrastructure to enable data-driven decision-making and analytics. Collaboration with cross-functional teams is essential to ensure cloud systems are robust, scalable, and optimized for performance.
Work location can be flexible. The role can be performed either nearshore from the Balkan region or in Switzerland.
Key Responsibilities
* Engineer and implement projects, changes, and improvements of the Azure Databricks platform, focusing on Monitoring, Scheduling, Load Balancing, Automation, Access, and Security.
* Utilize hands-on experience with Azure DevOps and Azure Data Factory; knowledge of Azure Synapse Analytics and Power BI is advantageous.
* Create and manage automation scripts using Python, PowerShell, or Azure CLI for deployment and management tasks.
* Implement best practices for performance and security.
* Write technical specifications, review code, and perform functional tests.
* Ensure security, integrity, and high availability of cloud systems.
* Monitor cloud infrastructure costs and identify optimization opportunities.
* Collaborate with teams to enhance cloud systems.
* Develop documentation and guidelines to improve processes.
Qualifications
* Extensive experience as a Cloud Engineer specializing in Azure Databricks, with knowledge of Azure Data Factory and Azure DevOps; familiarity with Azure Synapse Analytics and Power BI is a plus.
* Hands-on experience optimizing Databricks clusters, implementing Spark transformations, and supporting collaborative data environments.
* Proficiency in CI/CD pipelines with Azure DevOps, security, and compliance practices using Unity Catalog.
* Expertise in Python, PowerShell, and Azure CLI for automating deployment and management of cloud infrastructure.
* Ability to manage various data formats and sources, including CSV, JSON, Parquet, Delta Lake, ZIP, Excel, and REST APIs.
* Proficiency in English; German language skills are a plus.
#J-18808-Ljbffr