Get AI-powered advice on this job and more exclusive features.
Caser Grupo Helvetia is looking for an Azure Databricks DevOps Engineer to join its IT Asset Management team at the Helvetia Service Hub located in Madrid, providing services to the Helvetia Group Asset Management function. The selected individual will be part of a team of over 50 professionals, based primarily in Basel, Switzerland (70%), and in Madrid, Spain (30%)."
In this position, you will play a key role in designing, developing, and optimizing data pipelines and workflows. Your primary responsibility will be implementing scalable and high-performance data solutions using Azure Databricks, Airflow, Unity Catalog, Data Factory, and Git. You will collaborate closely with data engineers, architects, and business stakeholders to deliver high-quality data solutions that support analytics, reporting, and operational processes.
Responsibilities:
* Design, develop, and optimize ETL/ELT data pipelines using Azure Databricks and Apache Spark, ensuring scalability, reliability, and performance.
* Implement and manage Unity Catalog for metadata management, access control, and governance within the Azure environment.
* Develop, schedule, and orchestrate workflows using Apache Airflow and Azure Data Factory.
* Collaborate with data architects and engineers to define and implement data models, data governance standards, and best practices.
* Monitor and troubleshoot data pipelines, ensuring efficient processing and error handling.
* Write clean, modular, and reusable PySpark/Scala code following software engineering best practices.
* Integrate and manage Git-based version control to track changes, enable CI/CD practices, and ensure code quality.
* Work closely with business stakeholders to understand data requirements and ensure solutions align with business objectives.
* Optimize performance tuning for Spark-based transformations and large-scale distributed computing.
* Stay up to date with advancements in Azure, Databricks, and big data technologies, continuously improving system performance and efficiency.
Requirements:
* Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related technical field.
* Proven experience in developing and managing data pipelines in Azure Databricks.
* Proficiency in Apache Spark, PySpark, or Scala, with hands-on experience in large-scale data processing.
* Strong knowledge of Azure Data Services, including Azure Data Factory, Unity Catalog, and Azure Storage.
* Experience with Apache Airflow for workflow orchestration and job scheduling.
* Solid understanding of data modeling, ETL processes, and best practices for data engineering.
* Hands-on experience with Git for version control and CI/CD integration.
* Strong problem-solving skills with the ability to analyze, debug, and optimize data workflows.
* Familiarity with data governance principles, security, and compliance standards within cloud environments.
* Excellent communication and collaboration skills for working with cross-functional teams.
* Certifications in Azure, Microsoft Azure Data Engineer, or related cloud technologies.
* Scala
* Familiarity with dbt (data build tool) for data transformation and pipeline automation.
* Experience with streaming technologies (e.g., Spark Streaming, Kafka) for real-time data processing.
* Understanding of financial instruments and asset management data.
Languages
English – Advanced (most of the work will be conducted in English, in coordination with the Basel team).
Seniority level
* Seniority level
Not Applicable
Employment type
* Employment type
Full-time
Job function
* Industries
Insurance
Referrals increase your chances of interviewing at CASER by 2x
Sign in to set job alerts for “Data Engineer” roles.
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 2 weeks ago
Data Engineer - Advanced Analytics Madrid
Madrid, Community of Madrid, Spain 6 days ago
Tres Cantos, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 1 month ago
Madrid, Community of Madrid, Spain 3 weeks ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 1 month ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 1 day ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 1 day ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 6 days ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 3 weeks ago
Madrid, Community of Madrid, Spain 1 year ago
Madrid, Community of Madrid, Spain 1 day ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 3 days ago
Madrid, Community of Madrid, Spain 2 days ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 3 days ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 3 months ago
Sevilla La Nueva, Community of Madrid, Spain 3 months ago
Madrid, Community of Madrid, Spain 3 weeks ago
Madrid, Community of Madrid, Spain 2 months ago
Madrid, Community of Madrid, Spain 14 hours ago
Madrid, Community of Madrid, Spain 1 month ago
Madrid, Community of Madrid, Spain 1 week ago
Madrid, Community of Madrid, Spain 2 weeks ago
Madrid, Community of Madrid, Spain 2 weeks ago
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr