Join to apply for the DATA ENGINEER role at CASER
Position Overview
Caser Grupo Helvetia is seeking an Azure Databricks DevOps Engineer to join its IT Asset Management team at the Helvetia Service Hub in Madrid. The role involves designing, developing, and optimizing data pipelines and workflows, utilizing technologies such as Azure Databricks, Airflow, Unity Catalog, Data Factory, and Git. The successful candidate will work closely with data engineers, architects, and stakeholders to deliver high-quality data solutions supporting analytics and operational processes.
Responsibilities
* Design, develop, and optimize ETL/ELT data pipelines with Azure Databricks and Apache Spark.
* Manage Unity Catalog for metadata, access control, and governance.
* Develop and orchestrate workflows using Apache Airflow and Azure Data Factory.
* Collaborate with teams to define data models, standards, and best practices.
* Monitor and troubleshoot data pipelines for efficiency and error handling.
* Write clean, modular PySpark/Scala code following best practices.
* Use Git for version control, CI/CD, and code quality assurance.
* Engage with stakeholders to align solutions with business needs.
* Optimize performance for Spark transformations and distributed computing.
* Stay updated with advancements in Azure and big data technologies.
Minimum Requirements
* Bachelor's or Master’s degree in Computer Science, Engineering, Data Science, or related field.
* Experience managing data pipelines in Azure Databricks.
* Proficiency in Apache Spark, PySpark, or Scala for large-scale data processing.
* Knowledge of Azure Data Services, including Data Factory and Unity Catalog.
* Experience with Apache Airflow for workflow orchestration.
* Understanding of data modeling and ETL best practices.
* Experience with Git and CI/CD practices.
* Problem-solving skills for debugging and optimizing workflows.
* Knowledge of data governance, security, and compliance in cloud environments.
* Excellent communication and teamwork skills.
Desirable Skills
* Azure certifications or related cloud certifications.
* Scala programming experience.
* Familiarity with dbt and streaming technologies like Kafka or Spark Streaming.
* Knowledge of financial instruments and asset management data.
Languages
English – Advanced (required for collaboration with Basel team).
#J-18808-Ljbffr