Senior Databricks / Spark Data Engineer
Remote, Spain
About MDW
MDW is a leading consulting company specialized in Data, AI and BizApps, with offices in Geneva and Madrid. Recognized as Microsoft Partner of the Year with multiple awards, we help major organizations transform their business with Azure, Microsoft Fabric, Databricks and AI technologies.
Your Role
We are looking for a Senior Databricks / Spark Data Engineer to join our team and work on cutting-edge, enterprise-scale Data projects. Given the criticality of the production phase we are entering, we need a profile with deep expertise in Databricks and Big Data ecosystems, proven ability to deliver at very high data volumes, and the seniority to address complex challenges autonomously and with confidence.
What You Will Do
* Design and deliver enterprise-scale Databricks and Spark data solutions, handling very high data volumes with parallelised execution across countries and markets
* Build and evolve Data Solutions on Azure & Microsoft Fabric
* Take end-to-end ownership of data pipelines, from data processing to BI-ready datasets, with a strong focus on performance tuning and scalability
* Deliver real consulting projects across industries, reacting quickly to evolving requirements
* Provide personalized support to clients, working closely with them on-site in a hybrid model, ensuring effective collaboration and solutions that align with their business objectives
* Strengthen seniority, autonomy, and delivery confidence within the team
* Collaborate with an international team of experts at MDW to grow together
What We Value
Core Requirements – Databricks & Big Data
* Proven experience delivering Databricks projects at very high data volumes, with parallelised execution across countries or markets
* Strong Python and PySpark expertise, with deep knowledge of Spark performance tuning and optimization
* Hands-on experience with Declarative Automation Bundles and CI/CD pipelines (Azure DevOps)
* Experience with end-to-end delivery: from data processing to BI-ready datasets
* Experience with Batch Data workflow design, including Medallion Architecture
* Experience with Data Modelling: Star schemas, Snowflake schemas
* Knowledge of Azure Data Platform: Data Factory, Synapse, Data Lake Gen2, Azure Functions, Logic Apps, etc.
* Strong SQL skills
Infrastructure & DevOps
* Experience with CI/CD with Azure DevOps
* Knowledge of IaC with Terraform, Bicep, ARM, or Pulumi
Soft Skills & Language
* Ability to work with high autonomy and deliver quickly in a fast-paced, evolving project environment
* Strong client communication skills, ensuring effective management and a positive experience throughout the project
* Fluent in Spanish & English
Certifications (Valued)
* Databricks certifications (strongly valued)
* Azure certifications: DP-203, AZ-104, AZ-304...
* Experience working in an Agile environment
Nice to Have
* Real-time workflow design (Kafka, structured streaming, Delta Live Tables, Fabric Eventhouse, etc.)
* Master Data Management & Governance tools (Profisee, Purview)
* Knowledge of Microsoft Fabric platform
Why MDW?
* Competitive salary based on your experience
* 6 weeks holidays
* Work on disruptive Data & AI projects with top clients
* Grow your expertise in the Microsoft and Databricks ecosystem within a dynamic international team