We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions.

Key Responsibilities

- Design and architect Databricks Lakehouse solutions for large-scale enterprise data platforms
- Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization
- Define Databricks workspace architecture, cluster strategies, security models, and operational standards
- Design and develop scalable data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake
- Implement data ingestion frameworks supporting batch and real-time data processing
- Design Delta Lake architectures including:
- Delta tables
- Medallion architecture (Bronze/Silver/Gold layers)
- Data quality frameworks
- Schema evolution and optimization

- Perform Databricks performance tuning:

- Cluster optimization
- Spark job optimization
- Query tuning
- Cost optimization

- Implement Databricks governance and security using:

- Unity Catalog
- Access controls
- Data lineage
- Audit capabilities

- Lead migration projects from legacy data platforms to Databricks Lakehouse
- Define best practices for Databricks CI/CD, DevOps, and environment management
- Collaborate with data engineers, data scientists,



analysts, and enterprise architects
- Provide technical guidance, architecture documentation, and design reviews

Skills & Experience

- Strong experience in data engineering, big data, or solution architecture
- Strong hands‑on experience with Databricks Platform
- Expert knowledge of:
- Apache Spark
- PySpark / Scala
- Delta Lake
- SQL
- Data Lakehouse architecture
- ETL/ELT development

- Experience designing and delivering enterprise Databricks solutions
- Strong experience with cloud platforms:

- Azure Databricks (preferred)
- AWS Databricks
- Google Cloud Databricks

- Experience with:

- Databricks Workflows
- Jobs orchestration
- Notebooks
- Repos
- Cluster management
- Unity Catalog

Preferred Skills

- Databricks Certified Data Engineer Professional / Databricks Certified Architect
- Experience with:
- MLflow
- Databricks Machine Learning
- Feature Store
- Model deployment
- MLOps

- Knowledge of streaming technologies:

- Structured Streaming
- Kafka

- Experience with data governance, compliance, and enterprise security frameworks

Key Competencies

- Strong Databricks architecture and implementation expertise
- Ability to design scalable and cost-efficient Lakehouse solutions
- Strong troubleshooting and performance optimization skills
- Ability to lead technical discussions with enterprise stakeholders

#J-18808-Ljbffr

📌 Senior Databricks Solution Architect (Veglio)
🏢 Unison Group
📍 Veglio