Contract Type: Full-Time (40 hrs/week – long-term engagement)

Location: 100% Remote – Mexico, India or Sri Lanka

Experience: 5+ years of experience in Platform Engineering, Cloud Engineering, DevOps, Data Infrastructure, or related engineering roles. 

Role Overview

We are seeking an experienced Databricks Platform Engineer to design, build, automate, and operate our enterprise Databricks Lakehouse platform on AWS. This role bridges Cloud Platform Engineering, Data Infrastructure, and Data Engineering, with a strong focus on infrastructure automation, security, governance, and standardization. 

The ideal candidate will use Terraform and Infrastructure as Code (IaC) to deliver scalable and secure Databricks environments while establishing enterprise data governance through Databricks Unity Catalog. 

Key Responsibilities 

Infrastructure as Code & Platform Automation 

• Design, build, and maintain reusable Terraform modules for Databricks infrastructure across Development, Test, and Production environments. 
• Automate provisioning and lifecycle management of Databricks workspaces, compute resources, SQL warehouses, networking, security configurations, and storage integrations. 
• Manage IaC for AWS components including VPCs, security groups, IAM roles, private connectivity, and storage endpoints. 
• Develop and maintain CI/CD pipelines using GitHub Actions, GitLab CI, or similar platforms. 
• Implement deployment automation using Databricks Asset Bundles, Databricks APIs, Terraform, and related automation frameworks. 
• Establish reusable platform patterns and self-service “golden paths” for Data Engineering, Data Science, and Analytics teams. 

Unity Catalog & Data Governance 

• Design and implement the enterprise Unity Catalog architecture, including catalogs, schemas, volumes, storage credentials, and external locations. 
• Define governance standards that provide appropriate workload and team isolation while enabling secure data sharing and discoverability. 
• Implement and manage Role-Based Access Control (RBAC) and object-level permissions based on least-privilege principles. 
• Integrate Databricks access controls with enterprise identity and access management platforms. 
• Implement appropriate row-level and column-level security controls. 
• Establish data governance capabilities including lineage, auditing, classification, tagging, and access controls. 
• Automate governance policies through Terraform, APIs, and other platform automation mechanisms. 

Platform Operations & Security 

• Design and maintain compute and cluster policies to ensure standardized and secure platform usage. 
• Manage platform configuration, upgrades, capacity, and operational support. 
• Partner with AWS Cloud and Security teams to implement IAM controls, encryption, KMS integration, network security, and vulnerability remediation. 
• Troubleshoot Databricks platform, infrastructure, networking, access, and configuration issues. 
• Develop operational standards, architectural documentation, troubleshooting procedures, and platform runbooks. 

Required Skills & Qualifications 

• Experience: 5+ years of experience in Platform Engineering, Cloud Engineering, DevOps, Data Infrastructure, or related engineering roles. 
• Databricks: Strong hands-on experience administering enterprise Databricks environments, including workspaces, compute, SQL warehouses, cluster policies, jobs, and platform security. 
• Unity Catalog: Practical experience designing and implementing Unity Catalog, including catalogs, schemas, external locations, storage credentials, permissions, and governance controls. 
• Terraform: Advanced experience developing reusable Terraform modules and managing Terraform state, providers, and multi-environment deployment patterns. 
• AWS: Strong hands-on experience with IAM, S3, VPC networking, security groups, KMS, and PrivateLink/private connectivity. 
• CI/CD: Experience developing automated deployment pipelines using GitHub Actions, GitLab CI, Jenkins, or equivalent platforms. 
• Automation: Experience with Databricks APIs, Databricks Asset Bundles, CLI tools, and infrastructure automation frameworks. 
• Programming: Proficiency in Python and SQL for automation, validation, and troubleshooting. 
• Security: Strong understanding of cloud security, identity management, least-privilege access, encryption, and secrets management. 

Preferred / Nice-to-Have Skills 

• Experience with additional AWS data services, such as DynamoDB, Redshift, Glue, Athena, and Kinesis. 
• Familiarity with Azure data services, such as Synapse Analytics, Data Factory, ADLS, Cosmos DB, and Microsoft Fabric. 
• Exposure to modern cloud data, streaming, analytics, and AI/ML services across AWS and Azure. 

ID: P24