Staff Data Engineer (Azure Databricks Platform)

Position Summary Staff Data Engineer-Azure Databricks Platform We're looking for a highly skilled Platform Staff Engineer to lead the development and maintenance of our Databricks platform, supporting our healthcare delivery initiatives. You will be a hands-on leader, responsible for the architecture, implementation, and operational excellence of our data platform. You will be responsible for ensuring the platform's reliability, performance, and security, while also driving its strategic evolution to meet future business needs. This position requires a blend of deep technical expertise, a proactive mindset, and a commitment to operational excellence. This role sits at the intersection of platform engineering, cloud engineering, data governance, and developer enablement, requiring a blend of deep technical expertise, strategic thinking, and a passion for building scalable, reliable, and secure systems. You will be a trusted partner for our data engineers ensuring they have the tools and best practices to drive business value. Responsibilities ● Operational Excellence: Oversee the operational health of the platform. Maintain and enhance monitoring, logging, and alerting solutions to proactively identify and resolve issues. Lead incident response and perform root cause analysis for platform-wide problems. ● Platform Architecture & Strategy: Maintain and enhance the core components of our Azure Databricks environment. Define the strategic roadmap for the platform, including the adoption of new features, services, and best practices (e.g., Delta Lake, Unity Catalog, Databricks Workflows. Github CI/CD). ● Security & Governance: Manage the enforcement and enhancement of data security and governance policies. Manage the Unity Catalog to ensure fine-grained access control, data lineage, and compliance with internal and external regulations. ● Performance & Cost Optimization: Implement strategies for cost management and performance tuning at a platform level. …

…