Senior DevOps Engineer

We are seeking a hands-on Senior DevOps Engineer to join our Data and Analytics Platform team, driving the design and implementation of our next-generation cloud-based, streaming Data Mesh platform. This role is pivotal for enabling teams to create scalable, data-driven products and advancing our application modernization strategy.

Design and Implement Cloud Infrastructure:
Architect and deploy scalable cloud infrastructure using Infrastructure as Code (IaC) principles to support our Data Mesh platform. Experience with major cloud providers (AWS/Azure) and modern IaC tools (Terraform, CloudFormation) is essential.

Build Automation Pipelines:
Develop and maintain CI/CD pipelines for data products, ensuring reliable deployment, testing, and monitoring of data infrastructure and applications. Experience with tools like Jenkins, GitLab CI, or GitHub Actions is required.

Implement Platform Observability:
Design and deploy comprehensive monitoring, logging, and alerting solutions across the data platform. Experience with tools like Datadog, Prometheus, Grafana, ELK stack, or similar technologies is crucial.

Enable Self-Service Infrastructure:

• Develop automated provisioning systems for data infrastructure
• Create and maintain infrastructure templates and patterns
• Implement GitOps workflows for infrastructure management
• Build developer portals and self-service tooling

Security and Compliance Automation:

• Implement security controls and compliance checks as code
• Automate security scanning and vulnerability management
• Design and implement IAM and access management solutions
• Maintain audit trails and compliance reporting automation

Performance Optimization:

• Monitor and optimize platform performance
• Implement auto-scaling solutions design for high availability and disaster recovery
• Optimize cost through infrastructure efficiency

Lead Platform Re-Architecture:

• Transform our existing data platform architecture to align with Data Mesh principles
• Design and implement our Apache Iceberg adoption strategy, enabling ACID transactions and time travel capabilities
• Create standardized patterns for teams to effectively adopt and optimize Iceberg tables

Drive Release Excellence:

• Establish automated release processes that balance agility with platform stability
• Build deployment frameworks with comprehensive validation, monitoring, and rollback capabilities
• Implement coordinated release strategies across distributed domains

Enable Technical Success:

• Design reference architectures and implementation patterns for domain teams
• Lead technical design reviews and architecture governance
• Provide technical guidance during the transition to Data Mesh architecture

What We're Looking For
8+ years of experience in DevOps/Platform Engineering roles within a data operations team

Cloud & Infrastructure:

• Extensive experience with AWS, particularly EKS and EKS Anywhere
• Deep expertise in Kubernetes and container orchestration
• Strong background in Infrastructure as Code using Terraform and Terragrunt
• Experience with GitOps practices using ArgoCD
• Knowledge of distributed systems and event-driven architectures
• Strong knowledge of logging, monitoring, APM tracing and alerting tools (ELK, Datadog, Prometheus, Grafana, etc.)

Development & Engineering:

• Proficient in Java development and JVM optimization
• Experience with build tools (Maven/Gradle) and dependency management
• Strong background in unit testing frameworks and testing practices
• Expertise in Git version control and collaborative development
• Experience with Apache Kafka and Kafka Streams
• Background in monitoring and performance optimization for Java applications

CI/CD & Automation:

• Experience designing and managing Jenkins/Github/Gitlab pipelines
• Strong scripting skills (Python, Bash)
• Understanding of DevOps practices and automation principles

Preferred Qualifications:

• Experience with data mesh or similar distributed data architectures
• Relevant cloud certifications
• Experience building developer platforms or portals
• Background in DataOps practices