Intermediate/Senior DevInfraOps engineer to assist in maintaining the Data Engineering platform, implementing best practices, and enhancing underlying architecture. This involves establishing a solution for the acquisition, movement, and ingestion of large volumes of structured (future: unstructured) data daily, and making this data available for users.
Skills, in priority order (from must-have to nice-to-have), include:
- Kubernetes (K8s), Airflow/Dagster/Airbyte, Argo Workflows, Databricks
- Infrastructure as Code (IaC) - Terraform, Cloud Development Kit (CDK)
- Monitoring (Datadog, Splunk, PagerDuty) - managing a production system
- On-Premise Management – proficiency in upgrading and maintaining RHEL7/8 Cloud
- Management – proficiency in managing and maintaining EKS in AWS
- CI/CD – proficiency in setting up and managing GitLab pipelines
- AI stack - OpenAI, Claude 3, Pinecone, SageMaker, Bedrock APIs, Layer 3/7 networking (Envoy)
- Application authentication (Okta)
- Python
