Specific responsibilities include

• Manage large scale multi-nodes Kafka cluster environments residing on AWS and on-prem OpenShift platform.
• Handle all Kafka environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring.
• Perform high-level, day-to-day operational maintenance, support, and upgrades for the Kafka Cluster.
• Creation of key performance metrics, measuring the utilization, performance and overall health of the cluster.
• Capacity planning and implementation of new/upgraded hardware and software releases as well as for storage infrastructure.
• Research and recommend innovative, and where possible, automate key administration tasks.
• Ability to closely collaborate with product managers and lead engineers.
• Provide guidance in the creation and modification of standards and procedures
• Create topics, setup redundancy cluster, deploy monitoring tools, alerts
• Proactively monitor and setup alerting mechanism for Kafka Cluster and supporting hardware to ensure system health and maximum availability
• Perform high-level, day-to-day operational maintenance, support, and upgrades for the Kafka cluster
• Provide technical expertise and guidance to production support.

Job Requirements

• Solid Kafka Admin experience in managing critical 24/7 applications
• Design, build, assemble, and configure application or technical architecture components using business requirements.
• Hands-on experience with Kafka clusters hosted on ROSA in AWS and on-prem K8 Open Shift platform
• Experience in building Kafka pipelines using Terraform, Ansible, Cloud formation templates, shells etc.
• Experience in implementing security & authorization (permission based) on Kafka cluster.
• Aid System Administrators with setting up Kafka platform in provisioning, access lists Kerberos and SSL configurations.
• Experience in setting standards to automate deployments using Kubernetes, Docker, Chef or Jenkins
• Experience in open source and confluence Kafka, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center.
• Experience in Kafka Mirror Maker or Confluent Replicator
• Experience in High availability cluster setup, maintenance and ongoing 24/7 support
• Hands on experience in standing up and administrating Kafka platform from scratch which includes creating a backup & mirroring of Kafka Cluster brokers, broker sizing, topic sizing, h/w sizing, performance monitoring, broker security, topic security, consumer/producer access management (ACL)
• Knowledge of Kafka API (development experience is a plus)
• Knowledge of best practices related to security, performance, and disaster recovery.
• Ability to concentrate on a wide range of loosely defined complex situations, which require creativity and originality, where guidance and counsel may be unavailable.