Big Data Platform Engineer

Data Science

Skills

Description

As a leading telecommunications company with its two feet in the digitization of our society, we are eager to bring aboard Big Data professionals that share our passion for Big Data infrastructures and processing. It's our day to day mission to work together with our customers to provide a reliable & flexible platform as a foundation to host new applications and products. In your role as a Big Data Platform Engineer, you'll be part of a team that sits in a driving seat to define and wire all data components together. It is exactly this diversity of technologies and stakeholders what makes this challenging and exciting. If this sounds like a nice proposition, here's what you should have in store for us. Hope to welcome you in our dynamic team full of opportunities to help you grow in your job. JOB DESCRIPTION *Mid/senior level Big Data Platform Engineer *As a platform engineer, you are responsible for the engineering, integration and maintenance of the big data platform. The focus is on technology/infrastructure services, not the usage and development methodologies that use the Big Data platform. The latter is handled by data engineering teams with which you will be working closely. *You help to design the platform and the required services with focus on security, maintainability, flexibility, efficiency in a cost-efficient manner. *You work together with architects and analysts to come to a proper architectural design and solution *You ensure that the services are correctly deployed and maintained across the various quality environments, and that this maintenance and support can be done by operations team. *To accomplish this, you develop the right set of documentation (user, administration and expert documentation) and ensure handover and information sessions to colleagues as well as to core stakeholders. *Most importantly, you embrace the challenges related to container platform, hybrid cloud, multi cloud etc. A strong will to learn new technologies, services and concepts. You can swiftly adapt based on changing requirements and dare to challenge previous decisions. *Have an analytical mindset to select and apply the right technological service to solve problems. You can decompose complex problems and move on to simple, straight-forward solutions. *You will be expected to strive for excellence, continuously improving services and their behavior (including performance), but also to continuously deliver new functionalities. As such, you will need to prioritize workload and tasks and assist product owners and other stakeholders to make the right decision. *You are expected to work with a high degree of autonomy. While this increases efficiency, it also requires the team to know when to seek guidance and learn more from requirements (policies and standards) and processes. Self-direction and self-reflection are vital skills to have. *The platform has an affinity with open sources software components, and how to responsibly integrate and use those technologies within the broader enterprise context. Security aspects and manageability concerns are a continuous focal point. *You are expected to work with operational teams to investigate/solve the L3 incidents along with the vendors Experience *Data integration/movement platform specialist with experience in performing following activities: *Setting up E2E Kafka environments (Brokers, Connect, SchemaRegistry, Kerberos, mTLS, MirroMaker) on the cloud (AWS) *Setting up monitoring(Lenses, Prometheus, Grafana, ELK) *Setting up security(SASL/GSSAPI, mTLS) *Preparing Kafka and Kafka components (Connect, SR, SQLProcessors) deployment on k8s *CI/CD for topics/acls/connectors *Setting up E2E NiFi environments on the cloud *Setting up monitoring *Setting up security *Preparing Kafka and Kafka components *CI/CD for topics/acls/connectors *Experience in deploying/managing CDC data integration is a plus Technologies: *Apache Kafka *Apache Nifi *Prometheus + Grafana *Ansible/Python *Terraform *Gitlab CI/CD *Oracle Golden Gate (added advantage) *AWS or GCP cloud experience All applicants must be eligible to work in Belgium via a visa or EU nationality