What’s the challenge?
We have an exciting opportunity to join us at comparethemarket.com as a Senior Data Engineer. Your role will be pivotal in delivering and owning high performance data applications using cutting edge technology.
We are split into two teams, our Platform team that focuses on operational excellence and supporting our extensive data platform and our Streaming team that supports our bespoke Streaming platform built in Scala as well as a partner integration service built in Scala to process data received from them. As a Senior Engineer in our team, you will be driving the development of quality solutions for our customers and be proactive in adopting and championing best practices.
You’ll make a real impact by taking an active role in the team’s agile practices, technical decision making and development, generating value and continuously striving to improve the quality and reliability of our data.
What will it take to be a successful Senior Data Engineer?
Strong experience in Data Engineering - building strong, robust pipelines to ingest data from a variety of sources with a focus on real-time streaming (we process millions of events a day and deal with 100 TB’s of data here)
Experience in Scala, Kafka (Confluent platform if possible), AWS & Terraform
Demonstrable experience for modern software engineering principles (Agile, CI/CD, TDD/BDD)
Active learner, keeps up to date with industry knowledge and keen to learn new languages and tools
Self-motivated, natural leader and motivator. Passionate about Technology
Great communicator, influencer and negotiator
Ability to use your wealth of experience to find the right solution for any given task
Java (we would consider a strong Java Data Engineer who has a keen interest in learning about functional programming and building pipelines in Scala)
Kafka Streams, Kafka Connectors, KSQL
Why choose us?
You might already know us - we're the ones with the Meerkats! Compare the Market has helped revolutionise the insurance market and we have been growing rapidly since starting in 2006.
We are on a mission to transform the comparison industry, moving beyond endless forms and generic search results, to a highly personalised, service-led experience. In order to do so, we recognise that we need to consistently deliver seamless, personalised customer experiences through a connected data and technology ecosystem.
Diversity and Inclusion
We're a diverse community of dedicated, innovative and talented professionals. With an inclusive and open workplace, we encourage our people to create and share ideas – supporting their growth and celebrating their uniqueness.
We understand the importance of achieving a healthy lifestyle balance, whether it’s working remotely or flexibly we have an environment where people are empowered to embrace flexible working in a way that works for them and for the business.
'We are committed to offering equal chances to you throughout the application process, so if you require any special adjustments to be made, please let us know. We will work with you to make any appropriate arrangements.’