Senior Big Data Engineer (AI Enthusiast)

At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for jobseekers and companies around the world to find a perfect match in fair and equitable way. With over 20 brands across 30+ countries, we strive for fair and unbiased hiring. Join our team of 4,000+ employees and be part of reshaping the labour market and become the world's leading job platform.

As a Senior Big Data Engineer and AI Enthusiast, you will join the Marketplace Enablement department, working inside a cross-functional team to design and implement cutting edge, high-performance central streaming data that power our recruitment AI applications, including deploying advanced machine learning models. Your expertise will help create a more inclusive and efficient recruitment experience globallyly, supporting our mission to reshape the labour market for everyone. You will have a strong sense of ownership and proactively provide solutions before a situation deteriorates.

Your Responsibilities Lead the design and build of modern big data platforms and AI-powered tools that promote fair and inclusive hiring. Champion code quality by applying best practices like clean code, automated testing, CI/CD, observability, security, and Federated Governance. Design, implement, and optimize high-performance data pipelines and real-time applications tools like Apache Kafka and Apache Spark. Manage and optimize complex ETL/ELT processes and data orchestration workflows. Continuously measured, improved, and scale our data infrastructure, advocating for new technologies and architectural patterns. Collaborate and mentor in an international, agile, cross-functional team, helping others grow and succeed. Contribute to building an inclusive team culture where every voice is valued. Work in a collaborative environment that values diverse perspectives and inclusive decision-making.

Your Skills and Qualifications Excellent communication skills in spoken and written English (B2+ level). Genuine passion for leveraging data and AI to solve complex problems and make a tangible impact. Deep expertise and proven track record in Python for building large-scale data and backend applications (5+ years professional experience). Strong experience with AWS cloud services (e.g., EMR, Glue, ECS with Docker, Redshift, DynamoDB, S3) and understanding of cloud-native architectures (3+ years). Professional experience in Big Data architecture, Data Warehousing, designing complex ETL/ELT pipelines, and data pipeline orchestration (2+ years). Solid experience with Apache Kafka (MSK) for real-time data ingestion and processing (1+ years). Familiarity with Infrastructure as Code (Terraform) and Huggingface Transformers (considered a plus). We value different career paths — if you have career breaks, switched disciplines, or balanced work with family life, we still encourage you to apply. If you don't meet every single requirement but are excited about this role, we still would love to see you applying :)

Your benefits We're a community here that cares as much about your life outside work as how you feel when you're with us. Because your job shouldn't take over your life, it should enrich it. Here are some of the benefits we offer: 30 days of holidays + 2 extra days for Christmas and New Year's Eve

Hybrid working model

Company pension scheme

24/7 Employee Assistance Programs

Life Assurance Cover

Volunteering days

Job bike

Discounted parking or job ticket

In-house gym

In-house barista

And you can bring your dog to the office

Our commitment Equal opportunities are important to us. We believe that diversity and inclusion at The Stepstone Group are critical to our success as a global company, so we want to recruit, develop, and keep the best talent. We encourage applications from everyone, gender identity, sexual orientation, disability status, ethnicity, supply, age, family parental status, and any other characteristic. JOBV1_EN