The Data Engineer will design and develop data pipelines, manage data processing solutions, ensure data quality, and collaborate with team members to create scalable data assets.
UBDS Group is seeking skilled Data Engineers to design, build, and optimise modern data platforms. This role is suited to engineers with strong experience in data pipelines, cloud technologies, and distributed data processing, who can deliver scalable, high-quality data solutions.
You will work as part of a multidisciplinary team to develop reliable data assets that support a range of operational and analytical use cases.
Key Responsibilities- Design, develop, and maintain data pipelines and transformation processes using modern data engineering tools
- Build and manage batch and real-time data processing solutions
- Develop curated, reusable data assets to support downstream analytics and reporting
- Ensure data quality, integrity, and consistency across multiple data sources
- Work with data ingested via APIs, file-based ingestion (e.g. SFTP), and event streams (e.g. Kafka)
- Optimise data pipelines for performance, scalability, and cost efficiency
- Contribute to the development of data platform components and reusable engineering patterns
- Work within established architecture and engineering standards
- Collaborate with DevOps engineers, testers, and architects to deliver end-to-end data solutions
- Support onboarding of new data sources and integration into existing platforms
- Contribute to knowledge sharing and continuous improvement within engineering teams
- AWS (primary platform)
- S3, EMR, Glue
- Kafka for event-driven data processing
- Python and PySpark for data engineering and transformation
- Terraform (Infrastructure as Code)
- GitHub (version control and CI/CD)
- Azure data platforms
- Databricks and/or Microsoft Fabric
- Multi-cloud architectures and tooling
- Experience in data engineering, building and maintaining data pipelines
- Strong hands-on experience with Python and PySpark
- Experience working with AWS data services (e.g. S3, EMR, Glue)
- Experience with event-driven architectures, particularly Kafka
- Understanding of batch and real-time data processing
- Experience working with large-scale datasets and distributed systems
- Familiarity with data quality, validation, and schema management
- Experience using Terraform and GitHub within a DevOps environment
- Ability to work within defined engineering standards and best practices
- Experience with Azure, Databricks, or Microsoft Fabric
- Experience in regulated environments
- Exposure to data migration from legacy systems
- Understanding of analytics, reporting, or data product use cases
- Experience working in collaborative, cross-functional teams
Top Skills
AWS
Emr
Git
Glue
Kafka
Pyspark
Python
S3
Terraform
UBDS Digital London, England Office
30 Moorgate, 6th Floor, London, United Kingdom, EC2R 6DA
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and build scalable, secure data pipelines (streaming and batch), drive CI/CD and automation, participate in cloud migration and architecture discussions, collaborate with clients, and mentor internal teams to deliver production-grade data solutions.
Top Skills:
AirflowAWSBigQueryCircleCIContainerisationEltETLGraph DatabasesHadoopJavaJenkinsKafkaKinesisPythonRedshiftScalaSnowflakeSparkSpark StreamingSQL
Artificial Intelligence • Consulting
The AWS Data Engineer will design, build, and operationalize enterprise data solutions on AWS, mentoring fellow engineers and consulting with clients on data architecture and migration.
Top Skills:
AirflowAWSDatabricksDbtDynamoDBEmrGlueJavaKinesisLambdaPythonRedshiftScalaSnowflakeSpark
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and build cloud-based data pipelines using AWS tools, collaborate with clients, ensure data governance, and mentor junior engineers.
Top Skills:
AWSCi/CdDevOpsHadoopJavaKafkaKinesisPythonScalaSparkSpark Streaming
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
