Top Data Engineer Jobs in London

Design, build, and operate enterprise-grade, cloud-native data platforms and pipelines for Preqin's Company Intelligence. Provide technical architecture, end-to-end delivery, data governance, observability, and reliability while leading and mentoring a small team and partnering with product and analytics to deliver measurable business value.

Develop secure, scalable backend and data solutions using Java, Python, Spring Boot, Databricks, Kafka, and cloud technologies. Responsibilities include designing microservices and event-driven architectures, integrating SQL and NoSQL databases, building data workflows, writing automated tests, reviewing code, troubleshooting production systems, and improving operational stability through automation. The role also contributes to agile practices, AI-assisted development, communities of practice, and enterprise software engineering standards.

Designs and operates production-grade data pipelines and governed data products for technology resiliency and recovery-risk analytics. Responsibilities include data modeling, integration, lineage, quality controls, metadata management, distributed systems, database optimization, and scalable architecture. Partners with engineering, cybersecurity, controls, architects, analysts, and business stakeholders to translate requirements into technical solutions and deliver measurable operational outcomes.

Develop, operate, and maintain low-latency infrastructure that processes and redistributes market data and orders in live production. Partner with trading desks, exchanges, counterparties, and data providers to minimize outages, support trading, and ensure regulatory compliance. Responsibilities include production support, infrastructure reliability, performance analysis, and systems operation across multiple asset classes.

Design and maintain high-volume data pipelines, analytical stores, and data models supporting global KYC and customer onboarding. Modernize legacy verification data infrastructure using Python and Airflow, optimize queries and workflows, and ensure data quality, observability, auditability, and regulatory transparency. Collaborate with software engineers, data scientists, product analysts, and compliance stakeholders to define data contracts, schemas, monitoring standards, and automated testing frameworks.

Design, implement, operate, and troubleshoot CrowdStrike’s global multi-vendor, multi-cloud, hybrid data center networks. Build cloud connectivity and network constructs, develop infrastructure-as-code and automation using Ansible, Terraform, Python, and Go, improve resilience and deployment efficiency, support incident analysis and on-call rotations, and ensure compliance with regional data sovereignty and security requirements.

Build and maintain reliable Finance data pipelines connecting warehouses, CTRM platforms, brokers, FCMs, and financial systems. Automate workflows, implement reconciliations and controls, process external feeds, investigate P&L and cash discrepancies, and develop serverless AWS ETL solutions using Python, SQL, infrastructure as code, and CI/CD. Collaborate with Finance and technology stakeholders to improve data reliability, observability, and quality.

Design, build, operate, and evolve scalable, reliable, observable data lake and catalog platform services. Lead technical design, develop well-tested software, support SLOs and incident learning, collaborate with stakeholders, create implementation plans, mentor engineers, and contribute to platform architecture and open-source initiatives.

Design and deliver scalable data solutions on Azure, collaborate through code reviews and pairing, support data and telemetry understanding, conduct proof-of-concept work, and build reliable, tested, maintainable systems with CI/CD. The role contributes to engineering standards, agile delivery, data modeling, performance, cost optimization, and adoption of emerging data technologies.

Build and maintain scalable data pipelines, Snowflake data models, integrations, automation, and data quality controls for DK and PRH. Support BI reporting with Power BI and Microsoft Fabric, manage CI/CD, monitoring, documentation, governance, and incident resolution. Collaborate with technical and business stakeholders to translate requirements into secure, reliable data solutions, while contributing to technical reviews, continuous improvement, and data engineering best practices.

Designs, builds, and maintains scalable data pipelines and reliable data products supporting analytics, marketing optimization, econometric modeling, and regulatory reporting. Responsibilities include data quality, testing, monitoring, observability, solution architecture, cloud data platforms, workflow orchestration, stakeholder collaboration, and continuous improvement. The role partners with analysts, data scientists, marketing teams, and engineering groups to translate business requirements into resilient technical solutions.

Design, build, test, document, and support production data pipelines and data products on Snowflake using dbt Cloud. Develop dimensional models, enforce data quality and governance, optimize performance and cost, and support CI/CD and code reviews. Partner with underwriting, actuarial, operations, and engineering stakeholders to refine requirements, manage delivery expectations, and contribute to agile planning, platform standards, and the data product roadmap.

Leads data and integration engineering for digital product profile data across R&D, CMC, manufacturing, and commercial systems. Designs data models, pipelines, integrations, CI/CD processes, deployment environments, and knowledge-graph implementations. Supports AI and analytics initiatives, monitors data quality and system performance, troubleshoots recurring issues, documents technical standards, and coaches engineers and analysts. Ensures solutions meet security, privacy, GxP, validation, and change-control requirements.

Maintains and evolves real-time market data platforms, integrating, normalizing, processing, and distributing data from exchanges, vendors, and internal systems. Troubleshoots production issues, optimizes low-latency and high-throughput performance, develops automation, supports upgrades and deployments, and improves monitoring, resilience, capacity, and scalability. The role also involves stakeholder and vendor collaboration, compliance with market data licensing and exchange requirements, documentation, and occasional support outside standard business hours.

Build and maintain reliable data pipelines, transformations, data models, and supporting processes for business intelligence, reporting, analytics, and AI use cases. The role includes testing, deployment, code reviews, data-quality troubleshooting, documentation, and collaboration within agile cross-functional teams. Candidates should have strong SQL and Python skills, production pipeline experience, knowledge of data warehousing and data quality, cloud-platform familiarity, and understanding of CI/CD, version control, and testing practices.

Designs, builds, and improves scalable data ingestion and transformation pipelines using Python, SQL, dbt, Airflow, Spark, AWS, and Snowflake. Owns data lake and platform reliability, ETL processing, table refreshes, defect resolution, performance optimization, automation, and CI/CD delivery. The role also requires data modeling expertise, complex data problem-solving, and leadership through coaching and technical direction of Data Engineers.

Build and maintain data-driven applications, pipelines, and unified data models for reinsurance data. Improve data quality, structure, accessibility, scalability, and transformation processes while addressing legacy and decentralized systems. Collaborate with actuaries, analysts, business leaders, and technical teams to translate requirements into practical solutions. Explain technical concepts, challenge existing approaches, support product and project delivery, and contribute to agile data engineering initiatives.

Designs, builds, and optimizes scalable data pipelines integrating social, advertising, audience, platform, content, performance, and revenue data. Owns YouTube and Meta pipelines, ensuring reconciled finance-grade outputs. Develops analytics-ready datasets with BigQuery and dbt, maintains data quality, governance, monitoring, documentation, and security, and supports BI reporting. Collaborates with analysts and stakeholders to model data, automate processes, resolve issues, and adapt pipelines to changing APIs and schemas.

Design, provision, and operate a scalable, secure cloud data platform on AWS and Databricks using Terraform, automation (Python/Java), CI/CD, observability, and security best practices to support production analytics workloads.

Design, build, and optimize scalable data pipelines and processing systems for sensitive clinical and genomic data. Develop AWS-based data solutions, models, storage, orchestration, quality controls, testing, automation, and observability. Support secure data acquisition, transformation, lineage, metadata, de-identification, and delivery while collaborating with engineering and specialist teams. Enable research, machine learning, and visualization through reliable data products and contribute to organization-wide data engineering practices.

Build and operate AWS-based data pipelines, datasets, and analytics applications for business stakeholders. Integrate and validate internal and external data, develop BI views, maintain infrastructure through code and CI/CD, and contribute to LLM-powered projects. The role also involves documenting systems, using agentic coding tools, and translating ambiguous business questions into reliable data products and insights.

Lead architecture and engineering for B2B agentic commerce and machine-learning systems in global payments. Define multi-agent architectures, authorization, information barriers, MLOps pipelines, evaluation frameworks, safety guardrails, and model-risk evidence. Develop and review secure production code, establish AI-assisted engineering standards, integrate external agents, improve operational resilience, and guide technical teams and vendors. The role requires advanced Python, production LLM agent experience, secure distributed-systems expertise, and financial-services technology knowledge.

Maintains Cloudflare’s global data center network infrastructure across remote colocation sites. Responsibilities include configuring network equipment, ensuring operational availability, managing contractors and vendors, creating SOPs and operational plans, supporting deployments and migrations, improving processes through automation and AI tools, troubleshooting infrastructure, and participating in a 24/7 on-call rotation. The role also involves basic program management, documentation, status reporting, and limited travel.

Build and operate production data pipelines for patent, research, and company data using Python and Dagster. Model messy real-world data, develop agent evaluation harnesses, implement data quality monitoring, and deliver reliable data to product stores and APIs. Collaborate with product and ML teams, evaluate new data sources, troubleshoot source failures, and define measurable standards for data quality. The role involves working with large, noisy datasets and shaping how AI agents support data processing.

Design, develop, and maintain scalable data pipelines and architectures for analytics and reporting. Perform ETL and streaming data integration, data modeling, governance, quality management, and workflow optimization across cloud and big data environments. Translate business requirements into data solutions, monitor reliability and performance, and collaborate with stakeholders to advance Walmart’s data strategy.