Top Data Engineer Jobs in London

Build and maintain market, economic, research, news, and internal data pipelines for AI and quantitative models. Design scalable data architectures, implement data quality controls, manage metadata and lineage, create APIs and model-ready datasets, and optimize pipeline performance. Collaborate with data scientists and analysts while ensuring data security, privacy, permissions, and reliable integration across tools and models.

Build, maintain, and troubleshoot scalable AWS data pipelines using Python, PySpark, Spark, and Airflow. Perform root-cause analysis of pipeline and data-quality issues, apply dimensional modeling and slowly changing dimensions, and deliver governed data assets. Provision infrastructure with Terraform, containerize solutions with Docker, and manage GitLab CI/CD deployments. Work with AWS security, encryption, IAM, monitoring, and analytics services in a regulated government environment.

The Data Engineer will implement cloud-native data platforms, engineer scalable pipelines, and optimize cloud resources using Python and SQL, while applying agile methodologies and cloud automation tools.

Design and operate production data pipelines and standardized data models for a multi-tenant Azure SaaS platform serving container terminals. Build near-real-time and batch ELT pipelines from operational systems, develop Go-based connectors, implement data quality monitoring, and support source-to-curated transformations across PostgreSQL layers. Collaborate directly with terminal operations staff while owning architectural decisions and platform scalability.

Designs and evolves scalable data pipeline, orchestration, and inference infrastructure for autonomous driving data platforms. Responsibilities include developing DAG-based workflow systems, tooling, shared libraries, batch inference capabilities, and high-throughput processing workflows. The role focuses on reliability, performance optimization, resource prioritization, incident response, and collaboration with partner teams to support large-scale data ingestion, processing, evaluation, and machine learning operations.

As a Senior Data Science Engineer, you'll lead data science projects, develop algorithms, analyze datasets, and mentor junior team members to enhance trading operations and strategies.

Lead and contribute to building scalable, production-grade data systems for private markets. Architect end-to-end data workflows supporting analytics and AI/ML, ensure observability, testing, and operational maturity, influence technical strategy, engage senior stakeholders, and manage and mentor software engineers while partnering with product, data science, and business teams.

Build and operate production data pipelines using Python, SQL, Airflow, Snowflake, and Databricks. Implement ingestion sources, DAGs, schemas, monitoring, and catalog registration; investigate and resolve pipeline issues; review code; trace data lineage; support data security, governance, and reverse ETL integrations; collaborate with stakeholders; and mentor junior data engineers.

Design, build, and own scalable data pipelines from ingestion through analytics and incentive payouts. Lead data platform architecture, including Snowflake, dbt, Iceberg, and S3 workflows. Establish data quality, lineage, observability, performance, cost, and SLA standards. Partner with engineering and analysts to support telematics accuracy and driver incentive reporting. Develop maintainable data models and transformation layers while using AI coding tools to improve development speed and quality.

Build and evolve CluePoints’ enterprise data platform using Microsoft Fabric and Azure. Responsibilities include migrating Azure SQL and ADF workloads, integrating SaaS and web data, developing scalable ingestion and transformation pipelines, maintaining Kimball dimensional models, and establishing CI/CD, data quality, lineage, security, and monitoring. The role partners with analytics leadership and supports trusted Power BI reporting across multiple business functions.

As a Data Engineer, you'll develop data pipelines, manage data integration and transformation, and support analytics and intelligence projects for government clients.

The Data Engineer will analyze, clean, and manage various datasets to assist traders at Jane Street, automating ETL processes and ensuring data quality.

Designs and delivers scalable data pipelines and Lakehouse architectures using Databricks, Spark, PySpark, and Delta Lake. The role covers ETL/ELT, data governance, security, quality, performance optimization, cloud integration, and CI/CD. It involves technical leadership across customer programmes, architectural decision-making, stakeholder collaboration, mentoring, and supporting reliable production data solutions.

Design, build, and maintain performant PostgreSQL datastores, scalable data pipelines, and production-ready Python HTTP APIs. Implement automated testing, ensure data quality, and work with researchers and clients to deliver data features for financial analysis.

Build and operate scalable data pipelines, web scraping systems, and ML infrastructure for training, evaluating, and serving speech AI models. Responsibilities include improving GPU utilization, scheduling, observability, data quality, production reliability, and distributed-system troubleshooting. The role also contributes to MLOps practices and platform roadmap development.

Build and improve Lendable’s data platform by delivering scalable, reliable, secure, and cost-effective data solutions. Partner with product, analytics, and engineering teams to enable fast analytics and insight generation. Responsibilities include developing data-intensive systems, applying modern data architectures, managing cloud infrastructure and infrastructure-as-code, supporting CI/CD, evaluating build-versus-buy decisions, and improving platform operations while working autonomously in an ambiguous environment.

Design, build, and optimize cloud-based data platform components: ingest and transform varied datasets, create metadata-driven pipelines and data APIs, support AI/ML infrastructure, implement monitoring and metrics, and improve data processes to enable analytics and operational systems.

Responsible for designing and maintaining data pipelines, collaborating with teams for data quality, and developing engineering solutions using various tools and technologies.

The Data Engineer role involves modeling data as an Analytics Engineer, improving data infrastructure, managing data ingestion, and delivering training.

Build and own scalable Azure data platforms, SQL Server and Databricks environments, end-to-end pipelines, and maintainable data models. Apply Kimball dimensional modeling, solve complex client data problems, challenge unclear requirements, and communicate technical concepts clearly to technical and non-technical stakeholders. The role emphasizes production reliability, client impact, continuous learning, and collaboration in a fully remote European data consultancy.

Owns the roadmap and delivery of enterprise AI platform capabilities, including LLM and MCP gateways, agent runtimes, access controls, audit logging, governance, cost attribution, and cloud AI integrations. Partners with engineering, SRE, security, data, and business technology teams to define scalable architecture, APIs, self-service builder experiences, and prioritized product requirements. Leads products from problem definition through launch, adoption, and iteration while evaluating enterprise AI market trends and build-versus-buy decisions.

Lead design and delivery of metadata, meta-modelling, and distributed systems. Build secure, high-quality production code, drive adoption of AI-assisted engineering, improve SDLC automation and cloud-native delivery, set validation and governance standards, mentor engineers, and influence product and technical decisions.

Build, test, optimize, and maintain ELT pipelines supporting analytics, reporting, and data products. Deploy config-driven workflows to Databricks across a multi-tenant environment, extend data models, support CI/CD and ingestion projects, monitor pipeline health, troubleshoot failures, and collaborate with data and product teams to deliver reliable, well-tested data.

Develop and maintain Python applications, services, data processing solutions, and scalable ETL/ELT pipelines. Integrate structured and unstructured data from APIs, databases, files, and streaming systems while ensuring quality, performance, reliability, and security. Collaborate with engineering and architecture teams, contribute to data platform design, implement testing and monitoring, and support technical documentation and engineering standards.

Build and steward Databricks lakehouse solutions across EMEA and APAC, including agentic workflows, AI governance, semantic models, data models, and automated ingestion pipelines. Troubleshoot production failures, optimize performance, resolve data quality issues, and improve reporting. The role requires advanced SQL, Python, PySpark, Databricks, Azure, data modeling, and systems thinking, with opportunities to work on geospatial processing, master data management, and AI-amplified engineering practices.