Design, build, and operate cloud-native lakehouse and streaming data platforms and pipelines using Python, PySpark, dbt, Airflow, and orchestration/infrastructure tooling. Ensure performance, reliability, security, and observability; model and transform data for analytics; deploy with IaC, containers, and Kubernetes; mentor engineers and drive platform standards and best practices.
Shape how hundreds of thousands of UK investors use data to make confident, informed investment decisions. Join a team building modern, cloud-native data platforms that enable analytics, regulatory reporting, and data-driven products at scale. You’ll work with contemporary lakehouse and streaming patterns, strong engineering practices, and a culture that values ownership and continuous improvement. This role offers meaningful scope to influence platform standards and mentor others while growing your technical and leadership impact.
As a Lead Data Engineer at JPMorgan Chase within Personal Investing, you will design, build, and operate a robust cloud-native data platform and pipelines that power analytics, regulatory reporting, and data-promoten applications. You will help us deliver reliable, scalable, observable, and secure data solutions by applying strong software engineering fundamentals and modern data engineering patterns. You’ll work closely with partners across product, analytics, and engineering to translate business needs into resilient technical designs. You’ll also contribute to engineering excellence through best practices, mentoring, and thoughtful technical direction.
Job responsibilities
- Design scalable, reusable data processing and data quality frameworks using Python, PySpark, and dbt
- Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability
- Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations
- Model and transform data for analytics using SQL and dbt to support business intelligence and reporting workloads
- Write production-grade Python/PySpark code with disciplined testing, performance tuning, and maintainable object-oriented design
- Implement infrastructure-as-code (e.g., Terraform) to provision and manage cloud-based data platform components
- Containerize and deploy services using Docker and Kubernetes (and related tooling such as Helm)
- Collaborate with analysts, data scientists, and application teams to turn requirements into technical designs and delivered solutions
- Own critical data systems by improving reliability, scalability, security, and operational excellence
- Mentor junior engineers and influence the team’s technical direction through standards, reviews, and knowledge sharing
- Uses enterprise-authorized AI capabilities within the work environment to accelerate data platform and model design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements
- Applies reuse-first, AI-assisted practices within delivery and operational routines (e.g., backup/recovery validation and access control review support), ensuring traceability/auditability and alignment to resiliency and security expectations
Required qualifications, capabilities, and skills
- Degree in Computer Science or a STEM-related field (or equivalent)
- Demonstrated experience delivering in an agile, fast-paced engineering environment
- 8 years of recent, hands-on professional experience actively coding as a data engineer
- Strong software engineering fundamentals (system design, data structures, object-oriented programming, testing strategies, and end-to-end development lifecycle)
- Strong Python programming skills, including unit and integration testing
- Hands-on experience building and operating cloud-based data platforms using major cloud services (e.g., AWS, Google Cloud, or Azure)
- Experience with large-scale distributed data processing and performance tuning
- Hands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar)
- Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt)
- Experience designing and operating orchestration pipelines using Airflow or similar tools
- Experience designing and building streaming pipelines using Kafka, Pub/Sub, or similar messaging systems
- Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity
- Ability to review and validate AI-assisted outputs (e.g., model/design summaries or operational checklists) before use, escalating when uncertain and following data handling requirements
Preferred qualifications, capabilities, and skills
- Data modeling experience for analytics and reporting use cases
- Knowledge of security, risk, compliance, and governance considerations for data platforms
- Experience building continuous integration and continuous delivery automation for data and platform services
- Experience with container-based deployment environments (Docker, Kubernetes, etc.)
- Demonstrated ability to coach teammates on engineering practices and contribute to a collaborative, inclusive team culture
About UsJ.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent corporations, governments, wealthy individuals and institutional investors. Our first-class business in a first-class way approach to serving clients drives everything we do. We strive to build trusted, long-term partnerships to help our clients achieve their business objectives.
We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.
JPMorganChase London, England Office
25 Bank Street, Canary Wharf, London, United Kingdom, E14 5JP
Similar Jobs
Financial Services
Develop and maintain secure, scalable data pipelines, architectures, and models across the data lifecycle. Review enterprise data controls, customize tools for business outputs, and apply advanced SQL and NoSQL knowledge. Use authorized AI capabilities to support pipeline design, testing, documentation, and analysis while validating outputs, protecting sensitive data, and ensuring auditability, resiliency, and security.
Top Skills:
AgileAi-Assisted Data EngineeringData ModelingData PipelinesNosql DatabasesSdlcSQLStatistical Data Analysis
Financial Services
Lead architecture and engineering for B2B agentic commerce and machine-learning systems in global payments. Define multi-agent architectures, authorization, information barriers, MLOps pipelines, evaluation frameworks, safety guardrails, and model-risk evidence. Develop and review secure production code, establish AI-assisted engineering standards, integrate external agents, improve operational resilience, and guide technical teams and vendors. The role requires advanced Python, production LLM agent experience, secure distributed-systems expertise, and financial-services technology knowledge.
Top Skills:
A2AAg-UiCi/CdDatabricksDistributed SystemsGoogle AdkGpu InferenceGraphragIdentity And AuthorizationKnowledge GraphsKubernetesLanggraphLlm AgentsMcpMicro-Vm IsolationMl PlatformsMlflowMlopsMulti-Agent SystemsOpa/RegoOpenfgaPythonService Mesh
Financial Services
Leads development of secure, scalable data platforms supporting batch, streaming, ML, and LLM workloads. Designs distributed data systems using Spark, Ray, Kafka, Iceberg, and OpenSearch; productionizes model training, deployment, monitoring, and lifecycle management; and builds governed LLM solutions. Provides technical leadership, code review, incident response, vendor evaluation, responsible AI guidance, automation, and mentorship across engineering teams.
Top Skills:
Ci/CdCloud ComputingContainersDatabasesDistributed SystemsIcebergInfrastructure As CodeJavaKafkaKotlinKubernetesLarge Language ModelsMachine LearningMcpObservabilityOpensearchPythonRagRayScalaSparkVector Search
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
