Build and maintain market, economic, research, news, and internal data pipelines for AI and quantitative models. Design scalable data architectures, implement data quality controls, manage metadata and lineage, create APIs and model-ready datasets, and optimize pipeline performance. Collaborate with data scientists and analysts while ensuring data security, privacy, permissions, and reliable integration across tools and models.
As a Data Engineer, you will play a pivotal role in constructing the data foundation for our AI and quant model initiatives. Your responsibilities will encompass managing diverse data pipelines, from market and economic data to internal and news feeds. Additionally, you will be tasked with creating and maintaining a robust data architecture, ensuring data quality, and developing metadata systems. Your work will directly impact the performance and reliability of our client tools and quant models.
Responsibilities
- Build and maintain market, economic, research, news, and internal data pipelines, as well as Fenics/BGC data integration.
- Design and implement a scalable bronze/silver/gold data architecture to support our AI and quant models.
- Establish and monitor data quality gates to ensure data integrity and accuracy.
- Develop and manage instrument, ticker, country, client, source, and theme metadata for effective data categorization and retrieval.
- Create APIs and tables for seamless integration with agent tools, dashboards, and quant models.
- Implement data lineage tracking, user entitlements, and permissions management systems.
- Monitor and optimize data pipelines for performance and efficiency, ensuring smooth data flow.
- Collaborate with data scientists and analysts to understand their data needs and provide model-ready datasets.
- Stay updated with the latest trends in time-series and complex-data analysis, incorporating best practices into our data platform.
- Ensure data security and privacy compliance, adhering to industry standards and regulations.
- Strong proficiency in Python and SQL, with experience in production-level ETL/ELT processes.
- In-depth knowledge of cloud data stack technologies, such as Azure, Databricks, Delta Lake, or Snowflake.
- Experience with time-series data and complex data structures, demonstrating an understanding of data quality and monitoring.
- Ability to clean and transform messy data into structured, model-ready datasets, ensuring data integrity.
- Familiarity with ticker-level data processing, having worked with kdb+ or similar systems is an advantage.
- Excellent problem-solving and analytical skills, with a keen eye for detail and data accuracy.
- Strong communication and collaboration skills, able to work effectively with cross-functional teams.
- A proactive and self-motivated approach to work, with a passion for data engineering and continuous improvement.
- A bachelor's degree in Computer Science, Engineering, or a related field is preferred.
- 2+ years of relevant work experience in data engineering or a similar role.
Cantor Fitzgerald London, England Office
London, United Kingdom, 0
Similar Jobs
Artificial Intelligence • Machine Learning • Software
Design, build, deploy, and optimize distributed big-data ingestion, standardization, and ML pipelines at scale. Support platform reliability, cost controls, security accreditations, and customer integrations. Serve as technical lead in client meetings, mentor engineers, and recommend tools and best practices for data pipeline development and deployment.
Top Skills:
AWSAzureDatabricksDatadogDockerEc2GCPGitIamKubernetesOpensearchPostgresPythonRest ApisS3SparkSQLTerraform
Digital Media • Gaming • Software • Esports • Automation
Build and maintain near-real-time and batch regulatory reporting systems. Migrate legacy SQL Server reporting to GCP BigQuery, design cloud-native data models and ETL/ELT pipelines, implement validation, monitoring, CI/CD and IaC, and apply AI-assisted tooling to improve accuracy, traceability and performance for regulatory submissions.
Top Skills:
Ci/CdClaude CodeCloud ComposerCloud FunctionsCloud StorageEltETLGCPGitGitlabGoogle BigqueryMicrosoft Sql ServerPub/SubSQLTerraform
Digital Media • Gaming • Software • Esports • Automation
Build and maintain near-real-time and batch regulatory reporting systems. Migrate legacy SQL Server solutions to GCP BigQuery, design cloud-native data models and ETL/ELT pipelines, implement validations, monitoring, CI/CD and IaC, apply AI-assisted tooling, perform QA and documentation, and collaborate with senior engineers to ensure accurate, traceable regulator-ready outputs.
Top Skills:
Claude CodeGCPGitGitlabGoogle BigqueryGoogle Cloud ComposerGoogle Cloud FunctionsGoogle Cloud StorageGoogle Pub/SubMicrosoft Sql ServerSQLTerraform
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
