The Technical Lead Data Engineer will enhance the internal Data Platform by developing data pipelines, ensuring code quality, optimizing SQL databases, and managing cloud infrastructure. The role requires leadership in data engineering projects and collaboration across teams to ensure the effective use of technology and data analytics.
Our Client is a global investment advisor organization that utilizes advanced technologies and data analytics to improve the operational performance of their portfolio companies. Key objectives include delivering strong financial returns, offering diverse investment opportunities, enhancing portfolio company operations, and proactively managing risks. They have invested in proprietary technology and data-driven strategies to extract additional value.
Currently, they are expanding their data science team in London, seeking a Technical Lead Data Engineer to further develop their internal Data Platform.
Required Skills and Qualifications:
Must have:
- Python Development: Minimum 5 years of professional experience in production environments, emphasising performance optimisation and code quality.
- Ingestion and modelling:
- Experience with Python and orchestration tools like Airflow is beneficial.
- SQL Proficiency: Advanced knowledge of SQL:
- At least one of PostgreSQL, MySQL, MSSQL
- Ability to write complex queries and optimise database performance
- Utilise DPT tools.
- Infrastructure as Code: Experience with Terraform or equivalent tools to facilitate code-controlled infrastructure management.
- Cloud Experience: Over 3 years of hands-on experience with cloud providers, specifically:
- AWS or GCP is acceptable
- Azure is highly preferred due to project-specific requirements
- English – Upper-Intermediate or Advanced
-
Education
- Advanced degree (Master’s or Ph.D.) in Data Science, Statistics, Computer Science, Engineering, or a related field.
Nice to have:
- Data Pipeline Design: Strong understanding of designing robust and scalable data pipelines for large-scale applications.
- Version Control and CI/CD: Familiarity with Git-based workflows and continuous integration/deployment practices to ensure seamless code integration and deployment processes.
- Communication Skills: Ability to articulate complex technical concepts to technical and non-technical stakeholders alike.
Programming Languages
- Python
- PySpark
- DPT, SQLMesh (optional)
- PostgreSQL, MySQL, MSSQL
- Terraform
Tools and Technologies
- Git
- Azure DevOps
- JIRA
- Confluence
We offer:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
Top Skills
Python
Similar Jobs
Information Technology • Consulting
As a Middle Data Engineer, you will play a key role in enhancing the online car market by building and maintaining data pipelines, improving existing tools, and collaborating with senior engineering teams. Your expertise in data platforms and AWS services will drive significant improvements in user experience.
Top Skills:
Python
Artificial Intelligence
As a Data Platform Engineer, you will support the ML Platform Team in developing a high-scale compute and data platform, provision and expose data for researchers, implement workflow orchestrators, and ensure ML engineering best practices are followed.
Top Skills:
Python
Marketing Tech • Cryptocurrency
As a Senior Data Engineer at Auros, you will enhance and lead the management of the firm's trading data systems. Responsibilities include building high-throughput data architectures, developing real-time data collectors, analyzing data quality, and collaborating with traders to optimize data analysis requirements.
Top Skills:
Python
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
