The Data Engineer will develop and monitor BAU data pipelines, enhance processes, build new data pipelines, and contribute to the Data Lake technology in an Agile environment.
You will deliver
- Hands on development on Nexus Platform
- Monitoring daily BAU data pipelines and ensure our data solution is refreshed up to date every day
- Enhance the daily BAU process. Making it easier to monitor and less likely to fail and hands on development on Data Lake build, change and defect fix
- Building new data pipelines using existing frameworks and patterns
- Working with the team within an Agile framework using agile methodology tooling that controls our development and CI/CD release processes
- Contributing to the new Data Lake technology across the organisation to address a broad set of use cases across data science and data warehousing
Skills and Experience
Essential
- Experience with data solution BAU processes (ETL, table refresh etc.)
- Experience with integration of data from multiple data sources
- Experience in Big Data data integration technologies such as Spark, Scala, Kafka
- Experience in programming language such as Python or Scala.
Experience using AWS, DBT and Snowflake. - Analytical and problem-solving skills, applied to data solution
- Experience of CI/CD
- Good aptitude in multi-threading and concurrency concepts
- Familiarity with the fundamentals of Linux scripting language
Desirable
- Experience of ETL technologies
- AWS exposure (Athena, Glue, EMR, Step functions)
- Experience of Snowflake and DBT
- Experience with data solution BAU processes (ETL, table refresh etc
- Previous proficiency with ETL technologies (e.g. Talend, Informatica, Abinitio)
- Previous exposure to Python
- Previous exposure to own data solution BAU monitoring and enhancement
- Exposure to building applications for a cloud environment
#LI-Hybrid
#LI-TM1
We work with Textio to make our job design and hiring inclusive.
PermanentTop Skills
AWS
Dbt
ETL
Kafka
Linux
Nexus Platform
Python
Scala
Snowflake
Spark
NewDay London, England Office
7 Handyside Street, London, United Kingdom, N1C 4DA
Similar Jobs
Artificial Intelligence • Machine Learning • Software
The Senior Implementations Data Engineer will design, develop, and maintain data pipelines, ensure data security, and support customer integrations, working with cross-functional teams and stakeholders.
Top Skills:
AWSAzureDatabricksDockerGitKubernetesOpensearchPostgresPythonRest ApisSparkSQL
Financial Services
Lead development of strategic data pipelines and software solutions for Credit Trading. Collaborate with stakeholders to enhance operational stability, promote innovation, and ensure technology meets business needs.
Top Skills:
AWSAzureGCPKdb+OraclePostgresPythonVertica
Fintech • Information Technology • Financial Services
As a Vice President, you will lead Preqin's Data Engineering/BI team, develop data infrastructure, ensure stakeholder alignment, and promote data best practices. You will also drive the adoption of advanced analytics and AI/ML solutions.
Top Skills:
AWSAzureDbtKubernetesPythonSnowflakeSQLTerraform
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
