The Data Engineer will develop and monitor BAU data pipelines, enhance processes, build new data pipelines, and contribute to the Data Lake technology in an Agile environment.
You will deliver
- Hands on development on Nexus Platform
- Monitoring daily BAU data pipelines and ensure our data solution is refreshed up to date every day
- Enhance the daily BAU process. Making it easier to monitor and less likely to fail and hands on development on Data Lake build, change and defect fix
- Building new data pipelines using existing frameworks and patterns
- Working with the team within an Agile framework using agile methodology tooling that controls our development and CI/CD release processes
- Contributing to the new Data Lake technology across the organisation to address a broad set of use cases across data science and data warehousing
Skills and Experience
Essential
- Experience with data solution BAU processes (ETL, table refresh etc.)
- Experience with integration of data from multiple data sources
- Experience in Big Data data integration technologies such as Spark, Scala, Kafka
- Experience in programming language such as Python or Scala.
Experience using AWS, DBT and Snowflake. - Analytical and problem-solving skills, applied to data solution
- Experience of CI/CD
- Good aptitude in multi-threading and concurrency concepts
- Familiarity with the fundamentals of Linux scripting language
Desirable
- Experience of ETL technologies
- AWS exposure (Athena, Glue, EMR, Step functions)
- Experience of Snowflake and DBT
- Experience with data solution BAU processes (ETL, table refresh etc
- Previous proficiency with ETL technologies (e.g. Talend, Informatica, Abinitio)
- Previous exposure to Python
- Previous exposure to own data solution BAU monitoring and enhancement
- Exposure to building applications for a cloud environment
#LI-Hybrid
#LI-TM1
At NewDay, we value all types of diversity. We’re an equal opportunity employer and believe that our differences create a vibrant, authentic working culture. We want all our colleagues to feel able to bring their whole selves to work. We don’t discriminate on the basis of protected characteristics or identities. We make sure that every job is crafted to be inclusive and that people with disabilities or caring responsibilities can take part in the application and interview process.
Tell us if you need accommodations: We’ll put reasonable adjustments in place to support you.
We work with Textio to make our job design and hiring inclusive.
PermanentTop Skills
AWS
Dbt
ETL
Kafka
Linux
Nexus Platform
Python
Scala
Snowflake
Spark
NewDay London, England Office
7 Handyside Street, London, United Kingdom, N1C 4DA
Similar Jobs
Artificial Intelligence • Information Technology • Natural Language Processing • Software • Business Intelligence • Generative AI
The role entails leading POC solutions, collaborating with teams to drive customer success through technical demonstrations, and enhancing sales engineering processes.
Top Skills:
SaaS
Financial Services
As a Senior Lead Data Engineer, you'll enhance data solutions, mentor teams, drive innovation, and ensure data asset security within an agile environment.
Top Skills:
Ci/Cd PipelinesDatabricksIcebergNosql DatabasesParquetRelational Databases
Financial Services
As a Data Engineer II, you will enhance and deliver data solutions, manage data lifecycle, and execute troubleshooting on technical systems.
Top Skills:
Etl/EltGrafanaNoSQLPrometheusSQL
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
