The Senior Data Engineer will build and optimize data pipelines, collaborate with engineering teams, and ensure data quality and scalability.
At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio. We are currently looking for a Business Development Executive to join one of our clients' team. If you're looking for an exciting opportunity to grow in a innovative environment, this could be the perfect fit for you.
We are looking for a skilled Senior Data Engineer. This role, you will be responsible for building scalable data pipelines, working with large datasets, and optimizing data workflows in a cloud-based environment. You will collaborate closely with data and engineering teams to ensure efficient and reliable data processing.
Key Responsibilities
- Design, build, and maintain robust data pipelines (ETL/ELT)
- Work with Databricks for large-scale data processing using Spark and Delta Lake
- Optimize data workflows, performance, and data storage solutions
- Collaborate with cross-functional teams including data analysts, engineers, and stakeholders
- Ensure data quality, reliability, and scalability across systems
- Contribute to continuous improvement of data architecture and processes
Job requirements
- Strong experience with Databricks (Spark / Delta Lake)
- Proficiency in Python for data processing
- Strong knowledge of SQL
- Hands-on experience with ETL/ELT pipeline development
- Experience with at least one cloud platform (AWS, Azure, or GCP)
- Familiarity with Git and basic CI/CD practices
Nice to Have
- Experience with dbt
- Knowledge of streaming technologies (e.g., Kafka or similar)
- Experience with Terraform or Infrastructure as Code (IaC)
Top Skills
AWS
Azure
Ci/Cd
Databricks
Dbt
Delta Lake
GCP
Git
Kafka
Python
Spark
SQL
Terraform
Similar Jobs
Marketing Tech • Software
Lead the design and optimization of data pipelines and infrastructure, ensuring data availability and quality while collaborating with data scientists and adopting new technologies.
Top Skills:
AirflowDockerElkGCPGrafanaKubernetesLinuxPrometheusPythonSQLTerraformUnix
Big Data • Fintech • Mobile • Payments • Financial Services
As a Senior Software Engineer, you will lead backend development for the Merchant Data Platform, mentor junior engineers, ensure system reliability, and collaborate in product development. Your role involves project management, technical planning, and maintaining high availability of services.
Top Skills:
AWSKotlinKubernetesMySQLPython
Software • Cybersecurity
The Senior Data Engineer will evolve data infrastructure, manage data governance, and ensure reliability while collaborating with ML and analytics teams to provide well-governed datasets.
Top Skills:
SparkAthenaAvroAWSDockerKubernetesOrcParquetPysparkPythonRedshiftS3SQLTerraformTrino
What you need to know about the London Tech Scene
London isn't just a hub for established businesses; it's also a nursery for innovation. Boasting one of the most recognized fintech ecosystems in Europe, attracting billions in investments each year, London's success has made it a go-to destination for startups looking to make their mark. Top U.K. companies like Hoptin, Moneybox and Marshmallow have already made the city their base — yet fintech is just the beginning. From healthtech to renewable energy to cybersecurity and beyond, the city's startups are breaking new ground across a range of industries.
