Similar Jobs
As part of the Big Data Team, you will be using cutting edge technologies to work on collating data across different sources within GiG. Sources are all in real time, and you will be supporting an event driven micro service lambda architecture
Your focus is to work with the current agile team and contribute to the platform’s objectives and continue being an enabler to many other data technology stakeholders such as the Business Intelligence, Data Science and Quality Assurance teams. You will be up for a challenge, to continue maintaining and adding new features to a platform that ingests up to 5000 messages per second across multiple products such as Lottery, Sports and Casino.
Reporting to the Lead Big Data Engineer and supported by the Big Data Architect to follow best practices, you'll be working on a reload framework which will be used to make sure that any missed transactions are loaded automatically or by a click of a button. In order to maintain data integrity, daily reconciliation jobs are in place with the aim of comparing the data ingested in the last 24 hours with the data available in the same timeframe in the source. Any discrepancy will be highlighted and manually loaded back into the data platform. The aim of this task will be to automate this process while making sure to have efficient reload queries which make little or no impact on our sources and at the same time making sure to re-run the reconciliation framework to make sure the data has been loaded without any issues. Deduplication and Marts layer syncing needs to be taken into consideration as well.
What you will be doing:
- Familiarisation with the technical concepts used within the big data platform.
- Familiarisation with the business concepts of iGaming.
- Take part in all agile scrum meetings such as daily standups, refinement sessions, retro sessions, etc
- Implementation of basic fixes within the data platform.
- Implementation of a reload framework which eventually will be productionised.
Requirements:
- Ability to learn new technologies and big data principles
- Good analysis and problem solving skills
- Good communication skills
- Ability to learn about data requirements for the iGaming Business
- Understanding of coding principles.
- Knowing basic Unix commands for file manipulation
- Understanding the concepts and importance of CICD
- Ability to learn the concepts of Big Data and Event Driven architecture vs Traditional Data Architectures
What we offer:
- Exciting opportunities for career growth
- Hybrid-working model
- Complimentary office lunches
- Company social events
