The Business Data Infrastructure (BDI) Team is the beating heart of our business data and analytics ecosystem at Red Canary, ensuring that all insights, decisions, and innovations are powered by consistent, accurate, and accessible data. The BDI team takes a data product approach, where the hub serves as the platform for developing and managing data products. We defined data products as discrete, reusable datasets or pipelines that are curated to meet the needs of various stakeholders. These data products are treated like standalone assets, each with a clear purpose, defined ownership, and measurable quality standards. This ensures consistent delivery of tangible value to the spokes (the various departments across Red Canary).By aligning the design, governance, and delivery of data products with user needs, we ensure that data is not just accessible but also usable, meaningful, and trusted. Thus, BDI is not a passive data store, but a marketplace of data products, enabling our teams to innovate faster while maintaining enterprise-level standards and cohesion.
Challenges You Will Solve
We are seeking a Data Engineer to play a pivotal role in designing, building, and maintaining the data infrastructure that powers the organization's insights and decisions. This role will help ensure data is ingested, transformed, and stored securely and efficiently, enabling consistency, scalability, and accessibility across all teams. The data product approach will empower the Data Engineer role to act as a bridge between raw data and actionable insights, empowering departments to leverage reliable, well-structured datasets for innovation and growth – playing a critical role in the decision intelligence process at Red Canary.
What You'll Do
- Build and Maintain Scalable Data Infrastructure: Design, implement, and optimize data pipelines and storage solutions to support the ingestion, transformation, and distribution of high-quality data across the organization.
- Develop and Manage Data Products: Create reusable, well-documented, and user-centric data products tailored to the needs of specific teams, ensuring they meet defined performance and quality standards.
- Ensure Data Quality and Governance: Implement rigorous validation, monitoring, and governance processes to ensure data accuracy, consistency, and compliance with enterprise policies and regulatory requirements.
- Collaborate with Cross-Functional Teams: Partner with analysts, data scientists, and business stakeholders in the spokes to understand their needs and deliver solutions that empower data-driven decision-making.
- Continuously Optimize for Scalability and Efficiency: Stay updated on emerging technologies and industry best practices, using them to enhance the hub’s infrastructure and improve the usability and reliability of data products.
What You'll Bring
- Bachelor's or Master's degree or equivalent work experience in a relevant field such as Data Engineering, Mathematics, Statistics, Computer Science, or a related engineering discipline.
- 3+ years of proven experience as a Data Analyst, Data Engineer, data-focused Software Engineer, or similar – preferably in a startup or fast-paced environment.
- Strong knowledge of relational databases, distributed storage and semi/un-structured data.
- Substantial experience working within cloud computing platforms, such as AWS, GCP or Azure.
- Strong experience working with data pipeline execution and design – working with tools such as AWS Sagemaker, AWS Glue, Apache Airflow, Prefect or similar.
- Exposure to distributed computing, such as pySpark or similar is a plus.
- Experience with some of our primary data sources is a plus (Intacct, Salesforce, Zendesk, Jira)
- Excellent problem-solving and critical-thinking skills, with the ability to assess a complex problem set and distill insight into a focused, durable and understandable data product.
- Strong communication skills, with the ability to effectively present complex findings and recommendations to non-technical stakeholders.
- Self-motivated and proactive, with the ability to work independently and collaborate effectively within a team and across a distributed customer base.
Base salary for this role is $117,000 - $135,000 per year. This role is also eligible for participation in the company's bonus program. This role is eligible for a grant of stock options, subject to the approval of the company's board of directors. Application deadline is February 7, 2025.
