Staff Software Engineer - Data Integration Services

PhysicsX is developing a platform used by Data Scientists and Simulation Engineers to build, train, and deploy Deep Physics Models. As adoption grows, one of the most significant challenges is how data is accessed, shared, and governed across services, teams, and compute environments.

Simulation and AI workflows generate large and complex datasets — meshes, solver outputs, experiment artifacts, and derived model data — that must move seamlessly between interactive environments, automated pipelines, platform services, and external customer infrastructure.

Many engineering tools originate from POSIX filesystem assumptions, while modern cloud platforms are built around distributed services and object storage. Bridging these worlds requires preserving usability without inheriting the operational and scalability limitations of traditional mounting-based approaches.

Data permissioning is a core concern. This role will integrate fine-grained authorisation into the platform’s data model to enable secure multi-tenant data sharing.

We are looking for a Staff Software Engineer to lead the evolution of Data Integration Services — shared platform capabilities that define how data is discovered, accessed, exchanged, and governed across PhysicsX.

This role focuses on designing cloud-native data access patterns informed by filesystem semantics, distributed storage tradeoffs, and real-world workload behaviour. You will help teams interact with data predictably regardless of where it resides, while ensuring the platform remains scalable, secure, and operable at scale.

What You Will Do

• Define and drive the technical vision for Data Integration Services as a shared platform capability.
• Design scalable, cloud-native data access patterns supporting simulation, ML, and platform workloads without tightly coupled mounting solutions.
• Establish consistent abstractions for accessing datasets across APIs, services, and compute environments.
• Design integrations with storage ecosystems including:
  • Object storage platforms (S3-compatible systems, Azure Blob, GCS, MinIO)
  • Managed storage and filesystem abstraction platforms (e.g. FSx, JuiceFS, Alluxio, or similar systems)
  • Protocol-based access layers such as WebDAV or HTTP-based data interfaces
• Translate filesystem-oriented workflows into reliable distributed service patterns where appropriate.
• Enable efficient data sharing across teams and tenants while maintaining strong isolation and governance controls.
• Design dataset lifecycle capabilities including provisioning, versioning, and retention strategies.
• Define how permissioning integrates into the data plane and is enforced consistently across services and workflows.
• Partner with platform identity and security teams to ensure consistent access enforcement.
• Define standards for data contracts and compatibility between services.
• Evaluate build-vs-buy decisions across storage, data access, and metadata tooling.
• Establish observability and performance practices for large-scale data workflows.
• Mentor engineers and drive cross-team adoption of shared data integration patterns.

• A passion for the craft — a drive for engineering excellence and a commitment to raising technical standards across teams.
• Strong software engineering foundations — algorithms, data structures, and system design, with a focus on building clean, maintainable, and testable systems. Strong command of Golang and Python.
• Distributed systems experience — proven track record designing and operating production systems where data movement, consistency, and scalability are core concerns.
• API and platform design maturity — experience designing systems composed of multiple services or subsystems, with attention to long-term evolution, schema governance, and sustainable integration patterns.
• Architectural breadth — experience designing secure multi-tenant systems and navigating tradeoffs across storage models, compute environments, and cloud infrastructure.
• Reliability and observability mindset — experience defining operational guarantees, monitoring complex systems, and diagnosing production issues in distributed environments.
• Security and governance awareness — experience designing access control and data isolation mechanisms, ideally including fine-grained or relationship-based authorisation models.
• Cloud-native data systems experience — familiarity with object storage ecosystems and an understanding of filesystem and POSIX semantics, including the tradeoffs when adapting file-based workflows to distributed architectures.
• Diagnostic and optimisation skills — ability to identify performance bottlenecks across I/O, storage, and large-scale data workflows.
• Proven technical leadership — experience setting technical direction, driving consensus across teams, and delivering platform capabilities adopted beyond a single service.
• Communication and influence — ability to translate complex architectural ideas into actionable direction for engineers, product stakeholders, and leadership; experience mentoring and guiding engineers across the organisation.

• Experience building developer-facing data platforms or internal data infrastructure.
• Exposure to ML, simulation, or HPC workflows with large-scale datasets.
• Familiarity with data virtualisation or access acceleration technologies.
• Experience enabling hybrid or externally hosted compute to securely access shared data.
• Knowledge of dataset cataloging, metadata systems, or lineage tracking.
• Experience designing scalable alternatives to filesystem-based integration patterns.
• Prior experience shaping platform capabilities adopted organisation-wide.

• Equity options – share in our success and growth.
• 10% employer pension contribution – invest in your future.
• Free office lunches – great food to fuel your workdays.
• Flexible working – balance your work and life in a way that works for you.
• Hybrid setup – enjoy our new Shoreditch office while keeping remote flexibility.
• Enhanced parental leave – support for life’s biggest milestones.
• Private healthcare – comprehensive coverage.
• Personal development – access learning and training to help you grow.
• Work from anywhere – extend your remote setup to enjoy the sun or reconnect with loved ones.