Data Scientist & Engineer

Role Purpose 

We are building a new R&D capability focused on developing data-driven and AI-enabled products for legal services and the wider business of law. 

The Data Scientist & Engineer will work within R&D alongside the Data Programme to build the foundational data capabilities required for those products: trusted, reusable data assets, reliable pipelines, data quality, lakehouse models and analytical products. These capabilities will support R&D products, AI applications, client intelligence and firm-wide decision support. 

This is a hands-on hybrid data-engineering and applied-data-science role, with particular responsibility for building data products in Microsoft Fabric and Databricks. You will work with Data Programme colleagues to turn priority sources into governed, use-case-ready data assets using agreed patterns, without duplicating enterprise platform responsibilities. 

You will join a small, hands-on multidisciplinary team working collaboratively across discovery, prototyping, engineering, productionisation and continuous improvement. 

Key Responsibilities 

• Partner with the Data Programme, data owners and system teams to assess, access and combine priority internal and external data sources, understanding quality, permissions and fitness for use. 

• Design, build and operate reliable pipelines in Microsoft Fabric and Databricks to ingest, clean, standardise, enrich, version and publish structured, semi-structured and unstructured data as reusable data assets. 

• Create scalable lakehouse data models, schemas, data contracts and reusable features for entities such as clients, organisations, people, matters, sectors, jurisdictions, opportunities and legal topics. 

• Implement data-quality checks, lineage, provenance, observability, monitoring and change management within Fabric and Databricks workflows so data assets remain trusted over time. 

• Conduct exploratory data analysis to surface patterns, gaps, anomalies and data-quality issues, and identify opportunities for analysis, modelling or product use. 

• Develop, compare and validate appropriate statistical and machine-learning models for classification, ranking, recommendation, forecasting, anomaly detection, similarity or extraction problems. 

• Use NLP, embeddings and LLM-assisted techniques responsibly for classification, entity resolution, information extraction, clustering and relationship discovery across document-heavy data. 

• Publish curated datasets, features and analytical outputs through approved Fabric and Databricks data products, tables, APIs, search indexes, batch pipelines or product features. 

• Maintain reproducible code, tests, model and data documentation, evaluation evidence and clear statements of limitations and appropriate use. 

• Use AI agents and coding assistants responsibly to accelerate exploratory work, pipeline development, testing and documentation, while retaining ownership of analytical judgement and verification. 

Initial Focus 

• Establish foundational, governed and permission-aware data assets in Microsoft Fabric and Databricks, including curated datasets, metadata, lakehouse data models and quality measures. 

• Develop reusable pipelines in Microsoft Fabric and Databricks for entity resolution, enrichment, metadata, relationship mapping and data-quality monitoring. 

• Run EDA and model development that increases the value of priority data assets for client intelligence, opportunity identification, pricing, matter analysis and other high-value use cases. 

What Success Looks Like 

• The Data Programme and R&D teams can use documented, quality-controlled data assets in Fabric and Databricks without repeatedly cleaning and reconciling source data. 

• Analytical work identifies useful signals and viable opportunities, with transparent baselines, validation and limitations. 

• Successful models and features move beyond notebooks into maintainable Fabric and Databricks pipelines, APIs or products. 

Essential Skills and Experience 

• At least three years' professional experience in a hybrid data-engineering and applied-data-science role, including delivery of reliable pipelines, reusable data assets and useful analytical models in Microsoft Fabric and Databricks. 

• Advanced Python, SQL and PySpark, including experience with data transformation, analysis and common machine-learning libraries. 

• Hands-on experience designing and operating scalable data pipelines, lakehouse data models, Delta and SQL transformations, and data-quality controls in Microsoft Fabric and Databricks. 

• Strong grounding in exploratory data analysis, statistics, feature engineering, model selection, validation and error analysis. 

• Practical experience with NLP, information extraction, entity resolution, embeddings or other techniques relevant to document-heavy and relationship-rich data. 

• Good understanding of data lineage, provenance, metadata, access controls, permissions and the responsible use of sensitive data within an enterprise data estate. 

• Experience with Git, automated testing, API integration, CI/CD and version-controlled deployment of data workloads in Microsoft Fabric and Databricks. 

• Ability to translate a business question into a well-formed analytical problem and communicate findings, uncertainty and trade-offs clearly. 

• Proficiency with AI agents and agentic development workflows, coupled with rigorous validation of code, data transformations and analytical conclusions. 

Desirable Skills and Experience 

• Experience with Azure Data Lake, Azure Data Factory, dbt, Airflow, MLflow, Power BI, Microsoft Purview or comparable tools. 

• Experience with search indexes, vector databases, graph databases, knowledge graphs, ontologies or master-data-management techniques. 

• Experience with MLOps, model monitoring, experimental design, causal inference, forecasting, recommendation systems or optimisation. 

• Experience with legal, financial, client, market or professional-services data, especially licensed external data and restricted information.