Designs, develops, tests, optimizes, and maintains scalable batch and near-real-time data pipelines and architectures. Ensures data quality, builds API integrations, improves internal data processes, and supports machine learning, data science, BI, and product initiatives. The role requires experience with data warehouses, relational and NoSQL databases, data modeling, event-driven architectures, SQL, Python, Airflow, Spark, and AWS data services.
About the role
As a Data Engineer at Sporty, you will play a critical role in ensuring the smooth processing and handling of data for our machine learning and data science initiatives. Your primary responsibilities will include designing, building, testing, optimising, and maintaining data pipelines and architectures for various aspects of our rapidly growing business.
What you'll be doing
- Design, develop and maintain scalable batch ETL and near-real-time data pipelines and architectures for various parts of our business, on fast and versatile data sources with millions of changes per day
- Ensure all data provided is of the highest quality, accuracy, and consistency
- Identify, design, and implement internal process improvements for optimising data delivery and re-designing infrastructure for greater scalabilit
- Builds out new API integrations to support continuing increases in data volume and complexity
- Communicate with data scientist, MLOps engineers, product owners and BI analysts in order to understand business processes and system architecture for specific product features
What you'll bring
- Bachelor’s degree, or equivalent experience, in Computer Science, Engineering, Mathematics, or a related technical field
- 3+ years of experience in data engineering, data platforms, BI or related domain
- Experience in successfully implementing data-centric applications, such as data warehouses, operational data stores, and data integration projects
- Experience with large-scale production relational and NoSQL databases
- Experience with data modelling
- General understanding of data architectures and event-driven architectures
- Proficient in SQL
- Familiarity with one scripting language, preferably Python
- Experience with Apache Airflow & Apache Spark
- Solid understanding of cloud data services: AWS services such as S3, Athena, EC2, RedShift, EMR (Elastic MapReduce), EKS, RDS (Relational Database Services) and Lambda
Even better if
- Understanding of ML Models
- Understanding of containerisation and orchestration technologies like Docker/Kubernetes
- Relevant knowledge or experience in the gaming industry
What's in it for you
- Sporty is a remote first company in pursuit of sustainability
- A competitive salary + individual performance based bonuses every quarter
- 28 days paid annual leave
- Our core working hours are 10am-3pm in your local time zone with flexibility outside of this
- Referral bonuses & flash bonuses
- Top of the line equipment
- Annual company retreats to provide great internal networking opportunities
Interview Process
- Remote video screening with our Talent Acquisition Team
- Offline Take home assignment
- Remote video interview with Team Members (90 Mins)
If you're interested, we encourage you to apply! Every application is reviewed by a member of our team (AI is not used in our recruitment process), and we aim to respond within 48 hours.
*We are committed to maintaining a fair, secure and authentic recruitment process. Applications submitted using false, misleading or fraudulent information, including impersonated or fake profiles, will not be considered. Where reasonably necessary to protect the integrity and security of our recruitment process, we may verify a candidate’s identity and/or the accuracy of information provided during the recruitment process. Any such checks will be carried out in a proportionate and appropriate manner and in accordance with applicable data protection and employment laws.
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Build and operate production data pipelines supporting analytics, AI, and agentic workflows. Responsibilities include implementing canonical data models, maintaining Databricks or Snowflake platforms, monitoring reliability, responding to incidents, validating healthcare data mappings, improving performance and cost, and documenting architecture. Requires strong SQL and Python skills, cloud data platform experience, ETL/ELT orchestration expertise, and healthcare or pharmaceutical data experience.
Top Skills:
Ai/Ml WorkflowsDatabricksEltETLHedisOmopPythonSnowflakeSQL
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Owns the design, development, operation, and governance of data pipelines and integration patterns supporting Medical Affairs AI products. Builds APIs and ETL/ELT solutions connecting enterprise systems to analytics platforms, RAG pipelines, vector databases, and GraphRAG applications. Ensures data quality, privacy, lineage, compliance, and protection of sensitive information. Partners with architecture, engineering, product, and business stakeholders to deliver reusable, production-grade data integrations.
Top Skills:
Ai/MlApi GatewaysCi/CdEtl/EltEvent-Driven IntegrationGraph DatabasesGraphQLGraphragLow-Code/No-Code ToolsMiddlewareRagRestSalesforce Life Sciences/Marketing CloudSnowflakeSQLStreaming IntegrationVector DatabasesVeeva Crm
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Designs, builds, and operates production-grade agentic AI systems and orchestration frameworks. Responsibilities include prompt architecture, tool and API integrations, monitoring, evaluation, cost controls, reliability improvements, and governance documentation. The engineer collaborates with data science, data engineering, and governance teams to ensure reliable, compliant workflows using structured healthcare and pharmaceutical data.
Top Skills:
LanggraphLlm ApisMlopsPydantic Ai
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
