Design, train, and deploy vision-language models for construction intelligence using large-scale first-person video. Build training/evaluation pipelines, optimize models for edge and server inference, define benchmarks (video QA, temporal reasoning, activity recognition), and collaborate with hardware and data teams to productionize models.
About Ironsite
The Role
What You'll Build
Technical Challenges You'll Solve
What We're Looking For
Preferred Qualifications
Location & Compensation
Compensation
Construction is one of the most complex and labor-intensive industries, spending $7 trillion annually on labor, but productivity losses cost $1.6 trillion per year due to outdated management tools.
Ironsite leverages wearable cameras combined with human labeling and AI vision language models to drive on-site productivity, safety & training for crafts workers. We put cameras on construction workers' hard hats and vests, then use advanced computer vision to analyze what's actually happening on job sites.
We help teams reduce labor costs, improve safety, and deliver projects faster. To date, we’ve captured 50,000+ hours of construction footage across 7 states and have recently partnered with the nation’s #2 hard-hat manufacturer, Studson, to develop custom hardware purpose-built for the field.
We’ve raised over $13M from 8VC, South Park Commons, and 30+ leading operators and technologists, including Eric Glyman (Ramp), Jeff Dean (Google), and Scott Wu (Cognition). Now, we’re building the team to scale nationwide.
As an Applied ML Researcher on our team, you will report directly to the Chief Science Officer and be at the heart of our mission to build foundational spatial intelligence. You will tackle high-risk, high-impact research, leveraging our unparalleled proprietary dataset to train, benchmark, and deploy state-of-the-art VLMs that can interpret the complexity of a real-world construction site.
- Architect & Train Novel VLMs: Design, train, and iterate on general-purpose Vision-Language Models fine-tuned for "Construction Intelligence" using our massive, proprietary dataset of first-person video.
- Drive the Research Roadmap: Take a leading role in executing our research goals, including establishing baselines with state-of-the-art models and developing novel fine-tuning methodologies, long context architectures, and visual reasoning techniques.
- Build Scalable Pipelines: Develop and own the model training and evaluation pipelines, ensuring we can rapidly experiment, measure performance, and deploy models into production.
- Optimize for the Edge: Design and implement a hierarchical set of models for efficient, on-device inference on our wearable hardware as well as server-side inference. This includes developing lightweight, coarse classifiers for real-time analysis (e.g., safety event detection, info density classification), as well as heavy-weight server-side VLMs to deeply understand complex tasks.
- Define the Future of Spatial Intelligence: Spearhead the development and expansion of our "Construction Intelligence" benchmark, a comprehensive suite of tasks including video-question answering, temporal reasoning, activity recognition, and higher level data analysis reasoning across the construction site that will guide our research.
- Collaborate on System Design: Work closely with the hardware and data teams to explore model architectures such as a two-model system (lightweight segmenter, heavyweight insight extractor) and tool-use agents, to improve spatial understanding.
- Training large-scale models efficiently with limited compute budgets while maximizing performance
- Developing novel pre-training objectives that capture construction-specific knowledge and temporal reasoning
- Implementing efficient attention mechanisms and architectural innovations for long-context understanding of construction projects
- Designing evaluation metrics that measure real-world construction task performance beyond standard benchmarks
- Balancing model capability with deployment constraints for edge and mobile applications
What We're Looking For
Technical Excellence
- Fast learner eager to grow and expand their knowledge and capabilities, welcoming new challenges with a passion for their work
- Background in Computer Science, Machine Learning, AI, Robotics, Data Science or a related field with multiple relevant classes completed.
- Prior experience, internship, or personal project with hands-on experience designing and training deep learning models, particularly transformer-based architectures.
- Familiarity with one deep learning framework (e.g., PyTorch, TensorFlow, JAX).
- Proficiency in Python
Preferred Qualifications
- At least one publication in an AI/ML/CV conference.
- Experience doing research as part of a larger research lab or team
- Demonstrated experience with major deep learning frameworks (e.g., PyTorch, TensorFlow, JAX).
- Strong proficiency in Python and a solid foundation in software engineering principles.
- Experience working with and creating large-scale vision and/or language datasets.
- A strong interest in vision language models and the application of AI to solve real-world physical problems, including working with and understanding the day-to-day lives of construction workers.
Location & Compensation
- San Francisco Bay Area (on-site)
- Competitive salary and significant equity package
- Full benefits including health, dental, vision, and 401k +6% match
- Access to dedicated GPU compute resources for research and experimentation
The base pay range for this role is $180,000 – $350,000 per year.
Similar Jobs
Artificial Intelligence • Computer Vision • Hardware • Software
Lead design, training, and deployment of large-scale language and multimodal models for construction; build training infrastructure, data pipelines, evaluation and alignment (including RLHF), and production ML systems integrating human labeling and vision-language models.
Top Skills:
ClmComputer VisionData IngestionDistributed TrainingDpoExperiment TrackingFlashattentionGpu ClustersGradient AccumulationGradient CheckpointingLoraMixed PrecisionMlmMlopsModel ServingPpoPyTorchQloraRlhfTokenizationTransformer ArchitecturesVideo UnderstandingVision-Language Models
Artificial Intelligence • Machine Learning • Software • Industrial
The role involves building and improving multimodal foundation models that integrate various types of data for real-world applications, requiring strong research capabilities and deployment experience.
Top Skills:
Data EvaluationDeploymentExperimental DesignMachine LearningModel DevelopmentMultimodal ModelsSensor DataTime Series Data
eCommerce • Fashion • Retail • Sales • Wearables • Design
Maintain organized, customer-ready store by processing deliveries, stocking the sales floor, executing price changes and markdowns, auditing inventory/shrinkage, and supporting daily operational standards and cleanliness.
Top Skills:
Omnichannel SellingSocial Media
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
