Staff Perception Engineer (US Based)

At Andromeda Robotics, we’re not just imagining the future of human-robot relationships; we’re building it. Abi is the first emotionally intelligent humanoid companion robot, designed to bring care, conversation, and joy to the people who need it most. Backed by tier-1 investors and with customers already deploying Abi across aged care and healthcare, we’re scaling fast, and we’re doing it with an engineering-first culture that’s obsessed with pushing the limits of what’s possible.

This is a rare moment to join: we’re post-technical validation, pre-ubiquity, and building out the team that will take Abi from early access to global scale.

• Empathy - Kindness and compassion are at the heart of everything we do.
• Play - Play sharpens focus. It keeps us curious, fast and obsessed with the craft.
• Never Settle- A relentless ambition, bias toward action, and uncomfortable levels of curiosity.
• Tenacity - Tenacious under pressure, we assume chaos and stay in motion to adapt and progress.
• Unity- Different minds. Shared mission. No passengers.

The Role:
This role balances architectural leadership with strategic, hands-on work. You will lead the design and implementation of Abi’s perception stack, translating raw sensor data into semantic understanding that supports autonomous navigation and conversational AI. 

This dual challenge demands a systems thinker who can architect scalable perception pipelines and also implement foundational components to prove the architecture’s effectiveness. You will bridge multiple domains—sensor fusion, computer vision, audio processing, and ML deployment—while optimising for real-time performance on our embedded Jetson Orin AGX platform.

The Team:

You'll be the founding perception hire, collaborating deeply with autonomy, conversational AI, gestures, controls, audio engineering, and ML teams. You'll work with product owners to translate user needs into technical requirements, and with the broader engineering team to ensure your perception outputs enable downstream systems to flourish.

Key Responsibilities:

• Architect the Perception Stack: Design and own the full system from raw sensors through semantic understanding, including interface contracts, compute and latency budgeting, and robust architectural decisions.
• Lead Cross-Functional Design: Facilitate design workshops with autonomy, conversational AI, audio engineering, ML, and hardware teams to align resources and interface definitions.
• Implement Strategically: Develop core perception capabilities like face recognition, speaker diarisation, person detection and tracking, and sensor fusion to validate and instantiate the architecture.
• Own Production Systems: Build with production-readiness in mind, including graceful degradation, monitoring, debugging tools, and deployment pipelines.
• Make Technical Decisions: Drive build vs. buy, algorithm choices, and embedded deployment optimisations, focusing on performance and resource constraints.
• Scale the Team: Support recruitment and mentoring as the perception team grows.

What You'll Build:

• Scene Awareness Pipeline: Architect the unified perception system that serves navigation, gesture, and conversational needs.
• Person Detection & Tracking: Implement robust multi-person tracking using multiple cameras, LiDAR point clouds, and IMU data for both navigation obstacle avoidance and social scene understanding
• Audio-Visual Speaker Localisation: Fuse microphone arrays, face detection, gaze tracking, and skeleton pose to determine who is speaking and where Abi should direct her attention
• Sensor Fusion Architecture: Design the common data structures and synchronisation primitives that enable downstream systems to consume perception outputs efficiently, optimised to run on Jetson Orin AGX within strict latency budgets

Your work will enable other engineering teams to integrate their subsystems: 

• Audio engineering teams to conduct audio signal processing
• Machine learning teams to develop specialist models, such as emotional sentiment prediction models, and face recognition that enable personalised memory recall
• Hardware teams to readily change sensors as we upgrade Abi 

Your First 90 Days

Month 1: Architect the Perception Stack. Lead design workshops with autonomy, conversational AI, audio engineering, and ML teams to define the complete perception architecture. Establish interface contracts between perception and downstream consumers. Allocate compute and latency budgets across the Jetson Orin platform. Define data schemas, synchronisation primitives, and failure mode handling. Deliver the architectural blueprint that guides all perception work.

Month 2: Validate with Face Recognition & Speaker Diarisation. Implement face recognition and speaker diarisation systems to prove out the architecture. Use these as forcing functions to stress-test your interface definitions, budget allocations, and integration patterns. Iterate the architecture based on real deployment constraints.

Month 3: Establish Production Infrastructure. Build monitoring, testing frameworks, and deployment pipelines. Codify the architectural patterns and interface contracts. Prepare the foundation for the team to scale perception capabilities independently.

Ideally, You Have:

• Bachelor’s or Master’s degree in Computer Science, Robotics, Electrical Engineering, or related field
• 5-7+ years of experience building and shipping perception systems for robotics or autonomous vehicles
• Deep expertise in computer vision (object detection, tracking, 3D reconstruction, camera calibration)
• Strong fundamentals in sensor fusion (Kalman filtering, probabilistic estimation, multi-modal integration)
• Real-time embedded systems experience (CUDA, TensorRT, ROS2)
• Proven architectural skills with hands-on coding ability in C++ and Python
• Pragmatic mindset balancing off-the-shelf and custom solutions

Bonus Points If You Have:

• Audio processing background, including beamforming and source localisation
• Experience with face recognition systems and liveness detection
• Expertise with depth sensors such as stereo vision, structured light, or LiDAR
• Skills in human pose estimation and social gaze prediction mechanisms
• Experience optimising ML models for edge deployment
• A PhD in relevant fields (computer vision, robotics, signal processing)

Interview Process:

• 1st Round Interview: Recruiter phone screen to discuss background and fit
• Take-home assessment to architect the system and perform coding tasks involving data sets to fuse audio and visual inputs
• 2nd Round Interview: Technical interview focusing on perception engineering concepts, system design, and implementation challenges
• Final Round Interview: Cultural interview to ensure alignment with Andromeda Robotics' values and team fit

Benefits:
The expected base salary range for this role, when performed in our San Francisco office, is $150,000 - $250,000 USD, depending, of course, on skills and experience. The total compensation package may also include additional benefits or components based on the specific role. Details will be provided if an employment offer is made.

If you’re excited about this role but don’t meet every requirement, that’s okay—we encourage you to apply. At Andromeda Robotics, we celebrate diversity and are committed to creating an inclusive environment for all employees. Let’s build the future together.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Please note: At this time, we are generally not offering visa sponsorship for this role.