Quartermaster AI Logo

Quartermaster AI

Senior Data Engineer

Reposted 2 Days Ago
Hybrid
San Francisco, CA, USA
Senior level
Hybrid
San Francisco, CA, USA
Senior level
The Senior Data Engineer will design and maintain scalable data systems for maritime and defense applications, ensuring secure and resilient operations while collaborating with technical teams.
The summary above was generated by AI
About Us:

At Quartermaster AI, we believe the ocean should be a safe and sustainably managed resource for all. By leveraging cutting-edge AI, distributed sensing, and resilient data systems, we unlock capabilities that were only recently impossible. Our distributed open-ocean systems enable vessels and coastal infrastructure to sense, compute, and communicate, enhancing maritime domain awareness for those who need it most.

Job Description:

We are seeking a highly experienced Senior Data Engineer (7+ years experience) to design, build, and maintain secure, resilient, and scalable data systems supporting mission-critical maritime and defense applications.

This role requires deep expertise in AWS and Azure environments, distributed systems, and secure data architecture. The ideal candidate has experience working with government contractors, military systems, or regulated environments and understands the operational demands of real-time, high-availability data platforms.

Familiarity with GIS, streaming architectures, SDR/radio systems, radar pipelines, maritime domain awareness systems, or military data management is strongly preferred.

Key Responsibilities:
  • Design, build, and maintain scalable batch and real-time machine learning, data science, and data export pipelines.

  • Architect and manage AWS and Azure cloud infrastructure.

  • Administer and optimize databases including MongoDB and PostgreSQL using RDS and EC2-hosted environments.

  • Implement replication strategies, failover systems, and resiliency management for high-availability operations.

  • Design systems that strictly separate public and private/classified data.

  • Support high-throughput streaming ingestion and processing (sensor, radar, SDR, maritime feeds).

  • Develop infrastructure-as-code and automated deployment pipelines.

  • Ensure secure-by-design architecture aligned with compliance requirements.

  • Provide guidance and architectural leadership for handling data at scale.

  • Monitor performance, reliability, and cost optimization across environments.

  • Collaborate with DevSecOps, AI/ML, and mission teams to deliver operational capabilities.

  • Must be eligible to obtain/maintain a security clearance.

Qualifications (Required):
  • 7+ years of professional experience in Data Engineering.

  • Proven experience building production-grade data pipelines.

  • Strong hands-on experience with AWS (RDS, EC2, S3, IAM, VPC).

  • Experience with Azure data services.

  • Database expertise in MongoDB and PostgreSQL.

  • Proficiency in Python, TypeScript, and SQL.

  • Experience implementing high-availability, replication, and disaster recovery strategies.

  • Experience working in government, defense, or regulated environments.

  • Active security clearance OR ability to obtain/maintain clearance.

Qualifications (Preferred):
  • Experience with GIS systems and geospatial data architectures.

  • Experience with real-time streaming frameworks (Kafka, Kinesis, Spark, Flink).

  • Familiarity with SDR, radio frequency data, radar systems, GIS, or maritime domain awareness platforms.

  • Experience managing high-velocity time-series or sensor data.

  • AWS certifications (Solutions Architect, DevOps, Data Analytics).

  • Microsoft Azure certifications.

  • Experience with containerization (Docker, Kubernetes).

  • Infrastructure-as-code (Terraform, CloudFormation, Bicep).

  • Security+ or similar certification preferred in secure environments.

Work Environment:
  • Hybrid position requiring 3–5 days per week onsite.

  • Secure, mission-driven engineering environment.

  • Collaboration with cross-functional technical and operational teams.

  • Occasional schedule flexibility required for mission support.

Benefits:
  • Competitive salary

  • Opportunity to work on mission-critical maritime and defense technology

Similar Jobs

7 Days Ago
Hybrid
147K-185K Annually
Senior level
147K-185K Annually
Senior level
Fintech • Machine Learning • Payments • Software • Financial Services
Collaborate in Agile teams to develop and support data solutions, utilizing programming languages and cloud services to drive transformation in corporate functions.
Top Skills: AWSEmrGCPGurobiHadoopHiveJavaKafkaLinuxMapreduceAzureMySQLNoSQLPythonRedshiftScalaSnowflakeSparkSQLUnix
6 Days Ago
Hybrid
209K-262K Annually
Senior level
209K-262K Annually
Senior level
Fintech • Machine Learning • Payments • Software • Financial Services
The Sr. Lead Data Engineer will lead developers in data solutions, collaborate with Agile teams, utilize big data technologies, and develop cloud-based services.
Top Skills: AWSEmrGCPGurobiHadoopHiveJavaKafkaLinuxMapreduceAzureMySQLNosql DatabasesOpen Source RdbmsPythonRedshiftScalaSnowflakeSparkUnix
6 Days Ago
Hybrid
2 Locations
230K-286K Annually
Senior level
230K-286K Annually
Senior level
Fintech • Machine Learning • Payments • Software • Financial Services
Lead diverse technology projects and a team to create solutions for regulatory needs, leveraging distributed microservices and collaboration with product managers.
Top Skills: Amazon SagemakerAWSCSSDatabricksDockerEksGoHTMLJavaJavaScriptKubernetesMachine LearningNosql DatabasesOpen Source RdbmsPythonSQLTypescript

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account