The Data Engineer will analyze data quality, design prediction algorithms, collaborate with the engineering team, and generate business insights.
This is an exciting position to work on a very talented team of engineers. The ideal candidate's favorite words are learning, data, scale, and agility. You will leverage your strong collaboration skills and ability to extract valuable insights from highly complex datasets to ask the right questions and find the right answers. Additionally you enjoy putting the results of your work into meaningful reports that are then presented to the team.
Responsibilities
- Analyze raw data: assessing quality, cleansing, structuring for downstream processing
- Design accurate and scalable prediction algorithms
- Collaborate with engineering team to bring analytical prototypes to production
- Generate actionable insights for business improvements
Qualifications
- Bachelor's degree or equivalent experience in quantitative field (Statistics, Mathematics, Computer Science, Engineering, etc.)
- At least 5 years' of experience in quantitative analytics or data modeling
- Fluency in a programming language (Python and SQL)
- Familiarity with Big Data frameworks and visualization tools such as (Cassandra, Hadoop, Spark, Tableau)
AutoRoboto San Francisco, California, USA Office
San Francisco, California, United States, 94112
Similar Jobs
AdTech • Automotive • Big Data • Consumer Web
Administer and enhance Edmunds’ Databricks data platform and AWS infrastructure. Build and maintain ETL pipelines, infrastructure-as-code tooling using Terraform or CDK, and operational dashboards, alerts, and reports. Collaborate with business, engineering, analytics, security, and infrastructure teams to support data platform users and AI solutions. Evaluate new technologies, troubleshoot platform issues, and improve operational, cost, and security visibility.
Top Skills:
SparkAWSAws CdkDatabricksInfrastructure As Code (Iac)PythonScalaSQLTerraform
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Build and maintain data pipelines and scalable cloud infrastructure supporting analytics and machine learning. Integrate internal and external data sources, ensure data integrity, contribute to analytical investigations, document findings, and apply data governance, security, and performance optimization practices while collaborating with senior engineers and cross-functional teams.
Top Skills:
APIsAWSAzureCloud-Native DesignDistributed SystemsGCPGenerative AiLarge Language Models (Llms)MicroservicesPythonRelational DatabasesSQL
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Architects, deploys, and operates CrowdStrike’s data platform infrastructure and services. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and access controls, developing DBT data models, cataloging assets in OpenMetadata, writing Python automation, and mentoring engineers. The role requires cloud, container orchestration, SQL, data modeling, compliance, and AI technology expertise, along with eligibility for CJIS clearance.
Top Skills:
AIApache AirflowApache SupersetAWSAzureCi/CdDbtDockerGCPIamKubernetesMachine Learning PipelinesOciOpenmetadataPythonSQLTerraform
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
