Design, develop, and manage data infrastructure for internal and client projects, focusing on large-scale data pipelines, APIs, and cloud deployments.
Object Computing, Inc. (OCI)
OCI is seeking experienced Data Engineers to design, develop, and manage data infrastructure for internal and client projects within our AI/ML & Data Insights team.
About OCI
We provide a collaborative environment focused on innovation, leveraging open-source software and strategic partnerships with Amazon and Google to build transformative technology solutions.
Job Responsibilities
- Design, build, and optimize large-scale data pipelines (Azure, AWS, GCP)
- Develop data ingestion and storage solutions.
- Implement scalable APIs and ensure system performance.
- Manage big data infrastructure and cloud deployments.
- Collaborate with developers, designers, and data scientists.
- Work in an Agile/DevOps environment.
Core Competencies
- ETL data engineering (Databricks, SQL Server, Snowflake, BigQuery, Apache Spark).
- Proficiency in Go, Java, Python, or Scala.
- Proficiency in CI/CD pipelines and Infrastructure as Code (IaC) (Terraform, CDK, Ansible).
- Hands-on experience with event-driven architectures (Kafka, Pulsar).
- Strong knowledge of data warehousing, SQL/NoSQL databases, and cloud platforms.
- Experience with distributed computing, DevOps tools, and data governance.
- Familiarity with Delta Lake, Unity Catalog, Delta Sharing, and DLT.
Preferred Qualifications
- Degree in Computer Science, Data Engineering, or a related field, or equivalent experience.
- Experience with AI/ML-driven data solutions and real-time data processing.
- Expertise in building scalable APIs and integrating with modern analytics tools (Power BI, Tableau, QuickSight).
- Cloud certifications (Databricks, AWS, Azure, GCP).
Top Skills
Ansible
Spark
AWS
Azure
BigQuery
Cdk
Databricks
Delta Lake
Delta Sharing
GCP
Go
Java
Kafka
NoSQL
Pulsar
Python
Scala
Snowflake
SQL
SQL Server
Terraform
Unity Catalog
Similar Jobs
eCommerce • Information Technology • Retail • Industrial
As a Senior Data Engineer, you will design and maintain scalable data pipelines, develop data products, and collaborate across teams to deliver insights.
Top Skills:
SparkAws GlueCloudFormationDockerGithub ActionsKafkaKubernetesPostgresPythonSap S/4HanaScalaSnowflakeSQLTerraform
AdTech • Digital Media • Marketing Tech • Software • Automation
The Sr Data Engineer will design, implement, and maintain deployment and ETL pipelines while integrating various data sources and ensuring efficient developer experiences.
Top Skills:
Apache AirflowArgo Ci/CdArgo WorkflowsBazelCircle CiDockerFlywayGitHarnessJavaJenkinsKubernetesLookerPower BIPythonSnowflakeSQLThoughtspot
Healthtech • Insurance • Software
As a Software Engineer, you will design and operate frameworks for data processing, build internal APIs, and ensure data security. You will collaborate on systems that handle real-time and batch workloads efficiently.
Top Skills:
Auth0Cloud RunCloud StorageDockerGCPGoIamIcebergK NativeKubernetesPostgresPub/SubPythonRest ApisSQLTrino
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
