Design, build, and optimize data pipelines and analytical reports. Document requirements and data lineage, debug and test big-data workflows, collaborate with product and engineering partners, and translate business needs into code and insights.
Title: Senior Data Engineer
Location: Sunnyvale, CA
Position: 2 Month (Contract)
Summary
Provide business insights, while leveraging internal tools and systems, databases and industry data
Roles & Responsibilities
- Ability to document requirements, data lineage, subject matter in both business and technical terminology.
- Guide and learn from other team members.
- Demonstrated ability to transform business requirements to code, specific analytical reports and tools
- This role will involve coding, analytical modeling, root cause analysis, investigation, debugging, testing and collaboration with the business partners, product managers other engineering team.
Minimum Qualifications
- Very Strong engineering skills. Should have an analytical approach and have good programming skills.
- Minimum of 5+ years’ experience. Experience in retail business will be a plus.
- Excellent written and verbal communication skills for varied audiences on engineering subject matter
Must Have
- Strong analytical background
- Self-starter
- Must be able to reach out to others and thrive in a fast-paced environment.
- Strong background in transforming big data into business insights
Technical Requirements
- Knowledge/experience on Teradata Physical Design and Implementation, Teradata SQL Performance Optimization
- Experience with Teradata Tools and Utilities (FastLoad, MultiLoad, BTEQ, FastExport)
- Advanced SQL (preferably Teradata)
- Experience working with large data sets, experience working with distributed computing (MapReduce, Hadoop, Hive, Pig, Apache Spark, etc.).
- Strong Hadoop scripting skills to process petabytes of data
- Experience in Unix/Linux shell scripting or similar programming/scripting knowledge
- Experience in ETL/ processes
- Real time data ingestion (Kafka)
- BS degree in specific technical fields like computer science, math, statistics preferred
Education
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Lead design and operation of large-scale data pipelines and data APIs. Build Spark/PySpark workflows on Databricks, optimize job performance, manage data quality and observability, and develop MCP servers and AI-agent integrations. Mentor engineers, define standards, support production incidents and on-call rotations, and collaborate with stakeholders to deliver scalable data platform products.
Top Skills:
Apache IcebergApi GatewayAWSAws LambdaAws Rds/AuroraAzureDatabricksDatadogDbtFastapiFivetranGCPGoogle BigqueryLlms/Ai AgentsMcp ServersMs Sql ServerMySQLOraclePostgresPysparkPythonS3SecretsmanagerSnowflakeSnsSparkSplunkSQLSqs
.png)
Cloud • Insurance • Payments • Software • Business Intelligence • App development • Big Data Analytics
Design, build, and optimize BigQuery-based data solutions and data models. Manage data storage, partitioning, clustering, ETL, and data quality. Collaborate with architects, data scientists, and engineers to deliver scalable, secure cloud data infrastructure and documentation; senior hires provide technical leadership, code review, and process improvement.
Top Skills:
Ansi Sql)BigQueryConfluenceGoogle Cloud Platform (Gcp)JIRAPythonSql (Bigquery Sql
Cloud • Information Technology • Machine Learning
Own and evolve the data lake and analytics stack powering fleet observability. Build and maintain scalable ETL/ELT pipelines, operate data lake infrastructure (Iceberg, Trino, Airflow, Spark, Superset), produce analytics and executive reporting, create dashboards and runbooks, and implement data governance and observability.
Top Skills:
Apache AirflowApache HudiApache IcebergSparkApache SupersetAWSAzureDelta LakeEtl/EltGCPJavaManaged DatabasesNoSQLObject StoragePythonScalaSQLTrino
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
