Design and implement data integration and data warehouse solutions using big data technologies. Build cost-efficient, performant cloud data pipelines, apply data modeling and machine learning knowledge, and implement real-time streaming and aggregation patterns.
Role Big Data Engineer Responsibilities Implement Data integration and Data Warehouse based solutions using Big Data Technologies. Should be highly proficient in the use of Big Data / Open Source Technologies and standard techniques of Data Integration, Data Manipulation with hands-on contribution. Should be able to develop cost efficient and performant data pipelines in the cloud platform In-depth understanding of modern big data technology, including Data modeling and machine learning skills Knowledge of real time data streaming and aggregation architectural patterns and practice Essential Skills: 3+ Years hands on knowledge on SQL as well as SQL/NoSQL databases Proficient in programming languages such as Python and PySpark/Scala/Java
Hexaware Technologies San Ramon, California, USA Office
3001 Bishop Drive Suite 300, Office 3108, San Ramon, United States, 94583
Similar Jobs
Automotive • Big Data • Insurance • Software • Transportation
Designs, builds, and maintains scalable ETL/ELT pipelines and finance data models using Snowflake, AWS, dbt Core, Python, and SQL. Develops Medallion architecture data marts, integrates ERP systems, and partners with stakeholders on financial workflows. Implements data governance, RBAC, automated quality testing, observability, documentation, CI/CD, and secure deployments. Monitors Snowflake costs and optimizes queries while supporting Oracle EBS integrations and the migration to Workday.
Top Skills:
AWSCi/CdDbt CoreGitOracle EbsPythonSnowflakeSQLWorkday
Information Technology • Consulting
Build and maintain ETL and data pipelines using Python, PySpark, and AWS services. Orchestrate workflows, develop event-driven integrations, optimize data storage and queries, process API and JSON data, and implement data quality monitoring. Support CI/CD, production operations, cloud migration, data lake and lakehouse initiatives, and near-real-time systems while collaborating with stakeholders to deliver technical solutions.
Top Skills:
Amazon AthenaAmazon CloudwatchAmazon DynamodbAmazon EmrAmazon KinesisAmazon RdsAmazon RedshiftAmazon S3Amazon SnsAmazon SqsApache AirflowAPIsAws GlueAws LambdaAws Step FunctionsCi/CdData LakesData WarehousingGithub CopilotGitlabJSONLakehouseMcpOraclePysparkPythonSQLTerraform
Social Media
Lead Pinterest’s technical strategy and roadmap for scalable big data and AI infrastructure. Build frameworks for compute, job, resource, scheduling, and shuffle management across petabyte-scale datasets. Partner with internal customers, provide company-wide technical leadership, and improve data processing reliability, speed, and efficiency.
Top Skills:
AWSFlinkGoJavaKubernetesPythonPyTorchRayScalaSparkTensorFlow
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
