Lead big data projects employing AWS and various programming languages to enhance data engineering, security, and ETL processes. Ensure adherence to project standards and timelines while fostering team collaboration and resource management.
Strong experience in programming languages of SAS, PL/SQL,Unix shell scripts, Python. Proficiency in Amazon Web Services (AWS) architecture and technologies such as S3, Redshift, RDS, Glue (Catalog and ETL), EMR, Lambda, Step Functions, SQS/SNS, and ECS, CloudWatch, Secrets Manager among others Experience in one or more of the following software skills: API Gateway, JSON, PARQUET, XML, Docker, Urban Code, Jenkins, GITLab, Terraform Strong experience in data engineering, data warehousing, data security and ETL processes Experience creating and using APIs for developing or programming software Strong SQL Experience Strong analytical and problem-solving skills Ability to drive code reviews with peers and managers to ensure that each increment adheres to original vision as described in the user story and all standard resource libraries and architecture patterns as appropriate 5+ years' AWS cloud-based technology services experience Must be a self-starter and motivated to learn new technologies, skills, and knowledge required by duties and responsibilities in required timeframe Adept at leading resources, and people to ensure successful on schedule project completion
Top Skills
Api Gateway
AWS
Cloudwatch
Docker
Ecs
Emr
Gitlab
Glue
Jenkins
JSON
Lambda
Parquet
Pl/Sql
Python
Rds
Redshift
S3
SAS
Secrets Manager
Sns
SQL
Sqs
Step Functions
Terraform
Unix Shell Scripts
Urban Code
XML
Similar Jobs
Information Technology • Consulting
Lead the development of data pipelines and transformations in Azure Databricks, converting Scala programs to PySpark while leveraging various Azure technologies.
Top Skills:
AdfAzure Data Lake Gen 2Azure DatabricksDelta LakePysparkPythonSparkSynapse Analytics
Information Technology • Consulting
The Senior Azure Data Engineer will manage data ETL processes, mentor junior staff, and work on cloud technologies in Azure. Required experience includes Datawarehouse expertise.
Top Skills:
AdfAzureDatabricksDatawarehousePysparkPython
Information Technology • Consulting
The Big Data Lead will manage database development, ETL/ELT processes, and data warehousing, optimizing performance and ensuring data pipelines work reliably.
Top Skills:
AWSAws GlueAws S3AzureAzure BlobAzure Data FactoryAzure DevopsGitJenkinsOracleSnowflakeSQLTalendTeamcity
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
