Senior Data Engineer
BigCommerce is disrupting the e-commerce industry as the SaaS leader for fast-growing, mid-market businesses. We enable our customers to build intuitive and engaging stores to support every stage of their growth.
BigCommerce named a “Best and Brightest” place to work in San Francisco 2020 is looking for a Senior Data Engineer. This is an exciting opportunity to build a real time big data pipeline and SaaS analytics platform using latest cloud technologies for the #1 e-commerce SaaS platform.
An ideal candidate will have a background in Computer Science or a related technical field and possess extensive experience in building distributed scalable data pipelines and working with large data stores. You will be working with cross-functional teams in a highly visible role. You are gritty, focused on results, a self-starter and have demonstrated success in using engineering and analytics to drive the understanding, growth and success of a product.
What you’ll do
- Build scalable and reliable near real time data pipeline on cloud that collects, transforms, loads and curates data from various internal and external data sources
- Build a scalable distributed data store that will be central source of truth
- Experience in developing high performance, scalable and available web services
- Own data quality for the pipelines you build and make them auditable
- Evaluate new technologies and build prototypes for continuous improvements in Data Engineering
- Build integrations with external systems for transmitting and receiving event data
- Partner with Infrastructure and Engineering teams to ensure instrumentation, logging and monitoring is in place
Who you are
- Experience in using big data technologies such as Kafka, Spark, Hadoop, HBase or their equivalents
- Experience with AWS and/or GCP
- 5+ years of experience with Java, Scala and/or Python
- 5+ years of experience with SQL
- 3+ years of experience in building and monitoring near real time scalable ETL pipelines
- Experience with shell scripting
- Excellent written and verbal communication skills
- Bachelor's degree in CS, EE or MIS; or equivalent experience
Nice to have
- Experience with tools such as AirFlow for scheduling and monitoring of data workflow will be a plus
- Experience with managed data processing services of cloud platforms such as GCP and/or AWS
- Experience working with a team distributed over time zones
- Knowledge of Google Kubernetes Engine, Cloud Composer
Diversity & Inclusion at BigCommerce
We have the opportunity to build not only a great business, but a great company, with soul. Our beliefs and commitment to diversity and inclusion are a central part of achieving that.
Our dedication to diversity and inclusion is grounded in two things: a moral belief in the dignity, value, and potential of every individual, and a practical belief that diverse, inclusive teams will create the best outcomes for our customers, partners, employees, and company. We welcome everyone to be a part of our journey.
Current BigCommerce Employees: Please use the internal job board to apply for openings