The Staff Data Engineer will architect and build data pipelines, manage data strategy, ensure data quality, and mentor teams.
About PayJoy
PayJoy, a Public Benefit Corporation, is a mission-first credit provider dedicated to helping under-served customers in emerging markets to achieve financial stability and success. Our patented technology for secured credit provides an on-ramp for new customers to enter the credit system. Through PayJoy’s point-of-sale financing and card offerings, customers gain access to a modern quality of life. PayJoy’s credit also allows our customers to seize opportunities as micro-entrepreneurs, and acts as insurance for tough times. Through our cutting-edge machine learning, data science, and anti-fraud AI, we have served over 18 million customers as of 2025 while achieving solid profitability for sustainable growth.
This role
The Staff Data Engineer is responsible for ensuring that the organization has reliable, accessible, and well-organized data by designing and maintaining systems that process information in real time and in batches, defining a data strategy aligned with business goals, ensuring data quality and security, optimizing database performance, automating processes to increase efficiency, overseeing monitoring and rapid issue resolution, and providing leadership and guidance to other teams to ensure best practices in data usage and management
Responsibilities
- Architect and Build Data Pipelines: Build, optimize, and maintain reliable, scalable, and efficient data pipelines for both batch and real-time data processing.
- Data Strategy: Develop and maintain a data strategy aligned with business objectives, ensuring data infrastructure supports current and future needs.
- Streaming Expertise: Lead the development of real-time ingestion pipelines using Kafka/Kinesis, and design data models optimized for streaming workloads.
- Data Quality & Governance: Implement data quality checks, schema evolution, lineage tracking, and compliance using tools like Unity Catalog and Delta Lake etc.
- Tool & Technology Selection: Evaluate and implement the latest data engineering tools and technologies that will best serve our needs, balancing innovation with practicality.
- Automation and CI/CD: Drive automation of pipeline deployments, testing and monitoring using Terraform, CircleCi or similar tools.
- Performance Tuning: Regularly review, refine, and optimize SQL queries across different systems to maintain peak performance. Identify and address bottlenecks, query performance issues, and resource utilization. Setup best practices and work with developers on education of what they should be doing in the software development lifecycle to ensure optimal performance.
- Database Administration: Manage and maintain production AWS RDS MySQL, Aurora and postgres databases. Perform routine database operations, including backups, restores, and disaster recovery planning. Monitor database health, diagnose and resolve issues in a timely manner.
- Knowledge and Training: Serve as the primary point of contact for database performance and usage related knowledge, providing guidance, training, and expertise to other teams and stakeholders.
- Monitoring & Troubleshooting: Implement monitoring solutions to ensure high availability and troubleshoot data pipeline issues in real-time.
- Documentation: Maintain comprehensive documentation of systems, pipelines, and processes for easy onboarding and collaboration.
- Mentorship & Leadership: Mentor other engineers, review PRs, and establish best practices in data engineering.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.
- 12+ years of experience in data engineering, with at least 3+ years working in Databricks.
- Deep hands-on experience with Apache Spark (PySpark/SQL), Delta Lake, and Structured Streaming.
- Technical Expertise: Deep understanding of data engineering concepts, including ETL/ELT processes, data warehousing, big data technologies, and cloud platforms (e.g., AWS, Azure, GCP).
- Strong proficiency in Python, SQL, and data modeling for both OLTP and OLAP systems.
- Architectural Knowledge: Strong experience in designing and implementing data architectures, including real-time data processing, data lakes, and data warehouses.
- Tool Proficiency: Hands-on experience with data engineering tools such as Apache Spark, Kafka, Databricks, Airflow, and modern data orchestration frameworks.
- Innovation Mindset: A track record of implementing innovative solutions and reimagining data engineering practices.
- Experience with Databricks Workflows, Delta Live Tables (DLT), and Unity Catalog.
- Familiarity with stream processing patterns (exactly-once semantics, watermarking, checkpointing)
Benefits
- 100% Company-funded health insurance for employee and immediate family
- Company-funded employee life and disability insurance
- Paid vacation days, unlimited sick leave
- $2,000 USD annual Co-working Travel perk
- $2,000 USD annual Professional Development perk
- Phone finance, headphone benefit, home office equipment allowance and wellness perks
- Catered lunches
- Commuter benefit
PayJoy is proud to be an Equal Employment Opportunity employer and we welcome and encourage people of all backgrounds. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.
PayJoy Principles
Finance for the next billion * Ownership * Break Through Walls * Live Communication * Transparency & Directness * Focus on Scale * Work-Life Balance * Embrace Diversity * Speed * Active Listening
PayJoy San Francisco, California, USA Office
655 4th St, San Francisco, CA, United States, 94107
Similar Jobs
AdTech • Artificial Intelligence • Marketing Tech • Software • Analytics
The Staff Data Engineer will design and implement a unified semantic data layer, integrating heterogeneous data sources and enabling AI interactions while ensuring security and compliance across the organization.
Top Skills:
AerospikeAmazon S3Amazon SqsApache KafkaBigQueryDelta LakeDynamoDBGraphQLIcebergMongoDBMySQLPostgresRedshiftRestSnowflake
Cloud • Information Technology • Machine Learning
The Staff Business Systems Engineer will manage systems supporting Data Center operations, focusing on capacity planning, lifecycle management, integrations, and asset tracking across infrastructure and finance. Responsibilities include architecture design, data modeling, and ensuring operational compliance.
Top Skills:
CmdbDcimEamErpNetSuiteOracleSAPServicenowSunbird
Cloud • Information Technology • Security • Software • Cybersecurity
The Staff Data Center Operations Engineer will provision hardware, troubleshoot build environments, and collaborate with various teams to optimize processes for Zscaler's global cloud.
Top Skills:
AnsibleBashLinuxPxePythonUnix
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
