Lead architecture and delivery of real-time stream-processing pipelines using Apache Flink and Java. Build scalable, low-latency event-driven systems, optimize performance and fault tolerance, collaborate with cross-functional teams, mentor engineers, conduct design reviews, troubleshoot production, and guide cloud and containerized deployments with robust CI/CD and observability practices.
This role is for one of the Weekday's clients
Min Experience: 10 years
Location: USA
JobType: full-time
As a Lead Engineer, you will play a pivotal role in architecting and delivering complex stream-processing solutions, mentoring engineering teams, and collaborating closely with cross-functional stakeholders to translate business requirements into robust technical systems.
RequirementsKey Responsibilities
- Lead the architecture, design, and implementation of real-time data processing pipelines using Apache Flink.
- Develop and maintain high-performance backend services and distributed systems using Java.
- Design scalable event-driven architectures capable of handling high-throughput and low-latency workloads.
- Optimize streaming jobs for performance, fault tolerance, and resource efficiency.
- Ensure best practices in code quality, testing, observability, and CI/CD processes.
- Collaborate with data engineering, DevOps, and product teams to define technical roadmaps and system requirements.
- Conduct design reviews, troubleshoot production issues, and implement long-term reliability improvements.
- Mentor and guide engineers, fostering a culture of technical excellence and continuous improvement.
- Contribute to infrastructure decisions related to distributed processing, cloud deployment, and containerized environments.
- 10–12 years of overall experience in software engineering, with significant exposure to distributed systems.
- Strong hands-on expertise in Apache Flink, including stream processing concepts such as windowing, state management, checkpoints, and event-time processing.
- Advanced proficiency in Java, including concurrency, multithreading, memory management, and performance tuning.
- Deep understanding of data streaming architectures and real-time processing frameworks.
- Experience working with messaging systems (e.g., Kafka or similar platforms).
- Strong knowledge of data structures, algorithms, and system design principles.
- Experience deploying and managing applications in cloud environments (AWS, Azure, or GCP).
- Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes.
- Solid understanding of CI/CD pipelines, automated testing frameworks, and monitoring tools.
- Experience with SQL and NoSQL databases in high-scale environments.
- Proven experience leading engineering teams or owning major technical initiatives.
- Strong architectural decision-making abilities with a focus on scalability and maintainability.
- Excellent problem-solving and analytical skills.
- Ability to communicate complex technical concepts to both technical and non-technical stakeholders.
- Strong ownership mindset and commitment to delivering high-quality solutions.
- Experience with big data ecosystems and real-time analytics platforms.
- Exposure to performance benchmarking and capacity planning.
- Experience working in Agile/Scrum environments.
Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Lead end-to-end design, implementation, deployment, and operation of production AI/ML-enabled full-stack applications. Build LLM-powered features, design scalable backend services and data models, deploy on Azure, implement CI/CD and observability, and mentor engineers while ensuring secure, compliant integration into clinical workflows.
Top Skills:
Ai Frameworks And ApisAuthentication/AuthorizationAutomated TestingAzure Ai ServicesAzure App ServicesAzure ContainersAzure FunctionsAzure StorageCi/CdContainersLlmsAzureNext.JsNode.jsPostgresReactRestful ApisTypescript
Fintech • Machine Learning • Payments • Software • Financial Services
The Lead Front End Engineer will build and manage the marketing website, ensuring performance, scalability, and collaboration with design and marketing teams while optimizing workflows and ensuring accessibility standards.
Top Skills:
AWSContentfulFigmaGCPAzureNext.JsReactShadcn UiTailwind.CssTypescript
Fintech • Machine Learning • Payments • Software • Financial Services
Lead the development and maintenance of Capital One's marketing website, focusing on performance, collaboration with stakeholders, and continuous optimization of front-end technologies.
Top Skills:
AWSCi/CdContentful CmsFigmaGCPAzureNext.JsReactShadcn UiTailwind.CssTypescript
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
