Designs, builds, documents, tests, and maintains large-scale data pipelines across platforms and sources. Performs ETL and ELT while ensuring data governance, security, quality, validation, anomaly detection, and compliance. Collaborates with architects, analysts, and stakeholders to translate business needs into technical solutions, develops data models, and mentors junior data engineers.
The Sr Data Engineer will design and manage data systems, pipelines, and management tools. They will oversee junior data engineers, recommend and deploy data models, and lead projects involving data collection and storage systems. This role will design, build, document, test, and maintain critical data pipelines from large volumes of data from internal and external data sources cross different platforms.PRIMARY RESPONSIBILITIES:
- Collaborate with data architect, analysts, and stakeholders to identify and document needs/requirements for data pipelines and process automation
- Design, build, document, test, and maintain data pipelines using industry best practices
- Performs ETL, ELT operations in accordance with enterprise data governance and security standards
- Develop data quality and governance process to ensure the accuracy and quality of the data through inspection, validation, processing, anomaly detection
- Detail oriented and proactive in taking measures to avoid or mitigate issues.
- Excellent understanding of computer systems, integrations, databases, and data storage systems.
- Understanding of agile data engineering concepts and processes
- Ability to capture and document business processes and requirements and work with colleagues to create functional requirements.
- Ability to work with clients to translate business needs to technical solutions
- Ability to provide guidance, mentoring, and training for peers
- Exceptional interpersonal skills and ability to effectively communicate with teams across the entire organization.
- Demonstrates the organization’s core values, exuding behavior that is aligned with the firm’s culture
- Proficiency in Microsoft Office Suite (Word, Excel, PowerPoint, Outlook)
- Ability to learn any other appropriate program or software system used by the firm as necessary
- 5-7 years of relevant experience required
- BA/BS in relevant discipline required
- Vast experience with multiple development methodologies
- Experience of building large, complicated data pipelines cross different platforms, data sources, data structures
- Experience of working with relational databases such as SQL Server, Oracle, etc., and SQL scripting
- Experience of cloud platforms (like AWS, Azure, GCP) and its ETL tools and techniques of sourcing, maintaining, and updating data, data warehousing, data cleansing & transformation, etc.
- Programming experience in Python, Spark, and other similar languages
- Advanced programming knowledge and experience with large-scale processing engines are required, as well as a deeper understanding of database security and compliance tools.
- This position description is intended to describe the level of work required of the person performing in the role and is not a contract. The essential responsibilities are outlined; other duties may be assigned as needs arise or as required to support the organization. All requirements may be subject to reasonable accommodation to applicants and colleagues who need them for medical or religious reasons.
#LI-RL1
IND1
Click here for some insight into our culture!
The Baldwin Group will not accept unsolicited resumes from any source other than directly from a candidate who applies on our career site. Any unsolicited resumes sent to The Baldwin Group, including unsolicited resumes sent via any source from an Agency, will not be considered and are not subject to any fees for any placement resulting from the receipt of an unsolicited resume.
Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Design, build, and deploy AI/ML and generative AI solutions and full-stack applications. Develop frontend experiences, backend services, microservices, APIs, and cloud-native data integrations. Implement RAG, LLMs, vector DBs, dashboards, reporting, and self-service analytics to support quality, patient safety, and workflow automation. Provide technical leadership, mentor engineers, and apply CI/CD, testing, observability, security, and performance best practices.
Top Skills:
AgentsApache SupersetAutomated TestingAWSAzureAzure Ai ServicesCi/CdDatabricksEvent-Driven ArchitectureGCPGenerative AiJavaJavaScriptLangchainLlmsMicroservicesMicrosoft FabricObservabilityOpenaiPower BIPrompt EngineeringPythonRagReactRest ApisSemantic KernelSnowflakeSpring BootSQLTypescriptVector Databases
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Lead the strategic and operational direction of a specialized casualty claims team managing severe-exposure portfolios. Oversee claims managers, complex litigation, coverage strategy, reserves, settlements, legal vendors, and portfolio analytics. Collaborate with legal, underwriting, actuarial, reinsurance, and executive stakeholders to manage emerging risks, litigation trends, loss costs, and claim outcomes. Develop protocols and best practices for environmental, mass tort, toxic tort, latent injury, and other complex claims.
Top Skills:
Guidewire Claims System
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Build and operate AWS GovCloud data infrastructure across development, preproduction, and production environments. Establish PostgreSQL platforms, infrastructure-as-code with Chef, BI infrastructure, identity integrations, monitoring, observability, CI/CD, and disaster recovery. Support data pipelines, troubleshoot incidents, optimize databases, and ensure FedRAMP and FISMA compliance. Collaborate with security, compliance, DevOps, and data engineering teams while delivering greenfield infrastructure from architecture through production.
Top Skills:
AWSAws GovcloudBashChefCi/CdCloudwatchDatadogEltETLGitGitlabGoogle SamlJenkinsNagiosPostgresPythonRubySsoTableau ServerVpc
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine



