Architect and operate secure, compliant AWS data pipelines and analytics integrations. Ensure data quality, lineage, residency, and privacy-preserving engineering to meet HIPAA, SOC 2, GDPR, and CCPA. Lead data modeling, build CI/CD for data infrastructure, enforce least-privilege access, implement IaC governance, mentor engineers, perform code reviews, and produce runbooks and compliance evidence.
Position Summary
The Sr Analytics Engineer architects, builds, and operates secure, scalable data pipelines and analytics integrations within a governed AWS cloud environment. This role carries direct accountability for data quality, privacy-preserving engineering, and compliance with HIPAA, SOC 2, GDPR, and CCPA. The individual serves as the technical authority on analytics engineering standards and partners with Security, Legal, and Compliance teams to ensure all data products meet enterprise governance requirements.
AWS Technology Stack
Deep hands-on expertise required across the following AWS service areas:
- Data Storage & Lake: S3, Lake Formation, Glue Data Catalog, DynamoDB
- Processing & ETL: Glue (PySpark), Lambda, Step Functions, EMR
- Analytics & Query: Athena, Redshift / Redshift Serverless, QuickSight
- Integration & Streaming: Kinesis, API Gateway, Transfer Family, AppFlow
- Security & Encryption: KMS, IAM, Secrets Manager, Macie, GuardDuty, CloudTrail
- Monitoring & Compliance: CloudWatch, AWS Config, X-Ray, Systems Manager
- Networking & Isolation: VPC, PrivateLink, AWS Clean Rooms
Governance & Compliance Responsibilities Regulatory Compliance
- Design data architectures compliant with HIPAA (PHI encryption, minimum necessary access, audit retention), GDPR/CCPA (DSAR automation, right-to-erasure, consent management), SOC 2 Type II (access reviews, change management, continuous monitoring), and PCI DSS where applicable.
- Implement data residency and sovereignty controls ensuring data remains within approved AWS Regions.
Data Governance & Quality
- Enforce data classification standards (Public, Internal, Confidential, Restricted) via Macie and Lake Formation LF-Tags. Implement automated data quality frameworks with freshness, completeness, and accuracy SLAs.
- Maintain data lineage documentation, data retention/purge policies via S3 Lifecycle Rules and DynamoDB TTL, and participate in Data Governance Council reviews and compliance audits.
Security & Access Governance
- Implement least-privilege access models with quarterly reviews, IAM Access Analyzer scans, and encryption-at-rest/in-transit standards with KMS key policy management.
- Design masking, tokenization, and anonymization pipelines for non-production environments. Enforce IaC governance via CloudFormation/Terraform with security review gates and drift detection.
Responsibilities
Key Responsibilities
- Architect production-grade, fault-tolerant batch and real-time data pipelines within a compliance-governed AWS environment.
- Lead data modeling (dimensional, SCD, data vault) and build API/file-based integrations with encryption and audit logging at every point.
- Build CI/CD pipelines for data infrastructure with mandatory security scanning and approval gates.
- Conduct code reviews, mentor engineers, and produce technical documentation including ADRs, runbooks, and compliance evidence packages.
Qualifications
Qualifications Required
- Bachelor’s degree in Computer Science, Data Engineering, or related field; 7+ years in Analytics/Data Engineering.
- Expert SQL and Python (PySpark, Pandas, Boto3); deep AWS experience across the data, security, and governance stack.
- Demonstrated experience implementing governance frameworks and regulatory compliance controls (HIPAA, SOC 2, GDPR, CCPA) in production.
- Experience with IaC (CloudFormation, Terraform, CDK), CI/CD for data pipelines, and data security principles.
Preferred
- AWS certifications (Data Analytics Specialty, Security Specialty, Solutions Architect Professional).
- Experience with data quality frameworks (Great Expectations, dbt, Monte Carlo), and regulated industries (healthcare, financial services).
EXL Richmond, California, USA Office
Richmond, United States
Similar Jobs
Fintech • Financial Services
Design, build, and maintain scalable dimensional data models, semantic/metrics layers, and production data pipelines (dbt, Snowflake). Partner with Data Science, Data Engineering, BI, and DevOps to enable AI/ML, real-time analytics, governance, and data quality while mentoring junior engineers and enforcing best practices.
Top Skills:
CortexDatafoldDbtFivetranKafkaLlmPythonRagRedshiftSnowflakeSnowflake IntelligenceSQLStreamkapTableau
Fintech • Financial Services
Design, build, and maintain dimensional data models, semantic layer, and production data transformation pipelines (dbt/Snowflake). Partner with Data Engineering, Data Science, BI, and Product to deliver governed, AI-ready data products, monitor data quality, enable ML features, and coach junior engineers to uphold code and governance standards.
Top Skills:
CortexDatafoldDbtFivetranKafkaPythonRagRedshiftSnowflakeSnowflake IntelligenceSQLStreamkapTableau
Marketing Tech • Software
As the first Data Engineer, you will architect the Snowflake governed query layer, enable AI-driven tools, and define data access standards for various departments, significantly shaping Knak's data infrastructure and analytics capabilities.
Top Skills:
DbtGitPythonSnowflakeSQLTableau
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
