Lead design and implement cloud-based data platforms and GenAI solutions using AWS/Azure/GCP, Databricks, and Snowflake. Build scalable ETL/ELT pipelines, deploy ML/LLM models, define data governance and compliance, advise clients, manage delivery, and mentor cross-functional teams.
Title: Data and AI Architect (Lead/Manager Level)
Location: New York
Overview:
We are seeking a Data and AI Architect with a proven track record in designing and implementing data platforms and AI solutions using cloud technologies (AWS, Azure, GCP), Databricks, and Snowflake. The ideal candidate will combine technical expertise, hands-on experience, and leadership capabilities to deliver impactful solutions for our clients, including the development and deployment of GenAI solutions for next-generation business applications.
Key Responsibilities:
Solution Design and Implementation
- Architect and implement modern data platforms using cloud technologies such as AWS, Azure, and GCP.
- Build scalable and efficient data pipelines leveraging Databricks and Snowflake to support diverse client needs.
- Design and deploy AI/GenAI solutions, including machine learning models, natural language processing tools, and predictive analytics, to address complex business challenges.
Data Governance and Quality
- Develop data governance frameworks to ensure data quality, security, and compliance with industry regulations.
- Implement best practices for data lineage, monitoring, and cataloging using tools like Databricks and Snowflake.
- Work with clients to establish policies for managing data assets and ensuring compliance with GDPR, CCPA, and other regulatory requirements.
Client Engagement and Delivery
- Act as a trusted technical advisor to clients, translating business challenges into actionable data and AI strategies.
- Lead workshops, provide guidance on architecture decisions, and ensure alignment with client objectives.
- Manage solution delivery, including resource planning, timeline tracking, and adherence to budget constraints.
Technology Expertise and Innovation
- Stay current on advancements in cloud platforms, Databricks, Snowflake, and AI technologies to recommend innovative solutions.
- Explore and implement GenAI technologies to create transformative solutions such as personalized customer experiences, generative content creation, and intelligent automation.
- Contribute to the development of reusable frameworks and accelerators to enhance delivery efficiency.
Team Leadership and Development
- Provide technical leadership to cross-functional teams, ensuring high-quality delivery of data and AI projects.
- Mentor team members on cloud technologies, data platforms, and AI frameworks.
- Foster a collaborative environment to encourage innovation and continuous learning.
Qualifications:
- Experience: 6–10 years of experience in data architecture, engineering, or analytics, including client-facing or consulting roles.
- Cloud Expertise: Hands-on experience with AWS, Azure, and GCP, including data storage, compute, and analytics services.
- Data Platforms: Strong knowledge of Databricks and Snowflake, including experience with ETL/ELT workflows, performance tuning, and scaling.
- AI/GenAI Knowledge: Experience designing and deploying AI/GenAI solutions using frameworks like TensorFlow, PyTorch, Hugging Face, and advanced large language models (LLMs) such as OpenAI's GPT and Google's Gemini.
- Technical Skills: Proficiency in SQL, Python, Spark, and other data engineering and analytics tools.
- Leadership: Ability to lead technical teams and manage client engagements effectively.
- Education: Bachelor’s degree in Computer Science, Data Science, or a related field; advanced degree preferred.
Preferred Skills:
- Certification in AWS, Azure, or GCP architecture.
- Familiarity with tools such as dbt, Apache Airflow, and Kafka.
- Knowledge of customer data platforms and data-driven marketing strategies.
Compensation, Benefits and Duration
Minimum Compensation: USD 100,000
Maximum Compensation: USD 300,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Photon San Francisco, California, USA Office
San Francisco, United States
Similar Jobs
Information Technology
Sell CDW Security services and products across the full sales lifecycle. Develop strategic account plans, build relationships with enterprise clients and partners, manage pipeline and forecasts in Salesforce, coordinate cross-functional resources, mentor junior sellers, represent CDW at events, and achieve bookings, revenue, and profit targets while tracking market trends and driving customer security program adoption.
Top Skills:
AICloud SecurityData PrivacyEmerging Security TechnologiesIdentity And Access ManagementPhysical SecuritySalesforceSecure InfrastructureThird-Party Risk ManagementThreat And Vulnerability Management
Information Technology
Sell CDW Security services and products, manage full sales lifecycle, develop strategic account plans, build and grow client and partner relationships, coordinate internal resources, maintain Salesforce pipeline and forecasts, mentor junior sellers, represent CDW at industry events, and meet bookings, revenue, and profit targets.
Top Skills:
AICloud SecurityCyber Workforce DevelopmentEmerging Security TechnologiesIdentity And Access ManagementPhysical SecurityPrivacyRisk And ComplianceSalesforceSecure InfrastructureThird-Party Risk ManagementThreat And Vulnerability Management
Information Technology
Sell CDW Security services and products through full sales lifecycle management: strategic account planning, opportunity qualification, relationship building, partnering with services and OEMs, pipeline forecasting, and closing. Mentor junior team members, engage partners and marketing, maintain Salesforce, represent CDW at industry events, and drive revenue, bookings, and gross profit targets.
Top Skills:
Salesforce
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
