Define and validate legal AI behavior through prompt engineering, benchmark and evaluation design, dataset curation/annotation, and performance analysis. Collaborate with engineering, product, and customers to translate legal workflows into testable prompts, metrics, and features, and continually improve model accuracy and legal workflow quality.
This role sits at the intersection of legal practice and legal technology and plays an essential role in shaping the future of the product. In this role, you will bring a deep understanding of how attorneys work- their processes, use cases- to set the bar for how legal AI behaves, optimizing system prompts, designing evaluations, and ensuring every output aligns with real-world legal standards. You will be instrumental in measuring and continually improving the substantive accuracy and quality of the legal workflows and output the product produces.
By blending legal domain expertise, data analysis, and workflow modeling, you'll help ensure our AI tools are accurate, practical, and user-aligned. This hands-on role involves close collaboration with engineering, product, and design to define AI performance metrics, and requires designing and curating datasets reflecting how legal teams interact with documents.
Getting started- We want you to feel like part of the team early on! Our onboarding process will integrate you into the company with informative sessions on our product, policies, processes, and team structure and goals. [And you’ll spend meaningful time getting to know your direct reports and discover what excites and challenges them.]
- We’re excited for you to learn, grow, and contribute right away! We trust that you’ll bring experience and knowledge that will uplift and uplevel the team, but we don’t expect you to know everything on Day 1.
Prompt Optimization & Workflow Development:
- Systematically discover and test prompt engineering best practices to ensure the accuracy and reliability of AI outputs trusted by legal teams.
- Drive product enhancements by translating legal expertise into features.
- Collaborate with Engineering on LLM calls. Review and validate product outputs for quality.
- Translate client requests into concrete, testable prompts and evaluation criteria. Draft and iterate on system/developer prompts for AI systems.
Benchmark Design and Evaluation:
- Assess foundational model performance and suitability for various legal tasks.
- Collaborate with Product and customers to optimize genAI legal workflows. Develop datasets that accurately model legal tasks (e.g., document review, privilege detection), annotating data for edge cases and variability.
- Lead the creation of cutting-edge legal AI evaluations, designing and running rigorous tests to ensure outputs meet the high standards of legal professionals.
- Dataset curation and labelling for testing.
Data Analysis & Evaluation:
- Build and annotate complex legal datasets and document collections.
- Analyze model performance and iteratively improve AI solutions.
- Build metrics and reporting pipelines to monitor system quality and workflow performance.
- Analyze usage data to gauge effectiveness.
Collaboration:
- Serve as the legal voice of quality, advocating for precision, consistency, and clarity in AI outputs.
- Provide domain-specific legal expertise that guides feature development and product roadmap decisions.
- Act as the bridge between technical teams and legal domain expertise, translating complex legal concepts into clear product requirements and vice versa.
- Share findings with cross-functional stakeholders.
- Ideate new products based on your deep understanding of legal teams’ pain points.
- Participate in customer calls and feedback sessions.
- 3+ years of relevant legal experience, JD preferred.
- Exceptional written communication and a strong interest in the application of AI in the field of law.
- Familiarity with a coding/scripting language or coding agents (e.g., gemini-cli, claude code, codex-cli)
- Experience with generative AI in legal work, including evaluation of outputs, design/interpretation of metrics, and abstracting legal reasoning into repeatable steps.
- The ideal candidate is curious, detail-oriented, pragmatic, and able to bridge data science with product building, working cross-functionally to align data with user experience goals.
- The expected salary range for this role is between $163,000 - $206,000. The final offered salary will be dependent upon many factors including the candidate’s experience and skills. The base pay range is subject to change in the future.
- Equity program
- 401(k) retirement plan with company matching
- Health, dental, and vision
- Flexible Spending Accounts for health and dependent care expenses
- Paid parental leave and approximately 10 days (80 hours) per year of sick leave
- Seventeen paid vacation days plus 11 federal holidays
- Membership to Modern Health to help employees prioritize mental health and wellness
- Annual allocation for Learning & Development opportunities and applicable professional membership dues
- Company-sponsored life and disability insurance
- Find out more about our Benefits and Perks
- Work in Uptown Oakland, just steps from the BART line and dozens of restaurants and walking distance to Lake Merritt
- Flexible work-from-home days on Tuesdays and Fridays
- Monthly home internet reimbursement
- Select your preference of hardware (Mac or PC) and customize your desk setup
- Enjoy a wide variety of snacks and beverages in the office
- Bond over company-wide out-of-the-box events and fun activities with your team
- Time off for company-sponsored volunteer events and 4 paid hours per quarter to volunteer at a charitable organization of your choice
- Take advantage of learning and career development opportunities
- Ranked #9 on Glassdoor's Best Places to Work 2023 for US small and medium companies
- One of Wealthfront’s 2021 Career Launching Companies, and ranked #2 on the “2022 Bay Area Best Places to Work” list by the San Francisco Business Times and the Silicon Valley Business Journal
- One of Fast Company’s World's Most Innovative Companies for 2022 and proud contributor of free ediscovery resources to benefit the greater good through “Everlaw for Good”
- #LI-KP1
- #LI-Hybrid
Pursue Truth While Finding Yours
At Everlaw, we are deeply invested in pursuing the truth, for our clients and for our employees. We know that when you’re empowered to pursue your passions, it is reflected in the work. That’s why we’re committed to the professional growth of all our team members, offering an annual learning and development stipend and regular career check-ins with managers. If you’re looking for a place that values passion, integrity, and a desire to learn, we’d love to hear from you!
About Everlaw
We help law firms, government agencies, and corporations sift through millions of documents of evidence in big lawsuits and investigations to find the proverbial smoking gun (or needle in the haystack -- pick your metaphor). It's a multi-billion dollar space typically dominated by service-oriented vendors, and we're coming at it with cutting-edge technology and elegant design. It's working, and we've been growing very rapidly: we host hundreds of terabytes of data and work with all 50 state Attorneys General and hundreds of law firms on some of the most high-profile cases litigated today.
Everlaw is an equal opportunity employer. We pride ourselves on having a diverse workforce and we do not discriminate against any employee or applicant because of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition, or any other basis protected by law. We respect the gender, gender identity and gender expression of our applicants and employees, and we honor requests for pronouns. It is our policy to comply with all applicable national, state and local laws pertaining to nondiscrimination and equal opportunity, including the California Equal Pay Act. Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
We collect and process the personal information you provided along with your job application in accordance with our Applicants Privacy Notice and Notice at Collection.
When preparing to engage with Everlaw as a candidate, you may use AI tools for research, polishing application materials, and interview prep. However, any assessments (unless explicitly stated), remote interviews or live interviews must be completed independently without AI support. By submitting your application, you agree to adhere to these rules. Here's the link to our full policy, and please reach out with any questions!
We use Covey as part of our hiring and/or promotional processes. As part of the evaluation process, we provide Covey with job requirements and candidate-submitted applications. Certain features of the platform may qualify it as an Automated Employment Decision Tool (AEDT) under applicable regulations. For positions in New York City, our use of Covey complies with NYC Local Law 144. We began using Covey Scout for Inbound on the 9th of June, 2025.
Everlaw Oakland, California, USA Office
2101 Webster St, 1500, Oakland, CA, United States, 94612
Similar Jobs
Information Technology • Consulting
The Senior Legal Engineer leads customer implementations, advises on contracting processes, manages project timelines, and collaborates with cross-functional teams to enhance customer satisfaction.
Top Skills:
AIClmContracting Lifecycle ManagementSaaS
Artificial Intelligence • Internet of Things • Machine Learning • Software • Analytics • Industrial • Manufacturing
The Revenue Operations Manager will manage systems and processes that align sales, marketing, and leadership, ensuring data integrity and pipeline management while optimizing the commercial operations' tech stack.
Top Skills:
Ai ToolsCRMGitHubspot
Information Technology • Sales • Security • Cybersecurity • Automation
The Area Vice President will manage sales in the Eastern US, leading a team, driving sales growth, and building customer relationships while ensuring high performance and strategic collaboration across teams.
Top Skills:
CRMIdentity And Access ManagementIdentity SecurityItdrMfaNhiPam
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
