AI Legal Data Engineer

Scrutinize
Scrutinize

Other Engineering, Data Science · Part-time, Contractor

Remote

USD 75-100 / hour

Posted on Nov 19, 2026

About Scrutinize

Scrutinize uses data and AI to make state courts transparent and accountable. We are the only organization publishing data-driven records of individual judges at this scale, with a current focus on New York State. Since our founding in 2023, we have published profiles of more than 1,200 New York trial judges, each grounded in rigorous data and legal analysis. Our newest project, Decoded, is a public, plain-language record of how the New York Court of Appeals and each of its judges have ruled, sorted by the issues that shape life in New York. Decoded, and the projects our contractor will work on, use AI at all stages, from engineering unstructured data to running legal analyses at scale.

Our work is impactful: it is used by decision-makers, advocates, journalists, and voters. It has been recognized by The David Prize and Ironclad’s Legal AI Innovators Award, and backed by Fast Forward, Harvard Law School, Tyler Cowen’s Emergent Ventures, and others. Our collaborators include the New York Civil Liberties Union; clinics and centers at Harvard, NYU, and Cornell law schools; and leading watchdog groups like Reinvent Albany.

Why work with us

  • Your work is published and used

    What you build goes onto a public site that journalists, advocates, and voters consult, not into an internal tool.

  • You will have real ownership

    We are a very small team. You work directly with the founder, and your design decisions will shape the product and its impact.

  • The problem is hard and new

    The challenge is to use AI to analyze legal issues and legal datasets accurately and reliably enough to publish judge-by-judge records.

About the role

We are seeking a contractor who combines legal reasoning with hands-on AI engineering to implement several data development and data analysis projects. The work spans the full process: defining the legal questions, building the workflow that answers them, evaluating its accuracy, and improving it when the analysis fails.

This is not a position focused on using chatbots to draft legal work. You will build and test systems that analyze legal records at scale. You should be comfortable reading an appellate opinion closely and building the system that makes that analysis repeatable.

How the work is done

Our pipelines are built by directing AI coding agents (such as Claude Code and Codex), not by writing code by hand. You design the system, specify the work, and verify the result. We expect you to already work this way but to have experience with traditional coding and data pipeline designs.

What you will do

  • Specify the legal analysis

    Translate legal questions into clear instructions, categories, decision rules, and structured outputs. Distinguish holdings from dicta, majority reasoning from separate opinions, and genuine legal disagreement from differences in wording.

  • Build AI workflows

    Design and maintain multi-step workflows using language-model APIs, prompting, document processing, and structured extraction. Integrate them with existing datasets and pipelines.

  • Evaluate and improve accuracy

    Build test sets and validation criteria; compare results against source documents and expert judgments; investigate error patterns; and revise prompts, inputs, code, or methodology. Design controlled experiments that separate the real effect of a prompt or model change from run-to-run variation. Define when a result should be withheld for human review.

  • Make the work reproducible

    Maintain source references, traceable outputs, versioned code, and documentation. Improve reliability, efficiency, and scalability without sacrificing legal accuracy.

  • Work across disciplines

    Collaborate with Scrutinize’s partners to identify useful analyses and communicate both findings and limitations plainly.

Who should apply

We welcome applicants who developed their expertise through either law or engineering. A JD and appellate experience are preferred, but neither is required. What matters is demonstrated competence in both legal analysis and technical implementation.

  • Legal-first candidates

    A JD and substantial experience interpreting appellate opinions or other complex legal materials, together with a demonstrated record of building working AI systems.

  • Technical-first candidates

    Strong software and applied AI experience, together with meaningful experience working with judicial opinions, legal research, or other legal documents that require substantive interpretation.

Skills we need

  • Engineering through AI agents

    • Design

      Enough software background to decide what a pipeline should look like before any code exists: its stages, what each stage receives and returns, what gets saved, and how failures are recorded.

    • Specification

      Ability to give coding agents precise instructions and standing rules, and to break a large change into pieces an agent can complete correctly.

    • Verification

      Ability to establish that agent-written code does what was intended without taking the agent’s word for it: tests, adversarial audits, inspection of saved outputs, reading code, and controlled before-and-after comparisons.

    • Scale

      Experience running long, multi-stage jobs over thousands of documents, including resuming after failures, handling rate limits, and controlling cost.

  • AI development

    Experience creating, testing, and refining multi-step LLM workflows, including writing prompts and running systematic evaluations. Practical experience matters more than familiarity with a particular model.

  • Legal judgment

    Ability to identify legally significant distinctions, assess whether an interpretation is supported by a source, and recognize uncertainty rather than force a confident answer.

  • Working style

    Careful, independent problem-solving; clear writing; and the willingness to revisit an approach when evidence shows it is wrong.

Equal opportunity

Scrutinize selects and engages contractors and employees without regard to race, religion, color, national origin, sex, sexual orientation, disability, age, veteran status, ancestry, citizenship, marital status, gender identity, or any other status protected by applicable law. We consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need an accommodation during the application process, please let us know in your application.

How to apply

Fill out our application form and upload a resume. The form asks you to sign in with a Google account to upload it. The hiring process will include multiple interviews.

Scrutinize is an equal opportunity employer.