Overview

The Digital Harbor Foundation is seeking a 3-4 month contract engineer to help develop a new “LLM Arena” that compares educational Large Language Models (LLMs). The goal of the Software Engineer is to deliver a fully functional prototype designed for tutors, learners, and AI developers. This project includes creating a robust API schema, a user-friendly interface for tutoring sessions, logging mechanisms, pre- and post-test functions, and safeguards for respectful interactions. You will collaborate with university research labs, private organizations, and LEVI teams to build, test, and refine the prototype, ensuring it meets performance and reliability standards.

The Learning Engineering Virtual Institute (LEVI) is a research and development program to create educational technology that can double the rate of middle school math progress for students in under-resourced communities.

Digital Harbor Foundation is dedicated to fostering learning, creativity, productivity, and community through education with a vision of digital equity for everyone. Driven by our sincere belief that access to opportunity is a basic right, we take bold yet practical actions to support making a better future now. Through a portfolio of projects focused on developing leadership within communities, we support those closest to challenges to take deliberate actions based in design thinking approach, backed by data analysis, grounded in a practice of collective impact, and driven by a commitment to racial equity.

Education and Experience Requirements

• Prior professional or academic experience in software engineering, computer science, or a related field.

• Hands-on coding experience in Python, including familiarity with APIs and web frameworks.

• Ability to develop, debug, and optimize system architectures.

• Experience integrating or comparing AI models (LLMs preferred).

• Knowledge of data privacy and security best practices.

Knowledge, Skills and Abilities

• Ability to write, maintain, and document API endpoints and schemas.

• Working knowledge of SQL for data handling and logging.

• Experience with version control systems (e.g., Git).

• Familiarity with essential ML model concepts and how to integrate them into software products.

• Skill in building user-friendly interfaces with attention to accessibility.

• Capability to set up and maintain safeguards (harmful behavior detection).

• Comfortable conducting stress tests and troubleshooting performance or reliability issues.

• Competence in writing clear technical documentation for end users and developers.

Additional Notes

This position is a fully remote 3-4 month contract position starting March 2025. Collaboration with the project lead and additional stakeholders will be conducted virtually.

Role and Responsibilities

• Design and implement a functional API and schema to power the LLM Arena prototype.

• Coordinate with teachers, tutors, and AI developers to collect requirements and address integration needs.

• Develop tools and features aligned with the learning tasks (e.g., plotting, whiteboarding) and ensure they are functional within the arena.

• Implement system safeguards for detecting and mitigating harmful behavior, keeping all interactions on-topic and respectful.

• Build and refine the user interface to support multiple audiences (tutors, learners, and developers).

• Implement and document logging functions to collect conversation data for analysis.

• Enable and test pre- and post-assessment functions (multiple-choice tests).

• Conduct stress tests for platform reliability, privacy, and safeguard performance.

• Meet project milestones, including:

  • Midpoint: Completion of API schema documentation explaining prototype progress.

  • Final: Delivery of a “Tutoring Arena Walk-Through and Discussion” with a fully functional prototype.

Other Duties

Please note that this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Additional tasks may be assigned or adjusted as the project evolves.

Compensation

Compensation for this position is $35,000 - $50,000 for the contract length, commensurate with experience.

If our mission and vision align with your personal values, please apply!

A cover letter outlining your qualifications for the position along with your resume is required. Interviews will be conducted virtually.

Digital Harbor and LEVI are equal opportunity employers.