Job description

Founding Engineer – Open Source LLM Infrastructure | NYC (In-Person)

Compensation: $200K–$300K + Equity A well-funded NYC-based startup is seeking a Founding Engineer to build the future of large language model (LLM) infrastructure. This is an opportunity to join a small, expert team focused on open-source development at the intersection of machine learning, observability, and developer tooling. About the Role:

The successful candidate will work as a product-focused engineer, contributing to impactful, user-facing features across the stack. From building unified LLM APIs to optimizing inference and feedback loops, your work will directly influence the evolution of cutting-edge ML systems. Most contributions will be open source. What You’ll Do: Build and refine LLM observability tools, optimization workflows, and evaluation systems

Design performant full-stack features with measurable user impact

Collaborate with specialists in Rust, ML, and modern front-end ecosystems

Help shape engineering practices, product direction, and team culture

Ideal Candidate Has: A strong technical foundation and experience leading complex projects

Passion for developer experience and thoughtful product design

A growth mindset and eagerness to work across disciplines

Availability to work full-time in person in NYC

Why Join: Competitive compensation and meaningful equity

Work alongside world-class engineers and ML researchers

Fully open-source product development

Backed by top-tier investors with years of runway

This is a rare chance to get in early and help shape a high-impact platform at the core of the LLM revolution.

#LI-SM1