Job description
Staff AI Software Engineer
Salary up to $275K + Equity
San Mateo | High Impact AI Platform
US Citizen or GreenCard holder
A fast-growing AI startup is building LLM-powered systems transforming healthcare operations.
They’re looking for a Staff AI Software Engineer to design and scale the backend infrastructure powering production LLM applications used in real healthcare workflows.
This role is ideal for someone who enjoys taking LLM systems from prototype to reliable production infrastructure. You’ll be building APIs, data pipelines, and distributed systems that make AI dependable in mission-critical environments.
What You’ll Be Doing
- Design and build scalable backend services and APIs that orchestrate LLM systems in production
- Develop RAG pipelines and data ingestion systems that ground LLMs in healthcare data
- Build LLMOps infrastructure including monitoring, evaluation, cost tracking, and guardrails
- Optimize performance through caching, batching, and distributed workloads
- Implement secure and compliant systems designed for healthcare environments
- Partner with product and ML teams to bring AI into real operational workflows
- Provide technical leadership and system design guidance across AI infrastructure
What They’re Looking For
- 10+ years of software engineering experience building backend or distributed systems
- Proven experience working with LLMs on the backend in production environments
- Strong coding ability in Python plus one of Go, Java, or TypeScript
- Experience building systems using LangChain, Hugging Face, LlamaIndex, OpenAI APIs, Anthropic or similar frameworks
- Deep understanding of distributed systems and scalable API architecture
- Cloud infrastructure experience with AWS, GCP, or Azure
- Experience with Docker, Kubernetes, and cloud-native infrastructure
- Familiarity with LLMOps / MLOps practices including monitoring and model evaluation
Nice to Have
- Experience building RAG pipelines and working with vector databases
- Exposure to healthcare or regulated environments (FHIR, HL7, HIPAA)
- Background building enterprise SaaS or mission-critical platforms
Why This Role
- Work on real-world LLM systems used in healthcare
- High ownership role shaping core AI infrastructureSalary up to $275K + equity
- Opportunity to help define how AI is deployed in mission-critical environments
If you’re a senior engineer building real backend systems for LLMs, feel free to reach out.
