Senior Software Engineer – LLM Evaluation

Careerflow.ai is seeking a Senior Software Engineer to evaluate and enhance large language models through the creation of datasets and collaboration with researchers. The role involves curating code examples, refining AI-generated code, and working with cross-functional teams to improve AI-driven coding solutions. Responsibilities: Working on AI model training initiatives by curating code examples, building solutions, and correcting code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; Evaluate and refine AI-generated code to ensure that it is efficient, scalable, and reliable; Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks; Build agents that can verify the quality of the code and identify error patterns; Hypothesize on steps in the software engineering cycle (prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operational maintenance) and evaluate model capabilities on them; Design verification mechanisms that can automatically verify a solution to a software engineering task Qualifications: Several years of software engineering experience (+5 years), including 2+ years of continuous full-time experience at a top-tier product company (e.g., Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, IBM Research); Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools; Deep understanding of software architecture, design, development, debugging, and code quality/review assessment; Excellent oral and written communication skills for clear, structured evaluation rationales; Careerflow.ai; Careerflow - Career Copilot; Founded in 2022; San Francisco, California, USA; 11-50 employees; https://www.careerflow.ai; Current Stage; Early Stage; Total Funding; Pre Seed· $0.8M; Leadership Team; Puneet Kohli; Chief Executive Officer; Nikita Gupta; Co-Founder, COO; Recent News; USAA Propels FourBlock's AI Expansion to Support Veteran and Military Spouse Career Readiness; Company data provided by crunchbase; San Francisco; Date Posted; Years of Experience; Hidden Jobs; All Filters; Recommended; 2 months ago; LLM Engineer; Artificial Intelligence (AI) · Retail · Early Stage; United States; Mid, Senior Level; 3+ years exp; Why This Job Is A Match; Trustana is seeking an LLM Engineer to join their team and enhance their SaaS platform with AI features. The role involves applying large language models to automate product data management for retailers, brands, and distributors, while also mentoring teammates and guiding technical direction at a senior level.; Experience Level; Industry Experience; 98 applicants; APPLY WITH AUTOFILL; STRONG MATCH; 1 former colleague works here; Senior AI Engineer – LLM, RAG; Artificial Intelligence (AI) · Cloud Computing · Growth Stage; Palo Alto, CA; Senior Level; 5+ years exp; Why This Job Is A Match; BrightAI is a high-growth Physical AI company transforming how businesses interact with the physical world through intelligent automation. They are seeking a Senior AI Engineer – LLM, RAG to lead the development of Retrieval-Augmented Generation systems that leverage large language models and real-world knowledge sources to create intelligent assistants for industrial troubleshooting.; Experience Level; Industry Experience; 200+ applicants; APPLY WITH AUTOFILL; STRONG MATCH; Be an early applicant; AI Engineer; Artificial Intelligence (AI) · FinTech · Growth Stage; San Francisco, CA; $180K/yr - $280K/yr; Mid, Senior Level; 4+ years exp; Why This Job Is A Match; Comulate is transforming the insurance back office with AI, aiming to unlock significant savings in manual insurance operations. They are seeking an AI Engineer to build innovative products that leverage unstructured data and improve workflows i Required Skills: Python, JavaScript, ReactJS, C/C++, Java, Rust, Go, Software Architecture, Software Development, Debugging, Code Quality Assessment, Communication