Senior Prompt Engineer: LLM Migration & Optimization

Welo Data is seeking a Senior Prompt Engineer to lead the technical migration of template workflows into high-performance LLM autoraters. This role focuses on optimizing model performance using advanced tools and manual refinement to ensure high accuracy and nuance in automated systems. Responsibilities: Take full ownership of the end-to-end technical migration of templates to LLM autoraters; Utilize Automatic Prompt Generation (APG) and supervise Automated Prompt Optimization (APO) tools to push model performance past plateaus and deadlocks; Continuously measure quality against "gold data" baselines, tracking precision, recall, and $F_1$ scores to justify launch readiness; Manually draft and refine complex prompts to overcome anti-patterns and architecture gaps that automated tools can't solve Qualifications: Bachelor's, Master's, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field; 4+ years of experience tuning LLMs for strict, structured outputs, complex classification, and few-shot learning; High proficiency in identifying error patterns and using SQL or data analytics tools to monitor performance; Fast learner capable of mastering proprietary internal tools and 'Goose API' style interfaces with minimal oversight Required Skills: Prompt Engineering, Large Language Models (LLMs), Automatic Prompt Generation (APG), Automated Prompt Optimization (APO), SQL, Data Analytics, AI Model Evaluation, Software Engineering, Semantics, Logic, Chain-of-Thought (CoT) Prompting, Technical Documentation