AI Python Engineer

Role Overview

We are looking for a Senior Python GenAI Engineer to join internal teams working on AI-driven decision automation solutions.
The role focuses on evaluating, testing, and improving Generative AI agents used in risk assessment, workflow automation, and document processing.

Key Responsibilities

• Evaluate GenAI agent outputs for accuracy, consistency, and business relevance
• Design and apply AI evaluation frameworks (quality, hallucination, reliability, bias, latency)
• Test underwriting / risk-assessment AI workflows using real business scenarios
• Improve prompts, agent logic, orchestration flows, and output quality
• Collaborate with engineers, analysts, and stakeholders on production AI systems
• Support continuous optimization of AI agents in regulated environments
• Define KPIs and benchmarks for GenAI performance

Required Skills and Experience

• 6+ years of commercial software development experience
• Hands-on experience with GenAI / LLM products in production
• Understanding of AI agents / agentic workflowsKnowledge of banking, fintech, underwriting, insurance, or risk domains is a strong plus
• Familiarity with Microsoft Copilot Studio is beneficial
• Experience with LangChain / LangGraph or similar orchestration tools
• Experience with evaluation frameworks (RAGAS, DeepEval, LangSmith, prompt testing, etc.) Strong Python skills