Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years at TMS LLC, Los Angeles, CA

Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years at TMS LLC in Los Angeles, CA. Contract. Est. up to $200k. Apply with swipeandhired.

Location: Los Angeles, CA (Hybrid)

Duration: 3+ Months (Extension Possible)

Experience: 15+ Years
Job Summary
We are seeking a hands-on AI Engineer with expertise in Python, Agentic AI, AWS Bedrock AgentCore, and LLM evaluation to build, deploy, monitor, and continuously improve production-grade AI agents.
Key Responsibilities

• Develop AI agents using Python and Claude Agent SDK.

• Deploy agents using AWS Bedrock AgentCore Runtime.

• Design golden datasets and implement LLM-as-a-Judge evaluations.

• Integrate automated evaluations into CI/CD pipelines as quality gates.

• Implement Langfuse/OpenTelemetry for AI observability and tracing.

• Evaluate agent accuracy, task completion, tool execution, and performance.

• Support RAG pipelines and AWS data processing workflows.

Required Skills

• Python and Agentic AI development.

• AWS Bedrock AgentCore Evaluations / Bedrock evaluation.

• Golden-dataset design and curation.

• LLM-as-a-Judge design and calibration against human labels.

• Automated evaluation and CI gate integration.

• Langfuse and/or OpenTelemetry.

Good to Have

• RAGAS or equivalent evaluation frameworks.

• Trajectory evaluation and agentic red-teaming.

• Bedrock Guardrails, Strands Agents SDK, and GitHub Actions.

Qualifications

• 6–8+ years of software engineering, cloud engineering, or AI/ML experience.

• Hands-on experience with production AI agents and continuous evaluation.

• Strong problem-solving and troubleshooting skills.

All your information will be kept confidential according to EEO guidelines.