12 days ago

Senior AI Engineer

Arlington, United States of AmericaFull-timeRemoteMid · 5+ yearsAI Engineer

What you'll do

Design, build, and improve production agentic AI systems used to solve complex real-world problems.
Develop agent architectures for reasoning, planning, tool use, context management, memory, and multi-step task execution.
Build tools and capabilities that allow agents to securely interact with data, APIs, code, and external systems.
Develop model and inference infrastructure supporting multiple commercial and open-weight language models.
Evaluate new models, inference techniques, and emerging AI capabilities and determine how they can improve our production systems.
Build automated evaluation frameworks to measure agent quality, reliability, task completion, and regressions.
Develop datasets, benchmarks, and evaluation methodologies for complex agentic workflows.
Improve agent performance through prompt and context engineering, model selection, tool design, inference strategies, and architectural improvements.
Build scalable APIs, services, and infrastructure supporting agent execution and AI-powered product experiences.
Design systems for asynchronous and long-running agent workflows.
Build infrastructure for safe and reliable execution of agent-generated code and other computational workloads.
Improve system observability through structured logging, metrics, distributed tracing, dashboards, and automated alerting.

What they're looking for

U.S. Citizenship is required
5+ years of experience building production software, AI/ML systems, distributed systems, or similar technical systems.
Deep experience designing, building, and operating production AI agents or agent platforms.
Experience developing core agent infrastructure, including agent runtimes, tool execution, context management, memory, orchestration, or related platform capabilities.
Strong understanding of state-of-the-art agent architectures and the engineering tradeoffs involved in building reliable, production-grade agentic systems.
Experience designing agents that use tools, reason across multi-step tasks, interact with external systems, and operate over long-running or complex workflows.
Experience building evaluation systems for agents, including task-level evaluations, behavioral evaluations, regression testing, and production quality measurement.
Strong understanding of modern LLM systems, including model inference, context engineering, structured outputs, tool calling, retrieval, model selection, and techniques for improving agent performance.
Demonstrated ability to evaluate emerging models, research, and agent techniques and translate promising approaches into production systems.
Strong programming experience in Python and experience building production-quality software.
Experience designing scalable APIs, services, asynchronous systems, and event-driven architectures.
Experience operating production services using Kubernetes and cloud platforms such as AWS, GCP, or Azure.

Nice to have

Current possession of a U.S. security clearance, or the ability to obtain one with our sponsorship
Experience in or exposure to the nuances of a startup or other entrepreneurial environment
Experience building secure code execution environments or sandboxes for AI agents.
Experience with multi-agent architectures, agent-to-agent communication, or distributed agent execution.
Experience with fine-tuning, post-training, reinforcement learning, or synthetic data generation.
Experience building AI observability, tracing, and debugging infrastructure.
Experience optimizing inference latency, throughput, GPU utilization, or model-serving costs.
Experience with AI security, adversarial testing, or securing agentic systems.

Summarised by NextRaise from the employer’s description, which follows in full below.

Full description from employer

Company Description

Air is the leader in Enterprise Readiness. Our mission is to establish readiness as a real-time condition that is continuously achieved. Today, a dangerous Readiness Gap exists between what the front line needs and what is delivered. Our AI-native platform, Air Enterprise Readiness, aligns development, production, delivery, and sustainment into one coordinated execution system for government agencies and industrial suppliers. By revealing true capacity, exposing real constraints, coordinating resources, and executing at the speed of operational demands, the front line gets what it needs to succeed.

Job Description

We are seeking an experienced Senior AI Engineer to join our Agentic AI team as we scale our AI capabilities across all levels of the U.S. government.

Over the past year, we have seen rapid adoption of our AI agent, Ace. We are building the systems and infrastructure required to move agentic AI beyond demos and prototypes into reliable, production-grade software operating against complex, real-world problems.

This role sits at the intersection of applied AI and systems engineering. You will design and build agent architectures, model integrations, tools, evaluation systems, and infrastructure that enable increasingly capable AI systems. You will work across the AI stack - from experimentation and model behavior to production services and distributed infrastructure.

In order to do this job well: we are looking for engineers who understand that building great AI products requires more than calling a model API. You should be comfortable reasoning about how agents use context, tools, models, memory, and compute, and turning those ideas into reliable production systems.

This role is a full-time position based in our Pittsburgh, PA office or open to Remote Opportunities.

This role may require up to 25% travel, including periodic travel to our Pittsburgh, PA and Arlington, VA offices for team collaboration, planning activities, and in-person meetings.

Scope of Responsibilities

Design, build, and improve production agentic AI systems used to solve complex real-world problems.
Develop agent architectures for reasoning, planning, tool use, context management, memory, and multi-step task execution.
Build tools and capabilities that allow agents to securely interact with data, APIs, code, and external systems.
Develop model and inference infrastructure supporting multiple commercial and open-weight language models.
Evaluate new models, inference techniques, and emerging AI capabilities and determine how they can improve our production systems.
Build automated evaluation frameworks to measure agent quality, reliability, task completion, and regressions.
Develop datasets, benchmarks, and evaluation methodologies for complex agentic workflows.
Improve agent performance through prompt and context engineering, model selection, tool design, inference strategies, and architectural improvements.
Build scalable APIs, services, and infrastructure supporting agent execution and AI-powered product experiences.
Design systems for asynchronous and long-running agent workflows.
Build infrastructure for safe and reliable execution of agent-generated code and other computational workloads.
Improve system observability through structured logging, metrics, distributed tracing, dashboards, and automated alerting.
Investigate failures across models, agents, application code, and distributed infrastructure and turn those findings into systematic improvements.
Optimize model and agent systems for latency, throughput, reliability, and infrastructure cost.
Translate new AI research and emerging techniques into practical improvements to production systems.
Work closely with product, platform, security, and domain teams to bring new AI capabilities from experimentation to production.

Qualifications

U.S. Citizenship is required

Required Skills:

5+ years of experience building production software, AI/ML systems, distributed systems, or similar technical systems.
Deep experience designing, building, and operating production AI agents or agent platforms.
Experience developing core agent infrastructure, including agent runtimes, tool execution, context management, memory, orchestration, or related platform capabilities.
Strong understanding of state-of-the-art agent architectures and the engineering tradeoffs involved in building reliable, production-grade agentic systems.
Experience designing agents that use tools, reason across multi-step tasks, interact with external systems, and operate over long-running or complex workflows.
Experience building evaluation systems for agents, including task-level evaluations, behavioral evaluations, regression testing, and production quality measurement.
Strong understanding of modern LLM systems, including model inference, context engineering, structured outputs, tool calling, retrieval, model selection, and techniques for improving agent performance.
Demonstrated ability to evaluate emerging models, research, and agent techniques and translate promising approaches into production systems.
Strong programming experience in Python and experience building production-quality software.
Experience designing scalable APIs, services, asynchronous systems, and event-driven architectures.
Experience operating production services using Kubernetes and cloud platforms such as AWS, GCP, or Azure.
Strong understanding of distributed systems, containers, service orchestration, networking, storage, and scalable architectures.
Comfortable debugging complex failures spanning model behavior, agent execution, application code, and production infrastructure.
Able to move quickly between research and engineering: prototype an idea, evaluate it rigorously, understand why it works or fails, and turn successful approaches into reliable production systems.
Comfortable working at the frontier of a rapidly evolving field where established patterns may not yet exist.

Desired Skills:

Current possession of a U.S. security clearance, or the ability to obtain one with our sponsorship
Experience in or exposure to the nuances of a startup or other entrepreneurial environment
Experience building secure code execution environments or sandboxes for AI agents.
Experience with multi-agent architectures, agent-to-agent communication, or distributed agent execution.
Experience with fine-tuning, post-training, reinforcement learning, or synthetic data generation.
Experience building AI observability, tracing, and debugging infrastructure.
Experience optimizing inference latency, throughput, GPU utilization, or model-serving costs.
Experience with AI security, adversarial testing, or securing agentic systems.
Experience working in government, defense, or other mission-critical environments.

We firmly believe that past performance is the best indicator of future performance. If you thrive while building solutions to complex problems, are a self-starter, and are passionate about making an impact in global security, we’re eager to hear from you.

Air is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability and protected veterans status or any other characteristic protected by law.

Company

Air

Arlington, United States of America

Company facts come from this company's own listings. We only show what the postings themselves carry.