Principal QA Engineer – USA (Remote)

Full Time
USA
Posted 6 days ago

Contract Type: Full-Time Contract (40 hrs/week – long-term engagement)

Location: 100% Remote – Anywhere in the U.S.

Experience: 8–10 years of experience in SDET, quality engineering, or test architecture roles. 

Role Overview

We’re looking for a Principal SDET to serve as the quality engineering anchor for our AI-first engineering organization. In this role, you’ll define the long-term quality architecture, establish org-wide testing standards, and drive the strategy for how AI transforms quality engineering at scale. You’ll work closely with Tech Leads, Principal Engineers, and product leadership to ensure our systems — including their GenAI components — meet the highest standards of reliability, correctness, and performance. 

Key Responsibilities

Quality & Test Engineering 

  • Define and own the overall quality engineering strategy and test architecture across all product lines and engineering teams. 
  • Establish org-wide standards, frameworks, and best practices for test automation, performance testing, and reliability engineering. 
  • Lead the design of scalable, cloud-native test infrastructure on AWS, including containerized test environments and distributed test execution. 
  • Drive quality governance: define KPIs, coverage targets, and quality gates across the SDLC. 
  • Partner with Tech Leads and Principal Engineers to ensure testability is built into system and API designs from the start. 
  • Lead the adoption of Agile quality practices; own sprint-level and release-level quality reporting to leadership. 
  • Mentor senior and mid-level SDETs; develop engineering capability across the quality organization. 
  • Evaluate, select, and champion testing tools and platforms — including AI-powered quality solutions. 

AI-Augmented Testing 

  • Own the strategy for testing GenAI and LLM-powered systems at scale — including output validation, semantic similarity testing, hallucination detection, and behavioral regression. 
  • Architect evaluation frameworks for LLM applications, defining metrics, benchmarks, and automated quality gates for AI features. 
  • Partner with AI engineering teams to define testability requirements for LLM pipelines, RAG systems, and prompt frameworks. 
  • Build and evolve monitoring and observability approaches for AI system quality in production. 
  • Drive organization-wide adoption of GenAI tools to improve test productivity, coverage, and insight generation. 
  • Represent quality engineering in AI architecture discussions, ensuring new AI systems are built with measurability and testability in mind. 

Candidate Requirements 

  • 8–10 years of experience in SDET, quality engineering, or test architecture roles. 
  • Deep expertise across the full spectrum of testing: functional, integration, API, performance, security, and reliability. 
  • Proven track record of defining and owning test strategy and quality architecture across large engineering organizations. 
  • Strong proficiency in Python, JavaScript, or Java; able to design and review complex automation systems. 
  • Expert knowledge of CI/CD pipelines, cloud test infrastructure (AWS preferred), and containerized environments. 
  • Experience leading quality initiatives across cross-functional teams and influencing engineering standards. 
  • Strong communication and stakeholder management skills; able to present quality metrics and strategy to leadership. 
  • Proven ability to mentor senior engineers and grow the capability of a quality engineering team. 

Good to Have 

  • Experience designing evaluation frameworks for LLM or GenAI systems. 
  • Background in MLOps, AI observability, or model quality monitoring. 
  • Familiarity with chaos engineering and site reliability engineering (SRE) practices. 
  • Experience with open-source contributions to testing frameworks or AI quality tooling. 
  • Background in developer experience (DX) improvements that increase testability and code quality org-wide. 

ID: P22

Job Features

Job CategoryTechnical Leadership

Apply For This Job

A valid phone number is required.

U.S. Standard Demographic Questions

We invite applicants to share their demographic background. If you choose to complete this survey, your responses may be used to identify areas of improvement in our hiring process.

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey. Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file.

As set forth in DigitalT3’s Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.

Voluntary Self-Identification of Veteran Status (VEVRAA)

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows:

  • A "disabled veteran" is a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.
  • A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.
  • An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.
  • An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.

Voluntary Self-Identification of Disability (Form CC-305)

Form CC-305 | OMB Control Number 1250-0005 | Expires 07/31/2029

Why are you being asked to complete this form?

We are a federal contractor or subcontractor. The law requires us to provide equal employment opportunity to qualified people with disabilities. We have a goal of having at least 7% of our workers as people with disabilities. The law says we must measure our progress towards this goal. To do this, we must ask applicants and employees if they have a disability or have ever had one. People can become disabled, so we need to ask this question at least every five years.

Completing this form is voluntary, and we hope that you will choose to do so. Your answer is confidential. No one who makes hiring decisions will see it. Your decision to complete the form and your answer will not harm you in any way. If you want to learn more about the law or this form, visit the U.S. Department of Labor’s Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp.

How do you know if you have a disability?

A disability is a condition that substantially limits one or more of your "major life activities." If you have or have ever had such a condition, you are a person with a disability. Disabilities include, but are not limited to:

  • Alcohol or other substance use disorder
  • Autoimmune disorder (Lupus, Fibromyalgia, HIV/AIDS)
  • Blind or low vision
  • Cancer (past or present)
  • Cardiovascular or heart disease
  • Celiac disease
  • Cerebral palsy
  • Deaf or serious difficulty hearing
  • Diabetes
  • Disfigurement (burns, scars)
  • Epilepsy or other seizure disorder
  • Gastrointestinal disorders (Crohn's, IBS)
  • Intellectual or developmental disability
  • Mental health conditions (Depression, Anxiety, PTSD)
  • Missing limbs or partially missing limbs
  • Mobility impairment
  • Nervous system conditions (MS, Migraines, Parkinson's)
  • Neurodivergence (ADHD, Autism, Dyslexia)
  • Partial or complete paralysis
  • Pulmonary or respiratory conditions (Asthma)
  • Short stature (dwarfism)
  • Traumatic brain injury

PUBLIC BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. This survey should take about 5 minutes to complete.