Job Listing
COMPUTER SCIENTIST (AI TEST AND EVALUATION)
Defense Threat Reduction Agency · Posted 63 days ago
About the Role
This position may be filled using Direct Hire Authority: Z5CAV/Direct-Hire Authority (Certain DoD Personnel) PL 118-31, Sec 125OB (i)(2), 12/22/2023.This position is being filled under the memorandum from the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) "Expansion of Direct Hire Authority for Certain Personnel of the Department of Defense," dated August 12,2024. This position is part of the Defense Threat Reduction Agency,
What You'll Do
- →Tests AI/ML systems for security, robustness, trustworthiness, and reliability by conducting adversarial testing in operationally realistic environments, performing risk assessments, evaluating functionality and compatibility across diverse scenarios, and building assurance cases that demonstrate the level of confidence in AI system capabilities.
- →Develops and executes code-level testing and validation procedures for machine learning models, including automating testing processes, evaluating user experience and human-computer interaction factors, and recording and managing all test data with accuracy and integrity throughout the testing life-cycle.
- →Prepares detailed T&E reports and communicates findings, risks, and recommendations to technical and non-technical audiences, including senior leadership, by translating complex test data and evaluative conclusions into clear, actionable courses of action that inform decision-making across the organization.
- →Assesses AI systems for bias, security vulnerabilities, and high-impact risks by reviewing AI architectures, analyzing training data sets and system outputs for unintended bias, identifying failure modes and low-probability risks, and developing risk management plans and corrective actions to ensure ethical and trustworthy AI deployment.
- →Collaborates with cross-functional teams - including developers, data scientists, cybersecurity professionals, and operational users- to integrate AI T&E frameworks into project test strategies and ensure seamless coordination across DataOps, MLOps, and DevSecOps pipelines, including AI deployment within cloud and IT infrastructure environments.
- →Represents the organization in working groups, interagency forums, and professional eventsrelated to AI T&E and responsible AI, while monitoring evolving test and safety standards (e.g., MIL-STD 882E, DO-178C, ISO 26262) to ensure AI evaluation activities remain current, rigorous, and aligned with mission requirements and AI ethical principles.
Requirements
- ✓You MUST provide transcripts or other documentation to support your educational claims.
- ✓All materials must be submitted by the closing date of the announcement.
- ✓GRADUATE EDUCATION: One academic year of graduate education is considered to be the number of credits hours that your graduate school has determined to represent one academic year of full-time study.
- ✓Such study may have been performed on a full-time or part-time basis.
- ✓If you cannot obtain your graduate school's definition of one year of graduate study, 18 semester hours (or 27 quarter hours) should be considered as satisfying the requirement for one year of full-time graduate study.
- ✓For further information, visit: https://www2.ed.gov/about/offices/list/ous/international/usnei/us/edlite-visitus-forrecog.html
Personality Fit
Job ID: da1a1489-b89b-4d47-a57c-ab071044430e
Posted via USAJobs