Computer and Mathematical · SOC 15-1253
Software Quality Assurance Analysts and Testers
Develop and execute software tests to identify software problems and their causes. Test system modifications to prepare for implementation. Document software and application defects using a bug tracking system and report defects to software or web developers. Create and maintain databases of known defects. May participate in software design reviews to provide input on functional requirements, operational characteristics, product designs, and schedules.
Will AI replace this job?
The data cannot say, and anyone who gives you a percentage for “replaced” is guessing. What it does show is a gap between three things, for the 30 tasks O*NET lists for this occupation:
Could an LLM do it?
30%–80%
of tasks: an LLM alone could do the task or cut its time by half. Human raters say 30%, GPT-4 says 80%.
Counting tasks that need extra software around the model: 90%–93%.
Is it happening?
12 of 30 tasks
show measurable use in Claude conversations (Anthropic Economic Index). Anthropic’s occupation score, which counts only tasks that were also rated feasible, is 52%. Observed is a subset of “could” by construction.
Claude only, mostly US users, sampled periods in 2025. Not all AI use, and not employment.
What is left?
2 of 30 tasks
were rated “no LLM speed-up” by every rater (so, by Anthropic’s method, they cannot show up as observed use either). That is the part of the job these studies see no sign of AI touching.
It does not mean those tasks are safe. It means these methods found nothing there.
Task by task
Sorted with the tasks seen in use first. The three coloured squares are the three raters, in this order: human raters, GPT-4, GPT-4 with an alternative rubric. “Usage score” is Anthropic’s published task_penetration value; 0 means none seen.
| Task (O*NET 31.0) | Could an LLM? | Claude usage score |
|---|---|---|
| Document software defects, using a bug tracking system, and report defects to software developers. | all raters | 0.98 |
| Provide feedback and recommendations to developers on software usability and functionality. | all raters | 0.85 |
| Monitor bug resolution efforts and track successes. | all raters | 1.00 |
| Update automated test scripts to ensure currency. | all raters | 0.99 |
| Investigate customer problems referred by technical support. | all raters | 1.00 |
| Review software documentation to ensure technical accuracy, compliance, or completeness, or to mitigate risks. | all raters | 0.96 |
| Perform initial debugging procedures by reviewing configuration files, logs, or code pieces to determine breakdown source. | all raters | 0.94 |
| Design or develop automated testing tools. | all raters | 0.97 |
| Evaluate or recommend software for testing or bug tracking. | all raters | 0.79 |
| Modify existing software to correct errors, allow it to adapt to new hardware, or to improve its performance. (supplemental) | all raters | 0.96 |
| Store, retrieve, and manipulate data for analysis of system capabilities and requirements. (supplemental) | all raters | 0.96 |
| Test system modifications to prepare for implementation. | 2 of 3 raters | 0.99 |
| Identify, analyze, and document problems with program function, output, online screen, or content. | all raters | none seen |
| Develop testing programs that address areas such as database impacts, software scenarios, regression testing, negative testing, error or bug retests, or usability. | all raters | none seen |
| Design test plans, scenarios, scripts, or procedures. | all raters | none seen |
| Document test procedures to ensure replicability and compliance with standards. | all raters | none seen |
| Install, maintain, or use software testing programs. | all raters | none seen |
| Create or maintain databases of known test defects. | all raters | none seen |
| Develop or specify standards, methods, or procedures to determine product quality or release readiness. | all raters | none seen |
| Participate in product design reviews to provide input on functional requirements, product designs, schedules, or potential problems. | all raters | none seen |
| Plan test schedules or strategies in accordance with project scope or delivery dates. | all raters | none seen |
| Monitor program performance to ensure efficient and problem-free operations. | all raters | none seen |
| Conduct software compatibility tests with programs, hardware, operating systems, or network environments. | all raters | none seen |
| Identify program deviance from standards, and suggest modifications to ensure compliance. | all raters | none seen |
| Install and configure recreations of software production environments to allow testing of software performance. | all raters | none seen |
| Collaborate with field staff or customers to evaluate or diagnose problems and recommend possible solutions. | all raters | none seen |
| Coordinate user or third-party testing. | all raters | none seen |
| Conduct historical analyses of test results. | all raters | none seen |
| Visit beta testing sites to evaluate software performance. | no rater | none seen |
| Recommend purchase of equipment to control dust, temperature, or humidity in area of system installation. (supplemental) | no rater | none seen |
Try it with your own task list
Sources: task list from O*NET® Database (U.S. Department of Labor, CC BY 4.0; modified by noworkleft.com); “could” labels from Eloundou et al., GPTs are GPTs (OpenAI/University of Pennsylvania, MIT licence, labels from early-2023 models); “seen in use” from the Anthropic Economic Index (Claude usage only, CC BY). Dates, versions and limits.
Other jobs in Computer and Mathematical
- Computer Programmers (75% observed)
- Database Architects (58% observed)
- Information Security Analysts (49% observed)
- Web Developers (48% observed)
- Mathematical Science Occupations, All Other (48% observed)
- Computer User Support Specialists (47% observed)
- Data Scientists (46% observed)
- Operations Research Analysts (43% observed)
Data built 2026-11-06. O*NET 31.0. Task match rates: 99.4% with the OpenAI/Penn labels, 99.2% with Anthropic usage.