AI Researchers: Six Questions to Strengthen Moral Judgment

Updated 5 November 2026 · Standard formula, shown below

AI researchers can strengthen their moral judgment by asking six practical questions: Who benefits, who may be harmed, whose perspective is missing, what assumptions shape the work, how could it be misused, and how will its effects be monitored? These questions are a reflection tool, not a guarantee of ethical outcomes; their value comes from applying them throughout research and development, not just before launch.

Why moral reflection matters in AI research

AI research choices can influence who is represented in a dataset, what a model is optimized to do, how a system is evaluated, and where it is deployed. Those choices can distribute benefits and burdens unevenly. A model may perform well on an overall benchmark while failing for a particular language group, or an automated ranking system may make consequential decisions harder to challenge.

Technical performance alone cannot settle questions such as whether a system should be built, which errors are acceptable, or who should have authority over its use. Researchers therefore need to examine the context around the technology: affected communities, institutional incentives, likely users, and the consequences of failure. Ethical reflection does not replace measurement or governance. It helps identify what should be measured, whose interests count, and when a project needs to change or stop.

The six questions to ask

Who benefits from this work? Identify the people or organizations likely to gain from the system, and distinguish direct users from those who benefit indirectly. Ask whether the benefits are accessible to the people whose data, labor, or environments made the research possible.
Who could be harmed, and how? Consider direct and indirect harms, including inaccurate outputs, exclusion, surveillance, exposure of sensitive information, or overreliance on automated recommendations. Describe concrete scenarios rather than relying on the vague claim that a tool could be “biased.”
Whose perspective is missing? Ask whether the team has heard from people affected by the system, including those who are not typical users or who have limited ability to opt out. Consultation should happen early enough to change the project, not merely to endorse a decision already made.
What assumptions and values are built into the work? Objectives, labels, thresholds, and definitions of success all reflect choices. For example, optimizing for speed may come at the expense of careful review; a dataset label may treat a contested social category as if it were unambiguous.
How could the system be misused or behave unexpectedly? Look beyond the intended application. Could outputs be repurposed, combined with other data, or treated as more certain than they are? Consider foreseeable misuse alongside ordinary failure, without assuming that every possible misuse can be predicted.
How will the effects be monitored and addressed? Decide what evidence would reveal harm, who is responsible for reviewing it, and what changes are possible after deployment. Monitoring is meaningful only if there is a way to respond, such as restricting use, correcting data, informing affected people, or withdrawing the system.

How to use the questions in a project

These questions work best as repeated checkpoints. At project scoping, they can reveal that a proposed application has no clear public benefit or that an important group has not been consulted. During data collection, they can prompt scrutiny of consent, provenance, coverage, and privacy. During evaluation, they can push teams to test performance across relevant settings rather than relying on a single aggregate score.

A practical discussion can record each answer, the evidence behind it, unresolved uncertainties, and a named owner for follow-up. For example, a team developing a system that summarizes public-service applications might ask whether applicants can correct errors, whether the evaluation includes different writing styles and languages, and whether staff might mistake a summary for a verified fact. Those questions can lead to specific tests and safeguards.

Ask the questions before major commitments, when a project can still be redesigned.
Include researchers with different expertise and people familiar with the affected setting.
Turn concerns into testable checks, design changes, or explicit limits on use.
Revisit decisions when the data, model, users, or deployment context changes.

Common pitfalls and limits

A checklist can become box-ticking if teams treat an answer as proof that a project is safe. “We considered bias” is not evidence that relevant disparities were tested or resolved. Similarly, broad statements about social benefit can obscure who receives that benefit and who carries the risk.

Another mistake is to frame ethics as the responsibility of an individual researcher alone. Choices may be constrained by deadlines, funding, organizational policies, or deployment decisions made elsewhere. Teams can still document concerns and escalate them, but institutions also need clear accountability and the authority to act on findings.

Finally, no set of questions eliminates uncertainty. A system may have effects that are hard to predict, and public consultation cannot represent every affected person. The appropriate response is not to claim certainty, but to state limits, build in review, and avoid uses for which foreseeable risks cannot be responsibly managed.

From reflection to accountability

Moral reflection becomes more useful when it changes decisions. A team might revise an evaluation plan after discovering that its test data do not represent the intended setting. It might limit a model to a low-stakes assistive role rather than allowing it to make decisions, or conclude that a project should not proceed. These are substantive outcomes, not failures of innovation.

Documentation also matters. A concise record of intended use, excluded uses, known limitations, evaluation results, and monitoring responsibilities helps later users understand what the research does and does not establish. It can support review by colleagues, institutions, and affected communities. The six questions provide a starting structure; technical audits, privacy review, security analysis, and relevant institutional processes may also be needed.

Frequently Asked Questions

What are the six questions AI researchers should ask?
Ask who benefits, who could be harmed, whose perspective is missing, what assumptions shape the work, how the system could be misused or fail, and how its effects will be monitored and addressed.

Are these questions an official AI ethics standard?
No. They are a practical reflection framework, not a certification, legal test, or substitute for applicable institutional review and governance.

When should researchers ask these questions?
Throughout a project: during scoping, data work, design, evaluation, deployment planning, and later review. Answers may change as the system and its context change.

Can asking the questions make an AI system unbiased?
No. The questions can help teams identify assumptions and investigate unequal effects, but they cannot guarantee fairness. Claims about performance need relevant evidence and clear limits.

What should a research team do if it identifies a serious risk?
Document the concern, identify who can make or review the decision, and consider concrete responses such as redesign, restricted use, additional evaluation, or stopping the work. The right action depends on the risk and context.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.