Auditing AI Surveillance for Bias: City Guide

Updated 5 November 2026 · Standard formula, shown below

This guide explains how city governments can audit their AI surveillance systems for accuracy and bias. You’ll learn a practical, step-by-step process for testing these systems, interpreting results, and fixing problems. The goal is to ensure that AI tools used for public safety are fair, accurate, and accountable to the communities they serve.

What You’ll Need

Access to the AI system’s technical documentation and training data information
A sample of real-world data that the system has processed (with privacy protections)
Ground truth data: known correct outcomes for comparison (e.g., actual crime reports, verified identities)
Computing resources to run test scenarios (can be cloud-based)
A diverse audit team including data scientists, civil rights experts, and community representatives
Legal counsel to ensure compliance with privacy and civil liberties laws
Budget for independent testing if needed

Step-by-Step

Define the system’s purpose and scope. Write down exactly what the AI is supposed to do (e.g., detect shoplifting, recognize faces in public). Identify the inputs (camera feeds, sensor data) and outputs (alerts, matches).
Identify the protected groups. Determine which demographic groups are relevant to your city (by race, gender, age, disability, etc.) based on the system’s use and potential for disparate impact.
Collect a representative test dataset. Gather a set of inputs that reflect the diversity of your city’s population. Include edge cases: different lighting, angles, clothing, and backgrounds. Ensure the dataset has known correct outputs (ground truth).
Run the AI system on the test dataset. Use the system in a controlled environment, not in live operation. Record all outputs, including confidence scores and any error messages.
Calculate accuracy metrics. For each test case, compare the AI’s output to the ground truth. Compute:
- Overall accuracy: percentage of correct predictions.
- Precision: of all positive predictions, how many were correct?
- Recall: of all actual positives, how many were found?
- F1 score: harmonic mean of precision and recall.
Break down metrics by group. Calculate the same metrics separately for each demographic group in your test set. Compare group-specific accuracy, false positive rates, and false negative rates.
Test for bias. Look for significant differences between groups. For example, if the false positive rate for one race is 10% and for another it’s 2%, that’s a bias indicator. Use statistical tests (like chi-squared) to confirm differences aren’t due to chance.
Review the system’s decision-making process. Examine the model’s features and weights (if accessible). Identify which factors the AI relies on most. Check if any are proxies for protected characteristics (e.g., zip code as a proxy for race).
Document findings in a report. Create a clear, non-technical summary for the public and a detailed technical appendix. Include all metrics, test conditions, and limitations of the audit.
Make recommendations and implement fixes. Based on findings, suggest changes: retraining with more balanced data, adjusting decision thresholds, or discontinuing the system if bias is severe. Set a timeline for re-auditing.
Establish ongoing monitoring. Schedule regular audits (e.g., annually) and set up real-time performance tracking for accuracy and bias in live operations.

Common Mistakes

Testing only on the system’s training data. This gives a false sense of accuracy because the AI has already seen those examples. Always use new, unseen data.
Ignoring false negatives. In surveillance, a false negative (missing a threat) can be as harmful as a false positive. Report both.
Using a test set that doesn’t reflect the real population. If your city is 40% Hispanic but your test set is only 10%, your bias results will be misleading.
Only looking at overall accuracy. A system can be 95% accurate overall but 99% accurate for white men and 70% for Black women. Average hides disparities.
Not involving community stakeholders. Audits done in isolation may miss concerns that community members care about.
Failing to protect privacy during testing. Even test data may contain sensitive information. Anonymize everything.

Tips

Start small: Pilot the audit on a single system before scaling to all city AI.
Use independent auditors: If budget allows, hire an external firm to avoid conflicts of interest.
Make the audit repeatable: Document your test setup so you can replicate it in future audits.
Publish the results: Transparency builds public trust, even when findings are negative.
Consider intersectionality: Bias may be more pronounced for people who belong to multiple protected groups (e.g., older Black women).
Use open-source tools: Libraries like Fairlearn or AIF360 can help with bias metrics.

Troubleshooting

You don’t have access to the system’s internal code. Many commercial AI systems are black boxes. In that case, you can still audit by testing inputs and outputs. If the vendor refuses to provide any information, consider legal options or contractual requirements for transparency.

The test dataset is too small. If you can’t get enough real data, use synthetic data generation (creating realistic but artificial inputs). Be cautious: synthetic data may not capture all real-world variability.

Bias is found, but you can’t fix it immediately. The safest option is to temporarily disable the system in high-bias contexts until retraining is complete. If that’s not possible, at least increase human oversight for affected groups.

Vendor disputes your findings. Have an independent third party review your methodology. Publish the audit protocol in advance to preempt criticism.

Community members don’t trust the audit. Hold public forums to explain the process and results. Use plain language, not jargon.

Frequently Asked Questions

Q: How often should we audit?
A: At least annually, and whenever the system is updated or its deployment context changes significantly.

Q: What if the system is accurate but biased?
A: That’s possible. A system can be accurate overall but have high error rates for certain groups. Bias is a separate concern from accuracy; both need to be addressed.

Q: Can we audit a system we don’t fully understand?
A: Yes, you can do black-box testing. But you should also push for more transparency from vendors. Some cities have passed laws requiring algorithmic accountability.

Q: How do we define ‘bias’ in a legal sense?
A: Bias is often defined as a statistically significant difference in outcomes (like false positives) between protected groups. Consult your city’s legal team for the specific legal standard in your jurisdiction.

Q: What if we find bias but the vendor says it’s not their fault?
A: The responsibility lies with the city as the deployer. You can require the vendor to fix the issue as part of your contract, or you can stop using the system.

Q: Does the audit cover privacy issues?
A: This audit focuses on accuracy and bias. Privacy is a separate but related concern. Consider a parallel privacy impact assessment.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.