Protect Unpublished Research From AI Training and Scraping

Updated 5 November 2026 · Standard formula, shown below

Protecting unpublished research from AI training and scraping means reducing the chance that public-facing services collect your work, while limiting exposure if a file is shared or stored online. No single setting can guarantee that content will never be copied; start by identifying where the research lives and who needs access.

What You’ll Need

An inventory of research files, websites, repositories, collaboration platforms, and public profiles where drafts or data may appear.
Access to the relevant account settings, website configuration, and institutional IT or research data guidance.
A secure storage location with individual accounts and access controls, plus a backup plan.
If applicable, approval from your supervisor, coauthors, institution, funder, or research ethics office before changing sharing or retention settings.

Step-by-Step

Classify the material before changing anything. Separate unpublished manuscripts, notes, code, datasets, participant information, and material already approved for release. Mark sensitive or restricted data clearly. Human-subject information, confidential partner data, and regulated records need the safeguards specified by your institution; do not upload them to a public service to test its settings. Keep a record of which files are intended to remain private and who is authorized to access them.
Map every place a copy could be exposed. Check lab and personal websites, preprint or project pages, public code repositories, shared folders, cloud drives, collaboration tools, conference portals, and search-indexed documents. Look for forgotten test pages, old links, public profile attachments, and files shared as “anyone with the link.” Search for distinctive phrases from a draft in a search engine to identify publicly indexed copies. Do not paste confidential text into third-party search or detection tools.
Remove or restrict public copies you control. Take down unpublished files from public pages and repositories, or change their visibility to private or restricted. Review folder-level and file-level sharing separately: a private folder may contain a file with its own public link. Disable public-link access where it is not needed, remove former collaborators who no longer need access, and use named accounts rather than anonymous links. Tell your collaborators before changing shared resources so legitimate work is not disrupted.
Set website crawler preferences, but treat them as requests. If you operate a website, publish a robots.txt file at the site’s root and specify crawler rules for the paths you want compliant bots not to fetch. You can also add a robots directive to pages that should not be indexed. These controls are useful for well-behaved crawlers, but they do not prevent a scraper from ignoring them, and search indexing controls are not access controls. Do not publish a draft merely to apply a crawler rule; keep sensitive material behind authentication.
Check hosting and platform policies individually. Review the current privacy, retention, sharing, and model-improvement settings for each service holding your work. Settings differ by product, account type, and institution, and may change. Look for controls that govern whether submitted content is used to improve or train services, and whether administrators can manage those settings. If the documentation is unclear, ask the provider or your institution’s IT team in writing. A setting on one service does not change how another service handles copies.
Use approved, restricted storage for active research. Store working files in a university or organization-approved system when available. Require sign-in, grant access only to named collaborators, and use multifactor authentication where supported. Keep a separate backup in an approved location and test that you can restore it. Avoid sending drafts or data to public paste sites, anonymous file-sharing links, or tools whose data handling you have not reviewed.
Review connected apps and integrations. Check which browser extensions, research tools, automated workflows, and third-party apps can read or sync your files. Revoke access that is no longer required, and avoid granting broad permissions for convenience. Before using a writing, transcription, coding, or summarization service, check whether its terms allow submitted content to be retained or used for service improvement. Use only options cleared for the material’s sensitivity.
Ask coauthors and collaborators to follow the same safeguards. Agree on which draft versions may be shared, where they may be stored, and who can grant access. Ask collaborators to remove public copies and avoid uploading restricted content to unapproved services. For a project with external partners, document handling requirements and a contact for reporting accidental exposure.
Recheck periodically and before release. Review sharing permissions after staff changes, project milestones, or platform updates, and before submitting or publishing. Search for public copies again using non-sensitive phrases. When the work is ready for release, publish only the version and data approved for distribution, and apply the project’s licensing, attribution, and data-sharing decisions deliberately.

Common Mistakes

Relying on robots.txt alone: it asks compliant crawlers to stay away; it does not block access or stop noncompliant scraping.
Assuming “unlisted” means private: an unlisted page or link may still be forwarded, discovered, or accessed by a crawler. Use authentication and explicit permissions for confidential work.
Confusing removal with erasure: taking down a page does not guarantee that cached, archived, downloaded, or previously copied versions disappear.
Changing a shared folder without checking individual files: inherited permissions and direct links can behave differently. Verify the effective access for important documents.
Uploading a sample to a checker: a service that claims to detect scraping or AI use may itself receive and retain the text. Check its data practices before submitting anything.

Tips

Use a simple access list: file or location, owner, authorized people, sharing method, and next review date.
Keep a clean public project page with only approved abstracts, citations, and released materials; keep drafts in a restricted workspace.
Use version history and backups so accidental deletion or unwanted edits do not force you to loosen access controls.
When you do publish, consider a clear copyright notice or license. It communicates your terms but cannot technically prevent copying or determine how every service will handle the work.
For research involving people or confidential data, follow institutional policy and ask the data protection, ethics, or information-security contact before changing storage or sharing arrangements.

Troubleshooting

A supposedly private page appears in search: confirm the page is actually access-restricted, not merely marked “noindex.” Remove public access at the host, then request removal through the search engine’s available outdated-content process. Search-result removal may take time and does not remove copies elsewhere.
A public link still works after changing folder settings: inspect the file’s own sharing panel, inherited permissions, and link-sharing options. Revoke the link, test in a signed-out or separate browser session, and ask an administrator to check organization-wide sharing rules.
You find a copy on a site you do not control: save the URL and relevant dates without redistributing the material. Contact the site operator or hosting provider through its published reporting process, and consult your institution about appropriate next steps. Do not assume a request will remove every copy.
A service offers no clear training or retention setting: treat the content as potentially retained or used according to the provider’s stated terms. Do not submit sensitive unpublished material until your institution or the provider confirms an acceptable handling option.
You suspect an accidental disclosure: restrict access immediately if possible, preserve a record of what was exposed and when, and notify your supervisor or institutional security/privacy contact promptly. Follow their incident process rather than quietly deleting evidence or contacting affected parties on your own.

Frequently Asked Questions

Does robots.txt stop AI companies from scraping my website?
No. It communicates crawler preferences to compliant bots, but it is not a technical barrier. Use authentication or remove the content from public access when it must remain private.

Can I stop a service from training on a manuscript I already uploaded?
Check that service’s current controls and terms, then contact its support or your institution to ask about deletion and retention. A setting change may not undo earlier processing or copies.

Will a copyright notice prevent AI training?
A notice can state ownership and intended terms, but it cannot reliably prevent downloading or scraping. Pair it with access controls and suitable hosting practices.

Should I put unpublished work on a private repository?
It can be appropriate if the repository supports restricted access and meets your institution’s requirements. Verify visibility, link sharing, retention, and collaborator permissions before uploading.

Can I remove research from search results after taking it down?
You can request that a search engine refresh or remove an outdated result, but processing takes time. The request does not erase copies held by other sites, archives, or people who downloaded the file.

What if a coauthor has already shared a draft publicly?
Ask them to restrict or remove it, check whether the host offers cached-copy or removal requests, and notify the project lead or institution if the material is sensitive. Coordinate before changing shared files.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.