Linux Find Duplicate Files: Tools & Commands

Updated 6 November 2026 · Standard formula, shown below

To find duplicate files on Linux, you can use command-line tools like fdupes, rdfind, or fclones, or GUI applications like dupeGuru and czkawka. These tools compare file contents (usually by hash) and list or remove duplicates, saving disk space and reducing clutter.

Why Duplicates Happen

Duplicate files accumulate from various everyday activities: downloading the same attachment multiple times, copying photos from a camera without cleaning up, syncing folders across devices, or unzipping archives that contain files already present elsewhere. Backups and versioned folders often leave copies that are never cleaned. Even well-organized systems can harbor duplicates after a few years of use.

The problem is more than wasted disk space. Duplicates can cause confusion—editing the wrong copy, searching through redundant results, or accidentally deleting a version you needed. On large media libraries, duplicates also slow down backups and indexing. Identifying and removing them is a common maintenance task for Linux users.

How Duplicate Detection Works

Most tools follow the same basic strategy:

Size comparison: Files with different sizes cannot be identical, so tools first group files by size. This quickly eliminates most candidates.
Hash calculation: For files of the same size, the tool computes a cryptographic hash (like MD5, SHA-1, or SHA-256) of the contents. Identical hashes mean identical content (with negligible collision risk for SHA-1 and stronger).
Byte-by-byte verification: To be absolutely certain, some tools re-read the files and compare them byte by byte, avoiding any hash collision possibility.

This three-step process is efficient because hashing is fast but not free; grouping by size first dramatically reduces the number of files that need hashing.

Command-Line Tools

Here are the most popular command-line utilities, each with a slightly different focus:

fdupes: One of the oldest and most widely used. It compares files in given directories, lists duplicates, and can delete them interactively or automatically. Example: fdupes -r /home/user/Documents recursively finds duplicates.
rdfind: Uses a ranking algorithm to decide which file to keep (e.g., the one with the oldest creation time). It can also replace duplicates with hard links, saving space without deleting any file. Example: rdfind -makehardlinks true /home/user/Pictures.
fclones: A modern Rust-based tool that is extremely fast, especially on large datasets. It supports parallel processing and can group duplicates by content or by inode. Example: fclones group /home/user.
jdupes: A fork of fdupes that adds more options, such as ignoring hard links and using a faster hash. It is also available for Windows and macOS.

All of these are available in most distribution repositories. Install them with your package manager, e.g., sudo apt install fdupes on Debian/Ubuntu.

GUI Tools

If you prefer a graphical interface, several excellent options exist:

dupeGuru: Cross-platform, supports fuzzy matching for images and audio, and lets you preview files before deleting. It has a clean interface and is available in many repos.
czkawka: A fast, open-source tool written in Rust. It can find duplicate files, similar images, empty folders, and even broken files. The GUI is straightforward and shows previews.
fslint: An older utility that includes a GUI front-end for finding duplicates and other file-system issues. It is less maintained but still works.

GUI tools are particularly useful when you need to visually confirm that two files are indeed duplicates, especially for images or documents where the filename might differ.

Choosing the Right Tool

Your choice depends on your specific needs:

Quick cleanup: Use fdupes or jdupes for simple, interactive deletion.
Large datasets: Use fclones or rdfind for speed and efficiency.
Hard link replacement: Use rdfind with the -makehardlinks option to save space without deleting files.
Image/audio similarity: Use dupeGuru for fuzzy matching of similar (not identical) files.
GUI preference: Use czkawka or dupeGuru for a visual experience.

For most users, fdupes is the simplest starting point. It is well-documented, stable, and available everywhere.

Common Pitfalls and Safety

Before deleting duplicates, consider these important points:

Hard links: If two filenames point to the same inode (hard links), they are not duplicates in the sense of wasting space—they are the same file. Tools like fdupes can ignore hard links with the -H option.
Symlinks: Symbolic links are not duplicates; they are pointers. Most tools skip them by default.
Different filenames, same content: A file named report_final.pdf and report_FINAL.pdf may be identical. Tools will find them, but you must decide which name to keep.
Backup before deleting: Always make a backup or move duplicates to a temporary folder instead of deleting immediately. Many tools have a --dry-run option to show what would be done.
Permissions: You need read permission to hash files and write permission to delete them. Running as root is not recommended; use sudo only when necessary.

One common misconception is that hash collisions are a practical risk. For SHA-1 and stronger, the chance of two different files having the same hash is astronomically small. Still, tools like fdupes do a byte-for-byte check after hashing to be 100% sure.

Alternatives and Related Techniques

If you don’t want to install a dedicated tool, you can use standard Linux commands to find duplicates:

Using md5sum and sort: A one-liner like find . -type f -exec md5sum {} + | sort | awk '{if ($1 == prev) print $2; prev=$1}' lists files with duplicate hashes. This is a quick hack but not as efficient as dedicated tools.
Using find with -size: You can first list files of the same size, then manually compare. This is impractical for large sets.
Deduplication at the file-system level: Some file systems (like ZFS or Btrfs) support block-level deduplication, which automatically avoids storing duplicate blocks. This is a more advanced solution that requires planning at setup time.

These alternatives are useful when you cannot install new packages, but for regular maintenance, a dedicated tool is safer and faster.

Frequently Asked Questions

How do I find duplicate files in a specific directory?

Use fdupes -r /path/to/directory to search recursively. The -r flag tells it to go into subdirectories. You can also specify multiple directories: fdupes dir1 dir2 will find duplicates across both.

Can I delete duplicates automatically?

Yes, but with caution. fdupes -d will prompt you to choose which files to delete. For non-interactive deletion, you can use fdupes -d -N (keep the first file, delete the rest) or fdupes -d -N -f (keep the first, delete others, and list them). Always test with --dry-run first.

What is the difference between fdupes and rdfind?

Both find duplicates, but rdfind can automatically replace duplicates with hard links, which saves space without deleting any file. fdupes focuses on listing and deleting. rdfind also has a ranking algorithm to decide which file to keep based on age or path.

Are there any GUI tools for finding duplicates?

Yes, dupeGuru and czkawka are popular graphical tools. dupeGuru supports fuzzy matching for images and music, while czkawka is fast and also finds similar images, empty folders, and broken files.

How can I avoid creating duplicates in the future?

Use a file manager that warns you when copying a file with the same name, or use a deduplication tool periodically. For downloads, consider using a download manager that checks for existing files. Also, avoid storing multiple versions of the same document; use version control instead.

Is it safe to delete duplicates found by these tools?

Generally yes, but always review the list before deleting. Some files may have different names but identical content, and you might want to keep both if they are in different contexts. Always back up important data first, and use the tool’s dry-run option to see what will be deleted.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.