Bash Loop Through Files in Directory and Subdirectories

Often you may need to loop through files in directory and subdirectory to perform some task or the other. In this article, we will learn how to do this in a couple of simple ways. You can use these commands on almost every Linux distribution.

Bash Loop Through Files in Directory and Subdirectories

Here are a couple of ways to loop through file in directory & subdirectories in Bash.

1. Using find command

You can use find command to list all files that meet your requirements. It has the option to recursively search folders & subfolders. Here is the command to find and delete all .pdf and .doc files in /tmp subfolder on your system.

$ find /tmp -name '*.pdf' -or -name '*.doc' | xargs rm

The find command will simply list all the files in /tmp and its subfolders. The xargs commands will construct separate rm statements for each file present in the output of find command.

Alternatively, you can also use exec option in find command to delete files or perform some other action on them.

$ find /tmp -name '*.pdf' -or -name '*.doc' -exec rm -f {} \;

2. Using for loop

While find command is the best way to loop through folders and subfolders in Linux, it provides limited scope to perform complex operations on each file. In such cases, it is better to fall back to the good old ‘for loop’, to go through files in folders & subfolders and perform required tasks with them. Here is an example to loop through the files and folders in /tmp folder.

for f in /tmp/* tmp/**/* ; do

  # code to do something
  ...
done;

In the above code, we use /tmp/* to loop through the files and /tmp/*/** to for the subfolders in /tmp. Here is an example to remove all .pdf and .doc files in /tmp folder. This allows you to add multiple complex statements if we want.

for f in /tmp/*.pdf /tmp/*.doc tmp/**/*.pdf tmp/**/*.doc ; do
  rm "$f"
done

Alternatively, you can also run a for loop on the output of find command we used above. In this case, you don’t need to use xargs whose syntax can get tricky for complicated operations. Here you simply need to place your commands between do…done block.

for f in $(find /tmp -name '*.pdf' -or -name '*.doc'); do rm $f; done

But in this case, the shell loops through the files twice, once while executing find command and then again while running the for loop, so it can take some time, if there are too many files & folders to be listed.

Please note, when you use for loop, depending on your shell type and version, you may need to enable globstar with the following command, before the for loop. Otherwise, it may not recursively loop through the subdirectories.

shopt -s globstar
for f in /path/to/dir; do
...
done

Nevertheless, you can use any of the above methods, as per your requirements. You can even use these commands in shell scripts, or as cronjobs to automate tasks.

In this article, we have learnt how to loop through folders & subdirectories in bash, using find command as well as using for loop.

Also read:

How to Remove Git Ignore Files in Git Repository
How to Stop Python Code Execution After Certain Time
How to Convert Bytes to String in Python
How to Store Output of Cut Command in Shell Variable
How to Use Shell Variables in Awk Script