How to Merge Folders & Directories in Python

Python provides many functions and packages to easily work with files & folders. Sometimes you may need to combine folders & directories in Python. This is especially true in today’s world where you have so much data, organized in numerous files and folders. It is tedious to manually move folders and their contents from one place to another. Since there is no built-in Python function to do this, you will need to code your way out. Luckily, you can do this using os and shutil libraries. In this article, we will learn how to merge folders & directories in Python.

How to Merge Folders & Directories in Python

For our purpose, we will use os and shutil modules. os library provides tons of features for Python to interact with OS features such as its file system. shutil stands for shell utilities and provides a rich collection of functions to work with files and directories. It allows you to easily copy, move, rename and delete files and folders.

1. Create Python File

First, open terminal and run the following command to create a Python file. If you are using Windows, use a text editor like Notepad++ to create the Python file.

$ vi merge_folder.py

2. Import Libraries

Add the following lines to your Python script to set interpreter environment and import packages.

#!/usr/bin/env python

import os
import shutil

In the above code, the first line sets the execution environment and the next line imports libraries os and shutil.

3. Define Folder Paths

Next, we need to specify the folder paths to source and destination folders. Source folder is the folder to be merged and destination folder is the folder where we want these folders to be merged into.

source_directory = "path/to/source/folder"
destination_directory = "path/to/destination/folder"

We will merge all the content of source_directory into destination_directory.

4. Loop Through Source Directory

Next, we will loop through the source directory to get a list of all its files and sub directories. We will use os.listdir() function for this purpose. It returns a list of relative paths to all files & directories in input folder.

We will iterate through this list of file & directory paths using a for loop. In each iteration, we will concatenate the relative path of the list item with the source directory’s path to get full path to the file/directory. We will store it in source_file variable. We will also concatenate the path to file/directory with the destination folder path to obtain the full path to its final destination. We will store it in destination_file.

for file in os.listdir(source_directory):
  source_file = os.path.join(source_directory, file)
  destination_file = os.path.join(destination_directory, file)

5. Merge Directory Contents

Once you have defined the full paths to source and destination files, there are several ways to merge the folder contents. We will look at 4 different ways to do this.

Move Files & Folders

The simplest way is to just call shutil.move() function to move the files & folders from source directory to destination directory. If the destination directory does not exist, then it will be created. Add the following line at the end of your for loop above.

shutil.move(source_file, destination_file)

In this case, the above for loop will look like

for file in os.listdir(source_directory):
  source_file = os.path.join(source_directory, file)
  destination_file = os.path.join(destination_directory, file)
  shutil.move(source_file, destination_file)

Copy Files & Folders Without Metadata

If you only want to copy all files and subdirectories in source directory to the destination, without including their metadata, use shutil.copy() function.

shutil.copy(source_file, destination_file)

In this case, your for loop will look like

for file in os.listdir(source_directory):
  source_file = os.path.join(source_directory, file)
  destination_file = os.path.join(destination_directory, file)
  shutil.copy(source_file, destination_file)

Copy Files & Folders With Metadata

If you want to copy files & directories from source folder to destination folder, including their metadata, use shutil.copy2() function.

shutil.copy(source_file, destination_file)

In this case, your loop will look like

for file in os.listdir(source_directory):
  source_file = os.path.join(source_directory, file)
  destination_file = os.path.join(destination_directory, file)
  shutil.copy2(source_file, destination_file)

Create New Destination Directory Before Copying

If you explicitly want to create the destination directory before copying source folder’s contents into it, you can use shutil.copytree().

shutil.copytree(source_file, destination_file)

Then your for loop with look like the following.

for file in os.listdir(source_directory):
  source_file = os.path.join(source_directory, file)
  destination_file = os.path.join(destination_directory, file)
  shutil.copytree(source_file, destination_file)

6. Test Python Script

Your Python script will look like the following. The last line will vary depending on the function you choose to use.

#!/usr/bin/env python

import os
import shutil

source_directory = "path/to/source/folder"
destination_directory = "path/to/destination/folder"

for file in os.listdir(source_directory):
  source_file = os.path.join(source_directory, file)
  destination_file = os.path.join(destination_directory, file)
  shutil.move(source_file, destination_file)

Save and close the file. Make it into an executable using chmod function.

$ sudo chmod +x merge_folder.py

Test it with the following command.

$ python merge_folder.py

Conclusion

It can be challenging to manage large number of files and directories. It is easier to have a script that you can run to automate this. In this article, we have learnt 4 different ways to merge folders & directories in python. The basic structure of the script remains the same, except the last line. In all cases, you need to import libraries, define source & destination folder paths, loop through source folder to generate full source & destination paths, and use shutil functions to move/copy content. You can choose to move or copy the source folder’s contents, depending on your requirement. However, this method is to be used only when you need to merge folders from within your Python script or application. Otherwise, it is better to use Shell Scripting and commands for this purpose. This is because you can easily do it using cp/mv command in just one line.

How to Merge Folders & Directories in Python