How to Split Large File into Smaller Files

Often you may need to split a large file into smaller files in Linux, or even combine larger files into single combined file. In this article, we will learn how to split large file into smaller files.

How to Split Large File into Smaller Files

We will use split command to easily split large file into smaller files. Let us say you have a large file of 10Mb.

Split By Size

Here is the syntax of split command to split a large file into smaller files, by specifying the size of each smaller file.

split --bytes=<size of smaller files> /path/to/big/file prefix_for_smaller_files

Here is an example to split our large image /home/ubuntu/image.jpg into smaller files of 1mb size each.

$ split --bytes=1M /home/ubuntu/image.jpg new
OR
$ split -b=1M /home/ubuntu/image.jpg new

Now split command will split the 10Mb file into 10 1Mb files each with prefix new. The filenames of smaller files will be newaa.jpg, newab.jpg, …

If the specified file size doesn’t exactly divide the original file, then the last file will be smaller the specified byte size, with leftover data.

Please note, you can specify bytes (–bytes) as actual number of bytes (e.g. 500) or as short forms k, M, G respectively – for example 4k for 4kB, 5M for 5MB and 6G for 6Gb.

This method is very useful to split images and binary files which can be split based on file size.

Split by Lines

You can also split a large file into small files by specifying the number of lines using -l option. Here is the syntax to split a large file into smaller files by specifying the number of lines in each file.

split -l no_of_lines /path/to/large/file prefix_for_small_files

Here is an example to split large file /home/ubuntu/image.png into small files each with 1000 lines.

split -l 1000 /home/ubuntu/image.png new

In this case, split command will split the large image image.png into smaller files, each with 1000 lines in it.

This method is useful to split text based files such as codes, scripts, logs, SQL dumps, csv, text files.

Please note, the above split commands can be used for all file types, not just images.

Combine Smaller Files into Large File

If you want to combine the above created smaller files into large file, you can combine them using cat command. Here is a command to combine all the smaller files starting with prefix new into a larger file.

cat new* > newimage.jpg

In this case, the cat command will concatenate the smaller files into a large file newimage.jpg

Of course, there are many other ways to split a large file into smaller files and combine them back into the large file. We have used the simplest ways to do this using in-built tools like split and cat which are already present on most Linux systems. Also their syntax is so simple that you can easily embed this into your shell script or cronjobs.

Also read:

How to Sort Files into Folders in Linux
How to Add Newline After Pattern in Vim
How to Convert CRLF to LF in Linux
How to Store Command in Variable in Shell Script
Tar Directory to Exclude Files & Folders