How to Use Curl to Download Files in Linux

cURL is a useful tool that allows you to easily transfer data and files across systems over a network. It can be used as a tool as well as library. cURL supports all major protocols such as HTTP, HTTPS, FTP, SFTP and more. In this article, we will look at how to use curl to download files in Linux.

How to Use Curl to Download Files in Linux

Here are the steps to use curl to download files in Linux.

How to Install cURL

By default, curl is already installed on most Linux systems. However, if it is not present on your system, you can install it using the following command.

## Debian/Ubuntu Linux
$ sudo apt install curl
## Fedora/CentOS/RHEL
$ sudo dnf install curl
## OpenSUSE Linux users
$ sudo zypper install curl

You can check if cURL is installed on your system, with the following command

$ curl --version

Also read : How to Set or Change Time Zone in Ubuntu

We will look at some common example of using cURL command.

Download Files using cURL

The basic syntax of cURL is

curl [options] url_with_protocol

You need to specify the URL with protocol, from where you want to download the file.

Here is an example to download a file from a URL.

$ sudo curl --output your_file.pdf https://www.example.com/my_file.pdf 
OR
$ sudo curl -o your_file.pdf https://www.example.com/my_file.pdf

If you don’t specify the -o or –output option curl will save the downloaded file with a random name.

If you want to save the downloaded file with the same name, use -O (capital O) option

$ sudo curl -O https://www.example.com/my_file.pdf

Also read : How to Implement SSL/TLS in Apache Tomcat

Resume failed or paused downloads

If for some reason, your download was interrupted, then you can resume file download with the -C command

$ sudo curl -C - -O https://www.example.com/my_file.pdf

Download files with URL redirect

By default, curl does not download a file from a URL which gets redirected. You can do so using -L option.

$ sudo curl -L -O https://www.example.com/my_file.pdf

Also read : How to Extract data from JSON file using Python

Download Multiple Files

If you want to download multiple files from different URLs then you need to add each URL separately with its own -O option.

$ sudo curl -O url1 -O url2

$ sudo curl -O https://www.example.com/my_file.pdf \
            -O https://www.example.com/my_file2.pdf

If you have these URLs stored in a file (e.g. URLs.txt) then you can ask curl to pick those URLs from the file and download them, using -n command. Make sure you list 1 URL on each line of the file.

URLs.txt
------
https://www.example.com/my_file.pdf
https://www.example.com/my_file2.pdf
https://www.example.com/my_file3.pdf

Here is the command to download a list of URLs. We also use xargs command along with curl for this purpose.

xargs -n 1 curl -O < "urls.txt"

Also read : How to Install Laravel with NGINX

Download password protected file

Sometimes the file you want to download may be password protected. Here is the command to download a password protected file. You need to mention the username & password with -u option. Replace test_user & test_password below with your username & password.

$ curl -u test_user:test_password https://www.example.com/my_file.pdf

Download Files Using Proxy Server

Here is the syntax to download files using proxy server.

curl -x proxy-server-ip:PORT -O url

If it requires authentication, then you need to also provide username & password.

curl -u username:password -x proxy-server-ip:PORT -O url

Here is an example,

curl -x 'https://proxy.example.com' -v -O https://www.example.com/my_file.pdf

Also read : How to Check Cron Log with Linux

Rate Limits in Download

If you want to limit the download speed of your files, you can do so with the –limit-rate option.

curl --limit-rate {speed} url

Here is an example to limit download rate to 200kb/sec.

$ sudo curl --limit-rate 200k -O https://www.example.com/my_file.pdf

There are many more things you can do with cURL, but we have covered the most common ones. You can also import cURL as a library in Python/Java, which we will cover in another article.

Also read : How to Install CouchDB in Ubuntu