How to Parse CSV File in Shell

Sometimes you need to parse CSV file in shell, or read data from csv file. In fact, this is a common requirement in data processing and analytics systems. In this article, we will look at how to parse CSV file in shell. We will create a simple script to help you parse csv file. You can use it in most Linux systems such as Ubuntu, Debian, CentOS, Redhat, Fedora, CentOS.

How to Parse CSV File in Shell

Here are the steps to parse csv file in shell script.

1. Create empty shell script file

Open terminal and run the following command to create empty shell script file.

$ sudo vi parse_csv.sh

2. Add shell commands

Let us say, you have /home/input.csv file as follows

$ cat /home/input.csv

Output
id, qty, price, value
1,100,150,15
2,250,200,100
3,300,200,150

Add the following commands to your shell script. Replace /home/input.csv with your csv file’s path.

#! /bin/bash
while IFS="," read -r column1 column2 column3 column4
do
  echo "ID : $column1"
  echo "Quantity: $column2"
  echo "Price: $column3"
  echo "Value: $column4"
  echo ""
done <  <(tail -n +2 /home/input.csv)

In the above code, the first line specifies execution environment. We set the IFS (Input Field Separator) to “,”. We use read command to parse comma-delimited values into bash shell. In the above code we have skipped the header line 1, using tail command

done <  <(tail -n +2 /home/input.csv)

If you don’t want to skip header line, replace it with

done <  /home/input.csv

Also we are displaying one column value per line, per row. If you want to print all columns of each row per line, replace do..done

do
  echo "ID : $column1"
  echo "Quantity: $column2"
  echo "Price: $column3"
  echo "Value: $column4"
  echo ""
done 

with the following

do
  echo "ID : $column1" "Quantity: $column2" "Price: $column3" "Value: $column4"
  echo ""
done <  <(tail -n +2 /home/input.csv)

3. Make shell script executable

Run the following command to make your shell script executable.

$ sudo chmod +x parse_csv.sh

4. Run the shell script

Run the shell script with following command.

$ sudo ./parse_csv.sh

Output
ID : 1
Quantity : 100
Price : 150
Value: 15
ID : 2
Quantity : 250
Price : 200
Value: 100
ID : 3
Quantity : 300
Price : 200
Value: 150

In this article, we have looked at how to parse csv file in shell script. You may customize it as per your requirements. You can use it for almost every Linux system such as Ubuntu, Debian, Redhat, Fedora, CentOS, since we have used built-in functions of shell.

Also read:

How to Find All Symlinks to Folder/Directory
Shell Script to Count Number of Files in Directory
Shell Script to Count Number of Words in File
How to Enable & Disable Services in Linux
NGINX Alias vs Root