Sometimes you need to parse CSV file in shell, or read data from csv file. In fact, this is a common requirement in data processing and analytics systems. In this article, we will look at how to parse CSV file in shell. We will create a simple script to help you parse csv file. You can use it in most Linux systems such as Ubuntu, Debian, CentOS, Redhat, Fedora, CentOS.
How to Parse CSV File in Shell
Here are the steps to parse csv file in shell script.
1. Create empty shell script file
Open terminal and run the following command to create empty shell script file.
$ sudo vi parse_csv.sh
2. Add shell commands
Let us say, you have /home/input.csv file as follows
$ cat /home/input.csv Output id, qty, price, value 1,100,150,15 2,250,200,100 3,300,200,150
Add the following commands to your shell script. Replace /home/input.csv with your csv file’s path.
#! /bin/bash while IFS="," read -r column1 column2 column3 column4 do echo "ID : $column1" echo "Quantity: $column2" echo "Price: $column3" echo "Value: $column4" echo "" done < <(tail -n +2 /home/input.csv)
In the above code, the first line specifies execution environment. We set the IFS (Input Field Separator) to “,”. We use read command to parse comma-delimited values into bash shell. In the above code we have skipped the header line 1, using tail command
done < <(tail -n +2 /home/input.csv)
If you don’t want to skip header line, replace it with
done < /home/input.csv
Also we are displaying one column value per line, per row. If you want to print all columns of each row per line, replace do..done
do echo "ID : $column1" echo "Quantity: $column2" echo "Price: $column3" echo "Value: $column4" echo "" done
with the following
do echo "ID : $column1" "Quantity: $column2" "Price: $column3" "Value: $column4" echo "" done < <(tail -n +2 /home/input.csv)
3. Make shell script executable
Run the following command to make your shell script executable.
$ sudo chmod +x parse_csv.sh
4. Run the shell script
Run the shell script with following command.
$ sudo ./parse_csv.sh Output ID : 1 Quantity : 100 Price : 150 Value: 15 ID : 2 Quantity : 250 Price : 200 Value: 100 ID : 3 Quantity : 300 Price : 200 Value: 150
In this article, we have looked at how to parse csv file in shell script. You may customize it as per your requirements. You can use it for almost every Linux system such as Ubuntu, Debian, Redhat, Fedora, CentOS, since we have used built-in functions of shell.