How to avoid Delimiter occuring in column values in .csv file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to avoid Delimiter occuring in column values in .csv file
# 8  
Old 06-25-2012
@Sheel, the code you gave is giving me syntax error Smilie

Can this be done by writing a script if a one line command is not possible.
# 9  
Old 06-25-2012
Whats the error ?
# 10  
Old 06-25-2012
Code:
Code:
awk -F'"' 'BEGIN{OFS="\""}{gsub(",","|",$2);print}' file.csv
awk: syntax error near line 1
awk: illegal statement near line 1


Last edited by Franklin52; 06-25-2012 at 09:57 AM.. Reason: Please use code tags for data and code samples
# 11  
Old 06-25-2012
It worked fine for me. See if there are syntactical errors in your file. Here is the script you might be looking for :

Script - read.ksh
Code:
# STEP 1 - Replace the commas within double quotes with pipe & write it to a temp file
awk -F'"' 'BEGIN{OFS="\""}{gsub(",","|",$2);print}' inputFile.csv > tempFile.csv
 
# STEP 2 - Get the first and last columns from temp file
while read line
do
        firstColumn=`echo $line|awk -F"," '{print $1}'`
        echo "First Column= $firstColumn"
        lastColumn=`echo $line|awk -F"," '{print $NF}'`
        echo "Last Column= $lastColumn"
 
# STEP 3 - Replace pipe with comma
 finalLastColumn=`echo $lastColumn | sed 's/\|/\,/g'`
 echo "Finally Last Column= $finalLastColumn"
done < tempFile.csv

inputFile.csv
Code:
"asdf, tew,123",abcd,123456
abcd,"asdf, tew,123",123456
abcd,1234,"asdf, tew,123"

Test Output
Code:
./read.ksh
First Column= "asdf| tew|123"
Last Column= 123456
Finally Last Column= 123456
First Column= abcd
Last Column= 123456
Finally Last Column= 123456
First Column= abcd
Last Column= "asdf| tew|123"
Finally Last Column= "asdf, tew,123"

Missed reverting pipe to comma in first column. You can add these lines after step 3
Code:
finalFirstColumn=`echo $firstColumn | sed 's/\|/\,/g'`
 echo "Finally first Column= $finalFirstColumn"


Last edited by Sheel; 06-25-2012 at 10:12 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Identify duplicate values at first column in csv file

Input 1,ABCD,no 2,system,yes 3,ABCD,yes 4,XYZ,no 5,XYZ,yes 6,pc,noCode used to find duplicate with regard to 2nd column awk 'NR == 1 {p=$2; next} p == $2 { print "Line" NR "$2 is duplicated"} {p=$2}' FS="," ./input.csv Now is there a wise way to de-duplicate the entire line (remove... (4 Replies)
Discussion started by: deadyetagain
4 Replies

2. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

3. Shell Programming and Scripting

Remove the values from a certain column without deleting the Column name in a .CSV file

(14 Replies)
Discussion started by: dhruuv369
14 Replies

4. Shell Programming and Scripting

Fetching values in CSV file based on column name

input.csv: Field1,Field2,Field3,Field4,Field4 abc ,123 ,xyz ,000 ,pqr mno ,123 ,dfr ,111 ,bbb output: Field2,Field4 123 ,000 123 ,111 how to fetch the values of Field4 where Field2='123' I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Discussion started by: bharathbangalor
10 Replies

5. Linux

Filter a .CSV file based on the 5th column values

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Discussion started by: dhruuv369
2 Replies

6. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

7. Shell Programming and Scripting

How to delete a column/columns of a CSV file which has cell values with a string enclosed in " , "?

How can I delete a column from a CSV file which has comma separated value with a string enclosed in double quotes and a comma in between? I have a file 44.csv with 4 lines including the header like the below format: column1, column2, column3, column 4, column5, column6 12,455,"string with... (6 Replies)
Discussion started by: dhruuv369
6 Replies

8. Shell Programming and Scripting

Check to identify duplicate values at first column in csv file

Hello experts, I have a requirement where I have to implement two checks on a csv file: 1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit. 2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
Discussion started by: avikaljain
4 Replies

9. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies
Login or Register to Ask a Question