Sponsored Content
Top Forums Shell Programming and Scripting How to avoid Delimiter occuring in column values in .csv file Post 302661427 by Sheel on Monday 25th of June 2012 09:04:33 AM
Old 06-25-2012
It worked fine for me. See if there are syntactical errors in your file. Here is the script you might be looking for :

Script - read.ksh
Code:
# STEP 1 - Replace the commas within double quotes with pipe & write it to a temp file
awk -F'"' 'BEGIN{OFS="\""}{gsub(",","|",$2);print}' inputFile.csv > tempFile.csv
 
# STEP 2 - Get the first and last columns from temp file
while read line
do
        firstColumn=`echo $line|awk -F"," '{print $1}'`
        echo "First Column= $firstColumn"
        lastColumn=`echo $line|awk -F"," '{print $NF}'`
        echo "Last Column= $lastColumn"
 
# STEP 3 - Replace pipe with comma
 finalLastColumn=`echo $lastColumn | sed 's/\|/\,/g'`
 echo "Finally Last Column= $finalLastColumn"
done < tempFile.csv

inputFile.csv
Code:
"asdf, tew,123",abcd,123456
abcd,"asdf, tew,123",123456
abcd,1234,"asdf, tew,123"

Test Output
Code:
./read.ksh
First Column= "asdf| tew|123"
Last Column= 123456
Finally Last Column= 123456
First Column= abcd
Last Column= 123456
Finally Last Column= 123456
First Column= abcd
Last Column= "asdf| tew|123"
Finally Last Column= "asdf, tew,123"

Missed reverting pipe to comma in first column. You can add these lines after step 3
Code:
finalFirstColumn=`echo $firstColumn | sed 's/\|/\,/g'`
 echo "Finally first Column= $finalFirstColumn"


Last edited by Sheel; 06-25-2012 at 10:12 AM..
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

2. Shell Programming and Scripting

Check to identify duplicate values at first column in csv file

Hello experts, I have a requirement where I have to implement two checks on a csv file: 1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit. 2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
Discussion started by: avikaljain
4 Replies

3. Shell Programming and Scripting

How to delete a column/columns of a CSV file which has cell values with a string enclosed in " , "?

How can I delete a column from a CSV file which has comma separated value with a string enclosed in double quotes and a comma in between? I have a file 44.csv with 4 lines including the header like the below format: column1, column2, column3, column 4, column5, column6 12,455,"string with... (6 Replies)
Discussion started by: dhruuv369
6 Replies

4. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

5. Linux

Filter a .CSV file based on the 5th column values

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Discussion started by: dhruuv369
2 Replies

6. Shell Programming and Scripting

Fetching values in CSV file based on column name

input.csv: Field1,Field2,Field3,Field4,Field4 abc ,123 ,xyz ,000 ,pqr mno ,123 ,dfr ,111 ,bbb output: Field2,Field4 123 ,000 123 ,111 how to fetch the values of Field4 where Field2='123' I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Discussion started by: bharathbangalor
10 Replies

7. Shell Programming and Scripting

Remove the values from a certain column without deleting the Column name in a .CSV file

(14 Replies)
Discussion started by: dhruuv369
14 Replies

8. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

9. Shell Programming and Scripting

Identify duplicate values at first column in csv file

Input 1,ABCD,no 2,system,yes 3,ABCD,yes 4,XYZ,no 5,XYZ,yes 6,pc,noCode used to find duplicate with regard to 2nd column awk 'NR == 1 {p=$2; next} p == $2 { print "Line" NR "$2 is duplicated"} {p=$2}' FS="," ./input.csv Now is there a wise way to de-duplicate the entire line (remove... (4 Replies)
Discussion started by: deadyetagain
4 Replies
BUILDAPP(1)							   User Commands						       BUILDAPP(1)

NAME
buildapp - application to create common lisp images SYNOPSIS
buildapp --output OUTPUT-FILE [--flag1 value1 ...] DESCRIPTION
Required flags: --output OUTPUT-FILE Use OUTPUT-FILE as the name of the executable to create Entry-point flags: --entry NAME Use the function identified by NAME as the executable's toplevel function. Called with SB-EXT:*POSIX-ARGV* as its only argument. If NAME has a colon, it is treated as a package separator, otherwise CL-USER is the implied package. --dispatched-entry DNAME Specify one possible entry function, depending on the name of the file that is used to start the application. The syntax of DNAME is APPLICATION-NAME/ENTRY-NAME. If the name used to start the executable matches APPLICATION-NAME, use ENTRY-NAME as the entry point. This can be used to choose one of many possible entry points by e.g. symlinking names to the application executable. If APPLICA- TION-NAME is empty, the specified ENTRY-NAME is used as a default if no other application names match. There may be any number of dispatched entry points, but only one default. Action flags: --load FILE Load FILE. CL:*PACKAGE* is bound to the CL-USER package before loading --load-system NAME Load an ASDF system identified by NAME --require NAME Use CL:REQUIRE to load NAME --eval CODE Use CL:EVAL to evaulate CODE. The code is read with CL:READ-FROM-STRING in the CL-USER package There may be any number of load/load-system/require/eval flags. Each is executed in command-line order before creating an executable. Load path flags: --load-path DIRECTORY When handling a --load, search DIRECTORY for files to load --asdf-path DIRECTORY When handling a --load-system, search DIRECTORY for ASDF system files to load --asdf-tree DIRECTORY When handling a --load-system, search DIRECTORY and all its subdirectories for ASDF system files to load There may be any number of load-path/asdf-path/asdf-tree flags. asdf-path arguments take precedence over asdf-tree arguments. Other flags: --help Show this usage message --logfile FILE Log compilation and load output to FILE --sbcl PATH-TO-SBCL Use PATH-TO-SBCL instead of the sbcl program found in your PATH environment variable For the latest documentation, see http://www.xach.com/lisp/buildapp/ buildapp 1.1 July 2010 BUILDAPP(1)
All times are GMT -4. The time now is 10:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy