Help -- Modify awk command to handle "," delimiter


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help -- Modify awk command to handle "," delimiter
# 1  
Old 07-30-2014
Question Help -- Modify awk command to handle "," delimiter

Currently I am using this command to split a CSV file based on the distinct values in the 16th (position) column.

Code:
awk -F, 'NR==1 { hdr=$0; next } $16 != prev { prev=name=$16; gsub(/[^[:alnum:]_]/,"",name); $0 = hdr "\n" $0 } { print > ("/Directory/File."name".Download.csv") }' /Directory/File.ALL.Download.csv

Below is a sample of the data:

Code:
"Column1-Name", "Column2-Amount",... "Column16-ID"
" A, and B", " 100.00", " 000000001"
" CD", " 100.00", " 000000001"
" EF", " 100.00", " 000000001"

I'm unsure how to modify this to handle the introduction of commas within a field in the record (in the first record "A, and B"). I tried the following with no success:

Code:
awk -F FS="\",\"" 'NR==1 { hdr=$0; next } $16 != prev { prev=name=$16; gsub(/[^[:alnum:]_]/,"",name); $0 = hdr "\n" $0 } { print > ("/Directory/File."name".Download.csv") }' /Directory/File.ALL.Download.csv

All suggestions welcome, thanks!
# 2  
Old 07-30-2014
Try sth along this line:
Code:
awk     'NR>1   {FS=OFS="\""; $0=$0                                     # recalculate fields based on " delimiter
                 for (i=2; i<=NF; i+=2) gsub (/,/,"\001", $i)           # replace , in " delimited fields with a token
                 FS=OFS=","; $0=$0                                      # recalculate fields based on , delimiter

                 $2 = $1"abcd"                                          # do whatever you need

                 gsub ("\001", ",")                                     # just before printing anything, put back the inner ","
                }
         1
        ' file

# 3  
Old 07-30-2014
If you use awk, you've to write a lot of boiler-plate code. Try perl with the Text::CSV module. It will do this job neatly for you.

Code:
#! /usr/bin/perl

use warnings;
use strict;
use Text::CSV;

my ($csv, $fh, $row);
my (@fields);

$csv = Text::CSV->new ({ binary => 1, eol => $/ });

open $fh, "< file";

while ($row = $csv->getline($fh)) {
    @fields = @{ $row };
    print "@fields\n";
}

close $fh;

Code:
[user@host ~]$ cat file
a,b,"c,d,e",f,g
p,q,"r,s,t",u,v
[user@host ~]$ ./test.pl
a b c,d,e f g
p q r,s,t u v
[user@host ~]$

This User Gave Thanks to balajesuri For This Post:
# 4  
Old 07-30-2014
Quote:
Try perl with the Text::CSV module. It will do this job neatly for you.
Agreed, but unfortunately in this instance Perl is not available.

---------- Post updated at 12:30 PM ---------- Previous update was at 10:43 AM ----------

After some further review, I realized I don't need the FS= declaration. The following commond provides the desired result.

Code:
awk -F "\",\"" 'NR==1 { hdr=$0; next } $16 != prev { prev=name=$16; gsub(/[^[:alnum:]_]/,"",name); $0 = hdr "\n" $0 } { print > ("/Directory/File."name".Download.csv") }' /Directory/File.ALL.Download.csv

Thanks all for your input.

Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk "date" and "system" command

Hello experts! I need your help please I have a file.txt of which I want to extract 3rd and 4th columns with date with the form e.g.: 2016-11-25 03:14:50and pass them to "date" command, but also append the 9th column in a file as well. So I want to execute date -d '2016-11-25 03:14:50' ... (2 Replies)
Discussion started by: phaethon
2 Replies

2. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

3. Shell Programming and Scripting

Is it possible to use sed to handle the line contains BOTH "AA" and "BB"

if yes, can some expert give an example Lei (2 Replies)
Discussion started by: yanglei_fage
2 Replies

4. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

5. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

6. Shell Programming and Scripting

Modify or react on "cd" command

Hi folks, I found a pretty nice thread in this forum showing how to change the window title of a PuTTY window from within the shell. (Which I can't reference due having less than 5 postings?!?) I modified the solution a bit to have a function in my zshrc "wt()" changing the window title each... (8 Replies)
Discussion started by: mephistho
8 Replies

7. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

8. Shell Programming and Scripting

Modify print of command "ls -l -R"

Hi to all, Some help please, I want to modify using awk the print showed by command "ls -l -R" from: /home/user1/Mydocs/Year/2010/July/Experiments: total 1608 -rw-r--r-- 1 user1 user1 1431 Feb 2 22:45 Experiments1.TAR -rw-r--r-- 1 user1 user1 923 Feb 2 22:45 Tests_Exp1.TXT... (4 Replies)
Discussion started by: cgkmal
4 Replies

9. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies

10. UNIX for Advanced & Expert Users

Command similar to "touch" for modify File size

Hi All, I'm trying to find a command like similar to "touch" which would let me change the file size property. For ex: I have a file of size 1MB using the command i would like to set/update the size something like 1KB. Is it possible? Is there any such command which would accomplish this... (3 Replies)
Discussion started by: sriharshareddyk
3 Replies
Login or Register to Ask a Question