Visit Our UNIX and Linux User Community


utility


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting utility
# 1  
Old 04-16-2009
utility

hi experts,

Can you please help me out in removing delimiters with in double quotes from a CSV file.

input:
=====
a,"bnn,",dgd, "sagfh,dj",ad

output
=====
a,"bnn",dgd, "sagfhdj",ad


there are so mnay fileds in a row and there are millions of rows.

Thanks in an advance.
-subhendu-
# 2  
Old 04-16-2009
if you have Perl, you can use Text::CSV module
Code:
#!/usr/bin/perl

use Text::CSV;
my $csv = Text::CSV->new({allow_loose_quotes=>1});
my $file = "file";
open (CSV, "<", $file) or die $!;
while (<CSV>) {
        if ($csv->parse($_)) {
            foreach my $col ($csv->fields()){
              print $col."\n";
            }
        } else {
            my $err = $csv->error_input;
            print "Failed to parse line: $err";
        }
    }
close(CSV);

output:
Code:
# ./test.pl
a
bnn,
dgd
 "sagfh
dj"
ad

# 3  
Old 04-16-2009
An approach with awk:

Code:
awk -F "\"" '{
  for(i=2;i<NF;i+=2) {
    gsub(",", "", $i)
  }
}1' OFS="\"" file

Regards
# 4  
Old 04-16-2009
Quote:
Originally Posted by Franklin52
An approach with awk:

Code:
awk -F "\"" '{
  for(i=2;i<NF;i+=2) {
    gsub(",", "", $i)
  }
}1' OFS="\"" file

Regards
the "," is gone
Code:
 # awk -F "\"" '{ for(i=2;i<NF;i+=2) {gsub(",", "", $i) }}1' OFS="\"" file
a,"bnn",dgd, "sagfhdj",ad

# 5  
Old 04-16-2009
Quote:
Originally Posted by ghostdog74
the "," is gone
Code:
 # awk -F "\"" '{ for(i=2;i<NF;i+=2) {gsub(",", "", $i) }}1' OFS="\"" file
a,"bnn",dgd, "sagfhdj",ad

That's what the OP expected, check the desired output in the first post.

Regard
# 6  
Old 04-16-2009
my bad, misinterpretation.
# 7  
Old 04-17-2009
perl code:

Code:
my $str='a,"bnn,",dgd, "sagfh,dj",ad,"aa,bb,cc",asdf,asdf,"sdaf,asdfsdf,asdff",asdf';
print $str,"\n";
$str=~s/,
(?=
  (
    (?:[^"]*"[^"]*$)
   |
    (?:[^"]*"[^"]*(?:"[^"]*"[^"]*)*[^"]*$)
   )
)/|/xg;
print $str;

output:
Code:
a,"bnn,",dgd, "sagfh,dj",ad,"aa,bb,cc",asdf,asdf,"sdaf,asdfsdf,asdff",asdf
a,"bnn|",dgd, "sagfh|dj",ad,"aa|bb|cc",asdf,asdf,"sdaf|asdfsdf|asdff",asdf


Previous Thread | Next Thread
Test Your Knowledge in Computers #433
Difficulty: Easy
JSON, or JavaScript Object Notation, is a general-purpose data interchange format that is defined as a subset of JavaScript's object literal syntax.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Dialog utility

Hi, I am new to shell scripting, started learning script before one month. I can't understand the below script. dialog --title "Inputbox - To take input from you" --backtitle "Linux Shell\ Script Tutorial" --inputbox "Enter your name please" 8 60 2>/tmp/input.$$ In this script they are... (1 Reply)
Discussion started by: raj1983
1 Replies

2. Shell Programming and Scripting

How to Unzip a file using unzip utility for files zipped without zip utility ?

Hi, I need to zip/compress a data file and send to a vendor. The vendor does have only unzip utility and can accept only .ZIP files. I do not have zip utility in my server. How do I zip/compress the file so that it can be deflated using unzip command ? I tried gzip & compress commands, but... (1 Reply)
Discussion started by: Sabari Nath S
1 Replies

3. Programming

make utility

Hi, Could someone tell me why make utility is mostly preferred for compilation and generating executable in UNIX rather than any user created scripts which contains commands for compilation? (2 Replies)
Discussion started by: naan
2 Replies

4. AIX

Help with nmon utility

We have processes that run on our AIX box that sometimes run away and end up consuming 99% of the CPU. I'd like to create a script that would attempt to monitor when this happens and send an email alert with the PID and CPU %. Has anyone done such a thing? I know that you can run the nmon output to... (6 Replies)
Discussion started by: ssmith001
6 Replies

5. UNIX for Dummies Questions & Answers

Shred utility

Hi experts. We have a HP-Unix server, we deleted some files and folders from the server. Is there any utilities that we can use to ensure those deleted files and folders are not recoverable? We used rm command to delete these files. Cheers, RT:confused: (1 Reply)
Discussion started by: robtseng
1 Replies

6. HP-UX

kcalarm utility

Hi Does anybody know is there is any troubleshooting for kcalarm? How can i see what parameter exceded it's range while running command fr monitoring "any parameter"? Thanks :o (0 Replies)
Discussion started by: m_mouse
0 Replies

7. Solaris

Make utility

Hi Guys, I m very confused about the make/makefile utility in all unix. 1) My questions is why we need make. 2) Why some source code needs to complile. 3) I download the Bind 9 from Sunfreeware.com. I use pkgadd -d to install the bind. I 'm struck here becasue I can't find /etc/named.conf... (2 Replies)
Discussion started by: bigmoon
2 Replies

8. Programming

MAKE utility

I wrote a makefile, every thing is working fine, But One of the C header files which is created by me is kept in a different folder other than the current directory, I have given this PATH to VPATH Variable Example :- VPATH = /home/user1/projects/victor.h It gives an error as : file... (4 Replies)
Discussion started by: victorvvk
4 Replies

9. UNIX for Dummies Questions & Answers

zip utility on HP-UX

where can i find an zip utility similar than windows zip for HP-UX? Jesus (1 Reply)
Discussion started by: jgutierrez29
1 Replies

10. UNIX for Dummies Questions & Answers

What utility do I use for this?

I want to pull out the 3rd column of information and stick in a file. What is the Utility I use to do this? (8 Replies)
Discussion started by: James
8 Replies

Featured Tech Videos