Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

remove commas if touching any letters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting remove commas if touching any letters
# 1  
Old 03-20-2012
remove commas if touching any letters
I have a csv file that I am trying to edit. I need to remove any comma that is touching a letter. I used this command to find them.

Code:
grep [a-z], moz_places_good.csv | head -n 10 | grep [a-z],

I would think I could use some form of this to just delete the commas and not the letter before it. Just not sure what kind of case statements I need.

Code:
sed 's/[a-z],//g' moz_places_good.csv  > moz_places_good1.csv

Here are a few examples.

Code:
102660,"http://tinypic.com/images/404.gif","404.gif (GIF Image, 160x120 pixels)","moc.cipynit.",2,0,0,,20,1314761594939440,"BLHwEA5KNQrg"
104636,"http://en.wikipedia.org/wiki/Comparison_of_archive_formats","Comparison of archive formats - Wikipedia, the free encyclopedia","gro.aidepikiw.ne.",1,0,0,74,20,1299406155891077,"KpXR1unNI5-h"
104638,"http://en.wikipedia.org/wiki/List_of_archive_formats","List of archive formats - Wikipedia, the free encyclopedia","gro.aidepikiw.ne.",1,0,0,74,20,1299406111608099,"sDF5BkctvTeI"
104799,"http://en.wikipedia.org/wiki/EverBank_Field","EverBank Field - Wikipedia, the free encyclopedia","gro.aidepikiw.ne.",2,0,0,74,20,1309321478508918,"yxJjCMpKsXhr"

Code:
102660,"http://tinypic.com/images/404.gif","404.gif (GIF Image  160x120  pixels)","moc.cipynit.",2,0,0,,20,1314761594939440,"BLHwEA5KNQrg"
104636,"http://en.wikipedia.org/wiki/Comparison_of_archive_formats","Comparison  of archive formats - Wikipedia the free  encyclopedia","gro.aidepikiw.ne.",1,0,0,74,20,1299406155891077,"KpXR1unNI5-h"
104638,"http://en.wikipedia.org/wiki/List_of_archive_formats","List of  archive formats - Wikipedia the free  encyclopedia","gro.aidepikiw.ne.",1,0,0,74,20,1299406111608099,"sDF5BkctvTeI"
104799,"http://en.wikipedia.org/wiki/EverBank_Field","EverBank Field -  Wikipedia the free  encyclopedia","gro.aidepikiw.ne.",2,0,0,74,20,1309321478508918,"yxJjCMpKsXhr"

# 2  
Old 03-20-2012
How about, any comma that doesn't have a quote on the right side of it?

Code:
# Replace all ' we want to keep with `
# Delete all remaining '
# Turn ` back into ,"
sed 's/,"/`/g;s/,//g;s/`/,"/g'

[edit] No, that's not going to work. Hmm. This is why putting commas in CSV text is a bad idea, using them ends up being so ad-hoc...
# 3  
Old 03-20-2012
Since it's the quotes that determine whether a comma needs to be deleted, check on those.

When using " as the field separator, the first field will always be outside quotes, the second field always inside, the third field always outside, etc. So remove commas only from every second field. Then use " as the output separator too to prevent " being stripped...
Code:
 awk -F'"' -v OFS='"' '{ for(N=2; N<=NF; N+=2) gsub(/,/, "", $N); } 1' filename > outfile

# 4  
Old 03-21-2012
Quote:
Originally Posted by Corona688
How about, any comma that doesn't have a quote on the right side of it?
It might be possible, considering that remove those comma's which has a space next to them. However the OP has to confirm that this assumption is good to go.
Code:
sed 's/, / /g' inputfile

# 5  
Old 03-21-2012
Quote:
Originally Posted by Corona688
Since it's the quotes that determine whether a comma needs to be deleted, check on those.

When using " as the field separator, the first field will always be outside quotes, the second field always inside, the third field always outside, etc. So remove commas only from every second field. Then use " as the output separator too to prevent " being stripped...
Code:
 awk -F'"' -v OFS='"' '{ for(N=2; N<=NF; N+=2) gsub(/,/, "", $N); } 1' filename > outfile

This way works Smilie.

Quote:
Originally Posted by michaelrozar17
It might be possible, considering that remove those comma's which has a space next to them. However the OP has to confirm that this assumption is good to go.
Code:
sed 's/, / /g' inputfile

This won't work. You didn't anchor the comma to letters. I have a million commas in my file. I only wanna remove the ones touching letters.
# 6  
Old 03-21-2012
Ok. Try if the below helps..
Code:
sed 's/\([a-zA-Z]\), /\1 /g' inputfile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove unwanted commas from a .csv file?

how to remove unwanted commas from a .csv file Input file format "Server1","server-PRI-Windows","PRI-VC01","Microsoft Windows Server 2012, (64-bit)","Powered On","1,696.12","server-GEN-SFCHT2-VMS-R013,server-GEN-SFCHT2-VMS-R031,server-GEN-SFCHT2-VMS-R023"... (5 Replies)
Discussion started by: ranjancom2000
5 Replies

2. Shell Programming and Scripting

Remove rows containing commas with awk

Hello everyone, I have a dataset that looks something like: 1 3 2 2 3 4,5 4 3:9 5 5,9 6 5:6 I need to remove the rows that contain a comma in the second column and I'm not sure how to go about this. Here is an attempt. awk 'BEGIN {FS=" "} { if ($2!==,) print }'Any help is appreciated. (5 Replies)
Discussion started by: Rabu
5 Replies

3. Shell Programming and Scripting

Remove quotes and commas from field

In the attached file I am trying to remove all the "" and , (quotes and commas) from $2 and $3 and the "" (quotes) from $4. I tried the below as a start: awk -F"|" '{gsub(/\,/,X,$2)} 1' OFS="\t" enhancer.txt > comma.txt Thank you :). (6 Replies)
Discussion started by: cmccabe
6 Replies

4. Shell Programming and Scripting

Shell script that should remove unnecessary commas between double quotes in CSV file

i have data as below 123,"paul phiri",paul@yahoo.com,"po.box 23, BT","Eco Bank,Blantyre,Malawi" i need an output to be 123,"paul phiri",paul@yahoo.com,"po.box 23 BT","Eco Bank Blantyre Malawi" (5 Replies)
Discussion started by: mathias23
5 Replies

5. Shell Programming and Scripting

Remove leading commas in the file

Hi , I have a file with below records 965382,10310858,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 767010,10217614,3,10217616,10217622,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,... (4 Replies)
Discussion started by: vputtas@gmail.c
4 Replies

6. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:... (4 Replies)
Discussion started by: krishnix
4 Replies

7. Shell Programming and Scripting

shell script to remove extra commas from CSV outp file

Name,,,,,,,,,,,,,,,,,,,,Domain,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Contact,Phone,Email,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Location -----------------------,------------------------------------------------,-------,-----,---------------------------------,------------------------------------ ----... (1 Reply)
Discussion started by: sreenath1037
1 Replies

8. Shell Programming and Scripting

Remove duplicate commas after exporting excel file to csv

Hello everyone I'm new here and this is my first post so first of all I want to say that this is a great forum and I have managed to found most of my answers in these forums : ) So with that I ask you my first question: I have an excel file which I saved as a csv. However the excel file... (3 Replies)
Discussion started by: Spunkerspawn
3 Replies

9. UNIX for Dummies Questions & Answers

Inserting commas and replacing backslashes with commas

Hi, Newbie here. I have a file that consists of data that I want to convert to a csv file. For example: Jul 20 2008 1111 / visit home / BlackBerry8830/4.2.2 Profile/MIDP-2.0 Configuration/CLOC-1.1 VendorID/105 Jul 21 2008 22222 / add friend / BlackBerry8830/4.2.2 Profile/MIDP-2.0... (3 Replies)
Discussion started by: kangaroo
3 Replies

10. Linux

How do i remove commas(,) & spaces

Hey guys, I am very much new to shell scripts. So you ppl may feel that i am asking stupid question here. :D 1. I am using command line argument as an input variable. The user gets this value in his mail from client which has commas n spaces (Eg. 12,34,56,789) and the scripts... (5 Replies)
Discussion started by: anushree.a
5 Replies
Login or Register to Ask a Question