Help Replacing Characters in Flat File


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help Replacing Characters in Flat File
# 1  
Old 05-17-2007
Help Replacing Characters in Flat File

I was wondering if somebody could help me with something on UNIX. I have a file that looks like this -

"nelson,bill","bill","123 Main St","Mpls","MN",55444,8877,william

I want to replace all comma with pipes (|), except if the comma is within double quotes. (The first field is an example of this.) I can't do do a sed looking for "," and replacing that with a pipe because not all fields have double quotes around them. Side note - I do not have access to Perl, so that's not an option.
# 2  
Old 05-17-2007
Nelson,
See if this works for you:
Code:
sed -e 's/",/"|/g' -e 's/\([0-9]\),/\1|/g' input_file

# 3  
Old 05-17-2007
Shell,
won't work for this pattern:
Code:
"nelson,bill","bill","123 Main St","Mpls","MN",55444,8877,william,foo

# 4  
Old 05-17-2007
Vgersh,
I agree with you, but the sample is not precise as it has two strings
treated in two different ways:
Quote:
"bill"
william
I tried to solve the problem based on the sample data.
Thank you for analizing it.
# 5  
Old 05-17-2007
Thanks...But what if

That worked for the sample that I gave you, but I thought of another scenario that I need to account for. What would happen if I added another comma to the end and then some more text? Sample -

"nelson,bill,jr","bill","123 Main St","Mpls","MN",55444,8877,william,bill

I tried modifying your sed command and couldn't figure out how to make it work.
# 6  
Old 05-17-2007
echo '"nelson,bill,jr","bill","123 Main St","Mpls","MN",55444,8877,william,bill' | nawk -f doCSV.awk

doCSV.awk:
Code:
BEGIN { FS=SUBSEP; OFS="|" }

{
  result = setcsv($0, ",")
  print
}

function setcsv(str, sep, i) {
  gsub(/""/, "\035", str)
  gsub(sep, FS, str)

  while (match(str, /"[^"]*"/)) {
    middle = substr(str, RSTART+1, RLENGTH-2)
    gsub(FS, sep, middle)
    str = sprintf("%.*s%s%s", RSTART-1, str, middle,
      substr(str, RSTART+RLENGTH))
  }

  if (index(str, "\"")) {
    return ((getline) > 0) ? setcsv(str (RT != "" ? RT : RS) $0,
sep) : !setcsv(str "\"", sep)
  } else {
    gsub(/\035/, "\"", str)
    $0 = str

    for (i = 1; i <= NF; i++)
      if (match($i, /^"+$/))
        $i = substr($i, 2)

    $1 = $1 ""
    return 1
  }
}

# 7  
Old 05-17-2007
Nelson,
Per my previous reply, your sample led me to believe the following:
1) Every non-numeric field would be surrounded by double quotes.
except for "william".
2) Every numeric field would not be surrounded by double quotes.
If the specs give reason to question, no final solution will be found.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Replacing characters in a file

Suppose I have a file which has 1000 columns (5 SHOWN FOR EXAMPLE) two alphabets are separated by a space and then tab A A"\t"C C"\t"G G"\t"0 0"\t"T T A G"\t"C C"\t"G G"\t"A T"\t"0 0 G A"\t"0 0"\t"G C"\t"A A"\t"T C whenever there is a 0 0 in any column, the output should be printed as A... (12 Replies)
Discussion started by: rossi
12 Replies

2. Shell Programming and Scripting

Replacing multiple spaces in flat file

Greetings all I have a delimited text file (the delimiter is ';') where certain fields consist of many blanks e.g. ; ; and ; ; Before I separate the data I need to eliminate these blanks altogether. I tried the sed command using the following syntax: sed -i 's/; *;/;;/g' <filename> ... (15 Replies)
Discussion started by: S. BASU
15 Replies

3. Shell Programming and Scripting

Finding distinct characters from flat file

Hi....I need one help.... I'm having a files which is having the data as follows... a b c c d d d e f Now I need to find out distinct characters from this file and the output should be as follows - a b c d e f Can you please help me on this? I'm using KSH script. (18 Replies)
Discussion started by: Krishanu Saha
18 Replies

4. UNIX for Dummies Questions & Answers

How to remove numeric characters in the flat file

HI, can any one help me please .. i have flat file like qwer123rt ass3242ccf jjk654 kjh838ppp nhdg453ok hdkk34 i want remove numeric characters in the flat file i want output like this qwerrt assccf jjk kjhppp nhdgok hdkk help me... (4 Replies)
Discussion started by: rafimd1985
4 Replies

5. AIX

How to cut a flat file according to a certain number of characters?

hello everybody i am looking for a shell to cut a flat file (with a long unique line) according to a certain number of characters and redirect every result to an output file. here is an example MyFile : 12 3 456 12 3 456 12 3 456 ..... and i took every 9-characters including BLANKS... (6 Replies)
Discussion started by: fastlane3000
6 Replies

6. Shell Programming and Scripting

Merge lines in Flat file based on first 5 characters

Hi I have the fixed width flat file having the following data 12345aaaaaaaaaabbbbbbbbbb 12365sssssssssscccccccccc 12365sssss 12367ddddddddddvvvvvvvvvv 12367 vvvvv Here the first column is length 5 second is length 10 third is length 10 if the second or third column exceeds... (3 Replies)
Discussion started by: Brado
3 Replies

7. UNIX for Advanced & Expert Users

foreign characters in flat file

Hey, Is there anyway I anks, Pocha (12 Replies)
Discussion started by: pochaman
12 Replies

8. UNIX for Dummies Questions & Answers

Replacing characters in csv file

Hello all, This is my first post here, so please excuse me if this question is too obvious or has been asked before. I am new to Unix and although I tried to search your forum for the answer to my question, I could not find an answer that would help me. I have a 500MB csv file with numeric values... (1 Reply)
Discussion started by: finwhiz
1 Replies

9. UNIX for Dummies Questions & Answers

replacing the characters in a file

hi i want to replace the characters between positions 2 to 30 in each line in a file how to do it suggestions welcome (2 Replies)
Discussion started by: trichyselva
2 Replies

10. UNIX for Dummies Questions & Answers

replacing few characters in a file

Hi All, I have huge xml file. The file contains some comment tags . I have requirement to replace comment tag with another comment tag. Say for example : file X has -- Part of the file <?xml version="1.0" encoding="ISO-2"?><translationResults jobDate="20070123 23:20:51"... (1 Reply)
Discussion started by: purnakarthik
1 Replies
Login or Register to Ask a Question