Shell Programming and Scripting

Hello,

I am writing a script to remove multiples pipes (|) from a file. It needs to look like this data|data|data
instead of data|||data||data||||data

I have found the correct sed command to do this, but I need to do it in a loop and a for loop has not worked for me. It needs to be done on multiples files and then write to a new file. I'm running Solaris 8. I believe I need to use a while loop, but am unsure how do do this.
Here is what I have thus far:

#!/bin/sh
file="/file/directory/bin/temp/*.fixed"

#Removes extra spaces and Pipes in the file

for filename in *.fixed
do
sed 's/ * //g;s/||/|/g;s/|||/|/g' $file > newFile.fixed
done

Code:

for f in *.fixed;do sed 's/||*/|/g' "$f">new_"$f";done

or just:

Code:

awk '{gsub(/\|\|*/,"|");print>("new_"FILENAME)}' *.fixed

P.S. Use nawk or /usr/xpg4/bin/awk on Solaris.

more or less. The following will catch not only double and triple occurences of pipe-chars, but any number exceeding 1 (only the sed-code):

s/|[|]*/|/g

This replaces 1 or more occcurence of a pipe-char with exactly 1. The "[...]" means "any of the characters inside the brackets" and "*" means "zero or more of the last-mentioned expression", which is the brackets in this case. "[abc]*" would match any number of the characters a, b and c, i.e. "cbaabbc" or "a", but not "d".

About the multiple files issue, yes, you will have to use a loop. You can use the for-loop you have mentioned, but there will be a problem it the directory contains a very big number of files. "*" expands to a list of all filenames and this could get bigger than the maximum command line length in UNIX (for POSIX systems this is IIRC 4k, see /usr/include/sys/limits.h). You can overcome this by either using a temporary file:

Code:

ls -1 /files/to/change > /tmp/file
cat /tmp/file | while read file ; do
   sed ..... ${file} > ${file}.changed
   mv ${file}.changed ${file}
done

or you could use a pipe:

Code:

ls -1 /files/to/change | while read file ; do
   sed ..... ${file} > ${file}.changed
   mv ${file}.changed ${file}
done

bakunin

Thank you bakunin and radoulov.

bakunin
The code s/|[|]*/|/g worked. Thanks a lot.