Shell Programming and Scripting

Thanks bakunin and reborg for your ideas.

Bakunin's script captures the idea I originally intended, but it fails it work, I don't know why. Specifically, this line does not work properly:

Code:

print - "$line" | read word, line

I don't know if this is because of difference in systems (I am using Fedora Core 3), but I don't think this should be an issue, since we are using ksh, which is common among all Linux distributions and UNIX flavors.

On command shell I tried to run the following command, but it would not work as well:

Code:

echo 2224 test | read word line

when I echoed $word and $line they would be empty variables. Therefore, the script bakunin wrote would result in an infinite loop, although $line is read properly by the first 'read line' statement, piping doesn't seem to work. Furthermore, if it had worked I think the output would have been:

2224
test

rather than just:

2224

this is because $line becomes 'test' after the first run on the while loop, which would pass the inner loop condition again, causing $word this time to become 'test' and printing it. This is of course a simple programming problem, and can be fixed easily:

Code:

tempVar = ""
while [ -z "$tempVar" ] ; do
print - "$line" | read word, tempVar
 line = "" 
 print "$word" 
done

Any suggestions regarding the piping problem?

Sorry, I had a typo in my solution. You have already found the offending line, the comma is the error. It should read:

Code:

print - "$line" | read word line

and, when you call the whole script "tokenize.ksh", you could use it like:

Code:

cat anyfile | tokenize.ksh | some_script_using_tokens.ksh

If you meant by "tokenizing" that you want only the first word of each line going to the output (sorry, but that wasn't clear from the beginning, at least not for me), replace the while...done-loop by a sinqle

Code:

print - $line | read word junk
print - "$word"

The way the "read"-statement works is: put the first word encountered to the first variable, the second word to the second variable, etc. If you run out of variables put the rest of the line to the last variable. This means "read line" will read the whole line, regardless of how many word are in it, "read word line" will put the first word to $word, everything else to $line.

Since you mentioned that "echo 111 222 | read word line" doesn't work either I suppose your Kornshell implementation is as faulty as was my script. On my system (various AIX 5.x versions) it works perfectly.

bakunin