Shell Programming and Scripting

Hi ,

I Have an output from the a log file which I need to convert to a pipe delimiter, but the problem is , I am not able to find a proper delimiter b/w the words .

My log file

Code:

1. confirm the status     07:03:05 (U=AP2521)       ASBCS_39833.log           1268D7980: 891.118
2. Query status (ExecuteQuery)     08:39:28 (U=AP2521)      CNMSHK_39833.log          1268D7980: 891.118
3. confirm the status     07:22:03 (U=AP2521)       ASBCS_39833.log           1268D7980: 891.118

Expected output

Code:

1.|confirm the status|07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118
2.|Query status (ExecuteQuery)|08:39:28|(U=AP2521)|CNMSHK_39833.log|1268D7980:|891.118
3.|confirm the status|07:22:03|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

Since there is no proper delimiter , I tried to suppress the spaces and then change to pipe , but tr -s didn't work

Code:

tr -s '[:space:]' '|' < log.txt

Please need some help here

Thanks
Rashmi

Your tr needed to be two trcommands. I'm assuming that the white space is all space characters. If they are in fact tabs, that might be useful to know.
Assuming that the message text field always contains three space-separated "words", then we can build on your tr idea and then correct the message text field.

Code:

cat log.txt | tr -s ' ' | tr ' ' '|' | while read line
do
        part1=`echo "${line}"|cut -d\| -f1`
        part2=`echo "${line}"|cut -d\| -f2-4|tr '|' ' '`
        part3=`echo "${line}"|cut -d\| -f5-`
        #
        echo "${part1}|${part2}|${part3}"
done

./scriptname
1.|confirm the status|07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118
2.|Query status (ExecuteQuery)|08:39:28|(U=AP2521)|CNMSHK_39833.log|1268D7980:|891.118
3.|confirm the status|07:22:03|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

A sed suggestion:

Code:

sed 's/  */|/; h; s/.*  *\(..:..:...*\)/\1/; s/  */|/g; x; s/  *..:..:...*//; G; s/\n/|/' file

I am feeling lazy at the moment, so I assumed that there would only be one timestamp-ish sequence per line, xx:xx:xx where x is any character. If that's not a valid assumption (even if it is at the moment, it's best to be stricter to protect against the future), it's a very good idea to tighten ..:..:.. by explicitly matching digits.

Regards,
Alister

Thanks a lot Methyl and Alister :-) .

Alister , I am not that well versed with Sed , can you please explain me the code

... or using awk:

Code:

awk '$1=$1' OFS=\| log.txt

Note that there are spaces in the desired output's second field which your code will convert to pipes.

Regards,
Alister

---------- Post updated at 10:48 AM ---------- Previous update was at 09:32 AM ----------

If you don't understand sed at all, you should first read your manual page, because, while the following explanation doesn't assume much, it probably won't make a whole lot of sense if you don't understand basic sed commands such as s, G, and x.

I hope that you'll find the following explanation helpful.

sed has two "spaces" with which you can work, the pattern space and the hold space. Think of them as two memory locations, or variables, which are available to you. While there are commands to copy and append the contents of one space into the other, all text manipulation (substitutions and deletions) operate on the pattern space.

s/ */|/; h; s/.* *\(..:..:...*\)/\1/; s/ */|/g; x; s/ *..:..:...*//; G; s/\n/|/
That is a sed script consisting of 8 semicolon-delimited commands. Let's break it down, looking at each one in turn, to see how they manipulate the contents of the pattern and hold spaces to achieve our goal.

We begin our analysis at a point in time just after sed has read the first line of data from your original post. Bold red highlighting indicates something that changed.

===============================================

0.

Initial State

pattern space:
1. confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

hold space:
<EMPTY>

===============================================

1.

Substitute the first series of spaces with a pipe.
s/ */|/

pattern space:
1.|confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

hold space:
<EMPTY>

===============================================

2.

Copy the line as it now exists into the hold space (think of it as a location where we can put some text for later use, like a shell variable)
h

pattern space:
1.|confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

hold space:
1.|confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

===============================================

3.

Delete everything before the timestamp
s/.* *\(..:..:...*\)/\1/

pattern space:
<DELETED TEXT>07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

hold space:
1.|confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

===============================================

4.

Substitute all series of spaces with a pipe
s/ */|/g

pattern space:
07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

hold space:
1.|confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

===============================================

5.

exchange the pattern and hold space.
x

pattern space:
1.|confirm the status 07:03:05 (U=AP2521) ASBCS_39833.log 1268D7980: 891.118

hold space:
07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

===============================================

6.

Delete everything after the second field
s/ *..:..:...*//

pattern space:
1.|confirm the status<DELETED TEXT>

hold space:
07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

===============================================

7.

Append to the pattern space a newline followed by the contents of the hold space.
G

pattern space:
1.|confirm the status
07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

hold space:
07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

===============================================

8.

Substitute the first (in this case, the only) newline with a pipe.
s/\n/|/

pattern space:
1.|confirm the status|07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

hold space:
07:03:05|(U=AP2521)|ASBCS_39833.log|1268D7980:|891.118

===============================================

The pattern space now contains the desired output. Since the end of the script has been reached, sed will print the current contents of the pattern space before copying the next line of input into the pattern space.

Regards,
Alister