Shell Programming and Scripting

Hi,

I have a text file which when I do a 'cat' on it looks like below with the OK’s showing in green and any FAIL showing in red.

cat filename output:

===== MySQL Query Check =====
DB mpuser is alive. 733 = Expected 733 Tables. [ OK ]
DB mpuser_wf is alive. 61 = Expected 61 Tables. [ OK ]
===== MySQL Uptime Check =====
UpTime is 33024426 [ OK ]
===== Ensure all interfaces and DB connections are OK =====
=====> Determining DBM-01 & DBM-02 state ...
The states of DBM-01 and DBM-02 are match [ OK ]
DBM-01 DBMHB Online [ OK ]
DBM-01 DBMHB Linking [ FAIL ]

I want to get this output emailed using something like below:

“cat ${TMP_MAIL_OUTPUT} | mail -s "${EMAIL_SUBJECT}" ${EMAIL_RECIPIENTS}”

Also when I try to do a less on the file, it says it "may be a binary file" and when I go in see the following ESC and ^M characters. These ESC and ^M characters are also coming into the email output when I just want the email to contain the same output as shown on the cat output

[test@test01 jk]$ less mp_16_menu_runner.sh.20120912.regular.sh.email.30858.out
"mp_16_menu_runner.sh.20120912.regular.sh.email.30858.out" may be a binary file. See it anyway?
[HESC[2J===== MySQL Query Check =====
ESC[70G[ ESC[1;32mOKESC[0;39m ]^MDB mpuser is alive. 733 = Expected 733 Tables.
ESC[70G[ ESC[1;32mOKESC[0;39m ]^MDB mpuser_wf is alive. 61 = Expected 61 Tables.
===== MySQL Uptime Check =====
ESC[70G[ ESC[1;32mOKESC[0;39m ]^MUpTime is 33024426
..
..
..
===== Ensure all interfaces and DB connections are OK =====
=====> Determining ABM-01 & QBM-02 state ...
ESC[70G[ ESC[1;32mOKESC[0;39m ]^MThe states of ABM-01 and QBM-02 are match


I’ve tried things like piping file through below and outputting to tmp file and checking if the ESC[70G[ and ^M go away but with no luck at all

dos2unix
tr –d ^M
tr –d ‘[unct:]’
strings
iconv -f unicode -t ascii mp_16_menu_runner.sh.20120912.regular.sh.email.30858.out
sed 's/\033/,/g' mp_16_menu_runner.sh.20120912.regular.sh.email.30858.out

See below which confirms this is a text file
[test@test01 jk]$ file mp_16_menu_runner.sh.20120912.regular.sh.email.30858.out
mp_16_menu_runner.sh.20120912.regular.sh.email.30858.out: ASCII English text, with CR, LF line terminators, with escape sequences

Any help on how to remove these ESC and ^M characters much appreciated

Many Thanks

Moderator's Comments:

Please view this code tag video for how to use code tags when posting code and data.

Further to @vbe, please always post what Operating System and version you are running, what database engine and version you are running, and what commands you (or the script) typed to produce the output presented.
This is an awkward one because the output presented is formatted for your terminal and is unsuitable for use as an email attachment.
One approach is to try to convert the output format. A better approach is to write code to output the results in a more useful format.

You can both remove and add with sed:, making carriage returns into cr/lf and removing escape sequences:

Code:

sed '
    s/^M/&\
/g
    s/^[[0-9;]*[a-zA-Z]//g
 ' infile >outfile

The ^M is typed 'ctrl-v ctrl-m', the ^[ is 'ctrl-v escape'.

For anyone else with same issue, the following worked for me and was able to get what i needed

Code:

cat file | sed -e 's/\x1b\[70G//g' -e 's/\x1b\[1;32m//g' -e 's/\x1b\[0;39m//g' -e 's/\x1b\[1;33m//g' -e 's/\x1b\[1;31m//g' > test2

Good solution for the ESC sequences, but you did not remove the <carriage return> (^M, 0x0D) chars, so mail should still complain...?

Yes that was bizzare, whilst i could see the ^M characters, when i converted email output to HMTL it didnt complain about the ^M's at all and email output was clean

In HTML, unless you are in <PRE> or <TT>, all adjacent white space characters are rendered as one space character. This means you con break long lines up and even indent continuations in the raw byte stream and yet have a tight presentation that wraps to fit the window/frame/column/cell.