Shell Programming and Scripting

The problem with removing ALL non-keyboard characters is that the needed hidden characters will be deleted too.

The file is generated from mysql, then dumped to a file. The file will be sed to replace the comma with |. The | is the colseparator on the mysql.

I don't know much about chomp. I'll be researching on that.

---------- Post updated at 05:50 PM ---------- Previous update was at 05:33 PM ----------

This one worked. But it removed all the Hidden Characters.

I just want to remove the hidden character on 'InterUnit$' string and its result from sql query.

InterUnit column will give a Y or N result from the database.

---------- Post updated 06-24-11 at 02:40 PM ---------- Previous update was 06-23-11 at 05:50 PM ----------

Hi all, I am now facing a more complicated problem with hidden characters '$'.

I noticed that the '$' characters are printed in a particular pattern resulting for my CSV file to produce incorrect format upon the execution of the ksh on Linux.

Do you guys know how to remove unnecessary hidden characters on Linux in ksh?

There are unnecessary '$' characters on my output file, but not all '$'. As we know, '$' is like \n on hidden character when we used "set list" in vi.

If my question is unclear, please let me know how could I elaborate it better.

how you are taking dump (using which command) from mysql DB ?
what is your email command ? (which sends the file as attachment ) ?

I have a function to call the run_sql.ksh script using sqlplus.
Whenever I call the function, I dump the result on a $LOGFILE and also spool it on ${file}.csv
Now, the CSV file will be "sed" using the following commands:

Code:

#Remove commas seen on all results
 sed 's/,/ /g' ${file}.csv > $CSV_RESULT_PIPE
#Change pipes delimiter to comma
 sed 's/|/,/g' $CSV_RESULT_PIPE > $CSV_RESULT_UNZIPPED

The $CSV_RESULT_UNZIPPED file will be compressed to ${CSV_RESULT}
Then the following command:

Code:

uuencode ${CSV_RESULT} ${CSV_FILE_ZIPPED} > $EMAIL_ATTACHMENT

Then send the email using:

Code:

cat $EMAIL_HEADER | /usr/sbin/sendmail -t $RECIPIENTS

Note that the $EMAIL_HEADER includes the $EMAIL_ATTACHMENT, and I can receive the email successfully but the data in it are misplaced.

ok, in the inital csv file itself the $ value appears ?

${file}.csv # is the file having $value ?

Code:

 
#Remove commas seen on all results
sed 's/,/ /g' ${file}.csv > $CSV_RESULT_PIPE

Yes, it has. The value is the real file name to be generated. Please consider file=CSV_FILE:

Code:

 
export CSV_FILE=$PATH/MI_RESULT_OF_QUERY_DATETIME

The file is successfully generated and received but the content is not aligned properly.

Please describe what you see, mentioning what program you are using to view the file. If it is the "stairs" effect it will almost certainly be due to the line terminator.

This happens when you send a unix format text file to a MSDOS platform.
In unix the line terminator is just a linefeed character. This is conventionally represented as "$" or "\n" in unix.
In MSDOS the line terminator is two characters: carriage-return then linefeed.
You will probably need to convert the file to MSDOS format before using "uuencode".

We need to know what Operating System and version you have.
Where supplied, the program is variously called "ux2dos" or "unix2dos". Otherwise the conversion can be achieved with "awk". In your case you could presumably handle the whole conversion (including the CSV bit) in SQL.

deleted

wrong data