Request for advise on how to remove control characters in a UNIX file extracted from top command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Request for advise on how to remove control characters in a UNIX file extracted from top command
# 1  
Old 09-04-2013
Request for advise on how to remove control characters in a UNIX file extracted from top command

Hi,

Please excuse for posting new thread on control characters,
I am facing some difficulties in removing the control character from a file extracted from top command,
i am able to see control characters using more command and in vi mode, through cat control characters are not visible
through cat -v i am able to see the control characters.


i tried with sed , tr -dc, od command (not sure how to handle this) nothing worked and thought of reading line by line and removing control characters but not sure how to remove the control characters, Could anyone please advise how to remove the control characters.

below is my file in vi mode having control characters

Code:
^[[7m  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         ^[[m^O^[[K
^[[m^O12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.23 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O24435 aaaa_a    16   0  100m 3200 2624 S  0.0  0.0   0:00.19 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O25623 aaaa_a    15   0  100m 3192 2624 S  0.0  0.0   0:00.28 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O29634 aaaa_a    16   0  100m 3204 2632 S  0.0  0.0   0:00.35 xxx aaaa_axxxxxx                                                                               ^[[m^O


Thanks,
Regards,
karthikram
# 2  
Old 09-04-2013
Code:
 perl -pne 's/^[\[\d*m//g' file

Note the ^[ is a literal escape entered by hitting [CTRL]-V and then escape

On a side note top -b -n3 will run top 3 times with no control characters (mnemonic b for "batch" mode)

Last edited by Skrynesaver; 09-04-2013 at 05:00 AM.. Reason: added advice on the -b flag to top
# 3  
Old 09-04-2013
Hi Skrynesaver,

I am getting the below error , actually i dont know perl exactly. Could you please advise.

Code:
$perl -ne 's/^[\[\d*m//g' file_name.txt
Unmatched [ in regex; marked by <-- HERE in m/^[ <-- HERE \[\d*m/ at -e line 1.

# 4  
Old 09-04-2013
Did you try the tr option to remove control characters?

Code:
tr -d [:cntrl:] <filename >output

This User Gave Thanks to krishmaths For This Post:
# 5  
Old 09-04-2013
Code:
bash-3.2$ cat input.txt
^[[7m  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         ^[[m^O^[[K
^[[m^O12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.23 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O24435 aaaa_a    16   0  100m 3200 2624 S  0.0  0.0   0:00.19 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O25623 aaaa_a    15   0  100m 3192 2624 S  0.0  0.0   0:00.28 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O29634 aaaa_a    16   0  100m 3204 2632 S  0.0  0.0   0:00.35 xxx aaaa_axxxxxx                                                                               ^[[m^O

bash-3.2$ perl -pne 's/\^\[\[\d*m\^?O?//g; s/\^\[\[K//g' input.txt
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         
12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.23 xxx aaaa_axxxxxx                                                                               
24435 aaaa_a    16   0  100m 3200 2624 S  0.0  0.0   0:00.19 xxx aaaa_axxxxxx                                                                               
25623 aaaa_a    15   0  100m 3192 2624 S  0.0  0.0   0:00.28 xxx aaaa_axxxxxx                                                                               
29634 aaaa_a    16   0  100m 3204 2632 S  0.0  0.0   0:00.35 xxx aaaa_axxxxxx

This User Gave Thanks to MR.bean For This Post:
# 6  
Old 09-04-2013
Quote:
Originally Posted by karthikram
Hi Skrynesaver,

I am getting the below error , actually i dont know perl exactly. Could you please advise.

Code:
$perl -ne 's/^[\[\d*m//g' file_name.txt
Unmatched [ in regex; marked by <-- HERE in m/^[ <-- HERE \[\d*m/ at -e line 1.

See note on literal escape character in my original reply
This User Gave Thanks to Skrynesaver For This Post:
# 7  
Old 09-05-2013
Hi Skrynesaver/MR.Bean/krishmaths,

Thanks all of your input , Please excuse Skrynesaver i didnt see the note top -b d 3 command worked fine, through more , cat -v and vi i didnt see control characters,

Thanks i tried tr -d [:cntrl:] also it removes the caret symbol and character like [7m,[m[K[m,[m[m , i am able to see that i hope i can remove using sed command.

I tried
Code:
perl -pne 's/\^\[\[\d*m\^?O?//g; s/\^\[\[K//g' input.txt

but didnt work , actually i am using ksh.

i tried using
Code:
perl -pne '$_ =~ s/[^[:ascii:]]//g; print;' YourTextFile

-- command worked but still [* character was available ^ was removed.
and
Code:
perl -ne '$_ =~ s/\P{isPrint}//g; print;' YourTextFile


Thanks All for your valuable inputs.

Thanks,
Regards,
karthikram

---------- Post updated 09-05-13 at 02:15 AM ---------- Previous update was 09-04-13 at 10:09 PM ----------

Hi ,

I am not sure, whether i can request for another query,

Code:
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2961 aaaa_a    15   0  100m 3196 2624 S  0.0  0.0   0:01.10 ssh aaa_a@aaaaaa
12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.34 ssh aaa_a@aaaaaa

I need to take this output in html format i am unable to get $12 value which is the COMMAND value , if there is space it prints the start of the word and not printing the entire.

I am unable to find a way to
1. print the file in pipe separted so that i can send to html
2. i am unable to print the file in mail , where it is printing the whole file in single line.

I tried,
2. using while loop which array nothing is working to print line by line in the mail.

and the file is containing blank space after the 12th field COMMAND.

Please advise.

Thanks,
Regards,
karthikram

---------- Post updated at 02:56 AM ---------- Previous update was at 02:15 AM ----------

Hi,

1. To print the top command output file in mail , i tried the below , but writing the entire file in a single line.

Code:
IFS='\n'
i=1
while read -A line; do
echo "${line[${i}]}"
i=$((i + 1))
done < top_list.txt >> mail.txt

Code:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2961 aaaa_a 15 0 100m 3196 2624 S 0.0 0.0 0:01.10 ssh aaa_a@aaaaaa 12759 aaaa_a 16 0 100m 3188 2616 S 0.0 0.0 0:00.34 ssh aaa_a@aaaaaa


2. To record the top command output file in html format, the 12th field COMMAND value which has space so while appending to html more number of rows are creating instead of 12 field expected.

Code:
echo -e "<table border="0" cellpadding="0" cellspacing="0">" >> output_2.html
cat top_list.txt | head -1 > top_list.txt_temp
awk 'BEGIN{RS="\n";FS="\n";}
{
print "<TR>";
for (i = 1; i <= NF; i++)
{print "<TH width=\"100%\">" $i "</TH>";}
print "</TR>";
print "\n";
}' top_list.txt_temp >> output_2.html
echo -e "<tbody id="data">" >> output_2.html
rm -f top_list.txt_temp
more +2 top_list.txt > top_list.txt_temp
awk 'BEGIN{RS="\n";FS="\n";}
{
print "<TR>";
for (i = 1; i <= NF; i++)
{print "<TD width=\"100%\">" $i "</TD>";}
print "</TR>";
print "\n";
}' top_list.txt_temp >> output_2.html
echo -e "</tbody></table>" >> output_2.html
echo -e "<br>" >> output_2.html
echo -e "</body></html>" >> output_2.html

and 1 more thing , file is containing blank space after 12th field.

and Is there any approach to get the top command output in pipe separated file.

Please advise how i need to workaround.

Thanks,
Regards,
karthikram

---------- Post updated at 10:11 AM ---------- Previous update was at 02:56 AM ----------

Hi,

could anyone please advise, i was trying to get top command file into html where 12th field containing space , i am not sure to frame exact awk to command to accomadate the value inside 12th field instead of more fields after it.

Thanks,
Regards,
karthikram

Last edited by Franklin52; 09-05-2013 at 03:41 AM.. Reason: Please use code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Special control characters in file

Hi Guys, We receive some huge files on to Linux server. Source system use FTP mechanism to transfer these files on our server. Occasionally one record is getting corrupted while transfer, some control characters are injecting into the file. How to fix this issue ? please advice ? Sample... (2 Replies)
Discussion started by: srikanth38
2 Replies

2. Shell Programming and Scripting

How to view the control characters in a file?

Hello, How can I view control and special characters of a text file?. For example, space, tabs, new line chars etc. Can I use hexdump for it? Thanks (3 Replies)
Discussion started by: reddyr
3 Replies

3. UNIX for Dummies Questions & Answers

Control characters in UNIX

Hi, My files are showing some control characters in vi editor ^M ^@ and somtimes ^H I removed ^M with %s/^M//g command but how to represent ^@ and ^H e.g. for ^M it is hold ctrl then v and m.. Please help.. I am very new to unix.. (7 Replies)
Discussion started by: prabhat.diwaker
7 Replies

4. Shell Programming and Scripting

Extra control characters being added when I create a file using cat command

Hi, I am using Cygwin.I created a new file and type into it using cat > newfile. When I open this using vi editor, it contains loads of extra control characters. Whats happening? (1 Reply)
Discussion started by: erora
1 Replies

5. Shell Programming and Scripting

How to remove Ctlr-M characters from file from a unix script

There are 10 files present which have Ctlr-M characters appended to each line of all files. I have a unix script which processes the files in a loop. And there is an inner loop which processes each line in the file concerned. #inputFile is a variable which has the file name of the input... (2 Replies)
Discussion started by: akashtcs
2 Replies

6. Shell Programming and Scripting

display all possible control characters from .xml file in unix

Hi, I have a .xml file in unix. We are passing this file through a xml parser. But we are getting some control characters from input file and XML parser is failing for the control character in file.Now I am getting following error, Error at byte 243206625 of file filename_$.xml: Error... (1 Reply)
Discussion started by: fantushmayu
1 Replies

7. Shell Programming and Scripting

Hidden control characters in a Unix Text File!

Can anyone seem to know how to find out whether a UNIX text file has 'hidden' control characters? Can I view them using 'vi' by some command line options? If there are control characters in a text file which are invisible/hidden.. then how do I get rid of them? Your intelletual answers are... (6 Replies)
Discussion started by: kewl_guy
6 Replies

8. Shell Programming and Scripting

Which one is faster to remove control m characters?

I have a file with millions of records...Before I experiment, I would like to know which one is faster. Both the commands work absolutely fine on a smaller set of records. Please advice. sed 's/^M//g' ${INPUT_FILE} > tmp.txt mv tmp.txt ${INPUT_FILE} tr -d "\15" < ${INPUT_FILE} > ... (11 Replies)
Discussion started by: madhunk
11 Replies

9. Programming

Identifying and removing control characters in a file.

What is the best method to identify an remove control characters in a file. Would it be easier to do this in Unix or in C. (0 Replies)
Discussion started by: oracle8
0 Replies

10. UNIX for Dummies Questions & Answers

Remove control characters

Hi, When I do a man and save it into a file, I end up getting a lot of control characters. How can I remove them?? I tried this: /1,$ s/^H//g But I get an error saying "no previous regular expression". Can someone help me with this. Thanks, Aravind (5 Replies)
Discussion started by: aravind_mg
5 Replies
Login or Register to Ask a Question