Unix/Linux Go Back    


Gentoo Gentoo Linux is a versatile and fast, completely free Linux distribution geared towards developers and network professionals.

cpu%/mem% usage, scripting, dzen2: howto learn bash the hard way

Gentoo


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 08-02-2008
broli's Unix or Linux Image
broli broli is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 23 July 2014, 5:32 PM EDT
Location: Argentina
Posts: 219
Thanks: 0
Thanked 0 Times in 0 Posts
cpu%/mem% usage, scripting, dzen2: howto learn bash the hard way

I am trying to write a small (and rather simple) script to gather some info about the system and piping it to dzen2

first, i want to explain some things.
I know i could have used conky, but my intention was to expand my knowledge of bash, pipes and redirections inside a script, and to have fun (which i am)
some time later, i also decided to try to be more strict on the resource usage. this is a script that will constantly run on a low-end laptop.

also, dzen is not the important part of the script and can be simple ignored, is just a way to show the info. if you are interested in dzen, you can read more info here gotmor - dzen

so, lets talk about the script itself

i use top in batch mode to get the list of most cpu/mem using, but now, i would like to include the total too. (in percentage)

with the memory, i was thinking maybe using free, and doing some math on the used/cached fields. but that would involve a new app to run, when top already gave me the info.
but taking the info from top's output will require a heavy use of pipes and secondary apps like tail and head

with the cpu%, well, im lost .....
where could i find that info?
im thinking, maybe is better to just use the iddle% field of top

this is what i have done so far
general pastebin - broli - post number 1091305

any suggestion on how to improve what i have done will also be welcomed
Sponsored Links
    #2  
Old Unix and Linux 08-03-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
 
Join Date: Mar 2008
Last Activity: 28 March 2011, 6:41 AM EDT
Location: /there/is/only/bin/sh
Posts: 3,653
Thanks: 0
Thanked 11 Times in 9 Posts
For a small piece of code like that, I'm taking the liberty to quote

Quote:
Code:
#!/bin/bash

while [ 1 -eq 1 ]

The idiomatic way to code an endless loop is simply while true although you also see the obscure while : which avoids an external process (even though true is often a shell built-in in modern shells).

Quote:
Code:
do
	top -b -n 1 > /tmp/salidatop
	
	#primero cpu
	echo -n "^fg(green)CPU **"
	tail -n +8 /tmp/salidatop | sort -r -n -k9 | head -n 5 | while read line
	do 
		echo "$line" | awk ' { print "[^fg(cyan)",$12,"(^fg(red)",$9,"^fg(green)]--" } ' | tr -d ' ' | tr -d '\n'	 
	done

Is there a reason to feed awk a line at a time? Why not just


Code:
tail -n +8 /tmp/salidatop | sort -r -n -k9 | head -n 5 |
awk ' { printf "[^fg(cyan)",$12,"(^fg(red)",$9,"^fg(green)]--" } '

I don't see any spaces in the output, so tr -d ' ' seems superfluous. By using printf instead of print, there will be no trailing newline. As an aside, you could have combined the two tr:s to one: tr -d ' \n'

The same change could be applied below:

Quote:
Code:
	echo ">>"
	echo -n "^fg(green)MEM **"
	#segundo mem
	tail -n +8 /tmp/salidatop | sort -r -n -k10 | head -n 5 | while read line
	do 
		echo "$line" | awk ' { print "[^fg(cyan)",$12,"(^fg(red)",$10,"^fg(green)]--" } ' | tr -d ' ' | tr -d '\n'	 
	done

	echo ">>"
	
	sleep 5

done | dzen2 -ta l -u -l 1 -x 20 -y 710 -w 660 -e 'onstart=lower,uncollapse'

I don't understand the question about CPU percentage. You can add the user, system, and nice percentages, or just subtract the idle percentage from 100% as you note.

If you can replace the complex tail | head etc with a simple sed or awk script, that will probably help reduce the resource requirements. Perhaps you could pass some option to top to order the output like you want it, so you can avoid the separate sort -- that's probably the main bottleneck here (albeit a very minor one, with so little input).
Sponsored Links
    #3  
Old Unix and Linux 08-03-2008
broli's Unix or Linux Image
broli broli is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 23 July 2014, 5:32 PM EDT
Location: Argentina
Posts: 219
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by era View Post
The idiomatic way to code an endless loop is simply while true although you also see the obscure while : which avoids an external process (even though true is often a shell built-in in modern shells).
great, this sort of things is what i was expecting when i posted here Linux
i didn't used true, because i had the idea that it doesn't work (i might have got that form an old aix/tru64 at work)
i will google for more info on ":"

Quote:
Originally Posted by era View Post
Is there a reason to feed awk a line at a time? Why not just


Code:
tail -n +8 /tmp/salidatop | sort -r -n -k9 | head -n 5 |
awk ' { printf "[^fg(cyan)",$12,"(^fg(red)",$9,"^fg(green)]--" } '

well, the main reason is cause i didnt tough of it Linux
specially because of the approach i had with the while
Quote:
Originally Posted by era View Post
I don't see any spaces in the output, so tr -d ' ' seems superfluous.
well, i had spaces on my tests, dont know when they came from. i will check again
Quote:
Originally Posted by era View Post
By using printf instead of print, there will be no trailing newline.
great !! i didnt knew this. this is really good
Quote:
Originally Posted by era View Post
As an aside, you could have combined the two tr:s to one: tr -d ' \n'
didnt knew you could use tr like that

Quote:
Originally Posted by era View Post
I don't understand the question about CPU percentage. You can add the user, system, and nice percentages, or just subtract the idle percentage from 100% as you note.
my main problem is how to get it, or where to get that info.

Quote:
Originally Posted by era View Post
If you can replace the complex tail | head etc with a simple sed or awk script, that will probably help reduce the resource requirements. Perhaps you could pass some option to top to order the output like you want it, so you can avoid the separate sort -- that's probably the main bottleneck here (albeit a very minor one, with so little input).
i have the vague idea that i could use awk to avoid head | tail | cut , but i dont know how

could you give me some directions ?
specially to a good example
    #4  
Old Unix and Linux 08-04-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
 
Join Date: Mar 2008
Last Activity: 28 March 2011, 6:41 AM EDT
Location: /there/is/only/bin/sh
Posts: 3,653
Thanks: 0
Thanked 11 Times in 9 Posts
Obiously, awk is a fairly complete programming language, so you can rewrite simple text-processing utilities in awk, many of them easily.


Code:
awk 'NR==10 { exit 0 }1'  # head -10
awk '{ print $2 }'   # cut -f2 (splitting on runs of whitespace though)

tail is harder to emulate generically because the array type is rather crude. Maybe switch to sed (or Perl!) for that.

See also http://www.pement.org/awk/awk1line.txt

The first few lines of top output indicate the CPU usage. Read the manual page to learn what the fields mean.


Code:
Cpu(s): 16.8%us,  1.7%sy,  0.0%ni, 80.5%id,  0.8%wa,  0.1%hi,  0.1%si,  0.0%st

Sponsored Links
    #5  
Old Unix and Linux 08-04-2008
broli's Unix or Linux Image
broli broli is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 23 July 2014, 5:32 PM EDT
Location: Argentina
Posts: 219
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by era View Post
Obiously, awk is a fairly complete programming language, so you can rewrite simple text-processing utilities in awk, many of them easily.



Code:
awk 'NR==10 { exit 0 }1'  # head -10
awk '{ print $2 }'   # cut -f2 (splitting on runs of whitespace though)

so, if i read your post correctly, with awk i can simulate sime functions of tail or head...
what about sort?
i must admint my knowledge of awk is fairly simple. i ave seen more complex ones, and im reading this website AWK Language Programming - Table of Contents
readnig the temporal file with awk and doing all the text procesing will decrease the number of binaries that have to be loaded (less i/o)

Quote:
Originally Posted by era View Post
tail is harder to emulate generically because the array type is rather crude. Maybe switch to sed (or Perl!) for that.
well. i see now, that perl is a best solution than bash. after all perl was made to process text right? :P
and perl has always been in my todo-list
the problem there would be to pipe the info to dzen
because the way dzen works, i need an never ending loop, so dzen dosnt die
is posible to do that piping from a perl script?
Quote:
Originally Posted by era View Post
thanks for the link. is in my bookmarks now
Quote:
Originally Posted by era View Post
The first few lines of top output indicate the CPU usage. Read the manual page to learn what the fields mean.


Code:
Cpu(s): 16.8%us,  1.7%sy,  0.0%ni, 80.5%id,  0.8%wa,  0.1%hi,  0.1%si,  0.0%st

[/quote]
i know that. the problem is getting that info without using head | tail | cut or awk
spawning all those apps 4 times on the same text input file doesnt strike me as good performance


thanks for the replies !
Sponsored Links
    #6  
Old Unix and Linux 08-04-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
 
Join Date: Mar 2008
Last Activity: 28 March 2011, 6:41 AM EDT
Location: /there/is/only/bin/sh
Posts: 3,653
Thanks: 0
Thanked 11 Times in 9 Posts
Quote:
Originally Posted by broli View Post
so, if i read your post correctly, with awk i can simulate sime functions of tail or head...
what about sort?
Modern versions of awk have a sort command. I'm not sure if it was included even in the original awk.

Quote:
well. i see now, that perl is a best solution than bash. after all perl was made to process text right? :P
and perl has always been in my todo-list
the problem there would be to pipe the info to dzen
because the way dzen works, i need an never ending loop, so dzen dosnt die
is posible to do that piping from a perl script?
Certainly, but I'm not really suggesting you move to Perl just for this. There are really two schools of thought on this; skip awk, and only learn Perl; or move to Perl if you run into situations which awk cannot handle comfortably. If you need to process binary data or very large data sets, Perl was designed to overcome the limitations awk have in those areas.

But the requirements you have shown so far should be easy to handle in awk; it's just not clear from reading the code in which direction you want the script to grow, and/or I was too lazy to rewrite it all.
Sponsored Links
    #7  
Old Unix and Linux 08-04-2008
broli's Unix or Linux Image
broli broli is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 23 July 2014, 5:32 PM EDT
Location: Argentina
Posts: 219
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by era View Post
Modern versions of awk have a sort command. I'm not sure if it was included even in the original awk.
im on gentoo here, so is the latest stable gawk (sorry, my bad, for not telling you earlier)

Quote:
Originally Posted by era View Post
Certainly, but I'm not really suggesting you move to Perl just for this. There are really two schools of thought on this; skip awk, and only learn Perl; or move to Perl if you run into situations which awk cannot handle comfortably. If you need to process binary data or very large data sets, Perl was designed to overcome the limitations awk have in those areas.
the very reason i started with linux, the very reason im still with linux, is that there is something to learn, teh chalenge, and the satisfaction of personal growth.
"moving to perl" doesn't mean anything "bad"
i will finish this with bash (because i wont let this sucker win). and after that, i will try to make it in perl. just because i can (or rather "just because i still dont know how" )

Quote:
Originally Posted by era View Post
But the requirements you have shown so far should be easy to handle in awk; it's just not clear from reading the code in which direction you want the script to grow, and/or I was too lazy to rewrite it all.
there is only one more thing i want from this script. show the total of memory used, and the total of cpu used, while keeping it light

for the cpu. i had some doubts, but i have decided to show the iddle percentage. this is a personal thing, i will use it, so if i want/know iddle%, thats what im gonna get :P

for the memory, i have been researching.
i was using wmmem, so i went and read the code to and read how it calculated. i came up with the math behind it.
((total - cache - buffer) * 100 ) / total
now my problem is getting those 3 from top

unfortunately, all my work is in my personal laptop (at home), and usually, i dont spent time in this during the week, so i cant give you any particulars on what i have done so far to retrieve that data
i dont ask you to rewrite my script. actually i would hate that.
as i said, i want to solve this, or at least try

thanks for the great help so far !
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Should I learn bash scripting or is it going obsolete? KalEl Shell Programming and Scripting 9 10-31-2010 07:53 PM
HOWTO - Total memory and CPU usage ... without top? newbie_01 UNIX for Dummies Questions & Answers 2 09-21-2010 01:16 AM
Any book to learn perl scripting vkvishwakarma6 Shell Programming and Scripting 5 02-24-2010 02:05 AM
Help- To learn shell scripting tj23 Shell Programming and Scripting 2 06-20-2008 07:10 AM
newbie: way to learn more about server's resource usage blakekr UNIX for Dummies Questions & Answers 2 05-20-2008 12:43 AM



All times are GMT -4. The time now is 05:47 AM.