Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help With shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help With shell script
# 1  
Old 03-18-2010
Help With shell script
Hey Guys,

I have a script that my collegue helped me write it:
Code:
awk ' {if ($1 == "PAID_FILL_RATE") printf "%s:%-3.2f\n", $1, substr($4, 2, 99); else if ($1!="WINDOW" && $1~/^[^\|]/ && $1~/^[^0-9]/) printf "%s:%-3.2f ", $1, substr($4, 2, 99)} ' data/network_health*

And it gives the following output:

Code:
#:0.00 AD_REQUESTS:0.00 PAID_REVENUE:15.01 ABC_REVENUE:15.75 PAGE_VIEWS:-4.46 PAID_IMPRESSIONS:-8.22 PAID_CLICKS:3.46 PAID_ECPM:25.32 PAID_CTR:12.50 PAID_FILL_RATE:-7.15
AD_REQUESTS:-3.25 PAID_REVENUE:19.68 ABC_REVENUE:20.33 PAGE_VIEWS:-5.65 PAID_IMPRESSIONS:-7.99 PAID_CLICKS:5.19 PAID_ECPM:30.08 PAID_CTR:14.81 PAID_FILL_RATE:-4.82
AD_REQUESTS:-3.61 PAID_REVENUE:11.16 ABC_REVENUE:11.87 PAGE_VIEWS:-5.62 PAID_IMPRESSIONS:-7.45 PAID_CLICKS:1.47 PAID_ECPM:20.12 PAID_CTR:7.41 PAID_FILL_RATE:-3.93

The problem is that cacti (network monitoring tool) is not recognizing the data due to the 0.00 read in front of the data, and thats why the trouble.

Just for your reference, here is the input data:

Code:
WINDOW |TODAY |YESTERDAY |Y_CHANGE 
|2010-03-12|2010-03-11|03/12 - 03/11 (%)
00:00:00 - 08:00:00| | | 
AD_REQUESTS |76,712,939|77,664,044|-1.22 
PAID_REVENUE |24257.70 |21092.30 |15.01 
ABC_REVENUE |23007.95 |19876.68 |15.75 
PAGE_VIEWS |1608304 |1683381 |-4.46 
PAID_IMPRESSIONS |43567950 |47472365 |-8.22 
PAID_CLICKS |117688 |113747 |3.46 
PAID_ECPM |0.5568 |0.4443 |25.32 
PAID_CTR |0.0027 |0.0024 |12.50 
PAID_FILL_RATE |0.5792 |0.6238 |-7.15 
07:00:00 - 08:00:00| | | 
AD_REQUESTS |9,309,473 |9,621,876 |-3.25 
PAID_REVENUE |3183.77 |2660.34 |19.68 
ABC_REVENUE |3020.16 |2509.90 |20.33 
PAGE_VIEWS |255484 |270772 |-5.65 
PAID_IMPRESSIONS |4555517 |4951013 |-7.99 
PAID_CLICKS |14085 |13390 |5.19 
PAID_ECPM |0.6989 |0.5373 |30.08 
PAID_CTR |0.0031 |0.0027 |14.81 
PAID_FILL_RATE |0.4981 |0.5233 |-4.82 
06:00:00 - 07:00:00| | | 
AD_REQUESTS |7,917,841 |8,214,757 |-3.61 
PAID_REVENUE |2487.94 |2238.26 |11.16 
ABC_REVENUE |2345.53 |2096.60 |11.87 
PAGE_VIEWS |183887 |194846 |-5.62 
PAID_IMPRESSIONS |3902349 |4216501 |-7.45 
PAID_CLICKS |11363 |11198 |1.47 
PAID_ECPM |0.6376 |0.5308 |20.12 
PAID_CTR |0.0029 |0.0027 |7.41 
PAID_FILL_RATE |0.5013 |0.5218 |-3.93

As you see the script is calling the 1st and the 4th column after i did the modification to the script, except for that 0.00 in the front and the wrong value being pulled up for AD_REQUEST in the first instance instead of -1.22.

So the expected output i want to see is as follows:
Code:
AD_REQUESTS:-1.22 PAID_REVENUE:15.01 ABC_REVENUE:15.75 PAGE_VIEWS:-4.46 PAID_IMPRESSIONS:-8.22 PAID_CLICKS:3.46 PAID_ECPM:25.32 PAID_CTR:12.50 PAID_FILL_RATE:-7.15 AD_REQUESTS:-3.25 PAID_REVENUE:19.68 ABC_REVENUE:20.33 PAGE_VIEWS:-5.65 PAID_IMPRESSIONS:-7.99 PAID_CLICKS:5.19 PAID_ECPM:30.08 PAID_CTR:14.81 PAID_FILL_RATE:-4.82 AD_REQUESTS:-3.61 PAID_REVENUE:11.16 ABC_REVENUE:11.87 PAGE_VIEWS:-5.62 PAID_IMPRESSIONS:-7.45 PAID_CLICKS:1.47 PAID_ECPM:20.12 PAID_CTR:7.41 PAID_FILL_RATE:-3.93

Can anyone please suggest what modification do i need to make to the script. I have been trying to resolve this since the last 45 hours now !!

Thanks,

Any help is obviously greatly appreciated.
Last edited by Franklin52; 03-18-2010 at 04:14 PM.. Reason: Please use code tags!
# 2  
Old 03-18-2010
Question Can you simply filter the incoming data?
Not clear, but it seems that if you eliminated the lines that begin with | or number. Is that a correct assumption?
# 3  
Old 03-18-2010
So Basically what i need is to have the first and the fourth column of the input data all in a sequential line with the format of Field:value (example: AD_REQUEST:-1.22) and so in order to do that, i have been succesful in getting the data in correct format but it shows up on different lines. So i want them to come on the same line and get rid of the two garbage values showing up i.e "0.0" in the beginning and AD_REQUEST : 0.0 in the very first line should be pulling up -1.22.

I hope i was clear enough this time. Smilie
# 4  
Old 03-18-2010
If I put your sample data into data.txt and run through this perl script I get:

Code:
perl -nle 'while (/^([a-zA-Z_]+).*?\|.*?\|.*?\|([0-9\.-]+)/sg) {print "$1:$2 "}' data.txt
AD_REQUESTS:-1.22 
PAID_REVENUE:15.01 
ABC_REVENUE:15.75 
PAGE_VIEWS:-4.46 
PAID_IMPRESSIONS:-8.22 
PAID_CLICKS:3.46 
PAID_ECPM:25.32 
PAID_CTR:12.50 
PAID_FILL_RATE:-7.15 
AD_REQUESTS:-3.25 
PAID_REVENUE:19.68 
ABC_REVENUE:20.33 
PAGE_VIEWS:-5.65 
PAID_IMPRESSIONS:-7.99 
PAID_CLICKS:5.19 
PAID_ECPM:30.08 
PAID_CTR:14.81 
PAID_FILL_RATE:-4.82 
AD_REQUESTS:-3.61 
PAID_REVENUE:11.16 
ABC_REVENUE:11.87 
PAGE_VIEWS:-5.62 
PAID_IMPRESSIONS:-7.45 
PAID_CLICKS:1.47 
PAID_ECPM:20.12 
PAID_CTR:7.41 
PAID_FILL_RATE:-3.93

Or, if you want a single line:

Code:
$ perl -nle 'while (/^([a-zA-Z_]+).*?\|.*?\|.*?\|([0-9\.-]+)/sg) {$out.="$1:$2 ";} END {print $out;}' data.txt
AD_REQUESTS:-1.22 PAID_REVENUE:15.01 ABC_REVENUE:15.75 PAGE_VIEWS:-4.46 PAID_IMPRESSIONS:-8.22 PAID_CLICKS:3.46 PAID_ECPM:25.32 PAID_CTR:12.50 PAID_FILL_RATE:-7.15 AD_REQUESTS:-3.25 PAID_REVENUE:19.68 ABC_REVENUE:20.33 PAGE_VIEWS:-5.65 PAID_IMPRESSIONS:-7.99 PAID_CLICKS:5.19 PAID_ECPM:30.08 PAID_CTR:14.81 PAID_FILL_RATE:-4.82 AD_REQUESTS:-3.61 PAID_REVENUE:11.16 ABC_REVENUE:11.87 PAGE_VIEWS:-5.62 PAID_IMPRESSIONS:-7.45 PAID_CLICKS:1.47 PAID_ECPM:20.12 PAID_CTR:7.41 PAID_FILL_RATE:-3.93

Is that what you want?
Last edited by drewk; 03-18-2010 at 05:27 PM..
# 5  
Old 03-18-2010
Hammer & Screwdriver what about this command?
Code:
>grep "^[A-Z]" <choco.txt | grep -v "^WINDOW" | gawk -F"|" '{print $1":"$4}' | tr -d " " | tr "\n" " "

depending on your flavor of unix, might be gawk or awk.
# 6  
Old 03-18-2010
Or this:

Code:
egrep "^([a-zA-Z_]+ )(\|.*?){3}([0-9\.-]+)" <data.txt | awk -F"|" '{print $1":"$4}' | tr -d " " | tr "\n" " "

# 7  
Old 03-18-2010
Thanks guys,

I appreciate the help. But my output did not help me in getting the desired result in cacti.

Here is the script that i came up with before you guys responded:

Code:
awk -F"|" '{gsub(/\ /,"", $1); if ($1~/\_/) printf "%s:%-3.2f ", $1, $4} ' /etc/cacti/scripts/data/network_health*

And it does give me the desired result which i asked for earlier:

Code:
AD_REQUESTS:-1.22 PAID_REVENUE:15.01 ABC_REVENUE:15.75 PAGE_VIEWS:-4.46 PAID_IMPRESSIONS:-8.22 PAID_CLICKS:3.46 PAID_ECPM:25.32 PAID_CTR:12.50 PAID_FILL_RATE:-7.15 AD_REQUESTS:-3.25 PAID_REVENUE:19.68 ABC_REVENUE:20.33 PAGE_VIEWS:-5.65 PAID_IMPRESSIONS:-7.99 PAID_CLICKS:5.19 PAID_ECPM:30.08 PAID_CTR:14.81 PAID_FILL_RATE:-4.82 AD_REQUESTS:-3.61 PAID_REVENUE:11.16 ABC_REVENUE:11.87 PAGE_VIEWS:-5.62 PAID_IMPRESSIONS:-7.45 PAID_CLICKS:1.47 PAID_ECPM:20.12 PAID_CTR:7.41 PAID_FILL_RATE:-3.93

Now what i will have to do essentially is write 3 scripts; one for pulling each set and then chart it to 3 different graphs.

So here is what each set should look like for each script:
Set 1:
Code:
AD_REQUESTS:-1.22 PAID_REVENUE:15.01 ABC_REVENUE:15.75 PAGE_VIEWS:-4.46 PAID_IMPRESSIONS:-8.22 PAID_CLICKS:3.46 PAID_ECPM:25.32 PAID_CTR:12.50 PAID_FILL_RATE:-7.15

Set 2:
Code:
AD_REQUESTS:-3.25 PAID_REVENUE:19.68 ABC_REVENUE:20.33 PAGE_VIEWS:-5.65 PAID_IMPRESSIONS:-7.99 PAID_CLICKS:5.19 PAID_ECPM:30.08 PAID_CTR:14.81 PAID_FILL_RATE:-4.82

Set 3:

Code:
AD_REQUESTS:-3.61 PAID_REVENUE:11.16 ABC_REVENUE:11.87 PAGE_VIEWS:-5.62 PAID_IMPRESSIONS:-7.45 PAID_CLICKS:1.47 PAID_ECPM:20.12 PAID_CTR:7.41 PAID_FILL_RATE:-3.93



Any easy way to do this?

Thanks,
Choco
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script newbie- how to generate service log from shell script

Hi, I am totally a newbie to any programming languages and I just started an entry level job in an IT company. One of my recent tasks is to create a script that is able to show the log file of linux service (i.e. ntpd service) lets say, if I run my script ./test.sh, the output should be... (3 Replies)
Discussion started by: xiaogeji
3 Replies

2. Shell Programming and Scripting

Pass C shell array to another C shell script(csh) and shell(sh)

Dear Friends, Please help me on this my script name is send.csh In this i have written the statement like this set args = ( city state country price ) I want to pass this array to another c shell called receiver.csh. and i want to use it in this c shell or how to pass to... (2 Replies)
Discussion started by: SA_Palani
2 Replies

3. Shell Programming and Scripting

How to write config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

4. UNIX for Dummies Questions & Answers

How to write Config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

5. Shell Programming and Scripting

Unable to pass shell script variable to awk command in same shell script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk... (2 Replies)
Discussion started by: Ashunayak
2 Replies

6. Shell Programming and Scripting

Correct shell script to Call One shell script from another shell script

Hi All, I have new for shell scripting. Problem : I have one scrip at serv1 and path of server is /apps/dev/provimage/scripts and script name:extract_ancillary.bat. I need to call this script at server2(my working server) and execute at server2 . Please let me know how to build the... (5 Replies)
Discussion started by: Vineeta Nigam
5 Replies

7. Shell Programming and Scripting

call another shell script and pass parameters to that shell script

Hi, I basically have 2 shell scripts. One is a shell script will get the variable value from the user. The variable is nothing but the IP of the remote system. Another shell script is a script that does the job of connecting to the remote system using ssh. This uses a expect utility in turn. ... (2 Replies)
Discussion started by: sunrexstar
2 Replies

8. Shell Programming and Scripting

How to use ssh execute other shell script on other host (shell script include nohup)?

i want use ssh on the host01 to execute autoexec.sh on the host02 like following : host01> ssh host02 autoexec.sh autoexec.sh include nohup command like follwing : nohup /home/jack/deletedata.sh & after i execute ssh host02 autoexec.sh one the host01. i can't found deletedata.sh... (1 Reply)
Discussion started by: orablue
1 Replies

9. Shell Programming and Scripting

invoking a shell script inside cgi shell script

Hi, I have an HTML form through which I get some text as input. i need to run a shell script say script.sh inside a perl-cgi script named main_cgi.sh on the form input. I want to write the contents of the form in a file and then perform some command line operations like grep, cat on the text... (2 Replies)
Discussion started by: smriti_shridhar
2 Replies

10. Shell Programming and Scripting

How to Run a shell script from Perl script in Parent shell?

Hi Perl/UNIX experts, I have a problem in running a shell script from my perl script (auto.pl). I run the perl script using perl auto.pl from the shell prompt The shell script picks the files in "input" folder and procesess it. The shell script blue.sh has this code. export... (16 Replies)
Discussion started by: hifake
16 Replies
Login or Register to Ask a Question