write a script for this pattern...


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting write a script for this pattern...
# 1  
Old 05-21-2009
write a script for this pattern...

I have a file say test.txt, data in that file is in format such that
1st column represents number of executions of particular function
2nd column represents total amount of time
3rd column represents function name

I want to get the poor/slow performing function from this file.
So, ideally what need to do is...
(1) divide the 2nd column by 1st column
(make sure there is no divide by ZERO error)
(2) sort the output in such a way that the record with greater output of step(1) is at top
(3) depending on this sorting display the result file.
result should not contain the division but should contain the original value.

example...

inputfile...

Code:
$ cat test.txt
3       (0 sec, 804275 nanosec)  abcd
721     (253 sec, 632655866 nanosec)     efgh
1873    (21 sec, 887264375 nanosec)      hijk
2006    (0 sec, 922300 nanosec)  lmhn
3       (0 sec, 3332 nanosec)    sjhfdf
2       (0 sec, 2367 nanosec)    dkfi
580     (0 sec, 2138703 nanosec)         dfhjf
44486   (0 sec, 858447770 nanosec)       oet
21102   (739 sec, 891470054 nanosec)     lldff
140     (0 sec, 90282 nanosec)   sfowe
123     0 sec, 324590 nanosec)  xjjv
8939    (0 sec, 337230856 nanosec)       asadj
19      (0 sec, 1455434 nanosec)         sahjdi
210     (0 sec, 89131 nanosec)   posdio
3       (0 sec, 3750149 nanosec)         jhd
873     (260 sec, 715043557 nanosec)     lppeuv
16      (0 sec, 212024 nanosec)  hjass
13668   (2 sec, 506391927 nanosec)       nnuua

-------------<end of file>------------------------

here... after division we see these are greater...
253/721 = 0.35090152
260/873 = 0.29782359

so output should look like...
721 (253 sec, 632655866 nanosec) efgh
873 (260 sec, 715043557 nanosec) lppeuv
.
.

Last edited by Yogesh Sawant; 06-28-2009 at 05:03 AM.. Reason: added code tags
# 2  
Old 05-21-2009
if you have Python
Code:
#!/usr/bin/env python
import operator
d={}
for line in open("file"):
    line=line.strip().split()
    d.setdefault(line[-1],"")
    try:
        d[line[-1]] = int(line[1].strip("("))/int(line[0])
    except Exception,e: pass
for i in sorted(d.iteritems(), key=operator.itemgetter(1), reverse=True):
    print i

output
Code:
# ./test.py
('efgh', 0.35090152565880722)
('lppeuv', 0.29782359679266895)
('lldff', 0.035020377215429818)
('hijk', 0.011211959423384944)
('nnuua', 0.00014632718759145449)
('abcd', 0.0)
('sahjdi', 0.0)
('sfowe', 0.0)
('jhd', 0.0)
('posdio', 0.0)
('asadj', 0.0)
('oet', 0.0)
('sjhfdf', 0.0)
('lmhn', 0.0)
('dkfi', 0.0)
('xjjv', 0.0)
('hjass', 0.0)
('dfhjf', 0.0)


Last edited by ghostdog74; 05-22-2009 at 02:50 AM..
# 3  
Old 05-22-2009
this will solve your divide by zero problem
Code:
awk -F"[( ]" '{if ($1=="0"){print "0.000000 "$NF}else{printf "%f %s\n",($3+($5/1000000000))/$1,$NF}}' filename|sort -r -k1

# 4  
Old 06-11-2009
awk -F"[( ]" '{if ($1=="0"){print "0.000000 "$NF}else{printf "%f %s\n",($3+($5/1000000000))/$1,$NF}}'perf.fmt|sort -r -k1
Syntax Error The source line is 1.
The error context is
{if ($1=="0"){print "0.000000 (NF)}else{printf "%f >>> %s\n",($3+($5/1000000000))/$1(NF)}}perf. <<< fmt
awk: 0602-500 Quitting The source line is 1.





-----Post Update-----

After changing input file name from perf.fmt to perf, im getting below error

$ awk -F"[( ]" '{if ($1=="0"){print "0.000000 "$NF}else{printf "%f %s\n",($3+($5/1000000000))/$1,$NF}}'perf|sort -r -k1

awk: 0602-566 Cannot divide by zero.
The input line number is 1.
The source line number is 1.

# 5  
Old 06-12-2009
can anyone fix this problem?
# 6  
Old 06-12-2009
you have to give a space between awk and filename
Code:
awk -F"[( ]" '{if ($1=="0"){print "0.000000 "$NF}else{printf "%f %s\n",($3+($5/1000000000))/$1,$NF}}'<space>filename

IF the test file you provided is correct code works perfectly
Code:
/home/vidya_perl> cat vv
3 (0 sec, 804275 nanosec) abcd
721 (253 sec, 632655866 nanosec) efgh
1873 (21 sec, 887264375 nanosec) hijk
2006 (0 sec, 922300 nanosec) lmhn
3 (0 sec, 3332 nanosec) sjhfdf
2 (0 sec, 2367 nanosec) dkfi
580 (0 sec, 2138703 nanosec) dfhjf
44486 (0 sec, 858447770 nanosec) oet
21102 (739 sec, 891470054 nanosec) lldff
140 (0 sec, 90282 nanosec) sfowe
123 (0 sec, 324590 nanosec) xjjv
8939 (0 sec, 337230856 nanosec) asadj
19 (0 sec, 1455434 nanosec) sahjdi
210 (0 sec, 89131 nanosec) posdio
3 (0 sec, 3750149 nanosec) jhd
873 (260 sec, 715043557 nanosec) lppeuv
16 (0 sec, 212024 nanosec) hjass
13668 (2 sec, 506391927 nanosec) nnuua
/home/vidya_perl> awk -F"[( ]" '{if ($1=="0"){print "0.000000 "$NF}else{printf "%f %s\n",($3+($5/1000000000))/$1,$NF}}' vv |sort ->
0.351779 efgh
0.298643 lppeuv
0.035063 lldff
0.011686 hijk
0.001250 jhd
0.000268 abcd
0.000183 nnuua
0.000077 sahjdi
0.000038 asadj
0.000019 oet
0.000013 hjass
0.000004 dfhjf
0.000003 xjjv
0.000001 sjhfdf
0.000001 sfowe
0.000001 dkfi
0.000000 posdio
0.000000 lmhn

# 7  
Old 06-12-2009
Thanks vidyadhar85 for details.

Now im getting good results but...there is something diff from your results...check the 1st row. It has NaNQ.

$ $3+($5/1000000000))/$1,$NF}}' perf |sort -r -k1 | more <
NaNQ xjjv
0.351779 efgh
0.298643 lppeuv
0.035063 lldff
0.011686 hijk
0.001250 jhd
0.000268 abcd
0.000183 nnuua
0.000077 sahjdi
0.000038 asadj
0.000019 oet
0.000013 hjass
0.000004 dfhjf
0.000001 sjhfdf
0.000001 sfowe
0.000001 dkfi
0.000000 posdio
0.000000 lmhn


When I use this script for different input file which has similar format/pattern....I get all 1st column values as NaNQ.
Please let me know where is the problem.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk; pattern match, remove and re write

the following pattern match works correctly for me awk '/name="Fruits"/{f=1;next} /"name=Vegetables"/{f=0} f' filename This works well for me. Id like to temporarily move the match out of the file ( > newfile) and be able to stick it back in the same place at a later time. Is this... (7 Replies)
Discussion started by: TY718
7 Replies

2. Shell Programming and Scripting

Search pattern and write line into another file

Hi, I have a file which contains the below details.. My requirement is to fetch all the lines which are starting with "ABC_XY_" into 1 file and rest of the lines (not starting with "ABC_XY_") into another file. Could you please help with what command needs to be used? file1.txt ----------... (12 Replies)
Discussion started by: satyaatcgi
12 Replies

3. Shell Programming and Scripting

[Solved] Pattern match and write to separate files

I need to parse a file and depending on a patern match(in the insert job line) separate files have to be created with a line added (content in file2). Mapping for pattern match and add line : for Alpha 123 for Beta 234 for Gamma 345 no match (goes into another file) File 1 ... (3 Replies)
Discussion started by: w020637
3 Replies

4. Shell Programming and Scripting

Check for Pattern if exists write to file

Hi ! All I just want to search and write to new file if pattern is found in text file following are my text files by which I want to search Month and last column number my text file1 15-Jan-2011 25 ARTS 1255 125 125 178 198 15-Jan-2011 25 ARTS 1255 125 125 178 198 15-Jan-2011 25... (3 Replies)
Discussion started by: nex_asp
3 Replies

5. Shell Programming and Scripting

Script to compare pattern and print a different pattern in each line

Hi, I am writing a shell script to parse some files, and gather data. The data in the files is displayed as below. .......xyz: abz: ......qrt: .... .......xyz: abz: ......qrt: ... I have tried using awk and cut, but the position of these values keep changing, so I wasn't able to get... (2 Replies)
Discussion started by: Serena
2 Replies

6. Shell Programming and Scripting

PERL : Read an array and write to another array with intial string pattern checks

I have an array and two variables as below, I need to check if $datevar is present in $filename. If so, i need to replace $filename with the values in the array. I need the output inside an ARRAY How can this be done. Any help will be appreciated. Thanks in advance. (2 Replies)
Discussion started by: irudayaraj
2 Replies

7. IP Networking

read/write,write/write lock with smbclient fails

Hi, We have smb client running on two of the linux boxes and smb server on another linux system. During a backup operation which uses smb, read of a file was allowed while write to the same file was going on.Also simultaneous writes to the same file were allowed.Following are the settings in the... (1 Reply)
Discussion started by: swatidas11
1 Replies

8. Shell Programming and Scripting

grep a pattern and replace a value in it and write to the same file.

I have some complication with this, I have a file like below for DEV_1 till DEV_10. and the db values are set accordinly which are not unique. For example DEV1,DEV4,DEV6 can have the same target DB name. I waned to identify for DEV_2 and then replace the TARGET_DATABASE value with the new DB... (6 Replies)
Discussion started by: yesmani
6 Replies

9. Shell Programming and Scripting

To write a shell script which groups files with certain pattern, create a tar and zip

Hi Guru's, I have to write a shell script which groups file names based upon the certain matching string pattern, then creates the Tar file for that particular group of files and then zips the Tar file created for the respective group of files. For example, In the given directory these files... (3 Replies)
Discussion started by: rahu_sg
3 Replies

10. Shell Programming and Scripting

help needed .. Unable to write the data to new file after matching the pattern

Hi, i am pretty new to Unix environment ..... Can i get some help from any of you guyz on writing Unix script. my requirement is like reading a csv file, finding a specific pattern in the lines and repalce the string with new string and write it to another file. My file is file ABC123.dat... (3 Replies)
Discussion started by: prashant_jsw
3 Replies
Login or Register to Ask a Question