Unix/Linux Go Back    


Solaris The Solaris Operating System, usually known simply as Solaris, is a Unix-based operating system introduced by Sun Microsystems. The Solaris OS is now owned by Oracle.

Find highest value of a particular property in multiple files

Solaris


Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Agoyals1986's Unix or Linux Image
Agoyals1986 Agoyals1986 is offline
Registered User
 
Join Date: Feb 2017
Last Activity: 23 January 2018, 1:09 PM EST
Posts: 6
Thanks: 1
Thanked 0 Times in 0 Posts
Find highest value of a particular property in multiple files

I have multiple files with pattern of "*.tps (example:tps-20170307170421560-1053.tps)" in my log directory(files are in different sub directories).

entries in files are given below. I want to extract highest value of endtime accross all files.



Code:
"endTime :1488902691462"



Code:
{"endTime":1488902691462,"startTime":1488902691462,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Util_Algos_Ctp_Notional_Bar:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":0}}
{"endTime":1488902691471,"startTime":1488902691471,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Diff_Bar_No_Engine:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":9}}

Sponsored Links
    #2  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
RavinderSingh13's Unix or Linux Image
RavinderSingh13 RavinderSingh13 is offline Forum Advisor  
Registered User
 
Join Date: May 2013
Last Activity: 20 February 2018, 5:59 AM EST
Location: Chennai
Posts: 2,697
Thanks: 601
Thanked 1,282 Times in 1,152 Posts
Hello Agoyals1986,

Could you please try following and let me know if this helps you.


Code:
 awk -v RS=':|,' '/endTime/{getline;val=val>$0?val:$0} END{print val}'  Input_file

On a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk.

Thanks,
R. Singh
Sponsored Links
    #3  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 20 February 2018, 2:35 PM EST
Location: Aachen, Germany
Posts: 12,142
Thanks: 367
Thanked 3,739 Times in 3,436 Posts
Try also


Code:
grep -ho '"endTime":[^,]*' *.tps | sort -r | head -1

    #4  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Agoyals1986's Unix or Linux Image
Agoyals1986 Agoyals1986 is offline
Registered User
 
Join Date: Feb 2017
Last Activity: 23 January 2018, 1:09 PM EST
Posts: 6
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks @RudiC working fine on linux but its not working on Solaris

Last edited by Agoyals1986; 09-28-2017 at 01:25 PM..
Sponsored Links
    #5  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
MadeInGermany's Unix or Linux Image
MadeInGermany MadeInGermany is offline Forum Staff  
Moderator
 
Join Date: May 2012
Last Activity: 20 February 2018, 2:37 AM EST
Location: Simplicity
Posts: 3,981
Thanks: 342
Thanked 1,337 Times in 1,206 Posts
With perl


Code:
perl -ne 'BEGIN {$search="endTime"; $max=0} if (/"$search":([0-9]*)/ && $1>$max) {$max=$1; $fname=$ARGV;} END {printf "%s \"%s\" %s\n",$fname,$search,$max}' *.tps

Omit what you don't need!

Last edited by MadeInGermany; 09-28-2017 at 02:37 PM..
Sponsored Links
    #6  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Yoda's Unix or Linux Image
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 20 February 2018, 2:34 PM EST
Location: Galactic Empire
Posts: 3,661
Thanks: 257
Thanked 1,327 Times in 1,244 Posts
If you have python installed:-


Code:
import glob
import re

in_file = "*.tps"

e_time = []

for file in glob.glob(in_file):
    with open(file, 'r') as f:
        for line in f:
            pattern = re.search('(?<=endTime":)\w+', line)
            e_time.append(pattern.group(0))

print(max(e_time))

Sponsored Links
    #7  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is offline Forum Staff  
Moderator
 
Join Date: Nov 2008
Last Activity: 20 February 2018, 11:55 AM EST
Location: Amsterdam
Posts: 11,731
Thanks: 536
Thanked 3,406 Times in 3,001 Posts
Also for subdirectories:


Code:
find . -name '*.tps' -exec sed '/endTime/!d; s/{//; s/,.*//' {} + | sort -r | head -1

The Following User Says Thank You to Scrutinizer For This Useful Post:
Agoyals1986 (09-29-2017)
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Find highest records in table GDC Shell Programming and Scripting 10 07-03-2013 11:47 AM
Script to find & replace a multiple lines string across multiple php files and subdirectories spfc_dmt Shell Programming and Scripting 12 03-07-2012 10:05 AM
Find highest number - working but need help! unknownsolo Shell Programming and Scripting 8 10-21-2011 04:52 PM
Report a missing property and property value mis match script. jayka Shell Programming and Scripting 5 08-24-2011 11:22 AM
find the highest number in the file systemali Shell Programming and Scripting 30 03-23-2006 01:31 PM



All times are GMT -4. The time now is 03:40 PM.