Unix/Linux Go Back    


Solaris The Solaris Operating System, usually known simply as Solaris, is a Unix-based operating system introduced by Sun Microsystems. The Solaris OS is now owned by Oracle.

Find highest value of a particular property in multiple files

Solaris


Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Agoyals1986's Unix or Linux Image
Agoyals1986 Agoyals1986 is offline
Registered User
 
Join Date: Feb 2017
Last Activity: 2 October 2017, 4:59 AM EDT
Posts: 6
Thanks: 1
Thanked 0 Times in 0 Posts
Find highest value of a particular property in multiple files

I have multiple files with pattern of "*.tps (example:tps-20170307170421560-1053.tps)" in my log directory(files are in different sub directories).

entries in files are given below. I want to extract highest value of endtime accross all files.


Code:
"endTime :1488902691462"


Code:
{"endTime":1488902691462,"startTime":1488902691462,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Util_Algos_Ctp_Notional_Bar:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":0}}
{"endTime":1488902691471,"startTime":1488902691471,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Diff_Bar_No_Engine:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":9}}

Sponsored Links
    #2  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
RavinderSingh13's Unix or Linux Image
RavinderSingh13 RavinderSingh13 is offline Forum Advisor  
Registered User
 
Join Date: May 2013
Last Activity: 14 December 2017, 1:30 AM EST
Location: Chennai
Posts: 2,678
Thanks: 590
Thanked 1,275 Times in 1,146 Posts
Hello Agoyals1986,

Could you please try following and let me know if this helps you.

Code:
 awk -v RS=':|,' '/endTime/{getline;val=val>$0?val:$0} END{print val}'  Input_file

On a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk.

Thanks,
R. Singh
Sponsored Links
    #3  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 13 December 2017, 4:17 PM EST
Location: Aachen, Germany
Posts: 11,797
Thanks: 339
Thanked 3,659 Times in 3,357 Posts
Try also

Code:
grep -ho '"endTime":[^,]*' *.tps | sort -r | head -1

    #4  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Agoyals1986's Unix or Linux Image
Agoyals1986 Agoyals1986 is offline
Registered User
 
Join Date: Feb 2017
Last Activity: 2 October 2017, 4:59 AM EDT
Posts: 6
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks @RudiC working fine on linux but its not working on Solaris

Last edited by Agoyals1986; 09-28-2017 at 01:25 PM..
Sponsored Links
    #5  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
MadeInGermany's Unix or Linux Image
MadeInGermany MadeInGermany is offline Forum Staff  
Moderator
 
Join Date: May 2012
Last Activity: 14 December 2017, 2:10 AM EST
Location: Simplicity
Posts: 3,864
Thanks: 320
Thanked 1,289 Times in 1,167 Posts
With perl

Code:
perl -ne 'BEGIN {$search="endTime"; $max=0} if (/"$search":([0-9]*)/ && $1>$max) {$max=$1; $fname=$ARGV;} END {printf "%s \"%s\" %s\n",$fname,$search,$max}' *.tps

Omit what you don't need!

Last edited by MadeInGermany; 09-28-2017 at 02:37 PM..
Sponsored Links
    #6  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Yoda's Unix or Linux Image
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 13 December 2017, 10:07 PM EST
Location: Galactic Empire
Posts: 3,612
Thanks: 254
Thanked 1,317 Times in 1,234 Posts
If you have python installed:-

Code:
import glob
import re

in_file = "*.tps"

e_time = []

for file in glob.glob(in_file):
    with open(file, 'r') as f:
        for line in f:
            pattern = re.search('(?<=endTime":)\w+', line)
            e_time.append(pattern.group(0))

print(max(e_time))

Sponsored Links
    #7  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is online now Forum Staff  
Moderator
 
Join Date: Nov 2008
Last Activity: 14 December 2017, 2:59 AM EST
Location: Amsterdam
Posts: 11,651
Thanks: 518
Thanked 3,385 Times in 2,984 Posts
Also for subdirectories:

Code:
find . -name '*.tps' -exec sed '/endTime/!d; s/{//; s/,.*//' {} + | sort -r | head -1

The Following User Says Thank You to Scrutinizer For This Useful Post:
Agoyals1986 (09-29-2017)
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Find highest records in table GDC Shell Programming and Scripting 10 07-03-2013 11:47 AM
Script to find & replace a multiple lines string across multiple php files and subdirectories spfc_dmt Shell Programming and Scripting 12 03-07-2012 10:05 AM
Find highest number - working but need help! unknownsolo Shell Programming and Scripting 8 10-21-2011 04:52 PM
Report a missing property and property value mis match script. jayka Shell Programming and Scripting 5 08-24-2011 11:22 AM
find the highest number in the file systemali Shell Programming and Scripting 30 03-23-2006 01:31 PM



All times are GMT -4. The time now is 04:00 AM.