Unix/Linux Go Back    


Solaris The Solaris Operating System, usually known simply as Solaris, is a Unix-based operating system introduced by Sun Microsystems. The Solaris OS is now owned by Oracle.

Find highest value of a particular property in multiple files

Solaris


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Agoyals1986's Unix or Linux Image
Agoyals1986 Agoyals1986 is offline
Registered User
 
Join Date: Feb 2017
Last Activity: 16 April 2018, 8:21 AM EDT
Posts: 6
Thanks: 1
Thanked 0 Times in 0 Posts
Find highest value of a particular property in multiple files

I have multiple files with pattern of "*.tps (example:tps-20170307170421560-1053.tps)" in my log directory(files are in different sub directories).

entries in files are given below. I want to extract highest value of endtime accross all files.



Code:
"endTime :1488902691462"



Code:
{"endTime":1488902691462,"startTime":1488902691462,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Util_Algos_Ctp_Notional_Bar:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":0}}
{"endTime":1488902691471,"startTime":1488902691471,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Diff_Bar_No_Engine:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":9}}

Sponsored Links
    #2  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
RavinderSingh13's Unix or Linux Image
RavinderSingh13 RavinderSingh13 is offline Forum Advisor  
Registered User
 
Join Date: May 2013
Last Activity: 20 June 2018, 5:45 AM EDT
Location: Chennai
Posts: 2,739
Thanks: 619
Thanked 1,309 Times in 1,176 Posts
Hello Agoyals1986,

Could you please try following and let me know if this helps you.


Code:
 awk -v RS=':|,' '/endTime/{getline;val=val>$0?val:$0} END{print val}'  Input_file

On a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk.

Thanks,
R. Singh
Sponsored Links
    #3  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 20 June 2018, 6:12 AM EDT
Location: Aachen, Germany
Posts: 12,908
Thanks: 429
Thanked 3,966 Times in 3,647 Posts
Try also


Code:
grep -ho '"endTime":[^,]*' *.tps | sort -r | head -1

    #4  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Agoyals1986's Unix or Linux Image
Agoyals1986 Agoyals1986 is offline
Registered User
 
Join Date: Feb 2017
Last Activity: 16 April 2018, 8:21 AM EDT
Posts: 6
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks @RudiC working fine on linux but its not working on Solaris

Last edited by Agoyals1986; 09-28-2017 at 12:25 PM..
Sponsored Links
    #5  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
MadeInGermany's Unix or Linux Image
MadeInGermany MadeInGermany is offline Forum Staff  
Moderator
 
Join Date: May 2012
Last Activity: 20 June 2018, 2:59 AM EDT
Location: Simplicity
Posts: 4,119
Thanks: 357
Thanked 1,401 Times in 1,258 Posts
With perl


Code:
perl -ne 'BEGIN {$search="endTime"; $max=0} if (/"$search":([0-9]*)/ && $1>$max) {$max=$1; $fname=$ARGV;} END {printf "%s \"%s\" %s\n",$fname,$search,$max}' *.tps

Omit what you don't need!

Last edited by MadeInGermany; 09-28-2017 at 01:37 PM..
Sponsored Links
    #6  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Yoda's Unix or Linux Image
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 20 June 2018, 12:19 PM EDT
Location: Galactic Empire
Posts: 3,671
Thanks: 257
Thanked 1,331 Times in 1,247 Posts
If you have python installed:-


Code:
import glob
import re

in_file = "*.tps"

e_time = []

for file in glob.glob(in_file):
    with open(file, 'r') as f:
        for line in f:
            pattern = re.search('(?<=endTime":)\w+', line)
            e_time.append(pattern.group(0))

print(max(e_time))

Sponsored Links
    #7  
Old Unix and Linux 09-28-2017   -   Original Discussion by Agoyals1986
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is online now Forum Staff  
Moderator
 
Join Date: Nov 2008
Last Activity: 20 June 2018, 12:54 PM EDT
Location: Amsterdam
Posts: 11,829
Thanks: 545
Thanked 3,454 Times in 3,044 Posts
Also for subdirectories:


Code:
find . -name '*.tps' -exec sed '/endTime/!d; s/{//; s/,.*//' {} + | sort -r | head -1

The Following User Says Thank You to Scrutinizer For This Useful Post:
Agoyals1986 (09-29-2017)
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Find highest records in table GDC Shell Programming and Scripting 10 07-03-2013 10:47 AM
Script to find & replace a multiple lines string across multiple php files and subdirectories spfc_dmt Shell Programming and Scripting 12 03-07-2012 09:05 AM
Find highest number - working but need help! unknownsolo Shell Programming and Scripting 8 10-21-2011 03:52 PM
Report a missing property and property value mis match script. jayka Shell Programming and Scripting 5 08-24-2011 10:22 AM
find the highest number in the file systemali Shell Programming and Scripting 30 03-23-2006 12:31 PM



All times are GMT -4. The time now is 01:07 PM.