Find highest value of a particular property in multiple files


 
Thread Tools Search this Thread
Operating Systems Solaris Find highest value of a particular property in multiple files
# 1  
Old 09-28-2017
Find highest value of a particular property in multiple files

I have multiple files with pattern of "*.tps (example:tps-20170307170421560-1053.tps)" in my log directory(files are in different sub directories).

entries in files are given below. I want to extract highest value of endtime accross all files.

Code:
"endTime :1488902691462"

Code:
{"endTime":1488902691462,"startTime":1488902691462,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Util_Algos_Ctp_Notional_Bar:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":0}}
{"endTime":1488902691471,"startTime":1488902691471,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"GlobalRuleCache:Diff_Bar_No_Engine:ValidationRuleRepository","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"0","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":9}}

# 2  
Old 09-28-2017
Hello Agoyals1986,

Could you please try following and let me know if this helps you.
Code:
 awk -v RS=':|,' '/endTime/{getline;val=val>$0?val:$0} END{print val}'  Input_file

On a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk.

Thanks,
R. Singh
# 3  
Old 09-28-2017
Try also
Code:
grep -ho '"endTime":[^,]*' *.tps | sort -r | head -1

# 4  
Old 09-28-2017
Thanks @RudiC working fine on linux but its not working on Solaris

Last edited by Agoyals1986; 09-28-2017 at 01:25 PM..
# 5  
Old 09-28-2017
With perl
Code:
perl -ne 'BEGIN {$search="endTime"; $max=0} if (/"$search":([0-9]*)/ && $1>$max) {$max=$1; $fname=$ARGV;} END {printf "%s \"%s\" %s\n",$fname,$search,$max}' *.tps

Omit what you don't need!

Last edited by MadeInGermany; 09-28-2017 at 02:37 PM..
# 6  
Old 09-28-2017
If you have python installed:-
Code:
import glob
import re

in_file = "*.tps"

e_time = []

for file in glob.glob(in_file):
    with open(file, 'r') as f:
        for line in f:
            pattern = re.search('(?<=endTime":)\w+', line)
            e_time.append(pattern.group(0))

print(max(e_time))

# 7  
Old 09-28-2017
Also for subdirectories:
Code:
find . -name '*.tps' -exec sed '/endTime/!d; s/{//; s/,.*//' {} + | sort -r | head -1

This User Gave Thanks to Scrutinizer For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find and replace in multiple files

Hi, I have php files in main dir and sub dir's as well. I need to find "new mysqli('localhost', 'System', 'xxxxxx', 'System', '3306');" and replace as "new mysqli('localhost', 'unx_sys', 'yyyy', 'unx_sys', '3306');" I tried like: sed 's/new mysqli\(*\)\;$/new... (1 Reply)
Discussion started by: ashokvpp
1 Replies

2. Shell Programming and Scripting

Not to remove Files based on property value

Hi All, Need your help to fix one script. Main agenda is: 1. Read a property file. 2. Delete all files in directory except the name from Property file. I am trying to read property file for value then deleting all files from directory except THAT value/name. I have tried so far as... (3 Replies)
Discussion started by: sukhdip
3 Replies

3. Shell Programming and Scripting

Find highest records in table

Dear All, I have table files with different measurements for the same sensors. The first column indicate the sensor (7 different sensors and 16 measurements in the example below). I would like to find the best measurement for each sensor. The best measurement is determined by the higher value of... (10 Replies)
Discussion started by: GDC
10 Replies

4. Shell Programming and Scripting

Script to find & replace a multiple lines string across multiple php files and subdirectories

Hey guys. I know pratically 0 about Linux, so could anyone please give me instructions on how to accomplish this ? The distro is RedHat 4.1.2 and i need to find and replace a multiple lines string in several php files across subdirectories. So lets say im at root/dir1/dir2/ , when i execute... (12 Replies)
Discussion started by: spfc_dmt
12 Replies

5. Shell Programming and Scripting

Find highest number - working but need help!

Hello all, I am new to this and need some help or maybe steer me to the right direction! I wrote a script to get the highest number and prints it on the screen, the script basically asks the user to input numbers, and then prints the highest number! very simple it works like this $sh max.sh... (8 Replies)
Discussion started by: unknownsolo
8 Replies

6. Shell Programming and Scripting

Report a missing property and property value mis match script.

Hi All, I have 2 properties files - one is a master templete and other one is a node specific properties file, I need to comapre these 2 properties files and make sure the node Specific properties file contains all the properties in the master temple properties file else report the missing... (5 Replies)
Discussion started by: jayka
5 Replies

7. Programming

Help with find highest and smallest number in a file with c

Input file: #data_1 AGDG #data_2 ADG #data_3 ASDDG DG #data_4 A Desired result: Highest 7 Slowest 1 code that I try but failed to archive my goal :( #include <stdio.h> (2 Replies)
Discussion started by: cpp_beginner
2 Replies

8. Shell Programming and Scripting

Perl ? - How to find and print the lowest and highest numbers punched in by the user?

. . . . . . (3 Replies)
Discussion started by: some124one
3 Replies

9. UNIX for Dummies Questions & Answers

Design Options for Property Files

Dear all, Hello and Good Morning. I have a properties file in a specific directory in UNIX that can be accessed by certain users. This properties file is being used by a number of backend programs. The properties file contain the username and the password of the database as well. How do I design... (1 Reply)
Discussion started by: jackal28
1 Replies

10. Shell Programming and Scripting

find the highest number in the file

Hi, I have a file a.txt and it has values in it Eg :- I need to read through the file and find the number that is the greatest in them all. Can any one assit me on this. Thanks (30 Replies)
Discussion started by: systemali
30 Replies
Login or Register to Ask a Question