Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Regex in Shell Scripting to pick values

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Regex in Shell Scripting to pick values
# 15  
Old 01-20-2015
Hello radioactive9,

I have given previous command as per your input and it worked for me as follows.
Input_file:
Code:
cat test18
*IF *VALUE System.Page_Scan_Rate *GE 500
*IF *VALUE System.Page_Scan_Rate *EQ 12000
 *IF *VALUE Unix_Memory.Used_Swap_Space_Pct *GT 90.0
 *IF *VALUE Unix_Memory.Used_Swap_Space_Pct *GT 90.0
 *IF *VALUE KLZ_CPU.Busy_CPU *GE 95.00 *AND *VALUE KLZ_CPU.CPU_ID *EQ Aggregate
 *IF ( ( *VALUE Process.Execution_State *EQ Active *AND *VALUE Process.CPU_Pct *GT 95.00 ) *OR ( *VALUE Process.Execution_State *EQ Runnable *AND *VALUE Process.CPU_Pct *GT 95.00 ) )
 *IF *VALUE NT_Event_Log.Event_ID *EQ 11 *AND *VALUE NT_Event_Log.Source_U *EQ 'Disk'
 *IF ( ( *VALUE NT_Event_Log.Event_ID *EQ 50 *AND *VALUE NT_Event_Log.Source_U *EQ 'Ntfs' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 55 *AND *VALUE NT_Event_Log.Source_U *EQ 'NTFS' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 57 *AND *VALUE NT_Event_Log.Source_U *EQ 'Ftdisk' ) )
 *IF *VALUE KLZ_CPU.Busy_CPU *GT 90.00 *AND *VALUE KLZ_CPU.Busy_CPU *LT 95.00 *AND *VALUE KLZ_CPU.CPU_ID *EQ Aggregate
 *IF *VALUE NT_System.%_Total_Processor_Time *GE 95
 *IF *VALUE KLZ_VM_Stats.Net_Memory_Used_Pct *GE 85 *AND *VALUE KLZ_VM_Stats.Net_Memory_Used_Pct *LT 90
 *IF ( ( *VALUE NT_Event_Log.Event_ID *EQ 17 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 24 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 25 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 29 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 40960 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 46 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 4748 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) *OR ( *VALUE NT_Event_Log.Event_ID *EQ 50 *AND *VALUE NT_Event_Log.Log_Name_U *EQ 'System' ) )
 *IF *VALUE Linux_Process.Total_Busy_CPU_Pct *GT 95.00 *AND *VALUE Linux_Process.State *EQ Running
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'xinetd' )
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'sendmail' )
 *IF *VALUE Linux_VM_Stats.Swap_Pct_Used *GE 85 *AND *VALUE Linux_VM_Stats.Swap_Pct_Used *LT 90
 *IF *VALUE Disk.Inodes_Used_Percent *GT 85 *AND *VALUE Disk.Inodes_Used_Percent *LE 95
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'ntpd' )
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'nfsd' )
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'syslogd' )
 *IF *VALUE KLZ_Network.Interface_Status *EQ DOWN
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'crond' )
 *IF *VALUE KLZ_VM_Stats.Net_Memory_Used_Pct *GT 90
 *IF *VALUE Disk.Inodes_Used_Percent *GT 95
 *IF *VALUE Linux_Process.State *EQ Zombie *AND *COUNT Linux_Process.State *GT 10
 *IF *VALUE SMP_CPU.CPU_Busy *GT 85 *AND *VALUE SMP_CPU.CPU_Busy *LE 95 *AND *VALUE System.Load_Average_15_Min *GT 3.00 *AND *VALUE SMP_CPU.CPU_ID *EQ aggregate
 *IF *VALUE SMP_CPU.CPU_Busy *GT 85 *AND *VALUE SMP_CPU.CPU_Busy *LE 95 *AND *VALUE System.Load_Average_15_Min *GT 3.00 *AND *VALUE SMP_CPU.CPU_ID *EQ aggregate
 *IF *VALUE KLZ_NFS_Statistics.NFS_Null_Calls *GE 15
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'init' )
 *IF *MISSING Linux_Process.Process_Command_Name_U *EQ ( 'snmpd' )
 *IF *VALUE NT_Memory.Memory_Usage_Percentage *GE 95
 *IF *VALUE SMP_CPU.CPU_Busy *GT 95 *AND *VALUE System.Load_Average_15_Min *GT 5.00 *AND *VALUE SMP_CPU.CPU_ID *EQ aggregate
 *IF *VALUE SMP_CPU.CPU_Busy *GT 95 *AND *VALUE System.Load_Average_15_Min *GT 5.00 *AND *VALUE SMP_CPU.CPU_ID *EQ aggregate
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/cron' )
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/envd' )
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/inetd' )
 *IF *MISSING Process.Command *EQ ( 'init' )
 *IF *MISSING Process.Command *EQ ( 'lvmkd' )
 *IF *MISSING Process.Command *EQ ( '*netfmt' )
 *IF *MISSING Process.Command *EQ ( '*nktl_daemon' )
 *IF *MISSING Process.Command *EQ ( '*ntl_reader' )
 *IF *MISSING Process.Command *EQ ( 'statdaemon' )
 *IF *MISSING Process.Command *EQ ( 'swapper' )
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/syncer' )
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/syslogd' )
 *IF *MISSING Process.Command *EQ ( 'vhand' )
 *IF *VALUE System.Virtual_Memory_Percent_Used *GT 85.0 *AND *VALUE System.Virtual_Memory_Percent_Used *LT 95.0
 *IF *VALUE System.Virtual_Memory_Percent_Used *GT 85.0 *AND *VALUE System.Virtual_Memory_Percent_Used *LT 95.0
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/xntpd' )
 *IF *MISSING Process.Command *EQ ( '/usr/lib/inet/inetd' )
 *IF *MISSING Process.Command *EQ ( '/sbin/init' )
 *IF *MISSING Process.Command *EQ ( '/usr/sbin/syslogd' )
 *IF *MISSING Process.Command *EQ ( '/usr/lib/inet/xntpd' )
 *IF *VALUE System.Virtual_Memory_Percent_Used *GE 95.0
 *IF *VALUE System.Virtual_Memory_Percent_Used *GE 95.0
 *IF *VALUE System.Page_Scan_Rate *GE 300 *AND *VALUE System.Page_Scan_Rate *LT 500

Code as follows:
Code:
awk  '{match($0,/[a-zA-Z]+\.[a-zA-Z]+_[a-zA-Z]+_[a-zA-Z]+[[:space:]]\*+[(EQ)(LT)(LE)(GT)(GE)]+[[:space:]][0-9]+/);A=substr($0,RSTART,RLENGTH);if(A){print A}}'  test18

Output will be as follows:
Code:
System.Page_Scan_Rate *GE 500
System.Page_Scan_Rate *EQ 12000
Stats.Swap_Pct_Used *GE 85
Disk.Inodes_Used_Percent *GT 85
Disk.Inodes_Used_Percent *GT 95
Statistics.NFS_Null_Calls *GE 15
Memory.Memory_Usage_Percentage *GE 95
System.Page_Scan_Rate *GE 300

Now for your variable formula it is a bit different from input file provided so only my code was not working on this, made a minor change init, hope it may help this time.
Code:
formula="*IF *VALUE SMP_CPU.CPU_Busy *GT 95 *AND *VALUE System.Load_Average_15_Min *GT 5.00 *AND *VALUE SMP_CPU.CPU_ID *EQ aggregate"
echo $formula | awk  '{match($0,/[a-zA-Z]+_[a-zA-Z]+\.[a-zA-Z]+_[a-zA-Z]+[[:space:]]\*+[(EQ)(LT)(LE)(GT)(GE)]+[[:space:]][0-9]+/);A=substr($0,RSTART,RLENGTH);if(A){print A}}'

Output will be as follows:
Code:
SMP_CPU.CPU_Busy *GT 95

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
radioactive9
# 16  
Old 01-20-2015
To take account the last case:
Code:
 grep -o "\w*\.\w* \*\(EQ\|LT\|LE\|GT\|GE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\)" file

Regards.
This User Gave Thanks to disedorgue For This Post:
radioactive9
# 17  
Old 01-20-2015
Hi R. Singh

This is sweet Smilie.

Well for that type of formula or any other formula where we have multiple attribute and value we need all of them in separate lines / variables

Code:
formula="*IF *VALUE SMP_CPU.CPU_Busy *GT 95 *AND *VALUE System.Load_Average_15_Min *GT 5.00 *AND *VALUE SMP_CPU.CPU_ID *EQ aggregate"

what we need is

Code:
 
SMP_CPU.CPU_Busy *GT 95 
System.Load_Average_15_Min *GT 5.00 
SMP_CPU.CPU_ID *EQ aggregate

Check this command output

Code:
 
echo $formula | grep -o "\w*\.\w* \*\(EQ\|LT\|LE\|GT\|GE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\)" | tr '\n' ','


This is more like what we are looking comma separated all attributes + values

---------- Post updated at 06:42 AM ---------- Previous update was at 06:09 AM ----------

Quote:
Originally Posted by disedorgue
To take account the last case:
Code:
 grep -o "\w*\.\w* \*\(EQ\|LT\|LE\|GT\|GE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\)" file

Regards.

Hello

Almost ran through a huge environment with the line you provided 99 % success Smilie

Code:
 
formula=" *IF *VALUE NT_System.%_Total_Processor_Time *GE 95 "
formula= " *IF *VALUE MS_Exchange_DB.Log_Record_Stalls/sec *GT 100.00 "
formula= " *IF *VALUE MS_Exchange_Transport_Queues.Active_Non-SMTP_Delivery_Queue_Length *GT 500 *AND *VALUE MS_Exchange_Transport_Queues.Active_Non-SMTP_Delivery_Queue_Length *LE 750 "

Fails

looks like special charecter issue like _ or - or % or / in attribute.attribute Smilie

Regards
# 18  
Old 01-20-2015
It's normal, because '\w' is equivalent as '[[:alnum:]_]', so this below work with your input and should work in other case (not tested for other case):
Code:
grep -o "[^ ]*\.[^ ]* \*\(EQ\|LT\|LE\|GT\|GE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\)" file

I just replace '\w' by '[^ ]'.

Regards.

EDIT: better as:
Code:
grep -o "[^. ]*\.[^. ]* \*\(EQ\|LT\|LE\|GT\|GE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\)" file

Last edited by disedorgue; 01-20-2015 at 08:34 AM..
This User Gave Thanks to disedorgue For This Post:
radioactive9
# 19  
Old 01-20-2015
Hello radioactive9,

Could you please try following codes, I am sure they may help you.
Code1 :
Code:
echo $formula | awk  -F"*AND" '{for(i=1;i<=NF;i++){sub(/^[[:space:]]/,X,$i);match($i,/[a-zA-Z]+_[a-zA-Z]+\.[a-zA-Z]+_[a-zA-Z]+[[:space:]]\*+[(EQ)(LT)(LE)(GT)(GE)].*/);A=substr($i,RSTART,RLENGTH);if(A){print A};match($i,/[a-zA-Z]+\.[a-zA-Z]+_[a-zA-Z]+_[0-9]+\_[a-zA-Z]+[[:space:]]\*+[(EQ)(LT)(LE)(GT)(GE)].*/);A=substr($i,RSTART,RLENGTH);if(A){print A};}}'

Output will be as follows.
Code:
SMP_CPU.CPU_Busy *GT 95
System.Load_Average_15_Min *GT 5.00
SMP_CPU.CPU_ID *EQ aggregate

Code2:
Code:
echo $formula | awk  -F"*AND" '{for(i=1;i<=NF;i++){gsub(/^[[:space:]]/,X,$i);print $i}}'

Output will be as follows.
Code:
*IF *VALUE SMP_CPU.CPU_Busy *GT 95
*VALUE System.Load_Average_15_Min *GT 5.00
*VALUE SMP_CPU.CPU_ID *EQ aggregate

Here we need to keep two things in mind, both codes are looking for string
*AND so it should be there in each line, now code1 is looking for 2 regex values string_string.string_string and string.string_string_digits_string
so if any value doesn't satisfy that it will not print anything,
code2 is simple looking for string *AND and printing the fields not looking for any pattern just simply prinintg fields, now according to your post#1 what I understood is you are looking for a single match and pattern so I suggested like
that let's see if these codes may help you or not, finger crossed.


Thanks,
R. Singh
Last edited by RavinderSingh13; 01-20-2015 at 08:47 AM..
This User Gave Thanks to RavinderSingh13 For This Post:
radioactive9
# 20  
Old 01-22-2015
Hello

Both codes do help. But AND is something not common across all definations. Sometimes it might have a OR as well.

Infact I need to capture that portion of the string as well.

Something like this
Code:
 
 
formula="*IF ( ( *VALUE NT_System.Operating_System_Version *EQ 4.0 *AND *VALUE NT_System.%_Total_Processor_Time *GE 90 *OR *VALUE NT_System.%_Total_Processor_Time *LT 95 ) *AND ( *VALUE NT_System.Operating_System_Version *GE 5.0 *AND *VALUE NT_Processor.%_Processor_Time *LT 95 *AND *VALUE NT_Processor.Processor *EQ '_Total' *AND *VALUE NT_Processor.%_Processor_Time *GE 90 ) )"


The output for it should be

NT_System.Operating_System_Version *EQ 4.0 *AND
NT_System.%_Total_Processor_Time *GE 90 *OR
NT_System.%_Total_Processor_Time *LT 95 *AND
NT_System.Operating_System_Version *GE 5.0 *AND
NT_Processor.%_Processor_Time *LT 95 *AND
NT_Processor.Processor *EQ '_Total' *AND
NT_Processor.%_Processor_Time *GE 90


For

Code:
 
formula=" *IF ( ( *VALUE NT_System.Operating_System_Version *EQ 4.0 *AND *VALUE NT_System.%_Total_Processor_Time *GE 95 ) *OR ( *VALUE NT_System.Operating_System_Version *GE 5.0 *AND *VALUE NT_Processor.%_Processor_Time *GE 95 *AND *VALUE NT_Processor.Processor *EQ '_Total' ) )

The output should be

NT_System.Operating_System_Version *EQ 4.0 *AND
NT_System.%_Total_Processor_Time *GE 95 *OR
NT_System.Operating_System_Version *GE 5.0 *AND
NT_Processor.%_Processor_Time *GE 95 *AND
NT_Processor.Processor *EQ '_Total'

---------- Post updated at 05:30 AM ---------- Previous update was at 05:28 AM ----------

grep -o "[^. ]*\.[^. ]* \*\(EQ\|LT\|LE\|GT\|GE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\)"

This is very close just need to be able to capture *AND / *OR at the end of each line
# 21  
Old 01-22-2015
Hi,
Grep is not adapted for this request, because it can not suppress many characters of global regex (in red from example):
Code:
$ echo $formula
*IF ( ( *VALUE NT_System.Operating_System_Version *EQ 4.0 *AND *VALUE NT_System.%_Total_Processor_Time *GE 90 *OR *VALUE NT_System.%_Total_Processor_Time *LT 95 ) *AND ( *VALUE NT_System.Operating_System_Version *GE 5.0 *AND *VALUE NT_Processor.%_Processor_Time *LT 95 *AND *VALUE NT_Processor.Processor *EQ '_Total' *AND *VALUE NT_Processor.%_Processor_Time *GE 90 ) )
$ echo $formula | grep -o "[^. ]*\.[^. ]* \*\(EQ\|LT\|LE\|GT\|GE\|NE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\) *)* *\(\*\AND\|\*\OR\)*"
NT_System.Operating_System_Version *EQ 4.0 *AND
NT_System.%_Total_Processor_Time *GE 90 *OR
NT_System.%_Total_Processor_Time *LT 95 ) *AND
NT_System.Operating_System_Version *GE 5.0 *AND
NT_Processor.%_Processor_Time *LT 95 *AND
NT_Processor.Processor *EQ '_Total' *AND
NT_Processor.%_Processor_Time *GE 90 )

In bash, we can resolve by suppress before, but not the best solution:
Code:
$ echo ${formula//[)(]/} | grep -o "[^. ]*\.[^. ]* \*\(EQ\|LT\|LE\|GT\|GE\|NE\) \([^ ]*\|( *[^ ]* *)\|'[^']*'\) *)* *\(\*\AND\|\*\OR\)*"
NT_System.Operating_System_Version *EQ 4.0 *AND
NT_System.%_Total_Processor_Time *GE 90 *OR
NT_System.%_Total_Processor_Time *LT 95 *AND
NT_System.Operating_System_Version *GE 5.0 *AND
NT_Processor.%_Processor_Time *LT 95 *AND
NT_Processor.Processor *EQ '_Total' *AND
NT_Processor.%_Processor_Time *GE 90

Regards.
This User Gave Thanks to disedorgue For This Post:
radioactive9
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to calculate avg values of csv file using shell scripting .?

hi all i have a reporting work and i want it to be automated using shell scripting kindly let me know how can i make that possibe . eg data are :... (2 Replies)
Discussion started by: Avinash shaw
2 Replies

2. UNIX for Dummies Questions & Answers

How to use square values in Shell Scripting?

:wall: Hi I am a newbie with Shell Scripting who stuck while creating a shell script for Pythagoras theorem.I need to know how to add the squares for the value in shell scripting(for eg: b2 =a2 +c2). Thanks VR (4 Replies)
Discussion started by: VoraciousReader
4 Replies

3. Shell Programming and Scripting

Need to pick max values of the columns

Hi, I have sar disk reports like below sample: 01:01:00 hdisk24 0 0.0 0 0 0.0 0.0 hdisk15 0 0.0 0 3 0.0 5.5 hdisk20 0 0.0 2 1 0.0 1.9 hdisk19 1 ... (3 Replies)
Discussion started by: reddyr
3 Replies

4. UNIX for Dummies Questions & Answers

How to compare to values returned from sql in shell scripting?

hey i am using this code to connect to sql , store the value in variable and then compare it with another variable after some time by executing the same query but the desired result is not coming #!/bin/bash val=$(sqlplus -s rte/rted2@rel76d2 <<ENDOFSQL set heading off set feedback off... (11 Replies)
Discussion started by: ramsavi
11 Replies

5. Shell Programming and Scripting

Assigning array values using awk in shell scripting

hi My script as below #!/bin/ksh for i in `seq 1 7` do a=$(awk '{print $i}' /home/rama/expenese.txt) done for i in `seq 1 7` do echo "${a}" done content of expense.txt is as below 5032 210179 3110 132813874 53488966 11459221 5300794 I want output as... (6 Replies)
Discussion started by: Ramakrishna V
6 Replies

6. Shell Programming and Scripting

mapping of values in shell scripting

sample content of file1: SSTY1 2145228348 652011011715140100000002419005432092074 008801726143662 VDZX01 MIO2 008801726143662 SSRTY 2145228349 ... (3 Replies)
Discussion started by: vsachan
3 Replies

7. Shell Programming and Scripting

Need help to change XML values with shell scripting for Network Simulation

Hello, I don't have experience in this scripting and I need some help to read a value from an XML file and change it with a random number to use in simulator for different network scenarios. </Description><sim_comm_rounds>35</sim_comm_rounds><num_clusters>1</num_clusters><Clocking> I want to... (5 Replies)
Discussion started by: erhanasd
5 Replies

8. Shell Programming and Scripting

How to pick values from column based on key values by usin AWK

Dear Guyz:) I have 2 different input files like this. I would like to pick the values or letters from the inputfile2 based on inputfile1 keys (A,F,N,X,Z). I have done similar task by using awk but in that case the inputfiles are similar like in inputfile2 (all keys in 1st column and values in... (16 Replies)
Discussion started by: repinementer
16 Replies

9. UNIX for Dummies Questions & Answers

Concatenating arrays cell values in shell scripting

Hi All, I want to concatenate the array cell values and form a string.. Is it possible? for ex. I have an array word_array contains d u m b and after concatenating the string shld be 'dumb' thanks (2 Replies)
Discussion started by: mathur
2 Replies

10. Shell Programming and Scripting

Regex to pick up name from the following including carriage return at end of the line

has anyone got any suggestions how i would pick up the string as part of a substitution inclusive of the carriage return. ie i want to pick up <<NAME>> from the PS output but the <<; seems to be on the line before the NAME. Any ideas are appreciated! ... (3 Replies)
Discussion started by: Shakey21
3 Replies
Login or Register to Ask a Question