Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Substitute grep command at run time

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Substitute grep command at run time
# 1  
Old 09-04-2018
Substitute grep command at run time
HI
I am trying to use the following code in the shell script (using grep)

Code:
usage()
{
  echo "Usage: ./$0 <file name> <interval> <pattern>"
} 

METRICS_FILE=$1
INTERVAL=$2
PATTERN="$3" 
..

if [ "${PATTERN}" == "" ]
then
 PATTERN="grep Gx"
fi

	COUNT=`cat ${METRICS_FILE} | "${PATTERN}" |egrep "${start_tim}|${end_tim}" |awk -F"Count=" '{print $2}'|cut -d',' -f1 |perl -lne '$sum += $_ } { print $sum'`

..

The above is working fine in case I do not provide any argument for arg no 3, as in that case the hard-coded string "PATTERN="grep Gx"", is used.

But in case i provide argument 3 as "grep gx|grep -v RAR"

I am getting error.:
Quote:
./rate.sh metrics.csv.20180730_13_00 15 "grep gx|grep -v RAR"
./rate.sh: line 31: grep gx|grep -v RAR: command not found
Please suggest.
# 2  
Old 09-04-2018
Firstly, looks like the end of perl command is missing curly bracers in the end.
You should specify your operating system and shell used.

I cannot replicate behavior you are experiencing neither working or non working example on debian linux, using bash shell.

As for code, well i'm sure it could be written better, if you provide the relevant data to parse, and script expected input/output.

A PATTERN should be a string to match not a grep pipe grep command(s) inside a shell variable.
This is just wrong on so many levels and will not work as you expect or at all.

So my suggestion would be a complete rewrite Smilie

Also, for instance, you can specify a default value using shell builtin.
Code:
PATTERN=${1:-Gx}
echo $PATTERN

Meaning if $1 is not inputted or defined, it will be Gx otherwise it will be what is inputted/defined.

Hope that helps
Regards
Peasant.
# 3  
Old 09-05-2018
In Posix shell, bash, ksh and zsh, splitting the command line into the individual commands, takes place before parameter expansion. Hence, the shell doesn't see the pipe inside your pattern.

BTW, with this type of questions, you should always state which shell you are using.
This User Gave Thanks to rovf For This Post:
asifansari
# 4  
Old 09-05-2018
Its a bash shell.
Any ways to make it work.

------ Post updated at 12:05 PM ------

Thanks for the prompt reply..
As stated the script is working fine if I do not give 3rd argument, this means the syntax which you are pointing is not correct, is not the case.

Here is the complete code:
Code:
#!/bin/bash

usage()
{
  echo "Usage: ./$0 <metrics file name> <interval> <pattern>"
}  

METRICS_FILE=$1
INTERVAL=$2
PATTERN="$3"

if [ "${METRICS_FILE}" == "" ] || [ "${INTERVAL}" == "" ] 
then
	echo "Input not sufficient"
	usage
	exit
fi

if [ "${PATTERN}" == "" ]
then
 PATTERN="grep Gx"
fi

cat ${METRICS_FILE} |cut -d',' -f2,4|uniq > interval.txt
DUR=`expr ${INTERVAL} \* 60`
while read line
do
	
	start_tim=`echo ${line} |cut -d',' -f1`
	end_tim=`echo ${line} |cut -d',' -f2`
	
	COUNT=$(cat ${METRICS_FILE} | ${PATTERN} |egrep "${start_tim}|${end_tim}" |awk -F"Count=" '{print $2}'|cut -d',' -f1 |perl -lne '$sum += $_ } { print $sum')

	if [ "${COUNT}" == "" ]
	then
		continue;
	fi	
	RATE=`awk "BEGIN {print ${COUNT}/${DUR}}"`  	
#	echo "Interval: $line"	
	echo "COUNT: [${COUNT}] RATE: [${RATE}] [${start_tim}] [${end_tim}]"
	echo
done < interval.txt
rm -rf interval.txt 2>/dev/null

Sample Input File: Paste the contents in m.csv.

Code:
Start Time In MS=1532947500004,Start Time Local=Mon Jul 30 12:45:00 CEST 2018,End Time In MS=1532948400003,End Time Local=Mon Jul 30 13:00:00 CEST 2018,Site=site1,Group=Diameter,Application=Gx,Command=RAR,Destination Host=pcrf1.vVOLTE5MIRM.site1,Destination Realm=EPC.MNC010.MCC222.3GPPNETWORK.ORG,Egress Peer Origin Host=csb.vVOLTE5MIRM.site1,Egress Peer Origin Realm=vVOLTE5MIRM.site1,Ingress Peer Origin Host=c11-10-214-191-34-dcc-prf-mas-44,Ingress Peer Origin Realm=vodafone.it,Origin Host=c11-10-214-191-34-dcc-prf-mas-44,Origin Realm=vodafone.it,Outbound Message Processing=No Value,Result=DIAMETER_SUCCESS,Role=Routing Agent,Average Latency=6.621212121212121
Start Time In MS=1532947500004,Start Time Local=Mon Jul 30 12:45:00 CEST 2018,End Time In MS=1532948400003,End Time Local=Mon Jul 30 13:00:00 CEST 2018,Site=site1,Group=Diameter,Application=Gx,Command=CCR,Destination Host=pcrf1.vVOLTE5MIRM.site1,Destination Realm=EPC.MNC010.MCC222.3GPPNETWORK.ORG,Egress Peer Origin Host=csb.vVOLTE5MIRM.site1,Egress Peer Origin Realm=vVOLTE5MIRM.site1,Ingress Peer Origin Host=c11-10-214-191-34-dcc-prf-mas-44,Ingress Peer Origin Realm=vodafone.it,Origin Host=c11-10-214-191-34-dcc-prf-mas-44,Origin Realm=vodafone.it,Outbound Message Processing=No Value,Result=DIAMETER_SUCCESS,Role=Routing Agent,Average Latency=6.621212121212121

And run it as:
Code:
./r.sh m.csv 15
COUNT: [0] RATE: [0] [Start Time Local=Mon Jul 30 12:45:00 CEST 2018] [End Time Local=Mon Jul 30 13:00:00 CEST 2018]

To generate error:
Code:
[root@dsc02-1 vf]# ./r.sh m.csv 15 "grep Gx|grep -v RAR"
grep: RAR: No such file or directory

Moderator's Comments:
Mod Comment Converted QUOTE tags to CODE tags
# 5  
Old 09-05-2018
Maybe something in this way (not tested):

Code:
alias PATTERN="${3:-grep whatever you want here}"

.....

COUNT=$(PATTERN|egrep "${start_tim}|${end_tim}" | awk ... etc etc ...)

# 6  
Old 09-05-2018
I think I have already checked this, here is the execution results:

Code:
[root@dsc02-1 vf]# ./r.sh metrics.csv.20180730_13_00 15 "grep Gx|grep -v RAR"
+ METRICS_FILE=metrics.csv.20180730_13_00
+ INTERVAL=15
+ PATTERN='grep Gx|grep -v RAR'
+ '[' metrics.csv.20180730_13_00 == '' ']'
+ '[' 15 == '' ']'
+ '[' 'grep Gx|grep -v RAR' == '' ']'
+ cat metrics.csv.20180730_13_00
+ cut -d, -f2,4
+ uniq
++ expr 15 '*' 60
+ DUR=900
+ read line
++ echo Start Time Local=Mon Jul 30 12:45:00 CEST 2018,End Time Local=Mon Jul 30 13:00:00 CEST 2018
++ cut -d, -f1
+ start_tim='Start Time Local=Mon Jul 30 12:45:00 CEST 2018'
++ echo Start Time Local=Mon Jul 30 12:45:00 CEST 2018,End Time Local=Mon Jul 30 13:00:00 CEST 2018
++ cut -d, -f2
+ end_tim='End Time Local=Mon Jul 30 13:00:00 CEST 2018'
++ cat metrics.csv.20180730_13_00
++ grep 'Gx|grep' -v RAR
++ egrep 'Start Time Local=Mon Jul 30 12:45:00 CEST 2018|End Time Local=Mon Jul 30 13:00:00 CEST 2018'
++ awk -FCount= '{print $2}'
++ cut -d, -f1
grep: RAR: No such file or directory
++ perl -lne '$sum += $_ } { print $sum'
+ COUNT=
+ '[' '' == '' ']'
+ continue
+ read line
+ rm -rf interval.txt

What I used:
Code:
#!/bin/bash
set -x
usage()
{
  echo "Usage: ./$0 <metrics file name> <interval> <pattern>"
}  

METRICS_FILE=$1
INTERVAL=$2
#PATTERN="$3"
PATTERN="${3:-grep Gx}"

if [ "${METRICS_FILE}" == "" ] || [ "${INTERVAL}" == "" ] 
then
	echo "Input not sufficient"
	usage
	exit
fi

if [ "${PATTERN}" == "" ]
then
 PATTERN="grep Gx"
fi

cat ${METRICS_FILE} |cut -d',' -f2,4|uniq > interval.txt
DUR=`expr ${INTERVAL} \* 60`
while read line
do
	
	start_tim=`echo ${line} |cut -d',' -f1`
	end_tim=`echo ${line} |cut -d',' -f2`
	
	COUNT=$(cat ${METRICS_FILE} | ${PATTERN} |egrep "${start_tim}|${end_tim}" |awk -F"Count=" '{print $2}'|cut -d',' -f1 |perl -lne '$sum += $_ } { print $sum')

	if [ "${COUNT}" == "" ]
	then
		continue;
	fi	
	RATE=`awk "BEGIN {print ${COUNT}/${DUR}}"`  	
#	echo "Interval: $line"	
	echo "COUNT: [${COUNT}] RATE: [${RATE}] [${start_tim}] [${end_tim}]"
	echo
done < interval.txt
rm -rf interval.txt 2>/dev/null

# 7  
Old 09-05-2018
As I already explained, this doesn't work. Why don't you try the code which I have suggested?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Listen sharp time to run a command inside a script

Hello, Just wondered if there is any possibility to run a command at sharp time inside a script in linux. My question is not about crontab Example: #!/bin/bash cd /home/database for i in * do command 1 if time is 19:00, day is Monday then run command2 if time is 20:00, day is... (10 Replies)
Discussion started by: baris35
10 Replies

2. Shell Programming and Scripting

Run a command for specific amount of time with an auto key press

Hi, I have been trying to do a small fun project for myself. I want to run a command for 45 seconds. And to get the final output of this command, the script requires I push the "q" key on my keyboard and then the final output file becomes available. I tried the following script. But it... (12 Replies)
Discussion started by: jacobs.smith
12 Replies

3. Shell Programming and Scripting

Grep and substitute?

I have to parse ASCII files, output the relevant data to a comma-delimited file and load it into a database table. The specs for the file format have been recently updated and one section is causing problems. This is the original layout for that section. ... (2 Replies)
Discussion started by: alan
2 Replies

4. Shell Programming and Scripting

Using grep command to detect presence of script run

i have this line of code on a korn shell script to detect the presence of script run: ISRUNNING=`ps -eaf -o args | grep -i sfs_load_file.ksh | grep -v grep | wc -l` sometimes this returns either 1, 2, or 3. when it returns 2 or 3 that tells us that there are more than 1 script of... (8 Replies)
Discussion started by: wtolentino
8 Replies

5. Shell Programming and Scripting

Time taken to run grep in different OS

Hi , I am greping a keyword in all sql files in Solaris and Linux. Solaris bash-3.00$ time grep -iwc BEN_STARTUP_LERS_TL084701_WHO *.sql Load__v20130719-prod.sql:0 Load__v20130719-prod.sql:0 Load__v20130719-prod.sql:0 Load__v20130719-prod.sql:0 Load__v20130719-prod.sql:0... (4 Replies)
Discussion started by: millan
4 Replies

6. Ubuntu

run multiple command at the same time in one window terminal using multiplexer

Hi, I would like to ask if someone knows or accomplished this task in the terminal multiplexer in a single window with multiple splitted pane: In the script run multiple command at the same time in diff splitted pane or simulatneously. As an example: I would like to run iptraf, iotop, htop,... (2 Replies)
Discussion started by: jao_madn
2 Replies

7. AIX

time of a particular command run

Hello all, I need to find, what time a particular command was run in one of our AIX box. In our environment, we use 'powerbroker' to login as root and there are so many people who use this. I tried history command, which shown me similar to below: 406 ls -l | *user* 407 ls -l... (1 Reply)
Discussion started by: gsabarinath
1 Replies

8. Shell Programming and Scripting

Show date/time with tail|grep command

Hi, I have a log file without date/time, and I want that everytime tail|grep find something it displays the date/time and the line. I have tried something like this command but without any luck to display the date/time: tail -F catalina.out | sed "s/^/`date `/" | egrep ... (6 Replies)
Discussion started by: julugu
6 Replies

9. UNIX for Dummies Questions & Answers

Run Command in Specific Time ...

Guy's I want to make script to run this command solevel every Saturday at 8:00 clock exactly . Can you please help me and teach me how to do this ... (9 Replies)
Discussion started by: IT Helper
9 Replies

10. Shell Programming and Scripting

substitute the grep output

I have a file name called fruits. In this file the prices keep on changing & the order in which fruits are listed keep on changing. $ cat fruits fruitname price/pound redapples 30 grapes 50 oranges 20 $echo $custom_price 35 What I want to do is that if the file "fruits" contains... (1 Reply)
Discussion started by: jasmeet100
1 Replies
Login or Register to Ask a Question