Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

grep variables nested within grep

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting grep variables nested within grep
# 1  
Old 07-29-2010
grep variables nested within grep
Help,

I have a ksh script that has some variables within the grep command, I am then setting another variable that greps the variables that have greps within them.
Code:
cat $WORKINGDIR/meter_list.txt | while read meter
do
  serialnum=$(cat masterlogfile.txt | grep "$meter" | awk '{ print $19 }' | cut -c 2-10 )
  Thread_count=$(cat masterlogfile.txt | grep "$meter" | awk '{ print $4 }')
  Thread_serialnum_time=$(cat masterlogfile.txt | grep "$meter" | awk '{ print $2 }' | cut -c 1-4 )
  intime=$(cat masterlogfile.txt | grep 'Entering callExitsIn 2' | grep `echo $Thread_count` | grep `echo $Thread_serialnum_time` | awk  '{ print $2 }')
  outtime=$(cat masterlogfile.txt | grep 'Leaving callExitsIn 2'| grep `echo $Thread_count` | grep `echo $Thread_serialnum_time` | awk  '{ print $2 }')
  echo "Meter, "$serialnum, "came in on thread, "$Thread_count, "the time in was, "$intime, "and the time out was, "$outtime,  >> $WORKINGDIR/mail_file1.txt
done

when I get to the intime and outtime variables declaration the value of the called variable is not being passed.

I need the output string from the Thread_serial_num and the Thread count to be used in the grep "$Thread-count in the declaration of the last two variables.
I have tried "" , `echo $Thread-count`, forms of \<\>, <>, ${} no matter what I have used the value does not pass into the inner grep. Does any one have a workaround or solution to this.

I am in a pinch and need this by morning.
Last edited by srichard9; 07-29-2010 at 03:41 AM.. Reason: Please use code tags and indent your code
# 2  
Old 07-29-2010
Hi,

welcome to the forum.
Can you show the sample of "meter_list.txt" and "masterlogfile.txt" and the required stuffs?
# 3  
Old 07-29-2010
Some general things to note:
  • There is no need to cat files via pipe into other commands when they can take files as parameter themselves. You save a command and for time critical tasks you might want to spare all out you don't really need.
  • awk can "grep" too. It is one of it's easiest jobs to match patterns. You could write all those greps with one awk.
  • If not enclosed by single quotes, a variable will be substituted by the shell. There is no need to write `echo $VAR` to use the variables content as pattern.

For the problem that the content of your variable is not being passed, you should make sure 1st, that the content of the variable is the one you expect, before you pass it and if the grep/awk can match anything with this pattern.
A simple echo in the line after the declaration of the variable would be sufficient to check it out:

Code:
...
Thread_serialnum_time=...
echo $Thread_serialnum_time
...

Here an example how you could write such a lengthy pipe column instead:
Code:
# original
outtime=$(cat masterlogfile.txt | grep 'Leaving callExitsIn 2'| grep `echo $Thread_count` | grep `echo $Thread_serialnum_time` | awk  '{ print $2 }')

# how it could look like with just 1 command
outtime=$(awk '/Leaving callExitsIn 2/ && /'${Thread_count}'/ && /'${Thread_serialnum_time}'/ {print $2}' masterlogfile.txt)

# 4  
Old 07-29-2010
here is a copy of the masterlog_file I am grepping for the different colums.
Code:
[7/28/10 11:00:00:244 CDT] 00000560 SystemOut     O 28 Jul 2010 11:00:00:244 [INFO] Entering callExitsIn 2
[7/28/10 11:00:00:246 CDT] 00000560 SystemOut     O 28 Jul 2010 11:00:00:246 [INFO] Leaving callExitsIn 2
[7/28/10 11:00:00:808 CDT] 0000009f SystemOut     O 28 Jul 2010 11:00:00:808 [INFO] Entering callExitsIn 2
[7/28/10 11:00:00:811 CDT] 0000009f SystemOut     O 28 Jul 2010 11:00:00:811 [INFO] Using Where(MANUFACTURER = 'LG' AND SERIALNUM = '090678505')
[7/28/10 9:00:01:119 CDT] 00000037 SystemOut     O 28 Jul 2010 09:00:01:119 [INFO] Entering callExitsIn 2
[7/28/10 9:00:01:122 CDT] 00000037 SystemOut     O 28 Jul 2010 09:00:01:122 [INFO] Using Where(MANUFACTURER = 'LG' AND SERIALNUM = '105170900')
[7/28/10 11:00:01:194 CDT] 000000a0 SystemOut     O 28 Jul 2010 11:00:01:194 [INFO] Entering callExitsIn 2
[7/28/10 15:00:01:501 CDT] 0000009f SystemOut     O 28 Jul 2010 15:00:01:501 [INFO] Entering callExitsIn 2
[7/28/10 15:00:01:503 CDT] 0000009f SystemOut     O 28 Jul 2010 15:00:01:503 [INFO] Leaving callExitsIn 2
[7/28/10 9:00:01:541 CDT] 00000037 SystemOut     O 28 Jul 2010 09:00:01:541 [INFO] Leaving callExitsIn 2
[7/28/10 11:00:01:842 CDT] 000000a0 SystemOut     O 28 Jul 2010 11:00:01:842 [INFO] Leaving callExitsIn 2
[7/28/10 11:00:02:013 CDT] 0000009f SystemOut     O 28 Jul 2010 11:00:02:012 [INFO] Leaving callExitsIn 2
[7/28/10 11:00:02:429 CDT] 00000034 SystemOut     O 28 Jul 2010 11:00:02:429 [INFO] Entering callExitsIn 2
[7/28/10 11:00:02:431 CDT] 00000034 SystemOut     O 28 Jul 2010 11:00:02:431 [INFO] Leaving callExitsIn 2
[7/28/10 11:00:02:946 CDT] 0000003d SystemOut     O 28 Jul 2010 11:00:02:946 [INFO] Entering callExitsIn 2
[7/28/10 11:00:02:948 CDT] 0000003d SystemOut     O 28 Jul 2010 11:00:02:948 [INFO] Leaving callExitsIn 2
[7/28/10 8:00:02:953 CDT] 00000038 SystemOut     O 28 Jul 2010 08:00:02:953 [INFO] Entering callExitsIn 2
[7/28/10 8:00:03:396 CDT] 00000038 SystemOut     O 28 Jul 2010 08:00:03:396 [INFO] Leaving callExitsIn 2

the 4th column is the thread count, the second field is the time, and the last column with the SERIALNUM text is the serial number. I need the thread number and the time stamp from the row with the SERIALNUM text so that I can regrep to get the thread that goes with that row and associate it with a row above that, that has the first Entering callExitsIn 2 text associated with it. I can not get the prior associated thread entry until I have the entry that lists the serial number.
the output is giving all entries of the time no matter what the thread is:

output:
Code:
 
Meter, 070000079, came in on thread, 00000102, the time in was, 13:11:43:185 11:46:18:541 11:47:59:722 11:49:20:124, and the time out was, 13:11:40:739 13:11:44:024 11:46:19:
102 11:48:00:223 11:49:20:461,
Meter, 105456605, came in on thread, 00000038, the time in was, 13:11:42:306 15:11:43:520 13:11:44:565 11:46:29:764 11:46:50:928 11:47:38:335 11:47:56:521 11:48:12:733 11:48:
37:535 11:49:07:120, and the time out was, 15:11:43:884 13:11:44:029 13:11:45:131 11:46:30:082 11:46:51:458 11:47:38:887 11:47:57:092 11:48:13:202 11:48:38:037 11:49:07:881,
Meter, 105454746, came in on thread, 00000034, the time in was, 12:01:05:922 12:03:20:540 12:03:39:736, and the time out was, 12:01:06:295 12:03:20:868 12:03:40:361,
Meter, 056452251, came in on thread, 000000a0, the time in was, 0:16:09:121 10:16:30:326 10:50:16:339 9:50:16:919, and the time out was, 9:00:16:386 0:16:10:969 10:16:31:062
10:50:16:973,
Meter, 103636973, came in on thread, 00000038, the time in was, 13:10:01:328 13:10:39:961 13:10:48:721 13:10:52:072 13:10:58:711 13:11:07:375 13:11:15:522 13:11:21:916 13:11:
25:926 13:11:34:916 13:11:37:931 13:11:42:306 13:11:44:565 13:11:51:525 13:12:02:941 13:12:27:525 13:12:38:339 13:13:13:730 15:13:17:619 13:13:20:523 13:13:26:522 13:13:36:33
5 13:14:14:117 13:14:36:301 13:14:44:473 13:14:56:679 13:15:32:644 13:15:59:947 13:17:01:929 13:17:02:146 13:17:13:098 13:19:49:119, and the time out was, 13:10:01:734 13:10:
40:465 13:10:48:723 13:10:52:404 13:10:59:040 13:11:07:378 13:11:15:524 13:11:22:412 13:11:26:335 13:11:35:286 13:11:38:534 13:11:44:029 13:11:45:131 13:11:52:096 13:12:03:55
2 13:12:28:058 13:12:38:847 13:13:13:734 15:13:17:622 13:13:21:223 13:13:27:769 13:13:36:832 13:14:14:552 13:14:36:303 13:14:44:791 13:14:56:680 13:15:33:241 13:16:00:344 13:
17:02:148 13:17:02:645 13:17:13:439 13:19:49:904 8:40:13:112,
~

I should have only one in time and one out time with each thread for that serial number. the input file for the meter_list is simply a list of serial numbers that match the last column in the row with the SERIALNUM in it

Moderator's Comments:
Mod Comment You were already asked to use code tags, please do so, ty.
Last edited by zaxxon; 07-29-2010 at 05:04 AM..
# 5  
Old 07-29-2010
Did you check out what I wrote in my post? Also please use code tags as described in the PM, ty.
# 6  
Old 07-29-2010
I tried everything you suggested. I even put in the echos. Everything is as it should be until I get to the intime and outtime. these two variables show up as either nul or blank. Every other variable has the correct values in them. I tried using awk instead of grep and got the exact same output.... no difference as far as the output goes, but it does look better in awk than it did in grep. is there a next step?
# 7  
Old 07-29-2010
Add following lines in your code:
Code:
...
Thread_serialnum_time=$(cat masterlogfile.txt | grep "$meter" | awk '{ print $2 }' | cut -c 1-4 )
echo "Thread_count="$Thread_count
echo "Thread_serialnum_time="$Thread_serialnum_time
...

Both variables should have some value at this point. If not, they will neither have any value in the following lines.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using $variables in grep

Hi, I'm currently trying to use variables in grep on my script. Printing the variable via echo works fine. Also, if I hard coded the date of the appointment it works just fine. But, if I try to use the $DATE as an argument in grep it doesn't do anything. #!/bin/bash DATE=${2}/${3}/${1} ... (6 Replies)
Discussion started by: nuclearpenguin
6 Replies

2. Shell Programming and Scripting

Inconsistent `ps -eaf -o args | grep -i sfs_pcard_load_file.ksh | grep -v grep | wc -l`

i have this line of code that looks for the same file if it is currently running and returns the count. `ps -eaf -o args | grep -i sfs_pcard_load_file.ksh | grep -v grep | wc -l` basically it is assigned to a variable ISRUNNING=`ps -eaf -o args | grep -i sfs_pcard_load_file.ksh |... (6 Replies)
Discussion started by: wtolentino
6 Replies

3. Homework & Coursework Questions

Grep, Variables, IF statements... Fun

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: The issue I am having is part of a validation problem. My script will validate 3 or 4 parameters entered by the... (4 Replies)
Discussion started by: JonLaberge
4 Replies

4. Shell Programming and Scripting

Using awk or grep with Variables in a script

Good day Geeks, Am having an issue with using variables in a rather simple script, the script is as follows: #!/bin/bash ### Script written by Adigun Gbenga ### Date: April 28, 2012 array=( 1 2 3 4 5 29 7 8 9... (6 Replies)
Discussion started by: infinitydon
6 Replies

5. Shell Programming and Scripting

grep and variables ?

I would like to know if grep can extract the following requirement. I have the folllowing piece of SQL in a file and need to grep the FROM part. mp db-ter-fast-export C100_Input_Target_Table__table_ "${DB}"'/teradata.dbc' -select 'SELECT UPPER(trim(proj_id)) as proj_id, latest_job_id,... (9 Replies)
Discussion started by: anduzzi
9 Replies

6. Programming

Grep with C variables.

Hi all, I have a problem, I need a grep -v but, the patterns of the grep should be C variables. Example: char var1="h"; char var2="o"; char var3="d"; system("grep -v \"var1 var2 var3\" file.txt"); ---- ---- ---- I try it but... Can u help me? Thanks beforehand P.S. Sorry for... (4 Replies)
Discussion started by: AbelBs
4 Replies

7. Shell Programming and Scripting

How to use variables/array in grep command

Hi, I have a reqmt as i have some values in array and I want to search each value in a file by grep command. Here goes my scripting: #!/bin/ksh set -A ArrayA CENTER LEFT RIGHT echo "ArrayA contains: ${ArrayAİ*¨}" grep -e "${ArrayAİ*¨}" filename.txt The above grep is working for... (4 Replies)
Discussion started by: prashant43
4 Replies

8. Shell Programming and Scripting

MEM=`ps v $PPID| grep -i db2 | grep -v grep| awk '{ if ( $7 ~ " " ) { print 0 } else

Hi Guys, I need to set the value of $7 to zero in case $7 is NULL. I've tried the below command but doesn't work. Any ideas. thanks guys. MEM=`ps v $PPID| grep -i db2 | grep -v grep| awk '{ if ( $7 ~ " " ) { print 0 } else { print $7}}' ` Harby. (4 Replies)
Discussion started by: hariza
4 Replies

9. Shell Programming and Scripting

awk as grep with variables

i have such awk working fine but how to use variable instead of strings awk '/asdasd.*asda.*asdasd/' file2.txt This is not working: awk '/${a}.*${b}.*${c}/' file2.txt Thanks & regards Peter (7 Replies)
Discussion started by: pp56825
7 Replies

10. Shell Programming and Scripting

Using grep with variables

Being new to shell scripting I hope this question isn't too elementary but here goes. I run a grep statement - grep 'sqr' sqrmodule.par and the grep statement returns correctly the information that I'm looking for. Now I want to take the output from the grep statement and load it into a... (2 Replies)
Discussion started by: gettingstarted
2 Replies
Login or Register to Ask a Question