Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Filtering multiple files with variables

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filtering multiple files with variables
# 1  
Old 03-06-2009
Filtering multiple files with variables
Hi,
I spend few hours already searching this forum, but cannot find the solution matching exactly my case.
I have multiple log files, I need to filter them so I get info about certain event.

So we have files:
LOGA.txt
LOGB.txt
LOGC.txt
LOGD.txt
LOGE.txt

1. I need to grep lines in LOGA.txt matching pattern (RefId)
Code:
grep -i "-25093524" FILEA.txt

[40287] (03/05 10:46:28.256):{0x1148} {Keat, Eli} RefId=-25093524, EntryId=3,
[40669] (03/05 10:46:28.272):{0xD90} {Keat, Eli} RefId=-25093524, EntryId=3
[30081] (03/05 10:46:28.272):{0xD90} {Keat, Eli}, EntryId=3, RefId=-25093524, TransactionId=-911215752, Tag=68767

2. Now I want to search LOGB.TXT for value Tag= from LOGA.txt

I guess I need to get Tag="number" set as variable ($Tag), that's the problem I don't know how to do it

And I can do the same as in step 1
Code:
grep -i $Tag LOGB.txt

and then search for another patterns in other log files with similar procedure.

I got step 1 someway.
Code:
#!/bin/bash
LOGA=/logs/LOGA.txt
while read line
do
grep -i $1
done < $LOGA

Any help would be appreciated.
# 2  
Old 03-06-2009
Output the first grep to a file. Then from that file, get the TAG information into the shell, like this:
Code:
grep -- "$1" LOGA.txt >/tmp/resA-$$
tag=`sed -n '/Tag=/ s/^.*Tag=//p'  /tmp/resA-$$`

The sed command finds a line with Tag= in it, and then removed everything up to and including the = sign. The rest it prints out, which should be just your number. Now you can go grepping for that likewise.
Code:
grep -- "$tag" LOGB.txt >/tmp/resB-$$
tag=`sed -n '/Tag=/ s/^.*Tag=//p'  /tmp/resB-$$`

and so on.
# 3  
Old 03-09-2009
Thank you otheus,
it was very helpful, now I know how to find and set variable,
but now I am bit struggling to join commands together.
Code:
#!/bin/bash
grep -- $1 *LOGA*
grep -- $1 *LOGA* >/tmp/resA-$$
tag=`sed -n '/Tag=/ s/^.*Tag=//p'  /tmp/resA-$$`
echo "Tag=$tag"
grep -- "$tag" *LOGB*
echo "$tag"

Tag is set correctly, echo displays number for this tag but grep -- "$tag" *LOGB* shows nothing, if I put number instead of variable it works fine.
What do I need to do to make it working? Smilie

Thanks
# 4  
Old 03-09-2009
Insert "set -x" above the first line starting with "echo". The re-run it and you still what command bash actually executed. This is the way to debug scripts in the shell.
# 5  
Old 03-09-2009
otheus,
thanks for quick reply, the line I am looking for is obviously:

Code:
+ grep $'68722\r' BEA1_LOGB_01_20090305_0001.txt

what can it mean this "\r" there?
# 6  
Old 03-09-2009
The \r could indicate that this is a DOS file you are grep'ing. Also, the leading $ makes no sense. Do you know why it's there?
# 7  
Old 03-09-2009
these files are from Windows server, so it could make sense?
I don't know why $ sign is there, command is as I wrote grep -- "$tag" *LOGB*, interesting thing is when I try
grep -- "$tag" *

it shows some results from LOGA.txt but ignores LOGB.txt file
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Generate files and use csv data to replace multiple variables in a template

I have a source csv file consists of first field as variable name, and the rest are site-specific information (converted from excel file, where site -specific values in columns). I am trying to create a file for every site using a template and replace the multiple variables with values from the... (3 Replies)
Discussion started by: apalex
3 Replies

2. UNIX for Dummies Questions & Answers

Read in Multiple log files and output selected variables and values to cvs file

I have several problems with my problems: I hope you can help me. 1) the If else statement I am getting an error message. My syntax must be incorrect because the entire statement is throwing an error. For example in filew.log if these items don't exist Memsize, SASFoundation and also if... (0 Replies)
Discussion started by: dellanicholson
0 Replies

3. Shell Programming and Scripting

Reading multiple values from multiple lines and columns and setting them to unique variables.

Hello, I would like to ask for help with csh script. An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Discussion started by: FMMOLA
7 Replies

4. Shell Programming and Scripting

Running a script with multiple variables like 25 variables.

Hi All, i have a requirement where i have to run a script with at least 25 arguements and position of arguements can also change. the unapropriate way is like below. can we achieve this in more good and precise way?? #!/bin/ksh ##script is sample.ksh age=$1 gender=$2 class=$3 . . .... (3 Replies)
Discussion started by: Lakshman_Gupta
3 Replies

5. Shell Programming and Scripting

Awk/sed : help on:Filtering multiple lines to one:

Experts Good day, I want to filter multiple lines of same error of same day , to only 1 error of each day, the first line from the log. Here is the file: May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out) May 26 11:29:19 cmihpx02 vmunix: NFS... (4 Replies)
Discussion started by: rveri
4 Replies

6. UNIX for Advanced & Expert Users

Renaming Multiple Files that have variables in the name

I am trying to rename multiple files from lower case to upper case, but then I need to change the word SHIP to Ship and CSV to csv. This is a sample of the input file name input_file_ship_1234_20110106.csv I need this to look like this INPUT_FILE_Ship_1234_20110106.csv this is the result when I... (3 Replies)
Discussion started by: cventura
3 Replies

7. Shell Programming and Scripting

Filtering Multiple variables from a single column

Hi, I am currently filtering a file, "BUILD_TIMES", that has multiple column of information in it. An example of the data is as follows; Fri Nov 5 15:31:33 2010 00:28:17 R7_BCGNOFJ_70.68 Fri Nov 5 20:57:41 2010 00:07:21 R7_ADJCEL_80.6 Wed Nov 10 17:33:21 2010 00:01:13 R7_BCTTEST3_80.1X... (7 Replies)
Discussion started by: crunchie
7 Replies

8. Shell Programming and Scripting

Filtering issues with multiple columns in a single file

Hi, I am new to unix and would greatly appreciate some help. I have a file containing multiple colums containing different sets of data e.g. File 1: John Ireland 27_December_69 Mary England 13_March_55 Mike France 02_June_80 I am currently using the awk... (10 Replies)
Discussion started by: crunchie
10 Replies

9. Shell Programming and Scripting

Filtering multiple entries in a file

Hi, I have a requirement to filter multiple entries in a text file. Entries are a pair of 2 lines .Like line1 line2 ....... ........ Here line1, line2 consist of one pair. line1 and line2 contain different data. There can be multiple entries for the same pair. And there can be 'n' such... (2 Replies)
Discussion started by: suman_jakkula
2 Replies
Login or Register to Ask a Question