Visit Our UNIX and Linux User Community


Help with matching pattern inside a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with matching pattern inside a file
# 1  
Old 06-02-2011
Help with matching pattern inside a file

I have a huge file that has roughly 30304 lines. I need to extract specific info from that file. For example,
Code:
Box 1 > *aaaaaaaajjjj*
> hbbvjvj
> jdnnfddllll
> *dgdfhfekwjh*
Box 2 > *aaaaaaa'aj'jjj*
> dse hkjuejef bfdw
> dyeee
> dsewq
> *dgdfhfekwjh*
>feweiuei
Box 3 > *aaaa"aaaaj"jjj*
> fuiurhir
.
.
.
Box 100 >  *aaaa"'aaaajjjj*
> hhdfwiiji
>*dgdfhfekwjh*
>ggjhfhf

I hope you got the idea. I need to scan through the file and select the exact lines/rows that match my pattern and assign all these patterns to a specific Box and at the same time discarding everything that does not match my pattern.

Can someone help me out with this reqquest?

Last edited by pludi; 06-02-2011 at 03:10 PM..
# 2  
Old 06-02-2011
What pattern you are looking in the test data you provided?

a simple "grep" will work for you.
# 3  
Old 06-03-2011
Sorry, I did a bad job explaining the issue. Let me try again:
I have a list of files, each one ending with (.log). For example
file1.log
file2.log
file3.log
.
.
.
file100.log

Within each file,I want to search for particular fields. However, I need the output data to be concatenated into one single file and at the same time I want to be able to know which data belongs to file1.log, file2.log and so forth.
I wrote this script:
Code:
cat *.log > log1
while read line
do
awk  'NR==2{print; exit}' log1 >> /home/usr/Folder/file1
awk  'NR==5{print; exit}' log1 >> /home/usr/Folder/file2
awk  'NR==54{print; exit}' log1 >> /home/usr/Folder/file3
awk  'NR==16{print; exit}' log1 >> /home/usr/Folder/file4
awk  'NR==37{print; exit}' log1 >> /home/usr/Folder/file5
awk  'NR==69{print; exit}' log1 >> /home/usr/Folder/file6
awk  'NR==100{print; exit}' log1 >> /home/usr/Folder/file7
cat file1 file2 file3 file4 file5 file6 file7 > file.cvs
done<log1

and the output of the script is as follows:
HTML Code:
file1.log> I am okay today
file2.log> I am okay today
file3.log> I am okay today
file4.log> I am okay today
$ wow the weather is nice outside
$ wow the weather is bad outside
$ wow the weather is terrible
$ wow the weather sucks
I like to eat
go to the movies
play golf
read good books
$ Come back
$ Come back
$ Come back
$ Come back
unix is fun
what a mess
telecom
wireless
$ their history
$ their history
$ their history
$ their history
good job
excellent
fantastic
job well done
Rather, I want the output to show the final data in the following format:

HTML Code:
file1.log> I am okay today
$ wow the weather is nice outside
$ I like to eat
$ Come back
$ unix is fun
$ their history
$ good job
file2.log> I am okay today
$ wow the weather is bad outside
$ go to the movies
$ Come back
$ what a mess
$ their history
$ excellent
file3.log> I am okay today
$ wow the weather is terrible
$ play golf
$ telecom
$ their history
$ fantastic
$ Come back

file4.log> I am okay today
$ wow the weather sucks
$ read good books
$ Come back
$ wireless
$ their history
$ job well done
I cannot quite figure out how to do that yet. Any help is appreciated.

Thanks!
# 4  
Old 06-03-2011
Can you please post a sample of data from file1.log , file2.log and the outcome you are expecting and based on what you are doing the search.I'm not able to get how the script you did is OK.RegardsRavi
# 5  
Old 06-03-2011
What are these fields you want to search for?
Are they located in a file?
# 6  
Old 06-03-2011
Yes, each file contains a number of lines from which I need to get the fields. All I want to do is to tell the script to go to each file and grab specific fields and print the result into one single file. But when printing the result, I need to be able to know which data belongs to what file.
Note: I did not mention that earlier, each file has one header inside that particular file that identifies it. For example, file1.log would have for example A that tells me this is file A when I look at the data; thus it would be be headed by A followed the fields; file2.log would be headed by B followed by the fields, file21.log would be headed by U followed by the fields.

I hope that helps.

It it does not, how do I move line around. Let's say that I have:
1) A> reading is fun
2) B> hair spray
3) C> travel abroad
4) A>the sky is blue
5) B> train station
6) C>cup of water
Assuming there are thousands of lines, how can I write a script that looks at the whole data and prints all the lines related to A consecutively and then prints everything from B consecutively and goes down the line until the end of the file.
Thanks!
# 7  
Old 06-03-2011
Ernst, you are still not clear on your requirements.

You are not making it any easy for the members to help you.

Please specify:
1) Sample of input data.
2) Sample of strings you want to search for in the input data.
3) Describe in plain English as much details as possible what you want to do.
4) Display the expected output based on the sample data.

Previous Thread | Next Thread
Test Your Knowledge in Computers #202
Difficulty: Easy
At compile time, the interpreter parses Perl code into a recursive flat-space module.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy pattern inside the file

Hi all, I have files and have a missing record. I need copy the existing record and mark those values up. For example in the below file 11048 is missing. I need to copy 22001 and copy those create the values for 11048. I have 120 set of files and I need to do that on all files. Note the... (8 Replies)
Discussion started by: arunkumar_mca
8 Replies

2. Shell Programming and Scripting

Big pattern file matching within another pattern file in awk or shell

Hi I need to do a patten match between files . I am new to shell scripting and have come up with this so far. It take 50 seconds to process files of 2mb size . I need to tune this code as file size will be around 50mb and need to save time. Main issue is that I need to search the pattern from... (2 Replies)
Discussion started by: nitin_daharwal
2 Replies

3. Shell Programming and Scripting

Take only lastest file matching the pattern

Hi All , I have some fille names in a file which are sorted according to date. for example file1_123.log 23 Jul 0Kb file2_123.log 22 Jul 2Kb file3_123.log 20 Jul 0Kb file1_456.log 24 Jul 2Kb file2_678.log 22 Jul 0Kb file2_678.log 21 Jul 2Kb here 123 is a... (2 Replies)
Discussion started by: LoneRanger
2 Replies

4. Shell Programming and Scripting

Finding the pattern and replacing the pattern inside the file

i have little challenge, help me out.i have a file where i have a value declared and and i have to replace the value when called. for example i have the value for abc and ccc. now i have to substitute the value of value abc and ccc in the place of them. Input File: go to &abc=ddd; if... (16 Replies)
Discussion started by: saaisiva
16 Replies

5. Shell Programming and Scripting

Renumber position 88-94 inside all files matching criteria inside folder

There are 4 files inside one folder matching criteria i.e. File name = ABCJmdmfbsjopXXXXXXX_mm-dd-yyyy_XXX.data Here is the Code which find the files matching criteria:- TS=`date +"%m-%d-%Y"`| for fname in `find . -name "ABCJmdmfbsjop???????_${TS}*.data"` do # Matching File Processing Code.... (1 Reply)
Discussion started by: lancesunny
1 Replies

6. Shell Programming and Scripting

how to find the pattern inside the file and replace it

hello everybody, I have a group of file eg- sample1 sample2 sample3 sample4 each file contain this :- cat sample1 SEQ_NUM,1,UPESI1 My requirement is to change the value-UPESI1 to UPE10 in file which contain this pattern -UPESI1. any help is appreciated. (2 Replies)
Discussion started by: abhigrkist
2 Replies

7. Shell Programming and Scripting

Matching using Regex inside a file

I need scan through some files, then open the file one by one and scan inside the file using perl to see if it contain a start tag and end tag which the end tag is the mirror image of the start tag, the start tag and end tag only have 5 char. And inside the file there is "http://". It is just a... (5 Replies)
Discussion started by: blueblur
5 Replies

8. Programming

File Pattern Matching C++

Hi, I have large files with fixed length fields or fields seperated by delimeter. I would like to do validation on some or all fields to check for numeric or date or characters etc.. I would like to write this in C++. Please let me know if any one have any ideas on this. Thanks for all... (2 Replies)
Discussion started by: rameshmelam
2 Replies

9. Shell Programming and Scripting

Pattern matching for file

Hi All, I'm new to perl, My requirement is to check if particular file exists. e.g. filename.txt, filename1.txt, filename2.txt etc I tried the below code:- my $var1 = "filename.txt" if ( -e ($var1 = ~ /file\w/)) { print "File found \n"; } else { print "File not found \n"; } ... (0 Replies)
Discussion started by: doitnow
0 Replies

10. Shell Programming and Scripting

How to search a pattern inside a zipped file ie (.gz file) with out unzipping it

How to search a pattern inside a zipped file ie (.gz file) with out unzipping it? using grep command.. Bit urgent.. pls..help me (2 Replies)
Discussion started by: senraj01
2 Replies

Featured Tech Videos