Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Script to read file and extract data by matching pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to read file and extract data by matching pattern
# 8  
Old 06-02-2011
Thanks pravin the script works fine.. thank you so much
# 9  
Old 06-03-2011
Hey pravin.. sorry to get back and open the old thread.. I had one more requirement on the same topic.
i forgot to mention that i need to recursively find the dependent files.
for eg: file1 line 2 - zzzz has oooo as dependent file. now oooo might have in turn dependent files ( its not mentioned in example), so i need these files as well and this goes on till the last level. if the file has no dependent files then it will have its own name ( like uuuu:uuuu )
Another case could be: zzzz has xxxx as dependent file. but the 1st line has solved this dependency
please let me know If I'm confusing you a lot, I'll try to frame my question in a better way.
I would appreciate if anyone else could help me on this as well.

---------- Post updated at 05:31 PM ---------- Previous update was at 05:22 PM ----------

for example
-------------------------
file1
Code:
A:A,B,C,D
B:B,E,F
E:E
F:F.G,H
G:G
H:H
I:I
J:J

-------------------------
file2
Code:
/data/testing2/A
/data/test/B

------------------------
Now the script should list ( with path of course)
Code:
/data/testing2/A
/data/test/B
/data/testg2/C
/data/testing3/D
/data/testing1/E
/data/testing/F
/data/testing/G
/data/test2/H

Last edited by Franklin52; 06-03-2011 at 09:22 AM.. Reason: Please use code tags for code and data samples
# 10  
Old 06-04-2011
I hope this will resolve the issue.
Code:
#!/usr/bin/perl

sub seen_files {
        $filename=shift;
        @seenfld=split(" ",$seen{$filename});
        for($k=1;$k<=$#seenfld;$k++){
                @res=grep(/$seenfld[$k]/,@old_array);
                unless (scalar @res > 0 ) {
                        system("find /data -name $seenfld[$k]");
                        @old_array=(@old_array,$seenfld[$k]);
                }
        }
}


($file1,$file2)=($ARGV[0],$ARGV[1]);
open (FH1,"<","$file1") or die "Fail- $!\n";
open (FH2,"<","$file2") or die "Fail- $!\n";

while (<FH2>) {
        chomp;
        @flds=split(/\//);
        $lookup{$flds[$#flds]}++;
}

open (FH3,"<","$file1") or die "Fail- $!\n";
while (<FH3>) {
        chomp;
        if(/^(.+?):/) {
                @fld=split(/,/,substr($_,index($_,":")+1));
                $seen{$1}="@fld";
        }
}

while (<FH1>) {
chomp;
if(/^(.+?):/) {
        if ( exists $lookup{$1}) {
                @flds=split(/,/,substr($_,index($_,":")+1));
                for($i=0;$i<=$#flds;$i++) {
                @res=grep(/$flds[$i]/,@old_array);
                unless (scalar @res > 0 ) {
                        system("find /data -name $flds[$i]");
                        @old_array=(@old_array,$flds[$i]);
                        if (exists $seen{$flds[$i]} &&  $i > 0 ) {
                                seen_files($flds[$i]);
                                @seenfld_old=@seenfld;
                                for($p=1;$p<=$#seenfld_old;$p++){seen_files($seenfld_old[$p]);}
                        }
                        }
                   }
           }

        }
}

close(FH1);
close(FH2);
close(FH3);

# 11  
Old 06-06-2011
Hello Pravin, thanks the script is working perfectly fine. Could you please explain the script to me so that i can add error handling.
# 12  
Old 06-07-2011
Hello All,
I have a slight modification in my input files and need your help again. The logic of the script remains the same except that the input file1 and file2 are

File1:
Code:
Testing~xxxx~null:Testing~xxxx~null,Testing1~abcd~null,test~yyyy~null
Testing1~abcd~Null:Testing1~abcd~null,test~oooo~null,test1~zzzz~null
test1~zzzz:test1~zzzz~null,test~1234~null

File2:
Code:
Testing~xxxx~Null
test1~zzzz~null
test~oooo~
/data/Testing1/abcd~null
/data/test/yyyy.xml~null

Here the first word (test*) in file1 and file2 is the directory in which the file exist. eg: file xxxx is under /data/Testing

so i want to modify the script to

1. search file in specific directory.
2. remove ~null, ~ which is junk ( some time the "N" is caps ~Null or some time it is just ~) in both files.
3. file2 is mix and match of directory structure. first 3 lines have only the directory in which the file exist where as the last line has complete path with or without extension file extension. But the file names are unique so we can eliminate file extension.
i understand this is very confusing but any help would be appreciated.
Last edited by Franklin52; 06-07-2011 at 11:07 AM.. Reason: Please use code tags
# 13  
Old 06-07-2011
Can anyone help me here..

I am new in learning unix and finding some difficulty in preparing a small shell script. I want script to sort all the files given by user as input (either the exact full name of the file or say the files matching the criteria like files having names ZEB*). Then script will find all those files based on matching criteria and append the extension .sort to the sorted version of the file whilst retaining the original version of the file. Script should not sort files in sub-directories, only the top level directory it is run from.
This piece of code line is working for me but it does sort all the files present in that path/location.
Code:
find * -prune -type f |grep -v .sort| while read file
do
  sort "$file" > "$file".sort
done

I want some use friendly shell script which ask user to enter the path location and enter the file name and work only on those files enter by user as input.
Last edited by Franklin52; 06-08-2011 at 04:00 AM.. Reason: Code tags
# 14  
Old 06-07-2011
Quote:
Originally Posted by pankaj80
Can anyone help me here..

I am new in learning unix and finding some difficulty in preparing a small shell script. I want script to sort all the files given by user as input (either the exact full name of the file or say the files matching the criteria like files having names ZEB*). Then script will find all those files based on matching criteria and append the extension .sort to the sorted version of the file whilst retaining the original version of the file. Script should not sort files in sub-directories, only the top level directory it is run from.
This piece of code line is working for me but it does sort all the files present in that path/location.
find * -prune -type f |grep -v .sort| while read file
do
sort "$file" > "$file".sort
done
I want some use friendly shell script which ask user to enter the path location and enter the file name and work only on those files enter by user as input.

If you have new question, please post it in a separate thread.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies

2. Shell Programming and Scripting

Extract range from config file matching pattern

I have config file like this: server_name xx opt1 opt2 opt3 suboptions1 #suboptions - disabled suboptions2 pattern suboptions3 server_name yy opt1 opt2 opt3 suboptions1 pattern #suboptions - disabled suboptions2 So basically I want to extract the server... (1 Reply)
Discussion started by: nemesis911
1 Replies

3. Shell Programming and Scripting

IN Between Data after matching the Pattern

HI , I WANT TO RETRIVE IN BETWEEN DATA FROM PARENTHESIS AND I AM GETTING ERRORS WHILE RUN THE AWK.I HAVE 2 FILES AND WANT TO PROCESS 1ST FILE PATTERN TO 2ND FILE AND WRITES INTO OUTPUT FILE.THIS TIME I AM PUTTING WHERE EXACTLY I AM GETTING ERRORS.SO PLEASE HELP. PATTERN_FILE.TXT --------------... (1 Reply)
Discussion started by: andrew_11
1 Replies

4. UNIX for Beginners Questions & Answers

Shell - Read a text file with two words and extract data

hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line. Example that worked for me with text file... (5 Replies)
Discussion started by: dandaryll
5 Replies

5. Shell Programming and Scripting

Matching and extract data from a file

Gents, Matching columns 1-19 in file1 and 20-38 in file 2, I would like to extract the data in the same order of file2. file1 X 7494 11511 44149.00 48617.002 1 4321 44148.00 48198.00 49060.001 X 7494 11511 44149.00 48617.002 433 8641 44160.00 48198.00 49060.001 ... (10 Replies)
Discussion started by: jiam912
10 Replies

6. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

7. Shell Programming and Scripting

Want to read data from a file name.txt and search it in another file and then matching...

Hi Frnds... I have an input file name.txt and another file named as source.. name.txt is having only one column and source is having around 25 columns...i need to read from name.txt line by line and search it in source file and then save the result in results file.. I have a rough idea about the... (15 Replies)
Discussion started by: ektubbe
15 Replies

8. Shell Programming and Scripting

Removing data with pattern matching

I have the following: HH:MM:SS I want to use either % or # sign to remove :SS can somebody please provide me an example. I know how to do this in awk, but awk is too much overhead for something this simple since I will be doing this in a loop a lot of times. Thanks in advance to all... (2 Replies)
Discussion started by: BeefStu
2 Replies

9. Shell Programming and Scripting

help needed .. Unable to write the data to new file after matching the pattern

Hi, i am pretty new to Unix environment ..... Can i get some help from any of you guyz on writing Unix script. my requirement is like reading a csv file, finding a specific pattern in the lines and repalce the string with new string and write it to another file. My file is file ABC123.dat... (3 Replies)
Discussion started by: prashant_jsw
3 Replies

10. Shell Programming and Scripting

Script to find file name for non matching pattern

Hi, I want to list only the file names which do not contain a specific keyword or search string. OS: Solaris Also is there any way ; through the same script I can save the output of search to a CSV (comma seperated) so that the file can be used for inventory purpose. Any assistance will... (5 Replies)
Discussion started by: sujoy101
5 Replies
Login or Register to Ask a Question