extracting information from multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting extracting information from multiple files
# 1  
Old 11-06-2011
Bug extracting information from multiple files

Hello there,

I am trying to extract (string) information ( a list words) from 4 files and then put the results into 1 file. Currently I am doing this using grep -f list.txt file1 . and repeat the process for the other 3 files. The reasons i am doing that (a) I do know how to code (b) each file has a header that must be include with the string (c) I have to put the word and underneath it the results from 4 files and if there is no results from one and/or all I need to write a specific sentence like there is no results from file No. Below I will explain what I am trying to achieve.

(1) The list.txt
Gene1
Gene2
Gene3

(2) file1.txt
Chromosome Position Genes Mutation
1 251565465 Gene1 T/G
1 215465511 Gene3 G/A

(3) file 2.txt
Chromosome Position Genes Protein
1 251565471 Gene1 Damaged
1 215465614 Gene2 Pass

(4) file 3 and file 4.txt with different results

I want get the results file from the list.txt in a text as the format below
---
Gene1
(add a sentence) "From Mutation point of view" or file name
Chromosome Position Genes Mutation
1 251565465 Gene1 T/G
(add a sentence) "From Protein point of view" or file name
Chromosome Position Genes Protein
1 251565471 Gene1 Damaged
(add a sentence) "There was No results from file 3 and 4" or file name
(a gap, the follow the list with the next word)
Gene2
(add a sentence) "No resultsFrom Mutation point of view" or file name
(add a sentence) "From Protein point of view" or file name
Chromosome Position Genes Protein
1 215465614 Gene2 Pass

---

Any suggestions ?
I searched for grep for multiple files on the forum but it seems that perl will be needed and I am not an expert in coding.
# 2  
Old 11-06-2011
See if this will help...
Code:
awk '
FILENAME != "list"{
        if(filename!=FILENAME){
                filename=FILENAME;
                a[FILENAME"HDR"]=$0;
                next;
        }
        a[FILENAME$3]=$0;
        f[FILENAME]++;
        next;
}
{
        print $0
        for(i=1;i<=length(f);i++){
                print "File : "i
                print a[i"HDR"]
                print a[i$0]?a[i$0]:"No info found"
        }
        print "\n"
}
'  file1.txt file2.txt list

Make sure list is the last file in the argument.

--ahamed

Last edited by ahamed101; 11-06-2011 at 04:27 AM.. Reason: Edited the code to make the filename sorted!
This User Gave Thanks to ahamed101 For This Post:
# 3  
Old 11-06-2011
Thanks a lot ahmed i will give it ago and let you know.

Happy Eid

cheers

---------- Post updated at 04:38 PM ---------- Previous update was at 03:11 AM ----------

Hi again,

I tried the command above, and found there is No Info found for the list file. However, this is not the case when I take a word from the list file and do grep Gene1 file1.txt and I got results (The whole strings that I want). Also I wanted to know if I can put the result into an output file since the results will be too much to copy the displayed resutls. In addition when i get no info found for a specific file e.g
-
Gene000055930
File : 1

No info found
File : 2

No info found
File : 3

No info found
-
Is there a way I can modify the File : 1 into No resutls from "Mutation point of view"

cheers
# 4  
Old 11-07-2011
There was a bug... Corrected it with changes you have requested... Try this...
Code:
#!/bin/bash
awk '
FILENAME != "list"{
        if(filename!=FILENAME){
                filename=FILENAME;
                a[FILENAME"HDR"]=$0;
                next;
        }
        a[FILENAME$3]=$0;
        f[FILENAME]++;
        next;
}
{
        print $0
        for(i in f){
                if(a[i$0]){
                print "From "a[i"HDR"]" point of view"
                print a[i"HDR"]
                print a[i$0]
                }else{
                        print "No results from "a[i"HDR"]" point of view"
                }
        }
        print "\n"
}
'  file*.txt list > output

--ahamed

Last edited by ahamed101; 11-07-2011 at 12:33 PM..
# 5  
Old 11-15-2011
no results

hi Ahmed,

I tried the code got the output file with the headers from each however No results from all the files which when checked manually i got what i query on the list file. strange, i came to believe that only grep is good in these situation ? what do you recon?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting specific files from multiple .tgz files

Hey, I have number of .tgz files and want to extract the file with the ending *results.txt from each one. I have tried for file in *.tgz; do tar --wildcards -zxf $file *results.txt; doneas well as list=$(ls *.tgz) for i in $list; do tar --wildcards -zxvf $i *.results.txt; done... (1 Reply)
Discussion started by: jfern
1 Replies

2. Shell Programming and Scripting

Extracting non multiple files via script

Hi, Can somebody help me? I am testing a demo with the given function PATH525="/uscms/home/emily/READme/extra/data/" TEMP=temp FileName=DataFileName CopyFiles() { # PATHNAME="$paths" ... (9 Replies)
Discussion started by: emily
9 Replies

3. Shell Programming and Scripting

search information in multiple files and save in new files

hi everyone, im stuck in here with shell :) can you help me?? i have a directory with alot files (genbank files ... all ended in .gbk ) more than 1000 for sure ... and i want to read each one of them and search for some information and if i found the right one i save in new file with new... (6 Replies)
Discussion started by: andreia
6 Replies

4. Shell Programming and Scripting

Extracting lines based on identifiers into multiple files respectively

consider the following is the contents of the file cat 11.sql drop procedure if exists hoop1 ; Delimiter $$ CREATE PROCEDURE hoop1(id int) BEGIN END $$ Delimiter ; . . . . drop procedure if exists hoop2; Delimiter $$ CREATE PROCEDURE hoop2(id int) BEGIN END $$ (8 Replies)
Discussion started by: vivek d r
8 Replies

5. Shell Programming and Scripting

Extracting/condensing text from multiple files to multiples files

Hi Everyone, I'm really new to all this so I'm really hoping someone can help. I have a directory with ~1000 lists from which I want to extract lines from and write to new files. For simplicity lets say they are shopping lists and I want to write out the lines corresponding to apples to a new... (2 Replies)
Discussion started by: born2phase
2 Replies

6. Shell Programming and Scripting

Merging information from multiple files to a single file

Hello, I am new to unix and need help with a problem. I have 2 files each containing multiple columns of information ie; File 1 : A B C D E 1 2 3 4 5 File 2 : F G 6 7 I would like to merge the information from File 2 to File 1 so that the data reads as follows; File 1: A... (4 Replies)
Discussion started by: crunchie
4 Replies

7. Shell Programming and Scripting

Extracting columns from multiple files with awk

hi everyone! I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next} {print a, $2}' file1 file2 I added the file3, file4 and... (10 Replies)
Discussion started by: orcaja
10 Replies

8. UNIX for Dummies Questions & Answers

Extracting columns from multiple files with awk

hi everyone! I already posted it in scripts, I'm sorry, it's doubled I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next}... (1 Reply)
Discussion started by: orcaja
1 Replies

9. UNIX for Advanced & Expert Users

Extracting files with multiple links-perl

i want to write a perl script that gets/displays all those files having multiple links (in current directory) (4 Replies)
Discussion started by: guptesanket
4 Replies

10. Shell Programming and Scripting

Extracting information from Config files /text processing

Hello All, This is my first post on this forums, which I consider one of the best of its kind. The reason for my post is that I want to export some information form Nagios configuration files to a DB. I know that there are other tools available to do this, like NDO, monarch, etc... But I want to... (3 Replies)
Discussion started by: oconmx
3 Replies
Login or Register to Ask a Question