Finding data in large no. of files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding data in large no. of files
# 1  
Old 07-09-2012
Error Finding data in large no. of files

I need to find some data in a large no. of files. The data is in the following format :
Code:
VALUE A       VALUE B    VALUE C      VALUE D
10                   4                    65               1
12                   4.5                 65.5            2
10.75               5.1                 87               3
9.8                  4                    67               4


All the files have data is the same format (above). I need to write a script that copies those files (to a subdirectory, which also the script should create) that have ANY row satisfying the search criteria that : 10.5<VALUE A<11.5 && 4.5<VALUE B<5.5 && 80<VALUE C<90, and then also displays the VALUE D for those particular rows in the selected file which fulfill the above criteria.
The files are in "bin/models" which has two subdirectories,"model 1"and "model 2", each of which contain 10 data files. The files end in ".track". The new subdirectories are to be named "new_sub", and are to be created both in "model 1" and "model 2".

Thanks a TON in advance; I really need to know this one quick for a project!!
Moderator's Comments:
Mod Comment please use code tags to format data

Last edited by jim mcnamara; 07-09-2012 at 11:22 PM..
# 2  
Old 07-10-2012
Welcome to the forum, cooker97.
Did you try something? If yes, please post your efforts so that we can help you further.

Also, It is always better to post small points one-by-one instead of all summing up to a long sentence. just to increase readability and better understanding.

Regarding your problem, please post the sample output for the given input data.
# 3  
Old 07-10-2012
thx for the pointers clx.
Will keep 'em in mind next time.
I am a complete noob to linux, but I need this script for a project I'm working on.

The sample o/p should be smthing like this :

The script should simply create new sub-directories and copy those files there, any one row of which satisfy the criteria I mentioned. (i.e, no. of rows satisfying criteria >=1).

Also, it should give me VALUE D of those row(s) in the copied files which have satisfied the criteria.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Programming

C++ help in large data set

Hi All, We are trying to replace a 3rdparty where we don't know how they handled the reader part here. The query below is getting 197 * 2038017 row in the table. In the below code we are trying to run the query and execute in the DB part and fetch and read the record. That is where it is... (1 Reply)
Discussion started by: arunkumar_mca
1 Replies

2. UNIX for Beginners Questions & Answers

Transpose large data in UNIX

Hi I have the following sample of data: my full data dimention is 900,000* 1119 rs987435 C G 1 1 1 0 2 rs345783 C G 0 0 1 0 0 rs955894 G T 1 1 2 2 1 rs6088791 ... (7 Replies)
Discussion started by: marwah
7 Replies

3. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt (6 Replies)
Discussion started by: Fahmida
6 Replies

4. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

5. Shell Programming and Scripting

Using AWK to separate data from a large XML file into multiple files

I have a 500 MB XML file from a FileMaker database export, it's formatted horribly (no line breaks at all). The node structure is basically <FMPXMLRESULT> <METADATA> <FIELD att="............." id="..."/> </METADATA> <RESULTSET FOUND="1763457"> <ROW att="....." etc="...."> ... (16 Replies)
Discussion started by: JRy
16 Replies

6. Shell Programming and Scripting

finding and removing patterns in a large list of urls

I have a list of urls for example: Google Google Base Yahoo! Yahoo! Yahoo! Video - It's On Google The problem is that Google and Google are duplicates as are Yahoo! and Yahoo!. I'm needing to find these conical www duplicates and append the text "DUP#" in from of both Google and... (3 Replies)
Discussion started by: totus
3 Replies

7. UNIX for Dummies Questions & Answers

Quick question about finding a large file

what is the correct command for finding the largest file and displaying it without any error information? I can find it, but how do I display it in the same command? (6 Replies)
Discussion started by: raidkridley
6 Replies

8. Shell Programming and Scripting

Extracting data from large logs.

Hi all, I have a large log file that gets created daily. I need to be able to pull text out of the log when I hit a pattern including the 7 lines above it and the 3 lines below and output it to a new text file. Line 1 Line 2 Line 3 Line 4 Line 5 Line 6 Line 7 Pattern Line 9 Line 10... (11 Replies)
Discussion started by: bas
11 Replies

9. Shell Programming and Scripting

finding duplicate files by size and finding pattern matching and its count

Hi, I have a challenging task,in which i have to find the duplicate files by its name and size,then i need to take anyone of the file.Then i need to open the file and find for more than one pattern and count of that pattern. Note:These are the samples of two files,but i can have more... (2 Replies)
Discussion started by: jerome Sukumar
2 Replies
Login or Register to Ask a Question