11-15-2008
filtering one file based on results from other
Can anybody help me with writing a script for the data that I want to use from one file based on the data from another file. I have file1 in this form;
(the first field represents a well name and the second field represents the depth of interest)
FILE1
--------
DATA_35_0 -4065.517
DATA_48_0 -4043.609
............... .............
............... .............
The second file is of the form;
FILE2
-------
DATA_35_0 392409.2 2966457. 25.00000
DATA_35_0 392408.6 2966463. -2374.779
DATA_35_0 392408.4 2966463. -2474.773
DATA_35_0 392408.2 2966464. -2574.767
DATA_35_0 392408.2 2966464. -2674.762
DATA_35_0 392408.3 2966464. -2774.757
DATA_35_0 392408.5 2966464. -2874.752
DATA_35_0 392410.7 2966464. -3374.687
DATA_35_0 392411.2 2966463. -3474.669
DATA_35_0 392411.6 2966463. -3574.651
DATA_35_0 392412.1 2966462. -3674.633
DATA_35_0 392412.7 2966462. -3774.612
DATA_35_0 392413.2 2966462. -3874.585
DATA_35_0 392413.9 2966461. -3974.552
DATA_35_0 392414.6 2966461. -4074.514
DATA_35_0 392415.3 2966460. -4174.471
DATA_35_0 392416.1 2966460. -4274.429
DATA_35_0 392416.9 2966460. -4374.381
DATA_35_0 392417.8 2966459. -4474.324
DATA_35_0 392418.7 2966458. -4574.263
DATA_35_0 392419.5 2966458. -4674.216
............... ............. ............ ...............
I want to filter the file2 in a way that it only puts out the data where column 4 in the file2 is less than or equal to the second column in file 1. Thus in if file1 only had one record i.e. DATA_35_0 -4065.517, then I want the file2 to be filtered so that it it becomes (based on column 4) as follows;
DATA_35_0 392414.6 2966461. -4074.514
DATA_35_0 392415.3 2966460. -4174.471
DATA_35_0 392416.1 2966460. -4274.429
DATA_35_0 392416.9 2966460. -4374.381
DATA_35_0 392417.8 2966459. -4474.324
DATA_35_0 392418.7 2966458. -4574.263
DATA_35_0 392419.5 2966458. -4674.216
i.e the column 4 (and the coresponding other columns) are output wherever the column 4 is lessthan or equal to -4065.517
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I created a large file list using:
find . -type f -mtime +540 > test2.txt
..which searched recursively down the directory tree searching for any file older than 540 days.
I would like to filter the results removing the directory name and the "/" character, resulting in only a list of the... (3 Replies)
Discussion started by: fxvisions
3 Replies
2. UNIX for Dummies Questions & Answers
Hi all,
I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g.
COL1 COL2 COL3 ............... COL47
1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies
3. Shell Programming and Scripting
I have asked this question here before and got the answer too. Unfortunately I used only one record as an example and the script works fine for one record but not for more than one record.
Can anybody help me with writing a script for the data that I want to use from one file based on the... (13 Replies)
Discussion started by: digipak
13 Replies
4. Shell Programming and Scripting
I have a ";" delimited file.Whcih conatins a number fileds of length 4 charcters in 8th position
But there is a alphanumeric charcters like :
space, ";" , "," , "/" , "23-1" ,
"23 1" , "aqjhdj" , "jun-23" , "APR-04" , "4:00AM" , "-234" , "56784 ", "." , "+"
"_" , "&" , "*" , "^" , "%" , "!"... (2 Replies)
Discussion started by: indusri
2 Replies
5. Shell Programming and Scripting
Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times.
13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG
13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG
13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT
13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies
6. UNIX for Dummies Questions & Answers
Hi,
I am looking for an awk script which should help me to meet the following requirement:
File1 has records in following format
INF: FAILEd RECORD AB1234
INF: FAILEd RECORD PQ1145
INF: FAILEd RECORD AB3215
INF: FAILEd RECORD AB6114
............................ (2 Replies)
Discussion started by: mintu41
2 Replies
7. Shell Programming and Scripting
Hi friends,
I have one file like below. (.csv type)
SNo,data1,data2
1,1,2
2,2,3
3,3,2
and another file like below.
Exclude
data1
where Exclude should be treated as column name in file2.
I want the output shown below.
SNo,data2
1,2
2,3
3,2
Where my data1 column got removed from... (2 Replies)
Discussion started by: ks_reddy
2 Replies
8. Shell Programming and Scripting
please help solving the following. I have access to redhat linux cluster having 32gigs of ram.
I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates.
Lookup... (4 Replies)
Discussion started by: ritakadm
4 Replies
9. UNIX for Beginners Questions & Answers
Hi there,
I am trying to filter a big file with several columns using values on a column with values like (AC=5;AN=10;SF=341,377,517,643,662;VRT=1). I wont to filter the data based on SF= values that are (bigger than 400)
... (25 Replies)
Discussion started by: daashti
25 Replies
10. UNIX for Beginners Questions & Answers
Hi,
I tried filtering the records in a csv file using "awk" command listed below.
awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile
The output always has all the entries.
The same command worked for different users from one of the forum links.
content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
COMM(1) User Commands COMM(1)
NAME
comm - compare two sorted files line by line
SYNOPSIS
comm [OPTION]... FILE1 FILE2
DESCRIPTION
Compare sorted files FILE1 and FILE2 line by line.
When FILE1 or FILE2 (not both) is -, read standard input.
With no options, produce three-column output. Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and
column three contains lines common to both files.
-1 suppress column 1 (lines unique to FILE1)
-2 suppress column 2 (lines unique to FILE2)
-3 suppress column 3 (lines that appear in both files)
--check-order
check that the input is correctly sorted, even if all input lines are pairable
--nocheck-order
do not check that the input is correctly sorted
--output-delimiter=STR
separate columns with STR
--total
output a summary
-z, --zero-terminated
line delimiter is NUL, not newline
--help display this help and exit
--version
output version information and exit
Note, comparisons honor the rules specified by 'LC_COLLATE'.
EXAMPLES
comm -12 file1 file2
Print only lines present in both file1 and file2.
comm -3 file1 file2
Print lines in file1 not in file2, and vice versa.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report comm translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
join(1), uniq(1)
Full documentation at: <http://www.gnu.org/software/coreutils/comm>
or available locally via: info '(coreutils) comm invocation'
GNU coreutils 8.28 January 2018 COMM(1)