Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: filtering one file based on results from other- AGAIN
Top Forums Shell Programming and Scripting filtering one file based on results from other- AGAIN Post 302268756 by radoulov on Tuesday 16th of December 2008 08:43:48 AM
Old 12-16-2008
If I'm not missing something:

Code:
awk 'END {
  split(l, t)
  print k, t[2], t[3], t[4] - f
  }
!_[$1]++ {
  if (l) {
    split(l, t)
    print k, t[2], t[3], t[4] - f
    }
  k = $1; f = $NF
  }
{ l = $0 }'

So the result would be:

Code:
$ awk 'END {
  for(i=1; i<=c; i++) {
    split(r[i], t)
if (u[t[1]] > 1)
  print r[i]
  }
  }
NR == FNR {
  _[$1] = $2
  next
  }
$1 in _ && $4 <= _[$1] {
  r[++c] = $0
  u[$1]++
}' file1 file2|awk 'END {
  split(l, t)
  print k, t[2], t[3], t[4] - f
  }
!_[$1]++ {
  if (l) {
    split(l, t)
print k, t[2], t[3], t[4] - f
}
  k = $1; f = $NF
  }
{ l = $0 }'
DATA_444_0 299622.41 2686010.75 -7.57
DATA_451_0 299332.78 2684855.50 -6.04

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

filtering list results

I created a large file list using: find . -type f -mtime +540 > test2.txt ..which searched recursively down the directory tree searching for any file older than 540 days. I would like to filter the results removing the directory name and the "/" character, resulting in only a list of the... (3 Replies)
Discussion started by: fxvisions
3 Replies

2. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

3. Shell Programming and Scripting

filtering one file based on results from other

Can anybody help me with writing a script for the data that I want to use from one file based on the data from another file. I have file1 in this form; (the first field represents a well name and the second field represents the depth of interest) FILE1 -------- DATA_35_0 ... (2 Replies)
Discussion started by: digipak
2 Replies

4. Shell Programming and Scripting

filtering records based on numeric field value in 8th position

I have a ";" delimited file.Whcih conatins a number fileds of length 4 charcters in 8th position But there is a alphanumeric charcters like : space, ";" , "," , "/" , "23-1" , "23 1" , "aqjhdj" , "jun-23" , "APR-04" , "4:00AM" , "-234" , "56784 ", "." , "+" "_" , "&" , "*" , "^" , "%" , "!"... (2 Replies)
Discussion started by: indusri
2 Replies

5. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

6. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

7. Shell Programming and Scripting

Filtering first file columns based on second file column

Hi friends, I have one file like below. (.csv type) SNo,data1,data2 1,1,2 2,2,3 3,3,2 and another file like below. Exclude data1 where Exclude should be treated as column name in file2. I want the output shown below. SNo,data2 1,2 2,3 3,2 Where my data1 column got removed from... (2 Replies)
Discussion started by: ks_reddy
2 Replies

8. Shell Programming and Scripting

Filtering duplicates based on lookup table and rules

please help solving the following. I have access to redhat linux cluster having 32gigs of ram. I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates. Lookup... (4 Replies)
Discussion started by: ritakadm
4 Replies

9. UNIX for Beginners Questions & Answers

Filtering based on column values

Hi there, I am trying to filter a big file with several columns using values on a column with values like (AC=5;AN=10;SF=341,377,517,643,662;VRT=1). I wont to filter the data based on SF= values that are (bigger than 400) ... (25 Replies)
Discussion started by: daashti
25 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies

LEARN ABOUT CENTOS

end

END(7)							  PostgreSQL 9.2.7 Documentation						    END(7)

NAME

       END - commit the current transaction

SYNOPSIS

       END [ WORK | TRANSACTION ]

DESCRIPTION

       END commits the current transaction. All changes made by the transaction become visible to others and are guaranteed to be durable if a
       crash occurs. This command is a PostgreSQL extension that is equivalent to COMMIT(7).

PARAMETERS

       WORK, TRANSACTION
	   Optional key words. They have no effect.

NOTES

       Use ROLLBACK(7) to abort a transaction.

       Issuing END when not inside a transaction does no harm, but it will provoke a warning message.

EXAMPLES

       To commit the current transaction and make all changes permanent:

	   END;

COMPATIBILITY

       END is a PostgreSQL extension that provides functionality equivalent to COMMIT(7), which is specified in the SQL standard.

SEE ALSO

       BEGIN(7), COMMIT(7), ROLLBACK(7)

PostgreSQL 9.2.7						    2014-02-17								    END(7)
All times are GMT -4. The time now is 10:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy