Sponsored Content
Top Forums Shell Programming and Scripting filtering one file based on results from other- AGAIN Post 302267846 by digipak on Sunday 14th of December 2008 07:21:56 AM
Old 12-14-2008
Thanks a lot

Thank a lot both of you guys. Rodulov, I want to specially thank you as your script worked without anymore modification of the script. Perfectly done. Thanks a lot. Can I ask you one more question, after running your script I have a file in which there are lots of records that only occur once (It is normal for my data), I would like to run an awk (or any other command) to get rid of the records that only occur once (after I run Rodulov's script). Can somebody help me. For example I might have the data as follows;

So from the following data;
I would like to delete the Line 1, 2, 3, 4, 5, 6 and 7 but keep the rest as these the records on line 1, 2, 3, 4, 5, 6 and 7 occur only once (based on the first column). I would appreciate your help on this one.


DATA_96_0 315367.47 2711845.75 -6439.36
DATA_97_0 314705.38 2706544.75 -6480.00
DATA_98_0 314541.81 2717794.50 -6015.30
DATA_99_0 315198.31 2710142.75 -6416.00
DATA_9_0 299511.28 2719683.50 -6525.00
DATA_8_0 309508.59 2719213.25 -5422.90
DATA_84_0 318521.03 2722324.75 -7028.00
DATA_85_0 318399.84 2720022.25 -6658.59
DATA_85_0 318398.53 2720050.00 -6666.37
DATA_85_0 318396.75 2720078.50 -6670.63
DATA_85_0 318394.91 2720106.75 -6675.82
DATA_85_0 318393.62 2720135.00 -6684.01
DATA_85_0 318393.22 2720162.25 -6693.65
DATA_85_0 318393.22 2720190.75 -6706.62
DATA_85_0 318393.06 2720219.25 -6720.19
DATA_85_0 318392.25 2720247.00 -6732.92
DATA_85_0 318390.88 2720275.00 -6745.86
DATA_85_0 318389.41 2720301.00 -6755.37
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

filtering list results

I created a large file list using: find . -type f -mtime +540 > test2.txt ..which searched recursively down the directory tree searching for any file older than 540 days. I would like to filter the results removing the directory name and the "/" character, resulting in only a list of the... (3 Replies)
Discussion started by: fxvisions
3 Replies

2. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

3. Shell Programming and Scripting

filtering one file based on results from other

Can anybody help me with writing a script for the data that I want to use from one file based on the data from another file. I have file1 in this form; (the first field represents a well name and the second field represents the depth of interest) FILE1 -------- DATA_35_0 ... (2 Replies)
Discussion started by: digipak
2 Replies

4. Shell Programming and Scripting

filtering records based on numeric field value in 8th position

I have a ";" delimited file.Whcih conatins a number fileds of length 4 charcters in 8th position But there is a alphanumeric charcters like : space, ";" , "," , "/" , "23-1" , "23 1" , "aqjhdj" , "jun-23" , "APR-04" , "4:00AM" , "-234" , "56784 ", "." , "+" "_" , "&" , "*" , "^" , "%" , "!"... (2 Replies)
Discussion started by: indusri
2 Replies

5. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

6. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

7. Shell Programming and Scripting

Filtering first file columns based on second file column

Hi friends, I have one file like below. (.csv type) SNo,data1,data2 1,1,2 2,2,3 3,3,2 and another file like below. Exclude data1 where Exclude should be treated as column name in file2. I want the output shown below. SNo,data2 1,2 2,3 3,2 Where my data1 column got removed from... (2 Replies)
Discussion started by: ks_reddy
2 Replies

8. Shell Programming and Scripting

Filtering duplicates based on lookup table and rules

please help solving the following. I have access to redhat linux cluster having 32gigs of ram. I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates. Lookup... (4 Replies)
Discussion started by: ritakadm
4 Replies

9. UNIX for Beginners Questions & Answers

Filtering based on column values

Hi there, I am trying to filter a big file with several columns using values on a column with values like (AC=5;AN=10;SF=341,377,517,643,662;VRT=1). I wont to filter the data based on SF= values that are (bigger than 400) ... (25 Replies)
Discussion started by: daashti
25 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
TRUNCATE(1)							   User Commands						       TRUNCATE(1)

NAME
truncate - shrink or extend the size of a file to the specified size SYNOPSIS
truncate OPTION... FILE... DESCRIPTION
Shrink or extend the size of each FILE to the specified size A FILE argument that does not exist is created. If a FILE is larger than the specified size, the extra data is lost. If a FILE is shorter, it is extended and the extended part (hole) reads as zero bytes. Mandatory arguments to long options are mandatory for short options too. -c, --no-create do not create any files -o, --io-blocks treat SIZE as number of IO blocks instead of bytes -r, --reference=RFILE base size on RFILE -s, --size=SIZE set or adjust the file size by SIZE bytes --help display this help and exit --version output version information and exit The SIZE argument is an integer and optional unit (example: 10K is 10*1024). Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (pow- ers of 1000). SIZE may also be prefixed by one of the following modifying characters: '+' extend by, '-' reduce by, '<' at most, '>' at least, '/' round down to multiple of, '%' round up to multiple of. AUTHOR
Written by Padraig Brady. REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report truncate translation bugs to <http://translationproject.org/team/> COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
dd(1), truncate(2), ftruncate(2) Full documentation at: <http://www.gnu.org/software/coreutils/truncate> or available locally via: info '(coreutils) truncate invocation' GNU coreutils 8.28 January 2018 TRUNCATE(1)
All times are GMT -4. The time now is 09:25 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy