11-09-2007
As a clarification:
I am looking for one line per duplicate info. Therefore, the 4 lines input file (given as an example) is formed of two blocks, 2 lines each, containaing duplicate data. Thus the result is two lines.
AA BB CC DD EE FF GG HH KK ----- First Block
AA BB GG HH KK FF CC DD EE ----- First Block
AA BB CC DD EE UU VV XX ZZ ----- Second Block
AA BB VV XX ZZ UU CC DD EE ----- Second Blcok
The results should be like the following:
AA BB CC DD EE FF GG HH KK ----- Result for First Block
AA BB CC DD EE UU VV XX ZZ ----- Result for Second Block
I hope it is clear now!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi Guys...
Please Could you help me with the following ?
aaaa bbbb cccc sdsd
aaaa bbbb cccc qwer
as you can see, the 2 lines are matched in three fields...
how can I delete this pupicate ? I mean to delete the second one if 3 fields were duplicated ?
Thanks (14 Replies)
Discussion started by: yahyaaa
14 Replies
2. Shell Programming and Scripting
I am trying to figure out how to scan a file like so:
1 ralphs office","555-555-5555","ralph@mail.com","www.ralph.com
2 margies office","555-555-5555","ralph@mail.com","www.ralph.com
3 kims office","555-555-5555","kims@mail.com","www.ralph.com
4 tims... (17 Replies)
Discussion started by: totus
17 Replies
3. Shell Programming and Scripting
Hi,
snp.txt
CHR_A SNP_A BP_A_st BP_A_End CHR_B BP_B SNP_B R2 p-SNP_A p-SNP_B
5 rs1988728 74904317 74904318 5 74960646 rs1427924 0.377333 0.000740085 0.013930081
5 ... (12 Replies)
Discussion started by: genehunter
12 Replies
4. UNIX for Dummies Questions & Answers
I have a file1 that looks like this:
File 1
a b
b c
c e
d e
and a file 2 that looks like this:
File 2
b
c
e
e
Note that file 2 is the right hand column from file1. I want to remove any lines from file1 that begin with the column in file2. In this case the desired output... (6 Replies)
Discussion started by: kschiltz55
6 Replies
5. UNIX for Dummies Questions & Answers
hello all,
I have an input file with four columns like this with a lot of lines
and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies
6. Shell Programming and Scripting
Hi,
I need to concatenate some lines in a file based on the First 4 coloumns of a file .. (For Eg.)
Consider a file ...
I,01,000002,0666,00000.00,000,00,000,000, ,0
I,01,000002,0667,00000.00,000,00,000,000, ,0
I,01,000002,0666,00056.10
I,01,000002,0667,00056.10
I,01,000002,0666,00001... (6 Replies)
Discussion started by: Sri3001
6 Replies
7. UNIX for Advanced & Expert Users
Consider i have 2 directories a1 and a2.
under a1, i have below files
test1
test2
test3.
Under a2,i have below files.
test1
test2
test3
test4
test5My requirement is i will pass the directory names(2 parameters) and directory in which files needs to be removed.(3rd parameter)
a)first... (11 Replies)
Discussion started by: pandeesh
11 Replies
8. Shell Programming and Scripting
d df d d (1 Reply)
Discussion started by: larsk
1 Replies
9. Shell Programming and Scripting
Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker...
Column #1 is a simple ID, which is used to identify the duplicate.
Once dups are identified, I need to only keep the one... (2 Replies)
Discussion started by: kevinprood
2 Replies
10. Shell Programming and Scripting
Hi,
I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines
Command : sort -t'|' -nuk1 file.txt
Input :
38376KZ|09/25/15|1.057
38376KZ|09/25/15|1.057
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803... (2 Replies)
Discussion started by: parithi06
2 Replies
LEARN ABOUT DEBIAN
getcol
getcol(1) General Commands Manual getcol(1)
Name
getcol - Extract specified columns from an ASCII table file
Synopsis
getcol [-amv][-n num][-r lines][-s num] filename [column number range]
Description
Extract specified columns from an ASCII table file
Options
filename
Name of a ASCII table file. At least one of these must be present for any values to be printed. If it is stdin or STDIN, an ASCII
table is expected as standard input. If there is no input file, standard input is assumed.
@filename
Name of a file containing a list of ASCII table files. If this is present, any other file names on the command line will be
ignored.
field range
Print value of these columns for the number of lines of the table specified by the -n argument after the skippiing the number of
lines specified by the -s argument. A value of 0 causes the entire input line to be printed.
-a Sum all numeric columns selected, printing the sum on the line following the result. Columns with no sum are filled with ___.
(Added in version 2.6.9)
-b Input is bar-separate table file
-c Add count of number of lines in each column at end
-d <number>
Number of decimal places in f.p. output
-e Compute medians of selected columns
-f Print range of values in selected columns
-h Print Starbase tab table header
-i Input is tab-separate table file
-k Print number of columns on first line
-l <number>
Number of lines to add to each line
-m Compute the means of all numeric columns selected, printing the mean on the line following the result (or the line following the sum
if -a is used). Columns with no mean are filled with ___. (Added in version 2.6.9)
-n num Print selected columns for this many lines. If not specified, all lines will be read after the number of lines specified by -s have
been skipped.
-o OR conditions insted of ANDing them
-p Print only sum, mmean, sigma, median, or range, not entries
-r @listfile
-r line range Print columns from the lines specified as either the first nonzero number on each line of the file listfile or the
comma- and hyphen- delimitied range; i.e. 1-5,10-12 will print values from lines 1, 2, 3, 4, 5, 10, 11, and 12. (added in version
2.6.12)
-s num Skip this many line before starting to print values. If not specified, no lines will be skipped.
-t Starbase (tab-separated) table output
-v Print more information about process.
Web Page
http://tdc-www.harvard.edu/software/wcstools/getcol.html
Author
Doug Mink, SAO (dmink@cfa.harvard.edu)
8 November 2001 WCSTools getcol(1)