Remove duplicate columns in input file Post: 302498917

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file...

2. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ...

3. Shell Programming and Scripting

Formatting a file - Remove Duplicate

Hi I have a file in the following format. Basically the file contains tablename and their aliases: TABLE1 TABLE1 A TABLE2 TABLE2 B TABLE3 TABLE4 TABLE4 C TABLE4 Upon formatting an sql statement I am getting such output. Problem: Whenever a tablename appears with alias, it has...

4. UNIX for Dummies Questions & Answers

remove duplicate lines based on two columns and judging from a third one

hello all, I have an input file with four columns like this with a lot of lines and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5....

5. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later!

6. Shell Programming and Scripting

Remove Duplicate by considering multiple columns

hi friends, my input chr1 exon 35204 35266 gene_id "GOLGB1"; transcript_id "GOLGB1"; chr1 exon 42357 42473 gene_id "GOLGB1"; transcript_id "GOLGB1"; chr1 exon 45261 45404 gene_id "GOLGB1"; transcript_id "GOLGB1"; chr1 exon 50701 50778 gene_id "GOLGB1"; transcript_id "GOLGB1";...

7. Shell Programming and Scripting

How to Remove duplicate value from file?

if different branch code is available for same BIC code and one of the branch code is XXX.only one row will be stored and with branch code as XXX .rest of the rows for the BIC code will not be stored. for example if $7 is BIC code and $8 is branch code INPUT file are following...

8. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ...

9. Shell Programming and Scripting

Remove columns with duplicate entries

I have a 13gb file. It has the following columns: The 3rd column is basically correlation values. I want to delete those rows which are repeated between the columns: A B 0.04 B C 0.56 B B 1 A A 1 C D 1 C C 1 Desired Output: (preferably in a .csv format A,B,0.04 B,C,0.56 C,D,1...

LEARN ABOUT DEBIAN

empty-page

EMPTY-PAGE(1)							 ExactImage Manual						     EMPTY-PAGE(1)

NAME

       empty-page - empty page detector of the ExactImage toolkit

SYNOPSIS

       empty-page [option...] {-i | --input} input-file

       empty-page {-h | --help}

DESCRIPTION

       ExactImage is a fast C++ image processing library. Unlike many other library frameworks it allows operation in several color spaces and bit
       depths natively, resulting in low memory and computational requirements.

       empty-page counts dark pixels of a black and white image and decides with a threshold whether the page is most probably empty, and thus can
       be removed from the image processing stream.

OPTIONS

       -i file, --input file
	   Read image from the specified file.

       -m n, --margin n
	   Set width of border margin to skip. For speed reasons, the margin has to be a multiple of 8. The default is 16.

       -p x, --percentage x
	   Set fraction of permissible dark pixels. The default is 0.05 (5%).

       -h, --help
	   Display help text and exit.

EXAMPLES

	   $ empty-page -i test.tif
	   The image has 75461 dark pixels from a total of 1060992 (7.11231%).
	   non-empty

EXIT STATUS

       The exit status is 0 if the image is mostly white, 1 otherwise.

SEE ALSO

       exactimage(7)

AUTHORS

       Jakub Wilk <jwilk@debian.org>
	   Wrote this manual page for the Debian system.

       http://www.exactcode.de/site/open_source/exactimage/
	   This manual page incorporates texts found on the ExactImage homepage.

COPYRIGHT

       This manual page was written for the Debian system (and may be used by others).

       Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 or (at
       your option) any later version published by the Free Software Foundation.

       On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL-2.

empty-page							    09/09/2013							     EMPTY-PAGE(1)