Sponsored Content
Top Forums Shell Programming and Scripting Request to check:Remove duplicates Post 302673657 by raj_saini20 on Wednesday 18th of July 2012 08:17:42 AM
Old 07-18-2012
Code:
 
$ awk '{if (!a[$1]++){print}}'  inputfile


Last edited by raj_saini20; 07-18-2012 at 09:33 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Request to check:remove entries with N/A entries

Hi I have a file with numerous entries some entries are 1 mani 2 kavya 3 N/A 4 Praveeen 5 N/A and so on How to remove entries with N/A so the result will be 1 mani 2 kavya 3 Praveeen (6 Replies)
Discussion started by: manigrover
6 Replies

2. Shell Programming and Scripting

Request to check:remove entries more than once

Hi I have a file like this 1234 2345 567889 567889 2345 234899420 83743 2345 67890 67890 ................ so on I want to delete entries which are more than once like 2345, 567889 and 67890 so that these appear once (4 Replies)
Discussion started by: manigrover
4 Replies

3. Shell Programming and Scripting

Request to check:remove entries with blank spaces

Hi I want to remove entries with blank spaces in my input file: 123 234 456 678 56789 345346456 589 3454 345456 3454566............................ (2 Replies)
Discussion started by: manigrover
2 Replies

4. Shell Programming and Scripting

Request to check:remove entries with N/A mentioned

Hi I have a file with following entries 122 N/A 123 5654656 123423 43534543 4544 45435 435454 N/A i Have to remove entries with N/A so that only 123 5654656 123423 43534543 4544 45435 remain in output file can anybody guide for a code/unix/perl (2 Replies)
Discussion started by: manigrover
2 Replies

5. Shell Programming and Scripting

Request to check:remove entries more than once in different column

Hi I have a file 12m 345693460 12 1234 12 1234 34 345 34 345 And I want output fiel as 12m 345693460 12 1234 34 345 hw can it be done Thanks (1 Reply)
Discussion started by: manigrover
1 Replies

6. Shell Programming and Scripting

Request to check remove duplicates but write before it

Hi alll I have a file with following kind input I want in output duplicates should not be there but there should be numbering mentioned before that like (4 Replies)
Discussion started by: manigrover
4 Replies

7. Shell Programming and Scripting

Request to check:remove duplicates and write sytematically

Hi all I have a file with following input It contains 5 columns gene name drug drug ID disease approved Now the same gene is repeated many times with different data in column2,3 ,4,5 I want to arrange dat in such a way that there shuld be one entry in the column(no... (2 Replies)
Discussion started by: manigrover
2 Replies

8. Shell Programming and Scripting

Kindly check:remove duplicates with similar data in front of it

Hi all, I have 2 files containing data like this: so if there is same entry repeated in the column like1,2,3,4 I have to check if there is different entries column like 2,4 but similar entries for duplicatein column 2 like1,3 the output shuld be like this for first file ... (5 Replies)
Discussion started by: manigrover
5 Replies

9. Shell Programming and Scripting

Request to check:remove entries with duplicate numbers in first row

Hi I have a file 1 xyz 456 1 xyz 456 1 xyz 456 2 abc 8459 3 gfd 657 4 ghf 658 4 ghf 658 I want the output 1 xyz 456 2 abc 8459 3 gfd 657 4 ghf 658 (3 Replies)
Discussion started by: manigrover
3 Replies

10. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies
ICONV(1)							 Debian GNU/Linux							  ICONV(1)

NAME
iconv - Convert encoding of given files from one encoding to another SYNOPSIS
iconv -f encoding [-t encoding] [inputfile]... DESCRIPTION
The iconv program converts the encoding of characters in inputfile, or from the standard input if no filename is specified, from one coded character set to another. The result is written to standard output unless otherwise specified by the --output option. --from-code, -f encoding Convert characters from encoding. --to-code, -t encoding Convert characters to encoding. If not specified the encoding corresponding to the current locale is used. --list, -l List known coded character sets. -c Omit invalid characters from output. --output, -o file Specify output file (instead of stdout). --silent, -s Suppress warnings, but not errors. --verbose Print progress information. --help, -? Give help list. --usage Give a short usage message. --version, -V Print program version. ENCODINGS
The values permitted for --from-code and --to-code can be listed by the iconv --list command, and all combinations of the listed values are supported. Furthermore the following two suffixes are supported: //TRANSLIT When the string "//TRANSLIT" is appended to --to-code, transliteration is activated. This means that when a character cannot be represented in the target character set, it can be approximated through one or several similarly looking characters. //IGNORE When the string "//IGNORE" is appended to --to-code, characters that cannot be represented in the target character set will be silently discarded. AUTHOR
iconv was written by Ulrich Drepper as part of the GNU C Library. This man page was written by Joel Klecker <espy@debian.org>, for the Debian GNU/Linux system. 3rd Berkeley Distribution lenny ICONV(1)
All times are GMT -4. The time now is 08:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy