Sponsored Content
Full Discussion: Filtering the duplicates
Top Forums UNIX for Dummies Questions & Answers Filtering the duplicates Post 302662965 by koneru_18 on Wednesday 27th of June 2012 11:14:17 AM
Old 06-27-2012
Filtering the duplicates

Hello,

I want to filter all the duplicates of a record to one place. Sample input and output will give you better idea.

I am new to unix. Can some one help me on this?

Input:
Code:
7488	7389	chr1.fa	chr1.fa
3546	9887	chr5.fa	chr9.fa
7387	7898	chrX.fa	chr3.fa
7488	7389	chr1.fa	chr1.fa
7488	7389	chr21.fa	chr3.fa
7488	7389	chr1.fa	chr1.fa
9899	4577	chrX.fa	chrM.fa
3546	9887	chr5.fa	chr9.fa

My desired output:
Code:
7488	7389	chr1.fa	chr1.fa
7488	7389	chr1.fa	chr1.fa
7488	7389	chr1.fa	chr1.fa
3546	9887	chr5.fa	chr9.fa
3546	9887	chr5.fa	chr9.fa
7387	7898	chrX.fa	chr3.fa
7488	7389	chr21.fa	chr3.fa
9899	4577	chrX.fa	chrM.fa

Thanks in advance,


Moderator's Comments:
Mod Comment Please view this code tag video for how to use code tags when posting code and data.

Last edited by vbe; 06-27-2012 at 01:30 PM.. Reason: code tags please
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pipe filtering

Hi, The thing is that sometimes I need to filter an arbitrary output given a condition (think on the "filter" function present on some programming languages) from the shell (I use sh/bash). I tried this: #!/bin/sh # Usage: filter command COMMAND=$1 shift 1 xargs "$@" -I "{}" sh -c "if... (3 Replies)
Discussion started by: tokland
3 Replies

2. Shell Programming and Scripting

help need in filtering data

Hello Gurus, Please help me out of the problem. I ve a input file as below input clock; input a; //reset all input b; //input comment output c; output d; output e; input f; //output comment I need the output as follows: \\Inputs (1 Reply)
Discussion started by: user_prady
1 Replies

3. Shell Programming and Scripting

Please help me to do some filtering

I have to grep a pattern. scenario is like :- Suppose "/etc/sec/one" is a string, i need to check if this string contains "one" using any utility something like if /etc/sec/one | grep ; then Thanks in advance Renjesh Raju (3 Replies)
Discussion started by: Renjesh
3 Replies

4. Shell Programming and Scripting

Filtering a list

Hi all When I run the system command "rpm -qa |grep xmpp" it lists many files like xmpp-3.2.10.20111024-3 xmpp-3.2.10.201110_asd xmpp-3.2.10.201 and I want to uninstall all the rpms by using rpm -e. How can I do this??? NOTE: The number of rpms will vary... (3 Replies)
Discussion started by: Ananthdoss
3 Replies

5. AIX

Need help with filtering

Hi!! I have a bit of a task here and filtering/scripting not my strongest. I have to collect info of approx 1100 hdiskpower.so i have appended all the hdisk into a text file and i need it to run the command lscfg -vl to confirm if the drive is symmetrix. here's what i have so far at... (3 Replies)
Discussion started by: vpundit
3 Replies

6. Shell Programming and Scripting

Filtering

Hi I am interested in DNS resolving a set of sites and each time the output is different- $ host www.yahoo.com www.yahoo.com is an alias for fd-fp3.wg1.b.yahoo.com. fd-fp3.wg1.b.yahoo.com is an alias for ds-fp3.wg1.b.yahoo.com. ds-fp3.wg1.b.yahoo.com is an alias for... (1 Reply)
Discussion started by: jamie_123
1 Replies

7. Shell Programming and Scripting

Filtering out duplicates with the highest version number

Hi, I have a huge text file with filenames which which looks like the following ie uniquenumber_version_filename: e.g. 1234_1_xxxx 1234_2_vfvfdbb 343333_1_vfvfdvd 2222222_1_ggggg 55555_1_xxxxxx 55555_2_vrbgbgg 55555_3_grgrbr What I need to do is examine the file, look for... (4 Replies)
Discussion started by: mantis
4 Replies

8. Shell Programming and Scripting

PERL "filtering the log file removing the duplicates

Hi folks, I have a log file in the below format and trying to get the output of the unique ones based on mnemonic IN PERL. Could any one please let me know with the code and the logic ? Severity Mnemonic Log Message 7 CLI_SCHEDULER Logfile for scheduled CLI... (3 Replies)
Discussion started by: scriptscript
3 Replies

9. Shell Programming and Scripting

Need help on filtering

Hi experts, I have a file image.csv as below: COMPUTERNAME,23/07/2013,22/07/2013,21/07/2013,20/07/2013,19/07/2013,18/07/2013,17/07/2013 AED03852180,3,3,3,3,3,3,3 AED03852181,3,3,3,3,3,3,1 AED09020382,3,0,3,0,3,3,3 AED09020383,1,3,3,3,2,1,3 AED09020386,3,3,0,3,3,0,3 ... (4 Replies)
Discussion started by: zaq1xsw2
4 Replies

10. Shell Programming and Scripting

Filtering duplicates based on lookup table and rules

please help solving the following. I have access to redhat linux cluster having 32gigs of ram. I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates. Lookup... (4 Replies)
Discussion started by: ritakadm
4 Replies
html2po(1)						      Translate Toolkit 1.9.0							html2po(1)

NAME
html2po - convert HTML files to Gettext PO localization files SYNOPSIS
html2po [--version] [-h|--help] [--manpage] [--progress PROGRESS] [--errorlevel ERRORLEVEL] [-i|--input] INPUT [-x|--exclude EXCLUDE] [-o|--output] OUTPUT [-P|--pot] [-u|--untagged] [--keepcomments] [--duplicates DUPLICATESTYLE] DESCRIPTION
See: http://translate.sourceforge.net/wiki/toolkit/html2po for examples and usage instructions OPTIONS
--version show program's version number and exit -h/--help show this help message and exit --manpage output a manpage based on the help --progress show progress as: dots, none, bar, names, verbose --errorlevel show errorlevel as: none, message, exception, traceback -i/--input read from INPUT in htm, html, xhtml formats -x/--exclude exclude names matching EXCLUDE from input paths -o/--output write to OUTPUT in po, pot formats -P/--pot output PO Templates (.pot) rather than PO files (.po) -u/--untagged include untagged sections --keepcomments preserve html comments as translation notes in the output --duplicates what to do with duplicate strings (identical source text): merge, msgctxt (default: 'msgctxt') Translate Toolkit 1.9.0 html2po(1)
All times are GMT -4. The time now is 02:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy