How can I remove those duplicate sequence in UNIX?What command line I should type? Post: 302279496

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate...

2. UNIX for Dummies Questions & Answers

Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line? MI_AP MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR so that it will on output the one unique entry per line. MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN NT_CAR I can't find the same situation on the knowledge...

3. Shell Programming and Scripting

How to remove those sequence with same amino acid?What command line I should type?

My input is listed as: giNumber RefAminoAcid VarAminoAcid 10190711 P P 10190711 D D 109255248 I A 110349771 A ...

4. Shell Programming and Scripting

How can I calculate the total of nucleotide in Unix?What command line I should type?

For example, if I have the file whose content are: >HWI-EAS382_30FC7AAXX:7:1:927:1368 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA >HWI-EAS382_30FC7AAXX:7:1:924:1373 ACGAACTTTAAAGCACCTCTTGGCTCGTATGCCGTC I want my output calculate the total of nucleotide. So my output should look like this:...

5. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen...

6. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from...

7. Shell Programming and Scripting

Remove duplicate line on condition

Hi Ive been scratching over this for some time with no solution. I have a file like this 1 bla bla 1 2 bla bla 2 4 bla bla 3 5 bla bla 1 6 bla bla 1 I want to remove consecutive occurrences of lines like bla bla 1, but the first column may be different. Any ideasss??

8. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried...

9. Shell Programming and Scripting

Remove duplicate entries from the same line

Hello, I have a file which have several duplicate entries on the same line: File ID source 1 GM GF GM 2 GM GF GM GF GM GF GM GF GM GF 3 GM GF GM SF GM GF GM SF 4 FF FF FF FF 5 FF GM FF ...

10. Shell Programming and Scripting

Remove duplicate line starting with a pattern

HI, I have the below input file /* ----------------- cmdsDlyStartFWJ -----------------*/ UNIX_JOB CMDS065J RUN ANY CMDNAME sleep 5 AGENT CMDSHP USER proddata RUN MON,TUE,WED,THU,FRI DELAYSUB 02:00 /* "Triggers daily file watcher jobs" */ ENVAR...

LEARN ABOUT CENTOS

xmlstarlet

XMLSTARLET(1)                                                    xmlstarlet Manual                                                   XMLSTARLET(1)

NAME

       xmlstarlet - command line XML/XSLT toolkit

SYNOPSIS

       xmlstarlet [<options>] [<command>] [<cmd-options>]

INTRODUCTION

       XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files us-
       ing simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands.
       This  set of command line utilities can be used by those who deal with many XML documents on UNIX shell command prompt as well as for auto-
       mated XML processing with shell scripts.

OPTIONS

       --version
              Display the version of xmlstarlet.

       --help Display help.

COMMANDS

       Type: xmlstarlet <command> --help <ENTER> for command help

       Available commands include:

       ed (or edit)
              Edit/update XML document(s).

       sel (or select)
              Select data or query XML document(s) (XPATH, etc).

       tr (or transform)
              Transform XML documents(s) using XSLT.

       val (or validate)
              Validate XML document(s) (well-formed/DTD/XSD/RelaxNG).

       fo (or format)
              Format XML document(s).

       el (or elements)
              Display element structure of XML document.

       c14n (or canonic)
              XML canonicalization.

       ls (or list)
              List directory as XML.

       esc (or escape)
              Escape special XML characters.

       unesc (or unescape)
              Unescape special XML characters.

       pyx (or xmln)
              Convert XML into PYX format (based on ESIS - ISO 8879).

       p2x (or depyx)
              Convert PYX into XML.

REFERENCES

       XMLStarlet is a command line  toolkit  to  query/edit/check/transform  XML  documents  (for  more  information  see  http://xmlstar.source-
       forge.net/).

AUTHOR

       Mikhail Grushinskiy.

                                                                                                                                     XMLSTARLET(1)