Sponsored Content
Top Forums Shell Programming and Scripting How can I remove those duplicate sequence in UNIX?What command line I should type? Post 302279191 by patrick chia on Thursday 22nd of January 2009 04:48:26 AM
Old 01-22-2009
How can I remove those duplicate sequence in UNIX?What command line I should type?

The input is:
>HWI-EAS382_30FC7AAXX:4:1:1580:1465
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
>HWI-EAS382_30FC7AAXX:4:1:1062:1640
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
>HWI-EAS382_30FC7AAXX:4:1:272:629
AAAAAAAAGCTATAGTCTCGTCACACATACTCACAA
>HWI-EAS382_30FC7AAXX:4:1:1033:1135
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
>HWI-EAS382_30FC7AAXX:4:1:1421:27
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

My desired output is:
>HWI-EAS382_30FC7AAXX:4:1:1580:1465
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
>HWI-EAS382_30FC7AAXX:4:1:272:629
AAAAAAAAGCTATAGTCTCGTCACACATACTCACAA

What command line I should type to remove those duplicated sequence?
Thanks for all of your advise.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line? MI_AP MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR so that it will on output the one unique entry per line. MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN NT_CAR I can't find the same situation on the knowledge... (5 Replies)
Discussion started by: kharen11
5 Replies

3. Shell Programming and Scripting

How to remove those sequence with same amino acid?What command line I should type?

My input is listed as: giNumber RefAminoAcid VarAminoAcid 10190711 P P 10190711 D D 109255248 I A 110349771 A ... (4 Replies)
Discussion started by: patrick chia
4 Replies

4. Shell Programming and Scripting

How can I calculate the total of nucleotide in Unix?What command line I should type?

For example, if I have the file whose content are: >HWI-EAS382_30FC7AAXX:7:1:927:1368 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA >HWI-EAS382_30FC7AAXX:7:1:924:1373 ACGAACTTTAAAGCACCTCTTGGCTCGTATGCCGTC I want my output calculate the total of nucleotide. So my output should look like this:... (2 Replies)
Discussion started by: patrick chia
2 Replies

5. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

6. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

7. Shell Programming and Scripting

Remove duplicate line on condition

Hi Ive been scratching over this for some time with no solution. I have a file like this 1 bla bla 1 2 bla bla 2 4 bla bla 3 5 bla bla 1 6 bla bla 1 I want to remove consecutive occurrences of lines like bla bla 1, but the first column may be different. Any ideasss?? (23 Replies)
Discussion started by: jamie_123
23 Replies

8. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried... (12 Replies)
Discussion started by: bakere19
12 Replies

9. Shell Programming and Scripting

Remove duplicate entries from the same line

Hello, I have a file which have several duplicate entries on the same line: File ID source 1 GM GF GM 2 GM GF GM GF GM GF GM GF GM GF 3 GM GF GM SF GM GF GM SF 4 FF FF FF FF 5 FF GM FF ... (2 Replies)
Discussion started by: nans
2 Replies

10. Shell Programming and Scripting

Remove duplicate line starting with a pattern

HI, I have the below input file /* ----------------- cmdsDlyStartFWJ -----------------*/ UNIX_JOB CMDS065J RUN ANY CMDNAME sleep 5 AGENT CMDSHP USER proddata RUN MON,TUE,WED,THU,FRI DELAYSUB 02:00 /* "Triggers daily file watcher jobs" */ ENVAR... (5 Replies)
Discussion started by: varun22486
5 Replies
line(1) 						      General Commands Manual							   line(1)

NAME
line - Reads one line from standard input SYNOPSIS
line STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: line: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
None DESCRIPTION
The line command copies one line, up to and including a newline, from standard input and writes it to standard output. Use this command within a shell command file to read from your terminal. The line command always writes at least a newline character. NOTES
The line utility has no internationalization features and is marked LEGACY in XCU Issue 5. Use the read utility instead. EXIT STATUS
Success. End-of-File. EXAMPLES
To read a line from the keyboard and append it to a file, enter: echo 'Enter comments for the log:' echo ': c' line >>log This shell procedure displays the message: Enter comments for the log: It then reads a line of text from the keyboard and adds it to the end of the file log. The echo ': c' command displays a : (colon) prompt. See the echo command for information about the c escape sequence. SEE ALSO
Commands: echo(1), ksh(1), read(1), Bourne shell sh(1b), POSIX shell sh(1p) Functions: read(2) Standards: standards(5) line(1)
All times are GMT -4. The time now is 07:33 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy