Sponsored Content
Top Forums Shell Programming and Scripting How can I remove those duplicate sequence in UNIX?What command line I should type? Post 302279354 by cfajohnson on Thursday 22nd of January 2009 01:26:45 PM
Old 01-22-2009
Code:
awk '!x[$0]++' FILE

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line? MI_AP MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR so that it will on output the one unique entry per line. MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN NT_CAR I can't find the same situation on the knowledge... (5 Replies)
Discussion started by: kharen11
5 Replies

3. Shell Programming and Scripting

How to remove those sequence with same amino acid?What command line I should type?

My input is listed as: giNumber RefAminoAcid VarAminoAcid 10190711 P P 10190711 D D 109255248 I A 110349771 A ... (4 Replies)
Discussion started by: patrick chia
4 Replies

4. Shell Programming and Scripting

How can I calculate the total of nucleotide in Unix?What command line I should type?

For example, if I have the file whose content are: >HWI-EAS382_30FC7AAXX:7:1:927:1368 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA >HWI-EAS382_30FC7AAXX:7:1:924:1373 ACGAACTTTAAAGCACCTCTTGGCTCGTATGCCGTC I want my output calculate the total of nucleotide. So my output should look like this:... (2 Replies)
Discussion started by: patrick chia
2 Replies

5. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

6. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

7. Shell Programming and Scripting

Remove duplicate line on condition

Hi Ive been scratching over this for some time with no solution. I have a file like this 1 bla bla 1 2 bla bla 2 4 bla bla 3 5 bla bla 1 6 bla bla 1 I want to remove consecutive occurrences of lines like bla bla 1, but the first column may be different. Any ideasss?? (23 Replies)
Discussion started by: jamie_123
23 Replies

8. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried... (12 Replies)
Discussion started by: bakere19
12 Replies

9. Shell Programming and Scripting

Remove duplicate entries from the same line

Hello, I have a file which have several duplicate entries on the same line: File ID source 1 GM GF GM 2 GM GF GM GF GM GF GM GF GM GF 3 GM GF GM SF GM GF GM SF 4 FF FF FF FF 5 FF GM FF ... (2 Replies)
Discussion started by: nans
2 Replies

10. Shell Programming and Scripting

Remove duplicate line starting with a pattern

HI, I have the below input file /* ----------------- cmdsDlyStartFWJ -----------------*/ UNIX_JOB CMDS065J RUN ANY CMDNAME sleep 5 AGENT CMDSHP USER proddata RUN MON,TUE,WED,THU,FRI DELAYSUB 02:00 /* "Triggers daily file watcher jobs" */ ENVAR... (5 Replies)
Discussion started by: varun22486
5 Replies
crontab(5)							File Formats Manual							crontab(5)

Name
       crontab - clock daemon table file

Syntax
       /usr/lib/crontab

Description
       The  command  executes  at  specified dates and times according to the instructions in the file. The file consists of lines with six fields
       each.  The format for a line is as follows:

	      minute hour day month weekday command

       The following list defines each field in the line:

       minute (0-59)  The exact minute that the command sequence executes.

       hour (0-23)    The hour of the day that the command sequence executes.

       day (1-31)     The day of the month that the command sequence executes.

       month (1-12)   The month of the year that the command sequence executes.

       weekday (1-7)  The day of the week that the command sequence executes. Monday = 1, Tuesday = 2, and so forth.

       command	      The complete command sequence variable that is to be executed.  Note that the command string must conform  to  Bourne  shell
		      syntax.

       The first five integer fields may be specified as follows:

       o   A single number in the specified range

       o   Two numbers separated by a minus, meaning a range inclusive

       o   A list of numbers separated by commas, meaning any of the numbers

       o   An asterisk meaning all legal values

       The sixth field is a string that is executed by the shell at the specified times.  A percent sign (%) in this field is translated to a new-
       line character.	Only the first line of the command field, up to a percent sign (%) or end of line, is executed by the  shell.	The  other
       lines are made available to the command as standard input.

Examples
       The following example is part of a file:
       # periodic things
       0,15,30,45 * * * * (echo '^M' `date`; echo '') >/dev/console
       0,15,30,45 * * * * /usr/lib/atrun

       # daily stuff
       5 4 * * * sh /usr/adm/newsyslog
       15 4 * * * ( cd /usr/preserve; find . -mtime +7 -a -exec rm -f {} ; )
       20 4 * * * find /usr/msgs -mtime +21 -a ! -perm 444 -a ! -name bounds
	    -a -exec rm -f {} ;

       # NOTE: The above line is wrapped.

       # local cleanups
       30 4 * * * find /usr/spool/mqueue -type f -mtime +5 -name df-exec rm {} ;
       35 4 * * * find /usr/spool/mqueue -type f -mtime +5 -name tf-exec rm {} ;
       40 4 * * * find /usr/spool/rwho -type f -mtime +21 -exec rm {} ;
       #

       # redirecting error output
       0 17 * * 1,3,5 /bin/tar -cv /usr/sysads/smith > /dev/console 2>&1
       #

Files
See Also
       sh(1), cron(8)
       Guide to System Environment Setup

																	crontab(5)
All times are GMT -4. The time now is 06:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy