Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Remove duplicate entry in one line
Top Forums UNIX for Dummies Questions & Answers Remove duplicate entry in one line Post 302125478 by Shell_Life on Thursday 5th of July 2007 02:06:17 PM
Old 07-05-2007
Code:
while read mLine
do
  echo $mLine | tr ' ' '\n' | sort | uniq | paste -d' ' -s -
done < input_file

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

3. Shell Programming and Scripting

awk script to remove duplicate rows in line

i have the long file more than one ns and www and mx in the line like . i need the first ns record and first www and first mx from line . the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution. ... (4 Replies)
Discussion started by: kiranmosarla
4 Replies

4. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

5. Shell Programming and Scripting

Remove duplicate line on condition

Hi Ive been scratching over this for some time with no solution. I have a file like this 1 bla bla 1 2 bla bla 2 4 bla bla 3 5 bla bla 1 6 bla bla 1 I want to remove consecutive occurrences of lines like bla bla 1, but the first column may be different. Any ideasss?? (23 Replies)
Discussion started by: jamie_123
23 Replies

6. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried... (12 Replies)
Discussion started by: bakere19
12 Replies

7. Shell Programming and Scripting

Remove duplicate entries from the same line

Hello, I have a file which have several duplicate entries on the same line: File ID source 1 GM GF GM 2 GM GF GM GF GM GF GM GF GM GF 3 GM GF GM SF GM GF GM SF 4 FF FF FF FF 5 FF GM FF ... (2 Replies)
Discussion started by: nans
2 Replies

8. Shell Programming and Scripting

Remove duplicate line starting with a pattern

HI, I have the below input file /* ----------------- cmdsDlyStartFWJ -----------------*/ UNIX_JOB CMDS065J RUN ANY CMDNAME sleep 5 AGENT CMDSHP USER proddata RUN MON,TUE,WED,THU,FRI DELAYSUB 02:00 /* "Triggers daily file watcher jobs" */ ENVAR... (5 Replies)
Discussion started by: varun22486
5 Replies

9. Shell Programming and Scripting

Remove sections based on duplicate first line

Hi, I have a file with many sections in it. Each section is separated by a blank line. The first line of each section would determine if the section is duplicate or not. if the section is duplicate then remove the entire section from the file. below is the example of input and output.... (5 Replies)
Discussion started by: ahmedwaseem2000
5 Replies

10. UNIX for Dummies Questions & Answers

Using awk to remove duplicate line if field is empty

Hi all, I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following: FILE: 1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100 2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY) 3. CDC, 54321, TEST3,... (4 Replies)
Discussion started by: tugar
4 Replies

LEARN ABOUT OPENDARWIN

aclcheck

aclcheck(3SEC)					       File Access Control Library Functions					    aclcheck(3SEC)

NAME

       aclcheck - check the validity of an ACL

SYNOPSIS

       cc [ flag... ] file... -lsec [ library... ]
       #include <sys/acl.h>

       int aclcheck(aclent_t *aclbufp, int nentries, int *which);

DESCRIPTION

       The  aclcheck()	function  checks the validity of an ACL pointed to by aclbufp. The nentries argument is the number of entries contained in
       the buffer. The which parameter returns the index of the first entry that is invalid.

       The function verifies that an ACL pointed to by aclbufp is valid according to the following rules:

	 o  There must be exactly one GROUP_OBJ ACL entry.

	 o  There must be exactly one USER_OBJ ACL entry.

	 o  There must be exactly one OTHER_OBJ ACL entry.

	 o  If there are any GROUP ACL entries, then the group ID in each group ACL entry must be unique.

	 o  If there are any USER ACL entries, then the user ID in each user ACL entry must be unique.

	 o  If there are any GROUP or USER ACL entries, then there must be exactly one CLASS_OBJ (ACL mask) entry.

	 o  If there are any default ACL entries, then the following apply:

	      o  There must be exactly one default GROUP_OBJ ACL entry.

	      o  There must be exactly one default OTHER_OBJ ACL entry.

	      o  There must be exactly one default USER_OBJ ACL entry.

	      o  If there are any DEF_GROUP entries, then the group ID in each DEF_GROUP ACL entry must be unique.

	      o  If there are any DEF_USER entries, then the user ID in each DEF_USER ACL entry must be unique.

	      o  If there are any DEF_GROUP or DEF_USER entries, then there must be exactly one DEF_CLASS_OBJ (default ACL mask) entry.

	 o  If any of the above rules are violated, then the function fails with errno set to  EINVAL.

RETURN VALUES

       If the ACL is valid, alcheck() will return 0. Otherwise	errno is set to EINVAL and return code is set to one of the following:

       GRP_ERROR	       There is more than one GROUP_OBJ or DEF_GROUP_OBJ ACL entry.

       USER_ERROR	       There is more than one USER_OBJ or DEF_USER_OBJ ACL entry.

       CLASS_ERROR	       There is more than one CLASS_OBJ (ACL mask) or DEF_CLASS_OBJ (default ACL mask) entry.

       OTHER_ERROR	       There is more than one OTHER_OBJ or DEF_OTHER_OBJ ACL entry.

       DUPLICATE_ERROR	       Duplicate entries of USER, GROUP, DEF_USER, or DEF_GROUP.

       ENTRY_ERROR	       The entry type is invalid.

       MISS_ERROR	       Missing an entry. The which parameter returns -1 in this case.

       MEM_ERROR	       The system cannot allocate any memory. The which parameter returns -1 in this case.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Evolving			   |
       +-----------------------------+-----------------------------+
       |MT-Level		     |Unsafe			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       acl(2), aclsort(3SEC), attributes(5)

SunOS 5.10							    10 Dec 2001 						    aclcheck(3SEC)
All times are GMT -4. The time now is 12:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy