Sponsored Content
Top Forums UNIX for Dummies Questions & Answers removing line and duplicate line Post 302104913 by sb008 on Monday 29th of January 2007 01:48:06 PM
Old 01-29-2007
Could it be as simple as
sort -u
?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies

2. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

3. Shell Programming and Scripting

Duplicate Line Report per Section

I've been working on a script (/bin/sh) in which I have requested and received help here (in which I am very grateful for!). The client has modified their requirements (a tad), so without messing up the script to much, I come once again for assistance. Here are the file.dat contents: ABC1... (4 Replies)
Discussion started by: petersf
4 Replies

4. UNIX for Dummies Questions & Answers

Delete duplicate second line

Hi ALL I need a help I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that Eg: Input =035 (ABC)12324141241 =035 (XYZPQR)704124 =040 AB$QS$WEWR =035 (ABC)08080880809 =035 (XYZPQR)9809314 =040 ... (4 Replies)
Discussion started by: umapearl
4 Replies

5. Shell Programming and Scripting

Removing a line IF the next line contains string

So, I've been working on a project which takes layer 7 metadata from pcap dumps and archives it. However, there is a lot of dataless information that I don't want in my output. I know of ways to produce the output I want from the input file below, but I want a method of doing this, regardless of... (2 Replies)
Discussion started by: eh3civic
2 Replies

6. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

7. UNIX for Dummies Questions & Answers

Duplicate a line n times

I want a command equivalent to sed 'p;p' file My input file contains 1 2 3 If i want to input file like this 1 1 1 2 2 2 3 3 3 I use (2 Replies)
Discussion started by: nsuresh316
2 Replies

8. Shell Programming and Scripting

Duplicate third column to every line

Dear All, I have file input like this: INP901 5173 4114 INP902 5227 INP903 5284 INP904 5346 INP905 5400 INP906 5456 INP907 5511 INP908 5572 INP909 5622 INP910 5678 INP911 5739 INP912 5796 INP913 5845 INP914 5910 INP915 5965 (2 Replies)
Discussion started by: attila
2 Replies

9. Shell Programming and Scripting

Honey, I broke awk! (duplicate line removal in 30M line 3.7GB csv file)

I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code: awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies

10. UNIX for Beginners Questions & Answers

Get an output of lines in pattern 1st line then 10th line then 11th line then 20th line and so on.

Input file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (6 Replies)
Discussion started by: Sagar Singh
6 Replies
aclsort(3C)															       aclsort(3C)

NAME
aclsort() - sort an Access Control List (JFS File Systems only) SYNOPSIS
DESCRIPTION
The routine sorts JFS Access Control List (ACL) entries into the correct order to be accepted by the acl(2) system call. points to a buffer containing ACL entries; if non-zero, indicates that the permissions should be recalculated; and specifies the number of ACL entries in the buffer. sorts the contents of the ACL buffer as follows: Entries will be in order and Entries of type and will be sorted in increasing order by numeric ID. The call will succeed if all of the following are true: There is exactly one entry each of type and There is at most one entry each of type and Entries of type or may not contain duplicate entries. A duplicate entry is one of the same type containing the same numeric id, irrespective of permission bits. If the argument is zero and there are no entries of type and no entries of type the permissions of the and entries must be the same. If there are no entries of type and no entries of type and the entry is specified, then the entry must also be specified, and the permissions of the and entries must be the same. RETURN VALUE
Upon successful completion, the return value is 0. If there are duplicate entries, the return value is the position of the first duplicate entry. If there is more than one entry of type or they are treated as duplicate entries, and the return value is the position of the first duplicate entry. For all other errors, the return value is -1. NOTICES
The buffer is sorted by type and ID before checking for any failures. Therefore the buffer is always sorted, even if there is a failure. The position of a duplicate entry returned on failure is not the byte offset of the duplicate entry from its base; rather it refers to the entry number of the duplicate entry within the sorted buffer. Checks will be performed in order of entry type. If there are multiple failures, the failure returned will be the first encountered, for example, if the ACL buffer contains a duplicate entry and does not contain an entry, the return value will be the first duplicate entry. ACLs do not have to be sorted with prior to passing them to acl(2). DEPENDENCIES
is supported only on JFS file systems on the standard HP-UX operating system. AUTHOR
was developed by AT&T. SEE ALSO
acl(2), aclv(5). aclsort(3C)
All times are GMT -4. The time now is 12:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy