12-15-2011
Quote:
Originally Posted by
gowtham.varma
cat <inputfile> | sort -n | uniq
1) You don't need to use cat there.
2) Why are you using numeric sorting? I'm gonna guess the -n option doesn't do what you think it does.
3) "sort | uniq" == "sort -u"
4) This might be a bad solution. The OP only wants to get rid of duplicate headers. We can't assume there are no other duplicate lines in his input.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I'm sure this will be an easy question for you experts out there, but I have been searching the forum and working on this for a couple hours now and can't get it right.
I have a very messy data file that I am trying to tidy up - one of the issues is some records are split into multiple lines: ... (4 Replies)
Discussion started by: tink
4 Replies
2. Shell Programming and Scripting
Hi,
Please provide shell script to Remove empty lines(space) between two lines containing strings in a file.
Input File :
A1/EXT "BAP_BSC6/07B/00" 844 090602 1605
RXOCF-465 PDTR11 1
SITE ON BATTERY
A2/EXT... (3 Replies)
Discussion started by: sudhakaryadav
3 Replies
3. Shell Programming and Scripting
I have a file like this..
Maharastra Mumbai worli
Maharastra Mumbai navy
maharatra Pune
Maharastra Nagpur
Karnataka Bangalore
Karnataka Mysore
Karnataka Mangalore
Punjab Amritsar
punjab Jalandar
my expected outcome should be like this
Maharastra Mumbai worli
---------- ... (9 Replies)
Discussion started by: geeko
9 Replies
4. UNIX for Dummies Questions & Answers
Hi,
I have 1000 line text file.
I need only the 1st and 8th line from every set of 10 lines, that is,
1,8,11,18,21,21,28,31,38,... lines etc into a text file.
Please let me know how I can achieve the same.
Regards,
Don (2 Replies)
Discussion started by: donisback
2 Replies
5. UNIX for Dummies Questions & Answers
I have output like this:
USER_ID
12/31/69 19:00:00
12/31/69 19:00:00
USER_ID
12/31/69 19:00:00
12/31/69 19:00:00
USER_ID
12/31/69 19:00:00
12/31/69 19:00:00
USER_ID
12/31/69 19:00:00
12/31/69 19:00:00
...
where USER_ID is a unique user login followed by their login timestamp and... (6 Replies)
Discussion started by: MaindotC
6 Replies
6. Shell Programming and Scripting
I would like to remove carriage returns/line feeds in a text file, but in a specific cadence:
Read first line (Header Line 1), remove cr/lf at the end (replace it with a space ideally);
Read the next line (Line of Text 2), leave the cr/lf intact;
Read the next line, remove the cr/lf;
Read... (14 Replies)
Discussion started by: tomr2012
14 Replies
7. Shell Programming and Scripting
I have a file in which the data is stored in pairs of lines. The first line (beginining with ">") is a header, the second line is a sequence.
I would like to sort the file by species name. Desired output for the example file:
I can use
sort -t'_' -k2
to alphabetize headers in the... (1 Reply)
Discussion started by: pathunkathunk
1 Replies
8. UNIX for Dummies Questions & Answers
Hi Everybody! First post! Totally noobie.
I'm using the terminal to read a poorly formatted book.
The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this.
I was hoping to use grep -v " -- "... (5 Replies)
Discussion started by: AxeHandle
5 Replies
9. Shell Programming and Scripting
Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char.
thanks in advance
Source:-
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver;... (7 Replies)
Discussion started by: Jackceasar123
7 Replies
10. UNIX for Beginners Questions & Answers
Input file:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 (6 Replies)
Discussion started by: Sagar Singh
6 Replies
SORT(1) General Commands Manual SORT(1)
NAME
sort - sort a file of ASCII lines
SYNOPSIS
sort [-bcdfimnru] [-tc] [-o name] [+pos1] [-pos2] file ...
OPTIONS
-b Skip leading blanks when making comparisons
-c Check to see if a file is sorted
-d Dictionary order: ignore punctuation
-f Fold upper case onto lower case
-i Ignore nonASCII characters
-m Merge presorted files
-n Numeric sort order
-o Next argument is output file
-r Reverse the sort order
-t Following character is field separator
-u Unique mode (delete duplicate lines)
EXAMPLES
sort -nr file # Sort keys numerically, reversed
sort +2 -4 file # Sort using fields 2 and 3 as key
sort +2 -t: -o out # Field separator is :
sort +.3 -.6 # Characters 3 through 5 form the key
DESCRIPTION
Sort sorts one or more files. If no files are specified, stdin is sorted. Output is written on standard output, unless -o is specified.
The options +pos1 -pos2 use only fields pos1 up to but not including pos2 as the sort key, where a field is a string of characters delim-
ited by spaces and tabs, unless a different field delimiter is specified with -t. Both pos1 and pos2 have the form m.n where m tells the
number of fields and n tells the number of characters. Either m or n may be omitted.
SEE ALSO
comm(1), grep(1), uniq(1).
SORT(1)