01-27-2009
Might need a little more info.
What are the file contents?
You also need to redirect and then overwrite the original file.
sort -n $file | uniq > $ftemp
This ~should~ give you a numerically sorted file --- with all duplicate
lines removed.
HOWEVER!!
If you're thinking of changing this:
1 ant
3 cat
2 bat
4 ant
to this:
1 ant
2 bat
3 cat
No, that won't work.
10 More Discussions You Might Find Interesting
1. Infrastructure Monitoring
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead... (4 Replies)
Discussion started by: Nysif Steve
4 Replies
2. Shell Programming and Scripting
I have a file which contains data as below:
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf
nbk1j7o pageName=/jsp/common/index.jsf
nbk1j7o pageName=/jsp/common/index.jsf
nbk1wqe... (6 Replies)
Discussion started by: mohsin.quazi
6 Replies
3. Shell Programming and Scripting
Hello folks
I have a question for you gurus of sed or grep (maybe awk, but I would prefer the first two)
I have a file (f1) that says:
(actually, these are not numbers but md5sum, but for simplicity, let's assume these numbers.)
1
2
3
4
5And I have a file (f2) that says
1|a
1|b
1|c
2|d... (3 Replies)
Discussion started by: tukuyomi
3 Replies
4. UNIX for Dummies Questions & Answers
Hello! i have a text file.. which contains the data as follows
i want to merge the declarations lines pertaining to one datatype in to a single line as follows
i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I have a file with the lines as below:
C_10_A05_T7
C_10_A06_SP6
C_10_B05_SP6
C_10_B05_T7
C_10_B01_SP6
C_10_B01_T7
C_12_G07_SP6
C_12_G11_SP6
C_12_G11_T7
C_2_H18_T7
C_2_I02_SP6
C_2_I02_T7
C_2_I13_SP6
C_2_I17_SP6
The four segments of each line are connected by '_' symbols. I... (7 Replies)
Discussion started by: Fahmida
7 Replies
6. Shell Programming and Scripting
consider i have two files
cat onlyviews1.sql
CREATE VIEW V11
AS
SELECT id,
name,
FROM
etc etc
WHERE etc etc;
CREATE VIEW V22
AS
SELECT id,
name,
FROM
etc etc
WHERE etc etc;
CREATE VIEW V33
AS (10 Replies)
Discussion started by: vivek d r
10 Replies
7. Shell Programming and Scripting
Hello,
got a file with this structure:
33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26
33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27
33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26
33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies
8. Shell Programming and Scripting
Hello,
I am a java programmer but want to try unix for a purpose where I need to reduce a file using its first field.. Here is the sample data:
admin;2;0;;
admission;8;0;;
aman;1;0;;
caroline;0;4;;
cook;0;4;;
cook;2;0;;
far;0;3;;
far;1;5;;
I am explaining the dataset first. There... (5 Replies)
Discussion started by: shekhar2010us
5 Replies
9. Shell Programming and Scripting
Hi !
I am trying to remove doubbled entrys in a textfile only between delimiters.
Like that example but i dont know how to do that with sort or similar.
input:
{
aaa
aaa
}
{
aaa
aaa
}
output:
{
aaa
}
{ (8 Replies)
Discussion started by: fugitivus
8 Replies
10. UNIX for Dummies Questions & Answers
URGENT HELP IS NEEDED!!
I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but
- Currently, script is not printing lines to File3.txt in order.
- Also the matching lines are not moving out of File1.txt
... (1 Reply)
Discussion started by: High-T
1 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Discard all but one of successive identical lines from INPUT (or standard input), writing to OUTPUT (or standard output).
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines
-D, --all-repeated[=delimit-method]
print all duplicate lines delimit-method={none(default),prepend,separate} Delimiting is done with blank lines.
-f, --skip-fields=N
avoid comparing the first N fields
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
end lines with 0 byte, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use `sort -u' without
`uniq'.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
Report uniq bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
COPYRIGHT
Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info coreutils 'uniq invocation'
should give you access to the complete manual.
GNU coreutils 7.1 July 2010 UNIQ(1)