06-08-2011
The sample you posted is "line numbered" is it the output of an awk command of yours ?
If so, show it to us and provide and example of the very initial input file you have, before any formatting.
---------- Post updated at 12:33 AM ---------- Previous update was at 12:30 AM ----------
When you state :
"I need to remove the duplicate row"
Do you means that 2 same consecutive lines should :
a) appear only once ?
or
b) should not appear at all ?
This User Gave Thanks to ctsgnb For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
i have an input below.
As long as "x= 1" , i would want to capture 2 lines using sed or awk
for eg :
0001 x= 1 $---------------------------------..-.--..
0001 tt= 137 171 423 1682 2826 0
Pls help. Thanks in advance.
Note that the number of lines in each block do... (37 Replies)
Discussion started by: Raynon
37 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g.
COL1 COL2 COL3
A 1234 1234
B 3k32 2322
C Xk32 TTT
A NEW XX22
B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies
3. Shell Programming and Scripting
i have the long file more than one ns and www and mx in the line like .
i need the first ns record and first www and first mx from line .
the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution.
... (4 Replies)
Discussion started by: kiranmosarla
4 Replies
4. Shell Programming and Scripting
Hi,
I have a log file having size of 48mb.
For such a large log file. I want to get the message in a particular format which includes only unique error and exception messages.
The following things to be done :
1) To remove all the date and time from the log file
2) To remove all the... (1 Reply)
Discussion started by: Pank10
1 Replies
5. Shell Programming and Scripting
I need help with a script that will remove all HTML tags from an HTML document and remove any consecutive duplicate lines, and save it as a text document. The user should have the option of including the name of an html file as an argument for the script, but if none is provided, then the script... (7 Replies)
Discussion started by: clicstic
7 Replies
6. UNIX for Dummies Questions & Answers
Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column.
e.g.
a 1
a 2
a 3
b 1
c 1
gives
b 1
c 1
but requires 11 duplicates before it deletes.
Thanks for the help
Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies
7. Shell Programming and Scripting
Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed.
example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies
8. Shell Programming and Scripting
Hello all, I am working on a file like below:
site Date time value1 value2
0023 2014-01-01 00:00 32.0 23.7
0023 2014-01-01 01:00 38.0 29.9
0023 2014-01-01 02:00 85.0 26.6
0023 2014-01-01 03:00 34.0 25.3
0023 2014-01-01 04:00 37.0 23.8
0023 2014-01-01 05:00 80.0 20.3
0023 2014-01-01 06:00... (16 Replies)
Discussion started by: kathy wang
16 Replies
9. Shell Programming and Scripting
Hi,
In an ideal scenario, I will have a listing of db transaction log that gets copied to a DR site and if I have them all, they will be numbered consecutively like below.
1_79811_01234567.arc
1_79812_01234567.arc
1_79813_01234567.arc
1_79814_01234567.arc
1_79815_01234567.arc... (3 Replies)
Discussion started by: newbie_01
3 Replies
10. Shell Programming and Scripting
Hello,
I'm trying to remove the duplicate consecutive lines with specific string "WARNING".
File.txt
abc;
WARNING 2345
WARNING 2345
WARNING 2345
WARNING 2345
WARNING 2345
bcd;
abc;
123
123
123
WARNING 1234
WARNING 2345
WARNING 2345
efgh; (6 Replies)
Discussion started by: Mannu2525
6 Replies
SPLIT(1) FSF SPLIT(1)
NAME
split - split a file into pieces
SYNOPSIS
split [OPTION] [INPUT [PREFIX]]
DESCRIPTION
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default PREFIX is `x'. With no INPUT, or when INPUT is -, read standard
input.
Mandatory arguments to long options are mandatory for short options too.
-a, --suffix-length=N
use suffixes of length N (default 2)
-b, --bytes=SIZE
put SIZE bytes per output file
-C, --line-bytes=SIZE
put at most SIZE bytes of lines per output file
-l, --lines=NUMBER
put NUMBER lines per output file
--verbose
print a diagnostic to standard error just before each output file is opened
--help display this help and exit
--version
output version information and exit
SIZE may have a multiplier suffix: b for 512, k for 1K, m for 1 Meg.
AUTHOR
Written by Torbjorn Granlund and Richard M. Stallman.
REPORTING BUGS
Report bugs to <bug-coreutils@gnu.org>.
COPYRIGHT
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
LAR PURPOSE.
SEE ALSO
The full documentation for split is maintained as a Texinfo manual. If the info and split programs are properly installed at your site,
the command
info split
should give you access to the complete manual.
split (coreutils) 4.5.3 February 2003 SPLIT(1)