01-18-2016
Thanks MadeInGermany, forgot about that one, added to my post...
Last edited by Scrutinizer; 01-18-2016 at 01:46 PM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have following file content (3 fields each line):
23 888 10.0.0.1
dfh 787 10.0.0.2
dssf dgfas 10.0.0.3
dsgas dg 10.0.0.4
df dasa 10.0.0.5
df dag 10.0.0.5
dfd dfdas 10.0.0.5
dfd dfd 10.0.0.6
daf nfd 10.0.0.6
...
as can be seen, that the third field is ip address and sorted. but... (3 Replies)
Discussion started by: fredao
3 Replies
2. UNIX for Dummies Questions & Answers
Hey all, a relative bash/script newbie trying solve a problem.
I've got a text file with lots of lines that I've been able to clean up and format with awk/sed/cut, but now I'd like to remove the lines with duplicate usernames based on time stamp. Here's what the data looks like
2007-11-03... (3 Replies)
Discussion started by: mattv
3 Replies
3. Shell Programming and Scripting
Hi, I have a huge file which is about 50GB. There are many lines. The file format likes
21 rs885550 0 9887804 C C T C C C C C C C
21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0
21 rs303304 0 9941889 A A A A A A A A A A
22 rs303304 0 9941890 0 A A A A A A A A A
The question is that there are a few... (4 Replies)
Discussion started by: zhshqzyc
4 Replies
4. Shell Programming and Scripting
The question is not as simple as the title... I have a file, it looks like this
<string name="string1">RZ-LED</string>
<string name="string2">2.0</string>
<string name="string2">Version 2.0</string>
<string name="string3">BP</string>
I would like to check for duplicate entries of... (11 Replies)
Discussion started by: raidzero
11 Replies
5. Shell Programming and Scripting
Hello Folks,
Can some one help me with the removal of duplicate lines from a log file and send it to another log file. It's bit complicated as two lines are same but only difference is the timestamp, but some lines are uniq. Line has been seperated by colon's.
Log file:... (5 Replies)
Discussion started by: sim_je
5 Replies
6. Shell Programming and Scripting
hi to all
Does anyone know if there's a way to remove duplicate lines which we consider the same only if they have the first and the second column the same?
For example I have :
us2333 bbb 5
us2333 bbb 3
us2333 bbb 2
and I want to get
us2333 bbb 10
The thing is I cannot... (2 Replies)
Discussion started by: vlm
2 Replies
7. UNIX for Dummies Questions & Answers
Hi
I need this output. Thanks.
Input:
TAZ
YET
FOO
FOO
VAK
TAZ
BAR
Output:
YET
VAK
BAR (10 Replies)
Discussion started by: tara123
10 Replies
8. Shell Programming and Scripting
if I have the following lines in a file app.log
some lines here
<AAAA>
abc
<id>123456789</id>
ddd
</AAAA>some lines here too
<BBBB>
abc
<id>123456789</id>
ddd
</BBBB>some lines here too
<AAAA>
xyz
<id>987654321</id>
ssss
</AAAA>some lines here again...
How do I get the... (5 Replies)
Discussion started by: nariwithu
5 Replies
9. Shell Programming and Scripting
Hi,
I have a csv file which contains some millions of lines in it.
The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line).
I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies
10. Shell Programming and Scripting
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
LEARN ABOUT DEBIAN
text::trim
Text::Trim(3pm) User Contributed Perl Documentation Text::Trim(3pm)
NAME
Text::Trim - remove leading and/or trailing whitespace from strings
VERSION
version 1.02
SYNOPSIS
use Text::Trim;
$text = " important data
";
$data = trim $text;
# now $data contains "important data" and $text is unchanged
# or:
trim $text; # work in-place, $text now contains "important data"
@lines = <STDIN>;
rtrim @lines; # remove trailing whitespace from all lines
# Alternatively:
@lines = rtrim <STDIN>;
# Or even:
while (<STDIN>) {
trim; # Change $_ in place
# ...
}
DESCRIPTION
This module provides functions for removing leading and/or trailing whitespace from strings. It is basically a wrapper around some simple
regexes with a flexible context-based interface.
EXPORTS
All functions are exported by default.
CONTEXT HANDLING
void context
Functions called in void context change their arguments in-place
trim(@strings); # All strings in @strings are trimmed in-place
ltrim($text); # remove leading whitespace on $text
rtrim; # remove trailing whitespace on $_
No changes are made to arguments in non-void contexts.
list context
Values passed in are changed and returned without affecting the originals.
@result = trim(@strings); # @strings is unchanged
@result = rtrim; # @result contains rtrimmed $_
($result) = ltrim(@strings); # like $result = ltrim($strings[0]);
scalar context
As list context but multiple arguments are stringified before being returned. Single arguments are unaffected. This means that under
these circumstances, the value of $" ($LIST_SEPARATOR) is used to join the values. If you don't want this, make sure you only use single
arguments when calling in scalar context.
@strings = (" hello
", " there
");
$trimmed = trim(@strings);
# $trimmed = "hello there"
local $" = ', ';
$trimmed = trim(@strings);
# Now $trimmed = "hello, there"
$trimmed = rtrim;
# $trimmed = $_ minus trailing whitespace
Undefined values
If any of the functions are called with undefined values, the behaviour is in general to pass them through unchanged. When stringifying a
list (calling in scalar context with multiple arguments) undefined elements are excluded, but if all elements are undefined then the return
value is also undefined.
$foo = trim(undef); # $foo is undefined
$foo = trim(undef, undef); # $foo is undefined
@foo = trim(undef, undef); # @foo contains 2 undefined values
trim(@foo) # @foo still contains 2 undefined values
$foo = trim('', undef); # $foo is ''
FUNCTIONS
trim
Removes leading and trailing whitespace from all arguments, or $_ if none are provided.
rtrim
Like trim() but removes only trailing (right) whitespace.
ltrim
Like trim() but removes only leading (left) whitespace.
UNICODE
Because this module is implemented using perl regular expressions, it is capable of recognising and removing unicode whitespace characters
(such as non-breaking spaces) from scalars with the utf8 flag on. See Encode for details about the utf8 flag.
Note that this only applies in the case of perl versions after 5.8.0 or so.
SEE ALSO
Brent B. Powers' String::Strip performs a similar function in XS.
AUTHOR
Matt Lawrence <mattlaw@cpan.org>
ACKNOWLEDGEMENTS
Terrence Brannon <metaperl@gmail.com> for bringing my attention to String::Strip and suggesting documentation changes.
perl v5.10.1 2010-06-07 Text::Trim(3pm)