I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Hi please help me how to remove duplicate lines in any file.
I have a file having huge number of lines.
i want to remove selected lines in it.
And also if there exists duplicate lines, I want to delete the rest & just keep one of them.
Please help me with any unix commands or even fortran... (7 Replies)
greetings,
i'm hoping there is a way to cat a file, remove duplicate lines and send that output to a new file. the file will always vary but be something similar to this:
please keep in mind that the above could be eight occurrences of each hostname or it might simply have another four of an... (2 Replies)
Hi,
I have two files with below data::
file1:-
123|aaa|ppp
445|fff|yyy
999|ttt|jjj
555|hhh|hhh
file2:-
445|fff|yyy
555|hhh|hhh
The records present in file1, not present in file 2 should be writtent to the out put file.
output:-
123|aaa|ppp
999|ttt|jjj
Is there any one line... (3 Replies)
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
hi,
Please help me to write a command to delete duplicate lines from a file. And the size of file is 50 MB. How to remove duplicate lins from such a big file. (6 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Hi, all
I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
LEARN ABOUT CENTOS
git-stripspace
GIT-STRIPSPACE(1) Git Manual GIT-STRIPSPACE(1)NAME
git-stripspace - Remove unnecessary whitespace
SYNOPSIS
git stripspace [-s | --strip-comments] < input
DESCRIPTION
Clean the input in the manner used by Git for text such as commit messages, notes, tags and branch descriptions.
With no arguments, this will:
o remove trailing whitespace from all lines
o collapse multiple consecutive empty lines into one empty line
o remove empty lines from the beginning and end of the input
o add a missing
to the last line if necessary.
In the case where the input consists entirely of whitespace characters, no output will be produced.
NOTE: This is intended for cleaning metadata, prefer the --whitespace=fix mode of git-apply(1) for correcting whitespace of patches or
files in the repository.
OPTIONS -s, --strip-comments
Skip and remove all lines starting with comment character (default #).
-c, --comment-lines
Prepend comment character and blank to each line. Lines will automatically be terminated with a newline. On empty lines, only the
comment character will be prepended.
EXAMPLES
Given the following noisy input with $ indicating the end of a line:
|A brief introduction $
| $
|$
|A new paragraph$
|# with a commented-out line $
|explaining lots of stuff.$
|$
|# An old paragraph, also commented-out. $
| $
|The end.$
| $
Use git stripspace with no arguments to obtain:
|A brief introduction$
|$
|A new paragraph$
|# with a commented-out line$
|explaining lots of stuff.$
|$
|# An old paragraph, also commented-out.$
|$
|The end.$
Use git stripspace --strip-comments to obtain:
|A brief introduction$
|$
|A new paragraph$
|explaining lots of stuff.$
|$
|The end.$
GIT
Part of the git(1) suite
Git 1.8.3.1 06/10/2014 GIT-STRIPSPACE(1)