I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
I have following file content (3 fields each line):
23 888 10.0.0.1
dfh 787 10.0.0.2
dssf dgfas 10.0.0.3
dsgas dg 10.0.0.4
df dasa 10.0.0.5
df dag 10.0.0.5
dfd dfdas 10.0.0.5
dfd dfd 10.0.0.6
daf nfd 10.0.0.6
...
as can be seen, that the third field is ip address and sorted. but... (3 Replies)
Hi,
I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates.
Thanks in advance,
sudvishw :confused: (7 Replies)
Hi, I have a huge file which is about 50GB. There are many lines. The file format likes
21 rs885550 0 9887804 C C T C C C C C C C
21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0
21 rs303304 0 9941889 A A A A A A A A A A
22 rs303304 0 9941890 0 A A A A A A A A A
The question is that there are a few... (4 Replies)
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
Trying to cut down the size of some log files. Now that I write this out it looks more dificult than i thought it would be.
Need a bash script or command that goes sequentially through all lines of a file, and does this:
if field1 (space separated) is the number 2012 print the entire line. Do... (7 Replies)
Hi,
I have a csv file which contains some millions of lines in it.
The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line).
I don't want to use any pattern from the Header as I have some... (7 Replies)
Within my text file i have several thousand lines of text with some lines containing duplicate strings/words. I would like to entirely remove those lines which contain the duplicate strings.
Eg;
One and a Two
Unix.com is the Best
This as a Line Line
Example duplicate sentence with the word... (22 Replies)
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
LEARN ABOUT OPENSOLARIS
grablogs.conf
grablogs.conf(4) File Formats grablogs.conf(4)NAME
grablogs.conf - grablogs configuration for libgrablogs.so of the plugins of
gnome-system-log file
SYNOPSIS
/usr/lib/gnome-system-log/plugins/grablogs.conf
DESCRIPTION
The libgrablogs.so is a plugin for gnome-system-log(1), it colloct the log files from the system as many as possible. grablogs.conf is a
configuration file that contains a set of lines mixed with sh(1) syntax codes and individual
log files. libgrablogs.so will read the file try to get a log files list for
gnome-system-log(1). Users can copy the file into $HOME/.gnome2/gnome-system-log/plugins/`uname -p` to overwrite the system default one.
The grablogs.conf file contains the following configuration categories:
[configs]
Each line under this category is interpreted as a config file of System. The plugin will open the config file and try to find
all system paths of the logs.
[commands]
Each line under this category is interpreted as a shell command and will be execute through a pipe. And each line of the out-
put of the command will be interpreted as a log path.
[logs] Each line under this category is interpreted as a log path.
FILES
/usr/lib/gnome-system-log/plugins/grablogs.conf
The system default configuration file for the plugin libgrablogs.so
$HOME/.gnome2/gnome-system-log/plugins/`uname -p`/grablogs.conf
The user specific configuration file for the plugin libgrablogs.so
EMAMPLE
[configs]
/etc/syslog.conf
[commands]
for i in `svcs -aH -o FMRI | grep -v lrc `; do svcprop -p restarter/logfile $i 2>/dev/null || svcprop -q-p restarter/alt_logfile $i
2>/dev/null ; done
[logs]
/var/log/messages
/var/log/secure
/var/log/maillog
/var/log/cron
/var/log/Xorg.0.log
/var/log/XFree86.0.log
/var/log/auth.log
/var/log/cups/error_log
SEE ALSO gnome-system-log(1), pipelog.conf(1)gnome-utils 2.16.0 13 Oct 2006 grablogs.conf(4)