I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
I have following file content (3 fields each line):
23 888 10.0.0.1
dfh 787 10.0.0.2
dssf dgfas 10.0.0.3
dsgas dg 10.0.0.4
df dasa 10.0.0.5
df dag 10.0.0.5
dfd dfdas 10.0.0.5
dfd dfd 10.0.0.6
daf nfd 10.0.0.6
...
as can be seen, that the third field is ip address and sorted. but... (3 Replies)
Hi,
I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates.
Thanks in advance,
sudvishw :confused: (7 Replies)
Hi, I have a huge file which is about 50GB. There are many lines. The file format likes
21 rs885550 0 9887804 C C T C C C C C C C
21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0
21 rs303304 0 9941889 A A A A A A A A A A
22 rs303304 0 9941890 0 A A A A A A A A A
The question is that there are a few... (4 Replies)
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
Trying to cut down the size of some log files. Now that I write this out it looks more dificult than i thought it would be.
Need a bash script or command that goes sequentially through all lines of a file, and does this:
if field1 (space separated) is the number 2012 print the entire line. Do... (7 Replies)
Hi,
I have a csv file which contains some millions of lines in it.
The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line).
I don't want to use any pattern from the Header as I have some... (7 Replies)
Within my text file i have several thousand lines of text with some lines containing duplicate strings/words. I would like to entirely remove those lines which contain the duplicate strings.
Eg;
One and a Two
Unix.com is the Best
This as a Line Line
Example duplicate sentence with the word... (22 Replies)
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
LEARN ABOUT DEBIAN
control.ctl
CONTROL.CTL(5) File Formats Manual CONTROL.CTL(5)NAME
control.ctl - specify handling of Usenet control messages
DESCRIPTION
The file /etc/news/control.ctl is used to determine what action is taken when a control message is received. It is read by the parsecon-
trol script, which is called by all the control scripts. (For an explanation of how the control scripts are invoked, see innd(8).)
The file consists of a series of lines; blank lines and lines beginning with a number sign (``#'') are ignored. All other lines consist of
four fields separated by a colon:
message:from:newsgroups:action
The first field is the name of the message for which this line is valid. It should be either the name of the control message, or the word
``all'' to mean that it is valid for all messages.
The second field is a shell-style pattern that matches the email address of the person posting the message. (The poster's address is first
converted to lowercase.) The matching is done using the shell's case statement; see sh (1) for details.
If the control message is ``newgroup'' or ``rmgroup'' then the third field specifies the shell-style pattern that must match the group
being created or removed. If the control message is of a different type, then this field is ignored.
The fourth field specifies what action to take if this line is selected for the message. The following actions are understood:
doit The action requested by the control message should be performed. In most cases the control script will also send mail to usenet.
doifarg
If the control message has an argument, this is treated as a ``doit'' action. If no argument was given, it is treated as a ``mail''
entry. This is used in ``sendsys'' entries script so that a site can request its own newsfeeds(5) entry by posting a ``sendsys
mysite'' article. On the other hand, sendsys ``bombs'' ask that the entire newsfeeds file be sent to a forged reply-to address; by
using ``doifarg'' such messages will not be processed automatically.
doit=file
The action is performed, but a log entry is written to the specified log file, file. If file is the word ``mail'' then the record
is mailed. A null string is equivalent to /dev/null. A pathname that starts with a slash is taken as the absolute filename to use
as the log. All other pathnames are written to /var/log/news/file.log. The log is written by writelog (see newslog(8)).
drop No action is taken; the message is ignored.
log A one-line log notice is sent to standard error. Innd normally directs this to the file /var/log/news/errlog.
log=file
A log entry is written to the specified log file, file, which is interpreted as described above.
mail A mail message is sent to the news administrator.
Lines are matched in order; the last match found in the file is the one that is used. For example, with the following three lines:
newgroup:*:*:drop
newgroup:tale@*.uu.net:comp.*|misc.*|news.*|rec.*|sci.*|soc.*|talk.*:doit
newgroup:kre@munnari.oz.au:aus.*:mail
A newgroup coming from ``tale'' at a UUNET machine will be honored if it is in the mainstream Usenet hierarchy. If ``kre'' posts a new-
group message creating ``aus.foo'', then mail will be sent. All other newgroup messages are ignored.
HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.8, dated 1996/09/06.
SEE ALSO innd(8), newsfeeds(5), scanlogs(8).
CONTROL.CTL(5)