Remove lines that are subsets of other lines in File
Hello everyone,
Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done.
My problem is the following:
I have a file where each line is a sequence of IP addresses, example :
Code:
10.0.0.1 10.0.0.2
10.0.0.5 10.0.0.1 10.0.0.2
...
What I'd like to do, is to remove lines that are completely matched in other lines. In the previous example, "Line 1" would be deleted as it is contained in "Line 2".
So far, I've worked with python and set() objects to get the job done but I've got more than 100K lines and sets lookups are becoming time consuming as the program goes :/
Thanks for you help
Moderator's Comments:
Use code tags, thanks.
Last edited by zaxxon; 04-22-2015 at 06:31 AM..
Reason: code tags and missing a dot
All,
I have a text file with several entries like below:
personname
personname.domain.com
I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax.
Thanks (2 Replies)
Hi gurus,
i'm trying to remove a number of lines from a large file using the following command:
sed '1,5000d' oldfile > newfile
Somehow the lines in the old file are not deleted...
Am I doing this wrongly? Any suggestions? :confused:
Thanks! :)
wee (10 Replies)
A small question
I have a test.txt file
I have contents as:
a:google
b:yahoo
:
c:facebook
:
d:hotmail
How do I remove the line with :
my output should be
a:google
b:yahoo
c:facebook
d:hotmail (5 Replies)
Hi,
I'm not a expert in shell programming, so i've come here to take help from u gurus.
I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command.
I've a datatable csv of the below format -
--in file format
xx,xx,xx ,xx , , , , ,,xx,
xxxx,, ,, xxx,... (11 Replies)
Hey Gang-
I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this?
Example
wvm1234dev
wvm1234pro
uvm1122dev
uvm1122bku
uvm1344dev
I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
Hi,
I have a huge file which has Lacs of lines. File system got full.
I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
I have a file that contains the following:
Party_Id1;Party_id2;Party_id3;
1;2;3;
0
0
4;5;6;
0
7;8;9;
How can I adjust the file so it looks like this:
Party_Id1;Party_id2;Party_id3;
1;2;3;
4;5;6;
7;8;9;
I Think the '0' is something like a carriage return, I don't know. But how... (2 Replies)
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.
keepout:
user1
buser3
anuser19
notheruser27
database:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
I have been searching and trying to come up with an awk that will perform the following on a
converted text file (original is a pdf).
1. Since the first two lines are (begin with) text they are removed
2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies
LEARN ABOUT SUSE
mod-active
MOD-ACTIVE(8) InterNetNews Documentation MOD-ACTIVE(8)NAME
mod-active - Batch processing of newsgroups creation and removal commands
SYNOPSIS
mod-active [ctlinnd-command-file ...]
DESCRIPTION
mod-active is a Perl script that updates the active file based on its input lines of ctlinnd "newgroup", "rmgroup" and "changegroup"
commands. It pauses the server briefly while the existing active file is read and rewritten, which not only keeps innd from updating the
active file but also locks against other instances of mod-active.
The script must be run as the news user.
The input to mod-active can come either from one or more ctlinnd-command-file files named on the command line, or from the standard input.
Typically its input is the output from the docheckgroups or actsync commands. Every line which contains the string "ctlinnd newgroup",
"ctlinnd rmgroup", or "ctlinnd changegroup", optionally preceded by whitespace and/or the path to ctlinnd, is noted for the update.
Redundant commands, such as a newgroup directive for a group that already exists, are silently ignored. All other lines in the input are
also silently ignored. After the new active file has been generated, the existing one is renamed to active.old and the new one is moved
into place. The script then displays the differences between the two files. Any groups that were added to the active file are also added
to the active.times file with the string "checkgroups-update".
Please note that no syntax checking is performed on group names by mod-active.
BUGS
Though innd is paused while mod-active works, it is not inconceivable that there could be a conflict if something else tries to update the
active file during the relatively short time that mod-active is working. The two most realistic ways for this to happen are either by an
administrator concurrently doing a manual ctlinnd command, or by innd receiving a control message, then mod-active pausing the server, then
the control message handler script that innd forked running its own ctlinnd command while mod-active is working. Note that such scenarios
are very unlikely to happen.
HISTORY
Written by David C Lawrence <tale@isc.org> for InterNetNews. Converted to POD by Julien Elie.
SEE ALSO active(5), active.times(5), actsync(8), ctlinnd(8), docheckgroups(8), innd(8).
INN 2.5.2 2009-05-21 MOD-ACTIVE(8)
04-22-2015
