10-29-2013
Hi.
As alister said.
This behavior has been discussed in the forum, for example, I mentioned
sponge / absorb for pipelines and
re-writing-in-place in post #8 of
https://www.unix.com/shell-programming-and-scripting/221749-run-command-without-over-writing.html -- that approach either keeps data in memory (say a
perl array), or writes it to a scratch file first, both of which can preserve the inode if that is important. The
sponge approach
looks like the
tee solution, but the internal logic differs by
sponge collecting all the data until the end,
then writing to the file argument, whereas
tee streams the data to the file(s) as it comes in.
We cannot use the scratch file approach here because there is no additional disk space, so suggestions like the shell and dd above seemed appropriate. Both were validated on a file of 14M lines (~ 1 GB) with 50,000 lines being kept.
Best wishes ... cheers, drl
Last edited by drl; 10-30-2013 at 11:53 AM..
Reason: Minor typo.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
file:
1 xxxxxxx
2 xxx xxx
5 xxx xxx
...
180 xxxxxx
200 xxx
how to remove any lines with the first number range 1-180 (9 Replies)
Discussion started by: bluemoon1
9 Replies
2. Shell Programming and Scripting
Hi,
There seems to some hack attempts in my site. I have attached the index page of my site and I need to remove the below lines from the index page. The below lines are at the center of the file.
-->
</style>
<script>E V A L( unescape(... (5 Replies)
Discussion started by: gsiva
5 Replies
3. UNIX for Dummies Questions & Answers
All,
I have a text file with several entries like below:
personname
personname.domain.com
I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax.
Thanks (2 Replies)
Discussion started by: kjbaumann
2 Replies
4. Shell Programming and Scripting
Hi gurus,
i'm trying to remove a number of lines from a large file using the following command:
sed '1,5000d' oldfile > newfile
Somehow the lines in the old file are not deleted...
Am I doing this wrongly? Any suggestions? :confused:
Thanks! :)
wee (10 Replies)
Discussion started by: lweegp
10 Replies
5. Shell Programming and Scripting
A small question
I have a test.txt file
I have contents as:
a:google
b:yahoo
:
c:facebook
:
d:hotmail
How do I remove the line with :
my output should be
a:google
b:yahoo
c:facebook
d:hotmail (5 Replies)
Discussion started by: aronmelon
5 Replies
6. Shell Programming and Scripting
Hi,
I'm not a expert in shell programming, so i've come here to take help from u gurus.
I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command.
I've a datatable csv of the below format -
--in file format
xx,xx,xx ,xx , , , , ,,xx,
xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies
7. Shell Programming and Scripting
Hey Gang-
I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this?
Example
wvm1234dev
wvm1234pro
uvm1122dev
uvm1122bku
uvm1344dev
I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
Discussion started by: idiotboy
7 Replies
8. Shell Programming and Scripting
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.
keepout:
user1
buser3
anuser19
notheruser27
database:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies
9. Shell Programming and Scripting
Hello everyone,
Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done.
My problem is the following:
I have a file where each line is a sequence of IP addresses, example :
10.0.0.1 10.0.0.2
10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies
10. Shell Programming and Scripting
I have been searching and trying to come up with an awk that will perform the following on a
converted text file (original is a pdf).
1. Since the first two lines are (begin with) text they are removed
2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies
LEARN ABOUT SUSE
dbd::sponge
DBD::Sponge(3) User Contributed Perl Documentation DBD::Sponge(3)
NAME
DBD::Sponge - Create a DBI statement handle from Perl data
SYNOPSIS
my $sponge = DBI->connect("dbi:Sponge:","","",{ RaiseError => 1 });
my $sth = $sponge->prepare($statement, {
rows => $data,
NAME => $names,
%attr
}
);
DESCRIPTION
DBD::Sponge is useful for making a Perl data structure accessible through a standard DBI statement handle. This may be useful to DBD module
authors who need to transform data in this way.
METHODS
connect()
my $sponge = DBI->connect("dbi:Sponge:","","",{ RaiseError => 1 });
Here's a sample syntax for creating a database handle for the Sponge driver. No username and password are needed.
prepare()
my $sth = $sponge->prepare($statement, {
rows => $data,
NAME => $names,
%attr
}
);
o The $statement here is an arbitrary statement or name you want to provide as identity of your data. If you're using DBI::Profile it
will appear in the profile data.
Generally it's expected that you are preparing a statement handle as if a "select" statement happened.
o $data is a reference to the data you are providing, given as an array of arrays.
o $names is a reference an array of column names for the $data you are providing. The number and order should match the number and
ordering of the $data columns.
o %attr is a hash of other standard DBI attributes that you might pass to a prepare statement.
Currently only NAME, TYPE, and PRECISION are supported.
BUGS
Using this module to prepare INSERT-like statements is not currently documented.
AUTHOR AND COPYRIGHT
This module is Copyright (c) 2003 Tim Bunce
Documentation initially written by Mark Stosberg
The DBD::Sponge module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. In particular
permission is granted to Tim Bunce for distributing this as a part of the DBI.
SEE ALSO
DBI
perl v5.12.1 2007-09-26 DBD::Sponge(3)