For the version of tail that I used, as noted above:
so it looks like the GNU folks have made provisions for some large quantities of data. I have not looked at the source, however.
For the tests that I ran, I numbered all the lines, counted the final tail (50,000 of 14,000,000), and printed the first and last line of the output file. All seemed correct and intact. The system tail and the version in perl at http://cpansearch.perl.org/src/CWEST/ppt-0.14/bin/tail both return essentially immediately when asked for the last 5 lines of the 14M line file. I noticed some seeks in the perl version, so I'd guess that both those versions are using such performance techniques.
I don't disagree that some utilities will produce wrong results without warning, but that's not the right thing to do, and those systems should be avoided if possible, and especially if one will normally try to solve problems such as we are addressing here. Another possibility is to use the perl version, although I didn't rigorously test it (and the calling sequence is slightly different, but easily changed).
I might worry more about the shell capacity, although I didn't have any trouble, and I recall a test I did on an early version of SunOS, and was pleasantly surprised at its capacity for variable storage. Of course, the MMV for specific cases.
Best wishes ... cheers, drl
Last edited by drl; 10-27-2013 at 09:48 AM..
Reason: Minor typo.
Hi,
There seems to some hack attempts in my site. I have attached the index page of my site and I need to remove the below lines from the index page. The below lines are at the center of the file.
-->
</style>
<script>E V A L( unescape(... (5 Replies)
All,
I have a text file with several entries like below:
personname
personname.domain.com
I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax.
Thanks (2 Replies)
Hi gurus,
i'm trying to remove a number of lines from a large file using the following command:
sed '1,5000d' oldfile > newfile
Somehow the lines in the old file are not deleted...
Am I doing this wrongly? Any suggestions? :confused:
Thanks! :)
wee (10 Replies)
A small question
I have a test.txt file
I have contents as:
a:google
b:yahoo
:
c:facebook
:
d:hotmail
How do I remove the line with :
my output should be
a:google
b:yahoo
c:facebook
d:hotmail (5 Replies)
Hi,
I'm not a expert in shell programming, so i've come here to take help from u gurus.
I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command.
I've a datatable csv of the below format -
--in file format
xx,xx,xx ,xx , , , , ,,xx,
xxxx,, ,, xxx,... (11 Replies)
Hey Gang-
I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this?
Example
wvm1234dev
wvm1234pro
uvm1122dev
uvm1122bku
uvm1344dev
I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.
keepout:
user1
buser3
anuser19
notheruser27
database:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Hello everyone,
Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done.
My problem is the following:
I have a file where each line is a sequence of IP addresses, example :
10.0.0.1 10.0.0.2
10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
I have been searching and trying to come up with an awk that will perform the following on a
converted text file (original is a pdf).
1. Since the first two lines are (begin with) text they are removed
2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies
LEARN ABOUT OPENDARWIN
stdbuf
STDBUF(1) User Commands STDBUF(1)NAME
stdbuf - Run COMMAND, with modified buffering operations for its standard streams.
SYNOPSIS
stdbuf OPTION... COMMAND
DESCRIPTION
Run COMMAND, with modified buffering operations for its standard streams.
Mandatory arguments to long options are mandatory for short options too.
-i, --input=MODE
adjust standard input stream buffering
-o, --output=MODE
adjust standard output stream buffering
-e, --error=MODE
adjust standard error stream buffering
--help display this help and exit
--version
output version information and exit
If MODE is 'L' the corresponding stream will be line buffered. This option is invalid with standard input.
If MODE is '0' the corresponding stream will be unbuffered.
Otherwise MODE is a number which may be followed by one of the following: KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T,
P, E, Z, Y. In this case the corresponding stream will be fully buffered with the buffer size set to MODE bytes.
NOTE: If COMMAND adjusts the buffering of its standard streams ('tee' does for example) then that will override corresponding changes by
'stdbuf'. Also some filters (like 'dd' and 'cat' etc.) don't use streams for I/O, and are thus unaffected by 'stdbuf' settings.
EXAMPLES
tail -f access.log | stdbuf -oL cut -d ' ' -f1 | uniq
This will immediately display unique entries from access.log
BUGS
On GLIBC platforms, specifying a buffer size, i.e., using fully buffered mode will result in undefined operation.
AUTHOR
Written by Padraig Brady.
REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report stdbuf translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
Full documentation at: <http://www.gnu.org/software/coreutils/stdbuf>
or available locally via: info '(coreutils) stdbuf invocation'
GNU coreutils 8.28 January 2018 STDBUF(1)