Want to remove all lines but not latest 50 lines from a file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Want to remove all lines but not latest 50 lines from a file
# 29  
Old 10-29-2013
Hi.

As alister said.

This behavior has been discussed in the forum, for example, I mentioned sponge / absorb for pipelines and re-writing-in-place in post #8 of https://www.unix.com/shell-programming-and-scripting/221749-run-command-without-over-writing.html -- that approach either keeps data in memory (say a perl array), or writes it to a scratch file first, both of which can preserve the inode if that is important. The sponge approach looks like the tee solution, but the internal logic differs by sponge collecting all the data until the end, then writing to the file argument, whereas tee streams the data to the file(s) as it comes in.

We cannot use the scratch file approach here because there is no additional disk space, so suggestions like the shell and dd above seemed appropriate. Both were validated on a file of 14M lines (~ 1 GB) with 50,000 lines being kept.

Best wishes ... cheers, drl

Last edited by drl; 10-30-2013 at 11:53 AM.. Reason: Minor typo.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

3. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

4. Shell Programming and Scripting

Remove lines from file

Hey Gang- I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this? Example wvm1234dev wvm1234pro uvm1122dev uvm1122bku uvm1344dev I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
Discussion started by: idiotboy
7 Replies

5. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

6. Shell Programming and Scripting

remove : lines from file

A small question I have a test.txt file I have contents as: a:google b:yahoo : c:facebook : d:hotmail How do I remove the line with : my output should be a:google b:yahoo c:facebook d:hotmail (5 Replies)
Discussion started by: aronmelon
5 Replies

7. Shell Programming and Scripting

remove lines from file

Hi gurus, i'm trying to remove a number of lines from a large file using the following command: sed '1,5000d' oldfile > newfile Somehow the lines in the old file are not deleted... Am I doing this wrongly? Any suggestions? :confused: Thanks! :) wee (10 Replies)
Discussion started by: lweegp
10 Replies

8. UNIX for Dummies Questions & Answers

vi to remove lines in file

All, I have a text file with several entries like below: personname personname.domain.com I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax. Thanks (2 Replies)
Discussion started by: kjbaumann
2 Replies

9. Shell Programming and Scripting

To remove the lines in my file

Hi, There seems to some hack attempts in my site. I have attached the index page of my site and I need to remove the below lines from the index page. The below lines are at the center of the file. --> </style> <script>E V A L( unescape(... (5 Replies)
Discussion started by: gsiva
5 Replies

10. Shell Programming and Scripting

remove lines from file

file: 1 xxxxxxx 2 xxx xxx 5 xxx xxx ... 180 xxxxxx 200 xxx how to remove any lines with the first number range 1-180 (9 Replies)
Discussion started by: bluemoon1
9 Replies
Login or Register to Ask a Question
PHP(1)								Scripting Language							    PHP(1)

NAME
php - PHP Command Line Interface 'CLI' SYNOPSIS
php [options] [ -f ] file [[--] args...] php [options] -r code [[--] args...] php [options] [-B code] -R code [-E code] [[--] args...] php [options] [-B code] -F file [-E code] [[--] args...] php [options] -- [ args...] php [options] -a DESCRIPTION
PHP is a widely-used general-purpose scripting language that is especially suited for Web development and can be embedded into HTML. This is the command line interface that enables you to do the following: You can parse and execute files by using parameter -f followed by the name of the file to be executed. Using parameter -r you can directly execute PHP code simply as you would do inside a .php file when using the eval() function. It is also possible to process the standard input line by line using either the parameter -R or -F. In this mode each separate input line causes the code specified by -R or the file specified by -F to be executed. You can access the input line by $argn. While processing the input lines $argi contains the number of the actual line being processed. Further more the parameters -B and -E can be used to execute code (see -r) before and after all input lines have been processed respectively. Notice that the input is read from STDIN and therefore reading from STDIN explicitly changes the next input line or skips input lines. If none of -r -f -B -R -F or -E is present but a single parameter is given then this parameter is taken as the filename to parse and exe- cute (same as with -f). If no parameter is present then the standard input is read and executed. OPTIONS
--interactive -a Run PHP interactively. This lets you enter snippets of PHP code that directly get executed. When readline support is enabled you can edit the lines and also have history support. --php-ini path|file -c path|file Look for php.ini file in the directory path or use the specified file --no-php-ini -n No php.ini file will be used --define foo[=bar] -d foo[=bar] Define INI entry foo with value bar -e Generate extended information for debugger/profiler --file file -f file Parse and execute file --global name -g name Make variable name global in script. --help -h This help --hide-args -H Hide script name (file) and parameters (args...) from external tools. For example you may want to use this when a php script is started as a daemon and the command line contains sensitive data such as passwords. --info -i PHP information and configuration --syntax-check -l Syntax check only (lint) --modules -m Show compiled in modules --run code -r code Run PHP code without using script tags '<?..?>' --process-begin code -B code Run PHP code before processing input lines --process-code code -R code Run PHP code for every input line --process-file file -F file Parse and execute file for every input line --process-end code -E code Run PHP code after processing all input lines --syntax-highlight -s Output HTML syntax highlighted source --version -v Version number --stripped -w Output source with stripped comments and whitespace --zend-extension file -z file Load Zend extension file args... Arguments passed to script. Use '--' args when first argument starts with '-' or script is read from stdin --rfunction name --rf name Shows information about function name --rclass name --rc name Shows information about class name --rextension name --re name Shows information about extension name --rextinfo name --ri name Shows configuration for extension name --ini Show configuration file names FILES
php-cli.ini The configuration file for the CLI version of PHP. php.ini The standard configuration file will only be used when php-cli.ini cannot be found. EXAMPLES
php -r 'echo "Hello World ";' This command simply writes the text "Hello World" to standard out. php -r 'print_r(gd_info());' This shows the configuration of your gd extension. You can use this to easily check which image formats you can use. If you have any dynamic modules you may want to use the same ini file that php uses when executed from your webserver. There are more extensions which have such a function. For dba use: php -r 'print_r(dba_handlers(1));' php -R 'echo strip_tags($argn)." ";' This PHP command strips off the HTML tags line by line and outputs the result. To see how it works you can first look at the following PHP command 'php -d html_errors=1 -i' which uses PHP to output HTML formatted configuration information. If you then combine those two 'php ...|php ...' you'll see what happens. php -E 'echo "Lines: $argi ";' Using this PHP command you can count the lines being input. php -R '@$l+=count(file($argn));' -E 'echo "Lines:$l ";' In this example PHP expects each input line being a file. It counts all lines of the files specified by each input line and shows the summarized result. You may combine this with tools like find and change the php scriptlet. php -R 'echo "$argn "; fgets(STDIN);' Since you have access to STDIN from within -B -R -F and -E you can skip certain input lines with your code. But note that in such cases $argi only counts the lines being processed by php itself. Having read this you will guess what the above program does: skipping every second input line. TIPS
You can use a shebang line to automatically invoke php from scripts. Only the CLI version of PHP will ignore such a first line as shown below: #!/bin/php <?php // your script ?> SEE ALSO
For a more or less complete description of PHP look here: http://www.php.net/manual/ BUGS
You can view the list of known bugs or report any new bug you found at: http://bugs.php.net AUTHORS
The PHP Group: Thies C. Arntzen, Stig Bakken, Andi Gutmans, Rasmus Lerdorf, Sam Ruby, Sascha Schumann, Zeev Suraski, Jim Winstead, Andrei Zmievski. Additional work for the CLI sapi was done by Edin Kadribasic, Marcus Boerger and Johannes Schlueter. A List of active developers can be found here: http://www.php.net/credits.php And last but not least PHP was developed with the help of a huge amount of contributors all around the world. VERSION INFORMATION
This manpage describes php, version 5.3.2. COPYRIGHT
Copyright (C) 1997-2010 The PHP Group This source file is subject to version 3.01 of the PHP license, that is bundled with this package in the file LICENSE, and is available through the world-wide-web at the following url: http://www.php.net/license/3_01.txt If you did not receive a copy of the PHP license and are unable to obtain it through the world-wide-web, please send a note to license@php.net so we can mail you a copy immediately. The PHP Group 2010 PHP(1)