Want to remove all lines but not latest 50 lines from a file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Want to remove all lines but not latest 50 lines from a file
# 8  
Old 10-25-2013
Hi.

Modern shells can hold data in memory by using variables. Here's an example:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate keeping lines in shell memory, variables.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C specimen tail

# Stage new sample data file.
cp data0 data1

FILE=${1-data1}

pl " Input data file $FILE:"
specimen 3 $FILE

pl " Results:"
v1=$( tail -3 $FILE )
rm $FILE
echo "$v1" > $FILE
specimen 3 $FILE

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny, workstation) 
bash GNU bash 3.2.39
specimen (local) 1.17
tail (GNU coreutils) 6.10

-----
 Input data file data1:
Edges: 3:0:3 of 26 lines in file "data1"
Four score and seven years ago our fathers brought forth on this
continent a new nation, conceived in Liberty, and dedicated to
the proposition that all men are created equal.
   ---
under God, shall have a new birth of freedom--and that
government: of the people, by the people, for the people, shall
not perish from the earth.

-----
 Results:
Whole: 3:0:3 of 3 lines in file "data1"
under God, shall have a new birth of freedom--and that
government: of the people, by the people, for the people, shall
not perish from the earth.

So this holds the last 3 lines of Lincoln's Gettysburg address in memory, variable v1 , then removes the file, and finally writes the saved data to the file. One could also just truncate the file if one wanted to keep the inode.

This obviously has risks -- if the code is wrong or if something else bad happens, it's possible that the entire file could be lost. Of course, one has backups for such occasions.

This method works in the versions of Linux and bash as noted. Will it work in your system? I don't know, but you could test it, and, as usual, be prepared with a backup if things go awry.

Best wishes ... cheers, drl

( Originally posted by mistake in thread No Paging Space Available - The UNIX and Linux Forums )
# 9  
Old 10-26-2013
Bash:
Code:
n=50000; start=$(( $(wc -l file | cut -d' ' -f1) - $n + 1 )); sed -ni ${start},\$p file

Emanuele
# 10  
Old 10-26-2013
from sed1line.txt Smilie

Code:
sed -e :a -e '$q;N;50001,$D;ba'

# 11  
Old 10-26-2013
Quote:
Originally Posted by targzeta
Bash:
Code:
n=50000; start=$(( $(wc -l file | cut -d' ' -f1) - $n + 1 )); sed -ni ${start},\$p file

Emanuele
That won't work. The file system is full and the OP wants to edit in place, without using a temporary file.
# 12  
Old 10-26-2013
Quote:
Originally Posted by alister
That won't work. The file system is full and the OP wants to edit in place, without using a temporary file.
From sed(1):
Code:
-i[SUFFIX], --in-place[=SUFFIX]

              edit files in place (makes backup if SUFFIX supplied)

Emanuele
# 13  
Old 10-26-2013
If you really can't tail to another filesystem first, perhaps:
Code:
ex -sc '$-49999,$d | x' file


--
Quote:
Originally Posted by targzeta
From sed(1):
Code:
-i[SUFFIX], --in-place[=SUFFIX]

              edit files in place (makes backup if SUFFIX supplied)

Emanuele
That does not really edit in place. It creates a new file with a different inode.

Last edited by Scrutinizer; 10-26-2013 at 09:30 AM..
# 14  
Old 10-26-2013
Quote:
Originally Posted by targzeta
From sed(1):
Code:
-i[SUFFIX], --in-place[=SUFFIX]

              edit files in place (makes backup if SUFFIX supplied)

Emanuele
And, if you go to the info page for sed, it says:
Code:
-i extension
             Edit files in-place, saving backups with the specified extension.
             If a zero-length extension is given, no backup will be saved.  It
             is not recommended to give a zero-length extension when in-place
             editing files, as you risk corruption or partial content in situ-
             ations where disk space is exhausted, etc.

This User Gave Thanks to Don Cragun For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

3. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

4. Shell Programming and Scripting

Remove lines from file

Hey Gang- I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this? Example wvm1234dev wvm1234pro uvm1122dev uvm1122bku uvm1344dev I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
Discussion started by: idiotboy
7 Replies

5. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

6. Shell Programming and Scripting

remove : lines from file

A small question I have a test.txt file I have contents as: a:google b:yahoo : c:facebook : d:hotmail How do I remove the line with : my output should be a:google b:yahoo c:facebook d:hotmail (5 Replies)
Discussion started by: aronmelon
5 Replies

7. Shell Programming and Scripting

remove lines from file

Hi gurus, i'm trying to remove a number of lines from a large file using the following command: sed '1,5000d' oldfile > newfile Somehow the lines in the old file are not deleted... Am I doing this wrongly? Any suggestions? :confused: Thanks! :) wee (10 Replies)
Discussion started by: lweegp
10 Replies

8. UNIX for Dummies Questions & Answers

vi to remove lines in file

All, I have a text file with several entries like below: personname personname.domain.com I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax. Thanks (2 Replies)
Discussion started by: kjbaumann
2 Replies

9. Shell Programming and Scripting

To remove the lines in my file

Hi, There seems to some hack attempts in my site. I have attached the index page of my site and I need to remove the below lines from the index page. The below lines are at the center of the file. --> </style> <script>E V A L( unescape(... (5 Replies)
Discussion started by: gsiva
5 Replies

10. Shell Programming and Scripting

remove lines from file

file: 1 xxxxxxx 2 xxx xxx 5 xxx xxx ... 180 xxxxxx 200 xxx how to remove any lines with the first number range 1-180 (9 Replies)
Discussion started by: bluemoon1
9 Replies
Login or Register to Ask a Question