Removing lines from a file being used by another process using SHELL script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing lines from a file being used by another process using SHELL script
# 1  
Old 09-25-2012
Removing lines from a file being used by another process using SHELL script

Hi All,

Need a small help in writing a shell script which can delete a few lines from a file which is currently being used by another process.

File gets appended using tee -a command due to which its size is getting increased.
Contents like :
Code:
[INFO] 25/09/2012 05:18 Run ID:56579677-1

My requirement is to remove lines which are more than 1 month old...
for this i will cut month from the line i.e. 09 and backtrack to fetch the previous value i.e. 8
and remove all lines from top till the row fetched...

Query is can i remove the lines in the same file ? (what if another process comes to write in that file using tee -a)..

Thanks,
Nik

Last edited by Scrutinizer; 09-25-2012 at 04:39 AM.. Reason: code tags
# 2  
Old 09-25-2012
This is how I remove unwanted lines in syslog, it will give you the idea...
Code:
cat syslog.log|grep -vFi -e sudo -e tty?? \
-e smbd -e nmbd \
-e "above message repeats " \
-e " registrar/tcp: Conn" \
-e "-prod" >tata
sleep 2
cat tata>syslog.log

# 3  
Old 09-25-2012
I would guess that it depends on how you remove the top lines. This is at the bash prompt...
Code:
$ echo x > file1

$ ( ( echo a ; sleep 30 ; echo b ) | tee -a file1 ) &
[1] 7000

$ a


$ sed -i 1d file1
sed: cannot rename ./sedCDnEEq: Permission denied

$ cat file1
x
a

$ awk '{a[NR]=$0}END{close(FILENAME);for(i=2;i<=NR;i++)print a[i]>FILENAME}' file1

$ cat file1
a

$ b

[1]+  Done                    ( ( echo a; sleep 30; echo b ) | tee -a file1 )

$ cat file1
a
b

$

In the example above, I created a file then submitted a background process to append to the file. Using sed -i to remove the first line didn't work, but awk worked okay.
# 4  
Old 09-25-2012
Are you in linux? Try flock command. Beware: file locks are NOT mandatory, so every command accessing the file must do it through flock.

Here is an example:
Code:
$ cat file
25/08/2012
31/08/2012
4/09/2012
5/10/2012

$ flock file -c "sed '1,/\/08\/2012/ d' <file >tmpfile; cat tmpfile >file" & flock file -c "echo yadayadayada |tee -a file >/dev/null"
[1] 22160
[1]+  Completato              flock file -c "sed '1,/\/08\/2012/ d' <file >tmpfile; cat tmpfile >file"

$ cat file
4/09/2012
5/10/2012
yadayadayada

To have a real feel of what's happening, reset the original file content and use some sleeps, as in:
Code:
flock file -c "sed '1,/\/08\/2012/ d' <file >tmpfile; cat tmpfile >file; sleep 10" & \
sleep 3; echo "sed has worked; cat has worked, here is the file:"; cat file; \
echo; echo "waiting the last 7 seconds for tee..."; \
flock file -c "echo yadayadayada |tee -a file >/dev/null"; \
echo "tee has been able to write just now. Here is the file:"; cat file

To sum it up:

a) to delete your old lines, use flock file -c "sed ... <file >tmpfile; cat tmpfile >file";

b) prepend flock file -c "... to your tee -a file command accessing the file.
--
Bye
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to process a list of items and uncomment lines with that item in a second file

Hello, I have a src code file where I need to uncomment many lines. The lines I need to uncomment look like, C CALL l_r(DESNAME,DESOUT, 'Gmax', ESH(10), NO_APP, JJ) The comment is the "C" in the first column. This needs to be deleted so that there are 6 spaces preceding "CALL".... (7 Replies)
Discussion started by: LMHmedchem
7 Replies

2. UNIX for Beginners Questions & Answers

By pass a process in a Shell Script on file size

I wish to by pass a process if the file is over a certain size? not sure this makes sense current bit of the script below #if we are bypAssing the OCR if ; then echo Bypassing HOTFOLDER OCR HOT_FOLDER_DIR=$BATCH_POST_OCR_DIR; potential change below? would this work would I need... (1 Reply)
Discussion started by: worky
1 Replies

3. Shell Programming and Scripting

Shell script to report file size, pid and also kill the process

Hi All, Looking for a quick LINUX shell script which can continuously monitors the flle size, report the process which is creating a file greater than certain limit and also kill that process. Can someone please help me on this? (4 Replies)
Discussion started by: vasavimacherla
4 Replies

4. Shell Programming and Scripting

Shell script to pass the config file lines as variable on the respective called function on a script

I want to make a config file which contain all the paths. i want to read the config file line by line and pass as an argument on my below function. Replace all the path with reading config path line by line and pass in respective functions. how can i achieve that? Kindly guide. ... (6 Replies)
Discussion started by: sadique.manzar
6 Replies

5. Shell Programming and Scripting

What is the function of the following lines at the top of a shell script file: Directory and Script?

The file starts like this: Directory: <path to the script> Script: <script fife name> #!bin/ksh ##Comments <actual script> What is the use of the first two lines in the script? What if I save the file without them? What will be the effect? They are not comments. Im very new to this,... (4 Replies)
Discussion started by: remytom
4 Replies

6. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

7. Shell Programming and Scripting

Script for Removing Lines from File / Blocking internet connection

Hey all. I am trying to write some scripts and need some assistance. One: I already have a script that appends lines to a file. I need a script that will remove those lines from that file, and have no idea how to go about doing this. Just need the command (if any) that can remove lines. ... (2 Replies)
Discussion started by: Dysruption
2 Replies

8. Shell Programming and Scripting

Removing the entire file contents using unix shell script.

I need to remove the entire file contents in file using the shell script. Actually the grap -v command will create one more file and it occupy the space also. I need to remove the entire file contents without creating new file using the shell scripting. Please help me. (5 Replies)
Discussion started by: praka
5 Replies

9. Shell Programming and Scripting

help removing characters for output file in shell script

hi i'm new to shell scripts and have a small problem i am running a batch converter that returns all flash .flv files in a directory and create a png image from each one the problem i have is the $1 variable , its ok on the first call but on the secound call $1.png , i have extra... (1 Reply)
Discussion started by: wingchun22
1 Replies

10. Shell Programming and Scripting

How can i know file using another process on lunix shell script?

hi all, I did RedHat OS. How can i know myfile using another process on lunix shell script? Have you got any suggestions? (1 Reply)
Discussion started by: Tlg13team
1 Replies
Login or Register to Ask a Question