File write begin/end, recording time


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers File write begin/end, recording time
# 1  
Old 01-09-2020
File write begin/end, recording time

Hello all,

How can I find out the start and end of the writing file in the directory or recording time for writing file?

I have a directory where small ~ 1*MB temporary files are written.
After the end of the record, they are retrieved and erased.
I can only find out that the files are being written there.

Code:
while true; do ls --full-time temp | grep -v "insgesamt 0"; done

Thanks!

Last edited by RavinderSingh13; 01-09-2020 at 10:33 AM..
# 2  
Old 01-09-2020
Hello mrAibo,

You might get some joy from the output of the stat command. The file is Modified as it is created and Changed as the content is written, however this might not be safe for a polling process to be sure the file is complete.

What is the overall goal of the process you are looking to automate?

It might be safer to write a flag file when the real IO has completed, e.g. the file name might be abc123def456.output and this grows as the data is written/transferred. At the end of the writing process, just touch abc123def456.output.complete A reading process should look for files ending .complete and only then attempt to work with the real data file. Of course, there is then an extra file to tidy away and an extra IO, but the time taken should be negligible.


Does this better suit what you need, or have I missed the point?

Kind regards,
Robin
# 3  
Old 01-09-2020
Hello rbatte1,
I suppose that the other Server transfers the files too slow to our Server. The lag is ~3Sec., after file is ready it will archive and erased from directory.
The supporter on other side said that he see no problem. Now I have to approve that we are not guilty.
Sorry for my EnglishSmilie
# 4  
Old 01-10-2020
Solved with Inotify tools.
# 5  
Old 01-10-2020
inotify (in Linux) is most efficient.
You can also poll the file with fuser (or lsof).
Code:
file=tempfile
if [ ! -f $file ]
then
  echo "$file is not present"
elif pids=$(fuser $file 2>/dev/null); [ -z "$pids" ]
then
  echo "nobody reads or writes to $file"
else
 echo "$file is accessed by PIDs" $pids
fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

In ksh script what is this BEGIN and END in this function?

Can Someone please explain why BEGIN and END statement is used inside function? How does that help in scripting? function fileformatting { CleanupMask="xXxX" sed 's/^.//' < ${AllFile} > ${AllFile}.tmp echo $(wc -l ${AllFile}.tmp) `awk -v CleanupMask=${CleanupMask} ' BEGIN... (2 Replies)
Discussion started by: later_troy
2 Replies

2. Shell Programming and Scripting

Search ad replace using begin and end of the file

Hello Friends , Please help to create script for compare and replace if not matches of set of lines . * Primary* Servername Server1 Location R201 Rack 4 *End Primary* *Secondary* Server Name Server1 IPAddress 10.24.30.10 Application Apache *End of Secondary* Above... (4 Replies)
Discussion started by: rnary
4 Replies

3. Shell Programming and Scripting

BEGIN and END format in awk

I'm new to awk, trying to understand the basics. I'm trying to reset the counter everytime the program gets a new file to check. I figured in the BEGIN part it would work, but it doesn't. #!/bin/awk -f BEGIN {counter=0} { sum=0 for ( i=1; i<=NF;... (1 Reply)
Discussion started by: guitarist684
1 Replies

4. Shell Programming and Scripting

Begin/End blocks in awk: confused

I am trying to understand how to use the END block in awk without much success. I have this script that I found: gawk '{count++; keyword = $1} if (count == 3) keyword = "order this" else print keyword " " k } }' << orderfile Is that the way that the END block should be used? I am... (6 Replies)
Discussion started by: newbie2010
6 Replies

5. Shell Programming and Scripting

Get the sum of values in between begin and end in the file

Hi All, test file Begin Script Run at Thu Mar 14 09:24:16 PDT 2013 tst_accounts: ws zip: WS_out_20130313.tar.gz dat: test_20130313.dat count: 63574 loaded: xx pre-merge: xx post-merge: xx timestamp: Thu Mar 14 09:30:42 PDT 2013 tst_accounts: ws zip: WS_out_20130313.tar.gz dat: s_20130313.dat... (6 Replies)
Discussion started by: bmk
6 Replies

6. Programming

search a file between two begin and end strings in c

Can any one help me out with following problem... I want to search in a file which has two strings repeat each time(like start and end) i want to search between these two string in C programming. please help me with the solution. thanks in advance. (8 Replies)
Discussion started by: uday.sena.m
8 Replies

7. UNIX for Dummies Questions & Answers

Dump to another file for a begin and end Pattern

Hi All , I am newbie to linux shell scripting , below are the contents of my log file , i want the lines between a begin pattern and a end pattern for an instance , my begin Pattern is "Transaction Begins for Usr" and end pattern is "Transaction Ends for Usr" into another file Sample file... (1 Reply)
Discussion started by: SARAVANANE
1 Replies

8. Shell Programming and Scripting

Use of Begin IF ,END IF END not working in the sql script

Hi I have written a script .The script runs properly if i write sql queries .But if i use PLSQL commands of BEGIN if end if , end ,then on running the script the comamds are getting printed on the prompt . Ex :temp.sql After connecting to the databse at the sql prompt i type... (1 Reply)
Discussion started by: isha_1
1 Replies

9. Shell Programming and Scripting

begin end detection

Hi all, i am new to scripting. i need to write a code to detect begin and end of word that either begins or ends with t,th,d,dh,s,sh i have a set of words in a file containg one word per line. let the filename be aaa.txt. i have an another file bbb.txt which has two lines, just specifying the... (7 Replies)
Discussion started by: blkanth
7 Replies

10. Shell Programming and Scripting

BEGIN END questions

Why would you need to use this in a script? Why can't you just use print to print out what you want printed in the begining and print for what you want at the end. So this: nawk 'BEGIN {print "this is the first line"} {print $1 $2 $3} {print $5 $6} END {print "this is the last line"}' ... (2 Replies)
Discussion started by: llsmr777
2 Replies
Login or Register to Ask a Question