Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

File copy problem?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users File copy problem?
# 1  
Old 09-29-2012
File copy problem?
Hi All

I can't get my head around a problem I have with a control file.

The file is to control a "Listener" of sorts that listens on a named pipe. A script kicks off the listener in the background and passes it a control file. In the file it sets the Status field to pending. It then waits for the listener to get up and running and change its status to "Listening" before launching another job that uses the named pipe to report its status.

The control file: -

Code:
Name:m_fatal_error
Pid:20755
JobFile:/home/sshtest/muse/run/demo/120929165000/m_control/4-m_job_data_10519
LogFile:/home/sshtest/muse/run/demo/120929165000/m_control/m_run_log
TmpFile:/home/sshtest/muse/run/demo/120929165000/m_tmp/1_u_listener_4498.tmp
JobHost:tx5xn
Start:12/09/29-16:50:01
Finish:
Pipe:4-m_msg_data_10519
Status:Pending
Constants:/home/brad/wip/muse_root/lib/muse.constants
sshtest@ubuntu-dt64:~$ cat /home/sshtest/muse/run/demo/120929165000/m_tmp/1_u_listener_4498.tmp

The listener uses a function (that works 90% of the time) to change the status to Listening -

Code:
m_write_ctl_file_field ${C_MSG_JOB_STATUS} "Listening" "${MSG_CTL_FILE}"

Code:
m_write_ctl_file_field()
{
        TMP="$(sed -n ''${1}'p' ${3} | cut -d":" -f1):${2}"
        for s in ${PIPESTATUS[@]}; do [[ $s -eq 0 ]] || error_exit "Error:Pipe failed 2 (${SCRIPT})" ; done
        awk -v ln=${1} -v lo="${TMP}" '(NR == ln){print lo;next}{print $0}' "${3}" > "${U_MSG_TMPFILE}"
        cp "${U_MSG_TMPFILE}" "${3}" || error_exit "Error: cp failed in m_write_ctl_file_field"
}

C_MSG_JOB_STATUS is the line number for the status field
MSG_CTL_FILE is the control file
and "Listening" is my change of status

When I look in the tmp file this function uses after I time out waiting for the listener to change its status to listening -

Code:
Name:m_fatal_error
Pid:
JobFile:/home/sshtest/muse/run/demo/120929165000/m_control/4-m_job_data_10519
LogFile:/home/sshtest/muse/run/demo/120929165000/m_control/m_run_log
TmpFile:/home/sshtest/muse/run/demo/120929165000/m_tmp/1_u_listener_4498.tmp
JobHost:tx5xn
Start:12/09/29-16:50:01
Finish:
Pipe:4-m_msg_data_10519
Status:Listening
Constants:/home/brad/wip/muse_root/lib/muse.constants

So the function has written it to the tmp file. The check on the exit status for the cp command doesn't seem to error or I would see an error log in /tmp. So I can't understand why the tmp file and the control file aren't the same....

like I said, most of the time this code works.

I'm wondering if this is some sort of buffering problem and whether I need to adopt a safer method that explicitly closes the file or something???

Any help appreciated Smilie

Steady
# 2  
Old 09-29-2012
Have you checked the timestamps on ${MSG_CTL_FILE} and ${U_MSG_TMPFILE}?
Are you sure that the cp in m_write_ctl_file_field() didn't succeed and then something else updated ${MSG_CTL_FILE} changing the status back to Pending?
# 3  
Old 09-29-2012
Update to file access problem
Hi Don

Thanks for getting back to me.

I'm sure nothing else is writing it back again as each file has a unique name with a RND number appended. On top of that I don't have any code writing Pending to it at the moment.

I have just run this in a loop and then run an lsof and can see I have hundreds of open files in use by listeners that should be closed-

Code:
u_listene 29054    sshtest  mem       REG    8,1   134344  264820 /lib/i386-linux-gnu/ld-2.15.so
u_listene 29054    sshtest  mem       REG    8,1    13940  266883 /lib/i386-linux-gnu/libdl-2.15.so
u_listene 29054    sshtest  mem       REG    8,1  2932080  391615 /usr/lib/locale/locale-archive
u_listene 29054    sshtest  mem       REG    8,1    26256   16482 /usr/lib/i386-linux-gnu/gconv/gconv-modules.cache
u_listene 29054    sshtest    0r      CHR    1,3      0t0    4748 /dev/null
u_listene 29054    sshtest    1u      CHR  136,2      0t0       5 /dev/pts/2
u_listene 29054    sshtest    2u      CHR  136,2      0t0       5 /dev/pts/2
u_listene 29054    sshtest  255r      REG   8,17     3971 1705114 /home/brad/wip/muse_root/utils/u_listener~ (deleted)

The listener is u_listener and I have hundreds of files in use which probably explains why the problem is getting worse throughout the day... Smilie

ps shows that I have lots of open instances of the process -

Code:
sshtest@ubuntu-dt64:~$ ps -ef | grep u_listen
sshtest   2523     1  0 18:27 pts/2    00:00:00 /bin/bash /home/brad/wip/muse_root/utils/u_listener /home/sshtest/muse/run/demo/120929182746/m_control/1-m_msg_data_31692
sshtest   3132     1  0 17:45 pts/2    00:00:00 /bin/bash /home/brad/wip/muse_root/utils/u_listener /home/sshtest/muse/run/demo/120929174550/m_control/4-m_msg_data_7666
sshtest   5562     1  0 16:15 pts/2    00:00:00 /bin/bash /home/brad/wip/muse_root/utils/u_listener /home/sshtest/muse/run/demo/120929161543/m_control/1-m_msg_data_23624
sshtest   5659     1  0 16:22 pts/2    00:00:00 /bin/bash /home/brad/wip/muse_root/utils/u_listener /home/sshtest/muse/run/demo/120929162238/m_control/6-m_msg_data_21829
sshtest   7384     1  0 16:29 pts/2    00:00:00 /bin/bash /home/brad/wip/muse_root/utils/u_listener

I have just introduced some functionality to exit in the event of fatal errors and this problem started when I was running some test jobs that failed.

I hope this means that I just need to extend the clean exit to explicitely kill off listeners that haven't exited prior to exiting the main application.

Cheers

Steady
# 4  
Old 09-29-2012
I assume that you've verified that your RND number generator hasn't generated the same number for two or more temp files. Random is not the same as different; and for an application like this, different is what you need. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[Solved] Copy Problem

I facing a problem : I have a Source and Destination directory. The Source directory is linked to SVN and is updated by the Script to Head Revision. After that i copy the Source to Destination by this command: cp -r /SOURCE PATH /DESTIUNATION PATH Now if a delete a file in source and... (5 Replies)
Discussion started by: ankur328
5 Replies

2. Shell Programming and Scripting

how to copy the directory but not copy certain file

Hi experts cp bin root src /mnt but not copy bin/bigfile any help? ( I post this thread in the "redhat" forum wrongly, I don't know how to withdraw that question in that wrong forum) Thanks (6 Replies)
Discussion started by: yanglei_fage
6 Replies

3. Solaris

I can't copy any file to any machine | abort problem | freezing

I want to copy tar file to another machine. tar size is 4gb. Firstly I tried copy to windows machine with ftp client but copy operation didn't start. Now I have tryied to copy to solaris machine command with scp but copy was freezed. Picture is attached.... (1 Reply)
Discussion started by: getrue
1 Replies

4. AIX

Flash Copy Problem, has any one else seen this?

Around a month ago we suffered a prolonged power outage, due to circumstances the servers and storage arrays were still in use when the UPS dropped. One of the servers was running a flash copy on the database while batch processing had commenced. This is the only server that suffered any further... (2 Replies)
Discussion started by: gull04
2 Replies

5. UNIX for Dummies Questions & Answers

vi copy/paste problem

I'm having a problem copy/pasting from a txt file in windows to vi. What happens is I copy a chunk of text, go to the putty terminal, go into insert mode, and right click, and it will stop pasting at a random point and freeze up. Nothing I do gets out of it. This only happens on my account... (1 Reply)
Discussion started by: solidarity
1 Replies

6. Shell Programming and Scripting

Bash Copy-Move file problem

Hello, I made a script to copy files from one directory to another and move file after the copy is done. When files are present in the source directory there is no problem but when no file are present I'm getting an error. Please help !! --------------------- #!/bin/bash ... (2 Replies)
Discussion started by: lsimoneau
2 Replies

7. Shell Programming and Scripting

Copy files listed in a text file - whitespace problem.

Hi, Say I have this text file <copy.out> that contains a list of files/directories to be copied out to a different location. $ more copy.out dir1/file1 dir1/file2 dir1/file3 "dir1/white space" dir1/file4 If I do the following: $copy=`more copy.out` $echo $copy dir1/file1... (4 Replies)
Discussion started by: 60doses
4 Replies

8. UNIX for Dummies Questions & Answers

copy problem

how to copy and move to another directories?can any1 help mi? (2 Replies)
Discussion started by: yeah016
2 Replies

9. UNIX for Dummies Questions & Answers

Telnet and file Copy/Delete Problem

Hi there....I'm new for the UNIX... just wondering if there is any method that can telnet to a server without typing the userID and PWD each time... that is any command or scripts that allows me to enter the server directly? also...after i enter the server... i want to get some files then... (17 Replies)
Discussion started by: biglemon
17 Replies
Login or Register to Ask a Question