Sponsored Content
Full Discussion: recursive search and ftp
Top Forums UNIX for Dummies Questions & Answers recursive search and ftp Post 302512153 by DGPickett on Friday 8th of April 2011 01:32:51 PM
Old 04-08-2011
This should be simple, even with an ftp layer.

Do you need to worry about copying a file while being written? If so, you can detect others on the file with fuser. Can you wait on the writer to finish, near term? Are there other local readers that would confuse fuser? Can you make the files you want to move read-only, immediately once created, so implicity there are not being written?

First, both sender and recipient will cd to the head dir over the subtree in question. I like to use "find * -type f -newer /elsewhere/mark_file ! -newer /elsewhere/mark_next" and a marker file created with touch. 'touch' a new file at the start of every cycle, /elsewhere/mark_next. Then sleep a second so anything written in that second and also the next will be excluded. Sleep longer if it helps eliminate files being written. After the find and send, "mv /elsewhere/mark_next /elsewhere/mark_file" to support the next batch, seamlessly. Pipe the find to a "while read f" do loop that knows how to check for writing users and ftp the file using relative paths. If the files to be moved are read-only, test with 'if [ -w "$f" ]' and if writable, 'touch' their modify time forward so they are considered on the next pass.

Last edited by DGPickett; 04-08-2011 at 02:38 PM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Recursive FTP

I am trying to write a recursive FTP script and have come to a point where I need to test if the file is either a normal ascii file or a directory. My question is how do I test if the file is either ascii or directory. (1 Reply)
Discussion started by: aslamg
1 Replies

2. Shell Programming and Scripting

perl + Net::FTP::Recursive

Problem: It will not advance to the next user in the list. It always dies right after it sends the 2/2 files from the first users dir. $USERLIST="/export/home/mxdooley/perl_ftp/userlist"; $USER_DIR="/export/home/mxdooley/perl_ftp/homes";... (2 Replies)
Discussion started by: Optimus_P
2 Replies

3. Shell Programming and Scripting

non recursive search in the current directory only

Hi, Am trying for a script which should delete more than 15 days older files in my current directory.Am using the below piece of code: "find /tmp -type f -name "pattern" -mtime +15 -exec /usr/bin/ls -altr {} \;" "find /tmp -type f -name "pattern" -mtime +15 -exec /usr/bin/rm -f {} \;" ... (9 Replies)
Discussion started by: puppala
9 Replies

4. Shell Programming and Scripting

Recursive FTP -- here at last.

Over the past few weeks, I saw a couple of threads requesting recursive ftp: Copying files between 2 Unix server ftp from NT to UNIX I decided to try to write a script to accommodate these requests. The result is HardFeed. Here are a few examples of what it can do. HardFeed ftpserver... (52 Replies)
Discussion started by: Perderabo
52 Replies

5. UNIX for Advanced & Expert Users

Recursive directory search using ls instead of find

I was working on a shell script and found that the find command took too long, especially when I had to execute it multiple times. After some thought and research I came up with two functions. fileScan() filescan will cd into a directory and perform any operations you would like from within... (8 Replies)
Discussion started by: newreverie
8 Replies

6. UNIX for Advanced & Expert Users

Recursive search for filenames containing ..

Hi all, Since my gopher server doesn't like filenames containing 2 or more consecutive dots in a filename, I'd like to do a search for them and replace them with... well, let's say underscores... I've tried a oneliner or 2 from other posts, but they don't seem to work well with locating dots.... (3 Replies)
Discussion started by: Evert
3 Replies

7. Shell Programming and Scripting

recursive search and copy

Hello again. Well, I need help again sooner as I thought. Now I want to search for files with a known name within all subdirs, and copy the to differently named files in the same directory. For example if I had only one file to copy, I would just usecp fileName newFileNamebut to do this... (1 Reply)
Discussion started by: cabaciucia
1 Replies

8. Shell Programming and Scripting

Recursive search on pattern between two strings

Objective: Recursively search all files under a directory for SQL statements that end with ";" Sample input: UPDATE table1 set col=val UPDATE table2 set cola=vala ,colb=valb; UPDATE table3 set col=val Expected output: UPDATE table2 set cola=vala ,colb=valb; (1 Reply)
Discussion started by: krishmaths
1 Replies

9. OS X (Apple)

Search recursive

before posting, I have tried to find my answer elsewhere. no luck. I need to find a file buried in a folder somewhere. Master folder has 10 sub folders. each sub folder has folders too. I found this but it does nothing I am on Mac and use Applescript. do shell script "find... (2 Replies)
Discussion started by: sbrady
2 Replies
touch(1)							   User Commands							  touch(1)

NAME
touch, settime - change file access and modification times SYNOPSIS
touch [-acm] [-r ref_file | -t time] file... touch [-acm] [date_time] file... settime [-f ref_file] [date_time] file... DESCRIPTION
The touch utility sets the access and modification times of each file. The file operand is created if it does not already exist. The time used can be specified by -t time, by the corresponding time fields of the file referenced by -r ref_file, or by the date_time op- erand. If none of these are specified, touch uses the current time (the value returned by the time(2) function). If neither the -a nor -m options are specified, touch updates both the modification and access times. A user with write access to a file, but who is not the owner of the file or a super-user, can change the modification and access times of that file only to the current time. Attempts to set a specific time with touch will result in an error. The settime utility is equivalent to touch -c [date_time] file. OPTIONS
The following options are supported in the touch and settime utilities: touch The following options are supported for the touch utility: -a Changes the access time of file. Does not change the modification time unless -m is also specified. -c Does not create a specified file if it does not exist. Does not write any diagnostic messages concerning this condition. -m Changes the modification time of file. Does not change the access time unless -a is also specified. -r ref_file Uses the corresponding times of the file named by ref_file instead of the current time. -t time Uses the specified time instead of the current time. time will be a decimal number of the form: [[CC]YY]MMDDhhmm[.SS] where each two digits represent the following: MM The month of the year [01-12]. DD The day of the month [01-31]. hh The hour of the day [00-23]. mm The minute of the hour [00-59]. CC The first two digits of the year. YY The second two digits of the year. SS The second of the minute [00-61]. Both CC and YY are optional. If neither is given, the current year will be assumed. If YY is specified, but CC is not, CC will be derived as follows: +-----------------------------------------------------------+ | If YY is: CC becomes: | +-----------------------------------------------------------+ |69-99 19 | |00-38 20 | |39-68 ERROR | +-----------------------------------------------------------+ The resulting time will be affected by the value of the TZ environment variable. If the resulting time value precedes the Epoch, touch will exit immediately with an error status. The range of valid times is the Epoch to January 18, 2038. The range for SS is [00-61] rather than [00-59] because of leap seconds. If SS is 60 or 61, and the resulting time, as affected by the TZ environment variable, does not refer to a leap second, the resulting time will be one or two seconds after a time where SS is 59. If SS is not given, it is assumed to be 0. settime The following option is supported for the settime utility: -f ref_file Uses the corresponding times of the file named by ref_file instead of the current time. OPERANDS
The following operands are supported for the touch and settime utilities: file A path name of a file whose times are to be modified. date_time Uses the specified date_time instead of the current time. This operand is a decimal number of the form: MMDDhhmm[YY] where each two digits represent the following: MM The month of the year [01-12]. DD The day of the month [01-31]. hh The hour of the day [00-23]. mm The minute of the hour [00-59]. YY The second two digits of the year. YY is optional. If it is omitted, the current year will be assumed. If YY is specified, the year will be derived as follows: +-----------------------------------------------------------+ | YY Corresponding Year | |69-99 1969-1999 | |00-38 2000-2038 | |39-68 ERROR | +-----------------------------------------------------------+ If no -r option is specified, no -t option is specified, at least two operands are specified, and the first operand is an eight- or ten-digit decimal integer, the first operand will be assumed to be a date_time operand. Otherwise, the first operand will be assumed to be a file operand. USAGE
See largefile(5) for the description of the behavior of touch when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes). When existing file access and modification times cannot be determined (which will occur if a call to stat(2) fails), both times can be changed to known values by settime and utime(2). However, in this case, touch -a (without -m) and touch -m (without -a) will fail because the unchanged time cannot be preserved. ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of touch: LANG, LC_ALL, LANG, LC_ALL, LC_CTYPE, LC_MESSAGES, and NLSPATH. TZ Determine the timezone to be used for interpreting the time option-argument or the date_time operand. EXIT STATUS
The following exit values are returned: 0 The touch utility executed successfully and all requested changes were made. >0 An error occurred. The touch utility returned the number of files for which the times could not be successfully modified. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Committed | +-----------------------------+-----------------------------+ |Standard |See standards(5). | +-----------------------------+-----------------------------+ SEE ALSO
stat(2), time(2), utime(2), attributes(5), environ(5), largefile(5), standards(5) NOTES
Users familiar with the BSD environment will find that for the touch utility, the -f option is accepted but ignored. The -f option is unnecessary because touch will succeed for all files owned by the user regardless of the permissions on the files. SunOS 5.11 9 Sep 2008 touch(1)
All times are GMT -4. The time now is 06:20 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy