recursive search and ftp Post: 302512153

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers recursive search and ftp Post 302512153 by DGPickett on Friday 8th of April 2011 01:32:51 PM

04-08-2011

Registered User

This should be simple, even with an ftp layer.

Do you need to worry about copying a file while being written? If so, you can detect others on the file with fuser. Can you wait on the writer to finish, near term? Are there other local readers that would confuse fuser? Can you make the files you want to move read-only, immediately once created, so implicity there are not being written?

First, both sender and recipient will cd to the head dir over the subtree in question. I like to use "find * -type f -newer /elsewhere/mark_file ! -newer /elsewhere/mark_next" and a marker file created with touch. 'touch' a new file at the start of every cycle, /elsewhere/mark_next. Then sleep a second so anything written in that second and also the next will be excluded. Sleep longer if it helps eliminate files being written. After the find and send, "mv /elsewhere/mark_next /elsewhere/mark_file" to support the next batch, seamlessly. Pipe the find to a "while read f" do loop that knows how to check for writing users and ftp the file using relative paths. If the files to be moved are read-only, test with 'if [ -w "$f" ]' and if writable, 'touch' their modify time forward so they are considered on the next pass.

Last edited by DGPickett; 04-08-2011 at 02:38 PM..

DGPickett

View Public Profile for DGPickett

Find all posts by DGPickett

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Recursive FTP

I am trying to write a recursive FTP script and have come to a point where I need to test if the file is either a normal ascii file or a directory. My question is how do I test if the file is either ascii or directory.

2. Shell Programming and Scripting

perl + Net::FTP::Recursive

Problem: It will not advance to the next user in the list. It always dies right after it sends the 2/2 files from the first users dir. $USERLIST="/export/home/mxdooley/perl_ftp/userlist"; $USER_DIR="/export/home/mxdooley/perl_ftp/homes";...

3. Shell Programming and Scripting

non recursive search in the current directory only

Hi, Am trying for a script which should delete more than 15 days older files in my current directory.Am using the below piece of code: "find /tmp -type f -name "pattern" -mtime +15 -exec /usr/bin/ls -altr {} \;" "find /tmp -type f -name "pattern" -mtime +15 -exec /usr/bin/rm -f {} \;" ...

4. Shell Programming and Scripting

Recursive FTP -- here at last.

Over the past few weeks, I saw a couple of threads requesting recursive ftp: Copying files between 2 Unix server ftp from NT to UNIX I decided to try to write a script to accommodate these requests. The result is HardFeed. Here are a few examples of what it can do. HardFeed ftpserver...

5. UNIX for Advanced & Expert Users

Recursive directory search using ls instead of find

I was working on a shell script and found that the find command took too long, especially when I had to execute it multiple times. After some thought and research I came up with two functions. fileScan() filescan will cd into a directory and perform any operations you would like from within...

6. UNIX for Advanced & Expert Users

Recursive search for filenames containing ..

Hi all, Since my gopher server doesn't like filenames containing 2 or more consecutive dots in a filename, I'd like to do a search for them and replace them with... well, let's say underscores... I've tried a oneliner or 2 from other posts, but they don't seem to work well with locating dots....

7. Shell Programming and Scripting

recursive search and copy

Hello again. Well, I need help again sooner as I thought. Now I want to search for files with a known name within all subdirs, and copy the to differently named files in the same directory. For example if I had only one file to copy, I would just usecp fileName newFileNamebut to do this...

8. Shell Programming and Scripting

Recursive search on pattern between two strings

Objective: Recursively search all files under a directory for SQL statements that end with ";" Sample input: UPDATE table1 set col=val UPDATE table2 set cola=vala ,colb=valb; UPDATE table3 set col=val Expected output: UPDATE table2 set cola=vala ,colb=valb;

9. OS X (Apple)

Search recursive

before posting, I have tried to find my answer elsewhere. no luck. I need to find a file buried in a folder somewhere. Master folder has 10 sub folders. each sub folder has folders too. I found this but it does nothing I am on Mac and use Applescript. do shell script "find...

LEARN ABOUT OSF1

netrc

netrc(4)						     Kernel Interfaces Manual							  netrc(4)

NAME

       netrc, .netrc - Specifies automatic login information for ftp

SYNOPSIS

       $HOME/.netrc

DESCRIPTION

       The  .netrc  file contains the information used by the automatic login (autologin) feature of the ftp command.  It is a hidden file in your
       home directory and must be owned by the user logging in, or by the root user.   If the .netrc file contains a login  password,  the  file's
       permissions must be set to 600 (read and write by owner only).

       The  .netrc file can contain the following entries (separated by spaces, tabs, or newlines): Where host is the name of a remote host.  This
       entry begins the definition of the autologin process for the specified host.  All following entries up to the next machine entry or the end
       of  the	file apply to that host.  This is the same as machine, except that default matches any name.  There can be only one default token,
       and it must appear after all machine tokens.  This is normally used as follows: default login anonymous password  user@site  The  preceding
       command	line gives the user automatic anonymous ftp login to machines not specified in .netrc.	This can be overriden by using the -n flag
       to disable autologin.  Where user is the username to use at the remote host.  If this entry is found, the  autologin  process  initiates  a
       login  using  the specified name.  If this entry is missing, the autologin process fails.  Where password is the login password to be used.
       The autologin process supplies this password to the remote server.  A login password must be established at the remote host and that  pass-
       word  must  be  entered	in  this  file, or the autologin process fails and you are prompted for the login password.  Where password is the
       account password to be used.  If this entry is found and an account password is required at the remote host, the autologin process supplies
       the  password  to  the  remote  server.	 If  the remote host requires an account password but this entry is missing, the autologin process
       prompts for the account password.  Where macro is the name of an ftp subcommand macro.  The macro definition starts on the  following  line
       and  is	defined  to  contain all of the following ftp subcommands up to the next blank line.  If the macro is named init, ftp executes the
       macro upon successful completion of the autologin process.

FILES

       Contains automatic login information.

RELATED INFORMATION

       Commands:  ftp(1).  delim off

																	  netrc(4)