Sponsored Content
Top Forums Shell Programming and Scripting sftp - get newly created files on incremental basis Post 302480382 by ravi.videla on Tuesday 14th of December 2010 06:08:18 PM
Old 12-14-2010
sftp - get newly created files on incremental basis

Hi,

We have a sftp server which creates files daily and keeps 6 months of files on the server. We are creating a daily job to get the files and load into database. My problem is "how to get ONLY those files which got created after my last get". Let me provide some more details to it.

Below example shows files available on server as of day1, 2,3.

Code:
Day1:
customer_data_2010-12-09.2010-12-10_03.15.01.zip
customer_data_2010-12-09.2010-12-10_03.15.01.zip
customer_data_2010-12-10.2010-12-10_19.15.02.zip
customer_data_2010-12-10.2010-12-10_19.15.02.zip

Day2:
customer_data_2010-12-09.2010-12-10_03.15.01.zip
customer_data_2010-12-09.2010-12-10_03.15.01.zip
customer_data_2010-12-10.2010-12-10_19.15.02.zip
customer_data_2010-12-10.2010-12-10_19.15.02.zip
customer_data_2010-12-10.2010-12-11_03.15.01.zip
customer_data_2010-12-10.2010-12-11_03.15.01.zip

Day3:
customer_data_2010-12-09.2010-12-10_03.15.01.zip
customer_data_2010-12-09.2010-12-10_03.15.01.zip
customer_data_2010-12-10.2010-12-10_19.15.02.zip
customer_data_2010-12-10.2010-12-10_19.15.02.zip
customer_data_2010-12-10.2010-12-11_03.15.01.zip
customer_data_2010-12-10.2010-12-11_03.15.01.zip
customer_data_2010-12-11.2010-12-11_19.15.01.zip
customer_data_2010-12-11.2010-12-11_19.15.01.zip
customer_data_2010-12-12.2010-12-13_03.15.01.zip
customer_data_2010-12-12.2010-12-13_03.15.01.zip

To get files, Day1 is easier as we can simply get all 4 files. You can see two more files got created on day2. So we should get only those 2 onto database server. On day3, we see 4 more new files got created. Please let me know if you think of any easy way to get files from sftp server that are created after last get. Appreciate your help on this.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

How to FTP all newly created but the current open file?

An application running on HP-UX constantly generates new text log files ( I think using logpipe ). Any new file created requires to be ftp'ed to an offline server, however I want to make sure that the current file being written should not be transferred. For examples consider the following files... (3 Replies)
Discussion started by: indianya
3 Replies

2. UNIX for Dummies Questions & Answers

Newly created files default group and write permissions

Whenever I create a new file the group name is "dnn" and the file permissions are "-rw-r--r--". How do I get it so when I create files (with vi or other programs) that the default group is "sss" and the permissions are 770? (I am running HP-UNIX) Thanks, GoldFish (2 Replies)
Discussion started by: goldfish
2 Replies

3. UNIX for Dummies Questions & Answers

Forcing UID on Files/Dirs Created with SFTP?

I have a situation where I have to provide Windows based users with access to specific files and directories on a *nix web mail server. The users cannot use the CLI, so SSH is out. They've previously used a product called Webdrive to access *nix boxes via SFTP. The files and dirs they need... (5 Replies)
Discussion started by: deckard
5 Replies

4. UNIX for Dummies Questions & Answers

Help with command to find all newly created files in a given time period

Hi all! Can someone please help me create a command to accomplish the following task. I have a parent directory called ex. /var/www/parent and it has a bunch of sub-directories called /var/www/parent/1, var/www/parent/1/xyz/ and etc. What I would like to do is to count the number of files... (2 Replies)
Discussion started by: bbzor
2 Replies

5. Shell Programming and Scripting

How to find the newly created directory

Hi, I need to create new directory by increasing the number by 1 of extracted lastly created directory. e.g. Log\out_log_1\ Log\out_log_2\ Log\out_log_3\ become Log\out_log_1\ Log\out_log_2\ Log\out_log_3\ Log\out_log_4\ Can anyone help how to do it in c-shell... (3 Replies)
Discussion started by: Andre_2008
3 Replies

6. UNIX for Advanced & Expert Users

default size of a newly created folder

Hi all, In linux how to create a directory with specified size, so that it can be used only up to the mentioned size. Actually my question is, whether we can do directory quota in linux. mounting the directory in a partiton will do that, but do we have any other option... (1 Reply)
Discussion started by: anishkumarv
1 Replies

7. Solaris

Can't see Newly created LUN by SAN admin

hello, i am an oracle DBA and trying to scan a newly created LUN of 200 GB on fiber channel by SAN admin.we have solaris 10 and SANtoolkit is installed.i tried following to get the new LUN at my machine. go /opt/Netapp/Santoolkit/bin and then ./sanlun lun show but i see only the existing... (12 Replies)
Discussion started by: janakors
12 Replies

8. Shell Programming and Scripting

Copy an array to a newly created directory

hello everyone, I am new to perl script and trying to develop a script as follows. I am trying to Create an array for storing all file names. I am trying to copy $libs into "scratch". however i am unable to do so. Please suggest.. #!/usr/bin/perl use File::Copy; #use... (5 Replies)
Discussion started by: Rashid Khan
5 Replies

9. Hardware

Formatting a newly created lun

Hi , I have created one new lun in my SAN storage and make it visible to my HP servers , but the fdisk -l output is somehow confusing. Do not know what to do next ---------- fdisk -l /dev/sdo1 Disk /dev/sdo1 (Sun disk label): 64 heads, 32 sectors, 10238 cylinders Units =... (7 Replies)
Discussion started by: mishra.sankar
7 Replies

10. AIX

No iscsi available in newly created AIX wpar

AIX 7.1 New to WPAR, hopefully just missing something simple here. Creating the WPAR like this..... (The box where the WPAR is hosted does have an iscsi protocol device) mkwpar -h wpar08 -l -n wpar08 -N interface=en0 address=xxx.xx.xx.xxx netmask=255.255.255.0 -D devname=/dev/iscsi0 -D... (0 Replies)
Discussion started by: TomR
0 Replies
ZIPARCHIVE.OPEN(3)							 1							ZIPARCHIVE.OPEN(3)

ZipArchive::open - Open a ZIP file archive

SYNOPSIS
mixed ZipArchive::open (string $filename, [int $flags]) DESCRIPTION
Opens a new zip archive for reading, writing or modifying. PARAMETERS
o $filename - The file name of the ZIP archive to open. o $flags - The mode to use to open the archive. o ZipArchive::OVERWRITE o ZipArchive::CREATE o ZipArchive::EXCL o ZipArchive::CHECKCONS RETURN VALUES
o $Error codes - Returns TRUE on success or the error code. o ZipArchive::ER_EXISTS File already exists. o ZipArchive::ER_INCONS Zip archive inconsistent. o ZipArchive::ER_INVAL Invalid argument. o ZipArchive::ER_MEMORY Malloc failure. o ZipArchive::ER_NOENT No such file. o ZipArchive::ER_NOZIP Not a zip archive. o ZipArchive::ER_OPEN Can't open file. o ZipArchive::ER_READ Read error. o ZipArchive::ER_SEEK Seek error. EXAMPLES
Example #1 Open and extract <?php $zip = new ZipArchive; $res = $zip->open('test.zip'); if ($res === TRUE) { echo 'ok'; $zip->extractTo('test'); $zip->close(); } else { echo 'failed, code:' . $res; } ?> Example #2 Create an archive <?php $zip = new ZipArchive; $res = $zip->open('test.zip', ZipArchive::CREATE); if ($res === TRUE) { $zip->addFromString('test.txt', 'file content goes here'); $zip->addFile('data.txt', 'entryname.txt'); $zip->close(); echo 'ok'; } else { echo 'failed'; } ?> PHP Documentation Group ZIPARCHIVE.OPEN(3)
All times are GMT -4. The time now is 08:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy