Generate file and Upload to SFTP server.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Generate file and Upload to SFTP server.
# 1  
Old 03-15-2013
Oracle Generate file and Upload to SFTP server.

Xperts,

My requirement is something like this,

I have a sql script which i need to embed in a shell. The sql (oracle) script will generate a .csv file in some Unix directory. the approximate file size is around 10 mb which i need to upload to an sftp server. My concern here is how to make sure that the file generation is completed . Manually when i am running the script its taking 30 mins and the generated file size goes on increasing. Depending upon the table data the script may run for a long time too increasing the file size.

Any expert advice is deeply appreciated.Smilie
# 2  
Old 03-15-2013
Well, fuser will tell you if it is still open for output.

If you have ssh, you can pipe it up in real time to a temp name and decide later if it completed.

You could write a magic string on the file end, perhaps in SQL, and use sed -i or the like to remove it later.

You could run the query up there over tcp/ip and save the copying. The term tlisten comes to mind. Excel data import odbc is nice, too, and Access odbc link.

Often, you can see errors or success in the log if you divert non-data lines there.
# 3  
Old 03-15-2013
Thanks mate,
I will give a try for fuser. Is there any way to automate using Proginet?
# 4  
Old 03-16-2013
Hi DGPickett,


I tried the fuser. but it didn't display any pid for the file while that file generation is still in progress by an oracle instance....

Thanks for your support. And can u please tell me abt the magic string u mentioned as i am a newbie to scripting...Smilie
# 5  
Old 03-19-2013
Oracle

Hi DGPickett,


Actually I tried the fuser in my system and it worked fine. But in my Workstation i issued the following command
Quote:
/sbin/fuser CA_DBLT_18th_67567.csv
But it didn't return any pid where as i am sure that file writting is in progress. Pls suggest. I have also tried lsof but of no use.
# 6  
Old 03-19-2013
Well, fuser works in UNIX for all users and files, but maybe elsewhere it might balk at other users or their files.

If you select or echo some odd string on the end of the file (being careful it is not sorted), then when it arrives, you are done. Some people like to send a row total or checksum, for validation. Of course, something terrible might have gone wrong in an earlier query, so egrepping for nasty or happy phrases in the log is also good. And of course, you have to remove the trailer from the tabular data at some point.

Many apps write a substitute path until things are done, like downloads going into an *.part file. For uploads, you can upload each file to temp.tmp and then move/mv/rename it when you like it, which might just be when the upload is done, or when it also has the right size. For downloads, you need the other end to signal the file is ready, and then you need to mark the file ready to process, and again, renaming is very effective. The directory inode lock is your lock.

Some sites send an ack file with the count or empty, just to say the other, main file is OK, but I prefer using the name itself. What happens if an ack file is left around (unless the name has a date component)?
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File upload message in SFTP

Hi, Below script is running on AIX 7.1 ( 7100-04-05-1720 version ) server. Recently OpenSSH version installed on server got updated from OpenSSH_6.0p1 to OpenSSH_7.5p1 version. After this update we do not receive any file upload message after put/mput command in SFTP. sftp -b - user@server... (1 Reply)
Discussion started by: Juggernaut
1 Replies

2. Proxy Server

Upload file from desktop to UNIX server via putty

I have a requirement of following - 1. Transfer excel spreadsheet from desktop to unix server 2. Open the spreadsheet and read the sql row by row at a time 3. Run each sql on database using unix and then send output back to spreadsheet that was uploaded earlier and then transfer file back to... (12 Replies)
Discussion started by: Analabhatla
12 Replies

3. Shell Programming and Scripting

Script to upload latest file to other server via FTP

Hello, I have a script that finds the latest version of a file in a folder on my Minecraft server. I'm trying to come up with something that will then FTP that file over to my fileserver. Here's what I have that finds the newest file: find /home/mc/archive/sbhouse -type f -mtime +45 -exec... (7 Replies)
Discussion started by: nbsparks
7 Replies

4. Ubuntu

I want to upload file on remote machine in noninteractive mode through SFTP

Hi All, I want to upload file through SFTP in non interactive mode on remote server. please tell me what will have to do in oreder to do SFTP . (1 Reply)
Discussion started by: kulbhushan
1 Replies

5. Shell Programming and Scripting

How to Write sftp through "expect" for file upload ?

Hi Experts , I am new to unix programming please tell me how to write expect and hoe to call it for automated file upload process. help me really ! (0 Replies)
Discussion started by: kulbhushan
0 Replies

6. Shell Programming and Scripting

upload file to webdav server

Hi all i have a such question this is my shell script my script must upload file to webdav server ////////////////////////////////////////////////////////////////////////////////////////////////////////////// #!/bin/sh ###... (1 Reply)
Discussion started by: knut
1 Replies

7. Shell Programming and Scripting

sftp file upload problem

Hi All, I am trying to upload a text file from HP unix to Windows tectia server using sftp, the text file shows with new line character after upload. For EG : abc.txt file contains 123 456 aftre upload it shows as 123 456 i am using sftp version 2.0 TQ, (4 Replies)
Discussion started by: phani1312
4 Replies

8. Shell Programming and Scripting

upload excel file contents to server

Hi I have an excel file with only 5 rows .I want to uplaod those 5 rows to a database using a shell script.Is this possible.If so where do i start. Thnks (1 Reply)
Discussion started by: magikminox
1 Replies

9. Shell Programming and Scripting

stuck in perl cgi to upload a file to server

hi, i m working on a perl cgi script which uploads a file to the server. i m stuck. i hav written the errors. plz help. Sachin Kaw ______________________________________________________________________ #!/usr/bin/perl -w use CGI; use CGI qw(:standard); use strict; use POSIX... (4 Replies)
Discussion started by: sachin_kaw
4 Replies
Login or Register to Ask a Question