Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Automatically Load data from all files in directory

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Automatically Load data from all files in directory
# 1  
Old 10-04-2008
Error Automatically Load data from all files in directory
I'm new in Unix shell scripting and i need someone to help me to make Script that run all time watching my directory that files uploaded to it via FTP (/mydir/incoming_files), if any files exists in it then (if many files exists, then sort files and load them ascending) it‘ll checks the size of the file to make sure that it is not still uploading before it starts load data using oracle sql loader, if size of file is zero then move file to (/mydir/zero_size) directory, else use oracle sql loader to load data to oracle database , after finishing loading data from this file move it to loaded file directory (/mydir/loaded_files) and go to second file if exist do the same like first all time until the directory will be empty. (The directory will be some times empty if files to be uploaded is late but the default is not empty because files will be continue to be uploaded 24/7 *30 )


File name will be <in>_<some sequence>_<DDMMYYYY>_<HHMMSS>.dat


System HP UX and Oracle Database


Directories used:
  1. Script directory.(/mydir/scripts) contain scripts used
  2. Incoming files directory. (/mydir/incoming_files)
  3. Loaded files directory (/mydir/loaded_files) in this directory files that loaded to database will be moved to it.
  4. Discarded data files (/mydir/disc_files). This will be used by oracle sql loader to put discarded data file in it , and I need to make it name like <disc>_<original file name>.bad so if file name is in_000023_24082008_182433.dat contain discarded data then discarded file will be disc_in_000023_24082008_182433.bad
  5. Log files directory (/mydir/log_files) this contain log files created by oracle sql loader, and every file will be like it's incoming files with ext. log (in_000023_24082008_182433.log)
  6. Zero size files (/mydir/zero_size) move zero file size to it.
# 2  
Old 10-04-2008
You would help yourself if you also posted some of the script you already have. We do not write scripts from scratch here, but are willing to help those that help themselves
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Automatically determining directory path for scripts and programs

I have some C++ code in the following directory structure /home/chrisd/tatsh/trunk/hstmy/ ├── baseLib ├── bin │ ├── awk │ ├── bash │ ├── diag │ ├── ksh │ │ └── TAG201011 │ ├── old │ ├── perl │ ├── prog │ ├── py │ └── tcsh ├── docs ├── fortran ├── others... (0 Replies)
Discussion started by: kristinu
0 Replies

2. AIX

How to automatically load a script once I login?

Hi, may I know how to automatically load a script once I login? For example, I want to load "set -o vi" command once I login in my Aix. (2 Replies)
Discussion started by: ngaisteve1
2 Replies

3. Shell Programming and Scripting

Automatically execute all programs in a directory

Hello. The last days I extended an open-source C++ library by some faster functions. This library provides a lot of short test programs which guarantee that the library works exactly. The compilation of each test works in a Makefile automatically. What I need is a shell script which... (5 Replies)
Discussion started by: max3000
5 Replies

4. Shell Programming and Scripting

Moving files listed in a data file to a new directory using Perl

Hi, I have a data file that lists a number of files. I want to move the files named in that one to another directory. Here's what I have: #!/usr/bin/perl -w open(FILE, "<collision.txt"); my @lines=<FILE>; foreach my $lines (@lines) { system("mv $lines collisions/."); } close(FILE); ... (2 Replies)
Discussion started by: renthead720
2 Replies

5. Shell Programming and Scripting

sh and MySQL LOAD DATA

I have a csv file in a folder that is roughly 500,000 rows long. Rather than using PHP, I would like to use SH to run a MYSQL LOAD DATA command to load the data in, as I think it would be much faster and would not cause any memory problems associated with PHP. But the problem is, I am not... (18 Replies)
Discussion started by: worchyld
18 Replies

6. Shell Programming and Scripting

Checking the directory and concatenate the data of all the log files in that dir

Hi Gurus, I am new to unix and need your help to make a shell script. I have a requirement, would appreciate if you could please help me on it: Requirement: ------------- I will pass 2 parameters in shell script 1). Directory name say errors 2). file extension say .log First of all this... (4 Replies)
Discussion started by: anshulinpc
4 Replies

7. UNIX for Dummies Questions & Answers

Howto capture data from rs232port andpull data into oracle database-9i automatically

Hi, i willbe very much grateful to u if u help me out.. if i simply connect pbx machine to printer by serial port RS232 then we find this view: But i want to capture this data into database automatically when the pbx is running.The table in database will contain similar to this view inthe... (1 Reply)
Discussion started by: boss
1 Replies

8. Shell Programming and Scripting

Need help in wrting Load Script for a Load-Resume type of load.

hi all need your help. I am wrting a script that will load data into the table. then on another load will append the data into the existing table. Regards Ankit (1 Reply)
Discussion started by: ankitgupta
1 Replies

9. Shell Programming and Scripting

Removing files automatically from a directory after 30 days.

Hello; I have a directory that is collecting log and act files. I need to write a script that will remove these files once they are 30 days old. I have read through a number of threads on this site that have given me a great deal of information. However I have what seems to be a unique... (7 Replies)
Discussion started by: justinb_155
7 Replies

10. Shell Programming and Scripting

Checking and chaning directory permissions automatically

Hi all, My first post here. I need a small script to check the directory permssions on my /home/uploads and if there are any newly created directory in uploads chmod them to 1777. The upload directory is for my users who upload their pictures and I by default their directories are given... (4 Replies)
Discussion started by: apachi
4 Replies
Login or Register to Ask a Question