I need to read contents of directory and create a list of data files that match a certain pattern and process by renaming it and calling a existing .ksh script then archiving off to file another directory. Any suggestions or samples u could point me to on using .ksh perl or other to process through the list of data files? What would you recommend to strip off a suffix from a data file?. tia

test message

Please supply input files, data and desired results. It is much easier to understand from data rather than describing the situation.

To handle filename issues, do a >man basename
not clear exactly what you are trying to do.

file names:

BTIME_ACTUAL_WORK_HRS_wk_45_to_46.csv
BTIME_ACTUAL_WORK_HRS_wk_46_to_47.csv
...
sample processing:

1) strip off/rename each data file
e.g. BTIME_ACTUAL_WORK_HRS_wk_45_to_46.csv to
BTIME_ACTUAL_WORK_HRS.csv

2) call existing .ksh script (it will reference BTIME_ACTUAL_WORK_HRS.csv) each time

3) move/copy of BTIME_ACTUAL_WORK_HRS_wk_45_to_46.csv moved to /archive directory.

4) process next file

Ensure each file is only processed once
Copy of originally named file is archived
Process only non-zero byte files or wc > 3
Assumes job will be called from scheduler when files are ready/waiting

tia

I would humbly suggest you fix the existing ksh script to accept a file name argument. This is begging for ending up in a situation where you are not sure which file you have processed.

for f in BTIME_ACTUAL_WORK_HRS_*.csv; do
  cp -f "$f" BTIME_ACTUAL_WORK_HRS.cvs
  horrendous_legacy.ksh --majestic --righteous # FIXME: make it accept name of csv file
  mv "$f" archive/  # assume it's not really /archive/
done

I agree but the existing .ksh calls an Informatica process that references a specific flat file for a source. It doesn't allow a dynamic flat file source name so I have to replace the existing one it reads...