processing an input file and then the output file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting processing an input file and then the output file
# 1  
Old 02-18-2011
processing an input file and then the output file

Hi.

I am new to scripting and could really do with some advice on the best way to put a script together. Here is the scenario I am working to;

- i will get files via ftp to a tmp directory on the server
- all files will have a unique file name but with the same extension (.USM)
- for each file I need to run an application process that creates a new file by prefixing each file with 'mt' and appending '_file_01
- therefore from 1234.USM a new file name mt1234_file_01 will be created.
- a second process will then take mt1234_file_01 and make mt1234_file_02
- once the application processes have been created I will then need to move all the files to a new directory so that the FTP side of the script can run the next day and only new files are processed again.

The bit I am stuck on is defining the three types of files so that I can run a foreach command processing that set of files.

I have something if there is just one set of files which is like this;
PHP Code:
while (== 1)
   if ( 
$status == then
    set MARC_FILES 
= (*.USM)
   else
      
set MARC_FILES " "
   
endif
   foreach 
($MARC_FILES)
       if ( 
"$F!= "*.USM" then
# -- application script that takes a file and makes a new one
       
csh -f $proc/p_file_01 TST01,$F,mt$F_file01,
         
set MARCLOADERRORS=0
         
echo " " $DATA_DIR/marcload_email_txt.dat
endif
end 
What I need some help with is the logic in doing a foreach of the 1st set of newly created files and then the 2nd set of newly created files.

$F is set in the script above.
$F1 should me something like mt$F_file_01 ($F needs to lose the .USM extension)
$F2 should then be mt$F_file_02 (again where $F needs to lose the .USM extension)


help....
any pointers would be great as I am not sure where the best place is to start.
thanks,
Mark

---------- Post updated at 05:00 PM ---------- Previous update was at 04:59 PM ----------

forgot to mention. this is using .csh

the application scripts are using .csh so i wanted to keep this additional script in the same programming language.
# 2  
Old 02-20-2011
How about this:

Code:
set BN=`basename $F .USM`
set F1=mt${BN}_file_01
set F2=mt${BN}_file_02

B.T.W., I dont think if ( "$F" != "*.USM" ) then is doing what you want. Use switch/case to do wildcard tests:
Code:
switch ( $F )
  case "*.USM":
     # Code to process *.USM here ...
  breaksw;
endsw


Last edited by Chubler_XL; 02-20-2011 at 09:03 PM.. Reason: Fixup rogue color code formatting
# 3  
Old 02-21-2011
thanks for the advice Chubler_XL. I made a test script and managed to get someething to work if there was only one file named .USM.

PHP Code:
set F=("*.USM")
set BN=`basename $F .USM`
set F1=mt${BN}_file_01
set F2
=mt${BN}_file_02

echo $BN
echo $F1
echo $F2

switch ( $F )
  case 
"*.USM":
   echo 
'real script will change' $BN 'into '$F1
   
echo 'real script will then change '$F1 ' into '$F2
  breaksw
;
endsw 
The above script fails if there are multiple files with .USM extensions. I get this messages 'basename: too many arguments', '*.USM: Ambiguous.'

Could you help any further please?
# 4  
Old 02-21-2011
I was thinking more along the lines of this

Code:
#!/bin/csh
foreach F (*)
   switch ($F)
       case "*.OTH":
           echo "Some other processing you might want to do with .OTH file - $F"
       breaksw;
       case "*.USM":
           set BN=`basename $F .USM`
           set F1=mt${BN}_file_01
           set F2=mt${BN}_file_02
           echo 'real script will change' $BN 'into ' $F1
           echo 'real script will then change' $F1 'into ' $F2
        breaksw;
   endsw
end

# 5  
Old 02-22-2011
thanks again for your help and i've now got this working.

i'm stuck on a new problem that i am trying to solve. my original plan was to download all the files and process them in a holding directory. then, after processing, i was going to move them to the correct directory along with past files.

i can't do this now as the scripts need to run in a specific directory.

therefore the problem i am trying to work out is how to calculate what existing .USM files there are in a folder and run the scripts for each new .USM file only. as you can tell i am well outside my comfort zone...

---------- Post updated at 04:20 PM ---------- Previous update was at 02:43 PM ----------

i think i have the answer...

i download the files to a holding directory. Then do a

PHP Code:
ls *USM >> textfile.txt 
then i can move all my files to the directory where they are to be processed.

and then i can use;
PHP Code:
foreach (`cat textfile.txt`) 
this does a foreach on only the files listed in the text file. the script will not process old files in that directory as they are not on that list.

Thanks for all your help.
# 6  
Old 02-22-2011
if 1234.USM is file comin everyday. then you need that like this "mt1234_file_01".
then use cat 1234.USM > $TgtPath/mt1234_file_01.
just save that file in one directory like mt1234_file_01.
after file creation completed then take the last file and store that last two numbers into one variable. after that increment +1 to that and save the next file.

if you use date then files should not overwrite.

regards
rajesh
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to reformat output if input file is empty, but not if file has data in it

The below awk improved bu @MadeInGermany, works great as long as the input file has data in it in the below format: input chrX 25031028 25031925 chrX:25031028-25031925 ARX 631 18 chrX 25031028 25031925 chrX:25031028-25031925 ARX 632 14... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. UNIX for Dummies Questions & Answers

awk - Rename output file, after processing, same as input file

I have one input file ABC.txt and one output DEF.txt. After the ABC is processed and created output, I want to rename ABC.txt to ABC.orig and DEF to ABC.txt. Currently when I am doing this, it does not process the input file as it cannot read and write to the same file. How can I achieve this? ... (12 Replies)
Discussion started by: High-T
12 Replies

3. Shell Programming and Scripting

How to read each line from input file, assign variables, and echo to output file?

I've got a file that looks like this (spaces before first entries intentional): 12345650-000005000GL140227 ANNUAL HELC FEE EN 22345650-000005000GL140227 ANNUAL HELC FEE EN 32345650-000005000GL140227 ANNUAL HELC FEE EN I want to read through the file line by line,... (6 Replies)
Discussion started by: Scottie1954
6 Replies

4. Homework & Coursework Questions

Loop to Convert a list from an input file and output it to another file

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: A) Write a script, which will take input from a file and convert the number from Centigrade to Fahrenheit... (5 Replies)
Discussion started by: AliTheSnake
5 Replies

5. Shell Programming and Scripting

Recursive file processing from a path and printing output in a file

Hi All, The script below read the path and searches for the directories/subdirectories and for the files. If files are found in the sub directories then read the content of the all files and put the content in csv(comma delimted) format and the call the write to xml function to write the std... (1 Reply)
Discussion started by: Optimus81
1 Replies

6. UNIX for Dummies Questions & Answers

12. If an ‘88’ Record with BAI Code ‘902’ was found on input file and not written to Output file, re

This is my input file like this 03,105581,,015,+00000416418,,,901,+00000000148,,,922,+00000000354,,/ 49,+00000000000416920,00002/ 03,5313236,,015,+00231036992,,,045,+00231036992,,,901,+00000048428,,/ 88,100,+0000000000000,0000000,,400,+0000000000000,0000000,/ 88,902,+0000000079077,,/... (0 Replies)
Discussion started by: sgoud
0 Replies

7. Shell Programming and Scripting

split input data file and put into same output file

Hi All, I have two input file and need to generate a CSV file. The existing report just "GREP" the records with the Header and Tailer records with the count of records. Now i need to split the data into 25 records each in the same CSV file. id_file (Input file ) 227050994 232510151... (4 Replies)
Discussion started by: rasmith
4 Replies

8. Solaris

Processing user input in .sh file

Hi, I am new to shell scripting. script should accept the user value and then compare that value with the null. If null then assign the value "*" to the variable else will use the user inputed value. How to do this ? With Regards (3 Replies)
Discussion started by: milink
3 Replies

9. Shell Programming and Scripting

AWK Script to convert input file(s) to output file

Hi All, I am hoping someone can help me with some scripting I need to complete using AWK. I'm trying to process multiple fixed files to generate one concatenated fixed file in a standard format. The Input file is:- aaaa bbbbb ccccc 1 xxxx aaa bbb aaaa bbbbb ccccc 2 abcd aaa CCC... (9 Replies)
Discussion started by: jason_v_brown
9 Replies

10. Shell Programming and Scripting

Dynamic output file generation using a input text file with predefined output format

Hi, I have two files , one file with data file with attributes that need to be sent to another file to generate a predefined format. Example: File.txt AP|{SSHA}VEEg42CNCghUnGhCVg== APVG3|{SSHA}XK|"password" AP3|{SSHA}XK|"This is test" .... etc --------- test.sh has... (1 Reply)
Discussion started by: hudson03051nh
1 Replies
Login or Register to Ask a Question