Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Script to loop through the files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to loop through the files
# 1  
Old 11-29-2013
Script to loop through the files
Hi Experts,
Need some help regarding a requirement --

I may get any number of zipped files in a month (max 36). each of those wil contain a detail file and a header file. Also when the files arrive, there will already be previous days file lying around.

What I am trying to do is count the number of detail files and then loop through them and concatenating them (each corresponding header to deatil rec ) and then feed them to informatica one by one.

I have the below code --

Code:
i=1
while [ "$i" -lt `ls c4_ai_account[0-9][0-9]_`date +%Y%m%d`.dat | wc -w`+1 ]
do
    echo | cat c4_ai_account_[0-9]"$i"_`date +%Y%m%d`.hdr -  c4_ai_account_[0-9]"$i"_`date +%Y%m%d`.dat  > c4_account_test"$i".txt

echo "$i"
i=`expr $i + 1`
done

the file name will look like below
Code:
-rwxrwxrwx    1 l038028  users             6 Nov 18 13:35 c4_ai_account_01_20131129.dat
-rwxrwxrwx    1 l038028  users             8 Nov 18 13:35 c4_ai_account_02_20131129.dat
-rwxrwxrwx    1 l038028  users             9 Nov 18 13:35 c4_ai_customer_01_20131129.hdr
-rwxrwxrwx    1 l038028  users             9 Nov 18 13:35 c4_ai_account_01_20131129.hdr
-rwxrwxrwx    1 l038028  users             8 Nov 18 13:35 c4_ai_customer_02_20131129.hdr
-rwxrwxrwx    1 l038028  users             8 Nov 18 13:35 c4_ai_account_02_20131129.hdr
-rwxrwxrwx    1 l038028  users             7 Nov 18 13:36 c4_ai_customer_02_20131129.dat
-rwxrwxrwx    1 l038028  users            12 Nov 18 13:36 c4_ai_customer_01_20131129.dat
-rwxrwxr-x    1 l038028  users             6 Nov 29 13:38 c4_ai_account_03_20131129.dat
-rwxrwxr-x    1 l038028  users             9 Nov 29 13:39 c4_ai_account_03_20131129.hdr
-rwxrwxr-x    1 l038028  users             6 Nov 29 15:48 c4_ai_account_01_20131128.dat
-rwxrwxr-x    1 l038028  users             9 Nov 29 15:48 c4_ai_account_01_20131128.hdr

The above script is giving error -- The file c4_ai_account[0-9][0-9]_ does not exist.

Please advise on how I can achieve this. I am new to unix so any help will be greatly apprecaited.
Last edited by vbe; 11-29-2013 at 06:12 AM.. Reason: missing code hehe
# 2  
Old 11-29-2013
Code
I think breaking codes will be easier
Here's how it goes
Code:
dt=`date +%Y%m%d` # file's datstamp
ls -1 c4_ai_account[0-9][0-9]_${dt}.dat|wc -l>/tmp/detailfile #copying detail files to tmp file
for f in `cat /tmp/detailfile` #looping all files
do
fname=`cut -f1 -d'.'` #saving sile name
hdr_name=${fname}.hdr #creating header file name
cat $f>>$hdr_name #concatanating detail file to header
done

Hope this will hel[
Last edited by vbe; 11-29-2013 at 06:13 AM.. Reason: code tags - not icodes!
# 3  
Old 11-29-2013
i think you have forget a _ in your file name

c4_ai_account_[0-9][0-9]_
# 4  
Old 11-29-2013
Hi ashkul123...

Take a deep look at your error report, note your "while" line!
Code:
The file c4_ai_account[0-9][0-9]_ does not exist.

Now look at you filenames...
Code:
c4_ai_account_01_20131129.dat

Let us build it up you file correctly:-
Code:
currentdate=`date +%Y%m%d`
c4_ai_account_[0-9][0-9]_"$currentdate".dat

This User Gave Thanks to wisecracker For This Post:
ashkul123
# 5  
Old 11-29-2013
Not sure I understand your requirement correctly, but - if you want to concatenate .hdr files and .dat files, try this:
Code:
for i in c4*.hdr; do echo cat $i ${i/hdr/dat} ">" ${i/hdr/tot}; done

Remove the echo when happy with the result; you may add some tidying up after the concat succeeded.
# 6  
Old 12-01-2013
Hey guys, just came and saw the messages. Thank you so much for the help. Will try them out and revert back.

Cheers

---------- Post updated at 07:47 PM ---------- Previous update was at 06:42 PM ----------

Thanks, it worked.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Loop Script and not opening files containing spaces

Hello, I wrote a simple script, that basically wait for a *.dat-file in a certain folder, which is always a zipped file and extracts it. It worked before and i changed nothing in the script, but since last week i have the problem, that it doesnt extract files containing a space. How do i make... (4 Replies)
Discussion started by: blend_in
4 Replies

2. UNIX for Beginners Questions & Answers

Shell script to loop through files

Hi Team, I am new to shell scripting. I have the below requirement 1) Say if i am searching for 20160815 in a directory /dir 2) Now i need to get the files present in dir whose time stamp in greater than or equal to 20160815 3) Then i need to find the string 20160815 from the set of... (3 Replies)
Discussion started by: Rajendra Kalepu
3 Replies

3. Shell Programming and Scripting

Loop through multiple files in bash script

Hi Everybody, I'm a newbie to shell scripting, and I'd appreciate some help. I have a bunch of .txt files that have some unwanted content. I want to remove lines 1-3 and 1028-1098. #!/bin/bash for '*.txt' in <path to folder> do sed '1,3 d' "$f"; sed '1028,1098 d' "$f"; done I... (2 Replies)
Discussion started by: BabyNuke
2 Replies

4. UNIX for Dummies Questions & Answers

Using Shell Script To Loop Program Through Multiple Text Files

Hello, So I have approximately 300 files of raw data (.txt) files that I am using to perform statistical analysis. I have been able to construct a Fortran program that is able to perform my statistical analysis on a file by file basis. However, I now want to be able to loop program through... (19 Replies)
Discussion started by: Jimmyd24
19 Replies

5. UNIX for Dummies Questions & Answers

Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files: for i in 250.*; do || awk... (4 Replies)
Discussion started by: evelibertine
4 Replies

6. UNIX for Dummies Questions & Answers

Writing a loop to manipulate a script and store it in multiple output files

I have a script where the the 9th line looks like this: $filename=sprintf("250.1chr%d.ped", $N); I want to modify this script 1000 times, changing 250.1chr%d.ped to 250.2chr%d.ped, 250.3chr%.ped.......and so on all the way to 250.1000chr%d.ped and store each output in files called ... (4 Replies)
Discussion started by: evelibertine
4 Replies

7. UNIX for Dummies Questions & Answers

Foreach loop to run a perl script on multiple files

Hi, I have thousands of files in a directory that have the following 2 formats: 289620178.aln 289620179.aln 289620180.aln 289620183.aln 289620184.aln 289620185.aln 289620186.aln 289620187.aln 289620188.aln 289620189.aln 289620190.aln 289620192.aln.... and: alnCDS_1.fasta (1 Reply)
Discussion started by: greptastic
1 Replies

8. AIX

how to loop through non-empty files with shell script on AIX

I have av script that loops through some statistic files to create a report. We would like to only loop through non-empty files as these files create an empty report-line. I have figured out how to find the non-empty files, but not how to loop through only those files. Here is the code that finds... (4 Replies)
Discussion started by: Tessa
4 Replies

9. Shell Programming and Scripting

Script to loop through all files and directories.

I'm trying to write a script that will loop through all files and directories down from a path I give it, and change the permissions and ACL. I was able to do the obvious way and change the files and folders on the same level as teh path...but I need it to continue on deeper into the file... (2 Replies)
Discussion started by: cheetobandito
2 Replies

10. Shell Programming and Scripting

Help shell script to loop through files update ctl file to be sql loaded

I am currently trying to find a way to loop through files in a given directory and for each file modify a ctl file and sql load it. I have been using the sed command to change the infile, badfile parameters of the control file. I have not yet tried to sql load it. Requirement: files are ftp to... (1 Reply)
Discussion started by: dba_nh
1 Replies
Login or Register to Ask a Question