How to read max of 10 file at a time?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to read max of 10 file at a time?
# 1  
Old 06-22-2007
How to read max of 10 file at a time?

Hi All,

Please advise . Welcome more suggestions.

For examples, I have 1000 file with prefix x??? In fact, I want to convert them to x???.txt with max 10 files at a time. As such, I will need to call another script to read from those 10 *txt files and sleep 5000 to convert the next 10 again.

xaaa
;
;
xbbz

Hope to hear from.

Regards,
# 2  
Old 06-22-2007
You can do something like that:

Code:
count=0
limit=10
ls x??? | \
while read file
do
   mv $file $file.txt
   count=`expr $count+1`
   if [ $count -ge $limit ]
   then
      count=0
      /path/to/another_script &
      sleep 5000  # Wait more than an hour (5000 secs)
   fi
done

# 3  
Old 06-22-2007
Aigles, about your solution:
1) It does not work if the number of files is not multiple of 10.
2) The user is asking to work with 10 files at a time.
You are working with one at a time.

Here is one possible solution to rename the files:
Code:
for mFile in x???
do
  mv $mFile $mFile".txt"
done

Here is the solution to work with groups of 10 files at a time:
Code:
set -- x???.txt
printf "%s %s %s %s %s %s %s %s %s %s\n" $@ | \
while read m10Files
do
  echo "m10Files = "${m10Files}
done

# 4  
Old 06-22-2007
Quote:
Originally Posted by Shell_Life
Aigles, about your solution:
1) It does not work if the number of files is not multiple of 10.
2) The user is asking to work with 10 files at a time.
You are working with one at a time.
1) Exact. The following new version fix it.
2) The script is called every ten files. I add a variable containing the file names (can be used as a parameter for the script)

Code:
count=0
file_list=
limit=10
ls x??? | \
while read file
do
   mv $file $file.txt
   count=`expr $count+1`
   file_list="$file_list $file.txt"
   if [ $count -ge $limit ]
   then
      /path/to/another_script $file_list &
      sleep 5000  # Wait more than an hour (5000 secs)
      count=0
      file_list=
   fi
done
[ $count -gt 0 ] && /path/to/another_script $file_list &

I"m not sure that running the script in background is required. If not remove the &
# 5  
Old 06-22-2007
Hi aigles and Shell_Life,

Thanks for yours input. I will give a try.
Cheers
# 6  
Old 06-23-2007
Code:
count=0
limit=10
mydate=`date '+%Y%m%d%H%M%S'`

cd /var/tmp/VAS

for mFile in x????
do
     mv $mFile $mFile".txt"
     count=`expr $count+1`
     if [ $count -le $limit ]
     then
          sh up.sh 2>LOG/update.sh.$mydate.log &
          sleep 5000
          mv *out LOG/
          count=0
  fi
done

The counter doesn't increase at at and file are mv at all and. It won't stop at limit 10th...

Something wrong... Please advise
# 7  
Old 06-23-2007
Your if test in incorrect, replace -le by -ge :
Code:
     if [ $count -ge $limit ]

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find min and max time taken from a log file

You have a log file as attached in sample input with various operations and time taken by each of them. Write a script to find the min and max time taken for each operation. Sample output is attached. Sample Input is given as below: operation1,83621 operation2,72321 operation3,13288... (1 Reply)
Discussion started by: Chandan_Bose
1 Replies

2. Shell Programming and Scripting

Finding max of a column grouping by the time

Hi, I have the below text: 16:00 0.50 16:00 0.30 16:00 0.00 16:00 0.00 16:00 0.30 16:01 0.00 16:01 0.30 I want to find the max of the 2nd column grouping by the values in the 1st column using awk. So 16:00 0.50 16:01 0.30 I have tried (3 Replies)
Discussion started by: satishrao
3 Replies

3. Shell Programming and Scripting

Read multiple lines at a time from file

Hello All, I have a file like below.... dn: cn=user1,ou=org,o=org cn=user1 uid=user1 cn=user2,ou=org,o=org cn=user2 uid=user2 cn=user3,ou=org,o=org cn=user3 cn=user33 uid=user3 cn=user4,ou=org,o=org cn=user4 uid=user4 (6 Replies)
Discussion started by: s_linux
6 Replies

4. Shell Programming and Scripting

Read two lines at time from a file

Hello community, what I need to do is read 2 rows at time from a file. I have this simple solution: File to read: LINE1 LINE2 LINE3 LINE4 LINE5 LINE6 LINE7 LINE8Read routine:#!/bin/ksh sed '1,3d' /out.txt | while read line; do read line2 echo $line $line2 doneResult:LINE1... (5 Replies)
Discussion started by: Lord Spectre
5 Replies

5. Shell Programming and Scripting

File read time

hi, how to check the file latest read time.. (2 Replies)
Discussion started by: rsivasan
2 Replies

6. Shell Programming and Scripting

Need shell script to read two file at same time and print out in single file

Need shell script to read two file at same time and print output in single file Example I have two files 1) file1.txt 2) file2.txt File1.txt contains Aaa Bbb Ccc Ddd Eee Fff File2.txt contains Zzz Yyy Xxx (10 Replies)
Discussion started by: sreedhargouda
10 Replies

7. Shell Programming and Scripting

How to read/process a .gz file, one line at a time?

Hello I'm stuck trying to solve this KSH issue and I'm hoping someone out there can offer some suggestions. I want to read lots of large .gz files one line at a time in order to compare its Error entries with a list of known errors. I can't simply do "foreach ERROR do gzcat *.gz |grep... (2 Replies)
Discussion started by: dayscripter
2 Replies

8. UNIX for Dummies Questions & Answers

Read file and remove max length

Hi all, I tried to write a shell to read huge file and eliminate max length record which is wrong generated record. But I get an error remove_sp.sh: line 27: syntax error near unexpected token `else' remove_sp.sh: line 27: ` else $LINE >> REJFILE' My shell is here: #!/bin/sh... (5 Replies)
Discussion started by: mr_bold
5 Replies

9. Shell Programming and Scripting

why shell scripting takes more time to read a file

i have done a coding in shell scripting which reads a file line by line and does something....it takes more than 30 seconds to execute for a single file. when i do the same with perl scripting it takes less than a second. is that shell scripting is not efficient while working with large number of... (1 Reply)
Discussion started by: brkavi_in
1 Replies

10. UNIX for Advanced & Expert Users

Response time & IO max

in HP-UX how i can measure the response time and how can i find the maximum IO (1 Reply)
Discussion started by: salhoub
1 Replies
Login or Register to Ask a Question