Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Counter based on 10 files or less

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Counter based on 10 files or less
# 1  
Old 04-27-2019
Counter based on 10 files or less
there are around 676 files in a directory and I want to batch every 10 and then remaining 7 files as well for a batch pressing job simultaneously.

I'm trying to pick-up every 10 counter'd files -- batch these files and then move on with the next batch.. after resetting the Counters back to 0;

but what if I am left with remaining [1-9] files like for now I have 7 files which are not falling under ==10 rule .

Code:
FILECOUNTER=0;
for filex  in `ls`; 
do
let "FILECOUNTER++" 
FILENAMES="$filex\t$FILENAMES"
if [[  $FILECOUNTER  -eq 10  ]]
then
$BATCH_THE_PROCESS  `echo -e $FILENAMES`
FILECOUNTER=0; 
FILENAMES="";
fi
done


This loop is running fine for files till 670, but I'm missing last 7 files which are not accounted for in the if-else

Can someone please suggest to get a listing for every 10 or less ( if there are less than 10 files remaining)

Regards,
NMR
# 2  
Old 04-27-2019
Had you told us what OS and shell you are using, some smart solutions might have been presented by now.
# 3  
Old 04-27-2019
Code:
root# uname -a
Linux hippo 3.16.0-8-amd64 #1 SMP Debian 3.16.64-2 (2019-04-01) x86_64 GNU/Linux


Code:
root# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"


Code:
root# hostnamectl
   Static hostname: hippo
         Icon name: computer-server
           Chassis: server
        Machine ID: ---------------------------------------
           Boot ID: ---------------------------------------
  Operating System: Debian GNU/Linux 8 (jessie)
            Kernel: Linux 3.16.0-8-amd64
      Architecture: x86-64

Code:
root# echo $BASH_VERSION
4.3.30(1)-release

# 4  
Old 04-27-2019
With your recent bash, you have "Arithmetic Expansion" at your finger tips. You don't need the ´ls´ "command substitution". Try
Code:
for FN in *; do FL="$FL $FN"; if ! ((++CNT%10)); then echo "executing " $FL; FL=""; fi; done; [ "$FL" ] && echo "executing " $FL
executing  file1 file2 file3 file4 file5 file6 file7 file8 file9 file10
executing  file11 file12 file13 file14 file15 file16 file17 file18 file19 file20
executing  file21 file22 file23 file24 file25 file26 file27

. (Check for residual files added after Jim McNamara's post) Replace the echo with your actual command. You might also want to try e.g.
Code:
ls | paste -sd"         \n" | while read LN; do echo "executing " $LN; done

Last edited by RudiC; 04-27-2019 at 09:41 AM.. Reason: Check for residual files added after Jim McNamara's post
This User Gave Thanks to RudiC For This Post:
busyboy
# 5  
Old 04-27-2019
Code:
parms=""
cnt=0
cd /path/to/files
ls | while read fname
do
     parms="$parms $fname"
     cnt=$(($cnt +1))
     if [ $cnt -eq 10 ] ; then
         batch /path/to/myjobname parms
         cnt=0
         parms=""
     fi
done
# one answer to your question - run any amount of leftovers:
[ $cnt -gt 0 ]  &&   batch /path/to/myjobname $parms

Edit: oops Rudi beat me to it....
# 6  
Old 04-27-2019
Using shell arrays:
Code:
FL=($(ls))
while (( CNT <= ${#FL[@]})); do echo "executing " ${FL[@]:CNT:10}; ((CNT+=10)); done

Last edited by RudiC; 04-27-2019 at 10:18 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Move files with a certain suffix based on how many files are in another folder

Hello, First time poster. I am looking for a way to script or program the process of moving files from one folder to another, automatically, based on the count of files in the destination folder. I was thinking a shell script would work, but am open to the suggestions of the experts... (6 Replies)
Discussion started by: comtech
6 Replies

2. Shell Programming and Scripting

How to take input from different files for using the counter variable?

Hi All, I have script which call test utility and it takes token input which is kept in a separate file. now instead of taking tokens from single file,will keep 5 input files and script should read tokens from each files. EX : There will 5 token.txt file which will have say around 1000... (1 Reply)
Discussion started by: Optimus81
1 Replies

3. Shell Programming and Scripting

Divide an EBCDIC files into multiple files based on value at 45-46 bytes

Hi All, I do have an EBCDIC file sent from the z/os , this file has records with different record types in it, the type of record is identified by bytes 45-46 like value 12 has employee record value 14 has salaray record and etc.... we do now want to split the big ebcdic file into multiple... (3 Replies)
Discussion started by: okkadu
3 Replies

4. Shell Programming and Scripting

grouping log files based on counter

I have my log file as below 00:18:02 - Nothing normal; Garbage Collection kicked off & running from last 3 min... 00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min... 00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min...... (11 Replies)
Discussion started by: manas_ranjan
11 Replies

5. Shell Programming and Scripting

Using bash to separate files files based on parts of a filename

Hey guys, Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have: (a_b_c.txt) WB34_2_SLA8.txt WB34_1_SLA8.txt WB34_1_DB10.txt WB34_2_DB10.txt WB34_1_SLA8.txt WB34_2_SLA8.txt 77_1_SLA8.txt... (1 Reply)
Discussion started by: Breentax
1 Replies

6. Shell Programming and Scripting

File merging based on different counter loop

hello, File 1 main Group sub group MIT VAR_1D_DATA_TYPE 23-03-2012 MIT VAR_1D_DATA_TYPE 22-03-2012 MIT VAR_10D_DATA_TYPE 23-03-2012 MIT VAR_10D_DATA_TYPE 22-03-2012 MIT ... (0 Replies)
Discussion started by: manas_ranjan
0 Replies

7. Shell Programming and Scripting

sort the files based on timestamp and execute sorted files in order

Hi I have a requirement like below I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory My files looks like this PGABOLTXML1D_201108121235.xml... (1 Reply)
Discussion started by: saidutta123
1 Replies

8. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

9. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

10. Shell Programming and Scripting

rename files Ax based on strings found in files Bx

Hi, I'm not very experienced in shell scripting and that's probably why I came across the following problem: I do have several hundred pairs of text files (PF00x.spl and PF00x.shd) where the first file (PF00x.spl) needs to be renamed according a string that is included in the second file... (12 Replies)
Discussion started by: inCH
12 Replies
Login or Register to Ask a Question