Split list of files into an array and pass to function Post: 302931422

Sponsored Content

Top Forums Shell Programming and Scripting Split list of files into an array and pass to function Post 302931422 by Don Cragun on Tuesday 13th of January 2015 05:08:57 AM

01-13-2015

Registered User

It may have the list split up, but the 4 lists are no where close to containing the same number of elements. The construct grabbing a subset of the array elements in ksh93 and recent versions of bash is not:

Code:

${arrary[@]:start_index:end_index}

it is:

Code:

${arrary[@]:start_index:number_of_elements}

If we had a list of 10 files named 1 through 10 the four lists created by your code would be:

Code:

2:1 2
5:4 5 6 7 8
4:7 8 9 10
1:10

where the number before the colon is the number of files in the list and the numbers after the colon are the files in that list. (Note that files 7, 8, and 10 are in two lists and file 3 isn't in any list, and the number of files in the lists are 2, 5, 4, and 1.)

To get more even lists (and get each file in exactly one of your four lists), you could try something more like:

Code:

# collect list of file names
STATS_INPUT_FILENAMES=($(ls  './'$SET'/'$FOLD'/'*'in.txt'))
STATS_INPUT_FILENAMES=(1 2 3 4 5 6 7 8 9 10) # For testing only.

# get number of files
NUM_INPUT_FILES=${#STATS_INPUT_FILENAMES[@]}

# create 4 arrays, each with ~25% of filenames
NUM_FILE_LISTS='4'
# get size of each subset
BASE_LIST_SIZE=$(((NUM_INPUT_FILES) / NUM_FILE_LISTS))
LEFTOVER=$((NUM_INPUT_FILES % NUM_FILE_LISTS))
LIST_SIZE0=$((BASE_LIST_SIZE + (LEFTOVER > 0)))
LIST_SIZE1=$((BASE_LIST_SIZE + (LEFTOVER > 1)))
LIST_SIZE2=$((BASE_LIST_SIZE + (LEFTOVER > 2)))

FILE_LIST_0=("${STATS_INPUT_FILENAMES[@]:0:$LIST_SIZE0}")
FILE_LIST_1=("${STATS_INPUT_FILENAMES[@]:$LIST_SIZE0:$LIST_SIZE1}")
FILE_LIST_2=("${STATS_INPUT_FILENAMES[@]:$((LIST_SIZE0 + LIST_SIZE1)):$LIST_SIZE2}")
FILE_LIST_3=("${STATS_INPUT_FILENAMES[@]:$((LIST_SIZE0 + LIST_SIZE1 + LIST_SIZE2))}")

echo ${#FILE_LIST_0[@]}:${FILE_LIST_0[@]}
echo ${#FILE_LIST_1[@]}:${FILE_LIST_1[@]}
echo ${#FILE_LIST_2[@]}:${FILE_LIST_2[@]}
echo ${#FILE_LIST_3[@]}:${FILE_LIST_3[@]}

Which with the same list of 10 files produces the output:

Code:

Passing arrays to a function is tricky. The easier approach is to pass any fixed arguments as the 1st arguments to your functions and pass the filenames as a variable argument list with "${FILE_LIST_x[@]}".

Hope this helps...

This User Gave Thanks to Don Cragun For This Post:

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can we pass array with call by value in function

I want to pass an array in my function, And my function will be changing the elements of the array in the fuction, but it should not affect the values in my array variable of main function

2. Shell Programming and Scripting

Array split function & hashes

Hi, If this is the array that is being returned to me: How would I get the values for each of the 3 records? This works for 1 Record: foreach $item (@results) { ($id, $id2, $name, $date, $email) = split(/\|/, $item, 5); print "$name<br>"; }

3. Shell Programming and Scripting

How to pass an array from SHELL to C function

Hi, I have an output generated from a shell script like; 0x41,0xF2,0x59,0xDD,0x86,0xD3,0xEF,0x61,0xF2 How can I pass this value to the C function, as below; int main(int argc, char *argv) { unsigned char hellopdu={above value}; } Regards Elthox

4. Shell Programming and Scripting

Find and split the list of files with suffiz of seg**

Hi,. I am writing a script to get the new files and split them. Requirement Find the new files under the path "/wload/scmp/app/data/OAS" (There are 5 sub folders). Gunzip the files which are having .gz suffix. Put the list of files in the filename in the format...

5. Shell Programming and Scripting

Split the file and access that files through array and loop

Hi All, the below is my requirement.. i need to split the file based on line and put that files in a array and need to access that files through loop finally i should send the files through mail.. how can we achieve this ..I am new to shell script please guide me.. I am using KSH.. ...

6. Shell Programming and Scripting

How to pass an array to a function in shell script.?

hi, I have a array say SAP_ARRAY="s1.txt" SAP_ARRAY="s2.txt" how can i pass this full array to a function. here is the sample code i am using.. CHECK_NO_FILES() { FARRAY=$1 echo "FARRAY = $FARRAY" echo "FARRAY = $FARRAY" ...............

7. Shell Programming and Scripting

Question about sorting -- how to pass an array to a function

Hi, guys I just wanted to sort the elements of an array ascendingly. I know the following code does work well: array=(13 435 8 23 100) for i in {0..4} do j=$((i+1)) while ] do if } -le ${array} ]] then : else min=${array} ${array}=${array} ${array}=$min fi...

8. Shell Programming and Scripting

Pass array to a function and display the array

Hi All I have multiple arrays like below. set -A val1 1 2 4 5 set -A val2 a b c d . . . Now i would like to pass the individual arrays one by one to a function and display/ do some action. Note : I am using ksh Can you please advise any solution... Thanks in advance.

9. Shell Programming and Scripting

Pass an array to awk to sequentially look for a list of items in a file

Hello, I need to collect some statistical results from a series of files that are being generated by other software. The files are tab delimited. There are 4 different sets of statistics in each file where there is a line indicating what the statistic set is, followed by 5 lines of values. It...

10. Shell Programming and Scripting

How to pass and read an array in ksh shell script function.?

I'm able to read & print an array in varaible called "filelist" I need to pass this array variable to a function called verify() and then read and loop through the passed array inside the function. Unfortunately it does not print the entire array from inside the funstion's loop. #/bin/ksh...

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can we pass array with call by value in function

Discussion started by: ranjithpr

2. Shell Programming and Scripting

Array split function & hashes

Discussion started by: novera

3. Shell Programming and Scripting

How to pass an array from SHELL to C function

Discussion started by: elthox

4. Shell Programming and Scripting

Find and split the list of files with suffiz of seg**

Discussion started by: Satish Shettar

5. Shell Programming and Scripting

Split the file and access that files through array and loop

Discussion started by: kalidoss

6. Shell Programming and Scripting

How to pass an array to a function in shell script.?

Discussion started by: Little

7. Shell Programming and Scripting

Question about sorting -- how to pass an array to a function

Discussion started by: franksunnn

8. Shell Programming and Scripting

Pass array to a function and display the array

Discussion started by: Girish19

9. Shell Programming and Scripting

Pass an array to awk to sequentially look for a list of items in a file

Discussion started by: LMHmedchem

10. Shell Programming and Scripting

How to pass and read an array in ksh shell script function.?

Discussion started by: mohtashims