Shell Programming and Scripting

I have been trying to write a simple snip of bash shell code to import from 1 to 100 records into a BASH array.

I have a CSV file that is structured like:

Code:

record1,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record2,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record3,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record4,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record5 ......
record6 ......
...... and so on

I am trying to get this data into corresponding arrays as such:

Code:

$record1[item1-n]
$record2[item1-n]
$record3[item1-n]
$record4[item1-n]

Here is what i have so far:

++++++++++++++++++++++++++++++

Code:

#!/bin/bash
testfile="./test-dev/test-global-data-import.csv"
var1=`cat $testfile`
IFS=',' record1=( ${var1} )

echo ""
echo "Here is the data without the comma separators."
echo ""
echo ${record1[*]:0}
echo ""
echo "Here is the data with comma separators."
echo ""
echo "${record1[*]:0}"
echo ""

#EOF

++++++++++++++++++++++++++++++

Here are the problems I am having:

1. I am trying to have each record (line in the CSV file) become its own array, such as $record1[*]

2. There will be lots of strange formatting within the CSV file (i.e. IP addresses, street addresses w/ commas and periods) and right now each record has items 1-35 and it could possibly grow to <=100 fields.

3. I am a newbie :-( .... I have edited BASH scripts in the past ... but that no way make me proficient .... so can someone please help.

P.S. I am pouring over several resources on a daily basis (Beginners & Advanced BASH guides on tldp.org and I have a BASH book that I have my nose buried in constantly. I still need some expert assistance though.

Any help is greatly appreciated.

Could you try to explain what exactly do you want to achieve?
I won't recommend a pure shell for text manipulation.

Okay, so here is the 10k fly by ....

I have one group of persons inputting data about sites into a spreadsheet and saving it as someinputdata.csv. Then, I have several template files with VAR_1, VAR_2, VAR_3, and so on declared. I am trying to assign the CSV data to array elements so I can pluck out the data I need with a sed statement like this ...

I really thought it was a simple problem to solve. One group of people is collecting data and one group will be mass producing network devices with the cfg files.

+SNIP+

# set the template variable
TEMPLATE="some text file with VAR_1 in it"

# code here to insert the CSV data into the array ....
..... MISSING ......

# set the variable to the array element
VAR_1=$record1[12]

#
cat $TEMPLATE | sed -e "s/VAR_1/$VAR_1/"g | tee ./output/$VAR_1.cfg >/dev/null 2>&1

+SNIP+

Please post a sample of your input data and an example of the expected modified output after the substitution.

I'm not saying that reading an input record into a bash array is difficult,
I'm just saying that bash is not the right tool for parsing csv data.

Above in this post is the example of the input CSV data.

to re-iterate ....

I have a CSV file that is structured like:

Code:

record1,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record2,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record3,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record4,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record5 ......
record6 ......
...... and so on

I am trying to get this data into corresponding arrays as such:

Code:

$record1[item1-n]
$record2[item1-n]
$record3[item1-n]
$record4[item1-n]

Yes,
I've already read your original post, I mean that we need a representative sample.
Do you have any values with embedded commas, for example?

By expected output I mean the final product, not the intermediate variables/arrays.