Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Initializing array using awk from parsed csv

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Initializing array using awk from parsed csv
# 1  
Old 12-05-2010
Initializing array using awk from parsed csv
Hi all,

I am fairly new to unix scripting and I am struggling with some awk commands here. I want to create an array COMPANYCODE from some parsed csv values (single column) . I have been able to access the data in other ways but I would prefer if I could put each value in it's own array element. Here is what I have so far:

Code:
COMPANYCODE=( `awk -F ',' '{if ($1 == "") {print "-"}; print $1}' $TEMP_FILE | sed 's/<//' | sed 's/1//' | sed 's/COMPANYCODE//' | sed '/\\/,/file/d' | sed 's/1689c1//'` )

If I try and print out something like
Code:
echo ${COMPANYCODE[2]}

I get nothing. I know if I do something like

Code:
COMPANYCODE=$( awk -F ',' '{if ($1 == "") {print "-"}; print $1}' $TEMP_FILE | sed 's/<//' | sed 's/1//' | sed 's/COMPANYCODE//' | sed '/\\/,/file/d' | sed 's/1689c1//' )

and do
Code:
echo $COMPANYCODE

It will print all the values but they are not individual elements.

I have my IFS set to "," because some fields have spaces in them.

Any thoughts? Thanks in advance!
# 2  
Old 12-05-2010
The comma IFS is probably causing you some issues. Best bet is to aim at getting quotes around each company code and spaces between (are quotes allowed in your company codes?).
Then use eval:

Code:
$ eval COMPANYCODE=( $( echo '"ACME COMPUTERS" "MICROSOFT" "REST"'))
 
$ echo ${COMPANYCODE[2]}
REST
 
$ echo ${COMPANYCODE[0]}
ACME COMPUTERS

If you can supply a example of your TEMP_FILE I'm sure we could help simplify the code you use to generate the COMPANYCODE list.
# 3  
Old 12-05-2010
My temp file is simply

Code:
FIELD1, FIELD 2, FIELD 3, FIELD 4 etc
val1,     val2,      val3,     val4,  
test,1    test2,     test3,    test4,
x1,        x1,        x1,       x1

Ultimately I would like MYARRAY to contain val1, test1, x1 etc (first column)

I've tried using read while loops but they don't seem to help at all. As my code snippet shows, I am trying to use awk to put a single column into array elements.

awk section:

Code:
(awk -F ',' '{if ($1 == "") {print "-"}; print $1}' $TEMP_FILE

The csv values are separated by commas and if the field is blank I add a "-" character. I have print $1 for COMPANYCODE because it's the first column in the csv.

The reason I changed the IFS was because if I left it to the default value it would split fields with two spaces into separates lines of $COMPANYCODE
# 4  
Old 12-05-2010
OK this should work OK:

Code:
$ cat TEMP_FILE
FIELD1, FIELD 2, FIELD 3, FIELD 4 etc
val1,     val2,      val3,     val4,  
test,1    test2,     test3,    test4,
x1,        x1,        x1,       x1
,        blank,      company,  here
space ship,  has,      a,      space
 
$  eval COMPANYCODE=($(awk -F, 'NR>1{ print $1?"\""$1"\"":"-"}' TEMP_FILE ))
 
$  echo ${COMPANYCODE[1]} 
test
 
$ echo ${COMPANYCODE[3]}
-
 
$  echo ${COMPANYCODE[4]}
space ship

# 5  
Old 12-05-2010
Thanks for the help, would you mind explaining this section?

Code:
'NR>1{ print $1?"\""$1"\"":"-"}'

# 6  
Old 12-05-2010
NR>1 ==> If Row number is greater than 1 (skips heading line)

$1 ? A : B ==> if $1 is non null then A else B - A and B are expressions below

"\"' ==> String with a single double quote character (the backslash escapes the quote)

"\""$1"\"" ==> Field 1 with doulbe quotes around it (awk appends adjecent strings automatically)

"-" ==> String with a single hyphen.


So putting it all together

Ignore first line, otherwise
if field 1 is non null then print field 1 with quotes around it, otherwise
print a hyphen.
Last edited by Chubler_XL; 12-05-2010 at 10:08 PM..
This User Gave Thanks to Chubler_XL For This Post:
P8TRO
# 7  
Old 12-05-2010
Thank you so much! This works exactly right! I have been banging my head against the wall all day Smilie

I really appreciate it! Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to parse section of csv into array

In the awk below I am trying to parse the Sample Name below the section. The values that are extracted are read into array s(each value in a row seperated by a space) which will be used later in a bash script. The awk does execute but no values are printed. I am also not sure how to print in a row... (1 Reply)
Discussion started by: cmccabe
1 Replies

2. Shell Programming and Scripting

How to Assign an shell array to awk array?

Hello All, Can you please help me with the below. #!/bin/bash ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5 EXTRACT_DT:30-SEP-12 VER_NUM:1" ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5... (14 Replies)
Discussion started by: Ariean
14 Replies

3. Shell Programming and Scripting

Place the contents of a .CSV file to an array

Hi, I am trying to place the contents of a .CSV file to an array, but not sure how to do that. Here is my .CSV file content: App,SLA,Job name,Avg start time,Avg run time,Frequency,Downstream apps XYZ,,ABC14345,3:00 AM,00.04.00,Daily,STAMP XYZ,9:00,ABC12345,3:15 AM,00.05.00,Daily,STAMP ... (4 Replies)
Discussion started by: ajayakunuri
4 Replies

4. Shell Programming and Scripting

awk read column csv and search in other csv

hi, someone to know how can i read a specific column of csv file and search the value in other csv columns if exist the value in the second csv copy entire row with all field in a new csv file. i suppose that its possible using awk but i m not expertise thanks in advance (8 Replies)
Discussion started by: giankan
8 Replies

5. Shell Programming and Scripting

Shell script to populate an array from a csv file

Hi Guys, I've got a script that runs and collects statistics from one of our machines and outputs the data into a csv file. Now, that script runs fine and I managed to create another one (with a lot of help from this forum!!) to trim the csv file down to all the data that I need, rather than all... (9 Replies)
Discussion started by: jimbob01
9 Replies

6. Shell Programming and Scripting

CSV to SQL insert: Awk for strings with multiple lines in csv

Hi Fellows, I have been struggling to fix an issue in csv records to compose sql statements and have been really losing sleep over it. Here is the problem: I have csv files in the following pipe-delimited format: Column1|Column2|Column3|Column4|NEWLINE Address Type|some descriptive... (4 Replies)
Discussion started by: khayal
4 Replies

7. Programming

How to get the number of bytes parsed in libxml2

Hi, I am using the libxml2 sax parser to parse a in memory xml string along with validating it against a schema. I am using the following code: xmlSAXHandlerPtr sax_ = new xmlSAXHandler(); sax_->initialized = XML_SAX2_MAGIC; sax_->startElementNs =... (0 Replies)
Discussion started by: Sam Krishna
0 Replies

8. Shell Programming and Scripting

Shell snip to import CSV data into BASH array

I have been trying to write a simple snip of bash shell code to import from 1 to 100 records into a BASH array. I have a CSV file that is structured like: record1,item1,item2,item3,item4,etc.,etc. .... (<= 100 items) record2,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)... (5 Replies)
Discussion started by: dstrout
5 Replies

9. Shell Programming and Scripting

Array not initializing

Hi, I have to use array in shell script to perform a logic. When I use below statements inside a script and execute, it gives me an error: $ cat test.sh set -A arr 10 20 30 echo ${arr} $ sh test.sh a: -A: bad option(s) But at the same time I can run above two statements without error... (3 Replies)
Discussion started by: nandanjain
3 Replies

10. Shell Programming and Scripting

Read csv into Hash array?

Hi all experts, May I know how to read a csv file and read the content in a hash in PERL? Currently, I hard-coded and defined it in my code. I wanna know how to make up the %mymap hash thru reading the cfg.txt ==== csv file(cfg.txt): 888,444 999,333 === #!/usr/bin/perl my... (1 Reply)
Discussion started by: kinmak
1 Replies
Login or Register to Ask a Question