Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Turning CSV files into individual Variables

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Turning CSV files into individual Variables
# 1  
Old 03-25-2010
Turning CSV files into individual Variables
I want to be able to convert the following data from a CSV into individual variables from the columns 2 4 and 8
I can use awk to grab the columns using
Code:
var1=`cat text.csv | awk "," '{print $2}'`

but how do I create separate variables for each line.

Code:
595358 ,ECON1010 ,THU ,08:00 - 10:00 ,11 Mar 10 - 18 Mar 10 ,Weekly,85,Laboratory,Edit   
595410 ,ARTS1010 ,THU ,11:00 - 12:00 ,11 Mar 10 - 18 Mar 10 ,Weekly,85,Laboratory,Edit   
595449 ,GOVT1010 ,THU ,12:00 - 13:00 ,11 Mar 10 - 18 Mar 10 ,Weekly,85,Laboratory,Edit 


I want it to grab the following as individual variables.

Code:
595358 ,line1value1 ,THU ,line1value2 ,11 Mar 10 - 18 Mar 10 ,Weekly,85,line1value3,Edit   
595410 ,line2value1 ,THU ,line2value2 ,11 Mar 10 - 18 Mar 10 ,Weekly,85,line2value3,Edit   
595449 ,line3value1 ,THU ,line3value2 ,11 Mar 10 - 18 Mar 10 ,Weekly,85,line3value3,Edit 

The number of lines can change from day to day.

Any advice would be much appreciated .

Thanks in advance.
# 2  
Old 03-25-2010
Code:
sed 's/\(.[^,]* *,\)\(.*\)/\2/;s/,.[^,]*,/|/;s|,.*,\(.[^,]*\),.*|\|\1|' file

# 3  
Old 03-25-2010
Sorry I don't think i explained it well enough I want to be able to break it down so I can have each variable

So i end up with separate variables.

line1value1="econ1010"
line2value2="arts1010"
line3value3="govt1010"

If i do this
Code:
cat file.csv |awk -F "," '{print $2 $4 $8}'`

I get all the columns but but i want to be able to break it down further so that the value on each line becomes an individual variable.


# 4  
Old 03-25-2010
Should each variable be an array of values in each line?
# 5  
Old 03-25-2010
i guess that could work
# 6  
Old 03-25-2010
The try this
Code:
#!/bin/bash
n=0
while read L
do
    VAR1[$n]=$(echo "$L" | cut -d',' -f2)
    VAR2[$n]=$(echo "$L" | cut -d',' -f4)
    VAR3[$n]=$(echo "$L" | cut -d',' -f8)
    ((n++))
done < text.csv
for ((i=0; i<n; i++))
do    echo -e "$i\tValue1=${VAR1[$i]}\tValue2=${VAR2[$i]}\tValue3=${VAR3[$i]}"
done

# 7  
Old 03-25-2010
Thank you Frans that seems to do the trick.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Generate files and use csv data to replace multiple variables in a template

I have a source csv file consists of first field as variable name, and the rest are site-specific information (converted from excel file, where site -specific values in columns). I am trying to create a file for every site using a template and replace the multiple variables with values from the... (3 Replies)
Discussion started by: apalex
3 Replies

2. Programming

Python Reading Individual files and Regex through them

As a newbie to Python, I am trying to write a script in which is will add all the log files (*.log) from within a directory to a list, open the files and search for an ip using a regex and single it out (appending the ip's to the list). So far, I have: import re, os def list_files() content = ... (4 Replies)
Discussion started by: metallica1973
4 Replies

3. Shell Programming and Scripting

turning output of two lines into one CSV line

Hi, I am attempting to use sed on linux to do something trivial. I am also too embarassed to show you what I have tried so far! What I am trying to do should be trivial, if I knew what I was doing, but I don't. Would someone please help me? Here is my problemI have a ASCII file that has the... (4 Replies)
Discussion started by: Jon8987z
4 Replies

4. Solaris

Auditing Individual Files

I have some Solaris 9 systems and I'm interested in using the "fm" audit class to track changes to sensitive files but it's too verbose for it to be auditing to that level for EVERY file, so I was wondering if there were a way of restricting the audit of those events to particular files. I... (0 Replies)
Discussion started by: thmnetwork
0 Replies

5. Shell Programming and Scripting

Unpack individual files from tarball

Say you don't want to unpack the whole thing, just individual files or directories within a .tgz. How to do this? (1 Reply)
Discussion started by: stevensw
1 Replies

6. UNIX for Dummies Questions & Answers

Parse or cut concat variables to individual values

Hello I need to pass some environment parameters to a datastage job and am getting an error when trying to send the complete concatinated variable. I have decided to parse out just the values and send as parameters but am struggling to find the best way to do this (actually I am not very... (3 Replies)
Discussion started by: LynnC
3 Replies

7. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

8. Shell Programming and Scripting

list file content as individual variables

Hello, I've created a couple of files within a list using the command "ls -ltr | tail -2 > list" These files are the newest files placed within a directory. From the "list" file, I need to place the filenames as a variable. In which the newest file will be called "new_ctrl" the older file... (4 Replies)
Discussion started by: petersf
4 Replies

9. Shell Programming and Scripting

Turning files into an array

I have several files like this file A Good Bad Fair Strawberry 1 4 5 Banana 23 12 4 Plantain 0 0 1 Orange 0 0 0 file B Strawberry 1 1 3 Banana 2 ... (10 Replies)
Discussion started by: littleb
10 Replies

10. UNIX for Dummies Questions & Answers

Create individual tgz files from a set of files

Hello I have a ton of files in a directory of the format app.log.2008-04-04 I'd like to run a command that would archive each of these files as app.log.2008-04-04.tgz I tried a few combinations of find with xargs etc but no luck. Thanks Amit (4 Replies)
Discussion started by: amitg
4 Replies
Login or Register to Ask a Question