Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parse csv files by their names

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse csv files by their names
# 1  
Old 10-08-2013
Parse csv files by their names
HI all

I have multiple csv files with the names

VAR1_VAR2_VAR3_VAR4.csv

All the files have the same structure inside just values change.
I am trying to retrieve data from those files by fixing at each time one or more VAR.
I tried to write a script but I have 2 problems:

2- When I fix one variable (i.e, *_VAR2_VAR3_VAR4.csv ) on when VAR1 have 3 entries how to iterate this variable inside a loop ?

My script is here (My first tentative)
Code:
 #!/bin/bash          
      #OLDFIS=$IFS
      #IFS=","
      awk -F "," '
      #-------------------------- Table 1  ----------------------- -------#
 # Header begin    

 BEGIN {

      print"=========================================================="

      printf"%-8s %-8s %-8s %-8s %-8s\n","Network size", "P_Energy", "E_Energy",      
     "All_energy","Latency"

      print"=========================================================="
      }

 # Header end  


     NR==3 {printf "%-20s", $1 } NR==25 {printf "%-20s", $2 }   NR==12 {printf "%-20s", $2 }   
   NR==29 {printf "%-20s", $2 } NR==49 {printf "%-20s", $4 }' /...PATH/ *.csv >test.txt

      #-----------------------------------------------------------------------------#

Output:

Code:
======================================================================== 
    Network size       P_Energy           E_Energy           All_energy         Latency             
    =========================================================================
                    0.0402597           0.00767312          0.0479328           0.294311

With this code I have two problems:
1- Even I specify a large number of files I get only one row, should I add a loop?
2-I want to parse a set of files by fixing one variable (VAR1), and use its different value as the first column of my output.

Any ideas please?
# 2  
Old 10-10-2013
To 1.
But probably wanted printf only for the formatted output and forgot to add a \n and the end of your outputline.
And because printf is used in the shell, you kept using it rather than print from awk.

To 2.:
Maybe something like:
Code:
for var in VAR1 varA varB varC
do
	./script-parse-csv.sh $var_$VAR2_$VAR3_$VAR4.csv
done

Hope this helps
Last edited by sea; 10-10-2013 at 10:27 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need a script to parse data and output to csv

I am not too savvy with arrays and am assuming that what I am looking for needs arrays. This is my requirement. So I have the raw data that gets updated to a log as shown below StudentInfo: FullInfo = { Address = Newark Age = 20 Name= John } StudentInfo:... (2 Replies)
Discussion started by: sidnow
2 Replies

2. Shell Programming and Scripting

I have strigns below, in which we have files names. Can we parse them in one algorithm? If not, the

I have several strings with files names roundrobin_report_report_ALL_dbagadm_${Today}-${TM}_U.csv I want to say That I found a way to have prefix roundrobin_report_report_ALL_dbagadmU. letter=`echo $var | cut -d'.' -f 1 | rev | cut -d'_' -f 1 | rev` echo "NAME = $letter" prefix=`echo $var... (1 Reply)
Discussion started by: digioleg54
1 Replies

3. Shell Programming and Scripting

parse file names

trying to parse out all parts of this file name. REC=`echo "CAMXI.F0150.txt" | sed 's/.*\(*\).*/\1/'` export "FLRECL=$REC" FLECL=0150 I can get the numbers 0150 out of the file name. But need to capture first 5 bytes and extension. So i would export 3 variables (name, length, extension)... (5 Replies)
Discussion started by: rapalanlord
5 Replies

4. Shell Programming and Scripting

How to parse csv format?

Hi, I have a file with 3 fields in csv format: /tmp/foo/,MODIFY,bar/toto "/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"I would like to split fields in order to obtain the following: Line1: /tmp/foo/ MODIFY bar/totoLine2: /tmp/foo, bar/ ATTRIB,ISDIR toto, tata/foobarCan't find my way... (11 Replies)
Discussion started by: chebarbudo
11 Replies

5. Shell Programming and Scripting

Parse csv file

Hi, Our requirement is to parse the input file(.csv format). The each column in the file is delimited with comma. We need to take each column and apply some business validation rule. If data itself contains comma, then those fields are enclosed with double quotes ("). We can see this double... (7 Replies)
Discussion started by: vfrg
7 Replies

6. Shell Programming and Scripting

substrings from all files incl subdirs into csv with dir names

Greetings! I have multiple files, one per subdirectory, all with the same file name. All subdirectories are one level deep from the main directory. The data in the files is tab delimited between fields and record delimited with a newline. The subdirectory names have the date in the... (5 Replies)
Discussion started by: vtischuk@yahoo.
5 Replies

7. Shell Programming and Scripting

Unix Script to parse a CSV

I am writing a unix script that will parse a CSV and edit the values. My CSV looks like this 0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0 10,11,7,0,4,12,2,3,7,0,11,3,12,4,0,5,5,4,5,0,8,6,12,0,9,3,3,0,2,7,8... (16 Replies)
Discussion started by: RJ17
16 Replies

8. Shell Programming and Scripting

CSV File parse help in Perl

Folks, I have a bit of an issue trying to obtain some data from a csv file using PERL. I can sort the file and remove any duplicates leaving only 4 or 5 rows containing data. My problem is that the data contained in the original file contains a lot more columns and when I try ro run this script... (13 Replies)
Discussion started by: lodey
13 Replies

9. Shell Programming and Scripting

Re-usable function to parse csv files with different number of fields

Hi there, been pondering how to deal with this and hoping someone would give me an insight on this. I need help on creating a reusable bash funtion to parse csv files containing different number of fields (comma-seperated). My initial thought is to create function for each input csv file (20+... (2 Replies)
Discussion started by: jy2k7ca
2 Replies

10. UNIX for Advanced & Expert Users

How to Parse a CSV file into a Different Format

Hi I have a CSV file with me in this format Currency, USD, EUR, USD, 1.00, 1.32, EUR, 0.66, 1.00, How do I transpose the file to get to the format below. currency, currency, rate USD, USD, 1.00 USD, EUR, 1.32 EUR, USD, 0.66 EUR, EUR, 1.00 Thanks for your help We are using... (2 Replies)
Discussion started by: cdesiks
2 Replies
Login or Register to Ask a Question