Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Parse csv files by their names
Top Forums Shell Programming and Scripting Parse csv files by their names Post 302861209 by Jhon.c on Tuesday 8th of October 2013 11:16:43 AM
Old 10-08-2013
Parse csv files by their names
HI all

I have multiple csv files with the names

VAR1_VAR2_VAR3_VAR4.csv

All the files have the same structure inside just values change.
I am trying to retrieve data from those files by fixing at each time one or more VAR.
I tried to write a script but I have 2 problems:

2- When I fix one variable (i.e, *_VAR2_VAR3_VAR4.csv ) on when VAR1 have 3 entries how to iterate this variable inside a loop ?

My script is here (My first tentative)
Code:
 #!/bin/bash          
      #OLDFIS=$IFS
      #IFS=","
      awk -F "," '
      #-------------------------- Table 1  ----------------------- -------#
 # Header begin    

 BEGIN {

      print"=========================================================="

      printf"%-8s %-8s %-8s %-8s %-8s\n","Network size", "P_Energy", "E_Energy",      
     "All_energy","Latency"

      print"=========================================================="
      }

 # Header end  


     NR==3 {printf "%-20s", $1 } NR==25 {printf "%-20s", $2 }   NR==12 {printf "%-20s", $2 }   
   NR==29 {printf "%-20s", $2 } NR==49 {printf "%-20s", $4 }' /...PATH/ *.csv >test.txt

      #-----------------------------------------------------------------------------#

Output:

Code:
======================================================================== 
    Network size       P_Energy           E_Energy           All_energy         Latency             
    =========================================================================
                    0.0402597           0.00767312          0.0479328           0.294311

With this code I have two problems:
1- Even I specify a large number of files I get only one row, should I add a loop?
2-I want to parse a set of files by fixing one variable (VAR1), and use its different value as the first column of my output.

Any ideas please?
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

How to Parse a CSV file into a Different Format

Hi I have a CSV file with me in this format Currency, USD, EUR, USD, 1.00, 1.32, EUR, 0.66, 1.00, How do I transpose the file to get to the format below. currency, currency, rate USD, USD, 1.00 USD, EUR, 1.32 EUR, USD, 0.66 EUR, EUR, 1.00 Thanks for your help We are using... (2 Replies)
Discussion started by: cdesiks
2 Replies

2. Shell Programming and Scripting

Re-usable function to parse csv files with different number of fields

Hi there, been pondering how to deal with this and hoping someone would give me an insight on this. I need help on creating a reusable bash funtion to parse csv files containing different number of fields (comma-seperated). My initial thought is to create function for each input csv file (20+... (2 Replies)
Discussion started by: jy2k7ca
2 Replies

3. Shell Programming and Scripting

CSV File parse help in Perl

Folks, I have a bit of an issue trying to obtain some data from a csv file using PERL. I can sort the file and remove any duplicates leaving only 4 or 5 rows containing data. My problem is that the data contained in the original file contains a lot more columns and when I try ro run this script... (13 Replies)
Discussion started by: lodey
13 Replies

4. Shell Programming and Scripting

Unix Script to parse a CSV

I am writing a unix script that will parse a CSV and edit the values. My CSV looks like this 0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0 10,11,7,0,4,12,2,3,7,0,11,3,12,4,0,5,5,4,5,0,8,6,12,0,9,3,3,0,2,7,8... (16 Replies)
Discussion started by: RJ17
16 Replies

5. Shell Programming and Scripting

substrings from all files incl subdirs into csv with dir names

Greetings! I have multiple files, one per subdirectory, all with the same file name. All subdirectories are one level deep from the main directory. The data in the files is tab delimited between fields and record delimited with a newline. The subdirectory names have the date in the... (5 Replies)
Discussion started by: vtischuk@yahoo.
5 Replies

6. Shell Programming and Scripting

Parse csv file

Hi, Our requirement is to parse the input file(.csv format). The each column in the file is delimited with comma. We need to take each column and apply some business validation rule. If data itself contains comma, then those fields are enclosed with double quotes ("). We can see this double... (7 Replies)
Discussion started by: vfrg
7 Replies

7. Shell Programming and Scripting

How to parse csv format?

Hi, I have a file with 3 fields in csv format: /tmp/foo/,MODIFY,bar/toto "/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"I would like to split fields in order to obtain the following: Line1: /tmp/foo/ MODIFY bar/totoLine2: /tmp/foo, bar/ ATTRIB,ISDIR toto, tata/foobarCan't find my way... (11 Replies)
Discussion started by: chebarbudo
11 Replies

8. Shell Programming and Scripting

parse file names

trying to parse out all parts of this file name. REC=`echo "CAMXI.F0150.txt" | sed 's/.*\(*\).*/\1/'` export "FLRECL=$REC" FLECL=0150 I can get the numbers 0150 out of the file name. But need to capture first 5 bytes and extension. So i would export 3 variables (name, length, extension)... (5 Replies)
Discussion started by: rapalanlord
5 Replies

9. Shell Programming and Scripting

I have strigns below, in which we have files names. Can we parse them in one algorithm? If not, the

I have several strings with files names roundrobin_report_report_ALL_dbagadm_${Today}-${TM}_U.csv I want to say That I found a way to have prefix roundrobin_report_report_ALL_dbagadmU. letter=`echo $var | cut -d'.' -f 1 | rev | cut -d'_' -f 1 | rev` echo "NAME = $letter" prefix=`echo $var... (1 Reply)
Discussion started by: digioleg54
1 Replies

10. Shell Programming and Scripting

Need a script to parse data and output to csv

I am not too savvy with arrays and am assuming that what I am looking for needs arrays. This is my requirement. So I have the raw data that gets updated to a log as shown below StudentInfo: FullInfo = { Address = Newark Age = 20 Name= John } StudentInfo:... (2 Replies)
Discussion started by: sidnow
2 Replies

LEARN ABOUT DEBIAN

ifpps

ifpps(8)							netsniff-ng-toolkit							  ifpps(8)

NAME

       ifpps - fetch and format kernel network statistics

SYNOPSIS

       ifpps	 -d|--dev <netdev> [-t|--interval <sec>][-p|--promisc][-c|--term]
	    [-C|--csv][-H|--csv-tablehead][-l|--loop][-v|--version][-h|--help]

DESCRIPTION

       A tiny tool to provide top-like reliable networking statistics.	ifpps reads out the 'real' kernel statistics, so it does not give
       erroneous statistics on high I/O load.

OPTIONS

       ifpps --dev eth0
	   Fetch eth0 interface statistics.

       ifpps --dev eth0 --interval 60 --csv
	   Output eth0 interface statistics every minute in CSV format.

OPTIONS

       -h|--help
	   Print help text and lists all options.

       -v|--version
	   Print version.

       -d|--dev <netdev>
	   Device to fetch statistics for i.e., eth0.

       -p|--promisc
	   Put the device in promiscuous mode

       -t|--interval <time>
	   Refresh time in sec (default 1 sec)

       -c|--term
	   Output to terminal

       -C|--csv
	   Output in CSV format.  E.g. post-processing with Gnuplot et al.

       -H|--csv-tablehead
	   Print CSV table head.

       -l|--loop
	   Loop terminal output.

AUTHOR

       Written by Daniel Borkmann <daniel@netsniff-ng.org>

DOCUMENTATION

       Documentation by Emmanuel Roullit <emmanuel@netsniff-ng.org>

BUGS

       Please report bugs to <bugs@netsniff-ng.org>

								    2012-06-29								  ifpps(8)
All times are GMT -4. The time now is 10:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy