Sponsored Content
Full Discussion: Read .csv file
Top Forums Shell Programming and Scripting Read .csv file Post 302873491 by FabianDe on Tuesday 12th of November 2013 08:35:14 AM
Old 11-12-2013
Read .csv file

Hello,

this is my very first comment on this forum and i hope i don't mess it up. If i do, please forgive me (also for any language mistakes you may wanna know that i'm not native speaking).

As i just started doing a bigger paper for my studies i got a bunch of data in seperate csv-files which i'll have to analyze. Unfortunately the data logger messed up the naming for most of the files, so i have to fix that first in order to have a consistent naming. So i figured i'll write a little bash program for that. Here's roughly what i got so far:

Code:
#!/bin/bash
filelist="SP*"
echo "filenames" > filenames.dat
for file in $filelist
  do
	temp=$(head -1 $file)
#	echo "$temp" >> filenames.dat
	cp $file $temp
done

So what i'm doing is finding all files that have the string "SP" in it and add them to the list "filelist". Then i consecutively go through that list, open the file and save the first line in the variable "temp" (the correct name of the files is saved in that first line). Then i copy the files using the variable "temp" as new name.

Theoretically the first line consists of two tokens seperated by a semicolon (so two columns since csv-file), whereas i only need the first. But that's just an extra i don't necessarily need to work. The first line would be okay, too.

So my .csv-files look kinda like this:

Code:
SPY.SPYNODE.SIDE_R.IOM1.LaneA.SYSAP_IOM.SYSAP_IOM_IMU_READER.W_APPLI_IOM_IMU_READER_MEAS03_SigInsReceiveImuOutputToInsMeas03VelincY#VALUE;Integer
Timestamp;Value
1366346707344760064;0
1366346707360758016;0
1366346707376759040;0
1366346707392758016;0

As i thought it should work but did not ("Argument too long") i took a closer look to what actually happens. Apparently the head command doesn't work properly, so what i really does is saving the whole file (so including all the numbers also) in the variable "temp". If i then use that variable to create/copy my file the command is too long for the shell (actually the file consists of about 500 000 lines :-) ). So apparently the "head -1" doesn't work, but i don't know why. "head -n 1" didn't work either.

I have no idea what's going on and i don't know what i could change.

Can anybody help me out here and give me a piece of advice (i'd appreciate it if the advice was actually concerning this topic of course :-) ).

Thank you very much in advance.

Fa
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read Csv file

Hi All, I need to check if a csv file is empty, leaving the first line as the first line is header pls help Thanks (3 Replies)
Discussion started by: gwrm
3 Replies

2. Shell Programming and Scripting

Read Two lines in a CSV File and Compare

Hi , I have a CSV file ( file.csv) with some data as below: A,1,abc,x,y,z,,xyz,20100101,99991231 A,1,abc,x,y,z,234,xyz,20100101,99991231 I have to delete the duplicate line based on unique identifiers which are values in the fields- 2,3,4,8.These coulmns in both the rows have same... (6 Replies)
Discussion started by: Sheel
6 Replies

3. Shell Programming and Scripting

How can I read values from a CSV file using Shell?

SHELL SCRIPT Hi, I have a file in which contents are as follows: 9999,abdc,123 9988,aba_12,2323 and so on I want to read the contents of this file such that i can do echo "This is $a followed by $b an then $c" I tried the following but id did not work cat test | cut -d ',' -f1|... (7 Replies)
Discussion started by: mayanksargoch
7 Replies

4. Shell Programming and Scripting

read .csv file

Need UNIX script read below .csv file and print the line only records where type = TRN. Srno,Type,InputFileName,NewColumnData 1,TRN,File1.dat,11 2,TRN,File2.dat,12 3,TRN,File3.dat,13 4,REF,File4.dat, 5,REF,File5.dat, regards, santosh (4 Replies)
Discussion started by: santosh2k2
4 Replies

5. Shell Programming and Scripting

to read a CSV file and generate SQL output

Friends, This is what I need: I will pass a CSV file as an input, and I want my shell to be reading that CSV file, and based on the parameters it should generate SQLs and write those SQL in a different file in the same location. I'm new to Shell scripting. I'm currently working on a... (1 Reply)
Discussion started by: Ram.Math
1 Replies

6. UNIX for Dummies Questions & Answers

skip first line when doing a read of csv file

Folks, how do i skip the first line in a csv, while doing the read of a csv file in to a variable line by line. eg : do echo $line done < $rpt where rpt is path to csv file The initial 1st line is a garbage that i want to avoid, and start reading from 2nd line ... (2 Replies)
Discussion started by: venu
2 Replies

7. UNIX for Dummies Questions & Answers

When reading a csv file, counter to read 20 lines and wait for minute then read next 20 till end

Hello All, i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file. Any inputs are appreciated ... (3 Replies)
Discussion started by: victor.s
3 Replies

8. Shell Programming and Scripting

Read csv file in bash

how to I use IFS to read 2 files (csv) and run the followiung script ./naviseccli -h 1.2.3.4 storagegroup -addhlu -gname $hostname -hlu $hlu_num -alu $alu_num the csv file for $hostname is host1 host2 . . . for hlu and alu its alu,hlu (2 Replies)
Discussion started by: tdubb123
2 Replies

9. Shell Programming and Scripting

Not able to read .csv file until open in vi editor

Hi, I am facing a problem regarding .csv file, my script does not read .csv file and if i open this file in vi editor and perform :wq option then only my script reads the .csv file. Thanks (5 Replies)
Discussion started by: ranabhavish
5 Replies

10. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies
Text::CSV::Encoded::Coder::EncodeGuess(3pm)		User Contributed Perl Documentation	       Text::CSV::Encoded::Coder::EncodeGuess(3pm)

NAME
Text::CSV::Encoded::Coder::EncodeGuess - Text::CSV::Encoded coder class using Encode::Guess SYNOPSIS
use Text::CSV::Encoded coder_class => 'Text::CSV::Encoded::Coder::EncodeGuess'; use Spreadsheet::ParseExcel; my $csv = Text::CSV::Encoded->new(); $csv->encoding( ['ucs2', 'ascii'] ); # guessing ucs2 or ascii? $csv->encoding_to_combine('shiftjis'); my $excel = Spreadsheet::ParseExcel::Workbook->Parse( $file ); my $sheet = $excel->{Worksheet}->[0]; for my $row ( $sheet->{MinRow} .. $sheet->{MaxRow} ) { my @fields; for my $col ( $sheet->{MinCol} .. $sheet->{MaxCol} ) { my $cell = $sheet->{Cells}[$row][$col]; push @fields, $cell->{Val}; } $csv->combine( @fields ) or die; print $csv->string, " "; } DESCRIPTION
This module is inherited from Text::CSV::Encoded::Coder::Encode. USE
Except for 2 attributes, same as Text::CSV::Encoded::Coder::Encode. encoding_in $csv = $csv->encoding_in( $encoding_list_ref ); The accessor to an encoding for pre-parsing CSV strings. If no encoding is given, returns current $encoding, otherwise the object itself. $encoding_list_ref = $csv->encoding_in() When you pass a list reference, it might guess the encoding from the given list. $csv->encoding_in( ['shiftjis', 'euc-jp', 'iso-20022-jp'] ); If it cannot guess the encoding, the first encoding of the list is used. encoding $csv = $csv->encoding( $encoding_list_ref ); $encoding_list_ref = $csv->encoding(); You can pass a list reference to this attribute only: * For list data consumed by combine(). * For list reference returned by getline(). In other word, in "combine" and "print", it might guess an encoding for the passing list data. If it cannot guess the encoding, the first encoding of the list is used. SEE ALSO
Encode, Encode::Guess AUTHOR
Makamaka Hannyaharamitu, <makamaka[at]cpan.org> COPYRIGHT AND LICENSE
Copyright 2008-2010 by Makamaka Hannyaharamitu This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2010-04-26 Text::CSV::Encoded::Coder::EncodeGuess(3pm)
All times are GMT -4. The time now is 07:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy