Visit Our UNIX and Linux User Community


How to read and parse the content of csv file containing # as delimeter into fields using Bash?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to read and parse the content of csv file containing # as delimeter into fields using Bash?
# 1  
Old 02-22-2010
How to read and parse the content of csv file containing # as delimeter into fields using Bash?

Code:
#!/bin/bash
i=0
cat 1.csv | while read fileline
do
echo "$fileline"
IFS="#" flds=( $fileline )
nrofflds=${#flds[@]}
echo "noof fields$nrofflds"
fld=0
while [ "$fld" -lt "$nrofflds" ]
do
      echo "noof counter$fld" 
      echo "$nrofflds"
 
     #fld1="${flds[0]}" trying to store the content of line to fields but i am unable to do it. 
for example the line look like this A#B#C#D#E#F
i want to A in one feild B in other and so on 
 
      echo "$col2" 
      fld=$(($fld+1))
   done
 
 
 
i=$(($i+1))
echo "$i"
done

thanks
barani
# 2  
Old 02-22-2010
What exactly you are trying to do with the csv file as input ?.
# 3  
Old 02-22-2010
use tr
change the # to a tab - tab characters don't show in this example just enter tic (') then a tab, then another tic
Code:
cat 1.csv | tr '#' '	'| while read fileline
do
echo "$fileline"
IFS='	 ' flds=( $fileline )
nrofflds=${#flds[@]}
echo "noof fields$nrofflds"
fld=0
while [ "$fld" -lt "$nrofflds" ]
do
      echo "noof counter$fld" 
      echo "$nrofflds"
 
      fld1="${flds[0]}" 
 
      echo "$col2" 
      fld=$(($fld+1))
   done
 
 
 
i=$(($i+1))
echo "$i"
done

You have other errors as well - the fld variable is not initialized. You cannot do arithmetic comparisons on an unset variable.

Consider awk - not shell - it is meant for what you seem to want to do.
# 4  
Old 02-22-2010
how to do in awk

Hi,

the csv file with the delimeter #.

A#B#C#D#E#F#G#H
1#2#3#4#5#6#7#8
Z#x#c#V
7#2#8#9
N.

I want to read the file line by line and store in rowarray.
then the rowarray content should be spilt or made to fields using the delimeter #.
i am not sure can we store the fields in to column array.then it may be loaded to database.







then these fields are to be loaded to a oracle database.
may be in awk.provide the code if possible.

thanks
barani
# 5  
Old 02-22-2010
Quote:
Originally Posted by barani75
[...]
then these fields are to be loaded to a oracle database.
[...]
I would consider using Oracle SQL*Loader ...

Previous Thread | Next Thread
Test Your Knowledge in Computers #138
Difficulty: Easy
The Unix philosophy is a computer operating system idea that means the operating system provides a set of basic tools that perform a limited, well-defined function, a unified filesystem as the main method of communication, and a shell scripting and command language to combine modular tools to perform more complex workflows.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse file for fields and specific text

I have a file of ~500,000 entries in the following: file.txt chr1 11868 12227 ENSG00000223972.5 . + HAVANA exon . gene_id "ENSG00000223972.5"; transcript_id "ENST00000456328.2"; gene_type "transcribed_unprocessed_pseudogene"; gene_status "KNOWN"; gene_name "DDX11L1"; transcript_type... (17 Replies)
Discussion started by: cmccabe
17 Replies

2. UNIX for Advanced & Expert Users

Script to parse and compare information in two fields of file

Hello, I am working parsing a large input file1(field CFA) I have to compare the the file1 field(CFA byte 88-96) with the content of the file2(It contains only one field) and and insert rows equal in another file. Here is my code and sample input file: ... (7 Replies)
Discussion started by: GERMANOS
7 Replies

3. Shell Programming and Scripting

BASH script to parse XML and generate CSV

Hi All, Hope all you are doing good! Need your help. I have an XML file which needs to be converted CSV file. I am not an expert of awk/sed so your help is highly appreciated!! XML file looks like this: <l:event dateTime="2013-03-13 07:15:54.713" layerName="OSB" processName="ABC"... (2 Replies)
Discussion started by: bhaskar_m
2 Replies

4. Shell Programming and Scripting

Read csv file in bash

how to I use IFS to read 2 files (csv) and run the followiung script ./naviseccli -h 1.2.3.4 storagegroup -addhlu -gname $hostname -hlu $hlu_num -alu $alu_num the csv file for $hostname is host1 host2 . . . for hlu and alu its alu,hlu (2 Replies)
Discussion started by: tdubb123
2 Replies

5. Shell Programming and Scripting

Perl: Parse Hex file into fields

Hi, I want to split/parse certain bits of the hex data into another field. Example: Input data is Word1: 4f72abfd Output: Parse bits (5 to 0) into field word1data1=0x00cd=205 decimal Parse bits (7 to 6) into field word1data2=0x000c=12 decimal etc. Word2: efff3d02 Parse bits (13 to... (1 Reply)
Discussion started by: morrbie
1 Replies

6. Shell Programming and Scripting

Count the delimeter from a file and delete the row if delimeter count doesnt match.

I have a file containing about 5 million rows, in the file there are some records which has extra delimiter at random position. (we dont know the positions), now we have to Count the delimeter from each row and if the count of delimeter is not matching then I want to delete those rows from the... (5 Replies)
Discussion started by: Akumar1
5 Replies

7. Shell Programming and Scripting

Bash Script to read a file and parse each record

Hi Guys, I am new to unix scripting and I am tasked to parse through a CSV file delimited by #. Sample: sample.csv H#A#B#C D#A#B#C T#A#B#C H = Header D = Detail Record T = Tail What I need is to read the file and parse through it to get the columns. I have no idea on how... (8 Replies)
Discussion started by: 3vilwyatt
8 Replies

8. Shell Programming and Scripting

Parse csv file

Hi, Our requirement is to parse the input file(.csv format). The each column in the file is delimited with comma. We need to take each column and apply some business validation rule. If data itself contains comma, then those fields are enclosed with double quotes ("). We can see this double... (7 Replies)
Discussion started by: vfrg
7 Replies

9. UNIX for Dummies Questions & Answers

How do I read/find/replace fields in a csv datafile?

hello. I'm somewhat a novice here so please be patient. My stumbling block when loading csvs into ORACLE tables is this: I need to read a csv datafile, check several fields in each line, and if any of stated fields contain A ZERO only then replace it with a null/blank character. I had a... (9 Replies)
Discussion started by: MrCarter
9 Replies

10. Shell Programming and Scripting

Re-usable function to parse csv files with different number of fields

Hi there, been pondering how to deal with this and hoping someone would give me an insight on this. I need help on creating a reusable bash funtion to parse csv files containing different number of fields (comma-seperated). My initial thought is to create function for each input csv file (20+... (2 Replies)
Discussion started by: jy2k7ca
2 Replies

Featured Tech Videos