Sponsored Content
Top Forums Shell Programming and Scripting Shell snip to import CSV data into BASH array Post 302485587 by dstrout on Wednesday 5th of January 2011 02:57:14 PM
Old 01-05-2011
Shell snip to import CSV data into BASH array

I have been trying to write a simple snip of bash shell code to import from 1 to 100 records into a BASH array.

I have a CSV file that is structured like:

Code:
record1,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record2,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record3,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record4,item1,item2,item3,item4,etc.,etc. .... (<= 100 items)
record5 ......
record6 ......
...... and so on

I am trying to get this data into corresponding arrays as such:

Code:
$record1[item1-n]
$record2[item1-n]
$record3[item1-n]
$record4[item1-n]

Here is what i have so far:

++++++++++++++++++++++++++++++
Code:
#!/bin/bash
testfile="./test-dev/test-global-data-import.csv"
var1=`cat $testfile`
IFS=',' record1=( ${var1} )

echo ""
echo "Here is the data without the comma separators."
echo ""
echo ${record1[*]:0}
echo ""
echo "Here is the data with comma separators."
echo ""
echo "${record1[*]:0}"
echo ""

#EOF

++++++++++++++++++++++++++++++

Here are the problems I am having:

1. I am trying to have each record (line in the CSV file) become its own array, such as $record1[*]

2. There will be lots of strange formatting within the CSV file (i.e. IP addresses, street addresses w/ commas and periods) and right now each record has items 1-35 and it could possibly grow to <=100 fields.

3. I am a newbie :-( .... I have edited BASH scripts in the past ... but that no way make me proficient .... so can someone please help.

P.S. I am pouring over several resources on a daily basis (Beginners & Advanced BASH guides on tldp.org and I have a BASH book that I have my nose buried in constantly. I still need some expert assistance though. Smilie

Any help is greatly appreciated. Smilie

Last edited by radoulov; 01-05-2011 at 04:04 PM.. Reason: Code tags, please!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merging CSV data using a one liner from shell?

I'm trying to merge multiple CSV (comma separated value) files into one large master file. All files have a field that is unique to act as the key for entry/merging into the master file & and all files have the same number of fields that are in the master file. I'll give an example here: ... (2 Replies)
Discussion started by: jjinca
2 Replies

2. Shell Programming and Scripting

Shell script to format a .CSV data

Hi There I needed to write a Unix shell script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. Currently I am in the proess of Data Import to "Remedy System" (A one kind of incident mangement Application) and this... (8 Replies)
Discussion started by: Uday1982
8 Replies

3. UNIX for Advanced & Expert Users

shell script to format .CSV data

Hi all, I have written a shell script to search a specified directory (e.g. /home/user) for a list of specific words (shown as ${TMPDIR}/wordlist below). The script works well enough, but I was wondering if there was a way to display the line number that the word is found on? Thanks! cat... (1 Reply)
Discussion started by: tmcmurtr
1 Replies

4. Programming

import .txt and split word into array C

Hi, if I want to import .txt file that contain information and the number separate by space how can I split and put into array In C Example of .txt file 3 Aqaba 49789 10000 5200 25.78 6987 148976 12941 15.78 99885 35262 2501 22.98 Thank (3 Replies)
Discussion started by: guidely
3 Replies

5. Shell Programming and Scripting

Shell script to populate an array from a csv file

Hi Guys, I've got a script that runs and collects statistics from one of our machines and outputs the data into a csv file. Now, that script runs fine and I managed to create another one (with a lot of help from this forum!!) to trim the csv file down to all the data that I need, rather than all... (9 Replies)
Discussion started by: jimbob01
9 Replies

6. Shell Programming and Scripting

Converting variable space width data into CSV data in bash

Hi All, I was wondering how I can convert each line in an input file where fields are separated by variable width spaces into a CSV file. Below is the scenario what I am looking for. My Input data in inputfile.txt 19 15657 15685 Sr2dReader 107.88 105.51... (4 Replies)
Discussion started by: vharsha
4 Replies

7. Shell Programming and Scripting

Array comparision in bash shell

I'm not sure if i can put the problem in understandable form.Let me try: I have a array which is like and always fixed: Array1=(new inprogress pending Restored Resolved ) Other 2 array array2 which may varry but always subset of above array: Like it can be : Array2=(New inprogress... (2 Replies)
Discussion started by: InduInduIndu
2 Replies

8. Shell Programming and Scripting

Shell script to extract data from csv file

Hi everyone, I have a csv file which has data with different heading and column names as below. Static Data Ingested ,,,,,,,,,,,,Known Explained Rejections Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
Discussion started by: Vivekit82
14 Replies

9. Shell Programming and Scripting

Import 2 columns from 8 .csv files into pandas df (side by side) and write a new csv

I have 8 .csv files with 16 columns and "n" rows with no Header. I want to parse each of these .csv and get column and put the data into a new.csv. Once this is done, the new.csv should have 16 columns (2 from each input.csv) and "n" rows. Now, I want to just take the average of Column from... (3 Replies)
Discussion started by: Zam_1234
3 Replies

10. UNIX for Beginners Questions & Answers

Using bash script : How to Import data from a dsv file into multiple tables in mysql

HI I have a dsv file that looks like: <<BOF>> record_number|id_number|first name|last name|msisdn|network|points|card number|gender 312|9101011234011|Test Junior|Smith|071 123 4321|MTN|73|1241551413214444|M 313|9012023213011|Bob|Smith|27743334321|Vodacom|3|1231233232323244|M... (4 Replies)
Discussion started by: tera
4 Replies
UI::Dialog(3pm) 					User Contributed Perl Documentation					   UI::Dialog(3pm)

NAME
UI::Dialog SYNOPSIS
use UI::Dialog; my $d = new UI::Dialog ( backtitle => 'Demo', title => 'Default', height => 20, width => 65 , listheight => 5, order => [ 'zenity', 'xdialog' ] ); # Either a Zenity or Xdialog msgbox widget should popup, # with a preference for Zenity. $d->msgbox( title => 'Welcome!', text => 'Welcome one and all!' ); ABSTRACT
UI::Dialog is a OOPerl wrapper for the various dialog applications. These dialog backends are currently supported: Zenity, XDialog, GDia- log, KDialog, CDialog, and Whiptail. There is also an ASCII backend provided as a last resort interface for the console based dialog vari- ants. UI::Dialog is a class that provides a strict interface to these various backend modules. By using UI:Dialog (with it's imposed limi- tations on the widgets) you can ensure that your Perl program will function with any available interfaces. DESCRIPTION
UI::Dialog supports priority ordering of the backend detection process. So if you'd prefer that Xdialog should be used first if available, simply designate the desired order when creating the new object. The default order for detecting and utilization of the backends are as follows: (with DISPLAY env): Zenity, GDialog, XDialog, KDialog (without DISPLAY): CDialog, Whiptail, ASCII UI::Dialog is the result of a complete re-write of the UDPM CPAN module. This was done to break away from the bad choice of name (User- DialogPerlModule) and to implement a cleaner, more detached, OOPerl interface. EXPORT
None INHERITS
None CONSTRUCTOR
new( @options ) EXAMPLE my $d = new( title => 'Default Title', backtitle => 'Backtitle', width => 65, height => 20, listheight => 5, order => [ 'zenity', 'xdialog', 'gdialog' ] ); DESCRIPTION This is the Class Constructor method. It accepts a list of key => value pairs and uses them as the defaults when interacting with the various widgets. RETURNS A blessed object reference of the UI::Dialog class. OPTIONS The (...)'s after each option indicate the default for the option. An * denotes support by all the widget methods on a per-use policy defaulting to the values decided during object creation. debug = 0,1,2(0) order = [ zenity, xdialog, gdialog, kdialog, cdialog, whiptail, ascii ] (as indicated) PATH = [ /bin, /usr/bin, /usr/local/bin, /opt/bin ] (as indicated) backtitle = "backtitle" ('') * title = "title" ('') * beepbefore = 0,1(0) * beepafter = 0,1(0) * height = d+(20) * width = d+(65) * listheight = d+(5) * STATE METHODS
state( ) EXAMPLE if ($d->state() eq "OK") { $d->msgbox( text => "that went well" ); } DESCRIPTION Returns the state of the last dialog widget command. The value can be one of "OK", "CANCEL", "ESC". The return data is based on the exit codes (return value) of the last widget displayed. RETURNS a single SCALAR. ra( ) EXAMPLE my @array = $d->ra(); DESCRIPTION Returns the last widget's data as an array. RETURNS an ARRAY. rs( ) EXAMPLE my $string = $d->rs(); DESCRIPTION Returns the last widget's data as a (possibly multiline) string. RETURNS a SCALAR. rv( ) EXAMPLE my $string = $d->rv(); DESCRIPTION Returns the last widget's exit status, aka: return value. RETURNS a SCALAR. WIDGET METHODS
yesno( ) EXAMPLE if ($d->yesno( text => 'A binary type question?') ) { # user pressed yes } else { # user pressed no or cancel } DESCRIPTION Present the end user with a message box that has two buttons, yes and no. RETURNS TRUE(1) for a response of YES or FALSE(0) for anything else. msgbox( ) EXAMPLE $d->msgbox( text => 'A simple message' ); DESCRIPTION Pesent the end user with a message box that has an OK button. RETURNS TRUE(1) for a response of OK or FALSE(0) for anything else. inputbox( ) EXAMPLE my $string = $d->inputbox( text => 'Please enter some text...', entry => 'this is the input field' ); DESCRIPTION Present the end user with a text input field and a message. RETURNS a SCALAR if the response is OK and FALSE(0) for anything else. password( ) EXAMPLE my $string = $d->password( text => 'Enter some hidden text.' ); DESCRIPTION Present the end user with a text input field, that has hidden input, and a message. Note that the GDialog backend will provide a regular inputbox instead of a password box because gdialog doesn't support pass- words. GDialog is on it's way to the proverbial software heaven so this isn't a real problem. Use Zenity instead :) RETURNS a SCALAR if the response is OK and FALSE(0) for anything else. textbox( ) EXAMPLE $d->textbox( path => '/path/to/a/text/file' ); DESCRIPTION Present the end user with a simple scrolling box containing the contents of the given text file. RETURNS TRUE(1) if the response is OK and FALSE(0) for anything else. menu( ) EXAMPLE my $selection1 = $d->menu( text => 'Select one:', list => [ 'tag1', 'item1', 'tag2', 'item2', 'tag3', 'item3' ] ); DESCRIPTION Present the user with a selectable list. RETURNS a SCALAR of the chosen tag if the response is OK and FALSE(0) for anything else. checklist( ) EXAMPLE my @selection1 = $d->checklist( text => 'Select one:', list => [ 'tag1', [ 'item1', 0 ], 'tag2', [ 'item2', 1 ], 'tag3', [ 'item3', 1 ] ] ); DESCRIPTION Present the user with a selectable checklist. RETURNS an ARRAY of the chosen tags if the response is OK and FALSE(0) for anything else. radiolist( ) EXAMPLE my $selection1 = $d->radiolist( text => 'Select one:', list => [ 'tag1', [ 'item1', 0 ], 'tag2', [ 'item2', 1 ], 'tag3', [ 'item3', 0 ] ] ); DESCRIPTION Present the user with a selectable radiolist. RETURNS a SCALAR of the chosen tag if the response is OK and FALSE(0) for anything else. fselect( ) EXAMPLE my $text = $d->fselect( path => '/path/to/a/file/or/directory' ); DESCRIPTION Present the user with a file selection widget preset with the given path. RETURNS a SCALAR if the response is OK and FALSE(0) for anything else. dselect( ) EXAMPLE my $text = $d->dselect( path => '/path/to/a/file/or/directory' ); DESCRIPTION Present the user with a file selection widget preset with the given path. Unlike fselect() this widget will only return a directory selection. RETURNS a SCALAR if the response is OK and FALSE(0) for anything else. SEE ALSO
PERLDOC UI::Dialog::GNOME UI::Dialog::KDE UI::Dialog::Console UI::Dialog::Backend UI::Dialog::Backend::ASCII UI::Dialog::Backend::CDialog UI::Dialog::Backend::GDialog UI::Dialog::Backend::KDialog UI::Dialog::Backend::Nautilus UI::Dialog::Backend::Whiptail UI::Dialog::Backend::XDialog UI::Dialog::Backend::XOSD UI::Dialog::Backend::Zenity MAN FILES dialog(1), whiptail(1), zenity(1), gdialog(1), Xdialog(1), osd_cat(1), kdialog(1) and nautilus(1) BUGS
Please email the author with any bug reports. Include the name of the module in the subject line. AUTHOR
Kevin C. Krinke, <kckrinke@opendoorsoftware.com> COPYRIGHT AND LICENSE
Copyright (C) 2004 Kevin C. Krinke <kckrinke@opendoorsoftware.com> This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA perl v5.8.8 2004-08-01 UI::Dialog(3pm)
All times are GMT -4. The time now is 10:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy