#!/bin/bash
testfile="./test-dev/test-global-data-import.csv"
var1=`cat $testfile`
IFS=',' record1=( ${var1} )
echo ""
echo "Here is the data without the comma separators."
echo ""
echo ${record1[*]:0}
echo ""
echo "Here is the data with comma separators."
echo ""
echo "${record1[*]:0}"
echo ""
#EOF
++++++++++++++++++++++++++++++
Here are the problems I am having:
1. I am trying to have each record (line in the CSV file) become its own array, such as $record1[*]
2. There will be lots of strange formatting within the CSV file (i.e. IP addresses, street addresses w/ commas and periods) and right now each record has items 1-35 and it could possibly grow to <=100 fields.
3. I am a newbie :-( .... I have edited BASH scripts in the past ... but that no way make me proficient .... so can someone please help.
P.S. I am pouring over several resources on a daily basis (Beginners & Advanced BASH guides on tldp.org and I have a BASH book that I have my nose buried in constantly. I still need some expert assistance though.
Any help is greatly appreciated.
Last edited by radoulov; 01-05-2011 at 04:04 PM..
Reason: Code tags, please!
I'm trying to merge multiple CSV (comma separated value) files into one large master file. All files have a field that is unique to act as the key for entry/merging into the master file & and all files have the same number of fields that are in the master file.
I'll give an example here:
... (2 Replies)
Hi There
I needed to write a Unix shell script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file.
Currently I am in the proess of Data Import to "Remedy System" (A one kind of incident mangement Application) and this... (8 Replies)
Hi all,
I have written a shell script to search a specified directory (e.g. /home/user) for a list of specific words (shown as ${TMPDIR}/wordlist below). The script works well enough, but I was wondering if there was a way to display the line number that the word is found on?
Thanks!
cat... (1 Reply)
Hi, if I want to import .txt file that contain information and the number separate by space how can I split and put into array In C
Example of .txt file
3 Aqaba
49789 10000 5200 25.78
6987 148976 12941 15.78
99885 35262 2501 22.98
Thank (3 Replies)
Hi Guys,
I've got a script that runs and collects statistics from one of our machines and outputs the data into a csv file. Now, that script runs fine and I managed to create another one (with a lot of help from this forum!!) to trim the csv file down to all the data that I need, rather than all... (9 Replies)
Hi All,
I was wondering how I can convert each line in an input file where fields are separated by variable width spaces into a CSV file. Below is the scenario what I am looking for.
My Input data in inputfile.txt
19 15657 15685 Sr2dReader 107.88 105.51... (4 Replies)
I'm not sure if i can put the problem in understandable form.Let me try:
I have a array which is like and always fixed:
Array1=(new inprogress pending Restored Resolved )
Other 2 array array2 which may varry but always subset of above array:
Like it can be :
Array2=(New inprogress... (2 Replies)
Hi everyone,
I have a csv file which has data with different heading and column names as below.
Static Data Ingested
,,,,,,,,,,,,Known Explained Rejections
Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
I have 8 .csv files with 16 columns and "n" rows with no Header. I want to parse each of these .csv and get column and put the data into a new.csv. Once this is done, the new.csv should have 16 columns (2 from each input.csv) and "n" rows.
Now, I want to just take the average of Column from... (3 Replies)
HI
I have a dsv file that looks like:
<<BOF>>
record_number|id_number|first name|last name|msisdn|network|points|card number|gender
312|9101011234011|Test Junior|Smith|071 123 4321|MTN|73|1241551413214444|M
313|9012023213011|Bob|Smith|27743334321|Vodacom|3|1231233232323244|M... (4 Replies)
Discussion started by: tera
4 Replies
LEARN ABOUT DEBIAN
flow-import
flow-import(1) General Commands Manual flow-import(1)NAME
flow-import -- Import flows into flow-tools from other NetFlow packages.
SYNOPSIS
flow-import [-h] [-b big|little] [-d debug_level] [-f format] [-m mask_fields] [-V pdu_version] [-z z_level]
DESCRIPTION
The flow-import utility will convert data from cflowd and ASCII CSV files into flow-tools format.
OPTIONS -b big|little
Byte order of output.
-d debug_level
Enable debugging.
-f format Export format. Supported formats are:
0 cflowd
2 ASCII CSV
3 Cisco NFCollector
-h Display help.
-m mask_fields
Select fields for cflowd and ASCII formats. The mask_fields is built from a bitwise OR of the following:
UNIX_SECS 0x0000000000000001LL
UNIX_NSECS 0x0000000000000002LL
SYSUPTIME 0x0000000000000004LL
EXADDR 0x0000000000000008LL
DFLOWS 0x0000000000000010LL
DPKTS 0x0000000000000020LL
DOCTETS 0x0000000000000040LL
FIRST 0x0000000000000080LL
LAST 0x0000000000000100LL
ENGINE_TYPE 0x0000000000000200LL
ENGINE_ID 0x0000000000000400LL
SRCADDR 0x0000000000001000LL
DSTADDR 0x0000000000002000LL
SRC_PREFIX 0x0000000000004000LL
DST_PREFIX 0x0000000000008000LL
NEXTHOP 0x0000000000010000LL
INPUT 0x0000000000020000LL
OUTPUT 0x0000000000040000LL
SRCPORT 0x0000000000080000LL
DSTPORT 0x0000000000100000LL
PROT 0x0000000000200000LL
TOS 0x0000000000400000LL
TCP_FLAGS 0x0000000000800000LL
SRC_MASK 0x0000000001000000LL
DST_MASK 0x0000000002000000LL
SRC_AS 0x0000000004000000LL
DST_AS 0x0000000008000000LL
IN_ENCAPS 0x0000000010000000LL
OUT_ENCAPS 0x0000000020000000LL
PEER_NEXTHOP 0x0000000040000000LL
ROUTER_SC 0x0000000080000000LL
EXTRA_PKTS 0x0000000100000000LL
MARKED_TOS 0x0000000200000000LL
The default value is all fields applicable to the pdu_version.
-V pdu_version
Use pdu_version format output.
1 NetFlow version 1 (No sequence numbers, AS, or mask)
5 NetFlow version 5
6 NetFlow version 6 (5+ Encapsulation size)
7 NetFlow version 7 (Catalyst switches)
8.1 NetFlow AS Aggregation
8.2 NetFlow Proto Port Aggregation
8.3 NetFlow Source Prefix Aggregation
8.4 NetFlow Destination Prefix Aggregation
8.5 NetFlow Prefix Aggregation
8.6 NetFlow Destination (Catalyst switches)
8.7 NetFlow Source Destination (Catalyst switches)
8.8 NetFlow Full Flow (Catalyst switches)
8.9 NetFlow ToS AS Aggregation
8.10 NetFlow ToS Proto Port Aggregation
8.11 NetFlow ToS Source Prefix Aggregation
8.12 NetFlow ToS Destination Prefix Aggregation
8.13 NetFlow ToS Prefix Aggregation
8.14 NetFlow ToS Prefix Port Aggregation
1005 Flow-Tools tagged version 5
-z z_level
Configure compression level to z_level. 0 is disabled (no compression), 9 is highest compression.
EXAMPLES
Convert the cflowd file flows.cflowd to the flow-tools file flows. Store as Version 5 with compression level 5.
flow-import -V5 -z5 -f0 < flows.cflowd > flows
EXAMPLES
Convert the ASCII CSV data in flows.ascii to flow-tools format. The ASCII data must include all fields represented by 0xFF31EF in the
order listed above. Store as Version 5 with no compression.
flow-import -z0 -f2 -m0xFF31EF < flows.ascii > flows
BUGS
The pcap format is a hack.
AUTHOR
Mark Fullmer maf@splintered.net
SEE ALSO flow-tools(1)flow-import(1)
01-05-2011
