Sponsored Content
Full Discussion: Validate csv file
Top Forums UNIX for Beginners Questions & Answers Validate csv file Post 302971588 by Don Cragun on Friday 22nd of April 2016 05:55:34 AM
Old 04-22-2016
Your desired output format isn't clear, but maybe something like:
Code:
awk -F'"' '
{	nf = gsub(/,/, ",") + 1
	for(i = 2; i <= NF; i += 2)
		nf -= gsub(/,/, ",", $i)
	printf("%s\tresult:%d\n", $0, nf)
	tf += nf
}
END {	printf("Summary: fields:%d rows:%d average fields/row:%.2f\n",
		tf, NR, tf / NR)
}' OFS='"' file

producing the output:
Code:
bc,pqrs,1234,567,hhh	result:5
2.abc,pqrs,1234,567,hhh,"in,valid",end12	result:7
Summary: fields:12 rows:2 average fields/row:6.00

will give you what you want from your two sample input lines???
This User Gave Thanks to Don Cragun For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

validate csv file load

Hi All, I am using sqlldr to load my csv files into the database. The code in the sh script is as follows. sqlldr ${DBUSER}/${DBPASS}@${ORACLE_SID} \ data=myCSV.data \ bad=myCSV.bad \ control=myCSV.ctl \ ... (0 Replies)
Discussion started by: rahulrathod
0 Replies

2. Shell Programming and Scripting

How to validate a CSV file?

Hi. I think some people have already asked this, but the answers/questions seem to be about validating the contents inside a CSV file. I am simply after a simple variable solution (ie 0 = false, 1 = true) that I can use in my script to say that file so-and-so is actually a CSV file, or in some... (4 Replies)
Discussion started by: ElCaito
4 Replies

3. Shell Programming and Scripting

validate against a file

Hello all, I am having problem in writing a if condition for the following: I have a file Instance.dat which has: #Server Environment server1 dev server2 dev server3 sit #!/bin/ksh ENV=dev for i in $( cat Instances.dat | grep -v '#' |awk {'print $2'} ) do if ]... (7 Replies)
Discussion started by: chiru_h
7 Replies

4. Shell Programming and Scripting

Validate the file

How do we validate the header file. The file number should increament by 1 (position 17 to 19) if not abend the process. first week ABC0001 20100101123 second week ABC0001 20100108124 Third week ABC0001 20100115125 (7 Replies)
Discussion started by: zooby
7 Replies

5. Shell Programming and Scripting

How to: Validate a CSV file using an XSD?

Hi All, I was wondering if there is a utility/perl library/way of validating the contents of a CSV file using an XSD. i.e. Customer csv (including Header & Trailer) Cust_num, Cust_nme, Cust_typ, Cust_act_dte, Cust_loc, 101,Joe's Pizza,Retail,10121979,Detroit, 102,Sony... (6 Replies)
Discussion started by: luinzi
6 Replies

6. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

7. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

8. Shell Programming and Scripting

Script to ingest a csv, validate data and insert into Oracle

Hi all i would appreciate your help... I am looking for a set of unix commands which i can use to 1) ingest a csv file with a known format 2) validate the filename 3) validate the data/datatypes 4) Insert into an oracle db Can you help get me started? yogz888 (1 Reply)
Discussion started by: yogz888
1 Replies

9. Shell Programming and Scripting

Script to validate header in a csv file

Hi All; I am struggling to write a script that validates file header. Header file would be like below with TAB separated TRX # TYPE REF # Source Piece Code Destination Piece Code every time I need to check the txt file if the header was same as above fields if validation success... (6 Replies)
Discussion started by: heye18
6 Replies
CLUBAK(1)						     ClusterShell User Manual							 CLUBAK(1)

NAME
clubak - format output from clush/pdsh-like output and more SYNOPSIS
clubak [ OPTIONS ] DESCRIPTION
clubak formats text from standard input containing lines of the form "node:output". It is fully backward compatible with dshbak(1) but provides additonal features. For instance, clubak always displays its results sorted by node/nodeset. You do not need to use clubak when using clush(1) as all output formatting features are already included in. It is provided for other usages, like post-processing results of the form "node:output". Like clush(1), clubak uses the ClusterShell.MsgTree module of the ClusterShell library (see pydoc ClusterShell.MsgTree). INVOCATION
clubak should be started with connected standard input. OPTIONS
--version show clubak version number and exit -b, -c gather nodes with same output (-c is provided for dshbak(1) compatibility) -d, --debug output more messages for debugging purpose -L disable header block and order output by nodes -r, --regroup fold nodeset using node groups -s GROUPSOURCE, --groupsource=GROUPSOURCE optional groups.conf(5) group source to use -G, --groupbase do not display group source prefix (always @groupname) -S SEPARATOR, --separator=SEPARATOR node / line content separator string (default: :) -F, --fast faster but memory hungry mode (preload all messages per node) -T, --tree message tree trace mode; switch to enable ClusterShell.MsgTree trace mode, all keys/nodes being kept for each message element of the tree, thus allowing special output gathering --color=WHENCOLOR whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. WHENCOLOR is never, always or auto (which use color if standard output refers to a terminal). Color is set to [34m (blue foreground text) and cannot be modified. --diff show diff between gathered outputs EXIT STATUS
An exit status of zero indicates success of the clubak command. EXAMPLES
1. clubak can be used to gather some recorded clush(1) results: Record clush(1) results in a file: # clush -w node[1-7] uname -r >/tmp/clush_output # clush -w node[32-159] uname -r >>/tmp/clush_output Display file gathered results (in line-mode): # clubak -bL </tmp/clush_output 2. Another example, iterate over node* text files in current directory and gather characters count for all of them: # find -name "node*" -exec wc -c {} ; | awk '{ gsub("./","",$2); print $2": "$1 }' | clubak -bL node[1,3]: 7 node2: 9 SEE ALSO
clush(1), nodeset(1), groups.conf(5). BUG REPORTS
Use the following URL to submit a bug report or feedback: https://github.com/cea-hpc/clustershell/issues AUTHOR
Stephane Thiell, CEA DAM <stephane.thiell@cea.fr> COPYRIGHT
CeCILL-C V1 1.6 2012-03-28 CLUBAK(1)
All times are GMT -4. The time now is 08:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy