Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How do I format a Date field of a .CSV file with multiple commas in a string field?
Operating Systems Linux How do I format a Date field of a .CSV file with multiple commas in a string field? Post 302864085 by dhruuv369 on Tuesday 15th of October 2013 11:53:57 PM
Old 10-16-2013
How do I format a Date field of a .CSV file with multiple commas in a string field?
I have a .CSV file (file.csv) whose data are all enclosed in double quotes. Sample format of the file is as below:

Code:
column1,column2,column3,column4,column5,column6, column7, Column8, Column9, Column10
"12","B000QRIGJ4","4432","string with quotes, and with a comma, and colon: in between","4432","author1, name","890","88","11-OCT-11","12"
"4432","B000QRIGJ4","890","another, string with quotes, and with more than, two commas: in between","455","author2, name","12","455","12-OCT-11","55"
"11","B000QRIGJ4","77","string with, commas and (paranthesis) and : colans, in between","12","author3, name","333","22","13-OCT-11","232"

The 9th field is the date field in the format "DD-MMM-YY". I have to convert it to the format YYYY/MM/DD. I am trying to use the below code, but of no use.

Code:
awk -F, '
 BEGIN {
 split("JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC", month, " ")
 for (i=1; i<=12; i++) mdigit[month[i]]=i
 }
 { m=substr($9,4,3)
 $9 = sprintf("%02d/%02d/"20"%02d",mdigit[m],substr($9,1,2),substr($9,8,20))
 print
 }' OFS="," file.csv > temp_file.csv

The out put of the file temp_file.csv after executing the above code is as shown below.

Code:
column1,column2,column3,column4,column5,column6,column7,Column8,00/00/2000,Column10
"12","B000QRIGJ4","4432","string with quotes, and with a comma, and colon: in between","4432","author1,00/00/2000,"890","88","11-OCT-11","12"
"4432","B000QRIGJ4","890","another, string with quotes, and with more than, two commas: in between","455",00/00/2002, name","12","455","12-OCT-11","55"
"11","B000QRIGJ4","77","string with, commas and (paranthesis) and : colans, in between","12","author3,00/00/2000,"333","22","13-OCT-11","232"

As far as I understand, the issue is with the commas in the double quote as my code is taking them into consideration too... Please suggest on the below questions:

1) Does the double quoting all the values in all the fields make any difference? If they make any difference, how do I get rid of them from all the values except the strings with commas in them?
2) Any modifications to my code so I could format the 9th field which in the format "DD-MMM-YYYY" to YYYY/MM/DD
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

change field 2 date format

from this input WEBELSOLAR,29122009,1:1 WIPRO,15062010,2:3 ZANDUREALT,18012007,1:3 i want output as WEBELSOLAR,20091229,1:1 WIPRO,20100615,2:3 ZANDUREALT,20070118,1:3 basically input is in ddmmyyyy format and i was to convert it to yyyymmdd format (1 Reply)
Discussion started by: manishma71
1 Replies

2. Shell Programming and Scripting

csv file field needs to be changed current system date with awk

HI, I have csv file with records as shown below. 4102,Bangalore,G10,21,08/17/2011 09:28:33:188,99,08/17/2011 09:27:33:881,08/17/2011... (1 Reply)
Discussion started by: raghavendra.nsn
1 Replies

3. Shell Programming and Scripting

Replace field with commas with field without commas

Hey guys, I have the following text: 1,2,3,4,5,6,'NULL','when',NULL,1,2,0,'NULL' 1,2,3,4,5,6,'NULL','what','NULL',1,2,0,1 I need the same text with the word NULL without commas u know something like this: 1,2,3,4,5,6,NULL,'when',NULL,1,2,0,NULL 1,2,3,4,5,6,NULL,'what','NULL',1,2,0,1 ... (1 Reply)
Discussion started by: lmyk72
1 Replies

4. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:... (4 Replies)
Discussion started by: krishnix
4 Replies

5. Shell Programming and Scripting

Field validations in multiple files CSV

Hi, I am regular reader of this forum. My advanced thanks to everyone. Below given are the sample files INDATA (Main data) Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. . . N records (140000) eg GRPDATA (Reference file) (2 Replies)
Discussion started by: hyperion.krish
2 Replies

6. Shell Programming and Scripting

awk - CSV file - field with single or multiple spaces

Hi, In a csv file, I want to select records where first column has zero or multiple spaces. Eg: abc.csv ,123,a ,22,b ,11,c a,11,d So output should be: ,123,a ,22,b ,11,c Please advise (5 Replies)
Discussion started by: vegasluxor
5 Replies

7. Shell Programming and Scripting

Print particular string in a field of csv file

Hi, all I need your help and suggestions. I want to print particular strings in a field of a csv file and show them in terminal. Here is an example of the csv file. SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw... (9 Replies)
Discussion started by: refrain
9 Replies

8. Shell Programming and Scripting

Print particular string in a field of csv file - part 2

Hi, all I need your help and suggestions. I want to print particular strings in a field of a csv file and show them in terminal. Here is an example of the csv file. SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw... (7 Replies)
Discussion started by: refrain
7 Replies

9. Shell Programming and Scripting

CSV Split field to check multiple codes

Hello, For work i am trying to generate a combined csv file excisting out of 2 other csv files. The problem i am facing is that the first field on both files have multiple values in there which arent always the same. This first field is also the joining part. The layout of the files is as... (16 Replies)
Discussion started by: SDohmen
16 Replies

10. UNIX for Advanced & Expert Users

Help changing date format in the nth field

Hi, I have two (2) things that I want to do. First is to change the date format that is in the nth field from MM/DD/YY to YY/MM/DD. Preferably, I wish I know how to make it a 4-digit year but I don't. Problem is I can only assume it is a 20 century Second is somehow know how to figure out... (1 Reply)
Discussion started by: newbie_01
1 Replies

LEARN ABOUT PHP

htmlspecialchars_decode

HTMLSPECIALCHARS_DECODE(3)						 1						HTMLSPECIALCHARS_DECODE(3)

htmlspecialchars_decode - Convert special HTML entities back to characters

SYNOPSIS

       string htmlspecialchars_decode (string  $string, [int  $flags = ENT_COMPAT | ENT_HTML401])

DESCRIPTION

	This function is the opposite of htmlspecialchars(3). It converts special HTML entities back to characters.

	The converted entities are: &amp;, &quot; (when ENT_NOQUOTES is not set), &#039; (when ENT_QUOTES is set), &lt; and &gt;.

PARAMETERS

	      o $string
		- The string to decode.

	      o $flags
		-  A bitmask of one or more of the following flags, which specify how to handle quotes and which document type to use. The default
		is ENT_COMPAT | ENT_HTML401.

	      Available $flags constants

	      +--------------+---------------------------------------------------+
	      |Constant Name |							 |
	      | 	     |							 |
	      | 	     |			  Description			 |
	      | 	     |							 |
	      +--------------+---------------------------------------------------+
	      | 	     |							 |
	      | ENT_COMPAT   |							 |
	      | 	     |							 |
	      | 	     | Will  convert  double-quotes  and  leave  single- |
	      | 	     | quotes alone.					 |
	      | 	     |							 |
	      | 	     |							 |
	      | ENT_QUOTES   |							 |
	      | 	     |							 |
	      | 	     |	  Will convert both double and single quotes.	 |
	      | 	     |							 |
	      | 	     |							 |
	      |ENT_NOQUOTES  |							 |
	      | 	     |							 |
	      | 	     | Will  leave  both double and single quotes uncon- |
	      | 	     | verted.						 |
	      | 	     |							 |
	      | 	     |							 |
	      | ENT_HTML401  |							 |
	      | 	     |							 |
	      | 	     |		   Handle code as HTML 4.01.		 |
	      | 	     |							 |
	      | 	     |							 |
	      |  ENT_XML1    |							 |
	      | 	     |							 |
	      | 	     |		     Handle code as XML 1.		 |
	      | 	     |							 |
	      | 	     |							 |
	      |  ENT_XHTML   |							 |
	      | 	     |							 |
	      | 	     |		     Handle code as XHTML.		 |
	      | 	     |							 |
	      | 	     |							 |
	      |  ENT_HTML5   |							 |
	      | 	     |							 |
	      | 	     |		     Handle code as HTML 5.		 |
	      | 	     |							 |
	      +--------------+---------------------------------------------------+
RETURN VALUES

	Returns the decoded string.

CHANGELOG

       +--------+---------------------------------------------------+
       |Version |						    |
       |	|						    |
       |	|		     Description		    |
       |	|						    |
       +--------+---------------------------------------------------+
       | 5.4.0	|						    |
       |	|						    |
       |	|  The constants ENT_HTML401,  ENT_XML1,  ENT_XHTML |
       |	| and ENT_HTML5 were added.			    |
       |	|						    |
       +--------+---------------------------------------------------+
EXAMPLES

       Example #1

	      A htmlspecialchars_decode(3) example

	      <?php
	      $str = "<p>this -&gt; &quot;</p>
";

	      echo htmlspecialchars_decode($str);

	      // note that here the quotes aren't converted
	      echo htmlspecialchars_decode($str, ENT_NOQUOTES);
	      ?>

	      The above example will output:

	      <p>this -> "</p>
	      <p>this -> &quot;</p>

SEE ALSO

       htmlspecialchars(3), html_entity_decode(3), get_html_translation_table(3).

PHP Documentation Group 												HTMLSPECIALCHARS_DECODE(3)
All times are GMT -4. The time now is 01:58 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy