Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Change the delimiter from Comma to Pipeline

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Change the delimiter from Comma to Pipeline
# 1  
Old 12-17-2012
Linux Change the delimiter from Comma to Pipeline
Hello All,
I need to convert a csv file to pipeline delimiter file in UNIX. The data in file itself contains comma with double qouted qualifier apart from the comma separator. Let me know how to do it. Appreciate any help if awk can be used to do it.

Mentioned below is the sample record of file.

Sample Data:

Code:
Menstrain,A0022699,"Woodstock,IL,2",A0013195,"Rockford,IL",A0013243,"Chicago,IL","EMERGING,QUALITY"


Moderator's Comments:
Mod Comment Use code tags, thanks.
Last edited by zaxxon; 12-17-2012 at 04:26 AM.. Reason: code tags
# 2  
Old 12-17-2012
This is one method.. Smilie

Code:
$ awk -F '"' '{ for(i=1;i<=NF;i+=2){gsub(",","|",$i)}}1' OFS='"' file
Menstrain|A0022699|"Woodstock,IL,2"|A0013195|"Rockford,IL"|A0013243|"Chicago,IL"|"EMERGING,QUALITY"

# 3  
Old 12-17-2012
Linux
WIll that work for all. I mean this file will be extracted on monthly basis and it's not mandatory that the columns which now have double qoutes will have double qoutes when i download the file next month. Is that a generic code?
# 4  
Old 12-17-2012
Quote:
Originally Posted by Arun Mishra
WIll that work for all. I mean this file will be extracted on monthly basis and it's not mandatory that the columns which now have double qoutes will have double qoutes when i download the file next month. Is that a generic code?
Now try

Code:
$ cat file
Menstrain,A0022699,"Woodstock,IL,2",A0013195,"Rockford,IL",A0013243,"Chicago,IL","EMERGING,QUALITY"
Menstrain,A0022699,Woodstock,IL,2,A0013195,Rockford,ILA0013243,Chicago,EMERGING,QUALITY

$ awk -F '"' 'NF>1{ for(i=1;i<=NF;i+=2){gsub(",","|",$i)}}
NF==1{gsub(",","|")}1' OFS='"' file

Menstrain|A0022699|"Woodstock,IL,2"|A0013195|"Rockford,IL"|A0013243|"Chicago,IL"|"EMERGING,QUALITY"
Menstrain|A0022699|Woodstock|IL|2|A0013195|Rockford|ILA0013243|Chicago|EMERGING|QUALITY

# 5  
Old 12-17-2012
Quote:
Originally Posted by pamu
Now try

Code:
$ cat file
Menstrain,A0022699,"Woodstock,IL,2",A0013195,"Rockford,IL",A0013243,"Chicago,IL","EMERGING,QUALITY"
Menstrain,A0022699,Woodstock,IL,2,A0013195,Rockford,ILA0013243,Chicago,EMERGING,QUALITY

$ awk -F '"' 'NF>1{ for(i=1;i<=NF;i+=2){gsub(",","|",$i)}}
NF==1{gsub(",","|")}1' OFS='"' file

Menstrain|A0022699|"Woodstock,IL,2"|A0013195|"Rockford,IL"|A0013243|"Chicago,IL"|"EMERGING,QUALITY"
Menstrain|A0022699|Woodstock|IL|2|A0013195|Rockford|ILA0013243|Chicago|EMERGING|QUALITY

Pamu, Thanks very much for your answer.
Please check the details of the requirements below:

The next month i might get a record like this
Code:
"Menstrain,arr",A0022699,"Woodstock,IL",2,"A001,3195",RockfordIL,"A001,3243","Chicago,america",EMERGINGQUALITY

will your script work for the above ?

My point is the script should replace comma with pipeline at the same time it should check whether the column has comma inside the double qouted qualifier for each record and shouldnt replace those. A column will have double qoutes in the data only if it has comma inside it. Next month i may get different column which has comma inside it and being double qouted.
Last edited by Scrutinizer; 12-17-2012 at 05:30 AM.. Reason: code tagz
# 6  
Old 12-17-2012
Quote:
Originally Posted by Arun Mishra
Pamu, Thanks very much for your answer.
Please check the details of the requirements below:

The next month i might get a record like this
"Menstrain,arr",A0022699,"Woodstock,IL",2,"A001,3195",RockfordIL,"A001,3243","Chicago,america",EMERG INGQUALITY
will your script work for the above ?

My point is the script should replace comma with pipeline at the same time it should check whether the column has comma inside the double qouted qualifier for each record and shouldnt replace those. A column will have double qoutes in the data only if it has comma inside it. Next month i may get different column which has comma inside it and being double qouted.
Why you don't try by yourself...?

Please use code tags for code and data sample.
And always provide every details about input and desired output.
Please try to give your desired output.

please check..

Code:
$ cat file
Menstrain,A0022699,"Woodstock,IL,2",A0013195,"Rockford,IL",A0013243,"Chicago,IL","EMERGING,QUALITY"
Menstrain,A0022699,Woodstock,IL,2,A0013195,Rockford,ILA0013243,Chicago,EMERGING,QUALITY
"Menstrain,arr",A0022699,"Woodstock,IL",2,"A001,3195",RockfordIL,"A001,3243","Chicago,america",EMERG INGQUALITY

$ awk -F '"' 'NF>1{ for(i=1;i<=NF;i+=2){gsub(",","|",$i)}}
NF==1{gsub(",","|")}1' OFS='"' file
Menstrain|A0022699|"Woodstock,IL,2"|A0013195|"Rockford,IL"|A0013243|"Chicago,IL"|"EMERGING,QUALITY"
Menstrain|A0022699|Woodstock|IL|2|A0013195|Rockford|ILA0013243|Chicago|EMERGING|QUALITY
"Menstrain,arr"|A0022699|"Woodstock,IL"|2|"A001,3195"|RockfordIL|"A001,3243"|"Chicago,america"|EMERG INGQUALITY

# 7  
Old 12-17-2012
Sorry i didnt try that time. which i will take care in future before posting any thread. Wow Pamu ... Awesome.. great work.. that really worked.
Just a small help if the script can erase the double qoutes after the file has been converted to pipeline.

And if you have time please can you explain how the script works. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Insert a new column with sequence number (Delimiter as comma)

Hi All, I have a file which has data like a,b c,d e,f g,h And I need to insert a new column at the begining with sequence no( 1 to n) 1,a,b 2,c,d 3,e,f 4,g,h Please let me know how to acheive this in unix (3 Replies)
Discussion started by: weknowd
3 Replies

2. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

3. Shell Programming and Scripting

Replace comma delimiter by newline

The input file is as below AR,age,marks,roll,section,evin,25,80,456,A,atch,23,56,789,B,eena,24 ,78H245,C,Ps,ot,ecessary,hat,ame comes first then age and rest AR AZ,kevin,25,80,456,A,Satch,23,56,789,Satch,23,56,789,B,Meena,24,78,H245,C,AZ ................ ................ I am writting... (8 Replies)
Discussion started by: millan
8 Replies

4. Shell Programming and Scripting

To change the delimiter for first two columns

Dear Friends, I have file as below 1|sdf|rere|sert|trt|rtr i want to change the delimeter first three columns two fields expected output 1~sdf~rere|sert|trt|rtr Plz help (2 Replies)
Discussion started by: i150371485
2 Replies

5. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

6. Shell Programming and Scripting

nawk won't accept comma as delimiter

I have a text file delimited by commas, with three fields (no " marks). I want to use awk to create a fourth field which is equal to the line number + field 1 + .txt I've searched this forum and found the following nawk -v OFS=';' '{print $0, FNR}' myFile Which I've amended to change the... (2 Replies)
Discussion started by: b.hamilton
2 Replies

7. UNIX for Dummies Questions & Answers

Making a Tab delimiter file to Comma

How can i make a tab delimiter file to a comma delimiter??? (13 Replies)
Discussion started by: saggiboy10
13 Replies

8. Shell Programming and Scripting

Replacing Comma delimiter coming inside the data.

Hello, I am having flat file (Comma Delimiter) and the data in the file is as given below. EMPNO, ENAME, DESIGNATION, SALARY 10979, Arun Kumar, Cosultant, 35000 13555, Bidhu Shekar, Senior Consultant, 45000 15000, Kiran, Kumar, Senior, Consultant, 40000 If... (9 Replies)
Discussion started by: arunvasu2
9 Replies

9. Shell Programming and Scripting

Exporting data into Comma Delimiter.

Hi, Requirement: Exporting data from Oracle to UNIX into "Comma" delimiter. Help Needed: I was able to connect to Oracle and import the data. But please let me know while importing the data I need to make it into Comma delimiter flat file. For Example: Source Data - 100 ABC TLead... (6 Replies)
Discussion started by: arunvasu2
6 Replies

10. Shell Programming and Scripting

comma delimiter and space

I have a csv file and there is a problem which I need to resolve. Column1,Column2,Colum3,Column4 ,x,y,z ,d,c,v t,l,m,n ,h,s,k ,k,,y z,j, ,p Now if you see column1 for row 1 and row 4 though they are null there is a space but in case of row2 and row 5 there is no space. I want row... (3 Replies)
Discussion started by: RubinPat
3 Replies
Login or Register to Ask a Question