Linux convert Comma delimited file to pipe


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Linux convert Comma delimited file to pipe
# 1  
Old 11-06-2017
Linux convert Comma delimited file to pipe

I have file in linux with comma delimited and string fields in double quotations ", I need to convert them to pipe delimiter please share your inputs.
Example:
Input:
Code:
 "2017-09-30","ACBD,TVF","01234",NULL,18,NULL,"686091802","BANK OF ABCD, LIMITED, THE",790456

Output:
Code:
 2017-09-30|ACBD,TVF|01234|NULL|18|NULL|686091802|BANK OF ABCD, LIMITED, THE|790456

Thanks In advance...
Thanks,

Last edited by Don Cragun; 11-06-2017 at 03:34 PM.. Reason: Add CODE and ICODE tags.
# 2  
Old 11-06-2017
Hi, try:
Code:
awk '{for(i=1; i<=NF; i+=2) gsub(",","|",$i)}1' FS=\" OFS= file


Code:
2017-09-30|ACBD,TVF|01234|NULL|18|NULL|686091802|BANK OF ABCD, LIMITED, THE|790456


--
On Solaris use /usr/xpg4/bin/awk
# 3  
Old 11-06-2017
Or with sed
Code:
sed '
  s/$/,/
  s/"\{0,1\}\([^"]*\)"\{0,1\},/\1|/g
  s/|$//
' file

# 4  
Old 11-06-2017
Using python csv module:-
Code:
import csv

i_file = open('input.csv', 'rb')
o_file = open('output.csv', 'wb')

reader = csv.reader(i_file)

for row in reader:
        writer = csv.writer(o_file, delimiter='|')
        writer.writerow(row)

i_file.close()
o_file.close()

# 5  
Old 11-06-2017
Hi.

If you are going to be dealing with DSV (delimiter-separated-value data, including CSV and TSV), you may be interested in a suite of tools from eBay.

The one that is most applicable here is:
Code:
csv2tsv Convert Comma-separated file to tab-separated file (tsv). (doc)
Path    : ~/executable/csv2tsv
Version : v1.1.14
Type    : ELF 64-bit LSB executable, x86-64, version 1 (GNU ...)
Home    : https://github.com/eBay/tsv-utils-dlang

The tools are available in statically-linked executables as well as source, but the source is D, not all that common in my experience.

So for data on file z9:
Code:
"2017-09-30","ACBD,TVF","01234",NULL,18,NULL,"686091802","BANK OF ABCD, LIMITED, THE",790456

this code:
Code:
$ csv2tsv -t '|' z9

produces:
Code:
2017-09-30|ACBD,TVF|01234|NULL|18|NULL|686091802|BANK OF ABCD, LIMITED, THE|790456

On a system like:
Code:
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.9 (jessie) 
bash GNU bash 4.3.30

Tools in the suite are:
Code:
csv2tsv       Convert Comma-separated file to tab-separated file (tsv).
keep-header   Execute a command in a header aware fashion (tsv).
tsv-append    Concatenates multiple TSV files.
tsv-filter    Test fields for relationship, regular expression match, etc.
tsv-join      Match input lines against lines from a 'filter' file.
tsv-pretty    Output TSV data in a aligned format, cf. align.
tsv-sample    Randomize, sample input lines.
tsv-select    Re-order fields, cf. cut.
tsv-summarize Display results of aggregation operations on fields.
tsv-uniq      Filter out duplicate lines using fields (or line) as a key.

Best wishes ... cheers, drl
This User Gave Thanks to drl For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

2. Shell Programming and Scripting

How to cut a pipe delimited file and paste it with another file to form a comma separated outputfile

Hello ppl I have a requirement to split (cut in unix) a file (A.txt) which is a pipe delimited file into A1.txt and A2.txt Now I have to join (paste in unix) this A2.txt with external file A3.txt to form output file A4.txt which should be CSV (comma separated file) so that third party can... (25 Replies)
Discussion started by: etldev
25 Replies

3. Shell Programming and Scripting

Need a script to convert comma delimited files to semi colon delimited

Hi All, I need a unix script to convert .csv files to .skv files (changing a comma delimited file to a semi colon delimited file). I am a unix newbie and so don't know where to start. The script will be scheduled using cron and needs to convert each .csv file in a particular folder to a .skv... (4 Replies)
Discussion started by: CarpKing
4 Replies

4. Shell Programming and Scripting

how to convert comma delimited file to tab separator

Hi all, How can i convert comma delimited .csv file to tab separate using sed command or script. Thanks, Krupa (4 Replies)
Discussion started by: krupasindhu18
4 Replies

5. Shell Programming and Scripting

Linux - Script to generate the output delimited by Comma/Pipe

Hi All, I have a requirement where I need to go to a directory, list all the files that start with person* (for eg) & read the most recent file from the list of files. While browsing through the forum, i found that the command ls -t will list the files. I am trying to generate the output... (1 Reply)
Discussion started by: dsfreddie
1 Replies

6. Shell Programming and Scripting

How to convert a space delimited file into a pipe delimited file using shellscript?

Hi All, I have space delimited file similar to the one as shown below.. I need to convert it as a pipe delimited, the values inside the pipe delimited file should be as highlighted... AA ATIU2345098809 009697 005374 BB ATIU2345097809 005445 006518 CC ATIU9685098809 003215 003571 DD... (7 Replies)
Discussion started by: nithins007
7 Replies

7. Shell Programming and Scripting

Converting comma separated to pipe delimited file

Hi, I came across a very good script to convert a comma seperated to pipe delimited file in this forum. the script serves most of the requirement but looks like it does not handle embedded double quotes and commas i.e if the input is like 1234, "value","first,second", "LDC5"monitor",... (15 Replies)
Discussion started by: anijan
15 Replies

8. UNIX for Advanced & Expert Users

Urgent! need help! how to convert this file into comma delimited format

Hi experts, I need urget help! I have the a text file with this format: Types of fruits Name of fruits 1,1 Farm_no,1 apple,1 pineapple,1 grapes,1 orange,1 banana,1 2,2--->this is the record seperator Farm_no,2 apple,1 pineapple,1 grapes,3 orange,2 banana,1 3,3--->this is the... (2 Replies)
Discussion started by: natalie23
2 Replies

9. Shell Programming and Scripting

how to convert this file into comma delimited format

Hi experts, I need urget help! I have the a text file with this format: Types of fruits Name of fruits 1,1 Farm_no,1 apple,1 pineapple,1 grapes,1 orange,1 banana,1 2,2--->this is the record seperator Farm_no,2 apple,1 pineapple,1 grapes,3 orange,2 banana,1 3,3--->this is the... (1 Reply)
Discussion started by: natalie23
1 Replies

10. Shell Programming and Scripting

convert a pipe delimited file to a':" delimited file

i have a file whose data is like this:: osr_pe_assign|-120|wg000d@att.com|4| osr_evt|-21|wg000d@att.com|4| pe_avail|-21|wg000d@att.com|4| osr_svt|-11|wg000d@att.com|4| pe_mop|-13|wg000d@att.com|4| instar_ready|-35|wg000d@att.com|4| nsdnet_ready|-90|wg000d@att.com|4|... (6 Replies)
Discussion started by: priyanka3006
6 Replies
Login or Register to Ask a Question