Data Manipulation on a .csv file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Data Manipulation on a .csv file
# 1  
Old 05-15-2013
Data Manipulation on a .csv file

Hallo Friends,

I need you help. My file has 5000 or so lines and currently looks like below(sample).

Code:
Service Type,Origin,Destination,Rate Per Minute,Minimum Charge,Time Based Rate,Time Based From Day,Time Based To Day,Time Based From Time,Time Based To Time,Destination Prefix List,,
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284571|0215284580|0215284655',,
VoIS Community,Rest of RSA,Maxi Call,0.17,no,no,,,,,'0861',,
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,'35547210|35547425|35547550|35547700|35547870|35548270|35548570|35548660|35548855|35549440|35549930',,

I would like the output to be like below:

Code:
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284571
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284580
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284655',,
VoIS Community,Rest of RSA,Maxi Call,0.17,no,no,,,,,'0861',,
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547210
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547425
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547550
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547700
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547870
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548270
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548570
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548660
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548855
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35549440
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35549930',,

# 2  
Old 05-15-2013
Some like this?

Code:
awk -F, 'NR==1 {next} {n=split($11,a,"|")} {$11="";for (i=0;i++ <=n-1;) print $0,a[i]}' OFS="," file
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284571
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284580
VoIS Community,Rest of RSA,Datacash Extend Calls,0.36,no,no,,,,,0215284655'
VoIS Community,Rest of RSA,Maxi Call,0.17,no,no,,,,,'0861'
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,'35547210
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547425
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547550
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547700
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35547870
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548270
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548570
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548660
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35548855
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35549440
VoIS Global,All Of RSA,Albania - Amc Fixed 2,1.96,no,no,,,,,35549930'

This User Gave Thanks to Jotne For This Post:
# 3  
Old 05-15-2013
Thank you for your responce Jotne. One more question though lets say the file name is test.csv. Will the command be

Code:
$cat test.csv|awk -F, 'NR>1 {n=split($11,a,"|")} {for (i=0;i++ <=n;) print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,a[i]}' OFS=","


??

---------- Post updated at 03:13 PM ---------- Previous update was at 03:01 PM ----------

Jotne thank you very much. It worked.

Last edited by vgersh99; 05-15-2013 at 10:35 AM.. Reason: code tags, please!
# 4  
Old 05-15-2013
Nice, as you see I did a small change int the script, removed the $1..$10 and change to n-1

PS do not cat the file to awk, awk can read it directly itself like this
Code:
awk -F, 'NR==1 {next} {n=split($11,a,"|")} {$11="";for (i=0;i++ <=n-1;) print $0,a[i]}' OFS="," test.csv

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

2. Shell Programming and Scripting

Pax1.csv file manipulation

Hallo Team and happy Friday, I have a .csv file attached called MHG_Only_Billing_only_great_201408_SummaryDetails.csv The file is as below: $ cat MHG_Only_Billing_only_great_201408_SummaryDetails.csv... (1 Reply)
Discussion started by: kekanap
1 Replies

3. Shell Programming and Scripting

.csv file manipulation

Hall0 Team, I need you help. I have a .csv file called Pax.csv. It has 13 fields The fields that i am interested in is field 7, field 10(CallDuration),field 12 and field 13(TelkomAsif) Field 12(Cost) is affected by what is on field 7(CallType) if field7= VoIS Community Sharecall then Cost... (14 Replies)
Discussion started by: kekanap
14 Replies

4. Shell Programming and Scripting

Manipulation of file data with UNIX

Hello , How all doing today.. I have a little doubt in Unix (6 Replies)
Discussion started by: adisky123
6 Replies

5. Shell Programming and Scripting

CSV data format manipulation

Hi There I need a script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. I am using an application that will only take data in a particular format and need something that will convert without manual intervention. The... (4 Replies)
Discussion started by: rbggbr16
4 Replies

6. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

7. UNIX for Dummies Questions & Answers

Data file manipulation

Hi, I have two, double column data files (file1 and file2). I want to add the second column of file2 to as 3rd column of file1. But, the 3rd column values corresponds to the values of the 2nd column. example: file1: X Y ========= x1 y2 x3 y4 x2 y4 x5 y3 ========= file2: Y ... (7 Replies)
Discussion started by: gaurab
7 Replies

8. Shell Programming and Scripting

Data manipulation from a file

i have a file in follwing format 0110 1020 1011 1032 1020 2005 2003 1050 i want the output in such a way that all non zero numbers will be converted into 1 like this 0110 1010 1011 1011 1010 1001 1001 1010 (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

9. Shell Programming and Scripting

Data manipulation from one file

HI all i have a file consisting of following numbers 0000 0000 0000 0000 0000 1010 0000 0100 0000 0000 0000 1111 0000 1010 0000 0100 (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

10. Shell Programming and Scripting

csv file manipulation

Hi, I generated a csv file.But i have to manipulate it to make it user friendly. This is the structure of the input csv file. partyfailurereasonnumbercvfrejected12cvfapproved45cvfonhold23mvcunknown98mvconcall17pucapproved78puchold34actunknown19 The first line is the header. The... (6 Replies)
Discussion started by: amit.behera
6 Replies
Login or Register to Ask a Question