Add Empty columns at the end of csv file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Add Empty columns at the end of csv file
# 8  
Old 05-12-2010
proofed from within Solaris 10:

Code:
-> awk -F, '{$41=""}1' OFS="," file.csv
+ awk -F, {$41=""}1 OFS=, file.csv
awk: syntax error near line 1
awk: bailing out near line 1

-> nawk -F, '{$41=""}1' OFS="," file.csv
+ nawk -F, {$41=""}1 OFS=, file.csv
1,2,3,4,5,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1,2,3,4,5,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1,2,3,4,5,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1,2,3,4,5,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

# 9  
Old 05-13-2010
Quote:
Originally Posted by alister
The standard does not require recomputation of $0 when NF is assigned. However, it does if you assign to a field beyond NF (which also recomputes NF). Instead, try:
Code:
awk -F, '{$41=""}1' OFS="," file

I believe that should work with posix-compliant awks.

Regards,
Alister
Hi Alister,

you pointed right. How ever i did not get ,what do you mean by
Code:
 
"The standard does not require recomputation of $0 when NF is assigned. However, it does if you assign to a field beyond NF (which also recomputes NF)."

# 10  
Old 05-13-2010
It means that if you assign to a field beyond $NF, the necessary fields will be created, NF will be recomputed, and $0 will be rebuilt.

For example (with FS==OFS==","):
Code:
$0 == "a,b,c,d,e,f"
$NF == "f"
NF == 6
$10="g"
NF == 10  # NF has been recomputed
$NF == "g"
$0 == "a,b,c,d,e,f,,,,g"  # $0 has been recomputed to include null fields between the old value of $NF and the new.

Note, that when $0 is rebuilt, the fields are separated by the value of the output field separator, OFS, so the new $0 may not be the same as the original $0 with the new fields appended. For example, if the default setting of FS is used, consecutive instances of whitespace delimit a field, but the OFS is a single space. If there are many spaces between two fields, when $0 is recomputed, there will be only one.

With default values of FS and OFS:
Code:
$0 == "a b        c       d       e       f"
$NF == "f"
NF == 6
$10="g"
NF == 10  # NF has been recomputed
$NF == "g"
$0 == "a b c d e f    g"  # $0 has been recomputed to include null fields between the old value of $NF and the new.
                          # Note that the spaces between fields have been compressed to a single space.
                          # Also, this is not reversible; once done, if you did not save the original value of $0,
                          # it cannot be restored.

$0 is similary recomputed when you assign a value to a field, even if it already exists. If NF=6, $1=$1 will trigger this recomputation.
Code:
$0 == "a b        c       d       e       f"
$1=$1
$0 == "a b c d e f"  # $0 has been recomputed, the spaces between fields have been replaced by the value of OFS,
                     # but NF remains unchanged as we did not extend the number of fields in any way.

However, simply assinging a value to NF (instead of $NF) does not necessarily trigger the creation of fields and recomputation of $0.

Hope that helps,
Alister

Last edited by alister; 05-13-2010 at 09:52 AM..
This User Gave Thanks to alister For This Post:
# 11  
Old 05-13-2010
Thanks for your valuable inputs. I guess the csv file where I want to add empty columns at the end, contails a new line character at the end.

See below the input file format:
-> more test.csv
Code:
Object Type,Object Subtype,Object Name,CEMLI Type,Owner,Location,Created By, Creation Date, Updated By, Update Date, LastRecorded Usage
DB_OBJECT,ALERT,@GEPSAlphaPOSOEAInotification,CONFIGURATION,FND,ALR_ALERTS,CONCURRENT_MGR,10-Feb-10,CONCURRENT_MGR,test,

-> awk -F, '{NF=41}1' OFS="," test.csv > result.csv
-> more result.csv
Object Type,Object Subtype,Object Name,CEMLI Type,Owner,Location,Created By, Creation Date, Updated By, Update Date, LastRecorded Usage
, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
DB_OBJECT,ALERT,@GEPSAlphaPOSOEAInotification,CONFIGURATION,FND,ALR_ALERTS,CONCURRENT_MGR,10-Feb-10,CONCURRENT_MGR,test,
, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Is it possible to add those empty columns in between the last field seprator and new line characher?

Last edited by Scott; 05-13-2010 at 11:30 AM.. Reason: Code tags, please...
# 12  
Old 05-13-2010
Huh? Do you want to insert the empty fields into the line, and not at the end of it? Or are you dealing with multiline records? I'm not certain what you mean. Why don' t you post a sample record from your csv file, as well as how that record should appear after the desired transformation? That would be most helpful.

If that awk snippet isn't giving you the desired result, perhaps the following will:
Code:
awk -F, '{$41=""}1' OFS="," file

Regards,
Alister
# 13  
Old 05-13-2010
Can you post the output of:
Code:
head -3 test.csv |  od -An -t x1

Please use code tags.
# 14  
Old 05-13-2010
The result of command
Code:
-> head -3 test.csv |  od -An -t x1 
 4f 62 6a 65 63 74 20 54 79 70 65 2c 4f 62 6a 65
 63 74 20 53 75 62 74 79 70 65 2c 4f 62 6a 65 63
 74 20 4e 61 6d 65 2c 43 45 4d 4c 49 20 54 79 70
 65 2c 4f 77 6e 65 72 2c 4c 6f 63 61 74 69 6f 6e
 2c 43 72 65 61 74 65 64 20 42 79 2c 20 43 72 65
 61 74 69 6f 6e 20 44 61 74 65 2c 20 55 70 64 61
 74 65 64 20 42 79 2c 20 55 70 64 61 74 65 20 44
 61 74 65 2c 20 4c 61 73 74 52 65 63 6f 72 64 65
 64 20 55 73 61 67 65 0d 0a 44 42 5f 4f 42 4a 45
 43 54 2c 41 4c 45 52 54 2c 40 47 45 50 53 20 41
 6c 70 68 61 50 4f 53 4f 20 45 41 49 20 6e 6f 74
 69 66 69 63 61 74 69 6f 6e 2c 43 4f 4e 46 49 47
 55 52 41 54 49 4f 4e 2c 46 4e 44 2c 41 4c 52 5f
 41 4c 45 52 54 53 2c 43 4f 4e 43 55 52 52 45 4e
 54 5f 4d 47 52 2c 31 30 2d 46 65 62 2d 31 30 2c
 43 4f 4e 43 55 52 52 45 4e 54 5f 4d 47 52 2c 74
 65 73 74 2c 0d 0a 44 42 5f 4f 42 4a 45 43 54 2c
 41 4c 45 52 54 2c 40 47 45 50 53 20 41 6c 70 68
 61 50 4f 53 4f 20 53 75 70 70 6c 20 53 69 74 65
 20 73 65 74 75 70 20 61 6c 65 72 74 20 6e 6f 74
 69 66 79 2c 43 4f 4e 46 49 47 55 52 41 54 49 4f
 4e 2c 46 4e 44 2c 41 4c 52 5f 41 4c 45 52 54 53
 2c 43 4f 4e 43 55 52 52 45 4e 54 5f 4d 47 52 2c
 32 39 2d 4a 61 6e 2d 31 30 2c 43 4f 4e 43 55 52
 52 45 4e 54 5f 4d 47 52 2c 20 2c 0d 0a

Moderator's Comments:
Mod Comment Please use code tags

Last edited by Scott; 05-13-2010 at 12:24 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To add multiple columns in csv file

Hi Guys, Can you help to add static values at the end of the csv file with headers input_file id,name 1,neo 2,pull Expected id,name,status,entry,g_id 1,neo,done,2019-11-01T07:14:23,pass 2,pull,done,2019-11-01T07:14:23,pass My try but not able replacing properly and unable... (1 Reply)
Discussion started by: Master_Mind
1 Replies

2. Shell Programming and Scripting

Add 8 columns at the end of .csv file using awk

Hello all, I have a .csv file of 16 columns consists of bunch of numbers. 6.45E+01 1.17E+01 8.10E+04 8.21E+01 8.50E+00 1.20E+01 1.02E+01 1.88E+01 1.86E+04 3.53E+03 1.09E+07 3.82E+04 2.09E+03 3.57E+03 2.98E+03 3.93E+03 6.34E+01 3.23E+01 9.24E+04 ... (5 Replies)
Discussion started by: Zam_1234
5 Replies

3. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

4. Linux

Add empty columns at the end of a CSV file

I have a .CSV file (lets say named as file.csv) with numeric and string values. The string might contain commas hence they are enclosed in double quotes as in the below format. column1,column2,column3,column4,column5,column6,column7 12,455,"string, with, quotes, and with, commas, in... (3 Replies)
Discussion started by: dhruuv369
3 Replies

5. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

6. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

7. Shell Programming and Scripting

AWK : Add columns in the end of csv file

Hi everybody, I need some help please I have a csv file named masterFile1.csv header1,header2,header3 value1,value2,value3 value4,value5,value6 I am trying to add new columns in the end of the csv to have a new csv file named masterFile2.csv like this :... (3 Replies)
Discussion started by: villebonnais
3 Replies

8. Shell Programming and Scripting

Add columns in csv file

Hi everybody, I am new here and I need a reel help please A have a csv file and I want to add new in the end of the file Devisce Model,VMGuest Name,Host OS Name, Memory Size Value1,Value2,Value3,Value4 Value5,Value6,Value7,Value8 Value9,Value10,Value11,Value12 And I want to add to new... (3 Replies)
Discussion started by: villebonnais
3 Replies

9. UNIX for Dummies Questions & Answers

Merging Non-Empty Columns within a CSV

I am trying to place all my data in a single row (order doesn't matter). Note I am a Unix novice, so please go easy on me. Here is an example Raw data: row# (1) 45 64 23 (2) 32 1 6 56 (3) 32 45 Needs to be like this: row# (1) 45 (2) 32 (3) 32 ... (2 Replies)
Discussion started by: mmann1123
2 Replies

10. Filesystems, Disks and Memory

manipulate csv file to add columns

Hi, I have a csv file with a key composed by 3 columns and some other numeric fields and I need to obtain the partial amounts by some part of the key. This may be some difficult to understand, so better see an example, where my input file is: name,surname,department,y2004,y2005,y2006... (6 Replies)
Discussion started by: oscarmon
6 Replies
Login or Register to Ask a Question