Removing blank columns from a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing blank columns from a file
# 1  
Old 08-18-2008
Removing blank columns from a file

Hi,

I have a sample file as shown below --

id parent name dba
----------------------------------- ----------------------------------- ---------------------------------------- ----------------------------------------
0367169 BANK OF LITHUANIA
0108949 BRATTON LTD
0214359 HALYARD RE BV
0216856 VIKING CABLE AS
0400088 DEUTSCHE BANK AG
0212256 AXA SA
0020503 O/S COUPON SETTLEMENTS
0167528 TT UK LTD


The column under the headers parent and dba are fully blank. I want to convert this file into a csv. tried using sed s/" "*" "/,/g but that removes the whole blank columns as well... any ideas ??

Last edited by sumirmehta; 08-18-2008 at 05:24 PM.. Reason: Reformatted the input file
# 2  
Old 08-18-2008
Is it a fixed length format file? If so what is the length of the fields? You want to convert it into csv file, including the blank columns right?
If not any sample input and output?
# 3  
Old 08-18-2008
the file is basically the dump of a query/stored proc , assuming it is fixed format (each column width being n characters, say 10), how could it be done then....

another example of what is required


input

a b c
---------- ---------- ----------
1234 xyz
3456 ABC
5325 DEF


(it is not visible somehow over here, but there are blank spaces between a-b and b-c and similarly between values in other columns,
thus for row1, a=1234, b= , c=xyz , and similarly for the rest of the file)

and output would be something like this --
a,b,c
----------,----------,----------
1234,,xyz
3456,,ABC
5325,,DEF


The first two rows are not necessary and could be ignored in the final output (if that helps in any way)

Last edited by sumirmehta; 08-18-2008 at 05:49 PM.. Reason: typo error :-)
# 4  
Old 08-18-2008
well, as u said, if the format of the number is fixed ,that should be easy :
Code:
sed 's/\([0-9]\{4\}\) /\1,,/' filename

# 5  
Old 08-18-2008
there are certain variations to the file mentioned above

1. the width of a column is fixed (say 10) but that includes the numbers and padded spaces, so it could be "1234 " and "123456 "

2. there are other columns as well (3 shown here for simplicity), and any of the columns in between could be blank.

The real problem here is that as there are spaces in the columns which have values (padded spaces) as well as there are columns which are totally blank, it is difficult to identify the diff between them..
# 6  
Old 08-18-2008
You can use cut -c to specify the character columns to keep, but cut can't add commas without help. A regular expression matching the columns you want to keep might also be an option.

You can use code tags to post snippets with their whitespace preserved.
# 7  
Old 08-18-2008
the cut -c thing does work... but is a little bit crude ... any other ideas?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to find blank records in a file except for few columns

I have a file with the following format: X|High|2|GIC|DM||XHM|||6 Months X|Moderate|2|GIC|DM||XHM|||6 Months X|High|2|GCM|DM||XSF|||6 Months X|Med|2|GCM|DM||XSF|||6 Here there are ten columns but I need to print rows having blank records in any of the rows (except for 6th,8th and 9th... (10 Replies)
Discussion started by: chatwithsaurav
10 Replies

2. Shell Programming and Scripting

Remove blank columns from a tab delimited text file

Hello, I have some tab delimited files that may contain blank columns. I would like to delete the blank columns if they exist. There is no clear pattern for when a blank occurs. I was thinking of using sed to replace instances of double tab with blank, sed 's/\t\t//g' All of the examples... (2 Replies)
Discussion started by: LMHmedchem
2 Replies

3. Shell Programming and Scripting

Inserting blank columns in already present CSV file

Hi, i have a csv file which have headers and values of it like below : headers --> CI Ref SerialNumber LastScanDate values --> VMware-42,VMware-42,Tue, 20 May 2014 11:03:44 +0000 i want to have a above csv in below format : headers --> CI Name CI Description CI Ref... (6 Replies)
Discussion started by: omkar.jadhav
6 Replies

4. Shell Programming and Scripting

How to sort a text file if certain columns are blank?

Dear all, I am trying to sort a text file based on column 3, 10, 11 and 12. But certain column are blank for some lines. Column 3 has to be in ascending order after sorting. Part of my input file is as follows: CN727990 1 A01 4703 5083 73.28 - A_scaffold000011 4365605 4365985 73.28 +... (10 Replies)
Discussion started by: huiyee1
10 Replies

5. Shell Programming and Scripting

Removing blank lines from a file

Hi All, How do i remove continuos blank lines from a file. I have a file with data: abc; def; ghi; jkl; mno; pqr; In the above file, there are two blank lines. I want to remove, one out of them. My output should look like: (2 Replies)
Discussion started by: raosr020
2 Replies

6. Shell Programming and Scripting

Removing blank space in file

TT0000013101257 | JCJMMUJMMUB018 ... (0 Replies)
Discussion started by: sususa
0 Replies

7. UNIX for Dummies Questions & Answers

Removing columns from a text file that do not have any values in second and third columns

I have a text file that has three columns. But at the end of the text file, there are trailing lines that have missing second and third columns: 4 0.04972604 KLHL28 4 0.0497332 CSTB 4 0.04979822 AIF1 4 0.04983331 DECR2 4 0.04990344 KATNB1 4 4 4 4 How can I remove the trailing... (3 Replies)
Discussion started by: evelibertine
3 Replies

8. Shell Programming and Scripting

Extract columns from a file if the name dont exist put blank

Hi, I am very new to Unix script. Suppose i have a file with column header: NAME1 NAME2 Address Tel And I always need to make a file with column header: ID NAME1 NAME2 EMail Address Tel For the columns that do not exist in the file, I would still like to make a column with blank. ... (11 Replies)
Discussion started by: nightrider
11 Replies

9. UNIX for Dummies Questions & Answers

Removing blank lines in a file

Hi I have a text file that has blank lines at different places. How to remove all the blank lines in a file? Thanks Ashok (3 Replies)
Discussion started by: ashok.k
3 Replies

10. Shell Programming and Scripting

Removing blank spaces from a file?

Guys, I need some help... how can I remove the blank spaces between the lines below? (between the date and the hour fields) 21/05/07 00:05:00 99 21/05/07 00:10:01 99 21/05/07 00:15:00 99 21/05/07 00:20:00 99 21/05/07 00:25:00 99 I want to make the file... (4 Replies)
Discussion started by: dfs
4 Replies
Login or Register to Ask a Question