Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove nullable columns in lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove nullable columns in lines
# 1  
Old 10-03-2013
Remove nullable columns in lines
Hi Every one, my requirement is to remove the null columns in line, comma delimiter used
For example,

Code:
A,11,20,30,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

B1,,,,,, gem,plum,kite,,,,gud,bad,,,,,,,,,,,,,
B2,kiing,kong,height,,,,,,,,,,,,,,,,,,,,,,,,,rak,,,,,,,,,,,,,

B1,,,,,, gem,plum,kite,,,,gud,bad,,,,,,,,,,,,,
B2,kiing,kong,height,,,,,,,,,,,,,,,,,,,,,,,,,rak,,,,,,,,,,,,,

C,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,20

Here A and C are constant, B1 and B2 lines will be repeated many times, In line A i need to remove all the ',' after the value 30 and in line
B1 we have 5 nulls before value'Gem' i need to remove that and also remove all the ',' after rak, same for B2 and C

Logic like foreach line get A (iF A SH then Cut $1 to $5), can some one help me to fix this.

---------- Post updated at 01:50 PM ---------- Previous update was at 01:17 PM ----------

can some one help please !!!!
Last edited by joeyg; 10-03-2013 at 03:53 PM.. Reason: Wrap data and commands in CodeTags
# 2  
Old 10-03-2013
Bumping up posts or double posting is not permitted in these forums.

Please read the rules, which you agreed to when you registered, if you have not already done so.

You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future

Thank You.

The UNIX and Linux Forums.
# 3  
Old 10-03-2013
Sorry for that, could you please help me !!
# 4  
Old 10-03-2013
I don't understand the logics. There's no rak in line B1, nor in line C. If you want to remove commas before spaces or EOL, this won't work for C.

To remove ALL multiple commas, rty
Code:
sed -r 's/,+/,/g' file

# 5  
Old 10-03-2013
Code:
$ awk '{ line=""; for (i=1;i<=NF;i++) { if ($i) { { line = line ? line OFS $i : $i } } } print line }' FS=, OFS=, file
A,11,20,30

B1, gem,plum,kite,gud,bad
B2,kiing,kong,height,rak

B1, gem,plum,kite,gud,bad
B2,kiing,kong,height,rak

C,20

(which seems rather inelegant to me, but works)

EDIT: Note that this won't handle quoted fields which contain commas.
Last edited by CarloM; 10-03-2013 at 04:19 PM.. Reason: Caveat
This User Gave Thanks to CarloM For This Post:
RavinderSingh13
# 6  
Old 10-03-2013
Quote:
Code:
$ awk '{ line=""; for (i=1;i<=NF;i++) { if ($i) { { line = line ? line OFS $i : $i } } } print line }' FS=, OFS=, fileA,11,20,30B1, gem,plum,kite,gud,badB2,kiing,kong,height,rakB1, gem,plum,kite,gud,badB2,kiing,kong,height,rakC,20

(which seems rather inelegant to me, but works)

EDIT: Note that this won't handle quoted fields which contain commas.

Hello CarloM,

Could you please explain this.



Thanks,
R. Singh
# 7  
Old 10-03-2013
Thanks a lot for all your kind replies, What i wanted is
Code:
A,10,20,30,40,50,60,,,,,,,,,,,,,,,,,,,,,,
B1,,,,,,12,24,36,48,54,58,68,,,rak,,,,,,,,,,,,,,,,,,
B2,,,,,,,,,,,,,,,11,12,19,12,21,22,,,15,,17,13,,

B1,,,,,,12,24,36,48,54,58,68,,,rak,,,,,,,,,,,,,,
B2,,,,,,,,,,,,,,,11,12,19,12,21,22,,,15,,17,13,,

B1,,,,,,12,24,36,48,54,58,68,,,rak,,,,,,,,,,,,,,
B2,,,,,,,,,,,,,,,11,12,19,12,21,22,,,15,,17,13,,

B1,,,,,,12,24,36,48,54,58,68,,,rak,,,,,,,,,,,,,,
B2,,,,,,,,,,,,,,,11,12,19,12,21,22,,,15,,17,13,,

c,,,,,,,,,,,,,,,,,,,,,,,,,,,,90

In the above code i want to remove ' , 'only on the columns mentioned below
A- 6 to 29 (From 6th column to 29th column)
B1- 2 to 5 and 17 to 29 (From 2-5 column and 17- 29th column)
B2- 2 to 16 and 29 (From 2-16 column and 29th column)
C - 2 to 28 (From 2nd column to 28th column)

A line and C line will come only once in file, but b1 and b2 may repeat many time.Thanks again in advance.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times... (9 Replies)
Discussion started by: RalphNY
9 Replies

3. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

4. Shell Programming and Scripting

Remove lines with unique information in indicated columns

Hi, I have the 3-column, tab-separated following data: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to remove all of those lines in which the values of Columns 1 and 2 are identical. In this way, the results would be as follows: dot is-big 2 cat... (4 Replies)
Discussion started by: owwow14
4 Replies

5. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

6. UNIX for Dummies Questions & Answers

remove duplicate lines based on two columns and judging from a third one

hello all, I have an input file with four columns like this with a lot of lines and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies

7. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

8. Shell Programming and Scripting

Single command for add 2 columns and remove 2 columns in unix/performance tuning

Hi all, I have created a script which adding two columns and removing two columns for all files. Filename: Cust_information_1200_201010.txt Source Data: "1","Cust information","123","106001","street","1-203 high street" "1","Cust information","124","105001","street","1-203 high street" ... (0 Replies)
Discussion started by: onesuri
0 Replies

9. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

10. Shell Programming and Scripting

Remove lines with n columns

Hi folks - hope you are all well. I am trying to perform some pre-processing on a data file, to make sure it is in a valid format before performing a data upload. Each row of data in the file should consist of 10 comma delimited fields. Can anyone advise me of a sed/awk command that might... (2 Replies)
Discussion started by: Krispy
2 Replies
Login or Register to Ask a Question