Remove nullable columns in lines


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove nullable columns in lines
# 8  
Old 10-03-2013
Quote:
Originally Posted by RavinderSingh13
Hello CarloM,

Could you please explain this.
{ - for any input line.
line=""; - reset our output variable 'line' to empty.
for (i=1;i<=NF;i++) { - loop around the fields.
if ($i) { - if this field is not empty.
line = line ? line OFS $i : $i - if the 'line' variable is already set to something, append OFS (comma) and the current field to it. Otherwise just set it to the current field (this is so we don't get an extra OFS at the start of the line).
}
}
print line - print the string we just constructed.
} FS=, OFS=, file - set field separator & output field separators to ',', read input file 'file'.

---------- Post updated at 21:41 ---------- Previous update was at 21:16 ----------

Quote:
Originally Posted by skpshell
In the above code i want to remove ' , 'only on the columns mentioned below
If you want to restrict it by columns for certain first fields, you could do something like:
Code:
$ cat x.awk
$1 ~ /^A$/ {
   line="";
   for (i=1;i<=NF;i++) {
      if ($i || (! ((i >=6) && (i <= 29)))) {
         line = line ? line OFS $i : $i
      }
   }
   print line
}
$1 ~ /^B1$/ {
   line="";
   for (i=1;i<=NF;i++) {
      if ($i || (! ( ((i>=2) && (i<=5)) || ((i>=17) && (i<=29))))) {
         line = line ? line OFS $i : $i
      }
   }
   print line
}

$ awk -f x.awk FS=, OFS=, file
A,10,20,30,40,50,60
B1,,12,24,36,48,54,58,68,,,rak,,,,,
B1,,12,24,36,48,54,58,68,,,rak,
B1,,12,24,36,48,54,58,68,,,rak,
B1,,12,24,36,48,54,58,68,,,rak,

and similar for B2 and C.
# 9  
Old 10-04-2013
Hi Carol, Thanks a lot, but how do i remove the ',' at end of the line, and is there
any other way to do this.

Can some one please help ??????

Last edited by skpshell; 10-04-2013 at 02:53 AM..
# 10  
Old 10-04-2013
There are trailing commas because that's what you asked for - there are more than 29 fields in those lines. If you want to remove more then just change the field numbers the script is checking for.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times... (9 Replies)
Discussion started by: RalphNY
9 Replies

3. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

4. Shell Programming and Scripting

Remove lines with unique information in indicated columns

Hi, I have the 3-column, tab-separated following data: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to remove all of those lines in which the values of Columns 1 and 2 are identical. In this way, the results would be as follows: dot is-big 2 cat... (4 Replies)
Discussion started by: owwow14
4 Replies

5. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

6. UNIX for Dummies Questions & Answers

remove duplicate lines based on two columns and judging from a third one

hello all, I have an input file with four columns like this with a lot of lines and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies

7. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

8. Shell Programming and Scripting

Single command for add 2 columns and remove 2 columns in unix/performance tuning

Hi all, I have created a script which adding two columns and removing two columns for all files. Filename: Cust_information_1200_201010.txt Source Data: "1","Cust information","123","106001","street","1-203 high street" "1","Cust information","124","105001","street","1-203 high street" ... (0 Replies)
Discussion started by: onesuri
0 Replies

9. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

10. Shell Programming and Scripting

Remove lines with n columns

Hi folks - hope you are all well. I am trying to perform some pre-processing on a data file, to make sure it is in a valid format before performing a data upload. Each row of data in the file should consist of 10 comma delimited fields. Can anyone advise me of a sed/awk command that might... (2 Replies)
Discussion started by: Krispy
2 Replies
Login or Register to Ask a Question