Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Using 'sed' to delete or ignore columns in a dataset

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Using 'sed' to delete or ignore columns in a dataset
# 1  
Old 02-29-2008
Using 'sed' to delete or ignore columns in a dataset
Hi,

I want to know if its possible to delete or ignore columns in a large dataset using 'sed'. For example, I have the following dataset: -

20060714,X.XX,1,043004,Q,T,24.0000,1,25.5000,4,
20060714,X.XX,1,081209,Q,T,24.0000,1,25.5000,5,

As you can see, there are 10 columns here and the table that I am inserting into has 8 columns.

I want to delete the 3rd column (i.e. the 1's) and I want to delete the comma between Q and T. Finally I want to delete the comma at the end.

Is this possible with sed? Can any1 help me with this please?

I'll be very grateful!! Many Thanks, asif.
# 2  
Old 02-29-2008
maybe not the most elegant way, but hey it's Friday, and time to call it a day!
Code:
awk -F, '{ OFS=","; $3=""; print }' FILENAME | sed -e 's/,,//g' -e 's/,//3' -e s/.$//

# 3  
Old 02-29-2008
Quote:
Originally Posted by denn
maybe not the most elegant way, but hey it's Friday, and time to call it a day!
Code:
awk -F, '{ OFS=","; $3=""; print }' FILENAME | sed -e 's/,,//g' -e 's/,//3' -e s/.$//


Hey thanks for the reply. The file that I have is quote large (approx 5 million rows). How do i redirect the output or result of this command to a file without modifying this file?

Thank you very much!
# 4  
Old 03-01-2008
This is just a dump way, but it work...

# awk -F, '{print $1","$2","$4","$5$6","$7","$8","$9","$10}' FILENAME > newfile
[root@ganen test]# cat newfile
20060714,X.XX,043004,QT,24.0000,1,25.5000,4
20060714,X.XX,081209,QT,24.0000,1,25.5000,5
# 5  
Old 07-21-2008
I don't know how to post a question. So I am doing this. Please forgive me if I have made any mistake.
My question is : I have a file like
1:0.000003 2:0.000003 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0.000001

I want to remove the nos 1: 2: etc so that the output should look like
0.000003 0.000003 0 0 0 0 0 0 0 0 0.000001

How can I do it using linux?

Partha
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ignore dollar value in sed

Hi Guys, I need to replace the string based on specific value by keeping dollar sign input=$1 var=$(echo "@code_temp_table_$value_table"| sed -r "s/\@code/${input}/;s/(nz|sa)_\$value_/\$value1_\1_/" ) Expected if input=nz,sa then nz_temp_table_$value1_table else if any other... (5 Replies)
Discussion started by: Master_Mind
5 Replies

2. Shell Programming and Scripting

Selecting random columns from large dataset in UNIX

Dear folks I have a large data set which contains 400K columns. I decide to select 50K determined columns from the whole 400K columns. Is there any command in unix which could do this process for me? I need to also mention that I store all of the columns id in one file which may help to select... (5 Replies)
Discussion started by: sajmar
5 Replies

3. Shell Programming and Scripting

Ignore escape sequence in sed

Friends, In the file i am having more then 100 lines like, File1 had the values like this: #Example East.server_01=EAST.SERVER_01 East.server_01=EAST.SERVER_01 West.server_01=WEST.SERVER_01 File2 had the values like this: #Example EAST.SERVER_01=http://yahoo.com... (3 Replies)
Discussion started by: jothi basu
3 Replies

4. UNIX for Dummies Questions & Answers

Sed: delete columns 7,15,16

An extension from an earlier question. Now need a sed script to delete columns 7,15 and 16 from an example txt below.. Again, thanks in advance. 98M-01.WAV,98M,01,00:00:49,01:07:36:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", 98L-01.WAV,98L,01,00:00:51,01:01:45:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", (7 Replies)
Discussion started by: Vrc2250
7 Replies

5. Shell Programming and Scripting

awk based script to ignore all columns from a file which contains character strings

Hello All, I have a .CSV file where I expect all numeric data in all the columns other than column headers. But sometimes I get the files (result of statistics computation by other persons) like below( sample data) SNO,Data1,Data2,Data3 1,2,3,4 2,3,4,SOME STRING 3,4,Inf,5 4,5,4,4 I... (9 Replies)
Discussion started by: ks_reddy
9 Replies

6. Solaris

flarecreate for zfs root dataset and ignore multiple dataset

Hi All, I want to write a script to create flar images on multiple servers. In non zfs filesystem I am using -X option to refer a file to exclude mounts on different servers. but on ZFS -X option is not working. I want multiple mounts to be ignore on ZFS base system during flarecreate. I... (0 Replies)
Discussion started by: uxravi
0 Replies

7. Shell Programming and Scripting

Ignore first word using sed in PERL

Please help me in ignoring first word in a line example Input log 123^Babd^Basdf789^B098^Bouiou Desired output abd,asdf789,098,ouiou 123 should be ignored is this possible using sed regular expressions Use code tags - you got a PM with a guide. (2 Replies)
Discussion started by: thankful123
2 Replies

8. Programming

Extracting differences between two columns dataset (SQL command)

Hi, I have a table in my sqlite, here is an example (tab separated) 585 name1 chr1 + 1872 3533 3533 3533 6 1872,2041,2475,2837,3083,3315, 1920,2090,2560,2915,3237,3533, name2 The 10th and 11th columns have information in a comma separated format (not tab).... (0 Replies)
Discussion started by: labrazil
0 Replies

9. UNIX for Dummies Questions & Answers

Using 'sed' to delete or ignore columns in a dataset

Hi, I've already posted elsewhere but am posting again here coz im a newbie. I hope you forgive me this time. I want to know if its possible to delete or ignore columns in a large dataset using 'sed'. For example, I have the following dataset: - ... (0 Replies)
Discussion started by: aarif
0 Replies

10. Shell Programming and Scripting

Make sed ignore lines

Hi I use sed in a script for severall changes in files. I whish one of the substitutions I made to be aplied to every line that has the word "scripts" with the exception for the ones that start with "rsh", wich I wish sed to ignore . Is this possible? If yes, how can I do it? The substitution... (2 Replies)
Discussion started by: Scarlos
2 Replies
Login or Register to Ask a Question