Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Using 'sed' to delete or ignore columns in a dataset

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Using 'sed' to delete or ignore columns in a dataset
# 1  
Old 02-29-2008
Using 'sed' to delete or ignore columns in a dataset
Hi,

I've already posted elsewhere but am posting again here coz im a newbie. I hope you forgive me this time.

I want to know if its possible to delete or ignore columns in a large dataset using 'sed'. For example, I have the following dataset: -

20060714,X.XX,1,043004,Q,T,24.0000,1,25.5000,4,
20060714,X.XX,1,081209,Q,T,24.0000,1,25.5000,5,

As you can see, there are 10 columns here and the table that I am inserting into has 8 columns.

I want to delete the 3rd column (i.e. the 1's) and I want to delete the comma between Q and T. Finally I want to delete the comma at the end.

Is this possible with sed? Can any1 help me with this please?

I'll be extremely grateful if someone can help with this!! You can PM me or post back here.

Many Thanks, asif.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ignore dollar value in sed

Hi Guys, I need to replace the string based on specific value by keeping dollar sign input=$1 var=$(echo "@code_temp_table_$value_table"| sed -r "s/\@code/${input}/;s/(nz|sa)_\$value_/\$value1_\1_/" ) Expected if input=nz,sa then nz_temp_table_$value1_table else if any other... (5 Replies)
Discussion started by: Master_Mind
5 Replies

2. Shell Programming and Scripting

Selecting random columns from large dataset in UNIX

Dear folks I have a large data set which contains 400K columns. I decide to select 50K determined columns from the whole 400K columns. Is there any command in unix which could do this process for me? I need to also mention that I store all of the columns id in one file which may help to select... (5 Replies)
Discussion started by: sajmar
5 Replies

3. Shell Programming and Scripting

Ignore escape sequence in sed

Friends, In the file i am having more then 100 lines like, File1 had the values like this: #Example East.server_01=EAST.SERVER_01 East.server_01=EAST.SERVER_01 West.server_01=WEST.SERVER_01 File2 had the values like this: #Example EAST.SERVER_01=http://yahoo.com... (3 Replies)
Discussion started by: jothi basu
3 Replies

4. UNIX for Dummies Questions & Answers

Sed: delete columns 7,15,16

An extension from an earlier question. Now need a sed script to delete columns 7,15 and 16 from an example txt below.. Again, thanks in advance. 98M-01.WAV,98M,01,00:00:49,01:07:36:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", 98L-01.WAV,98L,01,00:00:51,01:01:45:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", (7 Replies)
Discussion started by: Vrc2250
7 Replies

5. Shell Programming and Scripting

awk based script to ignore all columns from a file which contains character strings

Hello All, I have a .CSV file where I expect all numeric data in all the columns other than column headers. But sometimes I get the files (result of statistics computation by other persons) like below( sample data) SNO,Data1,Data2,Data3 1,2,3,4 2,3,4,SOME STRING 3,4,Inf,5 4,5,4,4 I... (9 Replies)
Discussion started by: ks_reddy
9 Replies

6. Solaris

flarecreate for zfs root dataset and ignore multiple dataset

Hi All, I want to write a script to create flar images on multiple servers. In non zfs filesystem I am using -X option to refer a file to exclude mounts on different servers. but on ZFS -X option is not working. I want multiple mounts to be ignore on ZFS base system during flarecreate. I... (0 Replies)
Discussion started by: uxravi
0 Replies

7. Shell Programming and Scripting

Ignore first word using sed in PERL

Please help me in ignoring first word in a line example Input log 123^Babd^Basdf789^B098^Bouiou Desired output abd,asdf789,098,ouiou 123 should be ignored is this possible using sed regular expressions Use code tags - you got a PM with a guide. (2 Replies)
Discussion started by: thankful123
2 Replies

8. Programming

Extracting differences between two columns dataset (SQL command)

Hi, I have a table in my sqlite, here is an example (tab separated) 585 name1 chr1 + 1872 3533 3533 3533 6 1872,2041,2475,2837,3083,3315, 1920,2090,2560,2915,3237,3533, name2 The 10th and 11th columns have information in a comma separated format (not tab).... (0 Replies)
Discussion started by: labrazil
0 Replies

9. UNIX for Dummies Questions & Answers

Using 'sed' to delete or ignore columns in a dataset

Hi, I want to know if its possible to delete or ignore columns in a large dataset using 'sed'. For example, I have the following dataset: - 20060714,X.XX,1,043004,Q,T,24.0000,1,25.5000,4, 20060714,X.XX,1,081209,Q,T,24.0000,1,25.5000,5, As you can see, there are 10 columns here and the... (4 Replies)
Discussion started by: aarif
4 Replies

10. Shell Programming and Scripting

Make sed ignore lines

Hi I use sed in a script for severall changes in files. I whish one of the substitutions I made to be aplied to every line that has the word "scripts" with the exception for the ones that start with "rsh", wich I wish sed to ignore . Is this possible? If yes, how can I do it? The substitution... (2 Replies)
Discussion started by: Scarlos
2 Replies
Login or Register to Ask a Question

LEARN ABOUT DEBIAN

h5fromtxt

H5FROMTXT(1)							      h5utils							      H5FROMTXT(1)

NAME

       h5fromtxt - convert text input to an HDF5 file

SYNOPSIS

       h5fromtxt [OPTION]... [HDF5FILE]

DESCRIPTION

       h5fromtxt takes a series of numbers from standard input and outputs a multi-dimensional numeric dataset in an HDF5 file.

       HDF5  is a free, portable binary format and supporting library developed by the National Center for Supercomputing Applications at the Uni-
       versity of Illinois in Urbana-Champaign.  A single h5 file can contain multiple data sets; by default, h5fromtxt creates a  dataset  called
       "data",	but  this  can	be  changed  via  the -d option, or by using the syntax HDF5FILE:DATASET.  The -a option can be used to append new
       datasets to an existing HDF5 file.

       All characters besides the numbers (and associated decimal points, etcetera) in the input are ignored.  By default, the data is assumed	to
       be a two-dimensional MxN dataset where M is the number of rows (delimited by newlines) and N is the number of columns.  In this case, it is
       an error for the number of columns to vary between rows.  If M or N is 1 then the data is written as a one-dimensional dataset.

       Alternatively, you can specify the dimensions of the data explicitly via the -n size option, where size is e.g.	"2x2x2".   In  this  case,
       newlines  are ignored and the data is taken as an array of the given size stored in row-major ("C") order (where the last index varies most
       quickly as you step through the data).  e.g. a 2x2x2 array would be have the elements listed  in  the  order:  (0,0,0),	(0,0,1),  (0,1,0),
       (0,1,1), (1,0,0), (1,0,1), (1,1,0), (1,1,1).

       A simple example is:

	   h5fromtxt foo.h5 <<EOF
	   1 2 3 4
	   5 6 7 8
	   EOF

       which reads in a 2x4 space-delimited array from standard input.

OPTIONS

       -h     Display help on the command-line options and usage.

       -V     Print the version number and copyright info for h5fromtxt.

       -v     Verbose output.

       -a     If  the  HDF5  output file already exists, append the data as a new dataset rather than overwriting the file (the default behavior).
	      An existing dataset of the same name within the file is overwritten, however.

       -n size
	      Instead of trying to infer the dimensions of the array from the rows and columns of the input, treat the data as a sequence of  num-
	      bers  in row-major order forming an array of dimensions size.  size is of the form MxNxLx... (with M, N, L being numbers) and may be
	      of any dimensionality.

       -T     Transpose the input when it is written, reversing the dimensions.

       -d name
	      Write to dataset name in the output; otherwise, the output dataset is called "data"  by  default.   Alternatively,  use  the  syntax
	      HDF5FILE:DATASET.

BUGS

       Send bug reports to S. G. Johnson, stevenj@alum.mit.edu.

AUTHORS

       Written by Steven G. Johnson.  Copyright (c) 2005 by the Massachusetts Institute of Technology.

h5utils 							   March 9, 2002						      H5FROMTXT(1)