Sponsored Content
Full Discussion: deleting columns with NAs
Top Forums Shell Programming and Scripting deleting columns with NAs Post 302652195 by johnkim0806 on Wednesday 6th of June 2012 04:58:40 PM
Old 06-06-2012
deleting columns with NAs

I want to be able to delete columns whose data have more than 10 percent of NAs.
Code:
x1 x2 x3 x4
1 1 1 1
2 NA 2 2
1 2 1 NA
1 2 1 NA
NA 2 1 NA
1 2 1 NA
1 2 1 NA
1 2 1 NA
1 2 1 NA
1 2 1 NA
1 2 1 NA
1 2 1 NA
1 2 1 NA

so in this case i will delete x4.
lets say there are 100 tables with 500 columns each.
So I want to write 2 loops to extract and delete the columns with NAs.
Can you give me a hand?

Thanks Smilie

Last edited by Franklin52; 06-07-2012 at 03:58 AM.. Reason: Please use code tags for data and code samples
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting specific columns from a file

Hi Friends, I want to delete specific columns from a file. Say my file content is as follows: "1","a","ww1",1234" "2","b","wwr3","2222" "3","c","erre","3333" Now i want to delete the column 2 and 4 from this file. That is I want the file content to be: "1","ww1" "2","wwr3"... (11 Replies)
Discussion started by: premar
11 Replies

2. Shell Programming and Scripting

Deleting columns from CSV file

Hi All, Am working on perl script which should delete columns in existing CSV file. If my file is : AA,BB,CC,DD 00,11,22,33 00,55,66,77 00,99,88,21 AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC... Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies

3. Shell Programming and Scripting

deleting rows that dont have a certain # of columns

Hi, I want to delete rows that dont have a certain # of columns. In my case, rows that are less than 8 should be removed (those greater than 8 are ok). For instance: 1 2 3 4 5 6 7 8 2 3 2 4 3 2 1 5 1 2 3 4 5 6 8 2 2 4 3 1 1 1 1 1 1 1 1 1 after: 1... (8 Replies)
Discussion started by: gisele_l
8 Replies

4. Shell Programming and Scripting

Deleting columns by list or file

Dear specialists out there, please help a poor awk newbie: I have a very huge file to process consisting of 300000 columns and 1500 rows. About 20000 columns shall be deleted from that file. So it is clear, that I can't do this by writing down all the columns in an awk command like $1, $x etc.... (5 Replies)
Discussion started by: flxms
5 Replies

5. UNIX for Dummies Questions & Answers

Deleting all rows with empty columns

I have a text file that looks like this: 1 rs523634 8.22486 1 1 rs585160 8.22488 1 rs497228 8.2249 1 1 rs600933 8.225 1 rs480106 8.22531 1 rs600199 8.22533 1 rs529015 8.22534 1 rs598894 8.22534 I want to delete the rows with empty... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

7. Shell Programming and Scripting

Deleting columns passed as arguments to the script

Hi all, I am trying to delete columns in a file using a script. The columns that need to be deleted are passed as arguments to the script. The script should look like this > delete_columns.sh <file_name.txt> <column_numbers_to_be_deleted> The contents of the file_name.txt will be like ... (5 Replies)
Discussion started by: VNR
5 Replies

8. Shell Programming and Scripting

How to use awk for deleting columns?

Hi folks, How awk 'll help to do this file contains bunch of insert statement like below remove fourth column from all statement put it in new file INSERT INTO `tbl_medicalquestions` VALUES (1,'Is anyone waiting for an operation, post operative check up, any other hospital treatment or... (12 Replies)
Discussion started by: ashutoshverma
12 Replies

9. Shell Programming and Scripting

Deleting specific columns

Hi group, Can you please tell how to delete specific columns from a file. I know something like awk -F, '{ print $1" "$2" "15 }' input.txt > output.txt will delete all other columns. But this is in a way to copy some particular columns. But is there any other way to select just some... (11 Replies)
Discussion started by: smitra
11 Replies

10. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies
DELETE(7)							   SQL Commands 							 DELETE(7)

NAME
DELETE - delete rows of a table SYNOPSIS
DELETE FROM [ ONLY ] table [ [ AS ] alias ] [ USING usinglist ] [ WHERE condition | WHERE CURRENT OF cursor_name ] [ RETURNING * | output_expression [ [ AS ] output_name ] [, ...] ] DESCRIPTION
DELETE deletes rows that satisfy the WHERE clause from the specified table. If the WHERE clause is absent, the effect is to delete all rows in the table. The result is a valid, but empty table. Tip: TRUNCATE [truncate(7)] is a PostgreSQL extension that provides a faster mechanism to remove all rows from a table. By default, DELETE will delete rows in the specified table and all its child tables. If you wish to delete only from the specific table mentioned, you must use the ONLY clause. There are two ways to delete rows in a table using information contained in other tables in the database: using sub-selects, or specifying additional tables in the USING clause. Which technique is more appropriate depends on the specific circumstances. The optional RETURNING clause causes DELETE to compute and return value(s) based on each row actually deleted. Any expression using the table's columns, and/or columns of other tables mentioned in USING, can be computed. The syntax of the RETURNING list is identical to that of the output list of SELECT. You must have the DELETE privilege on the table to delete from it, as well as the SELECT privilege for any table in the USING clause or whose values are read in the condition. PARAMETERS
ONLY If specified, delete rows from the named table only. When not specified, any tables inheriting from the named table are also pro- cessed. table The name (optionally schema-qualified) of an existing table. alias A substitute name for the target table. When an alias is provided, it completely hides the actual name of the table. For example, given DELETE FROM foo AS f, the remainder of the DELETE statement must refer to this table as f not foo. usinglist A list of table expressions, allowing columns from other tables to appear in the WHERE condition. This is similar to the list of tables that can be specified in the FROM Clause [select(7)] of a SELECT statement; for example, an alias for the table name can be specified. Do not repeat the target table in the usinglist, unless you wish to set up a self-join. condition An expression that returns a value of type boolean. Only rows for which this expression returns true will be deleted. cursor_name The name of the cursor to use in a WHERE CURRENT OF condition. The row to be deleted is the one most recently fetched from this cur- sor. The cursor must be a non-grouping query on the DELETE's target table. Note that WHERE CURRENT OF cannot be specified together with a Boolean condition. See DECLARE [declare(7)] for more information about using cursors with WHERE CURRENT OF. output_expression An expression to be computed and returned by the DELETE command after each row is deleted. The expression can use any column names of the table or table(s) listed in USING. Write * to return all columns. output_name A name to use for a returned column. OUTPUTS
On successful completion, a DELETE command returns a command tag of the form DELETE count The count is the number of rows deleted. If count is 0, no rows matched the condition (this is not considered an error). If the DELETE command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and val- ues defined in the RETURNING list, computed over the row(s) deleted by the command. NOTES
PostgreSQL lets you reference columns of other tables in the WHERE condition by specifying the other tables in the USING clause. For exam- ple, to delete all films produced by a given producer, one can do: DELETE FROM films USING producers WHERE producer_id = producers.id AND producers.name = 'foo'; What is essentially happening here is a join between films and producers, with all successfully joined films rows being marked for dele- tion. This syntax is not standard. A more standard way to do it is: DELETE FROM films WHERE producer_id IN (SELECT id FROM producers WHERE name = 'foo'); In some cases the join style is easier to write or faster to execute than the sub-select style. EXAMPLES
Delete all films but musicals: DELETE FROM films WHERE kind <> 'Musical'; Clear the table films: DELETE FROM films; Delete completed tasks, returning full details of the deleted rows: DELETE FROM tasks WHERE status = 'DONE' RETURNING *; Delete the row of tasks on which the cursor c_tasks is currently positioned: DELETE FROM tasks WHERE CURRENT OF c_tasks; COMPATIBILITY
This command conforms to the SQL standard, except that the USING and RETURNING clauses are PostgreSQL extensions. SQL - Language Statements 2010-05-14 DELETE(7)
All times are GMT -4. The time now is 09:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy