Sponsored Content
Top Forums Shell Programming and Scripting how to delete duplicate rows based on last column Post 302347274 by reva on Wednesday 26th of August 2009 05:45:17 AM
Old 08-26-2009
ya i will follow from next post...

---------- Post updated 08-26-09 at 04:34 AM ---------- Previous update was 08-25-09 at 08:49 AM ----------

Thanks for the help i got it...

---------- Post updated at 04:45 AM ---------- Previous update was at 04:34 AM ----------

Hiii
now if i have data like shown below.how to sort it out. i mean delete duplicate entries in such a way that it must take the largest value in last column & it must choose a row which has many sets of values in the row.
For example the data in my file is
HTML Code:
1900  2  7  0   9.5000  76.5000 0.00 4.30 0.00 0.00 0.00 4.30
1900  2  7  0  10.8000  76.8000 0.00 6.00 0.00 0.00 0.00 6.00
1901 12  1  0  37.8000  66.0000 0.00 5.00 0.00 0.00 0.00 5.00
1901 12  1  0  37.8000  66.0000 0.00 4.60 3.00 3.50 3.50 4.60
1902  4 17 21  40.0000  71.0000 0.00 5.80 0.00 5.90 5.70 5.90
1902  8 12 17  39.5000  68.5000 0.00 6.00 0.00 6.20 5.90 6.20
1902  8 22  3  40.0000  77.0000 0.00 0.00 0.00 8.00 8.60 8.60
1902  8 22  3  40.0000  76.5000 0.00 0.00 0.00 0.00 8.20 8.20
1902  8 22  3  40.0000  76.5000 0.00 0.00 0.00 0.00 8.30 8.30
1903  5 16  6   5.3600  80.0000 0.00 4.50 0.00 5.00 0.00 5.00
1903  5 16  6   5.3600  80.0000 0.00 4.30 0.00 3.00 0.00 4.30
The output for it is
HTML Code:
1900  2  7  0  10.8000  76.8000 0.00 6.00 0.00 0.00 0.00 6.00
1901 12  1  0  37.8000  66.0000 0.00 4.60 3.00 0.00 3.50 4.60
1902  4 17 21  40.0000  71.0000 0.00 5.80 0.00 5.90 5.70 5.90
1902  8 12 17  39.5000  68.5000 0.00 6.00 0.00 6.20 5.90 6.20
1902  8 22  3  40.0000  77.0000 0.00 0.00 0.00 8.00 8.60 8.60
1903  5 16  6   5.3600  80.0000 0.00 4.50 0.00 5.00 0.00 5.00
Here it removes duplicates & checks for longest row with many values & largest value in last column.
If any one has an idea help me out..

Last edited by reva; 08-26-2009 at 12:06 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to delete duplicate rows in a file

I have a file content like below. "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Discussion started by: vamshikrishnab
5 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate rows of a file based on a value of a column

Hi, I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g. COL1 COL2 COL3 A 1234 1234 B 3k32 2322 C Xk32 TTT A NEW XX22 B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies

3. UNIX for Dummies Questions & Answers

forming duplicate rows based on value of a key

if the key (A or B or ...others) has 4 in its 3rd column the 1st A row has to form 4 dupicates along with the all the values of A in 4th column (2.9, 3.8, 4.2) . Hope I explain the question clearly. Cheers Ruby input "A" 1 4 2.9 "A" 2 5 ... (7 Replies)
Discussion started by: ruby_sgp
7 Replies

4. Ubuntu

delete duplicate rows with awk files

Hi every body I have some text file with a lots of duplicate rows like this: 165.179.568.197 154.893.836.174 242.473.396.153 165.179.568.197 165.179.568.197 165.179.568.197 154.893.836.174 how can I delete the repeated rows? Thanks Saeideh (2 Replies)
Discussion started by: sashtari
2 Replies

5. UNIX for Dummies Questions & Answers

Remove duplicate rows when >10 based on single column value

Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column. e.g. a 1 a 2 a 3 b 1 c 1 gives b 1 c 1 but requires 11 duplicates before it deletes. Thanks for the help Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies

6. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

7. UNIX for Dummies Questions & Answers

merging rows into new file based on rows and first column

I have 2 files, file01= 7 columns, row unknown (but few) file02= 7 columns, row unknown (but many) now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there e.g. file 01 James|0|50|25|10|50|30... (1 Reply)
Discussion started by: A-V
1 Replies

8. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

9. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies

10. Shell Programming and Scripting

Extract and exclude rows based on duplicate values

Hello I have a file like this: > cat examplefile ghi|NN603762|eee mno|NN607265|ttt pqr|NN613879|yyy stu|NN615002|uuu jkl|NN607265|rrr vwx|NN615002|iii yzA|NN618555|ooo def|NN190486|www BCD|NN628717|ppp abc|NN190486|qqq EFG|NN628717|aaa HIJ|NN628717|sss > I can sort the file by... (5 Replies)
Discussion started by: CHoggarth
5 Replies
DELETE(7)							   SQL Commands 							 DELETE(7)

NAME
DELETE - delete rows of a table SYNOPSIS
DELETE FROM [ ONLY ] table [ [ AS ] alias ] [ USING usinglist ] [ WHERE condition | WHERE CURRENT OF cursor_name ] [ RETURNING * | output_expression [ [ AS ] output_name ] [, ...] ] DESCRIPTION
DELETE deletes rows that satisfy the WHERE clause from the specified table. If the WHERE clause is absent, the effect is to delete all rows in the table. The result is a valid, but empty table. Tip: TRUNCATE [truncate(7)] is a PostgreSQL extension that provides a faster mechanism to remove all rows from a table. By default, DELETE will delete rows in the specified table and all its child tables. If you wish to delete only from the specific table mentioned, you must use the ONLY clause. There are two ways to delete rows in a table using information contained in other tables in the database: using sub-selects, or specifying additional tables in the USING clause. Which technique is more appropriate depends on the specific circumstances. The optional RETURNING clause causes DELETE to compute and return value(s) based on each row actually deleted. Any expression using the table's columns, and/or columns of other tables mentioned in USING, can be computed. The syntax of the RETURNING list is identical to that of the output list of SELECT. You must have the DELETE privilege on the table to delete from it, as well as the SELECT privilege for any table in the USING clause or whose values are read in the condition. PARAMETERS
ONLY If specified, delete rows from the named table only. When not specified, any tables inheriting from the named table are also pro- cessed. table The name (optionally schema-qualified) of an existing table. alias A substitute name for the target table. When an alias is provided, it completely hides the actual name of the table. For example, given DELETE FROM foo AS f, the remainder of the DELETE statement must refer to this table as f not foo. usinglist A list of table expressions, allowing columns from other tables to appear in the WHERE condition. This is similar to the list of tables that can be specified in the FROM Clause [select(7)] of a SELECT statement; for example, an alias for the table name can be specified. Do not repeat the target table in the usinglist, unless you wish to set up a self-join. condition An expression that returns a value of type boolean. Only rows for which this expression returns true will be deleted. cursor_name The name of the cursor to use in a WHERE CURRENT OF condition. The row to be deleted is the one most recently fetched from this cur- sor. The cursor must be a non-grouping query on the DELETE's target table. Note that WHERE CURRENT OF cannot be specified together with a Boolean condition. See DECLARE [declare(7)] for more information about using cursors with WHERE CURRENT OF. output_expression An expression to be computed and returned by the DELETE command after each row is deleted. The expression can use any column names of the table or table(s) listed in USING. Write * to return all columns. output_name A name to use for a returned column. OUTPUTS
On successful completion, a DELETE command returns a command tag of the form DELETE count The count is the number of rows deleted. If count is 0, no rows matched the condition (this is not considered an error). If the DELETE command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and val- ues defined in the RETURNING list, computed over the row(s) deleted by the command. NOTES
PostgreSQL lets you reference columns of other tables in the WHERE condition by specifying the other tables in the USING clause. For exam- ple, to delete all films produced by a given producer, one can do: DELETE FROM films USING producers WHERE producer_id = producers.id AND producers.name = 'foo'; What is essentially happening here is a join between films and producers, with all successfully joined films rows being marked for dele- tion. This syntax is not standard. A more standard way to do it is: DELETE FROM films WHERE producer_id IN (SELECT id FROM producers WHERE name = 'foo'); In some cases the join style is easier to write or faster to execute than the sub-select style. EXAMPLES
Delete all films but musicals: DELETE FROM films WHERE kind <> 'Musical'; Clear the table films: DELETE FROM films; Delete completed tasks, returning full details of the deleted rows: DELETE FROM tasks WHERE status = 'DONE' RETURNING *; Delete the row of tasks on which the cursor c_tasks is currently positioned: DELETE FROM tasks WHERE CURRENT OF c_tasks; COMPATIBILITY
This command conforms to the SQL standard, except that the USING and RETURNING clauses are PostgreSQL extensions. SQL - Language Statements 2010-05-14 DELETE(7)
All times are GMT -4. The time now is 05:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy