Sponsored Content
Top Forums Shell Programming and Scripting how to delete duplicate rows based on last column Post 302347204 by reva on Tuesday 25th of August 2009 06:21:34 AM
Old 08-25-2009
thanks a lot its working.but first few lines are been deleted in my file...

one more thing for the same data if i need the ouput as

1902 8 22 3 40.0000 77.0000 8.60
1902 9 30 6 38.5000 67.0000 7.70
1902 10 6 9 36.5000 70.5000 7.20
1902 12 4 22 37.8000 65.5000 4.90

that is just check for first 4 columns if its equal & other columns for largest value as shown in above ..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to delete duplicate rows in a file

I have a file content like below. "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Discussion started by: vamshikrishnab
5 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate rows of a file based on a value of a column

Hi, I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g. COL1 COL2 COL3 A 1234 1234 B 3k32 2322 C Xk32 TTT A NEW XX22 B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies

3. UNIX for Dummies Questions & Answers

forming duplicate rows based on value of a key

if the key (A or B or ...others) has 4 in its 3rd column the 1st A row has to form 4 dupicates along with the all the values of A in 4th column (2.9, 3.8, 4.2) . Hope I explain the question clearly. Cheers Ruby input "A" 1 4 2.9 "A" 2 5 ... (7 Replies)
Discussion started by: ruby_sgp
7 Replies

4. Ubuntu

delete duplicate rows with awk files

Hi every body I have some text file with a lots of duplicate rows like this: 165.179.568.197 154.893.836.174 242.473.396.153 165.179.568.197 165.179.568.197 165.179.568.197 154.893.836.174 how can I delete the repeated rows? Thanks Saeideh (2 Replies)
Discussion started by: sashtari
2 Replies

5. UNIX for Dummies Questions & Answers

Remove duplicate rows when >10 based on single column value

Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column. e.g. a 1 a 2 a 3 b 1 c 1 gives b 1 c 1 but requires 11 duplicates before it deletes. Thanks for the help Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies

6. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

7. UNIX for Dummies Questions & Answers

merging rows into new file based on rows and first column

I have 2 files, file01= 7 columns, row unknown (but few) file02= 7 columns, row unknown (but many) now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there e.g. file 01 James|0|50|25|10|50|30... (1 Reply)
Discussion started by: A-V
1 Replies

8. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

9. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies

10. Shell Programming and Scripting

Extract and exclude rows based on duplicate values

Hello I have a file like this: > cat examplefile ghi|NN603762|eee mno|NN607265|ttt pqr|NN613879|yyy stu|NN615002|uuu jkl|NN607265|rrr vwx|NN615002|iii yzA|NN618555|ooo def|NN190486|www BCD|NN628717|ppp abc|NN190486|qqq EFG|NN628717|aaa HIJ|NN628717|sss > I can sort the file by... (5 Replies)
Discussion started by: CHoggarth
5 Replies
funtablerowget(3)						SAORD Documentation						 funtablerowget(3)

NAME
FunTableRowGet - get Funtools rows SYNOPSIS
#include <funtools.h> void *FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, int *nrow) DESCRIPTION
The FunTableRowGet() routine retrieves rows from a Funtools binary table or raw event file, and places the values of columns selected by FunColumnSelect() into an array of user structs. Selected column values are automatically converted to the specified user data type (and to native data format) as necessary. The first argument is the Fun handle associated with this row data. The second rows argument is the array of user structs into which the selected columns will be stored. If NULL is passed, the routine will automatically allocate space for this array. (This includes proper allocation of pointers within each struct, if the "@" pointer type is used in the selection of columns. Note that if you pass NULL in the second argument, you should free this space using the standard free() system call when you are finished with the array of rows.) The third maxrow argument specifies the maximum number of rows to be returned. Thus, if rows is allocated by the user, it should be at least of size maxrow*sizeof(evstruct). The fourth plist argument is a param list string. Currently, the keyword/value pair "mask=transparent" is supported in the plist argument. If this string is passed in the call's plist argument, then all rows are passed back to the user (instead of just rows passing the filter). This is only useful when FunColumnSelect() also is used to specify "$region" as a column to return for each row. In such a case, rows found within a region have a returned region value greater than 0 (corresponding to the region id of the region in which they are located), rows passing the filter but not in a region have region value of -1, and rows not passing any filter have region value of 0. Thus, using "mask=transparent" and the returned region value, a program can process all rows and decide on an action based on whether a given row passed the filter or not. The final argument is a pointer to an int variable that will return the actual number of rows returned. The routine returns a pointer to the array of stored rows, or NULL if there was an error. (This pointer will be the same as the second argument, if the latter is non-NULL). /* get rows -- let routine allocate the row array */ while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ /* process all rows */ for(i=0; i<got; i++){ /* point to the i'th row */ ev = buf+i; /* rearrange some values. etc. */ ev->energy = (ev->pi+ev->pha)/2.0; ev->pha = -ev->pha; ev->pi = -ev->pi; } /* write out this batch of rows */ FunTableRowPut(fun2, buf, got, 0, NULL); /* free row data */ if( buf ) free(buf); } As shown above, successive calls to FunTableRowGet() will return the next set of rows from the input file until all rows have been read, i.e., the routine behaves like sequential Unix I/O calls such as fread(). See evmerge example code for a more complete example. Note that FunTableRowGet() also can be called as FunEventsGet(), for backward compatibility. SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funtablerowget(3)
All times are GMT -4. The time now is 04:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy