Sponsored Content
Top Forums Shell Programming and Scripting duplicates lines with one column different Post 302191921 by matrixmadhan on Monday 5th of May 2008 11:57:38 AM
Old 05-05-2008
Quote:
Originally Posted by dhanamurthy
Hi
Also i have one more question
if we are using like this

arr[key]=sprintf("%s %s", arr[key], $NF)
We are creating a map or relationship between the key and the elements.
I would like to do a file processing of nearly 3GB size file.
If thisis the case will there be any memory issues coming out.

Regards
Dhana

dhanumurthy Smilie

actually in awk - associative arrays; you are creating the associativity between key and value
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete lines in a file that have duplicates or derive the lines that aper once

Input: a b b c d d I need: a c I know how to get this (the lines that have duplicates) : b d sort file | uniq -d But i need opossite of this. I have searched the forum and other places as well, but have found solution for everything except this variant of the problem. (3 Replies)
Discussion started by: necroman08
3 Replies

2. Shell Programming and Scripting

How can i delete the duplicates based on one column of a line

I have my data something like this (08/03/2009 22:57:42.414)(:) king aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb (08/03/2009 22:57:42.416)(:) John cccccccccccc cccccvssssssssss baaaaa (08/03/2009 22:57:42.417)(:) Michael ddddddd tststststtststts (08/03/2009 22:57:42.425)(:) Ravi... (11 Replies)
Discussion started by: rdhanek
11 Replies

3. Shell Programming and Scripting

Delete Duplicates on the basis of two column values.

Hi All, i need ti delete two duplicate processss which are running on the same device type (column 1) and port ID (column 2). here is the sample data p1sc1m1 15517 11325 0 01:00:24 ? 0:00 scagntclsx25octtcp 2967 in3v mvmp01 0 8000 N S 969 750@751@752@ p1sc1m1 15519 11325 0 01:00:24 ? ... (5 Replies)
Discussion started by: neeraj617
5 Replies

4. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies

5. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies

6. Shell Programming and Scripting

Find duplicates in column 1 and merge their lines (awk?)

Hi, I have a file (sorted by sort) with 8 tab delimited columns. The first column contains duplicated fields and I need to merge all these identical lines. My input file: comp100002 aaa bbb ccc ddd eee fff ggg comp100003 aba aba aba aba aba aba aba comp100003 fff fff fff fff fff fff fff... (5 Replies)
Discussion started by: falcox
5 Replies

7. Shell Programming and Scripting

Remove duplicates according to their frequency in column

Hi all, I have huge a tab-delimited file with the following format and I want to remove the duplicates according to their frequency based on Column2 and Column3. Column1 Column2 Column3 Column4 Column5 Column6 Column7 1 user1 access1 word word 3 2 2 user2 access2 ... (10 Replies)
Discussion started by: corfuitl
10 Replies

8. Shell Programming and Scripting

Filter first column duplicates

Dear All, I really enjoy your help or suggestion for resolving an issue. Briefly, I have a file like this: a b c a d e f g h k g h x y z If the first column has the same ID, for example a, just remove it. The output should be this: f g h k g h x y z I was thinking to do it... (11 Replies)
Discussion started by: giuliangiuseppe
11 Replies

9. Shell Programming and Scripting

Count and keep duplicates in Column

Hi folks, I've got a csv file called test.csv Column A Column B Apples 1900 Apples 1901 Pears 1902 Pears 1903I want to count and keep duplicates in the first column. Desired output Column A Column B Column C Apples 2 1900 Apples ... (5 Replies)
Discussion started by: pshields1984
5 Replies

10. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies
DB2_FETCH_ASSOC(3)							 1							DB2_FETCH_ASSOC(3)

db2_fetch_assoc - Returns an array, indexed by column name, representing a row in a result set

SYNOPSIS
array db2_fetch_assoc (resource $stmt, [int $row_number = -1]) DESCRIPTION
Returns an array, indexed by column name, representing a row in a result set. PARAMETERS
o $stmt - A valid stmt resource containing a result set. o $row_number - Requests a specific 1-indexed row from the result set. Passing this parameter results in a PHP warning if the result set uses a forward-only cursor. RETURN VALUES
Returns an associative array with column values indexed by the column name representing the next or requested row in the result set. Returns FALSE if there are no rows left in the result set, or if the row requested by $row_number does not exist in the result set. EXAMPLES
Example #1 Iterating through a forward-only cursor If you call db2_fetch_assoc(3) without a specific row number, it automatically retrieves the next row in the result set. <?php $sql = "SELECT id, name, breed, weight FROM animals ORDER BY breed"; $stmt = db2_prepare($conn, $sql); $result = db2_execute($stmt); while ($row = db2_fetch_assoc($stmt)) { printf ("%-5d %-16s %-32s %10s ", $row['ID'], $row['NAME'], $row['BREED'], $row['WEIGHT']); } ?> The above example will output: 0 Pook cat 3.20 5 Rickety Ride goat 9.70 2 Smarty horse 350.00 Example #2 Retrieving specific rows with db2_fetch_assoc(3) from a scrollable cursor If your result set uses a scrollable cursor, you can call db2_fetch_assoc(3) with a specific row number. The following example retrieves every other row in the result set, starting with the second row. <?php $sql = "SELECT id, name, breed, weight FROM animals ORDER BY breed"; $result = db2_exec($stmt, $sql, array('cursor' => DB2_SCROLLABLE)); $i=2; while ($row = db2_fetch_assoc($result, $i)) { printf ("%-5d %-16s %-32s %10s ", $row['ID'], $row['NAME'], $row['BREED'], $row['WEIGHT']); $i = $i + 2; } ?> The above example will output: 0 Pook cat 3.20 5 Rickety Ride goat 9.70 2 Smarty horse 350.00 SEE ALSO
db2_fetch_array(3), db2_fetch_both(3), db2_fetch_object(3), db2_fetch_row(3), db2_result(3). PHP Documentation Group DB2_FETCH_ASSOC(3)
All times are GMT -4. The time now is 09:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy