Hi radoulov,

I think this might be more than just a one liner,eh?


10-a,1000010.0,500010.0,110.0
2-b,1000002.2,500002.2,102.3
9-b,1000009.9,500009.9,110.0
10-b,1000010.0,500010.0,110.1
9-a,1000009.9,500009.9,109.9
8-a,1000008.8,500008.8,108.8
3-b,1000003.3,500003.3,103.4
8-b,1000008.8,500008.8,108.9
7-a,1000007.7,500007.7,107.7
7-b,1000007.7,500007.7,107.8
6-b,1000006.6,500006.6,106.7
3-a,1000003.3,500003.3,103.3
6-a,1000006.6,500006.6,106.6
5-a,1000005.5,500005.5,105.5
5-b,1000005.5,500005.5,105.6
4-a,1000004.4,500004.4,104.4
4-b,1000004.4,500004.4,104.5
2-a,1000002.2,500002.2,102.2
1-a,1000001.1,500001.1,101.1
1-b,1000001.1,500001.1,101.2


Not sure what the output would be exactly.

But the objective would be to create two output files.

OutputFile1 would be the remainder records (i.e. the duplicates removed).
OutputFile2 would be the duplicates.

I apologize for the long winded description, but I am not a programmer.

#####

The objective would be achieved by,
taking fields 2 and 3 from the first record

10-a,1000010.0,500010.0,110.0

and comparing them to fields 2 and 3 from the remainder of the records,
within the range $2-1.1 to $2+1.1 and range $3-1.1 to $3+1.1,
to determine which of the remaining records are to be considered duplicates.

I think the duplicates would be:

9-b,1000009.9,500009.9,110.0
10-b,1000010.0,500010.0,110.1
9-a,1000009.9,500009.9,109.9

because their $2 and $3 fields satisfy the ranges.


So with record

10-a,1000010.0,500010.0,110.0

marked as a keeper and written to OutputFile1,

and with records

9-b,1000009.9,500009.9,110.0
10-b,1000010.0,500010.0,110.1
9-a,1000009.9,500009.9,109.9

identified as duplicates and written to OutputFile2.

These 4 records would now be removed from processing by
possibly creating a new temporary file/list with these 4 records removed.

This would be the new tempory file/list with the 4 records removed.

2-b,1000002.2,500002.2,102.3
8-a,1000008.8,500008.8,108.8
3-b,1000003.3,500003.3,103.4
8-b,1000008.8,500008.8,108.9
7-a,1000007.7,500007.7,107.7
7-b,1000007.7,500007.7,107.8
6-b,1000006.6,500006.6,106.7
3-a,1000003.3,500003.3,103.3
6-a,1000006.6,500006.6,106.6
5-a,1000005.5,500005.5,105.5
5-b,1000005.5,500005.5,105.6
4-a,1000004.4,500004.4,104.4
4-b,1000004.4,500004.4,104.5
2-a,1000002.2,500002.2,102.2
1-a,1000001.1,500001.1,101.1
1-b,1000001.1,500001.1,101.2


Now the "new first record" would be:

2-b,1000002.2,500002.2,102.3

and gets written to OutputFile1.

Compare fields 2 and 3
to fields 2 and 3 from the remainder of the records
in the tempory list,
within the range $2-1.1 to $2+1.1 and range $3-1.1 to $3+1.1,
to determine which of the remaining records are to be considered duplicates.


I think the duplicates would be:

3-b,1000003.3,500003.3,103.4
3-a,1000003.3,500003.3,103.3
2-a,1000002.2,500002.2,102.2
1-a,1000001.1,500001.1,101.1
1-b,1000001.1,500001.1,101.2

and get written to OutputFile2


The new temporary file/list with these 6 records removed would be:

8-a,1000008.8,500008.8,108.8
8-b,1000008.8,500008.8,108.9
7-a,1000007.7,500007.7,107.7
7-b,1000007.7,500007.7,107.8
6-b,1000006.6,500006.6,106.7
6-a,1000006.6,500006.6,106.6
5-a,1000005.5,500005.5,105.5
5-b,1000005.5,500005.5,105.6
4-a,1000004.4,500004.4,104.4
4-b,1000004.4,500004.4,104.5


8-a,1000008.8,500008.8,108.8
becomes the "new first record"
and gets written to OutputFile1

8-b,1000008.8,500008.8,108.9
7-a,1000007.7,500007.7,107.7
7-b,1000007.7,500007.7,107.8

are the duplicates and get written to OutputFile2


The new temporary file with these 4 records removed would be:

6-b,1000006.6,500006.6,106.7
6-a,1000006.6,500006.6,106.6
5-a,1000005.5,500005.5,105.5
5-b,1000005.5,500005.5,105.6
4-a,1000004.4,500004.4,104.4
4-b,1000004.4,500004.4,104.5

6-b,1000006.6,500006.6,106.7
becomes the "new first record"
and gets written to OutputFile1

6-a,1000006.6,500006.6,106.6
5-a,1000005.5,500005.5,105.5
5-b,1000005.5,500005.5,105.6

are the duplicates and get written to OutputFile2


The new temporary file with these 4 records removed would be:

4-a,1000004.4,500004.4,104.4
4-b,1000004.4,500004.4,104.5


4-a,1000004.4,500004.4,104.4
becomes the "new first record"
and gets written to OutputFile1

4-b,1000004.4,500004.4,104.5

is the duplicate and gets written to OutputFile2

Done
Whew!

Best Regards,
Kenny.