Remove duplicated records and update last line record counts


Login or Register to Reply

 
Thread Tools Search this Thread
# 8  
Old 1 Week Ago
Quote:
Originally Posted by nezabudka
Hi Don, thanks for the explanation.
Code:
awk 'BEGIN {FS=OFS=","} /^T/ {$2=length(A)} !A[$0]++'

Hi nezabudka,
Always glad to help.

This is another interesting way to do it. Unfortunately, the standards do not specify the behavior of the awk length built-in function when given an array name as an argument. This use is described on the GNU gawk man page and works in BSD awk version 20070501 (but is not documented in the BSD awk man page) that is installed on macOS Mojave (version 10.14.3).

I have no idea whether or not this will work (as an undocumented feature) on green_k's Solaris system in /usr/xpg4/bin/awk or nawk. I also do not know if gawk is installed on green_k's system.
This User Gave Thanks to Don Cragun For This Post:
nezabudka (1 Week Ago)
# 9  
Old 1 Week Ago
On top of what Don Cragun said, the last approach would not account for "duplicate duplicates".


Illogic nonsense... please disregard.

Last edited by RudiC; 1 Week Ago at 07:33 AM..
# 10  
Old 1 Week Ago
Quote:
Originally Posted by RudiC
On top of what Don Cragun said, the last approach would not account for "duplicate duplicates".
Hi RudiC,
I'm not sure what you mean. I don't see any reason why the code shown in post #7 should fail as long as all of the following are true:
  1. There are only "D" and "T" records in the input file.
  2. There is only one "T" record in the input file.
  3. The "T" record is the last record in the input file.
  4. The awk being used returns the number of elements in the array when length(array_name) is called.
The first three are true in the sample data provided in this thread. The fourth is true with gawk starting with version 3.1.5 according to the Linux 2.6 gawk man page available in the UNIX and Linux Man Pages repository. By experiment, it also works on the awk version 20070501 provided with macOS Mojave version 10.14.3.

Unlike the code in post #5, this code is not subtracting the number of duplicates found, it is directly setting the number of unique elements found.

Am I missing something?
This User Gave Thanks to Don Cragun For This Post:
RudiC (1 Week Ago)
# 11  
Old 1 Week Ago
Hi Don Cragun, sorry for posting that nonsense. My logics seem to require some lubrication. I may need some sleep. Post withdrawn.
# 12  
Old 1 Week Ago
Quote:
Originally Posted by RudiC
Hi Don Cragun, sorry for posting that nonsense. My logics seem to require some lubrication. I may need some sleep. Post withdrawn.
Hi RudiC,
I know the feeling. I'm just up this late because I checked to see what was going on here after resetting all of the clocks in the house. (Daylight Saving time kicked in here this morning when the clock should have hit 2am. I hate Daylight Saving time!)

Sleep tight.

- Don
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Join files, omit duplicated records from one file CHoggarth Shell Programming and Scripting 4 09-26-2017 08:15 AM
Remove duplicated bootlist entries Gabriander AIX 7 01-11-2017 09:04 AM
MYSQL - trigger update on record insert or update barrydocks Programming 5 05-19-2014 02:53 AM
How to remove duplicated lines? the_simpsons Shell Programming and Scripting 7 06-24-2013 12:55 PM
How to Remove the new line character inbetween a record machomaddy Shell Programming and Scripting 1 01-02-2012 09:51 AM
New file should store all the 7 existing filenames and their record counts and ftp th pr293 Shell Programming and Scripting 1 11-24-2011 03:03 AM
Remove rows with first 4 fields duplicated in awk tomahawk Shell Programming and Scripting 3 10-28-2011 08:10 AM
Hardcoding & Record counts in a file shruthidwh UNIX for Dummies Questions & Answers 7 08-22-2011 11:55 AM
Help with remove duplicated content perl_beginner Shell Programming and Scripting 3 08-22-2011 10:35 AM
Split a single record to multiple records & add folder name to each line ram2581 Shell Programming and Scripting 20 07-08-2011 05:44 PM
Sending e-mail of record counts in 3 or more files msrahman Shell Programming and Scripting 3 11-01-2010 11:17 AM
Help to Add and Remove Records only from first line/last line enjoy Shell Programming and Scripting 4 04-26-2010 05:02 AM
remove duplicated columns kamel.seg Shell Programming and Scripting 6 02-21-2008 06:36 AM
remove duplicated lines without sort lalelle Shell Programming and Scripting 6 08-21-2007 07:44 AM
remove duplicated xml record in a file under unix happyv Shell Programming and Scripting 8 09-20-2006 01:36 PM