Find duplicate based on 'n' fields and mark the duplicate as 'D'
Hi,
In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'.
In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create a new field at the end of the records as "C" or "D")
Last edited by machomaddy; 01-28-2012 at 06:37 AM..
Reason: Edited wrong Input "2010-12-31" to "2011-12-31" in the 4th record
I have a input file with formating:
6000000901 ;36200103 ;h3a01f496 ;
2000123605 ;36218982 ;heefa1328 ;
2000273132 ;36246985 ;h08c5cb71 ;
2000041207 ;36246985 ;heef75497 ;
Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Hi:
I've been searching the net but didnt find a clue. I have a file in which, for some records, some fields coincide. I want to compare one (or more) of the dissimilar fields and retain the one record that fulfills a certain condition. For example, on this file:
99 TR 1991 5 06 ... (1 Reply)
I have file that I need to remove the duplicates. The problem is, I need to only keep the one which has a unique 3rd field. Here is a sample file:
xxx.xxx:x:CISCO1.CLEVE61W:ERIE.NET:x:x:x:x:
xxx.xxx:x:CISCO2.CLEVE62W:OHIO.NET:x:x:x:x:
xxx.xxx:x:CISCO2.CLEVE62W:NORTH.NET:x:x:x:x:... (1 Reply)
Hello,
Although I have found similar questions, I could not find advice that
could help with our problem.
The issue:
We have several hundreds text files containing repeated blocks of text
(I guess back at the time they were prepared like that to optmize
printing).
The block of texts... (13 Replies)
Hi,
How can I remove duplicates from a file based on group on other column? for example:
Test1|Test2|Test3|Test4|Test5
Test1|Test6|Test7|Test8|Test5
Test1|Test9|Test10|Test11|Test12
Test1|Test13|Test14|Test15|Test16
Test17|Test18|Test19|Test20|Test21
Test17|Test22|Test23|Test24|Test5
... (2 Replies)
Hi ,
Some time i got duplicated value in my files ,
bundle_identifier= B
Sometext=ABC
bundle_identifier= A
bundle_unit=500
Sometext123=ABCD
bundle_unit=400
i need to check if there is a duplicated values or not if yes , i need to check if the value is A or B when Bundle_Identified ,... (2 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Dear folks
I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Hi,
My input looks like this (tab-delimited):
grp1 name2 firstname M 55 item1 item1.0
grp1 name2 firstname F 55 item1 item1.0
grp2 name1 firstname M 55 item1 item1.0
grp2 name2 firstname M 55 item1 item1.0
Using awk, I am trying to discard the records with common fields 2, 4, 5, 6, 7... (4 Replies)
Discussion started by: beca123456
4 Replies
LEARN ABOUT REDHAT
ale
ALE(1)ALE(1)NAME
ale - a synthetic capture engine and renderer
SYNOPSIS
ale [OPTIONS] original-frame [supplemental-frames ...] output-file
ale [HELP OPTIONS]
ale --version
DESCRIPTION
ALE is a free software synthetic capture engine and renderer accepting, as input, a sequence of image frames, and producing, as output,
rendered images, registration data, and 3D models (currently experimental).
EXAMPLES
ale --hu
Print the help page on program usage.
ale 001.ppm 002.ppm 003.ppm out.ppm
Render output file out.ppm from inputs 001.ppm, 002.ppm, and 003.ppm.
ale --scale 2 001.ppm 002.ppm 003.ppm out.png
Render output file out.png at 2x scale.
ale --perturb-upper=0 --trans-load=a.t --projective frame*.png out.png
Render output file out.png from inputs frame*.png, using projective transformations as described in the file a.t.
ale --3dv 0 out.png --projective frame*.png null:
Render output file out.png from inputs frame*.png, using a 3D scene model to determine transformations.
ale --visp last:sinc*lanc:6 sf:0 out/ .jpg in/*.jpg null:
Render a video stream from files in/*.jpg to the directory out/, stabilizing to the first frame, with a lanczos-windowed sinc filter.
USAGE
ale is a wrapper script that calls the executable ale-bin after extracting exposure EXIF information from image files (when the exiftool
utility is available) and converting from raw formats (when the dcraw utility is available).
When ale-bin has been linked with ImageMagick, input and output files may be of any kind supported by ImageMagick. Otherwise, only PPM
files can be read and written by ale-bin.
OPTIONS
OPTIONS are summarized via built-in help pages (see HELP OPTIONS below). More detailed manuals describing options and internal algorithms
are available for certain versions of ALE. See SEE ALSO below for more details on other manuals.
HELP OPTIONS
A listing of available help options can be obtained by invoking ale --hu.
ENVIRONMENT
The wrapper script ale recognizes the following environment variables:
ALE_BIN
ALE executable to use.
DCRAW
dcraw raw extraction tool to use.
EXIF_UTILITY
EXIF extraction utility to use.
ALE_COUNT_THREADS
Displays statistics on thread counts when set to a non-zero value.
PAGER
Text pager to use (e.g., for displaying help pages).
SEE ALSO
More detailed manuals for certain versions of ALE are distributed with source packages, and may also be available from:
http://auricle.dyndns.org/ALE/manuals/
03/31/2009 ALE(1)