Sponsored Content
Top Forums Shell Programming and Scripting Remove duplicates according to their frequency in column Post 302958189 by corfuitl on Tuesday 20th of October 2015 07:03:44 AM
Old 10-20-2015
Hi again,

Works perfectly now Smilie

Thanks for your time!

Best,
Kon
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies

2. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies

3. Shell Programming and Scripting

remove duplicates based on single column

Hello, I am new to shell scripting. I have a huge file with multiple columns for example: I have 5 columns below. HWUSI-EAS000_29:1:105 + chr5 76654650 AATTGGAA HHHHG HWUSI-EAS000_29:1:106 + chr5 76654650 AATTGGAA B@HYL HWUSI-EAS000_29:1:108 + ... (4 Replies)
Discussion started by: Diya123
4 Replies

4. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies

5. Shell Programming and Scripting

Remove duplicates within row and separate column

Hi all I have following kind of input file ESR1 PA156 leflunomide PA450192 leflunomide CHST3 PA26503 docetaxel Pa4586; thalidomide Pa34958; decetaxel docetaxel docetaxel I want to remove duplicates and I want to separate anything before and after PAxxxx entry into columns or... (1 Reply)
Discussion started by: manigrover
1 Replies

6. Shell Programming and Scripting

Remove Duplicates on multiple Key Columns and get the Latest Record from Date/Time Column

Hi Experts , we have a CDC file where we need to get the latest record of the Key columns Key Columns will be CDC_FLAG and SRC_PMTN_I and fetch the latest record from the CDC_PRCS_TS Can we do it with a single awk command. Please help.... (3 Replies)
Discussion started by: vijaykodukula
3 Replies

7. Shell Programming and Scripting

Count frequency of unique values in specific column

Hi, I have tab-deliminated data similar to the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to count the frequency of each individual "unique" value in the 1st column. Thus, the desired output would be as follows: dot 3 cat 1 hot 1 is... (5 Replies)
Discussion started by: owwow14
5 Replies

8. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

9. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies
Math(3pm)						User Contributed Perl Documentation						 Math(3pm)

NAME
PDL::Math - extended mathematical operations and special functions SYNOPSIS
use PDL::Math; use PDL::Graphics::TriD; imag3d [SURF2D,bessj0(rvals(zeroes(50,50))/2)]; DESCRIPTION
This module extends PDL with more advanced mathematical functions than provided by standard Perl. All the functions have one input pdl, and one output, unless otherwise stated. Many of the functions are linked from the system maths library or the Cephes maths library (determined when PDL is compiled); a few are implemented entirely in PDL. FUNCTIONS
acos Signature: (a(); [o]b()) The usual trigonometric function. Works inplace. acos does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. asin Signature: (a(); [o]b()) The usual trigonometric function. Works inplace. asin does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. atan Signature: (a(); [o]b()) The usual trigonometric function. Works inplace. atan does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. cosh Signature: (a(); [o]b()) The standard hyperbolic function. Works inplace. cosh does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. sinh Signature: (a(); [o]b()) The standard hyperbolic function. Works inplace. sinh does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. tan Signature: (a(); [o]b()) The usual trigonometric function. Works inplace. tan does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. tanh Signature: (a(); [o]b()) The standard hyperbolic function. Works inplace. tanh does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. ceil Signature: (a(); [o]b()) Round to integer values in floating-point format. Works inplace. ceil does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. floor Signature: (a(); [o]b()) Round to integer values in floating-point format. Works inplace. floor does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. rint Signature: (a(); [o]b()) Round to integer values in floating-point format. rint uses the 'round half to even' rounding method (also known as banker's rounding). Half-integers are rounded to the nearest even number. This avoids a slight statistical bias inherent in always rounding half-integers up or away from zero. If you are looking to round half-integers up (regardless of sign), try "floor($x+0.5)". If you want to round half-integers away from zero, try "floor(abs($x)+0.5)*($x<=>0)". Works inplace. rint does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. pow Signature: (a(); b(); [o]c()) Synonym for `**'. Works inplace. pow does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. acosh Signature: (a(); [o]b()) The standard hyperbolic function. Works inplace. acosh does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. asinh Signature: (a(); [o]b()) The standard hyperbolic function. Works inplace. asinh does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. atanh Signature: (a(); [o]b()) The standard hyperbolic function. Works inplace. atanh does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. erf Signature: (a(); [o]b()) The error function. Works inplace. erf does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. erfc Signature: (a(); [o]b()) The complement of the error function. Works inplace. erfc does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. bessj0 Signature: (a(); [o]b()) The regular Bessel function of the first kind, J_n Works inplace. bessj0 does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. bessj1 Signature: (a(); [o]b()) The regular Bessel function of the first kind, J_n Works inplace. bessj1 does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. bessy0 Signature: (a(); [o]b()) The regular Bessel function of the second kind, Y_n. Works inplace. bessy0 does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. bessy1 Signature: (a(); [o]b()) The regular Bessel function of the second kind, Y_n. Works inplace. bessy1 does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. bessjn Signature: (a(); int n(); [o]b()) The regular Bessel function of the first kind, J_n . This takes a second int argument which gives the order of the function required. Works inplace. bessjn does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. bessyn Signature: (a(); int n(); [o]b()) The regular Bessel function of the first kind, Y_n . This takes a second int argument which gives the order of the function required. Works inplace. bessyn does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. lgamma Signature: (a(); [o]b(); int[o]s()) log gamma function This returns 2 piddles -- the first set gives the log(gamma) values, while the second set, of integer values, gives the sign of the gamma function. This is useful for determining factorials, amongst other things. lgamma does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. badmask Signature: (a(); b(); [o]c()) Clears all "infs" and "nans" in $a to the corresponding value in $b. badmask can be run with $a inplace: badmask($a->inplace,0); $a->inplace->badmask(0); If bad values are present, these are also cleared. isfinite Signature: (a(); int [o]mask()) Sets $mask true if $a is not a "NaN" or "inf" (either positive or negative). Works inplace. Bad values are treated as "NaN" or "inf". erfi Signature: (a(); [o]b()) The inverse of the error function. Works inplace. erfi does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. ndtri Signature: (a(); [o]b()) The value for which the area under the Gaussian probability density function (integrated from minus infinity) is equal to the argument (cf erfi). Works inplace. ndtri does handle bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. polyroots Signature: (cr(n); ci(n); [o]rr(m); [o]ri(m)) Complex roots of a complex polynomial, given coefficients in order of decreasing powers. ($rr, $ri) = polyroots($cr, $ci); polyroots does not process bad values. It will set the bad-value flag of all output piddles if the flag is set for any of the input piddles. BUGS
Hasn't been tested on all platforms to ensure Cephes versions are picked up automatically and used correctly. AUTHOR
Copyright (C) R.J.R. Williams 1997 (rjrw@ast.leeds.ac.uk), Karl Glazebrook (kgb@aaoepp.aao.gov.au) and Tuomas J. Lukka (Tuomas.Lukka@helsinki.fi). Portions (C) Craig DeForest 2002 (deforest@boulder.swri.edu). All rights reserved. There is no warranty. You are allowed to redistribute this software / documentation under certain conditions. For details, see the file COPYING in the PDL distribution. If this file is separated from the PDL distribution, the PDL copyright notice should be included in the file. perl v5.14.2 2012-05-30 Math(3pm)
All times are GMT -4. The time now is 06:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy