I need to trim this down so that there a no duplicates in the first column. Actually, the entire row would be a duplicate, but I don't see any reason to look at anything other than the index value. There is no particular rational to the order and there could be any number of duplicates of a given row.
I need one, and only one, instance of each index value ("num" column value) in the file, not just the lines with num values that appear only once. There always seems to be some confusion about that with discussions of "unique" lines.
The only thing I could think of was to sort the rows on the num column value and then loop through checking if the num value was equal to the previous line. If it is not equal, copy it to a new array, etc.
Any suggestions? There always seems to be some simple one line solution that I don't know about.
I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records.
contents of file looks like
30002157,40342424,OTC,mart_rec,100, ,0
30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Hi, I've got a file that I'd like to uniquely sort based on column 2 (values in column 2 begin with "comp").
I tried sort -t -nuk2,3 file.txtBut got:
sort: multi-character tab `-nuk2,3'
"man sort" did not help me out
Any pointers?
Input:
Output: (5 Replies)
Hello experts,
I have a requirement where I have to implement two checks on a csv file:
1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit.
2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
I have a .CSV file with the below format:
"column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10
"12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Input
1,ABCD,no
2,system,yes
3,ABCD,yes
4,XYZ,no
5,XYZ,yes
6,pc,noCode used to find duplicate with regard to 2nd column
awk 'NR == 1 {p=$2; next} p == $2 { print "Line" NR "$2 is duplicated"} {p=$2}' FS="," ./input.csv
Now is there a wise way to de-duplicate the entire line (remove... (4 Replies)
Hi Gurus,
I have a file(weblog) as below
abc|xyz|123|agentcode=sample code abcdeeess,agentcode=sample code abcdeeess,agentcode=sample code abcdeeess|agentadd=abcd stereet 23343,agentadd=abcd stereet 23343
sss|wwq|999|agentcode=sample1 code wqwdeeess,gentcode=sample1 code... (4 Replies)
Dear folks
I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
I have csv file with 30, 40 columns
Pasting just three column for problem description
I want to filter record if column 1 matches CN or DN then,
check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345
and if column 2 contains 6789, 6789... (5 Replies)
Hi Experts,
I have csv file with 30, 40 columns
Pasting just 2 column for problem description.
Need to print error if below combination is not present in file
check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same.
For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
LEARN ABOUT SUNOS
num
Num(3o) OCaml library Num(3o)NAME
Num - Operation on arbitrary-precision numbers.
Module
Module Num
Documentation
Module Num
: sig end
Operation on arbitrary-precision numbers.
Numbers (type num ) are arbitrary-precision rational numbers, plus the special elements 1/0 (infinity) and 0/0 (undefined).
type num =
| Int of int
| Big_int of Big_int.big_int
| Ratio of Ratio.ratio
The type of numbers.
=== Arithmetic operations ===
val (+/) : num -> num -> num
Same as Num.add_num .
val add_num : num -> num -> num
Addition
val minus_num : num -> num
Unary negation.
val (-/) : num -> num -> num
Same as Num.sub_num .
val sub_num : num -> num -> num
Subtraction
val ( */ ) : num -> num -> num
Same as Num.mult_num .
val mult_num : num -> num -> num
Multiplication
val square_num : num -> num
Squaring
val (//) : num -> num -> num
Same as Num.div_num .
val div_num : num -> num -> num
Division
val quo_num : num -> num -> num
Euclidean division: quotient.
val mod_num : num -> num -> num
Euclidean division: remainder.
val ( **/ ) : num -> num -> num
Same as Num.power_num .
val power_num : num -> num -> num
Exponentiation
val abs_num : num -> num
Absolute value.
val succ_num : num -> num
succ n is n+1
val pred_num : num -> num
pred n is n-1
val incr_num : num Pervasives.ref -> unit
incr r is r:=!r+1 , where r is a reference to a number.
val decr_num : num Pervasives.ref -> unit
decr r is r:=!r-1 , where r is a reference to a number.
val is_integer_num : num -> bool
Test if a number is an integer
=== The four following functions approximate a number by an integer : ===
val integer_num : num -> num
integer_num n returns the integer closest to n . In case of ties, rounds towards zero.
val floor_num : num -> num
floor_num n returns the largest integer smaller or equal to n .
val round_num : num -> num
round_num n returns the integer closest to n . In case of ties, rounds off zero.
val ceiling_num : num -> num
ceiling_num n returns the smallest integer bigger or equal to n .
val sign_num : num -> int
Return -1 , 0 or 1 according to the sign of the argument.
=== Comparisons between numbers ===
val (=/) : num -> num -> bool
val (</) : num -> num -> bool
val (>/) : num -> num -> bool
val (<=/) : num -> num -> bool
val (>=/) : num -> num -> bool
val (<>/) : num -> num -> bool
val eq_num : num -> num -> bool
val lt_num : num -> num -> bool
val le_num : num -> num -> bool
val gt_num : num -> num -> bool
val ge_num : num -> num -> bool
val compare_num : num -> num -> int
Return -1 , 0 or 1 if the first argument is less than, equal to, or greater than the second argument.
val max_num : num -> num -> num
Return the greater of the two arguments.
val min_num : num -> num -> num
Return the smaller of the two arguments.
=== Coercions with strings ===
val string_of_num : num -> string
Convert a number to a string, using fractional notation.
val approx_num_fix : int -> num -> string
See Num.approx_num_exp .
val approx_num_exp : int -> num -> string
Approximate a number by a decimal. The first argument is the required precision. The second argument is the number to approximate.
Num.approx_num_fix uses decimal notation; the first argument is the number of digits after the decimal point. approx_num_exp uses scien-
tific (exponential) notation; the first argument is the number of digits in the mantissa.
val num_of_string : string -> num
Convert a string to a number.
=== Coercions between numerical types ===
val int_of_num : num -> int
val num_of_int : int -> num
val nat_of_num : num -> Nat.nat
val num_of_nat : Nat.nat -> num
val num_of_big_int : Big_int.big_int -> num
val big_int_of_num : num -> Big_int.big_int
val ratio_of_num : num -> Ratio.ratio
val num_of_ratio : Ratio.ratio -> num
val float_of_num : num -> float
OCamldoc 2012-06-26 Num(3o)