Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Remove duplicate words from column 1 Post 302948433 by jimmyf on Monday 29th of June 2015 04:23:12 PM
Old 06-29-2015
can the text be manipulated from horizontal to vertical for the output so there is only one row per instance?
Code:
dog, text1, text2, text3, text1 text2, text3, text1, text2, text3
cat, text1, text2, text3

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove duplicate rows of a file based on a value of a column

Hi, I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g. COL1 COL2 COL3 A 1234 1234 B 3k32 2322 C Xk32 TTT A NEW XX22 B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies

2. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen... (6 Replies)
Discussion started by: sam_2921
6 Replies

3. Shell Programming and Scripting

Remove duplicate line detail based on column one data

My input file: AVI.out <detail>named as the RRM .</detail> AVI.out <detail>Contains 1 RRM .</detail> AR0.out <detail>named as the tellurite-resistance.</detail> AWG.out <detail>Contains 2 HTH .</detail> ADV.out <detail>named as the DENR family.</detail> ADV.out ... (10 Replies)
Discussion started by: patrick87
10 Replies

4. UNIX for Dummies Questions & Answers

Remove duplicate rows when >10 based on single column value

Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column. e.g. a 1 a 2 a 3 b 1 c 1 gives b 1 c 1 but requires 11 duplicates before it deletes. Thanks for the help Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies

5. Shell Programming and Scripting

Remove very first pair of duplicate words

I have file which is almost look like below MMIT MMIT ... (2 Replies)
Discussion started by: manas_ranjan
2 Replies

6. UNIX for Dummies Questions & Answers

[SOLVED] remove lines that have duplicate values in column two

Hi, I've got a file that I'd like to uniquely sort based on column 2 (values in column 2 begin with "comp"). I tried sort -t -nuk2,3 file.txtBut got: sort: multi-character tab `-nuk2,3' "man sort" did not help me out Any pointers? Input: Output: (5 Replies)
Discussion started by: pathunkathunk
5 Replies

7. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies

8. Shell Programming and Scripting

Find duplicate words in first column between "10" repetiotions

hello I have a file of the form: nameA number number ... nameB number number ... nameA number number ... nameC number number ... nameD number number ... 10 nameA number number ... nameC number number ... nameB number number ... nameC number number ...... (4 Replies)
Discussion started by: phaethon
4 Replies

9. Shell Programming and Scripting

Filter file to remove duplicate values in first column

Hello, I have a script that is generating a tab delimited output file. num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 -1.6120 3 compound_03 -1.3053 1.8495 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

10. Shell Programming and Scripting

Remove duplicate values in a column(not in the file)

Hi Gurus, I have a file(weblog) as below abc|xyz|123|agentcode=sample code abcdeeess,agentcode=sample code abcdeeess,agentcode=sample code abcdeeess|agentadd=abcd stereet 23343,agentadd=abcd stereet 23343 sss|wwq|999|agentcode=sample1 code wqwdeeess,gentcode=sample1 code... (4 Replies)
Discussion started by: ratheeshjulk
4 Replies
Imager::Test(3pm)					User Contributed Perl Documentation					 Imager::Test(3pm)

NAME
Imager::Test - common functions used in testing Imager SYNOPSIS
use Imager::Test 'diff_text_with_nul'; diff_text_with_nul($test_name, $text1, $text2, @string_options); DESCRIPTION
This is a repository of functions used in testing Imager. Some functions will only be useful in testing Imager itself, while others should be useful in testing modules that use Imager. No functions are exported by default. FUNCTIONS
Test functions is_color1($color, $grey, $comment) Tests if the first channel of $color matches $grey. is_color3($color, $red, $green, $blue, $comment) Tests if $color matches the given ($red, $green, $blue) is_color4($color, $red, $green, $blue, $alpha, $comment) Tests if $color matches the given ($red, $green, $blue, $alpha) is_fcolor1($fcolor, $grey, $comment) is_fcolor1($fcolor, $grey, $epsilon, $comment) Tests if $fcolor's first channel is within $epsilon of ($grey). For the first form $epsilon is taken as 0.001. is_fcolor3($fcolor, $red, $green, $blue, $comment) is_fcolor3($fcolor, $red, $green, $blue, $epsilon, $comment) Tests if $fcolor's channels are within $epsilon of ($red, $green, $blue). For the first form $epsilon is taken as 0.001. is_fcolor4($fcolor, $red, $green, $blue, $alpha, $comment) is_fcolor4($fcolor, $red, $green, $blue, $alpha, $epsilon, $comment) Tests if $fcolor's channels are within $epsilon of ($red, $green, $blue, $alpha). For the first form $epsilon is taken as 0.001. is_image($im1, $im2, $comment) Tests if the 2 images have the same content. Both images must be defined, have the same width, height, channels and the same color in each pixel. The color comparison is done at 8-bits per pixel. The color representation such as direct vs paletted, bits per sample are not checked. Equivalent to is_image_similar($im1, $im2, 0, $comment). is_imaged($im, $im2, $comment) is_imaged($im, $im2, $epsilon, $comment) Tests if the two images have the same content at the double/sample level. $epsilon defaults to the platform DBL_EPSILON multiplied by four. is_image_similar($im1, $im2, $maxdiff, $comment) Tests if the 2 images have similar content. Both images must be defined, have the same width, height and channels. The cum of the squares of the differences of each sample are calculated and must be less than or equal to $maxdiff for the test to pass. The color comparison is done at 8-bits per pixel. The color representation such as direct vs paletted, bits per sample are not checked. isnt_image($im1, $im2, $comment) Tests that the two images are different. For regressions tests where something (like text output of "0") produced no change, but should have produced a change. test_colorf_gpix($im, $x, $y, $expected, $epsilon, $comment) Retrieves the pixel ($x,$y) from the low-level image $im and compares it to the floating point color $expected, with a tolerance of epsilon. test_color_gpix($im, $x, $y, $expected, $comment) Retrieves the pixel ($x,$y) from the low-level image $im and compares it to the floating point color $expected. test_colorf_glin($im, $x, $y, $pels, $comment) Retrieves the floating point pixels ($x, $y)-[$x+@$pels, $y] from the low level image $im and compares them against @$pels. is_color_close3($color, $red, $green, $blue, $tolerance, $comment) Tests if $color's first three channels are within $tolerance of ($red, $green, $blue). Test suite functions Functions that perform one or more tests, typically used to test various parts of Imager's implementation. image_bounds_checks($im) Attempts to write to various pixel positions outside the edge of the image to ensure that it fails in those locations. Any new image type should pass these tests. Does 16 separate tests. mask_tests($im, $epsilon) Perform a standard set of mask tests on the OO image $im. Does 24 separate tests. diff_text_with_nul($test_name, $text1, $text2, @options) Creates 2 test images and writes $text1 to the first image and $text2 to the second image with the string() method. Each call adds 3 "ok"/"not ok" to the output of the test script. Extra options that should be supplied include the font and either a color or channel parameter. This was explicitly created for regression tests on #21770. Helper functions test_image_raw() Returns a 150x150x3 Imager::ImgRaw test image. test_image() Returns a 150x150x3 8-bit/sample OO test image. Name: "basic". test_image_16() Returns a 150x150x3 16-bit/sample OO test image. Name: "basic16" test_image_double() Returns a 150x150x3 double/sample OO test image. Name: "basic_double". test_image_gray() Returns a 150x150 single channel OO test image. Name: "gray". test_image_gray_16() Returns a 150x150 16-bit/sample single channel OO test image. Name: "gray16". test_image_mono() Returns a 150x150 bilevel image that passes the is_bilevel() test. Name: "mono". test_image_named($name) Return one of the other test images above based on name. color_cmp($c1, $c2) Performs an ordering of 3-channel colors (like <=>). colorf_cmp($c1, $c2) Performs an ordering of 3-channel floating point colors (like <=>). AUTHOR
Tony Cook <tony@develop-help.com> perl v5.14.2 2012-05-11 Imager::Test(3pm)
All times are GMT -4. The time now is 07:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy