Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extracting column if above certain values and repeated over a number of times continuously
Top Forums UNIX for Dummies Questions & Answers Extracting column if above certain values and repeated over a number of times continuously Post 302456076 by newbeeuk on Thursday 23rd of September 2010 08:43:11 AM
Old 09-23-2010
Thank you very much.
The script did not work and returned with

"cut: invalid byte, character or field list"

I run the script on Centos 5.4

newbeeuk

---------- Post updated at 07:43 AM ---------- Previous update was at 05:29 AM ----------

I may not have explained clearly in my first post

Here is the another file for illustration, first 2 rows and first columns are identifier
Code:
id.1 A01 A01 A01 A68 A68
id.2 A5 A5 A5 A3 A3
1001 0.01 0 abc 0.136 0.136
1002 0.02 0 abc 0.262 0.183
1003 0.03 0 abc 0.662 0.662
1004 0.04 0 def 0.758 0.607
1005 0.05 0 def 0.855 0.855
1006 0.06 0 efg 0.867 0.867
1007 0 0 hij 0.872 0.692
1008 0.04 0 def 0.902 0.902
1009 0 0.06 def 0.51 0.906
1010 0 0 efg 0.51 0.906
1011 0 0.04 hij 0.51 0.969
1013 0.05 0 abc 0.07 0.743
1014 0.06 0 def 0.971 0.971
1015 0.07 0 def 0.971 0.743
1021 0 0.51 efg 0.996 0.996
1022 0 0.51 hij 0.996 0.996
1023 0 0.51 abc 0 0.995
1026 0.51 0 def 0 0.996
1027 0.51 0 poi 0 0.996
1028 0 0 sdf 0.05 0.996

I want everything that is greater than k (now assume >0.5)
and is continuous in that particular column for more than j rows (now assume>=3)
than I would like the output in 1 or multiple files in form of

first segment from column 3
Code:
id.1 A01
id.2 A5
1021 0.51
1022 0.51
1023 0.51

second segment from column 4
Code:
id.1 A68
id.2 A3
1003 0.662
1004 0.758
1005 0.855
1006 0.867
1007 0.872
1008 0.902
1009 0.51
1010 0.51
1011 0.51
1013 0.07

third segment also from column 5, split by 1 row with value < 0.5
Code:
id.1 A68
id.2 A3
1014 0.971
1015 0.971
1021 0.996
1022 0.996

and the last from column 6
Code:
id.1 A68
id.2 A3
1003 0.662
1004 0.607
1005 0.855
1006 0.867
1007 0.692
1008 0.902
1009 0.906
1010 0.906
1011 0.969
1013 0.743
1014 0.971
1015 0.743
1021 0.996
1022 0.996
1023 0.995
1026 0.996
1027 0.996
1028 0.996


none is coming from column 4 because all are alphabets and none from column 2 becasuse no continuous 3 row with value > 0.5

Thanks for help

newbeeuk
Last edited by Franklin52; 09-23-2010 at 10:02 AM.. Reason: Please use code tags!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how many times does this repeated sequence exist

need a script to determine daily how many times does the below repeated sequence exist in a log file, and if it shows failure to connect in the same file 200 PORT Command successful. 150 Opening BINARY mode data connection for rgr016.daily.0305. 226 Transfer complete. local: rgr016.daily.0306... (3 Replies)
Discussion started by: lichento
3 Replies

2. UNIX for Dummies Questions & Answers

To get the total of how many times the pattern is repeated.

Hi, I need the total of how many times the pattern is being repeated in a particular file. I tried with grep as below, but no go. grep -c best sample Contents of Sample file: ----------------------- This is the best site I have never come across. The best one indeed. Very best... (8 Replies)
Discussion started by: venkatesht
8 Replies

3. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on numerical values of a column

I have a text file where the second column is a list of numbers going from small to large. I want to extract the rows where the second column is smaller than or equal to 0.0001. My input: rs10082730 9e-08 12 46002702 rs2544081 1e-07 12 46015487 rs1425136 1e-06 7 35396742 rs2712590... (1 Reply)
Discussion started by: evelibertine
1 Replies

4. UNIX for Dummies Questions & Answers

Extracting rows from a space delimited text file based on the values of a column

I have a space delimited text file. I want to extract rows where the third column has 0 as a value and write those rows into a new space delimited text file. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

5. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

6. Shell Programming and Scripting

Choosing between repeated entries based on the "absolute values" of a column

Hello, I was looking for a way to select between the repeated entries (column1) based on the values of absolute values of column 3 (larger value). For example if the same gene id has FC value -2 and 1, I should get the output as -2. Kindly help. GeneID Description FC ... (2 Replies)
Discussion started by: Sanchari
2 Replies

7. Homework & Coursework Questions

Accepting a phrase and counting the number of times that it is repeated in a specific website

1. The problem statement, all variables and given/known data: Develop a shell script that accepts a phrase and counts the number of times that it is repeated in a specific website. Note: Im not sure if it's the whole website, or just a specific page but im guessing its thewhole website. ... (2 Replies)
Discussion started by: Zakerii
2 Replies

8. Shell Programming and Scripting

Extracting unique values of a column from a feed file

Hi Folks, I have the below feed file named abc1.txt in which you can see there is a title and below is the respective values in the rows and it is completely pipe delimited file ,. ... (4 Replies)
Discussion started by: punpun66
4 Replies

9. UNIX for Dummies Questions & Answers

Append no of times a column is repeated at the end

Hi folks, Iam working on a bash script, i need to print how many times column 2 repeated at the end of each line. Input.txt COL1 COL2 COL3 COL4 1 XX 45 N 2 YY 34 y 3 ZZ 44 N 4 XX 89 Y 5 XX 45 N 6 YY 84 D 7 ZZ 22 S Output.txt COL1 COL2 COL3 COL4 COL5 1 XX 45 N 3 2 YY 34... (6 Replies)
Discussion started by: tech_frk
6 Replies

10. Shell Programming and Scripting

Extracting values based on line-column numbers from multiple text files

Dear All, I have to solve the following problems with multiple tab-separated text file but I don't know how. Any help would be greatly appreciated. I have access to Linux mint (but not as a professional). I have multiple tab-delimited files with the following structure: file1: 1 44 2 ... (5 Replies)
Discussion started by: Bastami
5 Replies

LEARN ABOUT DEBIAN

sdl::gfx::imagefilter

pods::SDL::GFX::ImageFilter(3pm)			User Contributed Perl Documentation			  pods::SDL::GFX::ImageFilter(3pm)

NAME

       SDL::GFX::ImageFilter - image filtering functions

CATEGORY

       TODO, GFX

METHODS

   MMX_detect
       int gfx_image_MMX_detect()      CODE:	       SDL_imageFilterMMXdetect();

   MMX_off
       void gfx_image_MMX_off()      CODE:	     SDL_imageFilterMMXoff();

   MMX_on
       void gfx_image_MMX_on()	    CODE:	    SDL_imageFilterMMXon();

   add
       int gfx_image_add(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterAdd(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   mean
       int gfx_image_mean(Src1, Src2, Dest, length)	 unsigned char *Src1	  unsigned char *Src2	   unsigned char *Dest	    int length
	    CODE:	    RETVAL = SDL_imageFilterMean(Src1, Src2, Dest, length);	 OUTPUT:	   RETVAL

   sub
       int gfx_image_sub(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterSub(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   abs_diff
       int gfx_image_abs_diff(Src1, Src2, Dest, length)      unsigned char *Src1      unsigned char *Src2      unsigned char *Dest	int length
	    CODE:	    RETVAL = SDL_imageFilterAbsDiff(Src1, Src2, Dest, length);	    OUTPUT:	      RETVAL

   mult
       int gfx_image_mult(Src1, Src2, Dest, length)	 unsigned char *Src1	  unsigned char *Src2	   unsigned char *Dest	    int length
	    CODE:	    RETVAL = SDL_imageFilterMult(Src1, Src2, Dest, length);	 OUTPUT:	   RETVAL

   mult_nor
       int gfx_image_mult_nor(Src1, Src2, Dest, length)      unsigned char *Src1      unsigned char *Src2      unsigned char *Dest	int length
	    CODE:	    RETVAL = SDL_imageFilterMultNor(Src1, Src2, Dest, length);	    OUTPUT:	      RETVAL

   mult_div_by_2
       int gfx_image_mult_div_by_2(Src1, Src2, Dest, length)	  unsigned char *Src1	   unsigned char *Src2	    unsigned char *Dest      int
       length	   CODE:	   RETVAL = SDL_imageFilterMultDivby2(Src1, Src2, Dest, length);      OUTPUT:		RETVAL

   mult_div_by_4
       int gfx_image_mult_div_by_4(Src1, Src2, Dest, length)	  unsigned char *Src1	   unsigned char *Src2	    unsigned char *Dest      int
       length	   CODE:	   RETVAL = SDL_imageFilterMultDivby4(Src1, Src2, Dest, length);      OUTPUT:		RETVAL

   bit_and
       int gfx_image_bit_and(Src1, Src2, Dest, length)	    unsigned char *Src1      unsigned char *Src2      unsigned char *Dest      int length
	    CODE:	    RETVAL = SDL_imageFilterBitAnd(Src1, Src2, Dest, length);	   OUTPUT:	     RETVAL

   bit_or
       int gfx_image_bit_or(Src1, Src2, Dest, length)	   unsigned char *Src1	    unsigned char *Src2      unsigned char *Dest      int length
	    CODE:	    RETVAL = SDL_imageFilterBitOr(Src1, Src2, Dest, length);	  OUTPUT:	    RETVAL

   div
       int gfx_image_div(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterDiv(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   bit_negation
       int gfx_image_bit_negation(Src1, Dest, length)	   unsigned char *Src1	    unsigned char *Dest      int length      CODE:
		 RETVAL = SDL_imageFilterBitNegation(Src1, Dest, length);      OUTPUT:		 RETVAL

   add_byte
       int gfx_image_add_byte(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterAddByte(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   add_uint
       int gfx_image_add_uint(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned int C
	    CODE:	    RETVAL = SDL_imageFilterAddUint(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   add_byte_to_half
       int gfx_image_add_byte_to_half(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char
       C      CODE:	      RETVAL = SDL_imageFilterAddByteToHalf(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   sub_byte
       int gfx_image_sub_byte(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterSubByte(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   sub_uint
       int gfx_image_sub_uint(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned int C
	    CODE:	    RETVAL = SDL_imageFilterSubUint(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   shift_right
       int gfx_image_shift_right(Src1, Dest, length, N)      unsigned char *Src1      unsigned char *Dest      int length      unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftRight(Src1, Dest, length, N);	    OUTPUT:	      RETVAL

   shift_right_uint
       int gfx_image_shift_right_uint(Src1, Dest, length, N)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char
       N      CODE:	      RETVAL = SDL_imageFilterShiftRightUint(Src1, Dest, length, N);	  OUTPUT:	    RETVAL

   mult_by_byte
       int gfx_image_mult_by_byte(Src1, Dest, length, C)      unsigned char *Src1      unsigned char *Dest	int length	unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterMultByByte(Src1, Dest, length, C);	    OUTPUT:	      RETVAL

   shift_right_and_mult_by_byte
       int gfx_image_shift_right_and_mult_by_byte(Src1, Dest, length, N, C)	 unsigned char *Src1	  unsigned char *Dest	   int length
	    unsigned char N	 unsigned char C      CODE:	      RETVAL = SDL_imageFilterShiftRightAndMultByByte(Src1, Dest, length, N, C);
	    OUTPUT:	      RETVAL

   shift_left_byte
       int gfx_image_shift_left_byte(Src1, Dest, length, N)	 unsigned char *Src1	  unsigned char *Dest	   int length	   unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeftByte(Src1, Dest, length, N);      OUTPUT:		 RETVAL

   shift_left_uint
       int gfx_image_shift_left_uint(Src1, Dest, length, N)	 unsigned char *Src1	  unsigned char *Dest	   int length	   unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeftUint(Src1, Dest, length, N);      OUTPUT:		 RETVAL

   shift_left
       int gfx_image_shift_left(Src1, Dest, length, N)	    unsigned char *Src1      unsigned char *Dest      int length      unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeft(Src1, Dest, length, N);	   OUTPUT:	     RETVAL

   binarize_using_threshold
       int gfx_image_binarize_using_threshold(Src1, Dest, length, T)	  unsigned char *Src1	   unsigned char *Dest	    int length
	    unsigned char T	 CODE:		 RETVAL = SDL_imageFilterBinarizeUsingThreshold(Src1, Dest, length, T);      OUTPUT:
		 RETVAL

   clip_to_range
       int gfx_image_clip_to_range(Src1, Dest, length, Tmin, Tmax)	unsigned char *Src1	 unsigned char *Dest	  int length	  unsigned
       char Tmin      unsigned char Tmax      CODE:	      RETVAL = SDL_imageFilterClipToRange(Src1, Dest, length, Tmin, Tmax);	OUTPUT:
		 RETVAL

   normalize_linear
       int gfx_image_normalize_linear(Src1, Dest, length, Cmin, Cmax, Nmin, Nmax)      unsigned char *Src1	unsigned char *Dest	 int
       length	   int Cmin	 int Cmax      int Nmin      int Nmax	   CODE:	   RETVAL = SDL_imageFilterNormalizeLinear(Src1, Dest,
       length, Cmin, Cmax, Nmin, Nmax);      OUTPUT:	       RETVAL

   convolve_kernel_3x3_divide
       int gfx_image_convolve_kernel_3x3_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel3x3Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_5x5_divide
       int gfx_image_convolve_kernel_5x5_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel5x5Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_7x7_divide
       int gfx_image_convolve_kernel_7x7_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel7x7Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_9x9_divide
       int gfx_image_convolve_kernel_9x9_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel9x9Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_3x3_shift_right
       int gfx_image_convolve_kernel_3x3_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel3x3ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_5x5_shift_right
       int gfx_image_convolve_kernel_5x5_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel5x5ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_7x7_shift_right
       int gfx_image_convolve_kernel_7x7_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel7x7ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_9x9_shift_right
       int gfx_image_convolve_kernel_9x9_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel9x9ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   sobel_x
       int gfx_image_sobel_x(Src, Dest, rows, columns)	    unsigned char *Src	    unsigned char *Dest      int rows	   int columns	    CODE:
		 RETVAL = SDL_imageFilterSobelX(Src, Dest, rows, columns);	OUTPUT: 	  RETVAL

   sobel_x_shift_right
       int gfx_image_sobel_x_shift_right(Src, Dest, rows, columns, NRightShift)      unsigned char *Src      unsigned char *Dest      int rows
	    int columns      unsigned char NRightShift	    CODE:	    RETVAL = SDL_imageFilterSobelXShiftRight(Src, Dest, rows, columns,
       NRightShift);	  OUTPUT:	    RETVAL

   align_stack
       void gfx_image_align_stack()	 CODE:		 SDL_imageFilterAlignStack();

   restore_stack
       void gfx_image_restore_stack()	   CODE:	   SDL_imageFilterRestoreStack();

AUTHORS

       See "AUTHORS" in SDL.

perl v5.14.2							    2012-05-28					  pods::SDL::GFX::ImageFilter(3pm)
All times are GMT -4. The time now is 10:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy