Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Count Unique values from multiple lists of files
Top Forums Shell Programming and Scripting Count Unique values from multiple lists of files Post 302915726 by whegra on Thursday 4th of September 2014 11:48:10 AM
Old 09-04-2014
Count Unique values from multiple lists of files
Looking for a little help here.

I have 1000's of text files within a multiple folders.

Code:
YYYY/
       /MM
             /1000's Files

Eg.
Code:
2014/01/1000 files
2014/02/1237 files
2014/03/1400 files

There are folders for each year and each month, and within each monthly folder there are 1000's of txt files.

Within each txt file there are 1000's of values.

eg values
Code:
546546541121321321142839741ACME183012839081
56412314235412563416243banana31231928301928
239187239719231231231273192Orange1283018231

What I want to do is for each year/month is to find and count all the Unique Alpha upper of lowercase values and get a count for that value.

EG.

Code:
Year: 2008
Month: 01 

ACME: 40057
banana: 20253
Orange:14321
etc..

Then move on to the second monthly folder in 2008 and so on until all years and months parsed.

It would fine all the find unique values and count but do this for all the year and monthly folders.

Any help would be greatly appreciated, as I'd rather not do this manually.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to count Unique Values from a file.

Hi I have the following info in a file - <Cell id="25D"/> <Cell id="26A"/> <Cell id="26B"/> <Cell id="26C"/> <Cell id="27A"/> <Cell id="27B"/> <Cell id="27C"/> <Cell id="28A"/> I would like to know how would you go about counting all... (4 Replies)
Discussion started by: Prega
4 Replies

2. Shell Programming and Scripting

List unique values and count instances in .csv file

I need to take the second column of a .csv file and count the number of instances of each unique value in that same second column. I'd like the output to be value,count sorted by most instances. Thanks for any guidance! Data example: 317476,317756,0 816063,318861,0 313123,319091,0... (4 Replies)
Discussion started by: batcho
4 Replies

3. Shell Programming and Scripting

Find and count unique date values in a file based on position

Hello, I need some sort of way to extract every date contained in a file, and count how many of those dates there are. Here are the specifics: The date format I'm looking for is mm/dd/yyyy I only need to look after line 45 in the file (that's where the data begins) The columns of... (2 Replies)
Discussion started by: ronan1219
2 Replies

4. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

5. Shell Programming and Scripting

Looping through entire directory and count unique values

Hello, I`m a complete newbie to coding, please help with this problem. I have multiple files in a directory, I have to loop through the contents of each file and extract number of unique isoforms in that file. Each file is tab delimited and only the line with the first parent (column 3)... (1 Reply)
Discussion started by: ritakadm
1 Replies

6. Shell Programming and Scripting

Count frequency of unique values in specific column

Hi, I have tab-deliminated data similar to the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to count the frequency of each individual "unique" value in the 1st column. Thus, the desired output would be as follows: dot 3 cat 1 hot 1 is... (5 Replies)
Discussion started by: owwow14
5 Replies

7. Shell Programming and Scripting

Reading multiple values from multiple lines and columns and setting them to unique variables.

Hello, I would like to ask for help with csh script. An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Discussion started by: FMMOLA
7 Replies

8. Shell Programming and Scripting

Comparing multiple network files (edge lists)

I want to compare 4 edge-lists to basically see if an edge is present in all 4 networks. The issue is that an edge A-B in one file can be present as B-A in another file. Example: Input 1: net1.txt A B 0.1 C D 0.65 D E 0.9 E A 0.7 Input 2: net2.txt A Z 0.1 C D 0.65 E D 0.9 E A... (1 Reply)
Discussion started by: Sanchari
1 Replies

9. Shell Programming and Scripting

Print count of unique values

Hello experts, I am converting a number into its binary output as : read n echo "obase=2;$n" | bc I wish to count the maximum continuous occurrences of the digit 1. Example : 1. The binary equivalent of 5 = 101. Hence the output must be 1. 2. The binary... (3 Replies)
Discussion started by: H squared
3 Replies

10. Shell Programming and Scripting

Count number of unique values in each column of array

What is an efficient way of counting the number of unique values in a 400 column by 1000 row array and outputting the counts per column, assuming the unique values in the array are: A, B, C, D In other words the output should look like: Value COL1 COL2 COL3 A 50 51 52... (16 Replies)
Discussion started by: Geneanalyst
16 Replies

LEARN ABOUT DEBIAN

sdl::gfx::imagefilter

pods::SDL::GFX::ImageFilter(3pm)			User Contributed Perl Documentation			  pods::SDL::GFX::ImageFilter(3pm)

NAME

       SDL::GFX::ImageFilter - image filtering functions

CATEGORY

       TODO, GFX

METHODS

   MMX_detect
       int gfx_image_MMX_detect()      CODE:	       SDL_imageFilterMMXdetect();

   MMX_off
       void gfx_image_MMX_off()      CODE:	     SDL_imageFilterMMXoff();

   MMX_on
       void gfx_image_MMX_on()	    CODE:	    SDL_imageFilterMMXon();

   add
       int gfx_image_add(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterAdd(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   mean
       int gfx_image_mean(Src1, Src2, Dest, length)	 unsigned char *Src1	  unsigned char *Src2	   unsigned char *Dest	    int length
	    CODE:	    RETVAL = SDL_imageFilterMean(Src1, Src2, Dest, length);	 OUTPUT:	   RETVAL

   sub
       int gfx_image_sub(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterSub(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   abs_diff
       int gfx_image_abs_diff(Src1, Src2, Dest, length)      unsigned char *Src1      unsigned char *Src2      unsigned char *Dest	int length
	    CODE:	    RETVAL = SDL_imageFilterAbsDiff(Src1, Src2, Dest, length);	    OUTPUT:	      RETVAL

   mult
       int gfx_image_mult(Src1, Src2, Dest, length)	 unsigned char *Src1	  unsigned char *Src2	   unsigned char *Dest	    int length
	    CODE:	    RETVAL = SDL_imageFilterMult(Src1, Src2, Dest, length);	 OUTPUT:	   RETVAL

   mult_nor
       int gfx_image_mult_nor(Src1, Src2, Dest, length)      unsigned char *Src1      unsigned char *Src2      unsigned char *Dest	int length
	    CODE:	    RETVAL = SDL_imageFilterMultNor(Src1, Src2, Dest, length);	    OUTPUT:	      RETVAL

   mult_div_by_2
       int gfx_image_mult_div_by_2(Src1, Src2, Dest, length)	  unsigned char *Src1	   unsigned char *Src2	    unsigned char *Dest      int
       length	   CODE:	   RETVAL = SDL_imageFilterMultDivby2(Src1, Src2, Dest, length);      OUTPUT:		RETVAL

   mult_div_by_4
       int gfx_image_mult_div_by_4(Src1, Src2, Dest, length)	  unsigned char *Src1	   unsigned char *Src2	    unsigned char *Dest      int
       length	   CODE:	   RETVAL = SDL_imageFilterMultDivby4(Src1, Src2, Dest, length);      OUTPUT:		RETVAL

   bit_and
       int gfx_image_bit_and(Src1, Src2, Dest, length)	    unsigned char *Src1      unsigned char *Src2      unsigned char *Dest      int length
	    CODE:	    RETVAL = SDL_imageFilterBitAnd(Src1, Src2, Dest, length);	   OUTPUT:	     RETVAL

   bit_or
       int gfx_image_bit_or(Src1, Src2, Dest, length)	   unsigned char *Src1	    unsigned char *Src2      unsigned char *Dest      int length
	    CODE:	    RETVAL = SDL_imageFilterBitOr(Src1, Src2, Dest, length);	  OUTPUT:	    RETVAL

   div
       int gfx_image_div(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterDiv(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   bit_negation
       int gfx_image_bit_negation(Src1, Dest, length)	   unsigned char *Src1	    unsigned char *Dest      int length      CODE:
		 RETVAL = SDL_imageFilterBitNegation(Src1, Dest, length);      OUTPUT:		 RETVAL

   add_byte
       int gfx_image_add_byte(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterAddByte(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   add_uint
       int gfx_image_add_uint(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned int C
	    CODE:	    RETVAL = SDL_imageFilterAddUint(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   add_byte_to_half
       int gfx_image_add_byte_to_half(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char
       C      CODE:	      RETVAL = SDL_imageFilterAddByteToHalf(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   sub_byte
       int gfx_image_sub_byte(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterSubByte(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   sub_uint
       int gfx_image_sub_uint(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned int C
	    CODE:	    RETVAL = SDL_imageFilterSubUint(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   shift_right
       int gfx_image_shift_right(Src1, Dest, length, N)      unsigned char *Src1      unsigned char *Dest      int length      unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftRight(Src1, Dest, length, N);	    OUTPUT:	      RETVAL

   shift_right_uint
       int gfx_image_shift_right_uint(Src1, Dest, length, N)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char
       N      CODE:	      RETVAL = SDL_imageFilterShiftRightUint(Src1, Dest, length, N);	  OUTPUT:	    RETVAL

   mult_by_byte
       int gfx_image_mult_by_byte(Src1, Dest, length, C)      unsigned char *Src1      unsigned char *Dest	int length	unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterMultByByte(Src1, Dest, length, C);	    OUTPUT:	      RETVAL

   shift_right_and_mult_by_byte
       int gfx_image_shift_right_and_mult_by_byte(Src1, Dest, length, N, C)	 unsigned char *Src1	  unsigned char *Dest	   int length
	    unsigned char N	 unsigned char C      CODE:	      RETVAL = SDL_imageFilterShiftRightAndMultByByte(Src1, Dest, length, N, C);
	    OUTPUT:	      RETVAL

   shift_left_byte
       int gfx_image_shift_left_byte(Src1, Dest, length, N)	 unsigned char *Src1	  unsigned char *Dest	   int length	   unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeftByte(Src1, Dest, length, N);      OUTPUT:		 RETVAL

   shift_left_uint
       int gfx_image_shift_left_uint(Src1, Dest, length, N)	 unsigned char *Src1	  unsigned char *Dest	   int length	   unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeftUint(Src1, Dest, length, N);      OUTPUT:		 RETVAL

   shift_left
       int gfx_image_shift_left(Src1, Dest, length, N)	    unsigned char *Src1      unsigned char *Dest      int length      unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeft(Src1, Dest, length, N);	   OUTPUT:	     RETVAL

   binarize_using_threshold
       int gfx_image_binarize_using_threshold(Src1, Dest, length, T)	  unsigned char *Src1	   unsigned char *Dest	    int length
	    unsigned char T	 CODE:		 RETVAL = SDL_imageFilterBinarizeUsingThreshold(Src1, Dest, length, T);      OUTPUT:
		 RETVAL

   clip_to_range
       int gfx_image_clip_to_range(Src1, Dest, length, Tmin, Tmax)	unsigned char *Src1	 unsigned char *Dest	  int length	  unsigned
       char Tmin      unsigned char Tmax      CODE:	      RETVAL = SDL_imageFilterClipToRange(Src1, Dest, length, Tmin, Tmax);	OUTPUT:
		 RETVAL

   normalize_linear
       int gfx_image_normalize_linear(Src1, Dest, length, Cmin, Cmax, Nmin, Nmax)      unsigned char *Src1	unsigned char *Dest	 int
       length	   int Cmin	 int Cmax      int Nmin      int Nmax	   CODE:	   RETVAL = SDL_imageFilterNormalizeLinear(Src1, Dest,
       length, Cmin, Cmax, Nmin, Nmax);      OUTPUT:	       RETVAL

   convolve_kernel_3x3_divide
       int gfx_image_convolve_kernel_3x3_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel3x3Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_5x5_divide
       int gfx_image_convolve_kernel_5x5_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel5x5Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_7x7_divide
       int gfx_image_convolve_kernel_7x7_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel7x7Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_9x9_divide
       int gfx_image_convolve_kernel_9x9_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel9x9Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_3x3_shift_right
       int gfx_image_convolve_kernel_3x3_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel3x3ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_5x5_shift_right
       int gfx_image_convolve_kernel_5x5_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel5x5ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_7x7_shift_right
       int gfx_image_convolve_kernel_7x7_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel7x7ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_9x9_shift_right
       int gfx_image_convolve_kernel_9x9_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel9x9ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   sobel_x
       int gfx_image_sobel_x(Src, Dest, rows, columns)	    unsigned char *Src	    unsigned char *Dest      int rows	   int columns	    CODE:
		 RETVAL = SDL_imageFilterSobelX(Src, Dest, rows, columns);	OUTPUT: 	  RETVAL

   sobel_x_shift_right
       int gfx_image_sobel_x_shift_right(Src, Dest, rows, columns, NRightShift)      unsigned char *Src      unsigned char *Dest      int rows
	    int columns      unsigned char NRightShift	    CODE:	    RETVAL = SDL_imageFilterSobelXShiftRight(Src, Dest, rows, columns,
       NRightShift);	  OUTPUT:	    RETVAL

   align_stack
       void gfx_image_align_stack()	 CODE:		 SDL_imageFilterAlignStack();

   restore_stack
       void gfx_image_restore_stack()	   CODE:	   SDL_imageFilterRestoreStack();

AUTHORS

       See "AUTHORS" in SDL.

perl v5.14.2							    2012-05-28					  pods::SDL::GFX::ImageFilter(3pm)
All times are GMT -4. The time now is 05:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy