Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Store 'for' loop iterations in memory
Top Forums Shell Programming and Scripting Store 'for' loop iterations in memory Post 302949789 by lsantome on Thursday 16th of July 2015 04:15:16 AM
Old 07-16-2015
Store 'for' loop iterations in memory
hi all,

I'm writting a simple bash script to do a recursive paste over >10000 files with two columns each.

The files looks like this:
Code:
A1BG    3
A1CF    3
A2M    3
A4GALT    5
AAAS    2
AACS    2
AADAT    2
AAGAB    4
AAK1    3
AAMP    2
AANAT    3
AARS    2
AARS2    3
AARSD1    2
...


And the code is this:

Code:
SPHERE=Sphere.matrix.txt
rm -rf $SPHERE
for i in `ls *.spheres | sort`; do
    if [ -f $SPHERE ]; then
        cut -f2 $i \
        | paste $SPHERE - > $SPHERE.tmp
        mv $SPHERE.tmp $SPHERE
    else
        cat $i > $SPHERE
    fi
done

it opens the first file and writes it to the output file. then it opens the second file, it takes the second colums and it pastes it to the output file, and so on...

the code works propperly, but it gets progresively slower because in each cycle it has to open and overwrite a bigger file.

if I could store the result of aech iteration in memory instead of in disk, I think that the performnace would be much faster.

could you please give me some guidance in this?

thank you so much!
Last edited by Don Cragun; 07-16-2015 at 06:04 AM.. Reason: Add CODE tags for sample input.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

For Loop iterations

I have a silly beginners question here: I'm running a for loop within an awk command as follows, however the loop only runs once, instead of 2 through to 11 and then terminating. As the output to screen of x (line 6) shows x to equal 12, when it should be 1. --------------------------- awk... (2 Replies)
Discussion started by: sirtrancealot
2 Replies

2. Shell Programming and Scripting

Not able to store the results of perl recursive function when applied under for loop

Hi Perl Gurus , need URGENT HELP PLEASE !!!!! I have one recursive Perl function which takes path of any directory as argument and returns array containing all the sub folders inside it recursively. Now the problem is that it works well if i use it with one time but the problem is that when... (0 Replies)
Discussion started by: anthriksh2000
0 Replies

3. Shell Programming and Scripting

For loop scp transfers check if all iterations successful

All, I am using a for loop to SCP a bunch of files in a directory. I am having it then drop a .ready file name. Is there a way to check for the success of all iterations and then email upon fail or success? Example of part of my script: for file in $ORIGLOC/* do ] &&... (2 Replies)
Discussion started by: markdjones82
2 Replies

4. UNIX for Dummies Questions & Answers

Process getting terminated after sleep iterations

Hello All, I have a script which has a functionality to sleep for 300 seconds after it does some processing, so in the logs if i check after the 3 iteration of sleep it didn't write saying "sleeping for 300 seconds". I suspect putty would automatically terminate session as i cannot access it any... (1 Reply)
Discussion started by: Ariean
1 Replies

5. UNIX for Dummies Questions & Answers

Writing a loop to manipulate a script and store it in multiple output files

I have a script where the the 9th line looks like this: $filename=sprintf("250.1chr%d.ped", $N); I want to modify this script 1000 times, changing 250.1chr%d.ped to 250.2chr%d.ped, 250.3chr%.ped.......and so on all the way to 250.1000chr%d.ped and store each output in files called ... (4 Replies)
Discussion started by: evelibertine
4 Replies

6. UNIX for Dummies Questions & Answers

how to append into single file for multiple iterations

hi, i want to capture the CPU performance, memory , harddisk usage into a single file for every 10 seconds iterations and it shall run until i kill it. can somebody plz help me in this... thanx (1 Reply)
Discussion started by: bskumar4u
1 Replies

7. Shell Programming and Scripting

A way to store 2 random numbers from a for loop?

I have a for loop that cycles twice and generates 1 random number for each pass through. I would like to be able to store the two numbers to use later for arithmetics. Is there a way to do that? Right now I can only seem to use the last random number for anything. Thanks. (4 Replies)
Discussion started by: AxlVanDamme
4 Replies

8. Shell Programming and Scripting

Display no. of iterations

Hello.. I have a input file as follows 123,sl 345,ab 345,cd 123,qw 123,lk I want output file as follows.. --> print the no. of iterations at the end of each row. 123,sl 1 123.qw 2 123,lk 3 345,ab 1 345,cd 2 Thanks Suresh (3 Replies)
Discussion started by: suresh3566
3 Replies

9. Shell Programming and Scripting

awk loop using array:wish to store array values from loop for use outside loop

Here's my code: awk -F '' 'NR==FNR { if (/time/ && $5>10) A=$2" "$3":"$4":"($5-01) else if (/time/ && $5<01) A=$2" "$3":"$4-01":"(59-$5) else if (/time/ && $5<=10) A=$2" "$3":"$4":0"($5-01) else if (/close/) { B=0 n1=n2; ... (2 Replies)
Discussion started by: klane
2 Replies

10. Shell Programming and Scripting

Shell script to loop and store in array

I'm trying to achieve the follwoinig with no luck. Find the directories that are greater than 50GB in size and pick the owner of the directory as I would like to send an alert notification. du -sh * | sort -rh 139G Dir_1 84G Dir_2 15G Dir_3 ls -l Dir_1 drwx------ 2... (3 Replies)
Discussion started by: 308002184
3 Replies

LEARN ABOUT DEBIAN

sdl::gfx::imagefilter

pods::SDL::GFX::ImageFilter(3pm)			User Contributed Perl Documentation			  pods::SDL::GFX::ImageFilter(3pm)

NAME

       SDL::GFX::ImageFilter - image filtering functions

CATEGORY

       TODO, GFX

METHODS

   MMX_detect
       int gfx_image_MMX_detect()      CODE:	       SDL_imageFilterMMXdetect();

   MMX_off
       void gfx_image_MMX_off()      CODE:	     SDL_imageFilterMMXoff();

   MMX_on
       void gfx_image_MMX_on()	    CODE:	    SDL_imageFilterMMXon();

   add
       int gfx_image_add(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterAdd(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   mean
       int gfx_image_mean(Src1, Src2, Dest, length)	 unsigned char *Src1	  unsigned char *Src2	   unsigned char *Dest	    int length
	    CODE:	    RETVAL = SDL_imageFilterMean(Src1, Src2, Dest, length);	 OUTPUT:	   RETVAL

   sub
       int gfx_image_sub(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterSub(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   abs_diff
       int gfx_image_abs_diff(Src1, Src2, Dest, length)      unsigned char *Src1      unsigned char *Src2      unsigned char *Dest	int length
	    CODE:	    RETVAL = SDL_imageFilterAbsDiff(Src1, Src2, Dest, length);	    OUTPUT:	      RETVAL

   mult
       int gfx_image_mult(Src1, Src2, Dest, length)	 unsigned char *Src1	  unsigned char *Src2	   unsigned char *Dest	    int length
	    CODE:	    RETVAL = SDL_imageFilterMult(Src1, Src2, Dest, length);	 OUTPUT:	   RETVAL

   mult_nor
       int gfx_image_mult_nor(Src1, Src2, Dest, length)      unsigned char *Src1      unsigned char *Src2      unsigned char *Dest	int length
	    CODE:	    RETVAL = SDL_imageFilterMultNor(Src1, Src2, Dest, length);	    OUTPUT:	      RETVAL

   mult_div_by_2
       int gfx_image_mult_div_by_2(Src1, Src2, Dest, length)	  unsigned char *Src1	   unsigned char *Src2	    unsigned char *Dest      int
       length	   CODE:	   RETVAL = SDL_imageFilterMultDivby2(Src1, Src2, Dest, length);      OUTPUT:		RETVAL

   mult_div_by_4
       int gfx_image_mult_div_by_4(Src1, Src2, Dest, length)	  unsigned char *Src1	   unsigned char *Src2	    unsigned char *Dest      int
       length	   CODE:	   RETVAL = SDL_imageFilterMultDivby4(Src1, Src2, Dest, length);      OUTPUT:		RETVAL

   bit_and
       int gfx_image_bit_and(Src1, Src2, Dest, length)	    unsigned char *Src1      unsigned char *Src2      unsigned char *Dest      int length
	    CODE:	    RETVAL = SDL_imageFilterBitAnd(Src1, Src2, Dest, length);	   OUTPUT:	     RETVAL

   bit_or
       int gfx_image_bit_or(Src1, Src2, Dest, length)	   unsigned char *Src1	    unsigned char *Src2      unsigned char *Dest      int length
	    CODE:	    RETVAL = SDL_imageFilterBitOr(Src1, Src2, Dest, length);	  OUTPUT:	    RETVAL

   div
       int gfx_image_div(Src1, Src2, Dest, length)	unsigned char *Src1	 unsigned char *Src2	  unsigned char *Dest	   int length
	    CODE:	    RETVAL = SDL_imageFilterDiv(Src1, Src2, Dest, length);	OUTPUT: 	  RETVAL

   bit_negation
       int gfx_image_bit_negation(Src1, Dest, length)	   unsigned char *Src1	    unsigned char *Dest      int length      CODE:
		 RETVAL = SDL_imageFilterBitNegation(Src1, Dest, length);      OUTPUT:		 RETVAL

   add_byte
       int gfx_image_add_byte(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterAddByte(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   add_uint
       int gfx_image_add_uint(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned int C
	    CODE:	    RETVAL = SDL_imageFilterAddUint(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   add_byte_to_half
       int gfx_image_add_byte_to_half(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char
       C      CODE:	      RETVAL = SDL_imageFilterAddByteToHalf(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   sub_byte
       int gfx_image_sub_byte(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterSubByte(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   sub_uint
       int gfx_image_sub_uint(Src1, Dest, length, C)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned int C
	    CODE:	    RETVAL = SDL_imageFilterSubUint(Src1, Dest, length, C);	 OUTPUT:	   RETVAL

   shift_right
       int gfx_image_shift_right(Src1, Dest, length, N)      unsigned char *Src1      unsigned char *Dest      int length      unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftRight(Src1, Dest, length, N);	    OUTPUT:	      RETVAL

   shift_right_uint
       int gfx_image_shift_right_uint(Src1, Dest, length, N)	  unsigned char *Src1	   unsigned char *Dest	    int length	    unsigned char
       N      CODE:	      RETVAL = SDL_imageFilterShiftRightUint(Src1, Dest, length, N);	  OUTPUT:	    RETVAL

   mult_by_byte
       int gfx_image_mult_by_byte(Src1, Dest, length, C)      unsigned char *Src1      unsigned char *Dest	int length	unsigned char C
	    CODE:	    RETVAL = SDL_imageFilterMultByByte(Src1, Dest, length, C);	    OUTPUT:	      RETVAL

   shift_right_and_mult_by_byte
       int gfx_image_shift_right_and_mult_by_byte(Src1, Dest, length, N, C)	 unsigned char *Src1	  unsigned char *Dest	   int length
	    unsigned char N	 unsigned char C      CODE:	      RETVAL = SDL_imageFilterShiftRightAndMultByByte(Src1, Dest, length, N, C);
	    OUTPUT:	      RETVAL

   shift_left_byte
       int gfx_image_shift_left_byte(Src1, Dest, length, N)	 unsigned char *Src1	  unsigned char *Dest	   int length	   unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeftByte(Src1, Dest, length, N);      OUTPUT:		 RETVAL

   shift_left_uint
       int gfx_image_shift_left_uint(Src1, Dest, length, N)	 unsigned char *Src1	  unsigned char *Dest	   int length	   unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeftUint(Src1, Dest, length, N);      OUTPUT:		 RETVAL

   shift_left
       int gfx_image_shift_left(Src1, Dest, length, N)	    unsigned char *Src1      unsigned char *Dest      int length      unsigned char N
	    CODE:	    RETVAL = SDL_imageFilterShiftLeft(Src1, Dest, length, N);	   OUTPUT:	     RETVAL

   binarize_using_threshold
       int gfx_image_binarize_using_threshold(Src1, Dest, length, T)	  unsigned char *Src1	   unsigned char *Dest	    int length
	    unsigned char T	 CODE:		 RETVAL = SDL_imageFilterBinarizeUsingThreshold(Src1, Dest, length, T);      OUTPUT:
		 RETVAL

   clip_to_range
       int gfx_image_clip_to_range(Src1, Dest, length, Tmin, Tmax)	unsigned char *Src1	 unsigned char *Dest	  int length	  unsigned
       char Tmin      unsigned char Tmax      CODE:	      RETVAL = SDL_imageFilterClipToRange(Src1, Dest, length, Tmin, Tmax);	OUTPUT:
		 RETVAL

   normalize_linear
       int gfx_image_normalize_linear(Src1, Dest, length, Cmin, Cmax, Nmin, Nmax)      unsigned char *Src1	unsigned char *Dest	 int
       length	   int Cmin	 int Cmax      int Nmin      int Nmax	   CODE:	   RETVAL = SDL_imageFilterNormalizeLinear(Src1, Dest,
       length, Cmin, Cmax, Nmin, Nmax);      OUTPUT:	       RETVAL

   convolve_kernel_3x3_divide
       int gfx_image_convolve_kernel_3x3_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel3x3Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_5x5_divide
       int gfx_image_convolve_kernel_5x5_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel5x5Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_7x7_divide
       int gfx_image_convolve_kernel_7x7_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel7x7Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_9x9_divide
       int gfx_image_convolve_kernel_9x9_divide(Src, Dest, rows, columns, Kernel, Divisor)	unsigned char *Src	unsigned char *Dest
	    int rows	  int columns	   Sint16 *Kernel      unsigned char Divisor	  CODE: 	  RETVAL =
       SDL_imageFilterConvolveKernel9x9Divide(Src, Dest, rows, columns, Kernel, Divisor);      OUTPUT:		 RETVAL

   convolve_kernel_3x3_shift_right
       int gfx_image_convolve_kernel_3x3_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel3x3ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_5x5_shift_right
       int gfx_image_convolve_kernel_5x5_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel5x5ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_7x7_shift_right
       int gfx_image_convolve_kernel_7x7_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel7x7ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   convolve_kernel_9x9_shift_right
       int gfx_image_convolve_kernel_9x9_shift_right(Src, Dest, rows, columns, Kernel, NRightShift)	 unsigned char *Src	 unsigned char
       *Dest	  int rows	int columns	 Sint16 *Kernel      unsigned char NRightShift	    CODE:	    RETVAL =
       SDL_imageFilterConvolveKernel9x9ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift);      OUTPUT:		 RETVAL

   sobel_x
       int gfx_image_sobel_x(Src, Dest, rows, columns)	    unsigned char *Src	    unsigned char *Dest      int rows	   int columns	    CODE:
		 RETVAL = SDL_imageFilterSobelX(Src, Dest, rows, columns);	OUTPUT: 	  RETVAL

   sobel_x_shift_right
       int gfx_image_sobel_x_shift_right(Src, Dest, rows, columns, NRightShift)      unsigned char *Src      unsigned char *Dest      int rows
	    int columns      unsigned char NRightShift	    CODE:	    RETVAL = SDL_imageFilterSobelXShiftRight(Src, Dest, rows, columns,
       NRightShift);	  OUTPUT:	    RETVAL

   align_stack
       void gfx_image_align_stack()	 CODE:		 SDL_imageFilterAlignStack();

   restore_stack
       void gfx_image_restore_stack()	   CODE:	   SDL_imageFilterRestoreStack();

AUTHORS

       See "AUTHORS" in SDL.

perl v5.14.2							    2012-05-28					  pods::SDL::GFX::ImageFilter(3pm)
All times are GMT -4. The time now is 02:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy