Home Man
Search
Today's Posts
Register

OS X is a line of Unix-based graphical operating systems developed, marketed, and sold by Apple.

Slow FFT in ksh93 and awk.

Tags
awk, fft, ksh93, slow, solved

Login to Reply

 
Thread Tools Search this Thread
# 8  
Old 1 Week Ago
Another thing you can do to speed it up is keep it running - feed it input and read it output at need, rather than starting and killing it all the time. How to do that depends on what your input data would look like and what you'd want for your output data.
The Following User Says Thank You to Corona688 For This Useful Post:
wisecracker (1 Week Ago)
# 9  
Old 1 Week Ago
Pur awk FFT.

Quote:
Originally Posted by Corona688
Another thing you can do to speed it up is keep it running - feed it input and read it output at need, rather than starting and killing it all the time. How to do that depends on what your input data would look like and what you'd want for your output data.
There is one error and corrected for the correct results, note the minus sign in red:
SUMIMAG += -REAL_ARRAY[T]*SIN_ANGLE + IMAG_ARRAY[T]*COS_ANGLE;
Correct results:
Code:
Real:
 8.50 3.64 -1.71 0.05 1.50 -0.17 -0.29 0.48 0.50 0.48 -0.29 -0.17 1.50 0.05 -1.71 3.64
Imaginary:
 0.00 -1.93 -0.50 -0.24 -1.00 0.47 0.50 -1.22 0.00 1.22 -0.50 -0.47 1.00 0.24 0.50 1.93

Otherwise phenominal.
Thanks a lot...
(This has taught me more about awk than ANY tutorial or info from the mighty WWW and for that I can only say thank you.)

Last edited by wisecracker; 1 Week Ago at 06:44 AM.. Reason: Add title.
The Following User Says Thank You to wisecracker For This Useful Post:
Corona688 (1 Week Ago)
# 10  
Old 1 Week Ago
Just as a side note - awka is an awk -> C translator

I think it was created out of similar frustrations we have all had with awk performance in major applications. I've tried it with varying success.

May be worth considering:
GitHub - noyesno/awka: Revive awka - AWK to C Conversion Tool
The Following 2 Users Say Thank You to jim mcnamara For This Useful Post:
Corona688 (1 Week Ago), wisecracker (1 Week Ago)
# 11  
Old 1 Week Ago
Pure awk FFT.

OK a modified version of Corona688's awk translation:
Code:
# FFT_Awk.awk
# A modified translation by Corona688, WAY COOL!
# Don't feed it REAL/IMAG/K/T, those are local variables!
function fft(REAL_ARRAY, IMAG_ARRAY, K, T, SUMREAL, SUMIMAG,
        ANGLE)
{
        # PI to 14 decimal places.
        PI=3.14159265358979;
        for(K=1; K in REAL_ARRAY; K++)
        {
                SUMREAL=SUMIMAG=0;
                for(T=1; T in REAL_ARRAY; T++)
                {
                        ANGLE=(2*PI*(T-1)*(K-1))/N;
                        SUMREAL += REAL_ARRAY[T]*cos(ANGLE) + IMAG_ARRAY[T]*sin(ANGLE);
                        SUMIMAG += -REAL_ARRAY[T]*sin(ANGLE) + IMAG_ARRAY[T]*cos(ANGLE);
                }
                REAL[K]=SUMREAL
                IMAG[K]=SUMIMAG
        }
}

BEGIN {
        # REALSTR="1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5"
        # IMAGSTR="0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0"
        # 'IMAGSTR' can be builtin but not done in this DEMO.

        # Split into arrays.  INDEXES 1-16, not 0-15!
        N=split(REALSTR, REAL_ARRAY, " ");
        split(IMAGSTR, IMAG_ARRAY, " ");
        # Convert strings to floats.
        for(X in REAL_ARRAY) REAL_ARRAY[X] += 0.0
        for(X in IMAG_ARRAY) IMAG_ARRAY[X] += 0.0
        # N=16;    # Hardcoded 16 elements WITH '0.5' mid point padding for this DEMO!

        fft(REAL_ARRAY, IMAG_ARRAY);

        printf("Real:\n");
        for(X=1; X in REAL; X++)
                printf(" %.5f", REAL[X]);
        printf("\n\n");

        printf("Imaginary:\n");
        for(X=1; X in IMAG; X++)
                printf(" %.5f", IMAG[X]);
        printf("\n\n");

        printf("Absolute Values:\n");
        for(X=1; X<=N; X++)
		printf(" %.5f", sqrt(((REAL[X])^2)+((IMAG[X])^2)));
        printf("\n");

# awk -f FFT_Awk.awk -v N=16 -v REALSTR="1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5" IMAGSTR="0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0" /dev/null
#
# Real:
#  8.50000 3.63782 -1.70711 0.05143 1.50000 -0.17275 -0.29289 0.48350 0.50000 0.48350 -0.29289 -0.17275 1.50000 0.05143 -1.70711 3.63782
#
# Imaginary:
#  0.00000 -1.93071 -0.50000 -0.24147 -1.00000 0.46564 0.50000 -1.22361 0.00000 1.22361 -0.50000 -0.46564 1.00000 0.24147 0.50000 1.93071
#
# Absolute Values:
#  8.50000 4.11842 1.77882 0.24688 1.80278 0.49665 0.57947 1.31567 0.50000 1.31567 0.57947 0.49665 1.80278 0.24688 1.77882 4.11842
}

Is this a first?
Could it even pull views to UNIX from search engines?
Who knows but one thing I do know my eventual aim was to do this.
I chose the ksh-awk and ARexx versions earlier in this thread to see the possibilities of FFT on languages not really designed to do it, then to dig deep into awk programming to succeed at this also...
The awk only version, courtesy of Corona688, makes FFT available for all the shells floating point or not.
Again C688, superb, now to ether remove import dependencies for python, (already done a python version), or modify this for the builtin 'SPECAN' in AudioScope.sh...
Lastly, I hope this has been an interesting break from all the usual stuff posted on here.

Last edited by wisecracker; 1 Week Ago at 06:43 AM.. Reason: Add title...
The Following 2 Users Say Thank You to wisecracker For This Useful Post:
Corona688 (6 Days Ago), Neo (1 Week Ago)
# 12  
Old 1 Week Ago
WOW!
The awk version works on an AMIGA A1200 with HDD and 4MB extra memory using ADE the UNIX _emulator_ for AMIGA OS-3.x running ksh88 on top of a default AMIGA A1200 shell.
Now that is pretty damn good!
Thanks to Corona688 for his original translation.

@Corona688 with your permission can I upload to AMINET for we AMIGA fanatics too?

Aminet - Latest packages, last 14 days
# 13  
Old 1 Week Ago
Absolutely. I don't post code for people to not use it.

It's a kind of naive implementation though. Usually better to read values from stdin, but knowing how audioscope works I knew you'd need batches. That'd obscure how it really works a bit though. Maybe better as this, people can populate the arrays as they please.

It's not truly an FFT either, just a DFT -- which gives the same result but takes much more time to do so. Notice how it must do N*N loops for N input. FFT accomplishes the same with N log N loops.

Still, I hope it has its uses. And I'm grateful for you writing one in shell. That's the first DFT I've seen written in a language I really understand.

Last edited by Corona688; 1 Week Ago at 06:26 PM..
The Following 2 Users Say Thank You to Corona688 For This Useful Post:
Neo (1 Week Ago), wisecracker (1 Week Ago)
# 14  
Old 1 Week Ago
Quote:
Originally Posted by Corona688
Absolutely. I don't post code for people to not use it.

It's a kind of naive implementation though. Usually better to read values from stdin, but knowing how audioscope works I knew you'd need batches. That'd obscure how it really works a bit though. Maybe better as this, people can populate the arrays as they please.

It's not truly an FFT either, just a DFT -- which gives the same result but takes much more time to do so. Notice how it must do N*N loops for N input. FFT accomplishes the same with N log N loops.

Still, I hope it has its uses. And I'm grateful for you writing one in shell. That's the first DFT I've seen written in a language I really understand.
Yes it is technically a DFT but most people don't know what the difference is and FFT is what they know most, so I used that term. I will upload to AMINET as a DFT however.
And naive it is too, but I could stay with python and use the builtin import cmath version I have got to eliminate the 'scipy' and 'scipyIO' dependencies. I really didn't want to use python although my code works from Python 1.4.0 to the current 3.7.0.
Thanks a lot matey, will post the URL for you to see when I have uploaded it...
Bazza...

------ Post updated 19-09-18 at 06:19 AM ------

Hi Corona688...

Uploaded to the AMINET site at around 9:30pm UK local time last night, 18-09-2018.

At around 6:00am today 19-09-2018 UK local time it was added and has had 2 downloads, dls, already. It will be on show for 14 days and I expect at least 150 dls in that time.

Aminet - dev/gcc/DFT-FFT.awk.txt

Thanks a lot...

Bazza.

(Consider this thread wrapped up.)

I have no idea why it added my separate post to the end of this one...
The Following User Says Thank You to wisecracker For This Useful Post:
Corona688 (6 Days Ago)
Login to Reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Ksh93/AIX compatibility majinfrede AIX 2 04-26-2018 09:07 AM
Possible ksh93 Bug Expanding Variables? bakunin Shell Programming and Scripting 4 07-04-2017 01:54 AM
Ksh93 on Linux compatible with ksh93 on AIX Keith Turley UNIX for Advanced & Expert Users 6 10-22-2016 01:37 AM
Embed tcl in ksh93 script gio001 UNIX for Advanced & Expert Users 3 02-19-2013 12:37 PM
ksh93 different results using -x option lthorson Shell Programming and Scripting 8 10-30-2011 10:54 AM
Install ksh93 for cygwin devtakh UNIX for Advanced & Expert Users 1 07-10-2009 07:32 AM
Performance degradation with KSH93 i.f.schulz Shell Programming and Scripting 2 04-02-2009 10:27 AM
ksh88 or ksh93 Nestor Shell Programming and Scripting 3 03-13-2009 10:33 AM
ksh93 deprecation... curleb Shell Programming and Scripting 2 03-17-2008 08:36 PM
Strange array handling in ksh93 bakunin Shell Programming and Scripting 3 10-24-2005 12:49 PM


All times are GMT -4. The time now is 06:54 AM.

Unix & Linux Forums Content Copyright©1993-2018. All Rights Reserved.
UNIX.COM Login
Username:
Password:  
Show Password