I have not noticed, when viewing unix.com, any notable slowdown our our use of Font Awesome.
In fact, when I just checked, it shows the FA fonts cached in memory, with 0 additional delay.
I am guessing that those guys in that report you cited Akskay had their browser caching disabled in dev tools.
We use both Google Fonts and FA and I have never noticed any delay greater or less, then either one (after caching), so far. Of course, the initial loading will be faster depending on the CDN used and how those fonts are loaded, initially. If the Google CDN is faster than the FA CDN, that might be a factor in in initial load; but on the other hand, my experience is that networks change all the time (speed, performance), so drawing conclusions about loading time, can be very deceptive.
More importantly, is the caching, and after initial load, FB and Google Fonts (we use both) are cached in browser memory, and so are both very fast.