The main difference between AIX 5.3 and 6.1 compared to AIX 7.1 (on Power7) is ability to have up to 256 processors, and using smt4 - 1024 logical (scheduable) CPUs.
In AIX 7.1 TL1 - ASO was introduced as main new feature that
was AIX 7.1 only (as it has been introduced into AIX 6.1 TL8).
Basically, in terms of most features AIX 6.1 TL6, TL7, TL8 compares very well with AIX 7.1 TL0, TL1, TL2 respectively.
As mentioned above: AIX 6.1 only supports regular WPAR while AIX 7.1 supports regular and versioned WPAR (vWPAR is available in both AIX 5.2 - which I am told has been very popular with Oracle 8 customers/users) and more recently, AIX 5.3.
While AIX 5.3 can run natively on POWER7 it can only run in POWER6 emulation mode (and smt2) while as a vWPAR it can utilize SMT4, i.e. run in POWER7 mode. Also, as a WPAR it is mobile (whereas - if I am not mistaken - AIX 5.3 is not supported under LPM (Live Partition Mobility).
Basically, if your application is supported under AIX 7.1 I recommend the upgrade (from AIX 5.3) - rather than to AIX 6.1 as AIX 6.1 is already in it's 6th year while AIX 7.1 only in it's third. Hopefully, such an update will not be needed again in 4 to 5 years (remember AIX 5.3 is in it's 10th year (starting in 2004)).
Please excuse the rambling - and I hope this helps (someone
)