I'd like to share some experiences and what I found for NIS migration from Solaris 8 NIS to Linux platform.
I'm not an expert for both platforms, it's just when I tested both systems and found something really tricky. That might takes a lot of time for you to find the root cause. So, I think I can share some experiences from what I've found to help you saving time if you have the same need as me. And still, I have some questions unsolved, maybe some experts can give me suggestions.

Original NIS server: Solaris 8
New NIS server: Red hat Linux 6.5

I have to say that my experiences are built on Solaris 8 migrating to RHEL, Solaris 8 is an outdated system, Solaris 9, 10 above should be more advanced than 8, so I am not sure if it can works for other versions.

To save words, there were two questions I posted earlier.
User authentication failed while log in Solaris 8 client on Linux NIS server.
Solaris 8 MD5 encryption support

1. Linux 6.5 supports SHA512 password encryption as default, however, Solaris only supports DES encryption. There is no doubt that you have to modify Linux NIS encryption to support DES, otherwise, you may face the issue to authenticate a NIS user when you login to a solaris client.

Code:

A. run the following command to change the default encryption.
# authconfig --passalgo=descrypt --update.

B. run the following command to check if the above setting works.
# authconfig --test
pam_unix is always enabled
 shadow passwords are enabled
 password hashing algorithm is descrypt

C. check the following line of /etc/pam.d/system-auth,
change
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
to
password    sufficient    pam_unix.so shadow nullok try_first_pass use_authtok

2. If you still have Solaris 8 clients, /var/yp/Makefile has to be modified as the following,

Code:

A. MERGE_PASSWD=true  --> a shadow map won't be generated.

After that, restart your ypserv service and re-make your /var/yp/Makefile, it should works for a Solaris client to login a NIS user with no issues.

If you need a NIS slave server, it's recommended to build it under the same platform as NIS master has. A cross-platform method between two platforms may face the issue of ypfxrd synchronization since Solaris uses ndbm package instead of GNU dbm or Berkeley DB.

If you need password aging, that's the problem I am still working on it.

I found if you have Solaris clients in your domain, it is necessary to set "MERGE_PASSWD=true" to make NIS users can login from Solaris clients.
However, that won't generate a shadow map for NIS, and it's a MUST existence if you want password aging.

It seems like a conflict if you want both of them works. I still don't find a way out of this. Does someone has any workaround or solution for this issue??

I've been stuck on this task for couple weeks. If anyone is willing to share some experiences, I'll be appreciated.

Thanks.