We have umask defined under /etc/.login as 022.
I have my user specific umask defined in /userhome/.login as 002.
I understand ftp will not execute anything at destination, it simply transfers files.
But it seem to be using 022 as umask for the files transferred. How does ftp knows what umask to use, where does it get from on Solaris?

-rw-r--r-- 1 user group 2012 Jan 9 09:59 file.dat

Check this :
Synopsis - man pages section 4: File Formats

You will need to define a class of user(s) and defumask, or just defumask.

peasant thanks for point me in the right direction. I read that page, its not clear to me what the syntax is to create a class and define defumask.
I see that we have "/wh/etc/ftpd/ftpaccess".
If I have to allow group "allusers" to have 775 on folders and 664 on files transferred by uses that belong to "allusers" group, how would that look like?

I have tried this with default solaris 11 install, but it doesn't work.

Seems like the default service proftpd with it's configuration file /etc/proftpd.conf
So you will need to modify that configuration file and add this or similar directive :

<Directory> /export/home/test>
Umask 007 007
AllowOverwrite on
</Directory

Peasant,
I came across two possible solutions to this,, but could not try it yet as it requires root access and better understanding of how this works on solaris 8.

1.
use of defumask to control file permissions discussed here,,
NexentaStor Project - ftpd defumask - NexentaStor Project
it would be nice to get more explanation of how this works and how to do this for a specific user group or folder structure.

2.

# second one is possibly adding '-u 002'(default umask to ftpd to have 002 !?) to following entry to /etc/inetd.conf.
# But this may have unintended results as it may apply this to all ftp related uploaded files/folders on to the server.
# Since a server has multiple applications running and owned by different groups. how can we control this for a specific application user or group?
ftp stream tcp nowait <user/group?> /usr/sbin/tcpd /wh/sbin/in.ftpd -l -a -u 002