NFS with a NAS: permanently inconsistent directory state across clients
Hi,
I am having some NFS directory consistency problems with the below setup on a local (192.) network:
1. Different permissions (chmod) for the same NFS dir are reflected on different clients.
2. (more serious) an NFS dir created on client1 cannot be accessed on client2; this applies to some directories, not others; when this problem applies to a directory, it is a consistent problem.
Setup:
NFS server: Thecus N8800, 16Tb raw, RAID6
Client1: Sun Fire V210, Solaris 5.10 Generic_139555-08
Client2: Sun Fire V100, Solaris 5.10 Generic_118822-23
Both clients nfs-mount. Flags: vers=3,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
Attr cache: acregmin=3,acregmax=60,acdirmin=30,acdirmax=60
Use case on Client1:
CD to an nfs subdir:
Code:
cd /.../nfsdir
ls -la
drwxrwx---+ 56 user group 12288 Mar 13 15:28 .
drwxrwx---+ 3 user group 30 Mar 17 13:57 ..
drwxrwx---+ 3 user group 53248 Oct 3 04:41 somedir1
drwxrwxrwx+ 7 user group 4096 Mar 13 15:29 somedir2
All good. CD to somedir1 works; can LS and see files. Same for somedir2. Note: somedir2 was mkdir'ed on Client1.
Use case On Client2:
CD to the same nfs subdir. Listing files works, but the permissions are different than what is listed on Client1:
Code:
cd /.../nfsdir
ls -la
drwx------+ 56 user group 12288 Mar 13 14:28 .
drwx------+ 3 user group 30 Mar 17 12:57 ..
drwx------+ 3 user group 53248 Oct 3 04:41 somedir1
drwx---rwx+ 7 user group 4096 Mar 13 14:29 somedir2
PROBLEM1: the group permission for the same dir is different on client1 vs. client2.
CD to somedir1 works; can LS and see files. PROBLEM2: *cannot* CD to somedir2:
Code:
bash: cd: somedir2/: Not a directory
On both clients, 'group' is defined in /etc/groups with the same id; 'user' is defined in /etc/passwd with the same id.
When I un-mount and re-mount the nfs dir on client2, I am able to access the directory in question (subdir2). Permissions, however, are still different across clients.
Does anyone have suggestions as to what is going wrong with my NFS setup? I'll be happy to post more information.
NFS server is a Thecus N8800 Network Attached Storage. It runs Linux and is configurable via a web interface. Unfortunately, I don't see much useful info via the web UI on NFS status. I also have not tried rooting the box.
"nfsstat -m" on the client is in my original post.
Umm, no you didn't provide nfsstat data in your original post:
Quote:
Originally Posted by cosmojetz
Hi,
I am having some NFS directory consistency problems with the below setup on a local (192.) network:
1. Different permissions (chmod) for the same NFS dir are reflected on different clients.
2. (more serious) an NFS dir created on client1 cannot be accessed on client2; this applies to some directories, not others; when this problem applies to a directory, it is a consistent problem.
Setup:
NFS server: Thecus N8800, 16Tb raw, RAID6
Client1: Sun Fire V210, Solaris 5.10 Generic_139555-08
Client2: Sun Fire V100, Solaris 5.10 Generic_118822-23
Both clients nfs-mount. Flags: vers=3,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
Attr cache: acregmin=3,acregmax=60,acdirmin=30,acdirmax=60
Use case on Client1:
CD to an nfs subdir:
Code:
cd /.../nfsdir
ls -la
drwxrwx---+ 56 user group 12288 Mar 13 15:28 .
drwxrwx---+ 3 user group 30 Mar 17 13:57 ..
drwxrwx---+ 3 user group 53248 Oct 3 04:41 somedir1
drwxrwxrwx+ 7 user group 4096 Mar 13 15:29 somedir2
All good. CD to somedir1 works; can LS and see files. Same for somedir2. Note: somedir2 was mkdir'ed on Client1.
Use case On Client2:
CD to the same nfs subdir. Listing files works, but the permissions are different than what is listed on Client1:
Code:
cd /.../nfsdir
ls -la
drwx------+ 56 user group 12288 Mar 13 14:28 .
drwx------+ 3 user group 30 Mar 17 12:57 ..
drwx------+ 3 user group 53248 Oct 3 04:41 somedir1
drwx---rwx+ 7 user group 4096 Mar 13 14:29 somedir2
PROBLEM1: the group permission for the same dir is different on client1 vs. client2.
CD to somedir1 works; can LS and see files. PROBLEM2: *cannot* CD to somedir2:
Code:
bash: cd: somedir2/: Not a directory
On both clients, 'group' is defined in /etc/groups with the same id; 'user' is defined in /etc/passwd with the same id.
When I un-mount and re-mount the nfs dir on client2, I am able to access the directory in question (subdir2). Permissions, however, are still different across clients.
Does anyone have suggestions as to what is going wrong with my NFS setup? I'll be happy to post more information.
Thanks a lot!
What's the inode number of the problem directories and/or files?
I suspect the problem is with the server - Solaris is VERY particular about NFS implementations being EXACTLY according to the NFS spec.
Can a 64-bit process access the files/directories?
Ensure the mount points are directories with 755 permission! (umount to check.)
That too.
I see that nfsstat data and I see non-zero values in NFSv2 stats. Is the server Linux-based? Or Irix-based - nothing like inode numbers greater than 2 gig - the files are invisible to 32-bit applications running on Solaris since an inode number above 2 gig violates the specs for a 32-bit system.
The inode number of the problem file in question is: 1080094
Yes, the NFS server is Linux-based.
Re. ensure mount points are 755: yes, they are; otherwise I wouldn't be able to create any directories/files. Note that the problem occurs only with some newly created directories, but when it does occur, it is consistent (directory can never be read).
Regarding the other problem (perhaps related): even with freshly mounted nfs dirs, the two clients show different group permissions for all files/dirs:
Example of same dir:
Client1:
drwxrwx---+ 58 user group
Client2:
drwx------+ 58 user group
Both user and group are defined with same ids in /etc/passwd, /etc/group. What config might be faulty to warrant this behavior?
Good Morning,
I have 2 Solaris 9 machines sharing a NAS, and need to have users to be able to log in from the 2nd machine and get to all of their files on the NAS that were created on the 1st machine.
So far its working ok, but when users log in to the second machine, their user IDs show... (20 Replies)
I ssh in and am trying to add a directory permanently to $PATH in centos 7 and having issues. My current $PATH is
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
but when I do a sudo nano ~/.bashrc
# .bashrc
# User specific aliases and functions
alias... (7 Replies)
i've used only NFS and as many already found out, it can be or rather i should say, it is very unreliable.
based on the collective experiences of the members on this board, i would really appreciate it if someone can tell me what the next best file sharing method is?
is it DAS? SAN? NAS?
... (1 Reply)
Hello World,
We have a software repository server in our environment which we use as an NFS server.
Now this has been going on well before I was hired. Now, I observed many users not unmounting the NFS resources after their use. I ran showmount and it showed 513 current sessions.
:wall: Is... (7 Replies)
Hello;
I work now with a team which has based her "applications" on the following schema:
1 NFS client C1 (linux redhat 5) writes a file fic.dat on a NFS SERVER S (RH 5)
Another NFS client C2 is waiting for the same file fic.dat (on NFS server S1), and, when "fic.dat" appears, then makes... (8 Replies)
The UPS connected to the Disk Array portion of my Windows 2003 NAS burned up over the weekend. Reconnected it to a new UPS and re-booted the NAS box. Since then I have not been able to get my HPUX 10.2 box to mount the shared drives on the NAS. At boot, the NFS client & server subsystems do a... (0 Replies)
Hey guys,
My servers run Solaris 10. I have to move data from my NFS mount drives into the NAS storage area with all the same permissions ( for the users and programs and everything else).
Can any one help me please ??
Thanks.
MisterKhan (0 Replies)
