Shared File System Project Feasibility


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Shared File System Project Feasibility
# 1  
Old 05-16-2011
Shared File System Project Feasibility

I would be taking my masters degree project next year. So I am now in the process of thinking something that could help me get more in depth knowledge and will be a useful one in practice.

I have thought about creating shared file system for clustered environments (like GPFS,GFS).

I have below doubts before taking this idea any further.
1. Is it feasible to complete it in one year if I start some home work now ?
2. Is there a way to reuse any code from linux ? (the one from GFS)
3. Any documents to understand the implementation of any existing shared file systems .
4. Best book to start for file system implementations.
Please post your thoughts. That would guide me to choose the better option.

Thanks.
# 2  
Old 05-16-2011
Well, it is hardly worthwhile unless you have some extension or innovation to apply. After you dissect the open source of a SOTA SFS, you may find some ways to make it lower latency or higher bandwidth. Perhaps, using cluster-friendly networking like broadcast UDP. You could add in compression, perhaps somewhat like zip subtrees. You could make it a distributed peer file system aggregating a heterogenous ix of disks on all hosts of the cluster, with dynamic mirroring, where extra copies are made if a host disappears, and multiple copies exist for low churn high use files, or hybrid mirrroring, where one side of the mirror is compressed and the other takes the query load. Of course, a year is sufficient, depending on you anf your schedule.
# 3  
Old 05-18-2011
Thanks for you expert view. Though it would be hardly possible to match the performance of the commercial products, at least this would give me the understanding i am looking for. And innovation has to be found only after understanding the thing.

Can you suggest any book that could help me understand the file system implementation or any material about this topic.
# 4  
Old 05-19-2011
I'd get one running and look at the open source. Book writing is big business, but code and playing tells you more. Also, there are many good pieces on open source products online. Get started first, and buy the book if you find you need help.

I also fancied a UFS variant where directories are trimmed, maybe even organized for hash or tree search. Another idea is a dir that can hold small inodes, files and directories within them, recursively, with dynamic expansion and initial/utility compression. Saves both inodes and pages. All invisible to old software, give or take a funny inode #.

Compression like zip is the thing that really fascinates me, as zip seeks to the compressed file you want, and as CPUs are faster than disks, the flow rates through zip may exceed most disk speeds. You could even chop a file into super-pages and compress each separately, for easier update and seek time within the file. Integration with the VM would allow all the decompressed file or superblock to be cached to VM. It could be rewritten back to disk as a redundant uncompressed copy in case of more query.
This User Gave Thanks to DGPickett For This Post:
# 5  
Old 05-20-2011
Thanks for you tips here. I'll first start before getting deep into to optimizations :-)

Thanks again for your valuable tips.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Setting UID and GID for shared folder using NFS method in Linux system

Hi everyone, have a good day to you. I am trying to use NFS to share a folder between 2 linux systems. Let's say the server which is sharing the folder is server A and the client which need to access this shared folder is server B. In server B, i am having a Joe user which UID and GID is 500.... (1 Reply)
Discussion started by: michael_hoang
1 Replies

2. Shell Programming and Scripting

feasibility of opening a website link from unix and get a response in the form of xml or html

i just wanted to know whether is it possible to open a website link and get a response in the form of xml or html format... the website is of local network... for example something like this wget http://blahblah.samplesite.com/blachblahcblach/User/jsp/ShowPerson.jsp?empid=123456 ... (2 Replies)
Discussion started by: vivek d r
2 Replies

3. Shell Programming and Scripting

Shared File system- lib access issue

I have a C++ binary executable installed in a file system which is shared across multiple solaris boxes. When I start this executable from one of the boxes,I am able to start only 4 parallel instances and from the 5th instance onwards I am getting the following error. fatal: libdb2.so.1:... (2 Replies)
Discussion started by: prasperl
2 Replies

4. Solaris

what is the use of /etc/project file and project administration commands?

i have two doubts.. 1. what is the use /etc/project file. i renamed this file and when i tried to switch user or login with some user account the login was happening slowly. but when i renamed it to original name it was working fine... why so? 2. unix already has useradd and grouadd for... (4 Replies)
Discussion started by: chidori
4 Replies

5. Programming

Building an operating system for senior project?

Hey guys! this is my first post on this site. I was hoping you guys could help me out. I am going to be a senior this year and I want to build an operating system from scratch or at least do as much of as I can by myself. I want to build nothing big like ubuntu but something smaller looking that... (10 Replies)
Discussion started by: Hoonathan
10 Replies

6. AIX

changing values for nfs shared file system on aix

Hi, I want to change the values for shared file system in aix for that I have run the command smitty chnfsexp but I am not getting the all the values which I have seen while adding the file system while exporting example smitty chnfsexp but after selecting shared file system using F4... (3 Replies)
Discussion started by: manoj.solaris
3 Replies

7. UNIX for Dummies Questions & Answers

access collision with shared file system

Hello ALL, In my system, there are 14 machines running the same version of Linux RHEL4. The 14 machines use a NFS file system, i.e., a shared file system. My question is that if the programs in individual machines can access a common file simutaneously. Or, they have to access the file... (1 Reply)
Discussion started by: cy163
1 Replies

8. UNIX for Dummies Questions & Answers

Tell me some good unix project in system level programming...

this is very urgent plzzzzzzzzzzzzzzzz........... (0 Replies)
Discussion started by: stalin2020
0 Replies
Login or Register to Ask a Question