04-10-2013
4,673,
588
Join Date: Oct 2010
Last Activity: 1 February 2016, 3:35 PM EST
Location: Southern NJ, USA (Nord)
Posts: 4,673
Thanks Given: 8
Thanked 588 Times in 561 Posts
Wel, the SAN is on some sort of Network and provides Files in a System, so it is just a variant of NFS. Do not let labels distract you from the need to commit writes to disks not directly connected to your host, and to sense changes others may have made in those remote files, if they are accessible from many hosts.
Now, with the model of the Client file system, a commit to a local disk is a secure message that will be delivered to modify the remote disk, so it can be the end point of a commit. Of course, you cannot have others modifying that area, but there should be dedicated areas for each upstream host if it is a Commit space. Once a failed host or network is restored to operation, those 'dirty to NFS/SAN' pages can repatriated with no problem. While the network or NFS/SAN is down, the Client host can still read files and write files on the local cache, but will lock up if it needs to read, write, status or lock a remote file. A robustly structure system will find a way to save the state of processes locked up so they can survive local outages. Some operational files and databases could be on redundant local disk. Of course, if the clent host system is utterly lost without flushing to the remote store, those changes are lost. You can't get blood from a stone. You can define each class of risk, give it a reasonable probability, and come up with a recovery plan for each class.
Some shops have realized that an insert only serial append table structure allows them to identify ranges of change lost, so they know when orphan references show up. Such a structure can be though of as 100% history on all data. Disk is cheap, humans make lots of mistakes and mistakes are expensive.
I used such a pragma in a RAD app I developed for wafer fab training records. Sometimes I had to educate people that when a person left a role, they did not die (logically delete the person), they just lost the role. Eventually, the load of history became a burden, so we came up with administrative delete and update, but in such a way that even those actions, if a mistake, were reversible. I think it had 8 tables: students+teachers, courses, roles and training events, plus admin archive shadow tables of each. Template rows held hierarchical pieces, like departments, roles in departments, courses required in roles. Clustering kept history records for department/role/course/student adjacent for fast detection of status rows (latest) and active rows (latest is of active status) in views and subqueries.
Rather than buy a box, you can buy a cloud service. Let the RDBMS be out in the cloud. It is an appliance, after all. High speed fiber and an insert only database go well together: latency is a low priority. You send a query, you get a stream of rows. Your churn is a stream of rows in the reverse direction. Transactions just tell you the data arrived, are not necessary to exclude other updates, as there are no updates. Last row wins. You might want a lock of some sort at some level to keep two apps from inserting for the same keys at the same time, where row a fathers two new versions b and c, but as c is later it undoes all b changes.
