Persistent Object Database

This essay is about a suggested student project in Java programming. This essay gives a rough overview of how it might work. It does not describe an actual complete program. I have no source, object, specifications, file layouts or anything else useful to implementing this project. Everything I have to say to help you with this project is written below. I am not prepared to help you implement it; I have too many other projects of my own.

I do contract work for a living, which could include writing a program such as this. However, I don’t do people’s homework for them. That just robs them of an education.

You have my full permission to implement this project any way you please.

A persistent object database lets you do two things:

Code as if you had unlimited amounts of virtual RAM for your objects.
Shutdown and restart a program without losing your objects.

There are a number of commercial persistent object databases. The catch is they are all very expensive. The world needs a cheap or free POD (Persisten Object Database) that can be used by students, demos, small businesses and for prototyping large projects,

How could you write a very simple one? Start by implementing the Hermit Crab variable length file class. Use ordinary serialisation to save and restore objects. You can retrieve objects from the hermit crab file by serial number.

Basically you are simulating a segmented virtual memory system with an LRU (Least Recently Used) cache of objects. You need an array indexed by object number. With that array you can find out if the object is currently in RAM, and if so where in RAM it is. If it is not in RAM, you can use the object number to retrieve it. You also have to keep track if each RAM-based object is clean or dirty. Dirty means it has been changed, and eventually the disk copy will have to be updated. The simplest way to implement an LRU is to maintain a chain. Every time an object is touched, you move it to the head of the chain. The object on the tail end of the chain is then the least recently used.

Ideally you would code your application that used the POD as if the POD were not there. Some commercial PODs automatically insert code in your application to help give that illusion. The trickiest part of this project involves techniques to make the POD as transparent as possible. You might want to study some commercial PODs to get some ideas on how to handle this.

Let’s look at some of the problems:

If you get too many objects in RAM, you have to get rid of some, writing them to disk if dirty. There must be no pointers to those objects before they are released. The only pointers permitted to these persisted objects are object serial number style pointers, which will still work when the object is no longer in RAM. You peel them off from the end of the LRU chain.
If you put code in a getXXX method to ensure an object is RAM-resident, you still need at least the shell of an object pre-existent.
If you have grab and release methods for each object, you must deal with the problem of multiple nested grabs for the same object, and for failure to have a release for every grab. In a way you are back the bad old days of C++ manual memory allocation.
You might want to get clever and have an automatic release done when a module exits for all the grabs it did. That requires tweaking the stack, which is JVM dependent.
If the system crashes, you will lose everything. For a commercial system, you need transactions that are guaranteed to be applied in toto, or not at all, even if the system crashes.

Another totally different approach would spin off some independent jobs and use inter-task communication. You create as big a metavirtual VM as you need by spinning off additional jobs. You serialise the entire thing to disk only at startup and shutdown. A simple version would keep all the objects in a single VM, perhaps even the main VM.

You might like to study a commercial POD before attempting to write your own. See Objectstore and Poet.

	Please email your feedback for publication, errors, omissions, broken/redirected link reports and suggestions to improve this page to Roedy Green :
	Canadian Mind Products
	mindprod.com IP:[65.110.21.43]
	Your face IP:[38.103.63.16]	The information on this page is for non-military use only.
	You are visitor number 3,493.	Military use includes use by defence contractors.
	You can get a fresh copy of this page from:	or possibly from your local J: drive (Java virtual drive/Mindprod website mirror)
	http://mindprod.com/project/pod.html	J:\mindprod\project\pod.html