buffer : Java Glossary

Oveview

If you wrote a byte at a time, since the hardware works in 512-byte sectors at a time, the OS (Operating System) would need to read the sector, plop your byte into it and write the entire sector back. This would take at least 2 disk rotations, perhaps 3. Even if you wrote your data 512 bytes at a time, when you went to write the next sector, its spot would have just past the head, so you would have to wait an entire rotation for its spot to come round. If you wrote 131,072 bytes (still less than 1 physical track) at a pop, you could do that all in one rotation.

Ideally, if you have enough RAM (Random Access Memory), you do the I/O in one whacking huge file-sized unbuffered chunk. Java has a number of classes that let you process a file buffered in convenient small logical chunks, often line by line. The buffered classes transparently handle the physical I/O in bigger chunks, typically 4096 bytes. The classes store each large chunk for physical I/O in a separate piece of RAM called a buffer. Unless the buffer size for the physical I/O is at least twice as big as the size of the logical chunks you process, there is not much point in buffering. The extra buffering copying overhead will just slow you down.

The File I/O Amanuensis will teach you how to do I/O either buffered or unbuffered. You can try it both ways and see which works faster. You can also experiment with buffer sizes. The bigger the buffer, the fewer the physical I/Os you need to process the file. However, the bigger the buffer, the more virtual RAM you will use, which may trigger more swapping I/O. Further, there is not much point in having a whacking big buffer for a tiny file. It will take only a few I/Os to process the file anyway.

You will find that buffer sizes that are a power of two tend to work faster than other sizes. This is because disk and RAM hardware are designed around some magic sizes, typically 256, 512, 1024, 2048, 4096, 8192, 16,384, 32,768, 65,536, 131,072 and 262,144 bytes. Buffers that are powers of two naturally do I/O in physical chunks that align on powers of two boundaries in the file. This too makes the I/O more efficient because the hardware works typically in 512 byte sector chunks. If you do unbuffered I/O, likewise try to start your I/Os on boundaries that are even multiples of some power of two, the higher the power of two the better, e. g. it is better to start I/O on boundaries that are even multiples of 8096 rather than just 128. Sometimes it pays to pad your fixed-length records up to the next power of two. If you can help it, arrange your logical record size and buffer size so that logical records are aligned so that they never (or rarely) span two bufferfulls. It also helps to have your buffers aligned on physical RAM addresses that are even powers of two as well, though you have no control of that in Java.

In the olden days, CØBØL programs used double buffering. They used two or more buffers per file. The computer would read ahead filling buffers while the program was busy processing one of the previous buffers. Oddly, Java does not support this efficient serial processing technique, though sometimes the operating system maintains its own private set of read-ahead buffers behind the scenes. Unfortunately, the OS ’s cascaded buffering is less efficient than using a single layer. You have the overhead of copying plus the wasted RAM for the buffers that are not actually used for physical I/O. Java never has more than one buffer per file and hence cannot simultaneously process and do physical I/O, unless, of course, it uses Threads. Even with Threads, you can’t pull off double buffering with any ease.

The term double buffering also refers to a technique of constructing Images off screen then blasting them onscreen once they are complete, as a way of creating smoother animation.

If you wrote 128K a byte at a time using a 64K buffer there would be only two physical 64K I/Os. This would be slightly slower that using unbuffered I/O to write the entire 128K in one I/O because of the extra physical I/O, the RAM overhead for the buffer and the CPU (Central Processing Unit) overhead of copying the data to the buffer.

When To Buffer

• To process a file whole file a time, read the entire file in one giant unbuffered I/O.
• If a file is too large to process all in RAM, read it buffered and process it a chunk, line, field or char at a time.
• To copy files or download streams use the FileTransfer class which reads unbuffered a large chunk at a time.
• If you need the readLine method, you must use buffering.

Selecting a Buffer Size

• How big are the records, the bigger the records, the bigger the buffer.
• How many records are you planning to read? The more the bigger the buffer you can afford?
• How likely is the data to be in the OS buffer or hard disk firmware buffer? The better caching works, the less buffering you need to do. The more space you have the better caching works. The more other works going on at the same time, the less space will be available for your data.
• How much other i/o is going on, yanking the disk arms way from your data? The more, the bigger the buffers you want to avoid wasting disk seeks and rotational latency to get to your data.
• How much free RAM do you have. The more you have, the bigger the buffers you can afford.
• There are so many factors interacting, all can really do is tweak the buffer sizes and see if performance degrades or improves.

1. It first looks in the Java buffer.
2. If it is not there, it asks the OS to fill the Java buffer. This has surprisingly high CPU overhead. That is why Java keeps its own buffer.
3. The OS looks in its cache. If it not there, it schedules an I/O to the data, (plus some read-ahead sectors for sequential access).
4. The disk looks in its hardware cache. If it not there the disk seeks to the correct track and waits for the data to spin around under the read head.
5. It then reads the data over the SCSI/SATA etc. channel into the OS disk cache buffers.
6. From there it is copied into the Java buffer.
7. From there it is processed to become a Java primitive or object.

BufferedWriter vs BufferedOutputStream

When you use a Reader or Writer should you do your buffering with BufferedWriter/BufferedReader, BufferedOutputStream/BufferedInputStream or both? It turns out when you use BufferedWriter or a BufferedReader, you can improve performance by adding a BufferedOutputStream or BufferedInputStream into the chain.

When you use the Sun JVM (Java Virtual Machine), you should use both a BufferedWriter and a BufferedOutputStream with 25% of your buffer space allocated to the BufferedOutputStream. Keep in mind the BufferedWriters measure buffer space in 16-bit chars and BufferedOutputStreams measure them in 8-bit bytes. Further, you should use both a BufferedReader cascaded on a BufferedInputStream with 50% of your buffer space allocated to the BufferedInputStream. Keep in mind the BufferedReaders measure buffer space in 16-bit chars and BufferedInputStreams measure them in 8-bit bytes.

When you use Jet, you should use both a BufferedWriter/BufferedReader cascaded onto a BufferedOutputStream/BufferedInputStream with 75% of your buffer space allocated to the BufferedOutputStream/BufferedInputSTream.

Here is a benchmark I used to discover these rules of thumb.

Squirrel Cage Buffers

You have probably seen those wheels used to exercise pet rodents. Think of a buffer than works analogously allocating data around the inner rim. The next allocation is always done where the squirrel’s feet are touching the rim. A buffer that works the same way is useful when you want to maintain some history on a stream so that you can backtrack, but you don’t want to keep the entire history. You can also use such a buffer to implement a FIFO (First In First Out) queue. In the simplest case, you would have large array of bytes as your buffer. You write to the buffer until you hint the end, then you start over at the beginning. You keep track of where in the buffer your data start and end. That way you can scan backwards if needed. Old data are overwritten. In a queue, you don’t overwrite data until it has been processed. You use modulus to compute the slot to use.

In Java, your squirrel cage buffer would more typically be an array of references to your entries. When you overwrite, the objects corresponding to the overwritten references will be garbage collected. You’d think this common algorithm would have been implemented by some well-known Collection. If there is, I don’t know about it.