BOM : Java Glossary

BOM

BOMs (Byte Order Marks) are special characters at the beginning of a Unicode file to indicate whether it is big or little endian, in other words does the high or low order byte come first. These codes also tell whether the encoding is 8, 16 or 32-bit. You can recognise Unicode files by their starting byte order marks, and by the way Unicode-16 files are half zeroes and Unicode-32 files are three-quarters zeros.

UTF (Unicode Transformation unit) BOM (Byte Order Mark) (Byte Order Mark) Unicode-encoding Endian Indicators
0xfeff BOM as it appears encoded	Description
ef bb bf	UTF-8 endian, strictly speaking does not apply, though it uses big-endian most-significant-bytes first representation.
fe ff	UTF-16 for 16-bit internal UCS-2, big endian, Java network order
ff fe	UTF-16 for 16-bit internal UCS-2, little endian, Intel/Microsoft order. Note you must examine subsequent bytes to tell this apart from a UTF-32 BOM since they both start ff fe.
00 00 fe ff	UTF-32 for 32-bit internal UCS-4, big-endian, Java network order
ff fe 00 00	UTF-32 for 32-bit internal UCS-4, little endian, Intel/Microsoft order.

The actual Unicode character encoded in all cases is 0xfeff.

There are also variants of these encodings that have an implied endian marker.

Unfortunately, often applications, even Javac.exe, choke on these byte order marks. Java Readers don’t automatically filter them out. There is not much you can do but manually remove them.

Avoiding BOM s

How can you get rid of these pesky BOM s? Here are ideas, the ones I consider best/simplest near the top.

Use UTF-8. It does not use them. You can use native2ascii.exe to convert your given encoding to UTF-8.
Write a utility that reads the first character of a file. If it is a BOM, copy the rest of a file to a temp file, then delete the original and rename the temp to the original, effectively permanently chopping off the leading BOM. Unfortunately, this discards the useful information about the encoding of the file. It will be more efficient if you don’t use Readers, but use byte-based InputStreams instead.
Write an encoding UTF-16HIDEBOM that wraps itself around UTF-16 and install it as one of the official encodings.
Write a FilterInputStream that discards BOMs. And use it in your apps.
Lobby Sun to provide a solution.
Look for the character in your application code, and ignore it. This technique is very clumsy and will seriously interfere with your application logic.

TestBOM

This program tests how Java handles BOM s. It discovers than Java never inserts BOM and it never removes them on its own. You have to bypass, insert and delete them explicitly.

encoding
encoding recogniser
native2ascii.exe
Unicode
UTF

	available on the web at:	http://mindprod.com/jgloss/bom.html
	optional Replicator mirror of mindprod.com on local hard disk J:	J:\mindprod\jgloss\bom.html
	Please email your feedback for publication, letters to the editor, errors, omissions, typos, formatting errors, ambiguities, unclear wording, broken/redirected link reports, suggestions to improve this page or comments to Roedy Green : . If you want your message, your name or email kept confidential, not considered for public posting, please explicitly specify that. Unless you state otherwise, I will treat your message as a letter to the editor that I may or may not publish in the feedback section. After that, it will be too late to retract it. If you disagree with something I said, especially when sending an ad-hominem attack, a rant composed mainly of obscenities or a death threat, please quote the offending passage and cite the web page where you found it, tell me why you think it is wrong, and, if possible, provide some supporting evidence. I can’t very well fix erroneous or ambiguous text if I can’t find it.
Blog	Canadian Mind Products IP:[65.110.21.43] Your face IP:[54.224.75.101]
Feedback	You are visitor number 26,951.