HTTP : Java Glossary

Message Headers From Browser To Server

Beware using HttpURLConnection.setFollowRedirects( false); This reportedly causes trouble in recent JDKs. When it is set true, it will not automatically follow responses with: <META HTTP-EQUIV="Refresh".

Message Headers From Server To Browser

In the real world, the conversations between browser/client and server are much more complicated as slipshod than you might suppose. Each query often results is a flurry of permanent and temporary redirects back and forth. Each element on an HTML page must be requested independently. Sometimes servers will send back a fail error code, then send the page anyway. Or they will send a 404 with an OK text response code. Sometimes servers refuse HEAD requests, but accept the equivalent GET. Sometimes servers send back https: in response to an http: request. Sometimes servers give you a totally different page from the one you requested and don’t tell you the one you wanted is on longer available. Sometimes servers rediret to localhost, or send back gibberish messages. Sometimes a server won’t send you a page if you have recently previously requested it. They expect you to have cached it. Browsers just do their best to muddle through. When you start emulating browsers with code, you get pretty flaky programs.

Language and Charset

<!-- embedding language and charset inside an HTML document -->
<meta http-equiv="Content-Language" content="en">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

Sample Code

Code to do a GET

Code to do a POST

Code to do a HEAD

Code to do a Fetch (generic GET)

Code to do a PROBE (check if page present)

Base class for GET, POST, and PROBE

Code to READ

Under the Hood

1. URL.openConnection just sets up a place to build the HTTP header. It does no communicating with the outside world.
2. HTTPConnection.connect() requests sending the header to the server.
This request triggers opening a TCP/IP socket connection to the server. This is done by sending a SYN connection request packet. The server sends back an SYN+ACK. Then the client sends an ACK, upon which may be piggybacked some data.
3. This triggers sending the GET header composed of all the header fields set up before the .connect call. The GET request header includes a list of the encodings and compression algorithms the browser would like in response. .connect does not return until the HTTP header is safely sent out the wire.
4. The browser calls HTTPConnection.getResponseCode to see if request went ok. This blocks until the server responds with an HTTP response header.
5. Then the browser calls HTTPConnection.getInputStream and reads the text of the message from the server containing the requested web page. Using the standard TCP/IP protocol flow-control features, the server sends data only as fast the browser can read it.
6. The browser then scans the web page for the URLs of embedded images and puts out GET requests for them.
7. Then various images usually come back from the server on the original socket. The browser could elect to request each image on it own socket so they can arrive simultaneously.

Speeding Up HTTP

• gzip (compress) the responses from your server to the client. You might pre-compress the pages if you have no dynamic content even trivial SSI. Otherwise, you are putting a CPU burden on your server. Modern servers have plenty of spare CPU cycles, so even compressing on the fly will not likely be too onerous a burden. For this to work, the browser must send an Accept-Encoding: gzip in the request header. You also need to provide the uncompressed version too for older browsers (about 15% of browsers extant) that don’t announce they accept gzip.

  IE 5 & 6 had a bug involving javascript and gzip encoding. Microsoft failed to change the User-Agent string after the bug fix, and thus the server does not have enough information to determine whether or not the browser is defective.

• Compact the responses removing unnecessary white space from your HTML. See Compactor
• Strip out the HTML comments.
• Transfer-Encoding header: chunked. Use this when you don’t know in advance how long a long response will be, e.g. a database query result. You write the data in chunks, each preceeded by a length with a 0-length chunk as end-of-message marker.
• Keep-Alive
• Cookie scoping.

Learning More

RFC 2049 MIME part 2, non ASCII.

RFC 1945 HTTP 1.0 specification.

RFC 2045 MIME Part One: Format of Internet Message Bodies, specifies the various headers used to describe the structure of MIME messages.

RFC 2046 MIME Part Two: Media Types, describes the general structure of the MIME media typing system and defines an initial set of media types.

RFC 2047 MIME Part Three: Message Header Extensions for non-ASCII text

RFC 4288 and RFC 4289 MIME Part Four: Registration Procedures

RFC 2049 MIME Part Five: Conformance Criteria and Examples, Provides some illustrative examples of MIME message formats

RFC 2183 MIME Part Five: Conformance Criteria and HTTP Content-Disposition

RFC 2616 updates the HTTP protocol

RFC 2617: for details on how to send username and password in http headers to restrict access

RFC 2183 MIME Part Five: Conformance Criteria and HTTP Content-Disposition