Rails 3.1 is going to support HTTP streaming, aka chunked responses, this post explains what’s all about.
What Is HTTP Streaming?
Ordinary dynamic HTTP responses need a Content-Length header. Their timeline look like this:
HTTP request -> dynamic content generation -> HTTP response
Those are three serial steps because normally you need to generate the content in order to be able to know its size, and thus fill the Content-Length header of the response.
HTTP provides an alternative to this schema to be able to flush data as it is produced, known as chunked transfer encoding. That’s what we are referring to as streaming in recent commits.
Streamed responses have no Content-Length header. Rather, they have a Transfer-Encoding header with a value of “chunked”, and a body consisting of a series of chunks you write to the socket preceded by their individual sizes. Modulus details.
This is an example taken from Wikipedia:
HTTP/1.1 200 OK
This is the data in the first chunk
and this is the second one
Point is, you are able to flush chunks to the socket as soon as you have them, no need to wait for the whole thing to be generated.
When Do Web Browsers Fetch Assets?
Web broswers parse documents as their content is received. When they find an asset referenced, think an image, stylesheet, or script, a request to fetch them is fired. That happens in parallel while the document is being received and processed, no matter whether the content comes chunked or not.
Browsers have limits on the number of concurrent requests they are allowed to do, a global one (typically +30), and another per domain (nowadays typically 4 or 6), but within those limits, requests for getting assets happen as the content is parsed.
Trivia: While investigating this I discovered by accident that if the MIME type is unclear, for example “text/html” without an explicit charset, then web browsers buffer 1 KB of data firing no asset requests to be able to peek at the content and do an educated guess.
So What’s The Benefit Of Streaming?
Streaming is still being polished for Rails 3.1, expect another post in the future covering its practical aspects in Ruby on Rails applications.
Tony Gentilcore provided his insider’s guidance into this, thank you very much Tony! Also, thanks a lot to the Browserscope project for their really useful tables.