What if I told you that you could create a API backend that didn’t require any code? Crazy right? Wrong!
The first approach doesn’t scale. Every new saying is more for the client to download even though it needs exactly one. Not to mention that a quick View Source removes all of the mystery from your app.
The second can scale out to infinity and beyond, but now you need to code a backend, to create and maintain infrastructure. Now your afternoon project is real work.
What we need is a way to get one joke from the server without having to download them all and without having to spin up an API.
Enter HTTP Range
The format is quiet simple
The first number (zero indexed) offset of the starting byte, the second the end’s offset (not the size! I repeat, not the size!). If a web server supports HTTP Range then just that chunk of bytes will be returned (otherwise the header is ignored and entire file is returned as it normally would be).
While it rarely comes up for most developers, it’s what allows you to skip around when streaming audio or video. Because of that, byte serving is supported by most major web servers and many caching technologies.
Doing Something with Range
So, how does the help us? Well, in our witty example, if our sayings were in a file on the server and we knew the starting and ending offset of each one, we could use the following jQuery to read exactly one:
1 2 3 4 5 6 7 8 9 10 11 12 13
This presumes ‘sayings.json’ exists on the server and that ‘36-80’ is the range of a saying. How would we set that up? Given a text file like:
1 2 3 4
The following Ruby code will create two JSON files:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
sayings.json is the list of sayings in a JSON formatted array. We format the file as a JSON array so that if the server doesn’t support HTTP Range, we still get back valid data.
sayings-index.json is an array of the byte range of each line in the file. To create it we use File#pos to record the offset in the file before and after we write each line.
What’s with that magic
f.pos - 3 in the code? First “f.pos” returns
the position it’s going to write the next character to (-1), second,
we have the “,” needed to make the JSON valid (-2), and third, we have to
account for the line feed at the need of the line (-3). Thus, subtracting 3
from the position moves us back to the closing quote on the string.
The Codeless API
Once you have the files, your page can load the index and then a randomly selected saying:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Is it fast?
This is actually a very efficient way to fetch data. It boils down to two very low level syscalls:
Implementations vary, but if the web server caches the data it can be reduced to an even simpler memory look up. Something like:
A Familiar Pattern
While the pattern of finding data using a seek offset isn’t something most developers commonly run into, it’s one they take advantage of everyday. Pretty much every database engine there is uses it. In fact you could describe a database as an attempt to find the most efficient way to reduce a query to a seek offset.
Congratulations! You’ve just implement your own database using the HTTP protocol.
Next up, we’ll look at serving something more that just lines of text.