Previously while talking features of the HTTP Range header I wrote:
Last and likely least , you can read the last N bytes of a file by requesting a negative offset: […] Honestly, I’ve never come up with a use case for that.
A friend pointed out a possible use case, “tailing” a file on a server. As you surely know, the UNIX tail) command displays the last N lines of a file. We could get this effect by using a negative offset Range request. It might look something like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
The data returned will be a string,
response.body.split(/\r?\n/) splits it into lines allowing the for
lines that are terminated with either CR/LF or just LF.
Because we are reading bytes, not lines, it’s pretty likely that our
byte range will start in the middle of a line. For that reason it’s
best to discard the first line with
tail reads the file backwards in chunks until it gets N
lines. We can’t do that, requesting additional chunks would be
meaningless, the file might have changed. Instead, we need to read
more than we expect to need.
lines.last(n) will insure we get up to
“n” lines, so if we come up short nothing with break. You can adjust
buffer_size as needed.
“tail” is useful, but how about “tail -f”. The real “tail -f” works roughly like:
1 2 3 4 5 6 7 8 9
We can simulate this in HTTP:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
We make a HTTP HEAD request to get the size of the file. If it’s changed, we use the Range header to fetch just the new data.
A couple of caveats for HTTP
Because stat()ing doesn’t have a lot of overhead, the real tail doesn’t sleep. The HTTP version needs to so it doesn’t turn into a DoS attack. Based on your needs you might want to adjust that sleep or schedule the request to avoid blocking.
The code presumes it’s getting complete lines, and this is probably reasonable for logs. However, if incomplete lines become a problem, you would want buffer the last line if it doesn’t end with a line-feed and display it as part of the next request
You’d probably want some security and some error handling, but there you have it, tail, two ways, in HTTP.