2017-09-18 23:18:49
good
A service to gate web requests
For a high volume web site, there is generally no available downtime. We have to
find a way to update the application without cutting access.

One solution is to use a load balancer. If you have multiple servers, you remove
one server from the load balancer and you wait for the connections to vanish. You
perform your maintenance and inject back the server in the load balancer.

But sometime, long live connections take way too long to vanish. While the solution
using load balancer is a requirement, something lighter may be used to gate the HTTP request
for quick service refresh. This is the trli-stop service. This is a very simple service listening
on a Unix socket. It does very little.

!Normal operation

During normal operation, it does very little. For every web request, the web application connects
to trli-stop and wait for a single byte message. We will call it the GO byte. Trli-stop sends the GO byte immediately
at connection time. The web service then proceeds and close the connection to trli-stop at the end of the web request.

So it does very little. Unix sockets are so fast that we have not been able to measure any slowdown on the
web services.

!During maintenance

The maintenance sequence goes like this:

*We request trli-stop to stop all requests. It simply stops sending the GO byte at connection time so
the web services just sit there idle.
*We perform the maintenance
*We request trli-stop to resume operations.

Now trli-stop does a little more when asked to stop all requests. It does two things:

*It stops sending the GO byte for new connection.
*And it waits for all "in flight" connections to end. These are the web request
which have received their GO byte and are proceeding.

So when we ask the trli-stop service to stop the web requests, it succeeds quickly even if the
web browsers are keeping the connections open to the web service. When trli-stop confirms
that the web requests are all stopped, we know we can refresh the internal services, including
the database server. End user won't witness any failure or broken pages. They can experience
a small performance hit.