J E L L Y E N T
Learn-Most productive Mode for Better Rails Downtime

In the most up-to-date day I became once as quickly as having a conception to present a desire to the Postgres model on an utility I’ve
been engaged on. This could possibly require reasonably quantity of downtime, likely about 10
minutes.

The default resolution I’d attain for in these situations would be to enter Heroku’s
repairs mode, which serves an HTML repairs get page with a 503 Carrier
Unavailable
grunt code. This works however makes the utility fully
unusable at some level of the give a desire to, and I hoped to transfer making an try out a more in-depth resolution. On this
explicit case, I additionally wished with a conception to develop JSON responses as the
utility basically items an API for a cell app.

After exploring a handful of half of-baked solutions, I settled on the usage of a
learn-easiest connection to the database to restful enable reads however stay any
writes from occurring. Whereas the usage of the learn-easiest connection, the Postgres adapter
will raise an error any time we are trying to vary recordsdata for the length of the database, however we
can with out problems rescue this utter error and convert it to a particular person-going thru see. I
felt barely abnormal the usage of exceptions as the core of this workflow, however for the length of the stay, it
labored out in point of fact smartly, so I needed to portion the specifics.

It’s label noting that this resolution is severely smartly honorable to this
utter utility, which easiest items an API and has very learn-heavy usage,
however I have faith about it could possibly possibly possibly possibly possibly smartly be prolonged to work with a unfold of forms of app moreover.

Configuring Rails to Employ the Learn-Most productive Connection

If level to, Rails will expose the connection string in a DATABASE_URL env var to
join to the database. Following the Connection Desire notes for the length of the
Rails guides, I realized that I’d possibly smartly also assemble this DATABASE_URL usage explicit
and enable for a momentary override. To assemble this, I added an explicit url
property for the manufacturing ambiance with desired connection decision:

# config/database.yml

manufacturing: 
  <<:  *default
  url:  <%= ENV["DATABASE_URL_READ_ONLY"] || ENV["DATABASE_URL"] %>

With this in blueprint, I’m in a intention to enable the learn-easiest mode marvelous by surroundings the
DATABASE_URL_READ_ONLY env var:

heroku config:spot 
  DATABASE_URL_READ_ONLY='postgres://read_only_user:abc123...' 
  --far flung manufacturing

Likewise, to disable the learn-easiest mode, I’m in a intention to expose:

heroku config:unset DATABASE_URL_READ_ONLY --far flung manufacturing

Order: I became once as quickly as ready to assemble expose of Heroku’s Postgres Credentials interface to assemble
the learn-easiest particular person, however can make a selection up as correct with to you’re now no longer working with Heroku are trying with a conception to
expose these directions to assemble your learn-easiest particular person.

Error Handling

With a unfold of approaches I conception to be I found that I needed to discontinuance off factual just a few
a unfold of doable recommendations to self-discipline writes to the database, however the learn-easiest
connection labored smartly to scale again the total objects off in a single change. That talked about, it
became once as quickly as easiest half of the resolution, as I fully didn’t desire the errors making it to
customers.

Fortunately it became once as quickly as considerably straightforward to develop a centralized rescue
that can possibly possibly possibly possibly smartly enable me to take care of all of your errors. First, I created a module the usage of
Rails’s ActiveSupport::Residing effectivity:

# app/controllers/concerns/read_only_controller_support.rb
module ReadOnlyControllerSupport
  lengthen ActiveSupport:: Residing

  constructed-in assemble
    if ENV["DATABASE_URL_READ_ONLY"].level to?
      rescue_from ActiveRecord:: StatementInvalid assemble |error|
        if error.message.match?(/PG::InsufficientPrivilege/i)
          render(
            grunt: :service_unavailable,
            json: {
              recordsdata: "The app is within the intervening time in learn-easiest repairs mode. Please are trying all over again later.",
            },
          )
        else
          raise error
        stay
      stay
    stay
  stay
stay

When constructed-in, this module will expose Rails’s rescue_from methodology to grasp
doubtlessly relevant errors, after which we assemble a transient check interior that block
to assemble obvious we’re easiest capturing the relevant errors.

Order, the rescue_from correct judgment is easiest enabled when the DATABASE_URL_READ_ONLY
is made up our minds, so we’re ready to reuse the existence of that variable as a formula to scope
this habits.

I became once as quickly as then ready to embody that module in any relevant provoking controller:

# app/controllers/application_controller.rb
class ApplicationController < ActionController:: Rotten
  embody ReadOnlyControllerSupport
stay

# app/controllers/api/base_controller.rb
class Api:: BaseController < ActionController:: Rotten
  embody ReadOnlyControllerSupport
stay

Non-API Error Handling

My initial expose case for this learn-easiest mode easiest wished to beef up API requests,
however I’d possibly smartly also take into consideration extending it to HTML and to find-based fully interfaces.

The very first part I’d possibly make a selection up as correct with in solutions would be including a sitewide banner that acknowledged
that we were in a learn-easiest repairs mode to alert customers to the up-to-the-minute
grunt.

With that in blueprint, I build we would possibly possibly possibly possibly smartly also lengthen the error coping with for the length of the
ReadOnlyControllerSupport module to redirect the particular person motivate and prove a
relevant message:

rescue_from ActiveRecord:: StatementInvalid assemble |error|
  if error.message.match?(/PG::InsufficientPrivilege/i)
    respond_to assemble |structure|
      structure.json assemble
        # JSON erorr message as confirmed above
      stay

      structure.html assemble
        redirect_back(
          fallback_location: root_path,
          alert: "The app is within the intervening time in learn-easiest repairs mode. Please are trying all over again later.",
        )
      stay
    stay
  else
    raise error
  stay
stay

Scheduler and Background Jobs

One extra consideration right here would be spherical background jobs and scheduler
processes. For background jobs issues are considerably straightforward – we correct
must scale our employee pool down to zero for the learn-easiest length.

Scheduler processes are barely of trickier as I didn’t make a selection up as correct with a mechanism for
globally enabling or disabling them. With that in solutions, I build the colossal
resolution would be to easiest ever make a selection up as correct with scheduler processes enqueue jobs however now no longer
assuredly assemble any work past that.

Migrations

The final sticking level we bumped into became once as quickly as migrations. We make a selection up as correct with a launch expose
outlined in our Procfile that became once as quickly as configured to wing rake db:migrate.
Unfortunately, it appears that although no migrations wing, Rails will restful
are trying and jot down to the ar_internal_metadata desk as fragment of the db:migrate
expose, and Heroku will wing the discharge expose any time we adjust an env. In
my initial are trying, Heroku failed once I attempted to identify the
DATABASE_URL_READ_ONLY as the associated launch expose hit the learn-easiest
error when working rake db:migrate.

To work spherical this I wrote reasonably script that first assessments if there
are any migrations that can possibly possibly possibly possibly make a selection up as correct with to be wing, and easiest if there are, then runs rake
db:migrate
:

#!/bin/bash

spot -e

if bin/rails db:migrate:grunt | grep '^s+downs'; then
  bin/rails db:migrate
fi

This script became once as quickly as added to the repo as bin/migrate-if-wished, after which we
changed our name to rake db:migrate with bin/migrate-if-wished

Commerce (Oct 14, 2020)

After sharing this save up, a commenter on Hacker News known the
rails_failover gem
that their group at Discourse maintains. It appears to present
identical effectivity, however in a extra mighty and fully conception out formula. Appears to be grasp to be
like a mountainous formula to avoid losing into form this assemble of machine.

Learn More

Related Post

4 Comments

Leave a Comment

Recent Posts

Small Issues That Made Amiga Gigantic – Datagubbe.se
Tim Cook: This Is the No. 1 Reason We Accomplish iPhones in China
Naomi Wu’s 3DPrintMill CR-30 Now Live on Kickstarter
A Kid, a Minor Bike Accident and a $19,000 Medical Invoice
Penguin Random House Workers Confront Publisher About New Jordan Peterson E book

Recent Posts

Small Issues That Made Amiga Gigantic – Datagubbe.se
Tim Cook: This Is the No. 1 Reason We Accomplish iPhones in China
Naomi Wu’s 3DPrintMill CR-30 Now Live on Kickstarter
A Kid, a Minor Bike Accident and a $19,000 Medical Invoice
Penguin Random House Workers Confront Publisher About New Jordan Peterson E book
en_USEnglish
fr_FRFrench en_USEnglish