Adding Sitepress to Rails

I've been using Proofreader.io and thought it's about time I fixed some of the issues that Proofreader is designed to surface. Like SEO & social meta tags, sitemaps, robots.txt.

I choose Sitepress to handle the content side of things, which make doing a lot of this easier. Here's a walk though.

Install and setup Sitepress

Sitepress - "Build ambitious content websites in Rails"

Add the gems. You can skip markdown-rails and rouge if you don't want Markdown support or syntax highlighting.

gem "sitepress-rails"
gem "markdown-rails"
gem "rouge"

Run the installers. They will create the basic initializers and structure for the content. Content ends up in app/content, along with helpers and layouts.

bin/rails generate sitepress:install
bin/rails generate markdown_rails:install

Since these pages will be cached in production, we want to make sure they're signed out when generated. I don't actually know if this is needed, but I want it to work that way in development regardless.

I ended up creating a sub directory for the website controllers, app/controllers/website/.

I'm also using Pundit Can which requires that I skip some filters.

class Website::WebsiteController < Sitepress::SiteController
  layout "website"

  skip_scoped_check :show
  skip_authorized_check :show

  protected

  def signed_in?
    false
  end

  def current_user
    nil
  end

  def current_account
    nil
  end

  def setup_current
    Current.user = nil
    Current.true_user = nil
    Current.account = nil
    Current.ip = request.remote_ip
  end
end

Next update the routes so they user the overriden controller. I also removed sitepress_root in favor of using the root: true parameter.

sitepress_pages root: true, controller: "website/website"

For some reason, Slim isn't loaded correctly. I added an initialer for it.

config/initializers/sitepress.rb

ActionView::Template::Handlers.extensions << :slim

Restart the server

Now Sitepress should be working and serving pages!

SEO & Social

Turns out url_for doesn't add anything when passed a string, so we need this helper. Add to app/content/helpers/page_helper.rb.

module PageHelper
  # `url_for` doesn't create a url when a string is passed
  def path_url path
    URI.const_get(request.scheme.upcase).build({
      host: Rails.application.routes.default_url_options.fetch(:host, request.host),
      port: Rails.application.routes.default_url_options.fetch(:port, request.port),
      path: path
    }).to_s
  end
end

Meta Tags

Next up, meta tags. These are used by the search engines and social sites to get a better understanding of the page. Open Graph for Facebook, Twitter has some specifice tags, and keywords & description for search engines.

This definitely could be done without a gem, but I liked the Meta Tags gem and used that.

Add to the Gemfile and bundle.

gem "meta-tags"

Install it. This adds a config, but I didn't change anything there.

rails generate meta_tags:install

I want to use the frontmatter from Sitepress to override data for the meta tags. This maps the frontmatter to a hash for the set_meta_tags method.

Add this to app/content/helpers/page_helper.rb

...
  def to_meta_tags page
    {
      title: page.data.title,
      description: page.data.description,
      keywords: page.data.keywords,
      image_src: page.data.image,
      canonical: canonical(page),
      noindex: page.data.noindex,
      index: page.data.index,
      nofollow: page.data.nofollow,
      follow: page.data.follow,
      noarchive: page.data.noarchive,
      prev: page.data.prev,
      next: page.data.next,
      og: page.data.og,
      twitter: page.data.twitter
    }.compact
  end

  def canonical page
    path_url(page.request_path.gsub(/index\.html\z/, "").gsub(/\.html\z/, "").gsub(/\/\z/, ""))
  end
...

The canonical method strips index.html, .html, and tailing slashes to get a normalized URL. Also uses the other helper method, path_url we added above.

Sitepress has some layout support, but I used the build in Rails layouts. This is where we connect Sitepress to meta_tags with the set_meta_tags method.

app/views/layouts/website.html.slim

- set_meta_tags to_meta_tags(current_page)

- content_for :content do
  = yield

= render template: "layouts/application"

Add the display_meta_tags to the application layout, or head.html.slim in my case. Here is where I set the defaults for tags that might not be set.

= display_meta_tags site: "Proofreader.io",
  reverse: true,
  description: "Automates monitoring the technical details to make sure your content shines!",
  image_src: image_url("manolo-chretien-252147-mono.jpg"),
  og: { title: :title,
    description: :description,
    site_name: :site,
    url: :canonical,
    image: :image_src,
    type: "website",
  },
  twitter: { card: "summary", site: "@proofreaderio" }

Sitemap

It's a good idea to have a sitemap. To set up we'll need a route, controller, and view.

The route.

get '/sitemap.xml' => 'sitemaps#index', defaults: { format: 'xml' }

The controller.

class Website::SitemapsController < ApplicationController
  layout false

  skip_authorized_check :index
  skip_scoped_check :index

  def index
    render formats: :xml
  end
end

The view. Here we use Sitepress and path_url to create a map of pages. I'm removing any that have noindex frontmatter.

xml.instruct! :xml, version: "1.0"
xml.tag! "urlset", "xmlns" => "http://www.sitemaps.org/schemas/sitemap/0.9" do
  site = Sitepress::Site.new(root_path: "app/content")
  site.resources.reject{_1.data.noindex}.each do |resource|
    xml.tag! "url" do
      xml.tag! "loc", path_url(resource.request_path)
      xml.lastmod resource.asset.updated_at.strftime("%F")
    end
  end
end

Breadcrumbs

Not huge fan of breadcrumbs, but they are helpful for SEO and large content sites. Again we can the Sitepress meta data to create them. parents is based on the file/folder stucture of the content and is exactly what I needed.

In the app/views/layouts/website.html.slim file, I added this:

/ ...
- content_for :content do
  - current_page.parents.any?
    menu.flex.flex-row.gap-3.text-sm.text-gray-700.mb-4
      - current_page.parents.reverse.each_with_index do |parent, idx|
        li
          a href=parent.request_path = parent.data.title
        - if idx < current_page.parents.size - 1
          li \
/ ...

Robots.txt

Very similar to the sitemap, we can have a more dynamic robots.txt. You could also use the one from the public directory, but I liked the idea of having a bit more control by moving into Rails.

The route.

get "/robots.txt" => "website/robots#index", :defaults => {format: "text"}

The controller.

class Website::RobotsController < ApplicationController
  layout false

  skip_authorized_check :index
  skip_scoped_check :index

  def index
    render format: :text
  end
end

The view. Used ERB for this one. index.text.erb. Also added a reference to the sitemap.

<% if Rails.env.production? %>
User-Agent: *
Disallow:
<% else %>
User-Agent: *
Noindex: /
<% end %>

Sitemap: <%= path_url("/sitemap.xml") %>

Remove the on in public/

rm public/robots.txt

Error pages, 404, 422, 500

Not sure if this actually helps with SEO, but I like the idea of having nicer error pages. Of course, there are some cases where the app is actually broken and we'll end up with the Nginx default error pages. Even still, I moved into Rails.

Remove the defaults

rm public/{404,422,400}.html

Tell Rails to use our routes for the exceptions in config/application.rb.

    config.exceptions_app = routes

Here's the 404 page in app/content/pages/404.html. I'm using the Sitepress meta data to list some suggestions that be helpful. Also note the noindex frontmatter.

---
title: Oops, we couldn't find what you're looking for - 404
noindex: true
---

.container.mx-auto.prose.lg:prose-lg.py-20.px-4'
  h1 = current_page.data.title

  h2 Maybe something here?
  menu.flex.flex-col.gap-1
    - PageModel.all.reject{_1.data.noindex}.each do |page|
      li : a href=page.request_path = page.data.title

And the 422 page.

---
title: The change you wanted was rejected - 422
noindex: true
---

.container.mx-auto.prose.lg:prose-lg.py-20.px-4'
  h1 = current_page.data.title

  p Maybe you tried to change something you didn't have access to.

Lastly, the 500 page.

---
title: We're sorry, but something went wrong - 500
noindex: true
---

.container.mx-auto.prose.lg:prose-lg.py-20.px-4'
  h1 = current_page.data.title

  p We've been notified of the error and will investigate it shortly.

  p Feel free to contact us if you have any questions!
  p
    a href="/contact" Contact Us

Force HTTPS

This is built in! Probably already handled by Nginx.

# config/environments/production.rb

Rails.application.configure do
  # ...
  # force HTTPS on production
  config.force_ssl = true
  #...
end

Redirect www to non-www

Generally I prefer without www. Added this to the top of config/routes.rb. Any request starting with www. will be redirected without it.

Rails.application.routes.draw do
  constraints(host: /\Awww\./i) do
    get "(*any)" => redirect { |params, request|
      URI.parse(request.url).tap { |uri| uri.host = uri.host.gsub(/\Awww\./, "") }.to_s
    }
  end

  #...
end

Testing

Making the above change will break tests b/c the defaults host is www.example.com. Here's how to default to example.com.

Add to test/test_helper.rb.

#...
class ActionDispatch::IntegrationTest
  before { host! "example.com" }
end

Redirects

As the application grows, there will probably be cases where we want to redirect old URLs. I setup a new routes file for them in config/routes/redirects.rb.

Rails.application.routes.draw do
  #
  # Redirects: https://guides.rubyonrails.org/routing.html#redirection
  #
  # get '/stories', to: redirect('/articles')
  # get '/stories/:name', to: redirect('/articles/%{name}')
end

We need to tell Rails about those additional routes files in config/application.rb. This also allows us to organize the routes.

#...
    config.paths["config/routes.rb"].concat Dir[Rails.root.join("config/routes/*.rb")]
#...

References

A lot of this was adapted and updated from SEO & Ruby On Rails : the comprehensive guide 2018