Watch a process using Turbo streams

Dusty Candland | | cloudsh, ruby, rails, hotwire, turbo

After upgrading to Hotwire, I wanted to try out Turbo Streams. In CloudSh there is a background job that runs a Golang application to index sites. That seemed like a cool thing to use Turbo Streams so I can see the console output.

Turbo Streams just work. The basic flow is capture the process output in the background job, broadcast it to the UI, and follow along by scolling as new data comes in.

Turbo Streams

First I needed a new page to "watch" the output. I added a watch action and view.

class IndicesController < ApplicationController
# ...

def watch
end
end

The view setups up the Turbo Stream and a container for the log lines.

= turbo_stream_from @index

div#log_lines class="h-full overflow-auto bg-gray-900 text-white p-4 text-sm font-mono border-2 border-black" style="height: 50vh"

Next an ActiveModel is needed to broadcast, specifically with the Turbo::Broadcastable concern. I don't want this data stored in the database to I created a model to hold a line of output.

class LogLine
include ActiveModel::Model
include ActiveModel::Serialization
include ActiveModel::Attributes
include ActiveModel::AttributeMethods
include Turbo::Broadcastable
extend ActiveModel::Naming

attribute :line

alias_method :to_hash, :serializable_hash

def persisted?
false
end

def id
nil
end

def to_s
line
end

def to_html
line
end

def broadcast index
broadcast_append_to index
end
end

Turbo Streams will look for a view partial to render the broadcasts using the model being broadcast.

# app/views/log_lines/_log_line.html.slim
p = log_line.to_html

With this I can create a LogLine and broadcast it to the watch page for the Index. It can even be run from the rails console.

LogLine.new(line: "Test line").broadcast index

Watching the process

In the background job, I needed to capture the STDOUT and broadcast each line. Popen2e provides this.

    # ...
cmd = "cloudsh index #{index.domain} -d 100 -l #{limit} -x"

status = Open3.popen2e(cmd) { |stdin, stdout_and_stderr, wait_thr|
stdout_and_stderr.each { |line|
LogLine.new(line: line).broadcast index
}
wait_thr.value
}

LogLine.new(line: "Completed: exit: #{status.to_i}").broadcast index
# ...

Now after queuing the background job I redirect to the watch action created earlier.

Scrolling

With data streaming to the UI the problem became, the line would append, but not scroll into view. Making it very hard to watch the process.

The other side of Hotwire is Stimulus, so I created a controller and tried to hook into the turbo:before-stream-render. An after stream render event would have been better, but that's not an event in Turbo. Even so, I couldn't get the turbo:before-stream-render to fire.

I came accoss this Event to know a turbo-stream has been rendered discussion which suggests using MutationObserver. This was new to me, but integrates with Stimulus nicely.

Need to add the controller to the watch page on the container div.

  div#log_lines class="h-full overflow-auto bg-gray-900 text-white p-4 text-sm font-mono border-2 border-black" style="height: 50vh" data-controller="scroll"

The controller listens for childList changes, finds the last child, and scrolls it into view.

// app/javascript/controllers/scroll_controller.js
import { Controller } from 'stimulus'

export default class extends Controller {
connect () {
this.scroll = this.scroll.bind(this)
const config = { childList: true }

this.observer = new MutationObserver(this.scroll)
this.observer.observe(this.element, config)
}

disconnect () {
this.observer.disconnect()
}

scroll (mutationsList, observer) {
for (const mutation of mutationsList) {
if (mutation.type === 'childList') {
const children = this.element.children
children[children.length - 1].scrollIntoView()
}
}
}
}

This will scroll to the end even if you're trying to view higher up the logs. Might look at trying to handle that. Otherwise it works how I'd expect.

Colors

The cloudsh application outputs colors to the console using ANSI escape codes. I used some code from this ANSI escape code with html tags in Ruby? StackOverflow answer. Probably need something more robust, but it works for this proof of concept.


Webmentions

These are webmentions via the IndieWeb and webmention.io. Mention this post from your site: