· tutorials · 6 min read

HTML5 Canvas on Rails? Part 4

Series Overview

Vanilla rails doesn’t really give much guidance for how best to interact with an HTML5 canvas. This series represents my suggestion for how to build features in a way that conforms to StimulusJS and Rails conventions. To showcase this approach, I walk through building a proof-of-concept toy app that allows the user to draw annotations on top of an image and persist those annotations.

  1. The first post lays out the basic approach and why I decided to design the code in this way.
  2. The second post guides the reader through building out the necessary boilerplate for the app to function.
  3. The third post guides the reader through building the interactive canvas functionality.
  4. The final post contains a summary of this approach and the final code for the server-side HTML, Rails controller, and Stimulus controller.

You can head over to the github repo to inspect the full code.

If you’d like to skip the tutorial and view just the final Stimulus controller, Rails controller, and server-side rendered HTML form click here to jump directly to part 4.


Using the approach laid out in these blog posts, we’re able to keep with the spirit of using “sprinkles of Javascript” to make our server-rendered HTML “sparkle”. Delegating state management to the DOM and canvas manipulation/interactivity to our Stimulus controllers provides clear guidelines for how to separate concerns for different functionality. The approach lets each language/technology perform the role for which they are berst suited, and has an added benefit of minimizing the amount of Javascript, and arguably complexity, necessary to achieve the desired result.

While the Rails controller provides a RESTful approach to working with our resources, RawImage and Label, in a particular context, the StimulusJS controller is focused on handling click events to read/write data stored in the DOM and presenting those data visually.

Exercises for the reader

You didn’t think that this was as far as the journey goes, did you? There are a ton of additional other features that can, and probably should, be added in order to bring this idea to working production state. These include:

  • persisting a scaled version of the annotation, regardless of the canvas size
  • relatedly, enable dynamic scaling of canvas and annotations when resizing the window
  • additional features that Defne’s version has implemented, including:
    • adding dots to indicate vertices of the annotation
    • moving a single vertex
    • moving the entire annotation
    • binding the annotation to the borders of the canvas
    • closing the polygon
    • undo functionality
  • can you think of more?

Despite that, I hope this tutorial was helpful in demonstrating how we might extend the StimulusJS philsophy of injecting our HTML with “sprinkles of Javascript”. Happy coding!

Final State of the Rails Form and Stimulus Controller

Here are the 3 key files we built over the course of this tutorial in their entirety. Considering the complex interactivity of this feature, the two controllers are still relatively small (<100 lines of code).

# app/controllers/annotation_editor/raw_images_controller.rb
class AnnotationEditor::RawImagesController < ApplicationController
  before_action :set_raw_image, only: %i[show update]
  before_action :set_label, only: %i[show update]

  # GET annotation_editor/labels/1/raw_images/1
  def show
    @images = RawImage.all.order(:id)
    @labels = Label.all.order(:id)
    @previous_image = previous_record(@images, @raw_image)
    @next_image = next_record(@images, @raw_image)
    @previous_label = previous_record(@labels, @label)
    @next_label = next_record(@labels, @label)
    @annotation = Annotation.find_by(
      raw_image: @raw_image,
      label: @label
    ) || Annotation.new

  # PATCH/PUT annotation_editor/labels/1/raw_images/1
  def update
    @annotation = Annotation.find_by(
      raw_image: @raw_image,
      label: @label
    ) || Annotation.new(raw_image: @raw_image, label: @label)
    @annotation.landmarks = raw_image_params[:landmarks]
    if @annotation.save
      flash[:notice] = "Annotation saved!"
      flash[:alert] = "There was an error saving your annotation"
    redirect_to annotation_editor_label_raw_image_path(@label, @raw_image)


  # Use callbacks to share common setup or constraints between actions.
  def set_raw_image
    @raw_image = RawImage.find(params[:id])

  # Use callbacks to share common setup or constraints between actions.
  def set_label
    @label = Label.find(params[:label_id])

  # Only allow a list of trusted parameters through.
  def raw_image_params

  def next_record(collection, record)
    record_idx = collection.find_index(record)
    record_idx + 1 >= collection.length ? collection[0] : collection[record_idx + 1]

  def previous_record(collection, record)
    record_idx = collection.find_index(record)
    record_idx - 1 < 0 ? collection[-1] : collection[record_idx - 1]
// app/javascript/controllers/annotation_canvas_controller.js
import { Controller } from '@hotwired/stimulus';
import Konva from 'konva';

// Connects to data-controller="annotation-canvas"
export default class extends Controller {
  static targets = ['canvas', 'landmarks'];

  static values = { imageUrl: String };

  connect() {
    this.stage = new Konva.Stage({
      container: 'annotation-container',
      width: this.canvasTarget.clientWidth,
      height: this.canvasTarget.clientHeight,

    this.layer = new Konva.Layer();
    this.group = new Konva.Group({ name: 'annotation' });

    const currentImage = new Image();
    currentImage.src = this.imageUrlValue;
    currentImage.onload = () => {
      this.image = new Konva.Image({
        x: 0,
        y: 0,
        image: currentImage,
        width: this.stage.width(),
        height: this.stage.height(),

      // add the shape to the layer
    this.polygon = new Konva.Line({
      stroke: '#00F1FF',
      strokeWidth: 3,
      closed: false,
      fill: 'rgb(140,30,255,0.25)',
      id: 'annotation',
      name: 'polygon',


  handleClick() {
    const points = this.getCurrentPoints();
    const coordX = this.group.getRelativePointerPosition().x;
    const coordy = this.group.getRelativePointerPosition().y;
    points[points.length] = [coordX, coordy];
    this.landmarksTarget.value = JSON.stringify(points);

  getCurrentPoints({ flattened } = { flattened: false }) {
    const points = JSON.parse(this.landmarksTarget.value)
    if (flattened) return points.reduce((a, b) => a.concat(b), []);
    return points;

  drawAnnotation() {
    this.polygon.points(this.getCurrentPoints({ flattened: true }));

  reset() {
    this.landmarksTarget.value = JSON.stringify([]);

  disconnect() {
    // clean up the canvas when navigating away
# app/views/annotation_editor/raw_images/_form.html.erb

<%= form_with(
  model: [:annotation_editor, label, raw_image], 
  class: 'd-flex flex-column page-header',
) do |form| %>
  <% if raw_image.errors.any? %>
    <div id="error_explanation">
      <h2><%= pluralize(raw_image.errors.count, "error") %> prohibited this raw_image from being saved:</h2>

      <% raw_image.errors.full_messages.each do |message| %>
        <li><%= message %></li>
      <% end %>
  <% end %>
  <div class="d-flex flex-column " data-controller="annotation-canvas" 
    data-annotation-canvas-image-url-value=<%= url_for(raw_image.image) %> 
      <div class='d-flex flex-row justify-content-between'>
        <div id="image-carousel">
          <%= link_to annotation_editor_label_raw_image_path(label, previous_image) do  %>
            <i class="bi-chevron-left fas my-5" id="previous-image"></i>
          <% end %>
          <span  class="my-5 px-2"><strong>Image:</strong> <%= raw_image.id %>
          <%= link_to annotation_editor_label_raw_image_path(label, next_image) do  %>
            <i class="bi-chevron-right fas my-5" id="next-image"></i>
          <% end %>
        <div id="label-carousel">
          <%= link_to annotation_editor_label_raw_image_path(next_label, raw_image) do  %>
            <i class="bi-chevron-left fas my-5" id="previous-image"></i>
          <% end %>
          <span class="my-5 px-2"><strong>Label:</strong> <%= label.name %>
          <%= link_to annotation_editor_label_raw_image_path(previous_label, raw_image) do  %>
            <i class="bi-chevron-right fas my-5" id="next-image"></i>
          <% end %>
        <div class="btn btn-primary btn-lg my-4" 
             style="--bs-btn-padding-x: 3rem; --bs-btn-padding-y: 0.25rem;"
          <i class="bi-trash fas my-4" id="save"></i> 

        <%= form.button class: "btn btn-primary btn-lg my-4", style:"--bs-btn-padding-x: 3rem; --bs-btn-padding-y: 0.25rem;" do %>
          <i class="bi-save fas" id="save"></i> 
        <% end %>
          style="height: 70vh;"
    <%= form.hidden_field :landmarks, data: { annotation_canvas_target: 'landmarks'}, value: "#{@annotation.landmarks || []}" %>
<% end %>
Previous Post
Back to Blog