Your journey
from data science to animated web graphics

JAMES GOLDIE
DATA AND DIGITAL STORYTELLING LEAD, 360INFO

Hello!

I’m a data journalist at 360info

We deliver verified and reliable information to as many publishing, broadcasting and civic society outlets as possible

I lead our data efforts, publishing open graphics and datasets under Creative Commons

I also used to be a climate + health researcher!

Follow along today

Slides: https://positconf2024.talks.jamesgoldie.dev

Code: https://github.com/jimjam-slam/talk-positconf-aug2024

I’m in the Discord too!

Why I love Quarto

My career: research + data science -> communication -> data journalism

I’ve looked for tools to help bridge my existing technical skills

Quarto is a fantastic bridge

Today I want to bring you on this journey!

Wait, why is it a journey?

All of these have felt scary to me at some point:

Adding a(n S)CSS stylesheet to a Quarto or RMarkdown doc

Adding JavaScript to a Shiny app

Using D3.js to make a graphic

Build tools for JavaScript frameworks

But Quarto has everything you need to start learning!

First steps
Trying out OJS and Observable Plot

You can use OJS right now!

No need to download, add or configure anything

If you have Quarto, you’re ready to go! You can add OJS chunks to Quarto docs right now

Instead of…

Blah blah blah, anyway here's a cool chart:

```{r}
ggplot() +
  aes(x, y) +
  geom_point()
```

… We’ll use this:

Blah blah blah, anyway here's a cool chart:

```{ojs}
Plot.plot({
  marks: [
    // marks here...
  ]
})
```

Swap Observable Plot for ggplot2

Let’s say we’re plotting some basic population data:

Inputs.table(transpose(aus_pops), {
  height: 600,
  width: 600,
  style: {
    color: "red"
  }
})

Swap Observable Plot for ggplot2

Here’s our plot in R…

```{r}
#| output: false

library(ggplot2)
ggplot(aus_pops) +
  aes(x = city, y = pop) +
  geom_col(fill = "#222222ee") +
  theme_minimal() +
  theme(
    plot.background  = element_blank(),
    panel.background = element_blank(),
    panel.grid.major = element_blank(),
    panel.grid.minor = element_blank(),
    plot.title       = element_text(colour = "white"),
    axis.text        = element_text(colour = "white"),
    axis.title       = element_text(colour = "white")) +
  labs(title = "Population in millions")
```

Swap Observable Plot for ggplot2

… and here it is in Observable Plot:

```{ojs}
//| echo: fenced

md`Population in millions`

Plot.plot({
  marks: [
    Plot.barY(aus_pops, {
      x: "city",
      y: "pop",
      fill: "#222222ee"
    })
  ],
  style: {
    fontSize: 16
  },
  marginBottom: 45
})
```

md`Population in millions`

Plot.plot({
  marks: [
    Plot.barY(transpose(aus_pops), {
      x: "city",
      y: "pop",
      fill: "#222222ee"
    })
  ],
  style: {
    fontSize: 16
  },
  marginBottom: 45
})

Getting data into OJS

How do we load our data in?

If it’s already in R or Python, we can just use ojs_define():

ojs_define(aus_pops)

Or we can save the data to disk in R/Python…

write_csv(aus_pops, "aus_pops.csv")

… and read it back in:

aus_pops =
  FileAttachment("aus_pops.csv")
  .csv({ typed: true })

But why?

We can make more than just static plots!

Observable Plot has built-in tooltips, and we can make individual elements accessible too

Plot.tip(transpose(aus_pops), Plot.pointerX({
  x: "city",
  y: "pop",
  fill: "#333333",
  ariaHidden: true
}))

We can add this to marks… or we can just add tip: true to our other mark

md`Population in millions`

Plot.plot({
  marks: [
    Plot.barY(transpose(aus_pops), {
      x: "city",
      y: "pop",
      fill: "#222222ee",
      ariaLabel: d => d.city + ": " + d.pop + " million people"
    }),
    Plot.tip(transpose(aus_pops), Plot.pointerX({
      x: "city",
      y: "pop",
      fill: "#333333",
      ariaHidden: true
    })),
  ],
  style: {
    fontSize: 16
  },
  marginBottom: 45
})

Levelling up
Simulating animation and conditional content

Animating a map

cities = [
  { name: "Brisbane",  lat: -27.467778, lon: 153.028056 },
  { name: "New Delhi", lat: 28.613889,  lon: 77.208889 },
  { name: "Singapore", lat: 1.283333,   lon: 103.833333 },
  { name: "Istanbul",  lat: 41.013611,  lon: 28.955 },
  { name: "Paris",     lat: 48.856667,  lon: 2.352222 },
  { name: "Nairobi",   lat: -1.286389,  lon: 36.817222 },
  { name: "São Paulo", lat: -23.55,     lon: -46.633333 },
  { name: "Montreal",  lat: 45.508889,  lon: -73.554167 },
  { name: "Houston",   lat: 29.762778,  lon: -95.383056 },
  { name: "Vancouver", lat: 49.260833,  lon: -123.113889 },
  { name: "Honolulu",  lat: 21.306944,  lom: -157.858333 }
]

world =
  FileAttachment("/assets/naturalearth-land-110m.geojson")
  .json()

Plot.plot({
  marks: [
    Plot.geo(world, {
      fill: "#222222"
    }),
    Plot.dot(cities, {
      x: "lon",
      y: "lat",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      size: 6
    }),
    Plot.text(cities, {
      x: d => d.lon + 2,
      y: d => d.lat + 2,
      text: "name",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      fontSize: 18,
      textAnchor: "start"
    }),
  ]
})

Plot.plot({
  marks: [
    Plot.geo(world, {
      fill: "#222222"
    }),
    Plot.dot(cities, {
      x: "lon",
      y: "lat",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      size: 6
    }),
    Plot.text(cities, {
      x: d => d.lon + 2,
      y: d => d.lat + 2,
      text: "name",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      fontSize: 18,
      textAnchor: "start"
    }),
  ]
})

What about a globe?

We can add a projection to make this a globe

Plot.plot({
  marks: [
    Plot.graticule(),
    Plot.sphere(),
    /* previous marks */
  ],
  projection: {
    type: "orthographic",
    rotate: [50, -10]
  }
})

Plot.plot({
  marks: [
    Plot.graticule(),
    Plot.sphere(),
    Plot.geo(world, {
      fill: "#222222"
    }),
    Plot.dot(cities, {
      x: "lon",
      y: "lat",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      size: 6
    }),
    Plot.text(cities, {
      x: d => d.lon + 2,
      y: d => d.lat + 2,
      text: "name",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      fontSize: 18,
      textAnchor: "start"
    }),
  ],
  projection: {
    type: "orthographic",
    rotate: [50, -10]
  }
})

What about an animated globe?

OJS is reactive
If your plot includes something that changes, your plot will change too!

now

Let’s use this make an angle:

angle = (now / 40) % 360 - 180
angle

Then replace your plot’s rotation:

projection: {
  type: "orthographic",
  // `angle` is changing now!
  rotate: [angle, -10]
}

Plot.plot({
  marks: [
    Plot.graticule(),
    Plot.sphere(),
    Plot.geo(world, {
      fill: "#222222"
    }),
    Plot.dot(cities, {
      x: "lon",
      y: "lat",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      size: 6
    }),
    Plot.text(cities, {
      x: d => d.lon + 2,
      y: d => d.lat + 2,
      text: "name",
      fill: "#eb343d",
      stroke: "white",
      strokeWidth: 5,
      paintOrder: "stroke",
      fontSize: 18,
      textAnchor: "start"
    }),
  ],
  projection: {
    type: "orthographic",
    rotate: [angle, -10]
  }
})

You can animate with scrolling too!

My colleague Andrew Bray presented Closeread yesterday

Closeread lets you make scrollytelling docs with Quarto!

You can use crProgress instead of now to animate on scroll

Conditional content

Sometimes it’s nice to show different things in different circumstances

In R:

ifelse(condition,
  trueThing,
  falseThing)

In Python:

trueThing \
  if condition \
  else falseThing

In OJS:

condition ?
  trueThing :
  falseThing

But in OJS, this is reactive! It re-runs every time the condition changes.

You could use it for slides…

Here’s a simple timer:

trafficLight = {
  let i = 0;
  while (true) {
    yield Promises.tick(2000, ++i % 2)
  }
}
trafficLight

Draw text…

trafficLight ?
  md`Go for it!` :
  md`Waaait a minute`

… or graphics…

trafficLight ?
  md`![](/assets/positconf-a.png)` :
  md`![](/assets/positconf-b.png)`

… or a drawing…

svg`<svg style="margin-top:20px">
  <circle cx="20" cy="20" r="20"
    fill="${
      trafficLight  ? "#111111" : "orangered"
    }">
  </circle>
  <circle cx="70" cy="20" r="20"
    fill="${
      trafficLight ? "limegreen" : "#111111"
    }">
  </circle>
</svg>`

… or mixed content!

viewof selectedCities = Inputs.table(
  transpose(aus_pops),
  { required: false })

selectedCities.length > 2 ?
  svg`${selectedCityPlot}` :
  md`╳ Not enough cities selected`

selectedCities.length > 2 ?
  // NOTE - i've actually just inlined the plot since
  // it's using different data
  Plot.plot({
    marks: [
      Plot.barY(selectedCities, {
        x: "city",
        y: "pop",
        fill: "#222222ee",
        ariaLabel: d => d.city + ": " + d.pop + " million people"
      }),
      Plot.tip(transpose(aus_pops), Plot.pointerX({
        x: "city",
        y: "pop",
        fill: "#333333",
        ariaHidden: true
      })),
    ],
    style: {
      fontSize: 16
    },
    marginBottom: 45
  }) :
  md`╳ Not enough cities selected`

…or to make responsive graphics

Try resizing this window!

md`Window width is ${width} pixels`

horizBarPlot = Plot.plot({
  marks: [
    Plot.barY(transpose(aus_pops), {
      x: "city",
      y: "pop",
      fill: "#222222ee",
      ariaLabel: d => d.city + ": " + d.pop + " million people"
    }),
    Plot.tip(transpose(aus_pops), Plot.pointerX({
      x: "city",
      y: "pop",
      fill: "#bbbbbb",
      ariaHidden: true
    })),
  ],
  style: {
    fontSize: 16
  },
  marginBottom: 45,
  height: 275,
  width: 1200
})

vertBarPlot = Plot.plot({
  marks: [
    Plot.barX(transpose(aus_pops), {
      x: "pop",
      y: "city",
      fill: "#222222ee",
      ariaLabel: d => d.city + ": " + d.pop + " million people"
    }),
    Plot.tip(transpose(aus_pops), Plot.pointerY({
      x: "pop",
      y: "city",
      fill: "#bbbbbb",
      ariaHidden: true
    })),
  ],
  style: {
    fontSize: 32
  },
  marginLeft: 160,
  marginBottom: 80,
  height: 1000
})

width > 700 ?
  svg`${horizBarPlot}` :
  svg`${vertBarPlot}`

What are the limitations?

Most of the time, this is enough!

I can build interactives quickly and efficiently ✅

But I can’t transition from A to B easily.

In OJS, when you replace one thing with another, the first thing gets destroyed

I need something that is aware of the start and the finish

My wishlist

OJS’s reactivity is lovely… but I want to bypass it sometimes
I want to design graphics in the declarative way I do with ggplot2 and Observable Plot
I want more control over what I draw

Sverto
Using Svelte in Quarto

Reactivity: everyone’s doing it

Reactivity is really popular these days! All of these are reactive:

Category	Examples
Data	Shiny
App dev	SwiftUI (iOS), Jetpack Compose (Android)
Web dev	React, Svelte

In all of these, you:

Pass data or options that might change

Tell it what you want to show

It takes care of the updates when things change

I particularly like Svelte

It has tools for turning data into changing graphics, like transitions

Reactivity
Declarative graphics: everyone’s doing them

Sound familiar?

This is philosophically similar to declarative graphics: we want to focus on what we want to draw, not getting bogged down in how to do it

Enter Sverto

These controls are OJS…

viewof selectedCity = Inputs.select(
  new Map([
    ["Melbourne", "086338"],
    ["Brisbane", "040842"]
  ]),
  {
    value: "086338"
  }
)

viewof selectedVariable = Inputs.select(
  new Map([
    ["Daytime", "tmax"],
    ["Nighttime", "tmin"]
  ]),
  {
    value: "tmax"
  }
)

viewof selectedSeason = Inputs.select(
  new Map([
    ["Whole year", 0],
    ["January", 1],
    ["February", 2],
    ["March", 3],
    ["April", 4],
    ["May", 5],
    ["June", 6],
    ["July", 7],
    ["August", 8],
    ["September", 9],
    ["October", 10],
    ["November", 11],
    ["December", 12]
  ]),
  {
    value: 0
  }
)

viewof selectedMetric = Inputs.select(
  new Map([
    ["Hottest", "max"],
    ["Average", "mean"],
    ["Coldest", "min"],
  ]),
  {
    value: "Average"
  }
)

… but this chart is Svelte!
(with just a smidge of D3.js)

import { aq, op } from "@uwdata/arquero"

fullCity = aq.loadCSV(
  "time-series/" + selectedVariable + "." + selectedCity + ".daily.csv")

tidiedCity = fullCity
  .rename(aq.names("date", "value"))
  .filter(d => d.date !== null)
  .params({ selectedSeason: selectedSeason })
  .derive({
    year: d => op.year(d.date),
    month: d => op.month(d.date) + 1
  })

// filter unless "Whole year" is selected
filteredCity = selectedSeason == 0 ?
  tidiedCity :
  tidiedCity.filter(d => d.month == selectedSeason)

// now group by year and calculate the metrics
allMetrics = filteredCity
  .groupby("year")
  .rollup({
    min:  d => op.min(d.value),
    mean: d => op.mean(d.value),
    max:  d => op.max(d.value),
  })

// finally, select the year and whichever metric column is chosen by the user
finalData = allMetrics
  .select("year", selectedMetric)
  .rename(aq.names("year", "value"))

timeSeriesChart = new TimeSeriesChart.default({
  target: document.querySelector("#timeserieschart")
})

The chart transitions smoothly whenever our OJS controls change!

What can Sverto do?

Sverto lets you write Svelte components, like charts and maps

Then import them into Quarto quickly and easily

And drive them with OJS reactivity

Import the Svelte component

---
title: My Quarto doc
filters:
  - sverto
sverto:
  use:
    - example.svelte
---

Tell Sverto where to add it

```{ojs}
myChart = new example.default({
  target:
    document.querySelector("#here")
})
```

:::{#here}
:::

Update component with OJS

// eg. filter data sent to svelte
// by selected country
myChart.chartData =
  myData.filter(
    d => d.year == selectedYear)

Give it a try! https://sverto.jamesgoldie.dev

It’s dangerous to go alone
Take these!

Trying out OJS in Quarto?

Quarto docs
https://quarto.org/docs/interactive/ojs

Observable Plot docs
https://observablehq.com/plot

Observable Inputs docs
https://observablehq.com/documentation/inputs/overview

Levelling up with OJS?

360info’s OJS graphics
https://github.com/360-info

Observable’s examples
https://observablehq.com/explore#notebooks

Ready to use Svelte in Quarto?

Sverto
https://sverto.jamesgoldie.dev

Connor Rothschild: How to “Learn D3” in 2023
https://connorrothschild.github.io/v4/viz

Matthias Stahl: Svelte + D3 training
https://youtube.com/playlist?list=PLNvnDrMLMSipfbxSp1Z4v9Ydu2ud5i9V8&si=Fu326UCzouaMz3MM

Amelia Wattenberger: Fullstack D3 and Data Visualization
https://www.newline.co/fullstack-d3

Questions?
I’m in the Discord!

Thank you:

Posit for accepting my talk and Articulation for amazing speaker training

Andrew Bray for letting me pinch a Closeread example

Czepeku for letting me use their sweet backgrounds under licence

Slides at https://positconf2024.talks.jamesgoldie.dev

Bonus slides

What about D3.js?

Remember D3.js? We’ve used it before! Remember the scale functions?

D3 has lots of handy utilities, but it’s best known for tools that turn data into graphics:

d3.select("svg")
  // make circles for each data row
  .selectAll("circle")
  .data(mydata)
  .join("circle")
  // style them using data columns
  .attr("cx", d => d.income)
  .attr("cy", d => d.life_expec)
  .attr("r", d => d.population)
  .attr("fill", "blue")

D3 is lower level (ggplot2 users, think grid)

I failed to learn it for years!

D3’s “select” code is very imperative

As Connor Rothschild explains, you can use D3.js tools when it’s useful, but Svelte is often a better way to create graphics from data

What about D3.js?

The differences between D3 and Svelte are equivalent to the differences between instructions and authoring. In D3, we write instructions to tell JavaScript what to render; in Svelte, we write our output directly.

— Connor Rothschild, How to “Learn D3” in 2023

But web frameworks can be scary…

Most web frameworks have lovely browser sandboxes to try them out in…

Screenshot of Svelte.dev browser sandbox

But web frameworks can be scary…

But when you start a project for real, it gets confusing fast!

I designed Sverto with sensible defaults for Quarto users

'Stop doing math' meme with JavaScript Reddit

Your journeyfrom data science to animated web graphics

Hello!

Why I love Quarto

Wait, why is it a journey?

First stepsTrying out OJS and Observable Plot

You can use OJS right now!

Swap Observable Plot for ggplot2

Swap Observable Plot for ggplot2

Swap Observable Plot for ggplot2

Getting data into OJS

But why?

Levelling upSimulating animation and conditional content

Animating a map

What about a globe?

What about an animated globe?

You can animate with scrolling too!

Conditional content

You could use it for slides…

… or mixed content!

…or to make responsive graphics

What are the limitations?

My wishlist

SvertoUsing Svelte in Quarto

Reactivity: everyone’s doing it

ReactivityDeclarative graphics: everyone’s doing them

Enter Sverto

What can Sverto do?

It’s dangerous to go aloneTake these!

Trying out OJS in Quarto?

Levelling up with OJS?

Ready to use Svelte in Quarto?

Questions?I’m in the Discord!

Bonus slides

What about D3.js?

What about D3.js?

But web frameworks can be scary…

But web frameworks can be scary…

Your journey
from data science to animated web graphics

First steps
Trying out OJS and Observable Plot

Levelling up
Simulating animation and conditional content

Sverto
Using Svelte in Quarto

Reactivity
Declarative graphics: everyone’s doing them

It’s dangerous to go alone
Take these!

Questions?
I’m in the Discord!