A recipe for applications: Dataset & Web Components

R. S. Doiel,

Caltech Library, Digital Library Development

2025-06-12

Welcome to “A recipe for applications”

Welcome everyone. This is a talk and hands on workshop.

An approach to building web applications using Dataset and Web Components

Workshop: Dataset & Web Components

What we’ll do

What we’ll learn

Follow along at https://caltechlibrary.github.io/t2t3_dataset_web_apps/presentation1.html

Download the presentation zip file at https://github.com/caltechlibrary/t2t3_dataset_web_apps/releases

Getting started

You probably already have these, if not install them

Install dataset

We can start our first iteration of our application once you have these available.

Part 1: What are we building?

GOAL: A simple web application that lets us curate a list of recipes.

We’re going start from the back end and spend most of our time on the front end.

Part 1.1: What are the parts of our application?

  1. A web service for managing the recipe collection
  2. A page to browse recipes by name
  3. A page to display a recipe
  4. A page and web form to add or edit a recipe

Part 1.1: What is a recipe?

Part 1.1: Basic Strategy

  1. Setting up our web service
  2. Sketch your app using HTML
  3. Wire up and test

Part 1.2: Setting up our web service

  1. create our recipes.ds collection
  2. load sample data into our recipes.ds collection
  3. Configure and run our collection as a web service

Part 1.2: creating our collection

We use the dataset command line program to initialize a dataset collection.

dataset init recipes.ds

(Linux, macOS and Windows)

Part 1.2: Loading some sample data

Download sample data file recipes.jsonl (see: https://github.com/caltechlibrary/t2t3_dataset_web_apps/blob/main/recipes.jsonl)

cat recipes.jsonl | dataset load recipes.ds

On Windows:

Get-Content recipes.jsonl | dataset load recipes.ds

Part 1.2: Verify we loaded our data OK

dataset keys recipes.ds
dataset read recipes.ds frybread

Part 1.2: Example terminal interaction

$ dataset keys recipes.ds
blondies
brownies
cafe-con-leche
frybread
poi
waffles

Part 1.2: Example terminal interaction

$ dataset read recipes.ds frybread
{
    "ingredients": "backing powder,2 tsp\r\nflour,2 cups\r\nhot water,2/3 cups\r\nsalt,1 tsp salt\r\nshortening,2 Tbsp\r\n",
    "key": "frybread",
    "name": "Fry Bread",
    "procedure": "1. Combine flour, baking powder, and salt in a bowl.\r\n2. Use a pastry blender (or two butter knives) to cut the shortening into the flour.\r\n3. Add the hot water, and mix until the water is incorporated and you get a dough.\r\n4. Turn out the dough on a lightly floured board. Knead the dough until it is soft and smooth.\r\n5. Wrap the dough in plastic, and let the dough rest for 30 minutes.\r\n6. Divide the dough into 6 pieces, roll each into a ball, and roll each into a flat disk with a rolling pin.\r\n7. Brush one side of each disk with melted margarine and place on a barbecue over a 3 Mississippi fire.\r\n8. Brush the opposing side of the bread with margarine and flip the bread on the barbecue.\r\n9. Cook until both sides are golden brown. Serve hot.\r\n\r\n- You can substitute either 1/4 cup plain yogurt, or a 1/4 cup soured milk as a leavening agent\r\ninstead of baking powder. If you do, add 2 hours to the rest time for the dough, and leave\r\nthe dough somewhere warm. You can optionally include the baking powder as well to get a\r\nvery puffy version of frybread. If using yogurt or soured milk for leavening, use 1\2 cup\r\ninstead of 1/3 cup water.\r\n- You can use mayonnaise in place of shortening for a crispy, crunchier texture.\r\n- You can also fry the dough in hot oil over a stovetop. The dough cooks rapidly and will brown\r\nin about 12 seconds and must be turned over to allow the opposite side to brown, then be removed\r\nfrom the oil and placed sideways into a colander or large bowl lined with paper towels to allow\r\nthe oil to drain off the finished product\r\n\r\nurl: https://en.wikibooks.org/wiki/Cookbook:Fry_Bread_I\r\n"
}

Part 1.2: create the static content directories

mkdir htdocs
mkdir htdocs/modules
mkdir htdocs/css

Part 1.2: Setting up our web service

datasetd provides a turn key web service defined by a simple YAML file. It can host one or more dataset collections. It provides a static file service as well as a JSON API for each collection.

Create the file recipes_api.yaml (see: https://github.com/caltechlibrary/t2t3_dataset_web_apps/blob/main/recipes_api.yaml).

Part 1.2: recipes_api.yaml

#!/usr/bin/env -S datasetd -debug
host: localhost:8001
htdocs: htdocs
collections:
  - dataset: recipes.ds
    keys: true
    create: true
    # When we successfully create (or update via a POST) an object display it
    create_success: http://localhost:8001/display_recipe.html
    # When we fail go back to the current page.
    create_error: http://localhost:8001/edit_recipe.html
    read: true
    update: true
    delete: true
    query:
      list_recipes: |
        select src
        from recipes
        order by src->>'name'

Part 1.2: Starting and stopping the web service

Starting the web service.

datasetd -debug recipes_api.yaml

Part 1.2: Starting and stopping the web service

(on macOS, Linux or Window’s WSL)

chmod 775 recipes_api.yaml

Now you can shorten start up to

./recipes_api.yaml

Part 1.3: What about our static content?

The web service is running but haven’t populated the htdocs directory.

What do you see when you go to http://localhost:8001/?

Part 1.3: What should our recipe metadata look like?

key
A unique identifier for the recipe (needed by dataset to distinguish objects).
name
A line of text. Held by aninput element
ingredients
A CSV table. Held by a textarea or for display a table.
procedure
Free text describing how to prepare the dish. Held by a textarea element

We’ll need a submit button to save a new or edited recipe.

Part 1.3: What are our web pages?

htdocs/index.html
Display a list of our recipes
htdocs/display_recipe.html
A page that shows the recipe
htdocs/edit_recipe.html
A page used to add and edit recipes we’ve collected

Part 1.3: Populating our pages using JavaScript

We’ll create four modules, one specific to each HTML page and one utility module

htdocs/modules/list_recipes.js
Display a list of our recipes
htdocs/modules/display_recipe.js
A page that shows the recipe
htdocs/modules/edit_recipe.js
A page used to add and edit recipes we’ve collected
htdocs/modules/utils.js
This module handles retrieving data from the JSON API and finding the object’s key

Part 1.3: Fire up our web service

In a terminal run our startup command

datasetd -debug recipes_api.yaml

Part 1.3: Test using your web browser

  1. Go to http://localhost:8001
  2. In your browser turn on your developer tools
  3. Reload the page and explore using your developer tools
  4. Click through the site

Part 1.3: Debugging and improving

  1. Are there issues to debug?
  2. What happens when you add a recipe?
  3. What happens when you hen you update a receipt?
  4. Can any of this be improved?

Intermission

Let’s take a short break then we’ll comeback and iterate.

I’m available for questions.

Part 2: Recipes version 2

What we are doing next

Part 2.1: Bootstrap from version 1

dataset init recipes2.ds
cp recipes_api.yaml recipes_api2.yaml
cp -vR htdocs htdocs2

On Windows:

dataset init recipes2.ds
copy recipes_api.yaml recipes_api2.yaml
copy -Recurse htdocs htdocs2

(NOTE: The first line should look familiar, the others are just time savers)

Part 2.1: Updating our YAML configuration

Part 2.1: Testing a new instance

datasetd recipes_api2.yaml

(NOTE: note the “d” at the end of “datasetd”)

Part 2.2: Desirable changes

Part 2.2: Fixing web form submission

Part 2.2: Update the HTML for edit_recipe.html

Add the following at the bottom of the page before the </body>.

<script type="module">
  import { saveRecipe } from './modules/utils.js';
  document.addEventListener('DOMContentLoaded', () => {
    const form = document.addEventListener('submit', saveRecipe);
  });
</script>

Part 2.2: Restart datasetd and test

datasetd -debug recipes_api2.yaml

Test using your web browser.

Part 2.3: Improving the UI with Web Components

  1. Extend the HTML elements available
  2. Implement components as JavaScript Modules

Part 2.3: CL-web-components

https://github.com/caltechlibrary/CL-web-components/releases

Part 2.3: Copy the web components to the modules directory

unzip $HOME/Downloads/cl-web-components-0.0.6.zip *.js
mv -v *.js htdocs2/models/

Part 2.2: Adding CSVTextarea to edit_recipe.html

See: https://github.com/caltechlibrary/t2t3_dataset_web_apps/blob/main/htdocs2/edit_recipe.html

Part 2.2: What are the attributes needed in a <csv-textarea>?

Part 2.2: Restart recipes_api2.yaml and test

Start up our web service

dataset recipes_api2.yaml
  1. Point your browser at http://localhost:8002/edit_recipe.html
  2. Turn on your developer tools
  3. Test the web component, what’s the problem you see?

Part 2.2: Getting the table populated, update utils.js

CSVTextarea has the ability to be updated from CSV text. Let’s do that.

In edit_recipe.js you need to find this.

if (data["ingredients"] !== undefined) {
  ingredientsTextarea.innerHTML = data["ingredients"];
}

And replace it with something like this.

if (data["ingredients"] !== undefined) {
  ingredientsTextarea.fromCSV(data['ingredients']);
}

Part 2.2: Test and debug

Part 3: Exploring further

Part 3: Exploring Human Interfaces

Part 3: Exploring Human Interfaces

Part 3: My Recommendations

Part 3: What I am still mulling over?

Reference: Dataset

Reference: CL-web-components

CSVTextarea
Wraps a textarea element and presents a editable table of cells
AToZUL
Wraps a UL list and creates an A to Z list
SortableTable
Wraps an HTML table making it sort-able and filterable on a column

Reference: Programming Languages

Reference: Data formats

Thank you for listening