A recipe for applications: Dataset & Web Components (Part 2)

R. S. Doiel, rsdoiel@caltech.edu

Caltech Library, Digital Library Development

TBD

Welcome to “A recipe for applications”

Welcome everyone,

This presentation builds on what we built in the Part 1.

This workshop is focused on enhancing our application using Web Components.

What we’ll learn

What are Web Components?
Why use Web Components?
Anatomy of Web Components
Using Web Components

What we’ll do

Save our static content directory
Save our recipes_api.yaml
Create our first web component, “<hello-clock></hello-clock>”
Develop an A to Z web component, “<ul-a-to-z-list></ul-a-to-z-list>” (for listing our recipes)

Workshop: “A recipe for applications”

Follow along at https://caltechlibrary.github.io/t2t3_dataset_web_apps/presentation2.html
Download the presentation zip, https://github.com/caltechlibrary/t2t3_dataset_web_apps/releases
Make suggestions for improving the workshop at https://github.com/caltechlibrary/t2t3_dataset_web_apps/issues

Getting started, Part II

You’ll need the following from Part I

Terminal application
Text Editor
Web Browser
Dataset >= 2.3.2 (or latest release)
cURL or irm
The YAML, HTML and JavaScript you developed from Part I

Part 2.1: Setting up

Next steps

Save the old the htdocs directory under htdocs1
Save the old recipes_api.yaml as recipes_api1.yaml

Part 2.1: Copying htdocs and recipes_api.yaml

On macOS, Linux and Windows using WSL

cp -fR htdocs htdocs1
cp recipes_api.yaml recipes_api1.yaml

On Windows in PowerShell

copy -Recurse htdocs htdocs1
copy recipes_api.yaml recipes_api1.yaml

HOME WORK: Make recipes_api1.yaml work. Change the the htdocs attribute to point to htdocs1 directory. Compare version 1 with the results of the workshop. Discuss among friends.

Part 2.1: Starting up our web service

The web service works just like our session
We’ll be building browser side, updating the htdocs directory
- Ready to dive into Web Components?

datasetd recipes_api.yaml

Part 2.2: The “What” of Web Components

Web Components are a W3C standard to allowing you to extend HTML
- (Web Components inherit, including accessibility features)
A Web Component encapsulates the HTML, CSS and JavaScript use to create a new HTML Element
- (They encourage sensible code re-use)
Web Components are re-usable blocks of structure, function and presentation

Part 2.2: The “Why” of Web Components

Web components provide a sustainable way to extend HTML to fit our needs
They simplify use because they are expressed as HTML elements
Web components cleanly encapsulate HTML, CSS and JavaScript
- Their impact is constrained
- They are compatible with progressive enhancement techniques
- They can normalize user experience as a part of a design system

Part 2.2: The Anatomy of a Web Component

A component is implemented as a JavaScript class
- The class extends an existing HTML element (inheriting its features)
The class contains a connectedCallback() method
The component is “registered” using the customElements.define() method
- Example: customElements.define( 'hello-clock', HelloClock );

Part 2.2: Why extend HTML elements?

So we don’t have to define everything in JavaScript

DOM, life cycle, events, etc.

So we align with existing HTML element expectations

Part 2.2: What’s the `connectedCallback()` method?

This provides a “hook” into the browser’s render engine and event handler

Part 2.2.: What’s registration?

Registration tells the web browser this is a new HTML element type
- Web browsers ignore HTML elements they don’t understand
Registration configures the new element’s relationship with HTML parser and render engine

Part 2.2: The “Hello Clock” Web Component

Let’s create a web component called “hello-clock” (“htdocs/modules/hello-clock.js”)
“hello-clock” will extends “HTML element”
It will display a hello message and the time

<hello-clock>Hi There!</hello-clock>

This will display something like, “Hi There! 09:27:23 GMT-0700 (Pacific Daylight Time)”.

Part 2.2: “hello-clock.js” defines the web component

Create htdocs/modules/hello-clock.js

// Define our new element as a class
class HelloClock extends HTMLElement {
  // Hook used by browser to instantiate the element in the page
  connectedCallback() {
    // Get the current time as an object
    const d = new Date();
    // Update the inner text to include our time string
    const message = this.innerHTML.trim();
    this.textContent = `${message} ${d.toTimeString()}`;
  }
}
// This is how the browsers learns to use the new HTML element.
customElements.define( 'hello-clock', HelloClock );

Part 2.2: Now lets create an HTML Page using “Hello Clock”

htdocs/hello-clock.html

<!DOCTYPE html>
<html lang="en-US">
    <head>
        <title>Hello Clock Example</title>
        <link rel="stylesheet" href="css/style.css">
        <script type="module" src="modules/hello-clock.js" defer></script>
    </head>
    <body>
        <h1>Hello Clock Example</h1>
        <hello-clock>Hi there!</hello-clock>
    </body>
</html>

Part 2.2: Fire up our web service and test “Hello Clock”

Point your web browser at http://localhost:8001/hello-clock.html
Open your developer tools and reload the page
Notice the logs provided by Dataset Web Service

Part 2.2: Congratulations

You’ve built your first Web Component!!!!

Let’s take a quick break and stretch before moving forward

Part 2.3: Building an A to Z list web component

Q: What does the A to Z list web component do?

A: Wraps a standard UL list providing A to Z navigation.

Part 2.3: Building an A to Z list web component

Create an HTML file for testing, htdocs/ul-a-to-z-list.html
Create our Web Component, htdocs/modules/ul-a-to-z-list.js

macOS and Linux:

touch htdocs/ul-a-to-z-list.html htdocs/modules/ul-a-to-z-list.js

Windows:

New-Item -Path htdocs/ul-a-to-z-list.html ; New-Item htdocs/modules/ul-a-to-z-list.js

Part 2.3: Building an A to Z list web component

Here’s the HTML we’ll use for our test page, ul-a-to-z-list.html.

<!DOCTYPE html>
<html lang="en-US">
    <head>
        <title>A to Z List Clock Example</title>
        <link rel="stylesheet" href="css/style.css">
        <script type="module" src="modules/ul-a-to-z-list.js" defer></script>
    </head>
    <body>
        <h1>A to Z List Example</h1>

        <ul-a-to-z-list>
            <ul>
                <li>Alex</li> <li>Betty</li> <li>Connor</li> <li>David</li>
                <li>Edwina</li> <li>Fiona</li> <li>George</li> <li>Harry</li>
                <li>Ishmael</li> <li>Leonardo</li> <li>Millie</li> <li>Nellie</li>
                <li>Ollie</li> <li>Petra</li> <li>Quincy</li> <li>Rowino</li> <li>Selvina</li>
                <li>Terry</li> <li>Ulma</li> <li>Victorio</li> <li>Willamina</li> <li>Xavier</li>
                <li>Zoran</li>
            </ul>
        </ul-a-to-z-list>
    </body>
</html>

Part 2.3: Building an A to Z list web component

(source ul-a-to-z-list_v0.js)

Start out with a minimal Class definition and customElement definition

export class AToZList extends HTMLElement {
  constructor() {
    super();
  }

  connectedCallback() {
  }
}
customElements.define('ul-a-to-z-list', AToZList);

NOTE: The constructor and callback are empty. Open the web page. Use “the inspector” in your browser’s dev tools.

Part 2.3: Introducing Shadow DOM

(source ul-a-to-z-list_v1.js)

You can build your web component using the Shadow DOM. We need to include that in our constructor.

export class AToZList extends HTMLElement {
  constructor() {
    super();
    // This next line engages the Shadow DOM
    this.attachShadow({ mode: 'open' });
  }
  connectedCallback() {
  }
}
customElements.define('ul-a-to-z-list', AToZList);

Reload you web page, what does does it look like? What shows up in the inspector?

Part 2.3: What do we want the callback to do?

We use the connectedCallback() method to to call a render method. This is what makes our shadow DOM element visible.

export class AToZList extends HTMLElement {
  constructor() { super(); this.attachShadow({ mode: 'open' }); }

  connectedCallback() { this.render(); }

  render() {
    const template = document.createElement('template');
    template.innerHTML = '<!-- Our HTML markup will go here ... -->';
    this.shadowRoot.appendChild(template.content.cloneNode(true));
  }
}
customElements.define('ul-a-to-z-list', AToZList);

What happened in our web page? What does the inspector show?

Part 2.3: Basic Structure

(source ul-a-to-z-list_v2.js)

export class AToZList extends HTMLElement {
  constructor() { super(); this.attachShadow({ mode: 'open' }); }

  connectedCallback() { this.render(); }

  render() {
    const template = document.createElement('template');
    template.innerHTML = `<style>
        /* Basic styles */
        menu { list-style-type: none; padding: 0; }
      </style>
      <menu id="menu"></menu>
      <div id="list-container"></div>`;

    this.shadowRoot.appendChild(template.content.cloneNode(true));
  }
}
customElements.define('ul-a-to-z-list', AToZList);

Part 2.3: Display Items

(source ul-a-to-z-list_v3.js)

  render() {
    const template = document.createElement('template');
    template.innerHTML = `<style>
        /* Basic styles */
        menu { list-style-type: none; padding: 0; }
      </style>
      <menu id="menu"></menu>
      <div id="list-container"></div>`;

    this.shadowRoot.appendChild(template.content.cloneNode(true));

    const listContainer = this.shadowRoot.querySelector('#list-container');
    const ulElement = this.querySelector('ul');

    if (ulElement) {
      listContainer.appendChild(ulElement.cloneNode(true));
    }
  }

What happened in our web page? What shows up in the inspector?

Part 2.3: Organize Items by Starting Letter

(source ul-a-to-z-list_v4.js)

render() {
  const template = document.createElement('template');
  template.innerHTML = `<style>
      /* Basic styles */
      menu { list-style-type: none; padding: 0; }
      .letter-section { list-style-type: none; }
    </style>
    <menu id="menu"></menu>
    <div id="list-container"></div>`;

  this.shadowRoot.appendChild(template.content.cloneNode(true));

  const listContainer = this.shadowRoot.querySelector('#list-container');
  const ulElement = this.querySelector('ul');

  if (!ulElement) return;

  const items = Array.from(ulElement.querySelectorAll('li'));
  const alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
  const sections = {};

  items.forEach(item => {
    const firstLetter = item.textContent.trim()[0].toUpperCase();
    if (!sections[firstLetter]) sections[firstLetter] = [];
    sections[firstLetter].push(item);
  });
  Object.keys(sections).forEach(letter => {
    // NOTE: Menu setup will go here
    const section = document.createElement('ul');
    section.classList.add('letter-section');
    section.id = `section-${letter}`;

    sections[letter].forEach(item => {
      section.appendChild(item.cloneNode(true));
    });

    listContainer.appendChild(section);
  });
}

Part 2.3: Add Navigation Menu

(source ul-a-to-z-list_v5.js)

render() {
  // Previous code remains the same until the sections loop but we
  // need to grab the menu element in the template.
  const menu = this.shadowRoot.querySelector('#menu');

  Object.keys(sections).forEach(letter => {
    const menuItem = document.createElement('li');
    const menuLink = document.createElement('a');
    menuLink.href = `#section-${letter}`;
    menuLink.textContent = letter;
    menuItem.appendChild(menuLink);
    menu.appendChild(menuItem);

    // Rest of the section linking code goes here
  });
}

Part 2.3: Linking

(source ul-a-to-z-list_v6.js)

  // Section linking inside the `Object.keys(sections).forEach(letter => {` loop
  const section = document.createElement('ul');
      section.classList.add('letter-section');
      section.id = `section-${letter}`;

      sections[letter].forEach(item => {
        section.appendChild(item.cloneNode(true));
      });

      listContainer.appendChild(section);
  });

Part 2.3: Back to Menu Link

(source ul-a-to-z-list_v6.js)

render() {
  // This code goes below the `Object.keys(section).foreach( ...` loop

  const backToMenuLink = this.shadowRoot.querySelector('.back-to-menu');
  if (backToMenuLink) {
    backToMenuLink.addEventListener('click', (event) => {
      event.preventDefault();
      this.scrollToSection(menu);
    });
  }

  // Add event listeners to menu links for smooth scrolling
  const menuLinks = this.shadowRoot.querySelectorAll('menu li a');
  menuLinks.forEach(link => {
    link.addEventListener('click', (event) => {
      event.preventDefault();
      const targetSection = this.shadowRoot.querySelector(link.getAttribute('href'));
      this.scrollToSection(targetSection);
    });
  }); 
}

Part 2.3: Improve Scrolling

(source ul-a-to-z-list_v6.js)

// This is a new method, it can go after the render method in our class
scrollToSection(section) {
  const yOffset = -100;
  const y = section.getBoundingClientRect().top +
              window.pageYOffset + yOffset;

  window.scrollTo({ top: y, behavior: 'smooth' });
}

Part 2.3: Style and CSS

(source ul-a-to-z-list_v7.js)

render() {
  const template = document.createElement('template');
  template.innerHTML = `
<style>
  menu { list-style-type: none; padding: 0; }
  menu li { display: inline; margin-right: 10px; }
  .letter-section { list-style-type: none; }
  .letter-section li { text-decoration: none; font-weight: none; }
  .back-to-menu { display: block; margin-top: 20px; }
</style>
<menu id="menu"></menu>
<div id="list-container"></div> 
${this.hasAttribute('long') ? 
  '<a class="back-to-menu" href="#menu">Back to Menu</a>' : ''
}`;

  // Rest of the render method remains the same
}

Part 2.3: A final working A to Z list

(source ul-a-to-z-list.js)

/**
 * ul-a-to-z-list.js, this wraps a standard UL list providing A to Z navigation list
 */
export class AToZList extends HTMLElement {
  constructor() {
    super();
    this.attachShadow({ mode: 'open' });
  }

  connectedCallback() {
    this.render();
  }

  render() {
    const template = document.createElement('template');
    template.innerHTML = `
      <style>
        menu { list-style-type: none; padding: 0; }
        menu li { display: inline; margin-right: 10px; }
        .letter-section { list-style-type: none; }
        .letter-section li { text-decoration: none; font-weight: normal; }
        .back-to-menu { display: block; margin-top: 20px; }
      </style>
      <menu id="menu"></menu>
      <div id="list-container"></div>
      ${this.hasAttribute('long') ? '<a class="back-to-menu" href="#menu">Back to Menu</a>' : ''}
    `;

    this.shadowRoot.appendChild(template.content.cloneNode(true));

    const listContainer = this.shadowRoot.querySelector('#list-container');
    const menu = this.shadowRoot.querySelector('#menu');

    const ulElement = this.querySelector('ul');
    if (!ulElement) return;

    const items = Array.from(ulElement.querySelectorAll('li'));
    const alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
    const sections = {};

    items.forEach(item => {
      const firstLetter = item.textContent.trim()[0].toUpperCase();
      if (!sections[firstLetter]) {
        sections[firstLetter] = [];
      }
      sections[firstLetter].push(item);
    });

    alphabet.split('').forEach(letter => {
      if (sections[letter]) {
        const menuItem = document.createElement('li');
        const menuLink = document.createElement('a');
        menuLink.href = `#section-${letter}`;
        menuLink.textContent = letter;
        menuLink.addEventListener('click', (event) => {
          event.preventDefault();
          const targetSection = this.shadowRoot.querySelector(`#section-${letter}`);
          this.scrollToSection(targetSection);
        });
        menuItem.appendChild(menuLink);
        menu.appendChild(menuItem);

        const section = document.createElement('ul');
        section.classList.add('letter-section');
        section.id = `section-${letter}`;
        const sectionHeading = document.createElement('li');
        const sectionHeadingLink = document.createElement('a');
        sectionHeadingLink.href = `#menu`;
        sectionHeadingLink.textContent = letter;
        sectionHeadingLink.addEventListener('click', (event) => {
          event.preventDefault();
          this.scrollToSection(menu);
        });
        sectionHeading.appendChild(sectionHeadingLink);
        section.appendChild(sectionHeading);

        sections[letter].forEach(item => {
          const clonedItem = item.cloneNode(true);
          section.appendChild(clonedItem);
        });
        listContainer.appendChild(section);
      }
    });

    const backToMenuLink = this.shadowRoot.querySelector('.back-to-menu');
    if (backToMenuLink) {
      backToMenuLink.addEventListener('click', (event) => {
        event.preventDefault();
        this.scrollToSection(menu);
      });
    }
  }

  scrollToSection(section) {
    const yOffset = -100;
    const y = section.getBoundingClientRect().top + 
                      window.pageYOffset + yOffset;

    window.scrollTo({
      top: y,
      behavior: 'smooth'
    });
  }
}

customElements.define('ul-a-to-z-list', AToZList);

Part 2.3: Congratulations! You’ve created two web components!

<hello-clock> (minimal)
<ul-a-to-z-list> (complex)

Let’s take a quick break and stretch before moving forward

Part 2.4: Using our A to Z list

Modify is htdocs/index.html
Wrap the ul list with our <ul-a-to-z-list></ul-a-to-z-list>
Add our component module <script type="module" src="modules/ul-a-to-z-list.js" defer></script>
Test, what happens on page reload?

Part 2.4: Integrating the A to Z list

The web component internals are opaque to the outside elements
our index_recipes.js functions don’t see the internal UL list now
What do we do?

Part 2.4, Approaches

Pre-render the UL list (add middleware)
Merge the index_recipes into the web component module
Modify the web component, Modify index_recipes so they play nice

Part 2.4, Which approach?

Context, re-use and complexity are our constraints
- Is this problem a “common case”? Is there an easy solution?
- Is an increase in complexity worth supporting the common case?
- Do we drop the common case for unique element and build in a index_recipes function?

Before picking a path review our existing code base against our constraints

Part 2.4: Think about the innerHTML, look at `index_recipes.js`

listRecipes(): Gets the outer element, retrieves data and then invokes populateUL()
populateUL(): Populates the UL lists with the data

Are either easy to adapt to our component?

Part 2.4: Review `listRecipes()`

It could get the handle for the <ul-a-to-z-list></ul-a-to-z-list>
listRecipes() triggers retrieving the data
It invokes populateUL() which populates that innerHTML!

We can keep our component general with by adapting how listRecipes() works

Part 2.4: Taking advantage of inheriting HTML element

(source index_recipes_v2.js)

async function listRecipes() {
 const aToZList = document.querySelector('ul-a-to-z-list');
 console.log(aToZList);
 const ul = document.createElement('ul');
 const data = await getRecipes();
 populateUL(ul, data);
 aToZList.innerHTML = ul.outerHTML;
 // NOTE: we need to trigger aToZList render method still.
 // Feel ugly?
 aToZList.render()
}

What does the updates look like?

Part 2.4: Problems?

Is ordering is weird? Why?
Do we always have to called aToZList.render() to update?

Part 2.4: Ordering problem

Get rid of dummy text “Recipe name goes here”

Part 2.4: Adding an event handler to our component

Which event is triggered with the line aToZList.innerHTML = ul.outerHTML?

Update our component to use a mutation observer
Remove our render call from index_recipes.js.

(source ul-a-to-z-list_v8.js) (source index_recipes_v3.js)

Part 2.4: Lessons learned

Remember that our web components, inherit from HTML, we can treat them like HTML elements
Display elements can mutate, we should support mutation even when implementing display only components
- The JavaScript defining a web component extends HTML, that should be the focus
- The page level JavaScript is about adding behaviors at the page level
- Balance the areas of responsibilities
- A style element in a component can function like a style element in the page head element
  - CSS can be imported

Part 2.4: Things to remember

HTML5 provides allot of capability, not everything needs to be a component
Components let us encapsulate structure, behavior of visual style
Components work best when they enhance the semantic structure of HTML and the page

Reference: Dataset

Reference: Web Components

Reference: Programming Languages

HTML5, structured content
CSS, layout and style
JavaScript, behaviors and web components
SQL and SQLite
- SQLite Tutorial
- Or typing into your favorite LLM, but be prepared to validate the recommendations
MDN

Reference: Data formats

Thank you for listening

View presentations:
View the repository: https://github.com/caltechlibrary/t2t3_dataset_web_apps
Comment on this presentation: https://github.com/caltechlibrary/t2t3_dataset_web_apps/issues
Author: R. S. Doiel, mailto:rsdoiel@caltech.edu

A recipe for applications: Dataset & Web Components (Part 2)

Welcome to “A recipe for applications”

What we’ll learn

What we’ll do

Workshop: “A recipe for applications”

Getting started, Part II

You’ll need the following from Part I

Part 2.1: Setting up

Next steps

Part 2.1: Copying htdocs and recipes_api.yaml

Part 2.1: Starting up our web service

Part 2.2: The “What” of Web Components

Part 2.2: The “Why” of Web Components

Part 2.2: The Anatomy of a Web Component

Part 2.2: Why extend HTML elements?

Part 2.2: What’s the connectedCallback() method?

Part 2.2.: What’s registration?

Part 2.2: The “Hello Clock” Web Component

Part 2.2: “hello-clock.js” defines the web component

Part 2.2: Now lets create an HTML Page using “Hello Clock”

Part 2.2: Fire up our web service and test “Hello Clock”

Part 2.2: Congratulations

You’ve built your first Web Component!!!!

Part 2.3: Building an A to Z list web component

Part 2.3: Building an A to Z list web component

Part 2.3: Building an A to Z list web component

Part 2.3: Building an A to Z list web component

Part 2.3: Introducing Shadow DOM

Part 2.3: What do we want the callback to do?

Part 2.3: Basic Structure

Part 2.3: Display Items

Part 2.3: Organize Items by Starting Letter

Part 2.3: Add Navigation Menu

Part 2.3: Linking

Part 2.3: Back to Menu Link

Part 2.3: Improve Scrolling

Part 2.3: Style and CSS

Part 2.3: A final working A to Z list

Part 2.3: Congratulations! You’ve created two web components!

Part 2.4: Using our A to Z list

Part 2.4: Integrating the A to Z list

Part 2.4, Approaches

Part 2.4, Which approach?

Part 2.4: Think about the innerHTML, look at index_recipes.js

Part 2.4: Review listRecipes()

Part 2.4: Taking advantage of inheriting HTML element

Part 2.4: Problems?

Part 2.4: Ordering problem

Part 2.4: Adding an event handler to our component

Part 2.4: Lessons learned

Part 2.4: Things to remember

Reference: Dataset

Reference: Web Components

Reference: Programming Languages

Reference: Data formats

Thank you for listening

Part 2.2: What’s the `connectedCallback()` method?

Part 2.4: Think about the innerHTML, look at `index_recipes.js`

Part 2.4: Review `listRecipes()`