One place for hosting & domains

      Gatsby

      How To Create a Custom Source Plugin in Gatsby


      The author selected the Internet Archive to receive a donation as part of the Write for DOnations program.

      Introduction

      In building a website or application, often one of the most difficult tasks is pulling data from multiple sources and collating it into a uniform output. A common way to solve this is to use entirely different build systems for different parts of a site, but this sometimes adds complexity while making uniformity harder to achieve. This is where Gatsby, a data-driven Static Site Generator (SSG), can provide a solution.

      One of the core objectives of Gatsby is to solve this problem for developers, and source plugins are the main way it does so. A source plugin is a bundle of code that handles bringing data into the Gatsby ecosystem from a given source. Sources can be from the local filesystem as with Markdown files, databases, published data feeds, or even completely dynamic remote data sources such as APIs.

      In this tutorial, you will build your own custom source plugin to bring new data into Gatsby from a real-world API. You will also format the data so that it can be accessed throughout Gatsby, and by the end of the tutorial, have a working project that builds static HTML from your new dynamic data source.

      Prerequisites

      Before starting, here are a few things you will need:

      • A local installation of Node.js for running Gatsby and building your site. The installation procedure varies by operating system, but DigitalOcean has guides for Ubuntu 20.04 and macOS, and you can always find the latest release on the official Node.js download page.
      • Some familiarity with JavaScript, for working in Gatsby. The JavaScript language is an expansive topic, but a good starting spot is our How To Code in JavaScript series.
      • Some familiarity with web APIs, Node.js, and JSON.
      • A new Gatsby project, scaffolded from gatsby-starter-default. For satisfying this requirement and building a new Gatsby project from scratch, you can refer to Step 1 of the How To Set Up Your First Gatsby Website tutorial.
      • Some familiarity with React and JSX, as well as HTML elements, if you want to customize the user interface (UI) of your posts beyond what is covered in this tutorial.

      This tutorial was tested on Node.js v14.16.1, npm v6.14.12, Gatsby v3.13.0, and node-fetch v2.6.2.

      Step 1 — Scaffolding Files and Installing Dependencies

      When building something, the first step is always to get your tools and parts in order. In this step, you will put the initial building blocks of your source plugin in place by creating the necessary file structure and installing the dependencies your code will rely on.

      Since this will be a local plugin, create a directory within your Gatsby project to hold the source code of the plugin under a root-level plugins directory. You can do this by manually creating the folder in your file browser, or from the command line in the root of your project with the mkdir command:

      • mkdir -p plugins/my-custom-source-plugin

      Note: If you want to develop your plugin outside of your Gatsby project directory, you can do so, but it requires some extra steps to get Gatsby to pull the files into your site. For more on this, check out the official Gatsby documentation.

      Next, you need to create a package.json file to mark this directory as a Node.js package with its own dependencies. To create this file and pre-fill some required fields, use the following command:

      • cd plugins/my-custom-source-plugin
      • npm init -y

      This command navigates to your newly created plugin folder and then uses npm init to initialize a new package. The -y flag skips some questions that are irrelevant to this project and fills in the package.json file with the minimum required values.

      Now that package.json exists, you can add dependencies to your plugin that will make coding out functionality easier. Go ahead and install the only extra dependency you will be needing in this tutorial, node-fetch, by using the following command:

      • npm install node-fetch@^2

      Finally, create the gatsby-node.js file that will end up holding the main code of the source plugin:

      Note: If you are frequently building Gatsby plugins, you might find their plugin template helpful.

      Now that you have created the file structure to support your plugin and installed the initial dependencies, you will move on to giving instructions to Gatsby on how to find and load your plugin.

      Step 2 — Loading and Configuring the Plugin

      As is the case with any Gatsby plugin or theme, Gatsby has to be instructed on where and how to load the plugin from. To do this, you edit the main Gatsby config file gatsby-config.js , which resides in the root of your Gatsby project. Open the file in your editor of choice and add the following highlighted line:

      gatsby-config.js

      module.exports = {
      ...
        plugins: [
          `gatsby-plugin-react-helmet`,
          `gatsby-plugin-image`,
          {
            resolve: `gatsby-source-filesystem`,
            options: {
              name: `images`,
              path: `${__dirname}/src/images`,
            },
          },
          `my-custom-source-plugin`,
          `gatsby-transformer-sharp`,
      ...
      

      Because your plugin’s source code lives in the plugins directory, all that is required to get Gatsby to load it is to pass in the name of the sub-directory where it can be found within that folder. Your plugin also does not take any options at the moment, so there is no need to pass an options object to it in the Gatsby configuration block.

      Save gatsby-config.js and exit from the file.

      You have now configured Gatsby to load your custom source plugin, as well as told it exactly where to find the source code it should execute. In the next step, you will build out this source code to pull data into the Node.js runtime from your custom remote source.

      Step 3 — Pulling Raw Data Into Node.js

      In the previous step, you configured Gatsby to load and execute your custom source plugin’s code, but you still need to build out this code to accomplish the task of bringing new data into the Gatsby ecosystem. In this step, you will write the code that does this, fetching remote data via node-fetch and preparing it for use in future steps.

      Source plugins can pull data from almost anywhere, local or remote, but in this tutorial your source plugin will be specifically pulling titles and excerpts from the Computer Programming Books category in Wikipedia via their public API.

      Open your my-custom-source-plugin/gatsby-node.js file in your plugins directory and add the following code:

      plugins/my-custom-source-plugin/gatsby-node.js

      const fetch = require('node-fetch').default
      
      /**
       * Fetch a list of computer books from Wikipedia, with excerpts
       */
      async function getWikiProgrammingBooks() {
        const BASE_ENDPOINT = "https://en.wikipedia.org/w/api.php?action=query&format=json&utf8=1&redirects=1";
      
        // Get list of books
        const listEndpoint = new URL(BASE_ENDPOINT);
        listEndpoint.searchParams.append('list', 'categorymembers');
        listEndpoint.searchParams.append("cmtitle", "Category:Computer_programming_books");
        listEndpoint.searchParams.append("cmlimit", "10");
        const listResults = await (await fetch(listEndpoint.toString())).json();
      
      
        // Extract out the page IDs from the list
        const pageIds = listResults.query.categorymembers.map((listing) => listing.pageid);
      
        // Fetch details for page IDs
        const extractEndpoint = new URL(BASE_ENDPOINT);
        extractEndpoint.searchParams.append("pageids", pageIds.join("|"));
        extractEndpoint.searchParams.append("prop", "extracts|info");
        extractEndpoint.searchParams.append("exintro", "");
        extractEndpoint.searchParams.append("explaintext", "");
        extractEndpoint.searchParams.append("inprop", "url");
      
        const bookResult = await (await fetch(extractEndpoint.toString())).json();
      
        return Object.values(bookResult.query.pages);
      }
      

      In this code, you have created a reusable function that can be called to return a list of computer programming books, along with their page IDs (a unique ID within Wikipedia) and excerpts/extracts. In the first part of the function, you build up the right API URL to use to fetch the initial list of titles and IDs belonging to a specific category (Computer Programming Books). The URL constructor and interface is used to make modifying the query string more readable and manageable.

      You use the fetch method from node-fetch to make a GET request to the constructed URL, which returns a list of the book titles with their IDs. That response is then turned into an array of just the pageid values, subsequently used to query the Wikipedia API again, this time requesting extracts and meta info generated for the given page ID. The page IDs are joined by the pipe character (|), as the Wikipedia API uses this format to accept multiple IDs through a single string value.

      Finally, since the results for page excerpts come back as an object with each book listing nested under its own ID as a key, you use Object.values() to omit the page ID key and convert the results into an array before returning them.

      If you were to log the output of this function, it would look something like this:

      [
        {
          "pageid": 379671,
          "ns": 0,
          "title": "The C Programming Language",
          "extract": "The C Programming Language (sometimes termed K&R, after its authors' initials) is a computer programming book written by Brian Kernighan and Dennis Ritchie...",
          "fullurl": "https://en.wikipedia.org/wiki/The_C_Programming_Language",
          ...
        },
        ...
      ]
      

      Make sure to save your changes, but keep this file open as you will be adding more code to it in the next step.

      In this step you used node-fetch to retrieve remote source content and expose it within the gatsby-node.js file. In the next step, you will normalize the content as you use it to create new Gatsby nodes to use throughout the Gatsby project.

      Step 4 — Normalizing Data and Creating Nodes

      Fetching remote content and bringing it into gatsby-node.js in the previous step doesn’t mean that it is now accessible throughout Gatsby; in order to share data in a universal way, Gatsby uses the concept of nodes, which are shared across a unified GraphQL data layer. In this step, you will create these nodes, formatting your new content to match.

      Although you can now retrieve and access the results from Wikipedia by calling getWikiProgrammingBooks(), you still need to add the code to integrate this with Gatsby’s node system. In the same gatsby-node.js file from the previous step, add this new block of code to handle generating the nodes:

      plugins/my-custom-source-plugin/gatsby-node.js

      const fetch = require('node-fetch').default
      
      ...
      
      exports.sourceNodes = async ({ actions, createContentDigest, createNodeId }) => {
        // Arbitrary node type constant
        const BOOK_TYPE = 'BookWikiPage';
      
        // Get books
        const bookResults = await getWikiProgrammingBooks();
      
        // Convert raw book results to nodes
        for (const book of bookResults) {
          actions.createNode({
            ...book,
            id: createNodeId(`${BOOK_TYPE}-${book.pageid}`),
            parent: null,
            children: [],
            internal: {
              type: BOOK_TYPE,
              contentDigest: createContentDigest(book)
            }
          })
        }
      };
      

      In this code block, you are iterating over each book returned by getWikiProgrammingBooks and creating a Gatsby node for it via the createNode method. Each property and value passed into createNode has importance, and is worth consideration:

      • ...book is used to spread the key-value pairs from your Wikipedia API object into the Gatsby node you are creating. This means that later on you can access node.title, as it will be copied from book.title.
      • id is a globally unique value within Gatsby. To make each book’s ID unique within your own plugin, you are combining the book type with the Wikipedia page ID to form an ID string. However, because you can’t be sure what IDs other plugins are using, you’ve used the best practice of passing your ID to createNodeId, which is a Gatsby helper function that ensures the ID is turned into something globally unique.
      • parent is a field that can be used to link your node to another via ID, marking this node as a child. Since each book is its own entity, unconnected to other nodes, you have left this as null, signifying it does not have a parent.
      • children is similar to parent as a way to link nodes, but takes an array of IDs. Since each book has no children, you have left the array empty.
      • internal is an object that groups together fields highly-specific to Gatsby’s internal node management system and other plugins. It can only contain official fields, which is why you did not spread the book object into it.
      • type is a globally unique string that describes the type of node you are creating, and will be used later when querying for nodes via GraphQL.
      • contentDigestis a hash string, which is built from the contents of the node and the Gatsby createContentDigest helper utility. This field helps Gatsby detect when a node has changed, as the hash string will change if any properties of the book object are modified.

      You have just added code that takes your source content and creates new Gatsby nodes with it, sharing them across the Gatsby environment. In the next step, you will verify that these nodes appear in the GraphQL data layer and can be queried.

      Step 5 — (Optional) Inspecting Node Output with the GraphQL API

      By now, you have pulled your source content into Gatsby and used it to create new nodes. As an alternative to manually debugging with breakpoints or log statements, in this step you will use the interactive GraphQL IDE to verify that these new nodes are being created and are able to be queried with the GraphQL API.

      Go ahead and start up your local development server by running this command from the root of your Gatsby project:

      Note: At the time of writing this tutorial, a problem with Gatsby’s dependency chain introduced an error that can return the message Error: Cannot find module 'gatsby-core-utils' when attempting to start the development server. If you encounter this error, run the following:

      • npm install gatsby-core-utils

      This will re-install the Gatsby core utilities and will resolve the dependency problem. For more information on this, check out the GitHub issue for this Gatsby error.

      In addition to launching a live version of your Gatsby site, the develop command also exposes a local GraphQL server and IDE. To verify that your code in gatsby-node.js is creating all the book nodes, you will use this GraphQL query to fetch the book titles, page IDs, and Gatsby IDs:

      {
        allBookWikiPage {
          edges {
            node {
              title
              pageid
              id
            }
          }
        }
      }
      

      To run this query, either open up the interactive GraphQL IDE at localhost:8000/___graphql and paste the query into the left side before executing, or query it via cURL:

      • curl --location --request POST 'http://localhost:8000/___graphql'
      • --header 'Content-Type: application/json'
      • --data-raw '{
      • "query": "{ allBookWikiPage { edges { node { title pageid id } } } }"
      • }'

      The response JSON will look something like this:

      {
        "data": {
          "allBookWikiPage": {
            "edges": [
              {
                "node": {
                  "title": "The C Programming Language",
                  "pageid": 379671,
                  "id": "818771ca-40aa-5cfd-b9e7-fddff093d5ec"
                }
              },
              ...
            ]
          }
        },
        "extensions": {}
      }
      

      Having verified that your new custom source nodes have been created and are accessible in the GraphQL data layer, the next step is to use them to create visible content for visitors of your site or application.

      Step 6 — (Optional) Creating Pages Based on Nodes

      So far, all of the previous steps have been focused on creating internal Gatsby nodes, including the last step of verifying their creation and ability to be retrieved. However, these nodes are only visible to code running in your Gatsby project, not to visitors of your site or application. In this step, you will add a React page template file and wire it up to your nodes so that your source plugin content turns into actual public-facing webpages.

      There are multiple ways to create pages based on Gatsby nodes, but for this tutorial you will be using the File System Route API, which creates pages based on a special filename syntax.

      First, create an empty file in src/pages with a filename of {BookWikiPage.title}.js. The curly braces tell Gatsby that the filename is using the File System Route API, and inside the braces, BookWikiPage.title tells Gatsby to create a page for each unique book title. Note that you are no longer working on files within the plugins directory, but are now working inside of the main Gatsby project.

      Next, add code to that file that will take the book node and display it as a webpage:

      src/pages/{BookWikiPage.title}.js

      import { graphql } from "gatsby";
      import * as React from "react";
      import Layout from "../components/layout";
      import Seo from "../components/seo";
      
      export default function BookPageTemplate({ data: { bookWikiPage } }) {
        const { title, extract, fullurl } = bookWikiPage;
        return (
          <Layout>
            <Seo title={title} />
            <h1>{title}</h1>
            <blockquote>{extract}</blockquote>
      
            <i>This article uses material from the Wikipedia article <a href={fullurl} target="_blank" rel="noreferrer">"{title}"</a>, which is released under the <a href="https://creativecommons.org/licenses/by-sa/3.0/">Creative Commons Attribution-Share-Alike License 3.0</a>.</i>
          </Layout>
        );
      }
      
      export const pageQuery = graphql`
        query ($id: String!) {
          bookWikiPage(id: { eq: $id }) {
            title
            extract
            fullurl
          }
        }
      `;
      

      At the end of your code is an exported variable called pageQuery, which uses the Gatsby GraphQL tag. Gatsby will evaluate the GraphQL query that follows it, passing the results to the BookPageTemplate function.

      The BookPageTemplate function, which is a React component, then takes the results of the GraphQL query and displays them as part of a web page by embedding the values into the JSX that it returns. The title of the book is used as the main heading and title of the page, the extract is displayed as a block quote, and a link to the full Wikipedia entry page is embedded at the bottom.

      You also mark the BookPageTemplate function as the default export by using export default before its declaration, as Gatsby expects to find the React component responsible for producing the final rendered page as the default export of each page template file.

      Having added the React template code to the file, save the changes and close it. Navigate to http://localhost:8000/the-c-programming-language/ to render a sample page:

      Screenshot showing a book listing page generated for

      Note: For a more manual approach to creating pages based on nodes, you can use the createPages API inside of gatsby-node.js.

      To display a listing of these new nodes and their associated pages, you will also create a dedicated listing page, which will display all the books in one location. Under src/pages, create a new file with the filename of books.js. Next, add the following code into it:

      src/pages/books.js

      import { graphql, Link } from "gatsby";
      import * as React from "react";
      import Layout from "../components/layout";
      import Seo from "../components/seo";
      
      export default function BookListingsPageTemplate({ data: { allBookWikiPage } }) {
        return (
          <Layout>
            <Seo title="Programming Books Listing" />
            <p>Here are some computer programming books that have their own Wikipedia entries:</p>
      
            {allBookWikiPage.edges.map((edge) => {
              const node = edge.node;
              return (
                <details key={node.title}>
                  <summary>{node.title}</summary>
      
                  <div className="details-body">
                    <p>{node.extract}</p>
                    <div className="links">
                      <Link href={node.gatsbyPath}>Internal Page</Link>
                      <a rel="noreferrer" href={node.fullurl}>Wikipedia Page</a>
                    </div>
                  </div>
                </details>
              )
            })}
          </Layout>
        );
      }
      
      export const pageQuery = graphql`
        query {
          allBookWikiPage {
            edges {
              node {
                title
                extract
                gatsbyPath(filePath: "/{BookWikiPage.title}")
                fullurl
              }
            }
          }
        }
      `;
      

      Similar to the {BookWikiPage.title}.js page template, this file also uses the GraphQL pageQuery tag to pull in data from the GraphQL layer and pass it to a React component. However, while the previous template rendered a single book based on ID, this template will render a listing of all the books, while linking to the individual book pages created previously.

      Each book listing uses a <details> element, which enables the listing to expand to show the full excerpt of the book and links or to collapse to show just the title. Following best practices, you also pass a unique value to key while iterating through the array, using the Gatsby Link component for internal links and a tags for external links.

      The gatsbyPath(filePath: "/{BookWikiPage.title}") string in the GraphQL query uses the special gatsbyPath() function to retrieve the public path that will be created based on the File System Route API filename that is passed in.

      Save and exit from this file.

      Note: When changing data sources for components, the hot re-loading feature will sometimes return an error like the following: error Cannot query field "gatsbyPath" on type "BookWikiPage" graphql/template-strings. To fix this error, restart the development server manually by ending the process and running npm run develop again.

      With all the books on one page, even with collapsible sections things have gotten a little crowded, so the final step is to add some styling to make it easier for visitors to read the listing. Create a new stylesheet file at src/styles/books.css. You can do this in your file browser or with the command line from the root of your Gatsby project:

      • mkdir -p ./src/styles
      • touch ./src/styles/books.css

      Next, add the following CSS into the file:

      src/styles/books.css

      details {
        border: 1px dotted black;
        margin: 6px;
        padding: 6px;
      }
      
      .details-body {
        background-color: #eedeff;
        margin: 4px 0px 2px 12px;
        padding: 4px;
        border-radius: 4px;
      }
      
      .links {
        display: flex;
        justify-content: space-evenly;
      }
      

      This CSS adds a border around each book listing, spacing and margins within the listing, and spacing between the internal and external book links. After adding the CSS into the file, save and close it before moving on.

      Finally, update the book listing page template to pull this CSS file in:

      src/pages/books.js

      import { graphql, Link } from "gatsby";
      import * as React from "react";
      import Layout from "../components/layout";
      import Seo from "../components/seo";
      import "../styles/books.css";
      

      Save and close this file with the newly added CSS import line.

      To see the results, run the develop command again to bring up the development server and preview the new pages:

      You can now access your book listing page at localhost:8000/books/.

      Screenshot showing the book listings page, using the template file in the tutorial. Each section can be expanded to show the excerpt and links or collapsed to show just the title.

      You have now not only built a Gatsby source plugin from scratch, but also used it to generate pages based on a React template.

      Conclusion

      By following the steps in this tutorial, you have now finished building a custom source plugin that brings outside content into your Gatsby project and used it to power new pages within your site.

      There is a lot of depth to source plugins. If you are interested in following best practices and learning more advanced source plugin concepts, here are some areas that might interest you:

      If you would like to read more about Gatsby, check out the rest of the How To Create Static Web Sites with Gatsby.js series.



      Source link

      How To Generate Pages from Markdown in Gatsby


      The author selected the Internet Archive to receive a donation as part of the Write for DOnations program.

      Introduction

      One of the key features of the popular static site generator Gatsby is its flexibility in consuming content sources. Gatsby can generate static pages from almost any data source, such as a Content Management System (CMS), API, database, or even a local file system.

      Markdown files are a popular file-based source to use with Gatsby. Markdown is a markup language that is lightweight and designed for readability, which makes it ideal for documentation projects, blog posts, or any text-heavy websites.

      In this tutorial, you will create a Gatsby-powered static site that builds itself from local Markdown source files, using the gatsby-source-filesystem plugin to collect the files and the gatsby-transformer-remark plugin to convert them into HTML.

      Prerequisites

      Before starting, here are a few things you will need:

      • A local installation of Node.js for running Gatsby and building your site. The installation procedure varies by operating system, but DigitalOcean has guides for Ubuntu 20.04 and macOS, and you can always find the latest release on the official Node.js download page.
      • Some familiarity with JavaScript, for working in Gatsby. The JavaScript language is an expansive topic, but a good starting spot is our How to Code in JavaScript series.
      • Some familiarity with React and JSX, as well as HTML elements, if you want to customize the user interface (UI) of your posts beyond what is covered in this tutorial.
      • A new Gatsby project named markdown-tutorial, scaffolded from gatsby-starter-default. To build a new Gatsby project from scratch, you can refer to Step 1 of the How To Set Up Your First Gatsby Website tutorial.

      This tutorial was tested on Node.js v14.16.1, npm v6.14.12, Gatsby v3.10.2, gatsby-source-filesystem v3.10.0, and gatsby-transformer-remark v4.7.0.

      Step 1 — Standardizing Your Markdown Content

      Given Markdown’s flexibility as a markup language, documents written in Markdown can take on various forms and levels of complexity. Gatsby does not require you to format all your Markdown files in the same way, but to make your site and URLs easier to organize, you will first standardize how you write and store your Markdown files within this project.

      For this tutorial, you will add blog posts to your site, and each post will have a title, customized URL, publication date, and post body. You will start with a single post, Learning About Gatsby, where you can share your experience of learning about Gatsby with the world.

      In preparation for storing this and future posts, create a new directory under src, named blog. If you prefer to do this from the command line, you can do so with mkdir from the root directory of the project:

      Next, create an empty Markdown file within that new directory. For the purposes of this tutorial, filenames do not control the published URL or SEO, so the naming conventions here are just to keep things organized. Give your new post a filename of learning-about-gatsby.md.

      You can either do this in your IDE, or use the terminal with touch:

      • touch src/blog/learning-about-gatsby.md

      Finally, fill in the actual content of the post. Open the file in your editor of choice and add the following:

      markdown-tutorial/src/blog/learning-about-gatsby.md

      ---
      title: "Learning about Gatsby"
      slug: "/blog/learning-about-gatsby"
      date: "2021-08-01"
      ---
      
      ## What I'm Working On
      
      Right now, I'm working through a [DigitalOcean tutorial](https://www.digitalocean.com/community/tutorials) on using [Gatsby](https://www.gatsbyjs.com/) with Markdown.
      

      The top section enclosed by --- is called frontmatter and consists of key-value pairs. It is written in YAML, which is a configuration language outside of Markdown. Property keys go on the left, and your values go on the right. In your frontmatter block, you have defined values for your post’s title, the publication date, and the slug. The slug is the customized portion of the post’s URL that comes after your domain name (also known as the path).

      Any text below the frontmatter will become the post body. Within that area, you have a heading of What I'm Working On, with two hash symbols (##) at the beginning of the line. In Markdown, this indicates a level-2 heading, which Gatsby will convert into <h2></h2> in HTML. Within the text below that heading, you have some links that are written using Markdown syntax: [link_text](link_destination).

      Save your changes to learning-about-gatsby.md and then close the file.

      You have just created your first post using Markdown and saved it within your project’s source code. You have also standardized the format, including specific property values in the frontmatter of the blog posts. This keeps your source code organized and will be important for later steps. For now, the next step is to install and configure the plugins that are required for Gatsby to process this new Markdown post.

      Step 2 — Installing and Configuring the Required Plugins

      With your Markdown file in place, it is time to tell Gatsby where to find them and process them. In this step, you will install the plugins that are required to achieve this and update the Gatsby configuration to load them.

      Two plugins are required to process Markdown in Gatsby: gatsby-transformer-remark and gatsby-source-filesystem. gatsby-transformer-remark will parse your Markdown and convert your frontmatter into fields that Gatsby can query, and gatsby-source-filesystem will allow you to bring data from your local filesystem into your application.

      Install both at the same time by running the following command in your Gatsby project directory:

      • npm install gatsby-source-filesystem gatsby-transformer-remark

      After you install both plugins, open up the main Gatsby configuration file that lives in the root of your project folder: gatsby-config.js. By editing this file, you are telling Gatsby how to use the newly installed plugins to read your Markdown files and begin processing them. Add the following code to your config file:

      markdown-tutorial/gatsby-config.js

      module.exports = {
      ...
        plugins: [
          `gatsby-plugin-react-helmet`,
          `gatsby-plugin-image`,
          {
            resolve: `gatsby-source-filesystem`,
            options: {
              name: `images`,
              path: `${__dirname}/src/images`,
            },
          },
          {
            resolve: `gatsby-source-filesystem`,
            options: {
              name: `blog`,
              path: `${__dirname}/src/blog`,
            },
          },
          `gatsby-transformer-remark`,
          `gatsby-transformer-sharp`,
      ...
      

      The first block loads the gatsby-source-filesystem plugin and passes it an options object telling it to scan the src/blog directory for files and use the blog name as a label for the collection. The final line loads the gatsby-transformer-remark plugin with all default options, since you don’t need to customize them for this tutorial.

      You have now configured Gatsby to load the two plugins that are necessary for scanning in and parsing your Markdown files. In the next step, you will create a page template file for Gatsby to combine with your Markdown content and render as web pages.

      Step 3 — Creating a Page Template File

      Gatsby will now scan your Markdown files and process them with gatsby-transformer-remark, but it still needs instructions on how to display them in the browser. This step covers how to give it those instructions by adding a page template file (also known as a page template component).

      First, create an empty file in src/pages with a filename of {MarkdownRemark.frontmatter__slug}.js. The filename needs to exactly match this, because it uses a Gatsby API called the File System Route API, in which filenames dictate the routes (URLs) created on your site.

      Note: The File System Route API is a newer feature in Gatsby, and you might still see other Markdown page tutorials that involve the createPages API in gatsby-node.js instead. Although it is not deprecated, that API is no longer necessary for the use case covered by this tutorial. However, if you have a project that involves creating arbitrary pages that don’t mirror the file system or exceeds the capabilities of the File System Route API in some other way, you might still need to use the createPages approach.

      To create your component, add the following code to the file:

      markdown-tutorial/src/pages/{MarkdownRemark.frontmatter__slug}.js

      import { graphql } from "gatsby";
      import * as React from "react";
      import Layout from "../components/layout";
      import Seo from "../components/seo";
      
      export default function BlogPostTemplate({ data: { markdownRemark } }) {
        const { frontmatter, html } = markdownRemark;
        return (
          <Layout>
            <Seo title={frontmatter.title} />
            <h1>{frontmatter.title}</h1>
            <h2>{frontmatter.date}</h2>
            <div className="post-body" dangerouslySetInnerHTML={{ __html: html }} />
          </Layout>
        );
      }
      
      export const pageQuery = graphql`
        query ($id: String!) {
          markdownRemark(id: { eq: $id }) {
            html
            frontmatter {
              date(formatString: "MMMM DD, YYYY")
              title
            }
          }
        }
      `;
      

      This code has two main sections. The one at the end of the file is pageQuery, which uses the Gatsby GraphQL tag to evaluate the GraphQL query that follows it. The results of that query are passed to the BlogPostTemplate function and give you access to properties of the post fetched over GraphQL.

      The BlogPostTemplate function is a React component that returns JSX. Within the BlogPostTemplate you display the values of each post’s standardized frontmatter fields, the title and the date, in header elements. The actual body of the post you have placed into a <div> with a class of post-body, using React’s dangerouslySetInnerHTML property to directly echo out the HTML into the rendered result.

      Finally, using export default before declaring the BlogPostTemplate function is necessary in your code, because Gatsby will expect the default export of each page template file to be the React component responsible for producing the final rendered page.

      Now that you have added the template code to the file, save the changes and close it.

      In completing this step, you added a brand new page template file to your Gatsby project. With a filename that uses the File System Route API, it dynamically creates routes and pages based on GraphQL results and your Markdown source files. This was the final step to get Gatsby to generate pages from Markdown. In the next step, you will see these pages in action, as well as add new ones.

      With all the code now in place to turn your Markdown blog posts into web pages, you can now preview your work and add more Markdown content to your site.

      To preview your new blog posts and all the work you have done so far, run this command in your project directory:

      Once Gatsby is ready, it will prompt you to open your project in your web browser, which you can do by navigating to localhost:8000. However, this will first only show you the homepage of your Gatsby application instead of the blog post. To see your new Markdown blog post, navigate to localhost:8000/blog/learning-about-gatsby/. You will find your Markdown file rendered as HTML:

      Blog post titled

      From now on, you can continue to add new blog posts by creating Markdown files in the src/blog directory. Each time you create a new post, remember to follow the conventions you set up for your frontmatter. This means making sure to start the slug value with /blog/ followed by the custom path you want, as well as providing the post with a title and date.

      To test this out, copy your learning-about-gatsby.md Markdown file to be the base of a new post named continuing-learning.md:

      • cp src/blog/learning-about-gatsby.md src/blog/continuing-learning.md

      Next, open the new file and make the following highlighted changes to the content:

      markdown-tutorial/src/blog/continuing-learning.md

      --- 
      title: "Continuing to Learn"
      slug: "/blog/continuing-learning"
      date: "2021-08-01" 
      --- 
      
      ## Update
      
      I'm continuing to learn Gatsby to build some fast static sites!
      

      In this file, you kept the formatting the same but changed the title and slug of the frontmatter and the content of the blog post. Save the file then exit from it.

      Once your server has rebuilt your Gatsby site, navigate to http://localhost:8000/blog/continuing-learning in your browser. You will find the new post rendered at this URL:

      Blog post titled

      Note: If you wanted to add Markdown pages outside of blog, you could do so by modifying the slug to whatever path you would like. If you do this, make sure to use folders to keep your files organized.

      You now have new pages generated from Markdown in Gatsby and have previewed the results.

      Conclusion

      By following the steps in this tutorial, you added Markdown content to your Gatsby project, configured Gatsby to find the files and process them, and created template code to render each post as a new page. Although some of these steps could be omitted by using a Markdown-specific Gatsby Starter template, going through the process manually allows you to customize your Markdown-powered Gatsby site exactly how you want it.

      Although you can link to any new post or page using the Gatsby Link component, if your site has a large amount of Markdown files that are rapidly changing you might want to explore adding a dynamic listing as a next step, so visitors to your site can quickly find all your most recent posts and navigate to each one. To do so, you could use a Gatsby GraphQL tag to query the Markdown posts you want to list, and then iterate over them and display them as links. You can read more about this in this Adding a List of Markdown Blog Posts tutorial.

      If you would like to read more on Gatsby, check out the rest of the How To Create Static Web Sites with Gatsby.js series.



      Source link

      How To Convert a Gatsby Site to a Progressive Web App


      The author selected the Internet Archive to receive a donation as part of the Write for DOnations program.

      Introduction

      Gatsby is a popular framework for turning source material into static HTML files that are ready for instant deployment. Because of this, it is often called a Static Site Generator (SSG). As an SSG, it already has a positive impact on user experience (UX) by turning multiple content sources into an optimized website, but there is another layer available for UX improvement: Progressive Web App capabilities.

      A Progressive Web App, or PWA, is a type of website that goes beyond the usual limits of web capabilities, using newer technology to bridge the gap between your browser and your operating system. PWAs encompass a lot of different features, such as offline browsing, installation support, and newer web APIs. By combining these features, PWAs deliver your users an improved overall browsing experience, as well as the option to use your website like any other application, complete with its own icon and app window.

      Although there is a lot that goes into making an optimal PWA, Gatsby provides tools and support to streamline the process, such as a manifest file plugin (gatsby-plugin-manifest) and an offline plugin (gatsby-plugin-offline). This tutorial will walk you through using these plugins, as well as audit tools like Lighthouse, and by the end you will have learned how to take a Gatsby site and turn it into a fully-functional Progressive Web App.

      Prerequisites

      Before starting, here are a few things you will need:

      • A local installation of Node.js for running Gatsby and building your site. The installation procedure varies by operating system, but DigitalOcean has guides for Ubuntu 20.04 and macOS, and you can always find the latest release on the official NodeJS download page.
      • Some familiarity with JavaScript, for working in Gatsby. The JavaScript language is an expansive topic, but a good starting spot is our How to Code in JavaScript series.
      • An existing Gatsby project that does not yet have PWA support, but is otherwise functional. For satisfying this requirement and building a new Gatsby project from scratch, you can refer to Step 1 of the How to Set Up Your First Gatsby Website tutorial.
      • (Optional) An icon file for your website. In order to be installable, your PWA will need an icon with an original resolution of at least 512 x 512 pixels. If you do not have an icon in mind, this tutorial will instruct you to download a sample icon in Step 2.

      This tutorial was tested on Node v14.17.2, npm v6.14.13, Gatsby v3.9.0, gatsby-plugin-offline v4.8.0, and gatsby-plugin-manifest v3.8.0.

      Step 1 — Preparing Your Content (Optional)

      As a prerequisite, you already have created a functional Gatsby site that can be built and deployed. However, you might not have any content for your site yet. In this step, you will prepare a sample smart home user guide site to demonstrate what kind of content benefits from PWA features.

      As a smart home user guide is something likely to be visited multiple times by the same user, it makes a great example to showcase the main features of a PWA. The app-like qualities of a PWA, such as installation support and home screen icons, make it accessible on both mobile and desktop devices, and thanks to offline support, even when your home network fails, you or other residents can still access the guide.

      Building off the starter template, you can add a new page for the smart home guide under src/pages. Create a file named src/pages/internet-issues.js, and add the following sample page code:

      src/pages/internet-issues.js

      import * as React from "react"
      import { Link } from "gatsby"
      
      import Layout from "../components/layout"
      import Seo from "../components/seo"
      
      const IndexPage = () => (
        <Layout>
          <Seo title="Internet Issues" />
          <h1>Internet Issues - Troubleshooting</h1>
          <p>Having issues connecting to the internet? Here are some things to try in our house.</p>
          <ul>
            <li>Your Device
              <ul>
                <li>Is airplane mode on? Is WiFi enabled?</li>
                <li>Is the device within range of the router or physically connected to the network via ethernet?</li>
                <li>Are you connected to the correct network?</li>
              </ul>
            </li>
      
            <br />
      
            <li>The Router / Modem
              <ul>
                <li>Have you checked the ISPs status page to check for an outage? You can use your smartphone and mobile data to check this.</li>
                <li>Have you tried rebooting the router and/or modem?</li>
                <li>Have you checked to make sure that all physical connections to and from the router and modem are secure?</li>
              </ul>
            </li>
          </ul>
      
          <br/>
      
          <p>
            <Link to="/">Back to homepage</Link> <br />
          </p>
        </Layout>
      )
      
      export default IndexPage
      

      In this page code, you’ve provided troubleshooting instructions to your housemates or guests for when they are having trouble connecting to the internet. You have done so with a bulleted list, providing a link back to the homepage for easier navigation. Since this is a Gatsby project, you’ve created the entire page as a React component, which will nest your list inside a reusable Layout component and return the page as JSX so Gatsby can process it. For an optimized navigational experience, you’ve also used a Link component to link back to the homepage, instead of a regular HTML a tag.

      Make sure to save the file after updating it, and you can go ahead and close it since you won’t need to update it again in this tutorial.

      This page will be accessible at your_domain/internet-issues/, but you will also add a link to it from your homepage to make it easier to get to.

      Open up src/pages/index.js, and add a direct link to the new page within the React component IndexPage, as shown in the following highlighted code:

      src/pages/index.js

      import * as React from "react"
      import { Link } from "gatsby"
      import { StaticImage } from "gatsby-plugin-image"
      
      import Layout from "../components/layout"
      import Seo from "../components/seo"
      
      const IndexPage = () => (
        <Layout>
          ...
      
          <p>
            <Link to="/internet-issues/">Internet Issues Troubleshooting Page</Link> <br />
            <Link to="/page-2/">Go to page 2</Link> <br />
            ...
          </p>
        </Layout>
      )
      

      Save and close index.js with the added link.

      You’ve now built a brand new page for your smart home user guide and added a link to get to it from your homepage. In the next step, you will be adding a special file known as a manifest file, which instructs web browsers on the specifics of your PWA setup.

      Step 2 — Adding a Manifest File

      The next step is fulfilling one of the core requirements of PWAs by adding a manifest JSON file, manifest.json. The manifest file tells the web browser details about your site and how to display it to the user if it is installed on the users’s OS, specifying details such as what icon to use and how it should be launched. You will use gatsby-plugin-manifest to generate this file by initializing the plugin in your Gatsby config file.

      First, install the Gatsby plugin by running the following command in your Gatsby project directory:

      • npm install gatsby-plugin-manifest

      Next, you will provide some details to the plugin that tell it how you want the PWA to appear and act. You do this by editing the main Gatsby config file that lives in the root of your project directory, gatsby-config.js. Open this file and add the following code:

      gatsby-config.js

      module.exports = {
        plugins: [
          ...
      
          {
            resolve: `gatsby-plugin-manifest`,
            options: {
              name: `My Smart-Home Guide`,
              short_name: `SH Guide`,
              description: `Guide for residents of the ABC123 Smart Home`,
              start_url: `/`,
              background_color: `#0a68f0`,
              theme_color: `#0a68f0`,
              display: `standalone`,
              icon: `src/images/pwa-icon.png`,
              icons: [
                {
                  src: `/favicons/pwa-icon-192x192.png`,
                  sizes: `192x192`,
                  type: `image/png`
                },
                {
                  src: `/favicons/pwa-icon-512x512.png`,
                  sizes: `512x512`,
                  type: `image/png`
                }
              ]
            },
          }
      
          ...
        ]
      }
      

      Note: If you have started with the gatsby-starter-default template, you will already have some values for this plugin in your config file. In that case, overwrite the existing values with this code.

      There are a lot of values in this file, so here is a quick explanation:

      • name and short_name should correspond with the name of your site and how you want the site to appear to users when installed. short_name appears on the home screen of the user’s device, or other places where space is limited, and name appears everywhere else.
      • description should be text that describes the purpose of your application.
      • start_url is used to suggest to the browser which page should open when the user launches the PWA from their launcher. A value of /, as used here, tells the browser you would like the user to land on your homepage when opening the PWA.
      • background_color and theme_color are both directives to the browser on styling the PWA, and the values should correspond to CSS color strings. background_color is only used while waiting on the actual stylesheet to load, as a placeholder background color, whereas theme_color is potentially used in multiple places outside the PWA, such as surrounding the icon on an Android home screen.
      • display is a special value because it dictates how your entire site acts as a PWA, and, unlike other fields which support hundreds of different combinations, can be one of four possible values: fullscreen, standalone, minimal-ui, or browser. In your config, the value of standalone makes the PWA act like a standalone application outside the standard web browser. In practice, this means that it acts similar to a native application—it gets its own launcher icon, application window, and the URL address bar is hidden.
      • icon is not one of the standard manifest fields, but is a special field within the context of gatsby-plugin-manifest. By using this property, and providing a path to a file that meets the minimum requirements (at least 512x512 pixels, square), the Gatsby plugin will automatically transform the image into a site favicon, as well as inject into the manifest as the required icons manifest property. By specifying icons with an array of filenames, sizes, and image types, you are invoking the Hybrid Mode Configuration of the manifest plugin. This takes your single source icon file and transforms it into the filenames and sizes specified. This is not strictly necessary, but it avoids any possible issues with deployments in environments like Apache, which don’t work with the default /icons directory.

      Make sure to save the config file with your changes, but keep it open for the next step, where you will be adding another Gatsby plugin and configuring offline support.

      In the manifest values, the path used for icon was src/images/pwa-icon.png, but you still need to place an image file at that location before it will work. If you have a square image file that is at least 512 x 512 pixels, you can copy it to that path. Otherwise, you can use a pre-formatted image file selected for this tutorial. To use the tutorial icon file, either download the sample icon file for this tutorial and save it at src/images/pwa-icon.png, or if you prefer the command line, use cURL from your project root directory:

      • curl -o ./src/images/pwa-icon.png https://assets.digitalocean.com/articles/67869/1.png

      This will download the image to the correct part of your Gatsby application. This is the only image file you’ll need; Gatsby will automatically generate the 192x192 version.

      You have now configured your Gatsby project to serve a manifest JSON file with the correct values, which is a required part of enabling PWA capabilities. In the next step, you will be addressing another requirement of PWAs, offline support, by adding the feature via a service worker plugin, gatsby-plugin-offline.

      Step 3 — Adding Offline Support with Service Workers

      Another key component of PWAs is offline support, which you will implement with a piece of web technology known as a service worker. A service worker is essentially a bundle of JavaScript code that runs separately from all the code tied to the UI of the page you are on. This isolated code is also granted special privileges, such as the ability to alter the behavior of network requests, which is critical for implementing offline support. In this step, you will set up a robust service worker through the gatsby-plugin-offline plugin, configured through your Gatsby config file.

      Start by installing the gatsby-plugin-offline package and adding it to your dependencies. You can do so with:

      • npm install gatsby-plugin-offline

      Next, load the plugin through the Gatsby config, the same gatsby-config.js edited in the previous step:

      gatsby-config.js

      module.exports = {
        plugins: [
          ...
      
          {
            resolve: `gatsby-plugin-manifest`,
            options: {
              ...
            },
          },
          `gatsby-plugin-offline`,
      
          ...
        ]
      }
      

      Make sure to save the configuration file after adding the new plugin.

      Warning: Both the docs for gatsby-plugin-manifest and for gatsby-plugin-offline specify that gatsby-plugin-offline should always come after gatsby-plugin-manifest in the configuration array, as shown in this code snippet. This ensures that the manifest file itself can be cached.

      At this point, you’ve both added offline support and created a manifest file for your app. Next, this tutorial will explain the third necessary part of a PWA: having a secure context.

      Step 4 — Providing a Secure Context

      The last of the three basic minimum requirements for any PWA is that it run in a secure context. A secure context refers to a web environment in which certain baseline standards are met for authentication and security, and most often is referring to content served over HTTPS.

      A secure context can be achieved in many ways. Because of this, this tutorial will describe a few different strategies to get a secure context, then move forward with testing your Gatsby site locally.

      If you are deploying your Gatsby project through a static host, such as DigitalOcean’s App Platform, then it is likely that HTTPS is supported out of the box, with no setup required. For more information on deploying your app on App Platform, check out the How To Deploy a Gatsby Application to DigitalOcean App Platform tutorial.

      If you are deploying on a server that does not automatically provide HTTPS, but you have SSH access, you can use Let’s Encrypt to obtain and install a free TLS/SSL certificate. For example, if you are using Apache with Ubuntu 20.04, you can follow How To Secure Apache with Let’s Encrypt on Ubuntu 20.04 to use Certbot to handle this process. If you are deploying to a shared host, you will want to check their specific documentation pages for if, and how, SSL certificate installation might be available.

      For local testing, you don’t have to deal with obtaining or installing an SSL certificate. This is because most modern browsers treat localhost sites as a secure context, even without a TLS/SSL certificate installed or HTTPS protocol.

      You now have successfully added PWA support to your Gatsby project by meeting the three baseline criteria: HTTPS (or a localhost secure context), a manifest file, and a service worker. The next step is to test and verify that it shows up correctly, with PWA features enabled.

      Step 5 — Running Local Tests

      In this step, you will run some local tests to make sure your PWA functionality is working properly. This is an initial testing step before using the Lighthouse tool for a more comprehensive audit.

      To locally test your Gatsby site’s functionality as a PWA, build it and then serve the generated build directory:

      • npm run build
      • npm run serve

      Once it is ready, you will see the following:

      Output

      You can now view gatsby-starter-default in the browser. ⠀ http://localhost:9000/

      You can now visit that URL in your web browser, and if the browser supports PWAs, you will encounter some special additional UI elements. For example, on Chrome desktop, there will be a new button exposed in the address bar that, when clicked, asks if you would like to install the Gatsby site as an app, as shown in the following image:

      Screenshot showing a popup menu, originating from the Chrome desktop address bar, asking if you would like to

      If you want to test your site locally on a smartphone, this is possible with something like Chrome’s remote debugging tool (Android only), or a localhost tunneling service such as ngrok. On mobile, you will encounter the same option to install your site as an app, as shown in the following:

      Screenshot from an Android device, showing a

      This PWA prompt is different for each device, operating system, and browser. Additionally, certain features such as Add to Home Screen might only be available on certain devices. Certain browsers running on specific devices might not support PWAs entirely; check caniuse.com for more information on platform support.

      You have now verified that your Gatsby project can be built and served locally, with your browser successfully detecting that it offers PWA functionality. The next step will be a final check against what you have put together, and using the Lighthouse tool to check if there are areas for improvement.

      Step 6 — Running an Audit with Lighthouse

      At this point, you have a Gatsby site that meets all the core requirements for a PWA: it has HTTPS, a manifest, and a service worker for offline support. However, the concept of a PWA goes beyond any single requirement—it encompasses all of the facets working together, plus adhering to general guidelines. With this in mind, your last step is to use an audit tool to verify that you meet the baseline criteria, as well as gather information on how you can further optimize your Gatsby project to meet PWA best practices.

      There are a few different ways to audit your site as a PWA, but at the moment the gold standard is the Lighthouse Tool. If you have desktop Chrome installed, you can run an audit against your site directly in the web browser DevTools.

      First, navigate to your Gatsby site in Chrome, then open Chrome DevTools by right clicking anywhere on the webpage and selecting Inspect in the right click menu. Next, click the Lighthouse tab under DevTools. If you don’t see it, click the >> label next to the right-most tab to show tabs that are hidden due to size constraints.

      Now, to actually run the report, uncheck everything on the Lighthouse tab except for Progressive Web App, and then hit Generate Report to analyze your site:

      Screenshot showing the Lighthouse tab in desktop Chrome DevTools, with only the Progressive Web App report category checked

      You can also generate this report programmatically, via the Lighthouse Node.js CLI. This command will run the PWA-only audit and then open the report for viewing:

      • npx lighthouse http://localhost:9000 --only-categories=pwa --view

      However, using Lighthouse via CLI does not bypass the need to have Chrome installed; this just makes it easier to automate the process.

      The report generated by Lighthouse tells you a few things, broken down into categories. Some of the most important are:

      • Installable: This is the most important category, and addresses whether or not your site meets the three baseline criteria for being an installable PWA—HTTPS, manifest files, and a service worker.
      • PWA Optimized: These are things that you should be doing for an optimal PWA user experience, but aren’t strictly required for your PWA to be functional. Think of these as best-practice suggestions. Some examples of these are creating a custom splash screen to display during mobile app loading, setting a theme color for the address bar, and providing fallback content for when JavaScript is unavailable. If you’d like to look at the full list, check out the official Lighthouse documentation.

      By using the Lighthouse tool to audit your Gatsby PWA, you not only now have a functional PWA, but also have an idea of how it meets PWA requirements and best practices.

      Conclusion

      After following these steps, you now have a Gatsby site that can also function as a modern installable Progressive Web App, with strong offline support. You are now providing your users with the best of both worlds: They can browse your site as a normal web page, but they can also use it as they would a native application, with its own launcher icon, display window, and the reliable performance they expect from native applications.

      If you are looking for more ways to deliver the most optimized PWA experience possible, in addition to the Lighthouse PWA audit, Google also has a published a PWA checklist that will help. If you would like to read more on Gatsby, check out the rest of the How To Create Static Web Sites with Gatsby.js series.



      Source link