One place for hosting & domains

      Platform

      How To Deploy a Pre-Trained Question and Answer TensorFlow.js Model on App Platform


      The author selected Code 2040 to receive a donation as part of the Write for DOnations program.

      Introduction

      As the field of machine learning (ML) grows, so does the list of environments for using this technology. One of these environments is the web browser, and in recent years, there has been a surge in data-related frameworks targeting web-based machine learning models. An example of these frameworks is TensorFlow.js, TensorFlow’s JavaScript counterpart library for training, executing, and deploying machine learning models in the browser. In an attempt to make TensorFlow.js accessible to developers with limited or no ML experience, the library comes with several pre-trained models that work out of the box.

      A pre-trained machine learning model is ready-to-use machine learning you don’t have to train. TensorFlow.js includes 14 that suit a variety of use cases. For example, there is an image classifying model for identifying common objects and a body segmentation model for identifying body parts. The principal convenience of these models is that, as the name states, you don’t have to train them. Instead, you load them in your application. Besides their ease of use and pre-trained nature, these are curated models—they are accurate and fast, sometimes trained by the algorithms’ authors and optimized for the web browser.

      In this tutorial, you will create a web application that serves a Question and Answer (QnA) pre-trained model using TensorFlow.js. The model you will deploy is a Bidirectional Encoder Representations from Transformers (BERT) model that uses a passage and a question as the input, and tries to answer the question from the passage.

      You will deploy the app in DigitalOcean’s App Platform, a managed solution for building, deploying, and scaling applications within a few clicks from sources such as GitHub. The app you will create consists of a static page with two input fields, one for the passage and one for the question. In the end, you will have a written and deployed—from GitHub—a Question and Answer application using one of TensorFlow.js’ pre-trained models and DigitalOcean’s App Platform.

      Prerequisites

      To complete this tutorial, you will need:

      Step 1 — Creating the App’s Interface and Importing the Required Libraries

      In this step, you will write the app’s HTML code, which will define its interface and import the libraries for the app. The first of these libraries is TensorFlow.js, and instead of installing the package locally, you will load it from a content delivery network or CDN. A CDN is a network of servers spanning multiple locations that store content they provide to the internet. This content includes JavaScript files, such as the TensorFlow.js library, and loading it from a CDN saves you from packing them in your application. Similarly, you will import the library containing the Question and Answer model. In the next step, you will write the app’s JavaScript, which uses the model to answer a given question.

      In this tutorial, you will structure an app that will look like this:

      The app

      The app has five major elements:

      • A button that loads a pre-defined passage you can use to test the app.
      • An input text field for a passage (if you choose to write or copy your own).
      • An input text field for the question.
      • A button that triggers the prediction that answers the question.
      • An area to display the model’s output beneath the Answer! button (currently blank white space).

      Start by creating a new directory named tfjs-qna-do at your preferred location. In this directory, using a text editor of your choice, create a new HTML file named index.html and paste in the following code:

      index.html

      <!DOCTYPE html>
      <html lang="en-US">
      
      <head>
          <meta charset="utf-8" />
          <!-- Load TensorFlow.js -->
          <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
          <!-- Load the QnA model -->
          <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/qna"></script>
          <link href="./style.css" rel="stylesheet">
      </head>
      
      <body>
          <div class="main-centered-container">
              <h1>TensorFlow.js Pre-trained "QnA" BERT model</h1>
              <h3 class="header-border">Introduction</h3>
              <p>This application hosts TensorFlow.js' pre-trained Question and Answer model, and it attempts to answer the question
              using a given passage. To use it, write a passage (any!) in the input area below and a question. Then, click the
              "answer!" button to answer the question using the given passage as a reference. You could also click the test
              button to load a pre-defined input text.</p>
      
              <h4>Try the test passage!</h4>
              <div id='test-buttons'></div>
      
              <div>
                  <h4>Enter the model's input passage here</h4>
                  <textarea id='input-text' rows="20" cols="100" placeholder="Write the input text..."></textarea>
              </div>
      
              <div>
                  <h4>Enter the question to ask</h4>
                  <textarea id='question' rows="5" cols="100" placeholder="Write the input text..."></textarea>
              </div>
              <h4>Click to answer the question</h4>
              <div id="answer-button"></div>
              <h4>The model's answers</h4>
              <div id='answer'></div>
      
              <script src="./index.js"></script>
      
      
          </div>
      </body>
      
      </html>
      

      Here’s how that HTML breaks down:

      • The initial <html> tag has the <head> tag that’s used for defining metadata, styles, and loading scripts. Its first element is <meta>, and here you will set the page’s charset encoding to utf-8. After it, there are two <script> tags for loading both TensorFlow.js and the Question and Answer model from a CDN.
      • Following the two <script> tags, there’s a <link> tag that loads a CSS file (which you will create next).
      • Next, there’s the HTML’s <body>—the document’s content. Inside of it, there’s a <div> tag of class main-centered-container containing the page’s elements. The first is a <h1> header with the application title and a smaller <h3> header, followed by a brief introduction explaining how it works.
      • Under the introduction, there’s a <h4> header and a <div> where you will append buttons that populate the passage input text field with sample text.
      • Then, there are the app’s input fields: one for the passage (what you want the model to read) and one for the question (what you want the model to answer). If you wish to resize them, change the rows and cols attributes.
      • After the text fields, there’s a <div> with id button where you will later append a button that, upon clicking, reads the text fields’ text and uses them as an input to the model.
      • Last, there is a <div> with id answer that’s used to display the model’s output and a <script> tag to include the JavaScript code you will write in the following section.

      Next, you’ll add CSS to the project. In the same directory where you added the index.html file, create a new file named style.css and add the following code:

      style.css

      body {
        margin: 50px 0;
        padding: 0;
        font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial;
      }
      
      button {
        margin: 10px 10px;
        font-size: 100%;
      }
      
      p {
        line-height: 1.8;
      }
      
      .main-centered-container {
        padding: 60px;
        display: flex;
        flex-direction: column;
        margin: 0 auto;
        max-width: 960px;
      }
      
      .header-border {
        border: solid #d0d0d0;
        border-width: 0 0 1px;
        padding: 0 0 5px;
      }
      

      This CSS adds style to three HTML elements: body, buttons, and paragraphs (<p>). To body, it adds a margin, padding, and changes the default font. To button, it adds margin and increases the font size. To paragraph p, it modifies the line-height attribute. The CSS has a class main-centered-container that centers the content and another one, header-border, that adds a solid line to an element.

      In this step, you wrote the app’s HTML file. You imported the TensorFlow.js library and QnA model, and defined the elements you will use later to add the passage and the question you want to answer. In the following step, you will write the JavaScript code that reads the elements and triggers the predictions.

      Step 2 — Predicting with the Pre-Trained Model

      In this section, you will implement the web app’s JavaScript code that reads the app’s input fields, makes a prediction, and writes the predicted answers in HTML. You will do it in one function that’s triggered when you click the “answer” button. Once clicked, it will reference both input fields to get their values, use them as inputs to the model, and write its output in the <div> with id output you defined in Step 1. Then, you will run the app locally to test it before adding it to GitHub.

      In the project’s directory tfjs-qna-do/, create a new file named index.js, and declare the following variables:

      index.js

      let model;
      
      // The text field containing the input text
      let inputText;
      
      // The text field containing the question
      let questionText;
      
      // The div where we will write the model's answer
      let answersOutput;
      

      The first of the variables, model, is where you will store the QnA model; inputText and questionText are references to the input and question text fields; answersOutput is a reference to the output <div>.

      In addition to these variables, you will need a constant for storing the sample text you will use to test the app. We will use the Wikipedia article on DigitalOcean as a sample passage. Based on this passage, you could ask the model questions like “Where is DigitalOcean headquartered?” and hopefully, it will output “New York”.

      Copy this block into your index.js file:

      index.js

      // Sample passage from Wikipedia.
      const doText = `DigitalOcean, Inc. is an American cloud infrastructure provider[2] headquartered in New York City with data centers worldwide.[3] 
      DigitalOcean provides developers cloud services that help to deploy and scale applications that run simultaneously on multiple computers.
      DigitalOcean also runs Hacktoberfest which is a month-long celebration (October 1-31) of open source software run in partnership with GitHub and Twilio.
      `;
      

      Now, you will define the app’s functions, starting with createButton(). This function creates a button and appends it to an HTML element:

      index.js

      function createButton(innerText, id, listener, selector, disabled = false) {
        const btn = document.createElement('BUTTON');
        btn.innerText = innerText;
        btn.id = id;
        btn.disabled = disabled;
      
        btn.addEventListener('click', listener);
        document.querySelector(selector).appendChild(btn);
      }
      

      createButton() is a function that creates the app’s two buttons; since all the buttons work similarly, this function avoids repeating code. The function has five parameters:

      • innerText: the button’s text.
      • id: the button’s id.
      • listener: a callback function that’s executed when the user clicks the button.
      • selector: the <div> element where you will append the button.
      • disabled: a boolean value to disable or enable the button. This parameter’s default value is false.

      createButton() starts by creating an instance of a button and assigns it to the variable btn. Then, it sets the button’s innerText, id, and disabled attributes. The button uses a click event listener that executes a callback function whenever you click it. The function’s last line appends the button to the <div> element specified in the selector parameter.

      Next, you will create a new function named setupButtons() that calls createButton() two times to create the app’s buttons. Begin by creating the app’s Answer! button:

      index.js

      function setupButtons() {
        // Button to predict
        createButton('Answer!', 'answer-btn',
          () => {
            model.findAnswers(questionText.value, inputText.value).then((answers) => {
              // Write the answers to the output div as an unordered list.
              // It uses map create a new list of the answers while adding the list tags.
              // Then, we use join to concatenate the answers as an array with a line break
              // between answers.
              const answersList = answers.map((answer) => `<li>${answer.text} (confidence: ${answer.score})</li>`)
                .join('<br>');
      
              answersOutput.innerHTML = `<ul>${answersList}</ul>`;
            }).catch((e) => console.log(e));
          }, '#answer-button', true);
      }
      

      The first button the function creates is the one that triggers the predictions. Its first two arguments, innerText and id, are the button’s text and the identifier you want to assign to the button. The third argument is the listener’s callback (explained below). The fourth argument, selector, is the id of the <div> where you want to add the button (#answer-button), and the fifth argument, disabled, is set to true, which disables the button (to avoid predicting before the app loads the model).

      The listener’s callback is a function that’s executed once you click the Answer! button. Once clicked, the callback function first calls the pre-trained model’s function findAnswers(). (For more about the findAnswers() function, see the product documentation). findAnswers() uses as arguments the input passage (read from questionText) and the question (read from inputText). It returns the model’s output in an array that looks as follows:

      Model's output

      [ { "text": "New York City", "score": 19.08431625366211, "startIndex": 84, "endIndex": 97 }, { "text": "in New York City", "score": 8.737937569618225, "startIndex": 81, "endIndex": 97 }, { "text": "New York", "score": 7.998648166656494, "startIndex": 84, "endIndex": 92 }, { "text": "York City", "score": 7.5290607213974, "startIndex": 88, "endIndex": 97 }, { "text": "headquartered in New York City", "score": 6.888534069061279, "startIndex": 67, "endIndex": 97 } ]

      Each of the array’s elements is an object of four attributes:

      • text: the answer.
      • score: the model confidence level.
      • startIndex: the index of the passage’s first character that answers the question.
      • endIndex: the index of the answer’s last characters.

      Instead of displaying the output as the model returns it, the callback uses a map() function that creates a new array containing the model’s answers and scores while adding the list <li> HTML tags. Then, it joins the array’s elements with a <br> (line break) between answers and assigns the result to the answer <div> to display them as a list.

      Next, add a second call to createButton(). Add the highlighted portion to the setupButtons function beneath the first createButton:

      index.js

      function setupButtons() {
        // Button to predict
        createButton('Answer!', 'answer-btn',
          () => {
            model.findAnswers(questionText.value, inputText.value).then((answers) => {
              // Write the answers to the output div as an unordered list.
              // It uses map create a new list of the answers while adding the list tags.
              // Then, we use join to concatenate the answers as an array with a line break
              // between answers.
              const answersList = answers.map((answer) => `<li>${answer.text} (confidence: ${answer.score})</li>`)
                .join('<br>');
      
              answersOutput.innerHTML = `<ul>${answersList}</ul>`;
            }).catch((e) => console.log(e));
          }, '#answer-button', true);
      
        createButton('DigitalOcean', 'test-case-do-btn',
          () => {
           document.getElementById('input-text').value = doText;
          }, '#test-buttons', false);
      }
      

      This new call appends to the test-buttons <div> the button that loads the DigitalOcean sample text you defined earlier in the variable doText. The function’s first argument is the label for the button (DigitalOcean), the second argument is the id, the third is the listener’s callback that writes in the passage input text area the value of the doText variable, the fourth is the selector, and the last is a false value (to avoid disabling the button).

      Next, you will create a function, named init(), that calls the other functions:

      index.js

      async function init() {
        setupButtons();
        answersOutput = document.getElementById('answer');
        inputText = document.getElementById('input-text');
        questionText = document.getElementById('question');
      
        model = await qna.load();
        document.getElementById('answer-btn').disabled = false;
      }
      

      init() starts by calling setupButtons(). Then, it assigns some of the app’s HTML elements to the variables you defined at the top of the script. Next, it loads the QnA model and changes to false the disabled attribute of the answer (answer-btn) button.

      Last, call init():

      index.js

      init();
      

      You have finished the app. To test it, open your web browser and write the project’s directory absolute path with /index.html appended to it in the address bar. You could also open your file manager application (such as Finder on Mac) and click on “index.html” to open the web application. In this case, the address would look like your_filepath/tfjs-qna-do/index.html.

      To find the absolute path, go to the terminal and execute the following command from the project’s directory:

      Its output will look like this:

      Output

      /Users/your_filepath/tfjs-qna-do

      After launching the app, you will need to wait a few seconds while it downloads and loads the model. You will know it’s ready when the app enables the Answer! button. Then, you could either use the test passage button (DigitalOcean) or write your own passage, and then input a question.

      To test the app, click the “DigitalOcean” button. In the passage input area, you will see the sample text about DigitalOcean. In the question input area, write “Where is DigitalOcean headquartered?” From the passage, we can see the answer is “New York”. But will the model say the same? Click Answer! to find out.

      The output should look similar to this:

      Model's answers to "where is DO headquartered?"

      Correct! Although there are slight differences, four of the five answers say that DigitalOcean’s headquarters are in New York City.

      In this step, you completed and tested the web application. The JavaScript code you wrote reads the input passage and question and uses it as an input to the QnA model to answer the question. Next, you will add the code to GitHub.

      Step 3 — Pushing the App to GitHub

      In this section, you will add the web application to a GitHub repository. Later, you will connect the repository to DigitalOcean’s App Platform and deploy the app.

      Start by logging in to GitHub. From its main page, click the green button under your name or the plus sign on the screen’s upper right corner to create a new repository.

      Click to create a repository

      Click to create a repository

      Either choice takes you to the Create a new repository screen. In the Repository name field (after your username), name the repository tfjs-qna-do. Select its privacy setting, and click Create repository to create it.

      Create the repository

      Open a terminal and go to the project’s directory tfjs-qna-do/. There, execute the following command to create a new local Git repository:

      Next, stage the files you want to track with Git, which in this case, is everything in the directory:

      And commit them:

      • git commit -m "initial version of the app"

      Rename the repository’s principal branch to main:

      Then link your local repository with the remote one on GitHub:

      • git remote add origin git@github.com:your-github-username/tfjs-qna-do.git

      Last, push the local codebase to the remote repository:

      It will ask you to enter your GitHub credentials if this is the first time you are pushing code to it.

      Once you have pushed the code, return to the GitHub repository and refresh the page. You should see your code.

      The code in the repo

      In this step, you have pushed your web app’s code to a remote GitHub repository. Next, you will link this repository to your DigitalOcean account and deploy the app.

      Step 4 — Deploying the Web Application in DigitalOcean App Platform

      In this last section, you will deploy the Question and Answer app to DigitalOcean’s App Platform) from the GitHub repository created in Step 3.

      Start by logging into your DigitalOcean account. Once logged in, click the Create green button in the upper right corner and then on Apps; this takes you to the Create new app screen:

      Create a new DO App

      Here, choose GitHub as the source where the project resides.

      Choose the source

      If you haven’t linked your DigitalOcean account with GitHub, it will ask you to authorize DigitalOcean to access your GitHub. After doing so, select the repository you want to link: tfjs-qna-do.

      Link the repo

      Back on the App Platform window, select the app’s repository (tfjs-qna-do), the branch from where it will deploy the app (main), and check the Autodeploy code changes box; this option ensures that the application gets re-deployed every time you push code to the main branch.

      Choose the source (cont.)

      In the next screen, Configure your app, use the default configuration values. As an additional step, if you wish to change the web app’s HTTP requests route, from, for example, the default https://example.ondigitalocean.app/tfjs-qna-do, to https://example.ondigitalocean.app/my-custom-route, click on Edit under HTTP Routes and write my-custom-route in the ROUTES input.

      Configure the app

      Next, you will name the site. Again, you could leave the default tfjs-qna-do name or replace it with another one.

      Name the site

      Clicking Next will take you to the last screen, Finalize and launch, where you will select the app’s pricing tier. Since the app is a static webpage, App Platform will automatically select the free Starter plan. Last, click the Launch Starter App button to build and deploy the application.

      Finalize and launch

      While the app is deployed, you will see a screen similar to this:

      Deploying app

      And once deployed, you will see:

      Deployed

      To access the app, click on the app’s URL to access your app, which is now deployed in the cloud.

      Deployed app

      Conclusion

      As the field of machine learning expands, so do its use cases alongside the environments and platforms it reaches – including the web browser.

      In this tutorial, you have built and deployed a web application that uses a TensorFlow.js pre-trained model. Your Question and Answer web app takes as input a passage along with a question and uses a pre-trained BERT model to answer the question according to the passage. After developing the app, you linked your GitHub account with DigitalOcean and deployed the application in App Platform without needing additional code.

      As future work, you could trigger the app’s automatic deployment by adding a change to it and pushing it to GitHub. You could also add a new test passage and format the answer’s output as a table to improve its readability.

      For more information about TensorFlow.js, refer to its official documentation.



      Source link

      DigitalOcean App Platform: New Features and Enhancements


      How to Join

      This Tech Talk is free and open to everyone. Register below to get a link to join the live stream or receive the video recording after it airs.

      Date Time RSVP
      September 1, 2021 11 a.m.–12 p.m. ET / 3–4 p.m. GMT

      About the Talk

      App Platform is DigitalOcean’s Platform as a Service (PaaS) that lets you build, deploy, and scale applications quickly. Let’s take a look at all its cool new features, and how you can use them.

      What You’ll Learn

      This Talk Is Designed For

      Developers that want to simplify your deployments

      Resources

      App platform docs

      To join the live Tech Talk, register here.



      Source link

      How To Build a Rate Limiter With Node.js on App Platform


      The author selected the COVID-19 Relief Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Rate limiting manages your network’s traffic and limits the number of times someone repeats an operation in a given duration, such as using an API. A service without a layer of security against rate limit abuse is prone to overload and hampers your application’s proper operation for legitimate customers.

      In this tutorial, you will build a Node.js server that will check the IP address of the request and also calculate the rate of these requests by comparing the timestamp of requests per user. If an IP address crosses the limit you have set for the application, you will call Cloudflare’s API and add the IP address to a list. You will then configure a Cloudflare Firewall Rule that will ban all requests with IP addresses in the list.

      By the end of this tutorial, you will have built a Node.js project deployed on DigitalOcean’s App Platform that protects a Cloudflare routed domain with rate limiting.

      Prerequisites

      Before you begin this guide, you will need:

      Step 1 — Setting Up the Node.js Project and Deploying to DigitalOcean’s App Platform

      In this step, you will expand on your basic Express server, push your code to a GitHub repository, and deploy your application to App Platform.

      Open the project directory of the basic Express server with your code editor. Create a new file by the name .gitignore in the root directory of the project. Add the following lines to the newly created .gitignore file:

      .gitignore

      node_modules/
      .env
      

      The first line in your .gitignore file is a directive to git not to track the node_modules directory. This will enable you to keep your repository size small. The node_modules can be generated when required by running the command npm install. The second line prevents the environment variable file from being tracked. You will create the .env file in further steps.

      Navigate to your server.js in your code editor and modify the following lines of code:

      server.js

      ...
      app.listen(process.env.PORT || 3000, () => {
          console.log(`Example app is listening on port ${process.env.PORT || 3000}`);
      });
      

      The change to conditionally use PORT as an environment variable enables the application to dynamically have the server running on the assigned PORT or use 3000 as the fallback one.

      Note: The string in console.log() is wrapped within backticks(`) and not within quotes. This enables you to use template literals, which provides the capability to have expressions within strings.

      Visit your terminal window and run your application:

      Your browser window will display Successful response. In your terminal, you will see the following output:

      Output

      Example app is listening on port 3000

      With your Express server running successfully, you’ll now deploy to App Platform.

      First, initialize git in the root directory of the project and push the code to your GitHub account. Navigate to the App Platform dashboard in the browser and click on the Create App button. Choose the GitHub option and authorize with GitHub, if necessary. Select your project’s repository from the dropdown list of projects you want to deploy to App Platform. Review the configuration, then give a name to the application. For the purpose of this tutorial, select the Basic plan as you’ll work in the application’s development phase. Once ready, click Launch App.

      Next, navigate to the Settings tab and click on the section Domains. Add your domain routed via Cloudflare into the field Domain or Subdomain Name. Select the bullet You manage your domain to copy the CNAME record that you’ll use to add to your domain’s Cloudflare DNS account.

      With your application deployed to App Platform, head over to your domain’s dashboard on Cloudflare in a new tab as you will return to App Platform’s dashboard later. Navigate to the DNS tab. Click on the Add Record button and select CNAME as your Type, @ as the root, and paste in the CNAME you copied from the App Platform. Click on the Save button, then navigate to the Domains section under the Settings tab in your App Platform’s Dashboard and click on the Add Domain button.

      Click the Deployments tab to see the details of the deployment. Once deployment finishes, you can open your_domain to view it on the browser. Your browser window will display: Successful response. Navigate to the Runtime Logs tab on the App Platform dashboard, and you will get the following output:

      Output

      Example app is listening on port 8080

      Note: The port number 8080 is the default assigned port by the App Platform. You can override this by changing the configuration while reviewing the app before deployment.

      With your application now deployed to App Platform, let’s look at how to outline a cache to calculate requests to the rate limiter.

      Step 2 — Caching User’s IP Address and Calculating Requests Per Second

      In this step, you will store a user’s IP address in a cache with an array of timestamps to monitor the requests per second of each user’s IP address. A cache is temporary storage for data frequently used by an application. The data in a cache is usually kept in quick access hardware like RAM (Random-Access Memory). The fundamental goal of a cache is to improve data retrieval performance by decreasing the need to visit the slower storage layer underneath it. You will use three npm packages: node-cache, is-ip, and request-ip to aid in the process.

      The request-ip package captures the user’s IP address used to request the server. The node-cache package creates an in-memory cache which you will use to keep track of user’s requests. You’ll use the is-ip package used to check if an IP Address is IPv6 Address. Install the node-cache, is-ip, and request-ip package via npm on your terminal.

      • npm i node-cache is-ip request-ip

      Open the server.js file in your code editor and add following lines of code below const express = require('express');:

      server.js

      ...
      const requestIP = require('request-ip');
      const nodeCache = require('node-cache');
      const isIp = require('is-ip');
      ...
      

      The first line here grabs the requestIP module from request-ip package you installed. This module captures the user’s IP address used to request the server. The second line grabs the nodeCache module from the node-cache package. nodeCache creates an in-memory cache, which you will use to keep track of user’s requests per second. The third line takes the isIp module from the is-ip package. This checks if an IP address is IPv6 which you will format as per Cloudflare’s specification to use CIDR notation.

      Define a set of constant variables in your server.js file. You will use these constants throughout your application.

      server.js

      ...
      const TIME_FRAME_IN_S = 10;
      const TIME_FRAME_IN_MS = TIME_FRAME_IN_S * 1000;
      const MS_TO_S = 1 / 1000;
      const RPS_LIMIT = 2;
      ...
      

      TIME_FRAME_IN_S is a constant variable that will determine the period over which your application will average the user’s timestamps. Increasing the period will increase the cache size, hence consume more memory. The TIME_FRAME_IN_MS constant variable will also determine the period of time your application will average user’s timestamps, but in milliseconds. MS_TO_S is the conversion factor you will use to convert time in milliseconds to seconds. The RPS_LIMIT variable is the threshold limit of the application that will trigger the rate limiter, and change the value as per your application’s requirements. The value 2 in the RPS_LIMIT variable is a moderate value that will trigger during the development phase.

      With Express, you can write and use middleware functions, which have access to all HTTP requests coming to your server. To define a middleware function, you will call app.use() and pass it a function. Create a function named ipMiddleware as middleware.

      server.js

      ...
      const ipMiddleware = async function (req, res, next) {
          let clientIP = requestIP.getClientIp(req);
          if (isIp.v6(clientIP)) {
              clientIP = clientIP.split(':').splice(0, 4).join(':') + '::/64';
          }
          next();
      };
      app.use(ipMiddleware);
      
      ...
      

      The getClientIp() function provided by requestIP takes the request object, req from the middleware, as parameter. The .v6() function comes from the is-ip module and returns true if the argument passed to it is an IPv6 address. Cloudflare’s Lists requires the IPv6 address in /64 CIDR notation. You need to format the IPv6 address to follow the format: aaaa:bbbb:cccc:dddd::/64. The .split(':') method creates an array from the string containing the IP address splitting them by the character :. The .splice(0,4) method returns the first four elements of the array. The join(':') function returns a string from the array combined with the character :.

      The next() call directs the middleware to go to the next middleware function if there is one. In your example, it will take the request to the GET route /. This is important to include at the end of your function. Otherwise, the request will not move forward from the middleware.

      Initialize an instance of node-cache by adding the following variable below the constants:

      server.js

      ...
      const IPCache = new nodeCache({ stdTTL: TIME_FRAME_IN_S, deleteOnExpire: false, checkperiod: TIME_FRAME_IN_S });
      ...
      

      With the constant variable IPCache, you are overriding the default parameters native to nodeCache with the custom properties:

      • stdTTL: The interval in seconds after which a key-value pair of cache elements will be evicted from the cache. TTL stands for Time To Live, and is a measure of time after which cache expires.
      • deleteOnExpire: Set to false as you will write a custom callback function to handle the expired event.
      • checkperiod: The interval in seconds after which an automatic check for expired elements is triggered. The default value is 600, and as your application’s element expiry is set to a lesser value, the check for expiry will also happen sooner.

      For more information on the default parameters of node-cache, you will find the node-cache npm package’s docs page useful. The following diagram will help you to visualise how a cache stores data:

      Schematic Representation of Data Stored in Cache

      You will now create a new key-value pair for the new IP address and append to an existing key-value pair if an IP address exists in the cache. The value is an array of timestamps corresponding to each request made to your application. In your server.js file, create the updateCache() function below the IPCache constant variable to add the timestamp of the request to cache:

      server.js

      ...
      const updateCache = (ip) => {
          let IPArray = IPCache.get(ip) || [];
          IPArray.push(new Date());
          IPCache.set(ip, IPArray, (IPCache.getTtl(ip) - Date.now()) * MS_TO_S || TIME_FRAME_IN_S);
      };
      ...
      

      The first line in the function gets the array of timestamps for the given IP address, or if null, initializes with an empty array. In the following line, you are pushing the present timestamp caught by the new Date() function into the array. The .set() function provided by node-cache takes three arguments: key, value and the TTL. This TTL will override the standard TTL set by replacing the value of stdTTL from the IPCache variable. If the IP address already exists in the cache, you will use the existing TTL; else, you will set TTL as TIME_FRAME_IN_S.

      The TTL for the current key-value pair is calculated by subtracting the present timestamp from the expiry timestamp. The difference is then converted to seconds and passed as the third argument to the .set() function. The .getTtl() function takes a key and IP address as an argument and returns the TTL of the key-value pair as a timestamp. If the IP address does not exist in the cache, it will return undefined and use the fallback value of TIME_FRAME_IN_S.

      Note: You require the conversion timestamps from milliseconds to seconds as JavaScript stores them in milliseconds while the node-cache module uses seconds.

      In the ipMiddleware middleware, add the following lines after the if code block if (isIp.v6(clientIP)) to calculate the requests per second of the IP address calling your application:

      server.js

      ...
          updateCache(clientIP);
          const IPArray = IPCache.get(clientIP);
          if (IPArray.length > 1) {
              const rps = IPArray.length / ((IPArray[IPArray.length - 1] - IPArray[0]) * MS_TO_S);
              if (rps > RPS_LIMIT) {
                  console.log('You are hitting limit', clientIP);
              }
          }
      ...
      

      The first line adds the timestamp of the request made by the IP address to the cache by calling the updateCache() function you declared. The second line collects the array of timestamps for the IP address. If the number of elements in the array of timestamps is greater than one (calculating requests per second needs a minimum of two timestamps), and the requests per second are more than the threshold value you defined in the constants, you will console.log the IP address. The rps variable calculates the requests per second by dividing the number of requests with a time interval difference, and converts the units to seconds.

      Since you had defaulted the property deleteOnExpire to the value false in the IPCache variable, you will now need to handle the expired event manually. node-cache provides a callback function that triggers on expired event. Add the following lines of code below the IPCache constant variable:

      server.js

      ...
      IPCache.on('expired', (key, value) => {
          if (new Date() - value[value.length - 1] > TIME_FRAME_IN_MS) {
              IPCache.del(key);
          }
      });
      ...
      

      .on() is a callback function that accepts key and value of the expired element as the arguments. In your cache, value is an array of timestamps of requests. The highlighted line checks if the last element in the array is at least TIME_FRAME_IN_S in the past than the present time. As you are adding elements to your array of timestamps, if the last element in value is at least TIME_FRAME_IN_S in the past than the present time, the .del() function takes key as an argument and deletes the expired element from the cache.

      For the instances when some elements of the array are at least TIME_FRAME_IN_S in the past than the present time, you need to handle it by removing the expired items from the cache. Add the following code in the callback function after the if code block if (new Date() - value[value.length - 1] > TIME_FRAME_IN_MS).

      server.js

      ...
          else {
              const updatedValue = value.filter(function (element) {
                  return new Date() - element < TIME_FRAME_IN_MS;
              });
              IPCache.set(key, updatedValue, TIME_FRAME_IN_S - (new Date() - updatedValue[0]) * MS_TO_S);
          }
      ...
      

      The filter() array method native to JavaScript provides a callback function to filter the elements in your array of timestamps. In your case, the highlighted line checks for elements that are least TIME_FRAME_IN_S in the past than the present time. The filtered elements are then added to the updatedValue variable. This will update your cache with the filtered elements in the updatedValue variable and a new TTL. The TTL that matches the first element in the updatedValue variable will trigger the .on('expired') callback function when the cache removes the following element. The difference of TIME_FRAME_IN_S and the time expired since the first request’s timestamp in updatedValue calculates the new and updated TTL.

      With your middleware functions now defined, visit your terminal window and run your application:

      Then, visit localhost:3000 in your web browser. Your browser window will display: Successful response. Refresh the page repeatedly to hit the RPS_LIMIT. Your terminal window will display:

      Output

      Example app is listening on port 3000 You are hitting limit ::1

      Note: The IP address for localhost is shown as ::1. Your application will capture the public IP of a user when deployed outside localhost.

      Your application is now able to able to track the user’s requests and store the timestamps in the cache. In the next step, you will integrate Cloudflare’s API to set up the Firewall.

      Step 3 — Setting Up the Cloudflare Firewall

      In this step, you will set up Cloudflare’s Firewall to block IP Addresses when hitting the rate limit, create environment variables, and make calls to the Cloudflare API.

      Visit the Cloudflare dashboard in your browser, log in, and navigate to your account’s homepage. Open Lists under Configurations tab. Create a new List with your_list as the name.

      Note: The Lists section is available on your Cloudflare account’s dashboard page and not your Cloudflare domain’s dashboard page.

      Navigate to the Home tab and open your_domain’s dashboard. Open the Firewall tab and click on Create a Firewall rule under the Firewall Rules section. Give your_rule_name to the Firewall to identify it. In the Field, select IP Source Address from the dropdown, is in list for the Operator, and your_list for the Value. Under the dropdown for Choose an action, select Block and click Deploy.

      Create a .env file in the project’s root directory with the following lines to call Cloudflare API from your application:

      .env

      ACCOUNT_MAIL=your_cloudflare_login_mail
      API_KEY=your_api_key
      ACCOUNT_ID=your_account_id
      LIST_ID=your_list_id
      

      To get a value for API_KEY, navigate to the API Tokens tab on the My Profile section of your Cloudflare dashboard. Click View in the Global API Key section and enter your Cloudflare password to view it. Visit the Lists section under the Configurations tab on the account’s homepage. Click on Edit beside your_list list you created. Get the ACCOUNT_ID and LIST_ID from the URL of your_list in the browser. The URL is of the format below:
      https://dash.cloudflare.com/your_account_id/configurations/lists/your_list_id

      Warning: Make sure the content of .env is kept confidential and not made public. Make sure you have the .env file listed in the .gitignore file you created in Step 1.

      Install the axios and dotenv package via npm on your terminal.

      Open the server.js file in your code editor and the add following lines of code below the nodeCache constant variable:

      server.js

      ...
      const axios = require('axios');
      require('dotenv').config();
      ...
      

      The first line here grabs the axios module from axios package you installed. You will use this module to make network calls to Cloudflare’s API. The second line requires and configures the dotenv module to enable the process.env global variable that will define the values you placed in your .env file to server.js.

      Add the following to the if (rps > RPS_LIMIT) condition within ipMiddleware above console.log('You are hitting limit', clientIP) to call Cloudflare API.

      server.js

      ...
          const url = `https://api.cloudflare.com/client/v4/accounts/${process.env.ACCOUNT_ID}/rules/lists/${process.env.LIST_ID}/items`;
          const body = [{ ip: clientIP, comment: 'your_comment' }];
          const headers = {
              'X-Auth-Email': process.env.ACCOUNT_MAIL,
              'X-Auth-Key': process.env.API_KEY,
              'Content-Type': 'application/json',
          };
          try {
              await axios.post(url, body, { headers });
          } catch (error) {
              console.log(error);
          }
      ...
      

      You are now calling the Cloudflare API through the URL to add an item, in this case an IP address, to your_list. The Cloudflare API takes your ACCOUNT_MAIL and API_KEY in the header of the request with the key as X-Auth-Email and X-Auth-Key. The body of the request takes an array of objects with ip as the IP address to add to the list, and a comment with the value your_comment to identify the entry. You can modify value of comment with your own custom comment. The POST request made via axios.post() is wrapped in a try-catch block to handle errors if any, that may occur. The axios.post function takes the url, body and an object with headers to make the request.

      Change the clientIP variable within the ipMiddleware function when testing out the API requests with a test IP address like 198.51.100.0/24 as Cloudflare does not accept the localhost’s IP address in its Lists.

      server.js

      ...
      const clientIP = '198.51.100.0/24';
      ...
      

      Visit your terminal window and run your application:

      Then, visit localhost:3000 in your web browser. Your browser window will display: Successful response. Refresh the page repeatedly to hit the RPS_LIMIT. Your terminal window will display:

      Output

      Example app is listening on port 3000 You are hitting limit ::1

      When you have hit the limit, open the Cloudflare dashboard and navigate to the your_list’s page. You will see the IP address you put in the code added to your Cloudflare’s List named your_list. The Firewall page will display after pushing your changes to GitHub.

      Warning: Make sure to change the value in your clientIP constant variable to requestIP.getClientIp(req) before deploying or pushing the code to GitHub.

      Deploy your application by committing the changes and pushing the code to GitHub. As you have set up auto-deploy, the code from GitHub will automatically deploy to your DigitalOcean’s App Platform. As your .env file is not added to GitHub, you will need to add it to App Platform via the Settings tab at App-Level Environment Variables section. Add the key-value pair from your project’s .env file so your application can access its contents on the App Platform. After you save the environment variables, open your_domain in your browser after deployment finishes and refresh the page repeatedly to hit the RPS_LIMIT. Once you hit the limit, the browser will show Cloudflare’s Firewall page.

      Cloudflare's Error 1020 Page

      Navigate to the Runtime Logs tab on the App Platform dashboard, and you will view the following output:

      Output

      ... You are hitting limit your_public_ip

      You can open your_domain from a different device or via VPN to see that the Firewall bans only the IP address in your_list. You can delete the IP address from your_list through your Cloudflare dashboard.

      Note: Occasionally, it takes few seconds for the Firewall to trigger due to the cached response from the browser.

      You have set up Cloudflare’s Firewall to block IP Addresses when users are hitting the rate limit by making calls to the Cloudflare API.

      Conclusion

      In this article, you built a Node.js project deployed on DigitalOcean’s App Platform connected to your domain routed via Cloudflare. You protected your domain against rate limit misuse by configuring a Firewall Rule on Cloudflare. From here, you can modify the Firewall Rule to show JS Challenge or CAPTCHA instead of banning the user. The Cloudflare documentation details the process.



      Source link