One place for hosting & domains

      Create

      How To Create an Intelligent Chatbot in Python Using the spaCy NLP Library


      The author selected the COVID-19 Relief Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Interacting with software can be a daunting task in cases where there are a lot of features. In some cases, performing similar actions requires repeating steps, like navigating menus or filling forms each time an action is performed. Chatbots are virtual assistants that help users of a software system access information or perform actions without having to go through long processes. Many of these assistants are conversational, and that provides a more natural way to interact with the system.

      To create a conversational chatbot, you could use platforms like Dialogflow that help you design chatbots at a high level. Or, you can build one yourself using a library like spaCy, which is a fast and robust Python-based natural language processing (NLP) library. spaCy provides helpful features like determining the parts of speech that words belong to in a statement, finding how similar two statements are in meaning, and so on.

      In this tutorial, you will create a chatbot that not only helps users simplify their interactions with a software system, but is also intelligent enough to communicate with the user in natural language (American English in this tutorial). The chatbot will use the OpenWeather API to tell the user what the current weather is in any city of the world, but you can implement your chatbot to handle a use case with another API.

      Prerequisites

      Before you begin, you will need the following:

      This tutorial assumes you are already familiar with Python—if you would like to improve your knowledge of Python, check out our How To Code in Python 3 series. This tutorial does not require foreknowledge of natural language processing.

      Step 1 — Setting Up Your Environment

      In this step, you will install the spaCy library that will help your chatbot understand the user’s sentences.

      Having set up Python following the Prerequisites, you’ll have a virtual environment. Let’s activate that environment.

      Make sure you are in the directory where you set up your environment and then run the following command:

      • source my_env/bin/activate

      Now install spaCy:

      Finally, you will download a language model. spaCy’s language models are pre-trained NLP models that you can use to process statements to extract meaning. You’ll be working with the English language model, so you’ll download that.

      Run the following command:

      • python -m spacy download en_core_web_md

      If you run into an error like the following:

      Output

      ERROR: Failed building wheel for en-core-web-md

      You need to install wheel:

      Then download the English-language model again.

      To confirm that you have spaCy installed properly, open the Python interpreter:

      Next, import spaCy and load the English-language model:

      >>> import spacy
      >>> nlp = spacy.load("en_core_web_md")
      

      If those two statements execute without any errors, then you have spaCy installed.

      Now close the Python interpreter:

      >>> exit()
      

      You now have everything needed to begin working on the chatbot. In the next section, you’ll create a script to query the OpenWeather API for the current weather in a city.

      Step 2 — Creating the City Weather Program

      In this section, you will create a script that accepts a city name from the user, queries the OpenWeather API for the current weather in that city, and displays the response.

      First, create and open a Python file called weather_bot.py with your preferred editor:

      Next, you’ll create a function to get the current weather in a city from the OpenWeather API. This function will take the city name as a parameter and return the weather description of the city.

      Add the following code into your weather_bot.py file:

      weather_bot.py

      import requests
      
      api_key = "your_api_key"
      
      def get_weather(city_name):
          api_url = "http://api.openweathermap.org/data/2.5/weather?q={}&appid={}".format(city_name, api_key)
      
          response = requests.get(api_url)
          response_dict = response.json()
      
          weather = response_dict["weather"][0]["description"]
      
          if response.status_code == 200:
              return weather
          else:
              print('[!] HTTP {0} calling [{1}]'.format(response.status_code, api_url))
              return None
      

      First, you import the requests library, so you are able to work with and make HTTP requests. Make sure to replace your_api_key with your own API key. The next line begins the definition of the function get_weather() to retrieve the weather of the specified city.

      In this function, you construct the URL for the OpenWeather API. This URL returns the weather information (temperature, weather description, humidity, and so on) of the city and provides the result in JSON format. After that, you make a GET request to the API endpoint, store the result in a response variable, and then convert the response to a Python dictionary for easier access.

      On the next line, you extract just the weather description into a weather variable and then ensure that the status code of the API response is 200 (meaning there were no issues with the request). Finally, you return the weather description.

      If there is an issue with the request, the status code is printed out to the console, and you return None.

      To test the script, call the get_weather() function with a city of your choice (for example, London) and print the result. Add the highlighted code following your function:

      ~/weather_bot.py

      import requests
      
      def get_weather(city_name):
      
        ...
      
        return weather
      
      weather = get_weather("London")
      print(weather)
      

      Save and run the script:

      You will receive a result like the following:

      Output

      scattered clouds

      Having completed that successfully, you can now delete the last two lines from the script.

      Open it with:

      Then delete the two highlighted lines at the end of the file:

      ~/weather_bot.py

      import requests
      
      def get_weather(city_name):
      
        ...
      
        return weather
      
      weather = get_weather("London")
      print(weather)
      

      Save and close the file.

      You now have a function that returns the weather description for a particular city.

      In the next step, you’ll create a chatbot capable of figuring out whether the user wants to get the current weather in a city, and if so, the chatbot will use the get_weather() function to respond appropriately.

      Step 3 — Creating the Chatbot

      In the previous two steps, you installed spaCy and created a function for getting the weather in a specific city. Now, you will create a chatbot to interact with a user in natural language using the weather_bot.py script.

      You’ll write a chatbot() function that compares the user’s statement with a statement that represents checking the weather in a city. To make this comparison, you will use the spaCy similarity() method. This method computes the semantic similarity of two statements, that is, how similar they are in meaning. This will help you determine if the user is trying to check the weather or not.

      To begin, open the script:

      Then, import spaCy and load the English language model:

      ~/weather_bot.py

      import spacy
      import requests
      
      nlp = spacy.load("en_core_web_md")
      
      . . .
      
      

      After the get_weather() function in your file, create a chatbot() function representing the chatbot that will accept a user’s statement and return a response.

      Following your definition, add the highlighted code to create tokens for the two statements you’ll be comparing. Tokens are the different meaningful segments of a statement, like words and punctuation. This is necessary to allow spaCy to compute the semantic similarity:

      ~/weather_bot.py

      import spacy
      
      . . .
      
      def chatbot(statement):
        weather = nlp("Current weather in a city")
        statement = nlp(statement)
      

      Here the weather and statement variables contain spaCy tokens as a result of passing each corresponding string to the nlp() function.

      Save and close your file.

      Next you’ll be introducing the spaCy similarity() method to your chatbot() function. The similarity() method computes the semantic similarity of two statements as a value between 0 and 1, where a higher number means a greater similarity. You need to specify a minimum value that the similarity must have in order to be confident the user wants to check the weather.

      For example, if you check the similarity of statements 2 and 3 with statement 1 following, you get:

      1. Current weather in a city
      2. What is the weather in London? (similarity = 0.86)
      3. Peanut butter and jelly (similarity = 0.31)

      To try this for yourself, open the Python interpreter:

      Next, import spaCy and load the English-language model:

      >>> import spacy
      >>> nlp = spacy.load("en_core_web_md")
      

      Now let’s create tokens from statements 1 and 2:

      >>> statement1 = nlp("Current weather in a city")
      >>> statement2 = nlp("What is the weather in London?")
      

      Finally, let’s obtain the semantic similarity of the two statements:

      >>> print(statement1.similarity(statement2))
      

      You will receive a result like this:

      Output

      0.8557684354027663

      Setting a low minimum value (for example, 0.1) will cause the chatbot to misinterpret the user by taking statements (like statement 3) as similar to statement 1, which is incorrect. Setting a minimum value that’s too high (like 0.9) will exclude some statements that are actually similar to statement 1, such as statement 2.

      We will arbitrarily choose 0.75 for the sake of this tutorial, but you may want to test different values when working on your project.

      Let’s add this value to the script. First, open the file:

      Then add the following highlighted code to introduce the minimum value:

      ~/weather_bot.py

      import spacy
      
      . . .
      
      def chatbot(statement):
        weather = nlp("Current weather in a city")
        statement = nlp(statement)
        min_similarity = 0.75
      

      Now check if the similarity of the user’s statement to the statement about the weather is greater than or equal to the minimum similarity value you specified. Add the following highlighted if statement to check this:

      ~/weather_bot.py

      import spacy
      
      . . .
      
      def chatbot(statement):
        weather = nlp("Current weather in a city")
        statement = nlp(statement)
        min_similarity = 0.75
      
        if weather.similarity(statement) >= min_similarity:
          pass
      

      The final step is to extract the city from the user’s statement so you can pass it to the get_weather() function to retrieve the weather from the API call. Add the following highlighted for loop to implement this:

      ~/weather_bot.py

      import spacy
      
      ...
      
      def chatbot(statement):
        weather = nlp("Current weather in a city")
        statement = nlp(statement)
        min_similarity = 0.75
      
        if weather.similarity(statement) >= min_similarity:
          for ent in statement.ents:
            if ent.label_ == "GPE": # GeoPolitical Entity
              city = ent.text
              break
      

      To do this, you’re using spaCy’s named entity recognition feature. A named entity is a real-world noun that has a name, like a person, or in our case, a city. You want to extract the name of the city from the user’s statement.

      To extract the city name, you get all the named entities in the user’s statement and check which of them is a geopolitical entity (country, state, city). To do this, you loop through all the entities spaCy has extracted from the statement in the ents property, then check whether the entity label (or class) is “GPE” representing Geo-Political Entity. If it is, then you save the name of the entity (its text) in a variable called city.

      You also need to catch cases where no city was entered by adding an else block:

      ~/weather_bot.py

      import spacy
      
      ...
      
      def chatbot(statement):
        weather = nlp("Current weather in a city")
        statement = nlp(statement)
        min_similarity = 0.75
      
        if weather.similarity(statement) >= min_similarity:
          for ent in statement.ents:
            if ent.label_ == "GPE": # GeoPolitical Entity
              city = ent.text
              break
            else:
              return "You need to tell me a city to check."
      

      Now that you have the city, you can call the get_weather() function:

      ~/weather_bot.py

      import spacy
      
      ...
      
      def chatbot(statement):
        weather = nlp("Current weather in a city")
        statement = nlp(statement)
        min_similarity = 0.75
      
        if weather.similarity(statement) >= min_similarity:
          for ent in statement.ents:
            if ent.label_ == "GPE": # GeoPolitical Entity
              city = ent.text
              break
            else:
              return "You need to tell me a city to check."
      
          city_weather = get_weather(city)
          if city_weather is not None:
            return "In " + city + ", the current weather is: " + city_weather
          else:
            return "Something went wrong."
        else:
          return "Sorry I don't understand that. Please rephrase your statement."
      

      Recall that if an error is returned by the OpenWeather API, you print the error code to the terminal, and the get_weather() function returns None. In this code, you first check whether the get_weather() function returns None. If it doesn’t, then you return the weather of the city, but if it does, then you return a string saying something went wrong. The final else block is to handle the case where the user’s statement’s similarity value does not reach the threshold value. In such a case, you ask the user to rephrase their statement.

      Having completed all of that, you now have a chatbot capable of telling a user conversationally what the weather is in a city. The difference between this bot and rule-based chatbots is that the user does not have to enter the same statement every time. Instead, they can phrase their request in different ways and even make typos, but the chatbot would still be able to understand them due to spaCy’s NLP features.

      Let’s test the bot. Call the chatbot() function and pass in a statement asking what the weather is in a city, for example:

      ~/weather_bot.py

      import spacy
      
      . . .
      
      def chatbot(statement):
      
      . . .
      
      response = chatbot("Is it going to rain in Rome today?")
      print(response)
      

      Save and close the file, then run the script in your terminal:

      You will receive output similar to the following:

      Output

      In Rome, the current weather is: clear sky

      You have successfully created an intelligent chatbot capable of responding to dynamic user requests. You can try out more examples to discover the full capabilities of the bot. To do this, you can get other API endpoints from OpenWeather and other sources. Another way to extend the chatbot is to make it capable of responding to more user requests. For this, you could compare the user’s statement with more than one option and find which has the highest semantic similarity.

      Conclusion

      You have created a chatbot that is intelligent enough to respond to a user’s statement—even when the user phrases their statement in different ways. The chatbot uses the OpenWeather API to get the current weather in a city specified by the user.

      To further improve the chatbot, you can:



      Source link

      How To Use Migrations to Create and Manage Database Tables in Laravel



      Part of the Series:
      How To Build a Links Landing Page in PHP with Laravel and Docker Compose

      Laravel is an open-source PHP framework that provides a set of tools and resources to build modern PHP applications. In this project-based tutorial series, you’ll build a Links Landing Page application with the Laravel framework, using a containerized PHP development environment managed by Docker Compose.

      At the end, you’ll have a one-page website built with Laravel and managed via Artisan commands where you can share relevant links to an audience on social channels and presentations.

      Laravel database migrations allow developers to quickly bootstrap, destroy, and recreate an application’s database, without the need to log into the database console or run any SQL queries.

      In this guide, you’ll create a database migration to set up the table where you’ll save the application links. In order to do that, you’ll use the Artisan command-line tool that comes with Laravel by default. At the end, you will be able to destroy and recreate your database tables as many times as you want, using only artisan commands.

      To get started, first make sure you’re in the application’s root directory and your Docker Compose development environment is up and running:

      • cd ~/landing-laravel
      • docker-compose up -d

      Output

      landing-laravel_app_1 is up-to-date landing-laravel_nginx_1 is up-to-date landing-laravel_db_1 is up-to-date

      Next, create a database migration to set up the links table. Laravel Migrations allow developers to programmatically create, update, and destroy database tables, working as a version control system for your database schema.

      To create a new migration, you can run the make:migration Artisan command and that will bootstrap a new class on your Laravel application, in the database/migrations folder. This class will contain a default boilerplate code.

      Remember to use docker-compose exec app to run the command on the app service container, where PHP is installed:

      • docker-compose exec app php artisan make:migration create_links_table

      Output

      Created Migration: 2020_11_18_165241_create_links_table

      Notice that the migration name is generated based on the current date and time, and the name provided as argument to the make:migration command. For that reason, your migration file name will differ slightly.

      Open the generated migration class using your editor of choice:

      • nano database/migrations/2020_11_18_165241_create_links_table

      Next, update the up method to include the table columns you’ll need to store the app data.

      Replace the current content of your migration class with the following code. The highlighted values are the only lines that need adding, so if you prefer, you can also only copy those highlighted lines and include them into your Schema::create definition:

      database/migrations/2020_10_12_171200_create_links_table.php

      <?php
      
      use IlluminateDatabaseMigrationsMigration;
      use IlluminateDatabaseSchemaBlueprint;
      use IlluminateSupportFacadesSchema;
      
      class CreateLinksTable extends Migration
      {
          /**
           * Run the migrations.
           *
           * @return void
           */
          public function up()
          {
              Schema::create('links', function (Blueprint $table) {
                  $table->id();
                  $table->string('url', 200);
                  $table->text('description');
                  $table->boolean('enabled')->default(true);
                  $table->timestamps();
              });
          }
      
          /**
           * Reverse the migrations.
           *
           * @return void
           */
          public function down()
          {
              Schema::dropIfExists('links');
          }
      }
      

      In addition to the default fields that are included in the table definition that is automatically generated with the Artisan command, you’re including three new fields in this table:

      • url : A string field to save the link URL.
      • description : A text field to save the link description.
      • enabled : A field to store the state of the link, whether it’s enabled or not. The boolean Schema type will generate a tinyint unsigned field to store a value of either 0 of 1.

      Save your migration file when you’re done adding these fields. Next, run the migration with:

      • docker-compose exec app php artisan migrate

      Output

      Migration table created successfully. Migrating: 2014_10_12_000000_create_users_table Migrated: 2014_10_12_000000_create_users_table (152.46ms) Migrating: 2014_10_12_100000_create_password_resets_table Migrated: 2014_10_12_100000_create_password_resets_table (131.12ms) Migrating: 2019_08_19_000000_create_failed_jobs_table Migrated: 2019_08_19_000000_create_failed_jobs_table (101.06ms) Migrating: 2020_11_18_165241_create_links_table Migrated: 2020_11_18_165241_create_links_table (60.20ms)

      You’ll notice that other migrations were also executed along with the create_links_table. That is because the default Laravel installation comes with migrations for users (with a users table and a password_resets table) and for queued jobs (with a failed_jobs table). Because our demo application won’t use these features, it is safe to remove those migrations now; however, you may also opt to leave them in place if you are working on an application of your own and you plan on developing it further. All migration files are located at database/migrations in the app’s root folder.

      For more detailed information on database migrations, please refer to our guide on How To Use Database Migrations and Seeders to Abstract Database Setup in Laravel.

      In the next part of this series, you’ll create a custom Artisan command to list, insert, and delete entries in the app’s links table.



      Source link

      How To Create a Database Model in Laravel with Eloquent



      Part of the Series:
      How To Build a Links Landing Page in PHP with Laravel and Docker Compose

      Laravel is an open-source PHP framework that provides a set of tools and resources to build modern PHP applications. In this project-based tutorial series, you’ll build a Links Landing Page application with the Laravel framework, using a containerized PHP development environment managed by Docker Compose.

      At the end, you’ll have a one-page website built with Laravel and managed via Artisan commands where you can share relevant links to an audience on social channels and presentations.

      Eloquent is an object relational mapper (ORM) included by default within the Laravel framework. It facilitates the task of interacting with database tables, providing an object-oriented approach to inserting, updating, and deleting database records, while also providing a streamlined interface for executing SQL queries.

      Eloquent uses database models to represent tables and relationships in supported databases. The name of the database table is typically inferred from the model name, in plural form. For instance, a model named Link will use links as its default table name.

      You can use the artisan make:model command line helper to generate new models for your application. To create a new Eloquent model for your links table, run:

      • docker-compose exec app php artisan make:model Link

      Output

      Model created successfully.

      This will generate a new file containing a barebones model class. Even though this class has no apparent properties or methods, when operating the model via facades, you have access to the underlying Eloquent database classes that are able to identify database table structures and represent them as fully-functional objects.

      For your reference, this is the automatically generated model class:

      app/Models/Link.php

      <?php
      
      namespace AppModels;
      
      use IlluminateDatabaseEloquentFactoriesHasFactory;
      use IlluminateDatabaseEloquentModel;
      
      class Link extends Model
      {
          use HasFactory;
      }
      
      

      For the purpose of this series, you don’t need to make any changes to this file. If you want to extend the application in the future, you might use this model to create custom methods for the Link class that involve database operations. Additionally, if you want to create relationships between the Link model and other models, you’ll need to include a method representing the relationship in at least one of the sides. For detailed information about Eloquent relationships, please refer to the official documentation.

      In the next part of this series, you’ll create Artisan commands that will use this model to select, insert, and delete links on your database.



      Source link