One place for hosting & domains

      Applications

      How To Serve Flask Applications with uWSGI and Nginx on Ubuntu 20.04


      Not using Ubuntu 20.04?


      Choose a different version or distribution.

      A previous version of this tutorial was written by Justin Ellingwood

      Introduction

      In this guide, you will build a Python application using the Flask microframework on Ubuntu 20.04. The bulk of this article will be about how to set up the uWSGI application server and how to launch the application and configure Nginx to act as a front-end reverse proxy.

      Prerequisites

      Before starting this guide, you should have:

      • A server with Ubuntu 20.04 installed and a non-root user with sudo privileges. Follow our initial server setup guide for guidance.
      • Nginx installed, following Steps 1 through 3 of How To Install Nginx on Ubuntu 20.04.
      • A domain name configured to point to your server. You can purchase one on Namecheap or get one for free on Freenom. You can learn how to point domains to DigitalOcean by following the relevant documentation on domains and DNS. This tutorial assumes you’ve created the following DNS records:

        • An A record with your_domain pointing to your server’s public IP address.
        • An A record with www.your_domain pointing to your server’s public IP address.

      Additionally, it may be helpful to have some familiarity with uWSGI, the application server you’ll set up in this guide, and the WSGI specification. This discussion of definitions and concepts goes over both in detail.

      Step 1 — Installing the Components from the Ubuntu Repositories

      Your first step will be to install all of the pieces that you need from the Ubuntu repositories. The packages you need to install include pip, the Python package manager, to manage your Python components. You’ll also get the Python development files necessary to build uWSGI.

      First, update the local package index:

      Then install the packages that will allow you to build your Python environment. These will include python3-pip, along with a few more packages and development tools necessary for a robust programming environment:

      • sudo apt install python3-pip python3-dev build-essential libssl-dev libffi-dev python3-setuptools

      With these packages in place, you’re ready to move on to creating a virtual environment for your project.

      Step 2 — Creating a Python Virtual Environment

      A Python virtual environment is a self-contained project directory that contains specific versions of Python and the Python modules required for the given project. This is useful for isolating one application from others on the same system by managing each one’s dependencies separately. In this step, you’ll set up a Python virtual environment from which you’ll run your Flask application.

      Start by installing the python3-venv package, which will install the venv module:

      • sudo apt install python3-venv

      Next, make a parent directory for your Flask project:

      Move into the directory after you create it:

      Create a virtual environment to store your Flask project’s Python requirements by typing:

      • python3.8 -m venv myprojectenv

      This will install a local copy of Python and pip into a directory called myprojectenv within your project directory.

      Before installing applications within the virtual environment, you need to activate it. Do so by typing:

      • source myprojectenv/bin/activate

      Your prompt will change to indicate that you are now operating within the virtual environment. It will look something like this: (myprojectenv)user@host:~/myproject$.

      Step 3 — Setting Up a Flask Application

      Now that you are in your virtual environment, you can install Flask and uWSGI and then get started on designing your application.

      First, install wheel with the local instance of pip to ensure that your packages will install even if they are missing wheel archives:

      Note: Regardless of which version of Python you are using, when the virtual environment is activated, you should use the pip command (not pip3).

      Next, install Flask and uWSGI:

      Creating a Sample App

      Now that you have Flask available, you can create a sample application. Flask is a microframework. It does not include many of the tools that more full-featured frameworks might, and exists mainly as a module that you can import into your projects to assist you in initializing a web application.

      While your application might be more complex, in this example you’ll create your Flask app in a single file, called myproject.py:

      • nano ~/myproject/myproject.py

      The application code will live in this file. It will import Flask and instantiate a Flask object. You can use this to define the functions that you want to be run when a specific route is requested:

      ~/myproject/myproject.py

      from flask import Flask
      app = Flask(__name__)
      
      @app.route("/")
      def hello():
          return "<h1 style="color:blue">Hello There!</h1>"
      
      if __name__ == "__main__":
          app.run(host="0.0.0.0")
      

      Essentially, this defines what content to present to whoever accesses the root domain. Save and close the file when you’re finished. If you used nano to edit the file, as in the previous example, do so by pressing CTRL + X, Y, and then ENTER.

      If you followed the initial server setup guide, you should have a UFW firewall enabled. To test the application, you need to allow access to port 5000:

      Now, you can test your Flask app by typing:

      You will see output like the following, including a helpful warning reminding you not to use this server setup in production:

      Output

      * Serving Flask app "myproject" (lazy loading) * Environment: production WARNING: Do not use the development server in a production environment. Use a production WSGI server instead. * Debug mode: off * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

      Visit your server’s IP address followed by :5000 in your web browser:

      http://your_server_ip:5000
      

      You will see something like this:

      Flask sample app

      When you are finished, hit CTRL + C in your terminal window to stop the Flask development server.

      Creating the WSGI Entry Point

      Next, create a file that will serve as the entry point for your application. This will tell your uWSGI server how to interact with it.

      Call the file wsgi.py:

      In this file, import the Flask instance from your application and then run it:

      ~/myproject/wsgi.py

      from myproject import app
      
      if __name__ == "__main__":
          app.run()
      

      Save and close the file when you are finished.

      Step 4 — Configuring uWSGI

      Your application is now written with an entry point established. You can move on to configuring uWSGI.

      Testing Whether uWSGI Can Serve the Application

      As a first step, test to make sure that uWSGI can correctly serve your application by passing it the name of your entry point. This is constructed by the name of the module (minus the .py extension) plus the name of the callable within the application. In the context of this tutorial, the name of the entry point is wsgi:app.

      Also, specify the socket so that it will be started on a publicly available interface, as well as the protocol, so that it will use HTTP instead of the uwsgi binary protocol. Use the same port number, 5000, that you opened earlier:

      • uwsgi --socket 0.0.0.0:5000 --protocol=http -w wsgi:app

      Visit your server’s IP address with :5000 appended to the end in your web browser again:

      http://your_server_ip:5000
      

      You will see your application’s output again:

      Flask sample app

      When you have confirmed that it’s functioning properly, press CTRL + C in your terminal window.

      You’re now done with your virtual environment, so you can deactivate it:

      Any Python commands will now use the system’s Python environment again.

      Creating a uWSGI Configuration File

      You have tested that uWSGI is able to serve your application, but ultimately you will want something more robust for long-term usage. You can create a uWSGI configuration file with the relevant options for this.

      Place that file in your project directory and call it myproject.ini:

      • nano ~/myproject/myproject.ini

      Inside, start the file off with the [uwsgi] header so that uWSGI knows to apply the settings. Below that, specify module itself — by referring to the wsgi.py file minus the extension — and the callable within the file, app:

      ~/myproject/myproject.ini

      [uwsgi]
      module = wsgi:app
      

      Next, tell uWSGI to start up in master mode and spawn five worker processes to serve actual requests:

      ~/myproject/myproject.ini

      [uwsgi]
      module = wsgi:app
      
      master = true
      processes = 5
      

      When you were testing, you exposed uWSGI on a network port. However, you’re going to be using Nginx to handle actual client connections, which will then pass requests to uWSGI. Since these components are operating on the same computer, a Unix socket is preferable because it is faster and more secure. Call the socket myproject.sock and place it in this directory.

      Next, change the permissions on the socket. You’ll be giving the Nginx group ownership of the uWSGI process later on, so you need to make sure the group owner of the socket can read information from it and write to it. Also, add the vacuum option and set it to true; this will clean up the socket when the process stops:

      ~/myproject/myproject.ini

      [uwsgi]
      module = wsgi:app
      
      master = true
      processes = 5
      
      socket = myproject.sock
      chmod-socket = 660
      vacuum = true
      

      The last thing to do is set the die-on-term option. This can help ensure that the init system and uWSGI have the same assumptions about what each process signal means. Setting this aligns the two system components, implementing the expected behavior:

      ~/myproject/myproject.ini

      [uwsgi]
      module = wsgi:app
      
      master = true
      processes = 5
      
      socket = myproject.sock
      chmod-socket = 660
      vacuum = true
      
      die-on-term = true
      

      You may have noticed that these lines do not specify a protocol like you did from the command line. That is because by default, uWSGI speaks using the uwsgi protocol, a fast binary protocol designed to communicate with other servers. Nginx can speak this protocol natively, so it’s better to use this than to force communication by HTTP.

      When you are finished, save and close the file.

      With that, uWSGI is configured on your system. In order to give you more flexibility in how you manage your Flask application, you can now configure it to run as a systemd service.

      Step 5 — Creating a systemd Unit File

      Systemd is a suite of tools that provides a fast and flexible init model for managing system services. Creating a systemd unit file will allow Ubuntu’s init system to automatically start uWSGI and serve the Flask application whenever the server boots.

      Create a unit file ending in .service within the /etc/systemd/system directory to begin:

      • sudo nano /etc/systemd/system/myproject.service

      Inside, start with the [Unit] section, which is used to specify metadata and dependencies. Then put a description of the service here and tell the init system to only start this after the networking target has been reached:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=uWSGI instance to serve myproject
      After=network.target
      

      Next, open up the [Service] section. This will specify the user and group that you want the process to run under. Give your regular user account ownership of the process since it owns all of the relevant files. Then give group ownership to the www-data group so that Nginx can communicate easily with the uWSGI processes. Remember to replace the username here with your username:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=uWSGI instance to serve myproject
      After=network.target
      
      [Service]
      User=sammy
      Group=www-data
      

      Next, map out the working directory and set the PATH environmental variable so that the init system knows that the executables for the process are located within your virtual environment. Also, specify the command to start the service. Systemd requires that you give the full path to the uWSGI executable, which is installed within your virtual environment. Here, we pass the name of the .ini configuration file you created in your project directory.

      Remember to replace the username and project paths with your own information:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=uWSGI instance to serve myproject
      After=network.target
      
      [Service]
      User=sammy
      Group=www-data
      WorkingDirectory=/home/sammy/myproject
      Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
      ExecStart=/home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini
      

      Finally, add an [Install] section. This will tell systemd what to link this service to if you enable it to start at boot. In this case, set the service to start when the regular multi-user system is up and running:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=uWSGI instance to serve myproject
      After=network.target
      
      [Service]
      User=sammy
      Group=www-data
      WorkingDirectory=/home/sammy/myproject
      Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
      ExecStart=/home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini
      
      [Install]
      WantedBy=multi-user.target
      

      With that, your systemd service file is complete. Save and close it now.

      You can now start the uWSGI service you created:

      • sudo systemctl start myproject

      Then enable it so that it starts at boot:

      • sudo systemctl enable myproject

      Check the status:

      • sudo systemctl status myproject

      You will see output like this:

      Output

      ● myproject.service - uWSGI instance to serve myproject Loaded: loaded (/etc/systemd/system/myproject.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-05-20 13:21:39 UTC; 8h ago Main PID: 22146 (uwsgi) Tasks: 6 (limit: 2345) Memory: 25.5M CGroup: /system.slice/myproject.service ├─22146 /home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini ├─22161 /home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini ├─22162 /home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini ├─22163 /home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini ├─22164 /home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini └─22165 /home/sammy/myproject/myprojectenv/bin/uwsgi --ini myproject.ini

      If you see any errors, be sure to resolve them before continuing with the tutorial. Otherwise, you can move on to configuring your Nginx installation to pass requests to the myproject.sock socket.

      Step 6 — Configuring Nginx to Proxy Requests

      Your uWSGI application server is now up and running, waiting for requests on the socket file in the project directory. In this step, you’ll configure Nginx to pass web requests to that socket using the uwsgi protocol.

      Begin by creating a new server block configuration file in Nginx’s sites-available directory. To keep in line with the rest of the guide, the following example refers to this as myproject:

      • sudo nano /etc/nginx/sites-available/myproject

      Open up a server block and tell Nginx to listen on the default port 80. Additionally, tell it to use this block for requests for your server’s domain name:

      /etc/nginx/sites-available/myproject

      server {
          listen 80;
          server_name your_domain www.your_domain;
      }
      

      Next, add a location block that matches every request. Within this block, include the uwsgi_params file that specifies some general uWSGI parameters that need to be set. Then pass the requests to the socket you defined using the uwsgi_pass directive:

      /etc/nginx/sites-available/myproject

      server {
          listen 80;
          server_name your_domain www.your_domain;
      
          location / {
              include uwsgi_params;
              uwsgi_pass unix:/home/sammy/myproject/myproject.sock;
          }
      }
      

      Save and close the file when you’re finished.

      To enable the Nginx server block configuration you’ve just created, link the file to the sites-enabled directory:

      • sudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled

      With the file in that directory, you can test for syntax errors by typing:

      If this returns without indicating any issues, restart the Nginx process to read the new configuration:

      • sudo systemctl restart nginx

      Finally, adjust the firewall once again. You no longer need access through port 5000, so you can remove that rule. Then, you can allow access to the Nginx server:

      • sudo ufw delete allow 5000
      • sudo ufw allow 'Nginx Full'

      You will now be able to navigate to your server’s domain name in your web browser:

      http://your_domain
      

      You will see your application output:

      Flask sample app

      If you encounter any errors, trying checking the following:

      • sudo less /var/log/nginx/error.log: checks the Nginx error logs.
      • sudo less /var/log/nginx/access.log: checks the Nginx access logs.
      • sudo journalctl -u nginx: checks the Nginx process logs.
      • sudo journalctl -u myproject: checks your Flask app’s uWSGI logs.

      Step 7 — Securing the Application

      To ensure that traffic to your server remains secure, obtain an SSL certificate for your domain. There are multiple ways to do this, including getting a free certificate from Let’s Encrypt, generating a self-signed certificate, or buying one from a commercial provider. For the sake of expediency, this tutorial explains how to obtain a free certificate from Let’s Encrypt.

      First, install Certbot and its Nginx plugin with apt:

      • sudo apt install certbot python3-certbot-nginx

      Certbot provides a variety of ways to obtain SSL certificates through plugins. The Nginx plugin will take care of reconfiguring Nginx and reloading the config whenever necessary. To use this plugin, type the following:

      • sudo certbot --nginx -d your_domain -d www.your_domain

      This runs certbot with the --nginx plugin, using -d to specify the names you’d like the certificate to be valid for.

      If this is your first time running certbot on this server, you will be prompted to enter an email address and agree to the terms of service. After doing so, certbot will communicate with the Let’s Encrypt server, then run a challenge to verify that you control the domain you’re requesting a certificate for.

      If that’s successful, certbot will ask how you’d like to configure your HTTPS settings:

      Output

      Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1: No redirect - Make no further changes to the webserver configuration. 2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for new sites, or if you're confident your site works on HTTPS. You can undo this change by editing your web server's configuration. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

      Select your choice then hit ENTER. The configuration will be updated, and Nginx will reload to pick up the new settings. certbot will wrap up with a message telling you the process was successful and where your certificates are stored:

      Output

      IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at: /etc/letsencrypt/live/your_domain/fullchain.pem Your key file has been saved at: /etc/letsencrypt/live/your_domain/privkey.pem Your cert will expire on 2020-08-18. To obtain a new or tweaked version of this certificate in the future, simply run certbot again with the "certonly" option. To non-interactively renew *all* of your certificates, run "certbot renew" - Your account credentials have been saved in your Certbot configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Certbot so making regular backups of this folder is ideal. - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le

      If you followed the Nginx installation instructions in the prerequisites, you will no longer need the redundant HTTP profile allowance:

      • sudo ufw delete allow 'Nginx HTTP'

      To verify the configuration, navigate once again to your domain, using https://:

      https://your_domain
      

      You will see your application output once again, along with your browser’s security indicator, which should indicate that the site is secured.

      Conclusion

      In this guide, you created and secured a basic Flask application within a Python virtual environment. Then you created a WSGI entry point so that any WSGI-capable application server can interface with it, and then configured the uWSGI app server to provide this function. Afterwards, you created a systemd service file to automatically launch the application server on boot. You also created an Nginx server block that passes web client traffic to the application server, thereby relaying external requests, and secured traffic to your server with Let’s Encrypt.

      Flask is a simple yet flexible framework meant to provide your applications with functionality without being too restrictive about structure or design. You can use the general stack described in this guide to serve the flask applications that you design.



      Source link

      How To Develop Applications on Kubernetes with Okteto


      The author selected Girls Who Code to receive a donation as part of the Write for DOnations program.

      Introduction

      The Okteto CLI is an open-source project that provides a local development experience for applications running on Kubernetes. With it you can write your code on your local IDE and as soon as you save a file, the changes can be pushed to your Kubernetes cluster and your app will immediately update. This whole process happens without the need to build Docker images or apply Kubernetes manifests, which can take considerable time.

      In this tutorial, you’ll use Okteto to improve your productivity when developing a Kubernetes-native application. First, you’ll create a Kubernetes cluster and use it to run a standard “Hello World” application. Then you’ll use Okteto to develop and automatically update your application without having to install anything locally.

      Prerequisites

      Before you begin this tutorial, you’ll need the following:

      Step 1 — Creating the Hello World Application

      The “Hello World” program is a time-honored tradition in web development. In this case, it is a simple web service that responds “Hello World” to every request. Now that you’ve created your Kubernetes cluster, let’s create a “Hello World” app in Golang and the manifests that you’ll use to deploy it on Kubernetes.

      First change to your home directory:

      Now make a new directory called hello_world and move inside it:

      • mkdir hello_world
      • cd hello_world

      Create and open a new file under the name main.go with your favorite IDE or text editor:

      main.go will be a Golang web server that returns the message Hello world!. So, let’s use the following code:

      main.go

      package main
      
      import (
          "fmt"
          "net/http"
      )
      
      func main() {
          fmt.Println("Starting hello-world server...")
          http.HandleFunc("/", helloServer)
          if err := http.ListenAndServe(":8080", nil); err != nil {
              panic(err)
          }
      }
      
      func helloServer(w http.ResponseWriter, r *http.Request) {
          fmt.Fprint(w, "Hello world!")
      }
      

      The code in main.go does the following:

      • The first statement in a Go source file must be the package name. Executable commands must always use package main.
      • The import section indicates which packages the code depends on. In this case it uses fmt for string manipulation, and net/http for the HTTP server.
      • The main function is the entry point to your binary. The http.HandleFunc method is used to configure the server to call the helloServer function when a request to the / path is received. http.ListenAndServe starts an HTTP server that listens on all network interfaces on port 8080.
      • The helloServer function contains the logic of your request handler. In this case, it will write Hello world! as the response to the request.

      You need to create a Docker image and push it to your Docker registry so that Kubernetes can pull it and then run the application.

      Open a new file under the name Dockerfile with your favorite IDE or text editor:

      The Dockerfile will contain the commands required to build your application’s Docker container. Let’s use the following code:

      Dockerfile

      FROM golang:alpine as builder
      RUN apk --update --no-cache add bash
      WORKDIR /app
      ADD . .
      RUN go build -o app
      
      FROM alpine as prod
      WORKDIR /app
      COPY --from=builder /app/app /app/app
      EXPOSE 8080
      CMD ["./app"]
      

      The Dockerfile contains two stages, builder and prod:

      • The builder stage contains the Go build tools. It’s responsible for copying the files and building the Go binary.
      • The prod stage is the final image. It will contain only a stripped down OS and the application binary.

      This is a good practice to follow. It makes your production containers smaller and safer since they only contain your application and exactly what is needed to run it.

      Build the container image (replace your_DockerHub_username with your Docker Hub username):

      • docker build -t your_DockerHub_username/hello-world:latest

      Now push it to Docker Hub:

      • docker push your_DockerHub_username/hello-world:latest

      Next, create a new folder for the Kubernetes manifests:

      When you use a Kubernetes manifest, you tell Kubernetes how you want your application to run. This time, you’ll create a deployment object. So, create a new file deployment.yaml with your favorite IDE or text editor:

      The following content describes a Kubernetes deployment object that runs the okteto/hello-world:latest Docker image. Add this content to your new file, but in your case replace okteto listed after the image label with your_DockerHub_username:

      ~/hello_world/k8s/deployment.yaml

      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: hello-world
      spec:
        selector:
          matchLabels:
            app: hello-world
        replicas: 1
        template:
          metadata:
            labels:
              app: hello-world
          spec:
            containers:
            - name: hello-world
              image: your_DockerHub_username/hello-world:latest
              ports:
              - containerPort: 8080
      

      The deployment manifest has three main sections:

      • metadata defines the name for your deployment.
      • replicas defines how many copies of it you want running.
      • template tells Kubernetes what to deploy, and what labels to add. In this case, a single container, with the okteto/hello-world:latest image, listening on port 8080, and with the app: hello-world label. Note that this label is the same used in the selector section.

      You’ll now need a way to access your application. You can expose an application on Kubernetes by creating a service object. Let’s continue using manifests to do that. Create a new file called service.yaml with your favorite IDE or text editor:

      The following content describes a service that exposes the hello-world deployment object, which under the hood will use a DigitalOcean Load Balancer:

      k8s/service.yaml

      apiVersion: v1
      kind: Service
      metadata:
        name: hello-world
      spec:
        type: LoadBalancer
        ports:
          - protocol: TCP
            port: 80
            targetPort: 8080
            name: http
        selector:
          app: hello-world
      

      The service manifest has four main sections:

      • metadata tells Kubernetes how to name your service.
      • type tells Kubernetes how you want to expose your service. In this case, it will expose it externally through a Digital Ocean Load Balancer.
      • The ports label tells Kubernetes which ports you want to expose, and how to map them to your deployment. In this case, you will expose port 80 externally and direct it to port 8080 in your deployment.
      • selector tells Kubernetes how to direct traffic. In this case, any pod with the app: hello-world label will receive traffic.

      You now have everything ready to deploy your “Hello World” application on Kubernetes. We will do this next.

      Step 2 — Deploying Your Hello World Application

      In this step you’ll deploy your “Hello World” application on Kubernetes, and then you’ll validate that it is working correctly.

      Start by deploying your application on Kubernetes:

      You’ll see the following output:

      Output

      deployment.apps "hello-world" created service "hello-world" created

      After about one minute or so, you will be able to retrieve your application’s IP. Use this kubectl command to check your service:

      • kubectl get service hello-world

      You’ll see an output like this listing your Kubernetes service objects. Note your application’s IP in the the EXTERNAL-IP column:

      Output

      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-world ClusterIP your_cluster_ip your_external_ip 8080/TCP 37s

      Open your browser and go to your_external_ip listed for your “Hello World” application. Confirm that your application is up and running before continuing with the next step.

      Hello World Okteto

      Until this moment, you’ve followed a fairly traditional pathway for developing applications with Kubernetes. Moving forward, whenever you want to change the code in your application, you’ll have to build and push a new Docker image, and then pull that image from Kubernetes. This process can take quite some time. Okteto was designed to streamline this development inner-loop. Let’s look at the Okteto CLI and see just how it can help.

      Step 3 — Installing the Okteto CLI

      You will now improve your Kubernetes development productivity by installing the Okteto CLI. The Okteto command line interface is an open-source project that lets you synchronize application code changes to an application running on Kubernetes. You can continue using your favorite IDE, debuggers, or compilers without having to commit, build, push, or redeploy containers to test your application–as you did in the previous steps.

      To install the Okteto CLI on a macOS or Linux machine, run the following command:

      • curl https://get.okteto.com -sSfL | sh

      Let’s take a closer look at this command:

      • The curl command is used to transfer data to and from a server.
      • The -s flag suppresses any output.
      • The -S flag shows errors.
      • The -f flag causes the request to fail on HTTP errors.
      • The -L flag makes the request follow redirects.
      • The | operator pipes this output to the sh command, which will download and install the latest okteto binary in your local machine.

      If you are running Windows, you can alternately download the file through your web browser and manually add it to your $PATH.

      Once the Okteto CLI is installed, you are ready to put your “Hello World” application in development mode.

      Step 4 — Putting Your Hello World Application in Development Mode

      The Okteto CLI is designed to swap the application running on a Kubernetes cluster with the code you have in your machine. To do so, Okteto uses the information provided from an Okteto manifest file. This file declares the Kubernetes deployment object that will swap with your local code.

      Create a new file called okteto.yaml with your favorite IDE or text editor:

      Let’s write a basic manifest where you define the deployment object name, the Docker base image to use, and a shell. We will return to this information later. Use the following sample content file:

      okteto.yaml

      name: hello-world
      image: okteto/golang:1
      workdir: /app
      command: ["bash"]
      

      Prepare to put your application in development mode by running the following command:

      Output

      ✓ Development environment activated ✓ Files synchronized Namespace: default Name: hello-world Welcome to your development environment. Happy coding! default:hello-world /app>

      The okteto up command swaps the “Hello World” application into a development environment, which means:

      • The Hello World application container is updated with the docker image okteto/golang:1. This image contains the required dev tools to build, test, debug, and run the “Hello World” application.

      • A file synchronization service is created to keep your changes up-to-date between your local filesystem and your application pods.

      • A remote shell starts in your development environment. Now you can build, test, and run your application as if you were in your local machine.

      • Whatever process you run in the remote shell will get the same incoming traffic, the same environment variables, volumes, or secrets as the original “Hello World” application pods. This, in turn, gives you a highly realistic, production-like development environment.

      In the same console, now run the application as you would typically do (without building and pushing a Docker image), like this:

      Output

      Starting hello-world server...

      The first time you run the application, Go will download your dependencies and compile your application. Wait for this process to finish and test your application by opening your browser and refreshing the page of your application, just as you did previously.

      Now you are ready to begin developing directly on Kubernetes.

      Step 5 — Developing Directly on Kubernetes

      Let’s start making changes to the “Hello World” application and then see how these changes get reflected in Kubernetes.

      Open the main.go file with your favorite IDE or text editor. For example, open a separate console and run the following command:

      Then, change your response message to Hello world from DigitalOcean!:

      main.go

      package main
      
      import (
          "fmt"
          "net/http"
      )
      
      func main() {
          fmt.Println("Starting hello-world server...")
          http.HandleFunc("/", helloServer)
          if err := http.ListenAndServe(":8080", nil); err != nil {
              panic(err)
          }
      }
      
      func helloServer(w http.ResponseWriter, r *http.Request) {
          fmt.Fprint(w, "Hello world from DigitalOcean!")
      }
      

      It is here that your workflow changes. Instead of building images and redeploying containers to update the “Hello World” application, Okteto will synchronize your changes to your development environment on Kubernetes.

      From the console where you executed the okteto up command, cancel the execution of go run main.go by pressing CTRL + C. Now rerun the application:

      • default:hello-world /app> go run main.go

      Output

      Starting hello-world server...

      Go back to the browser and reload the page for your “Hello World” application.

      Hello world DigitalOcean

      Your code changes were applied instantly to Kubernetes, and all without requiring any commits, builds, or pushes.

      Conclusion

      Okteto transforms your Kubernetes cluster into a fully-featured development platform with the click of a button. In this tutorial you installed and configured the Okteto CLI to iterate your code changes directly on Kubernetes as fast as you can type code. Now you can head over to the Okteto samples repository to see how to use Okteto with different programming languages and debuggers.

      Also, if you share a Kubernetes cluster with your team, consider giving each member access to a secure Kubernetes namespace, configured to be isolated from other developers working on the same cluster. This great functionality is also provided by the Okteto App in the DigitalOcean Kubernetes Marketplace.



      Source link

      How To Serve Flask Applications with Gunicorn and Nginx on Ubuntu 20.04


      In this guide, you will build a Python application using the Flask microframework on Ubuntu 20.04. The majority of this tutorial is about how to set up the Gunicorn application server to run the application and how to configure Nginx to act as a front-end reverse proxy.

      Introduction

      In this guide, you will build a Python application using the Flask microframework on Ubuntu 20.04. The bulk of this article will be about how to set up the Gunicorn application server and how to launch the application and configure Nginx to act as a front-end reverse proxy.

      Prerequisites

      Before starting this guide, you should have:

      • A server with Ubuntu 20.04 installed and a non-root user with sudo privileges. Follow our initial server setup guide for guidance.
      • Nginx installed, following Steps 1 and 2 of How To Install Nginx on Ubuntu 20.04.
      • A domain name configured to point to your server. You can purchase one on Namecheap or get one for free on Freenom. You can learn how to point domains to DigitalOcean by following the relevant documentation on domains and DNS. Be sure to create the following DNS records:

        • An A record with your_domain pointing to your server’s public IP address.
        • An A record with www.your_domain pointing to your server’s public IP address.
      • Familiarity with the WSGI specification, which the Gunicorn server will use to communicate with your Flask application. This discussion covers WSGI in more detail.

      Step 1 — Installing the Components from the Ubuntu Repositories

      Our first step will be to install all of the pieces we need from the Ubuntu repositories. This includes pip, the Python package manager, which will manage our Python components. We will also get the Python development files necessary to build some of the Gunicorn components.

      First, let’s update the local package index and install the packages that will allow us to build our Python environment. These will include python3-pip, along with a few more packages and development tools necessary for a robust programming environment:

      • sudo apt update
      • sudo apt install python3-pip python3-dev build-essential libssl-dev libffi-dev python3-setuptools

      With these packages in place, let’s move on to creating a virtual environment for our project.

      Step 2 — Creating a Python Virtual Environment

      Next, we’ll set up a virtual environment in order to isolate our Flask application from the other Python files on the system.

      Start by installing the python3-venv package, which will install the venv module:

      • sudo apt install python3-venv

      Next, let’s make a parent directory for our Flask project. Move into the directory after you create it:

      • mkdir ~/myproject
      • cd ~/myproject

      Create a virtual environment to store your Flask project’s Python requirements by typing:

      • python3 -m venv myprojectenv

      This will install a local copy of Python and pip into a directory called myprojectenv within your project directory.

      Before installing applications within the virtual environment, you need to activate it. Do so by typing:

      • source myprojectenv/bin/activate

      Your prompt will change to indicate that you are now operating within the virtual environment. It will look something like this: (myprojectenv)user@host:~/myproject$.

      Step 3 — Setting Up a Flask Application

      Now that you are in your virtual environment, you can install Flask and Gunicorn and get started on designing your application.

      First, let’s install wheel with the local instance of pip to ensure that our packages will install even if they are missing wheel archives:

      Note


      Regardless of which version of Python you are using, when the virtual environment is activated, you should use the pip command (not pip3).

      Next, let’s install Flask and Gunicorn:

      • pip install gunicorn flask

      Creating a Sample App

      Now that you have Flask available, you can create a simple application. Flask is a microframework. It does not include many of the tools that more full-featured frameworks might, and exists mainly as a module that you can import into your projects to assist you in initializing a web application.

      While your application might be more complex, we’ll create our Flask app in a single file, called myproject.py:

      • nano ~/myproject/myproject.py

      The application code will live in this file. It will import Flask and instantiate a Flask object. You can use this to define the functions that should be run when a specific route is requested:

      ~/myproject/myproject.py

      from flask import Flask
      app = Flask(__name__)
      
      @app.route("/")
      def hello():
          return "<h1 style="color:blue">Hello There!</h1>"
      
      if __name__ == "__main__":
          app.run(host="0.0.0.0")
      

      This basically defines what content to present when the root domain is accessed. Save and close the file when you’re finished.

      If you followed the initial server setup guide, you should have a UFW firewall enabled. To test the application, you need to allow access to port 5000:

      Now you can test your Flask app by typing:

      You will see output like the following, including a helpful warning reminding you not to use this server setup in production:

      Output

      * Serving Flask app "myproject" (lazy loading) * Environment: production WARNING: Do not use the development server in a production environment. Use a production WSGI server instead. * Debug mode: off * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

      Visit your server’s IP address followed by :5000 in your web browser:

      http://your_server_ip:5000
      

      You should see something like this:

      Flask sample app

      When you are finished, hit CTRL-C in your terminal window to stop the Flask development server.

      Creating the WSGI Entry Point

      Next, let’s create a file that will serve as the entry point for our application. This will tell our Gunicorn server how to interact with the application.

      Let’s call the file wsgi.py:

      In this file, let’s import the Flask instance from our application and then run it:

      ~/myproject/wsgi.py

      from myproject import app
      
      if __name__ == "__main__":
          app.run()
      

      Save and close the file when you are finished.

      Step 4 — Configuring Gunicorn

      Your application is now written with an entry point established. We can now move on to configuring Gunicorn.

      Before moving on, we should check that Gunicorn can serve the application correctly.

      We can do this by simply passing it the name of our entry point. This is constructed as the name of the module (minus the .py extension), plus the name of the callable within the application. In our case, this is wsgi:app.

      We’ll also specify the interface and port to bind to so that the application will be started on a publicly available interface:

      • cd ~/myproject
      • gunicorn --bind 0.0.0.0:5000 wsgi:app

      You should see output like the following:

      Output

      [2020-05-20 14:13:00 +0000] [46419] [INFO] Starting gunicorn 20.0.4 [2020-05-20 14:13:00 +0000] [46419] [INFO] Listening at: http://0.0.0.0:5000 (46419) [2020-05-20 14:13:00 +0000] [46419] [INFO] Using worker: sync [2020-05-20 14:13:00 +0000] [46421] [INFO] Booting worker with pid: 46421

      Visit your server’s IP address with :5000 appended to the end in your web browser again:

      http://your_server_ip:5000
      

      You should see your application’s output:

      Flask sample app

      When you have confirmed that it’s functioning properly, press CTRL-C in your terminal window.

      We’re now done with our virtual environment, so we can deactivate it:

      Any Python commands will now use the system’s Python environment again.

      Next, let’s create the systemd service unit file. Creating a systemd unit file will allow Ubuntu’s init system to automatically start Gunicorn and serve the Flask application whenever the server boots.

      Create a unit file ending in .service within the /etc/systemd/system directory to begin:

      • sudo nano /etc/systemd/system/myproject.service

      Inside, we’ll start with the [Unit] section, which is used to specify metadata and dependencies. Let’s put a description of our service here and tell the init system to only start this after the networking target has been reached:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=Gunicorn instance to serve myproject
      After=network.target
      

      Next, let’s open up the [Service] section. This will specify the user and group that we want the process to run under. Let’s give our regular user account ownership of the process since it owns all of the relevant files. Let’s also give group ownership to the www-data group so that Nginx can communicate easily with the Gunicorn processes. Remember to replace the username here with your username:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=Gunicorn instance to serve myproject
      After=network.target
      
      [Service]
      User=sammy
      Group=www-data
      

      Next, let’s map out the working directory and set the PATH environmental variable so that the init system knows that the executables for the process are located within our virtual environment. Let’s also specify the command to start the service. This command will do the following:

      • Start 3 worker processes (though you should adjust this as necessary)
      • Create and bind to a Unix socket file, myproject.sock, within our project directory. We’ll set an umask value of 007 so that the socket file is created giving access to the owner and group, while restricting other access
      • Specify the WSGI entry point file name, along with the Python callable within that file (wsgi:app)

      Systemd requires that we give the full path to the Gunicorn executable, which is installed within our virtual environment.

      Remember to replace the username and project paths with your own information:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=Gunicorn instance to serve myproject
      After=network.target
      
      [Service]
      User=sammy
      Group=www-data
      WorkingDirectory=/home/sammy/myproject
      Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
      ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app
      

      Finally, let’s add an [Install] section. This will tell systemd what to link this service to if we enable it to start at boot. We want this service to start when the regular multi-user system is up and running:

      /etc/systemd/system/myproject.service

      [Unit]
      Description=Gunicorn instance to serve myproject
      After=network.target
      
      [Service]
      User=sammy
      Group=www-data
      WorkingDirectory=/home/sammy/myproject
      Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
      ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app
      
      [Install]
      WantedBy=multi-user.target
      

      With that, our systemd service file is complete. Save and close it now.

      We can now start the Gunicorn service we created and enable it so that it starts at boot:

      • sudo systemctl start myproject
      • sudo systemctl enable myproject

      Let’s check the status:

      • sudo systemctl status myproject

      You should see output like this:

      Output

      ● myproject.service - Gunicorn instance to serve myproject Loaded: loaded (/etc/systemd/system/myproject.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-05-20 14:15:18 UTC; 1s ago Main PID: 46430 (gunicorn) Tasks: 4 (limit: 2344) Memory: 51.3M CGroup: /system.slice/myproject.service ├─46430 /home/sammy/myproject/myprojectenv/bin/python3 /home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app ├─46449 /home/sammy/myproject/myprojectenv/bin/python3 /home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app ├─46450 /home/sammy/myproject/myprojectenv/bin/python3 /home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app └─46451 /home/sammy/myproject/myprojectenv/bin/python3 /home/sammy/myproject/myprojectenv/bin/gunicorn --workers 3 --bind unix:myproject.sock -m 007 wsgi:app

      If you see any errors, be sure to resolve them before continuing with the tutorial.

      Step 5 — Configuring Nginx to Proxy Requests

      Our Gunicorn application server should now be up and running, waiting for requests on the socket file in the project directory. Let’s now configure Nginx to pass web requests to that socket by making some small additions to its configuration file.

      Begin by creating a new server block configuration file in Nginx’s sites-available directory. Let’s call this myproject to keep in line with the rest of the guide:

      • sudo nano /etc/nginx/sites-available/myproject

      Open up a server block and tell Nginx to listen on the default port 80. Let’s also tell it to use this block for requests for our server’s domain name:

      /etc/nginx/sites-available/myproject

      server {
          listen 80;
          server_name your_domain www.your_domain;
      }
      

      Next, let’s add a location block that matches every request. Within this block, we’ll include the proxy_params file that specifies some general proxying parameters that need to be set. We’ll then pass the requests to the socket we defined using the proxy_pass directive:

      /etc/nginx/sites-available/myproject

      server {
          listen 80;
          server_name your_domain www.your_domain;
      
          location / {
              include proxy_params;
              proxy_pass http://unix:/home/sammy/myproject/myproject.sock;
          }
      }
      

      Save and close the file when you’re finished.

      To enable the Nginx server block configuration you’ve just created, link the file to the sites-enabled directory:

      • sudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled

      With the file in that directory, you can test for syntax errors:

      If this returns without indicating any issues, restart the Nginx process to read the new configuration:

      • sudo systemctl restart nginx

      Finally, let’s adjust the firewall again. We no longer need access through port 5000, so we can remove that rule. We can then allow full access to the Nginx server:

      • sudo ufw delete allow 5000
      • sudo ufw allow 'Nginx Full'

      You should now be able to navigate to your server’s domain name in your web browser:

      http://your_domain
      

      You should see your application’s output:

      Flask sample app

      If you encounter any errors, trying checking the following:

      • sudo less /var/log/nginx/error.log: checks the Nginx error logs.
      • sudo less /var/log/nginx/access.log: checks the Nginx access logs.
      • sudo journalctl -u nginx: checks the Nginx process logs.
      • sudo journalctl -u myproject: checks your Flask app’s Gunicorn logs.

      Step 6 — Securing the Application

      To ensure that traffic to your server remains secure, let’s get an SSL certificate for your domain. There are multiple ways to do this, including getting a free certificate from Let’s Encrypt, generating a self-signed certificate, or buying one from another provider and configuring Nginx to use it by following Steps 2 through 6 of How to Create a Self-signed SSL Certificate for Nginx in Ubuntu 20.04. We will go with option one for the sake of expediency.

      Install Certbot’s Nginx package with apt:

      • sudo apt install python3-certbot-nginx

      Certbot provides a variety of ways to obtain SSL certificates through plugins. The Nginx plugin will take care of reconfiguring Nginx and reloading the config whenever necessary. To use this plugin, type the following:

      • sudo certbot --nginx -d your_domain -d www.your_domain

      This runs certbot with the --nginx plugin, using -d to specify the names we’d like the certificate to be valid for.

      If this is your first time running certbot, you will be prompted to enter an email address and agree to the terms of service. After doing so, certbot will communicate with the Let’s Encrypt server, then run a challenge to verify that you control the domain you’re requesting a certificate for.

      If that’s successful, certbot will ask how you’d like to configure your HTTPS settings:

      Output

      Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access. ------------------------------------------------------------------------------- 1: No redirect - Make no further changes to the webserver configuration. 2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for new sites, or if you're confident your site works on HTTPS. You can undo this change by editing your web server's configuration. ------------------------------------------------------------------------------- Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

      Select your choice then hit ENTER. The configuration will be updated, and Nginx will reload to pick up the new settings. certbot will wrap up with a message telling you the process was successful and where your certificates are stored:

      Output

      IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at: /etc/letsencrypt/live/your_domain/fullchain.pem Your key file has been saved at: /etc/letsencrypt/live/your_domain/privkey.pem Your cert will expire on 2020-08-18. To obtain a new or tweaked version of this certificate in the future, simply run certbot again with the "certonly" option. To non-interactively renew *all* of your certificates, run "certbot renew" - Your account credentials have been saved in your Certbot configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Certbot so making regular backups of this folder is ideal. - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le

      If you followed the Nginx installation instructions in the prerequisites, you will no longer need the redundant HTTP profile allowance:

      • sudo ufw delete allow 'Nginx HTTP'

      To verify the configuration, navigate once again to your domain, using https://:

      https://your_domain
      

      You should see your application output once again, along with your browser’s security indicator, which should indicate that the site is secured.

      Conclusion

      In this guide, you created and secured a simple Flask application within a Python virtual environment. You created a WSGI entry point so that any WSGI-capable application server can interface with it, and then configured the Gunicorn app server to provide this function. Afterwards, you created a systemd service file to automatically launch the application server on boot. You also created an Nginx server block that passes web client traffic to the application server, relaying external requests, and secured traffic to your server with Let’s Encrypt.

      Flask is a very simple, but extremely flexible framework meant to provide your applications with functionality without being too restrictive about structure and design. You can use the general stack described in this guide to serve the flask applications that you design.



      Source link