One place for hosting & domains

      Storage

      Host a Static Site using Linode Object Storage


      Updated by Linode

      Contributed by

      Linode

      Note

      Object Storage is currently in a closed early access Beta, and you may not have access to Object Storage through the Cloud Manager or other tools. To gain access to the Early Access Program (EAP), open up a Customer Support ticket noting that you’d like to be included in the program, or e-mail objbeta@linode.com – beta access is completely free.

      Additionally, because Object Storage is in Beta, there may be breaking changes to how you access and manage Object Storage. This guide will be updated to reflect these changes if and when they occur.

      Why Host a Static Site on Object Storage?

      Static site generators are a popular solution for creating simple, fast, flexible, and attractive websites that are easy to update. You can contribute new pages and content to a static site in two steps:

      1. First, write the content for your site’s new page using Markdown, an easy-to-learn and light-weight markup language.

      2. Then, tell your static site generator to compile your Markdown (along with other relevant assets, like CSS styling, images, and JavaScript) into static HTML files.

      The second compilation step only needs to happen once for each time that you update your content. This is in contrast with a dynamic website framework like WordPress or Drupal, which will reference a relational database and compile your HTML every time a visitor loads your site.

      Benefits of Hosting on Object Storage

      Traditionally, these static HTML files would be served by a web server (like NGINX or Apache) running on a Linode. Using Object Storage to host your static site files means you do not have to worry about maintaining your site’s infrastructure. It is no longer necessary to perform typical server maintenance tasks, like software upgrades, web server configuration, and security upkeep.

      Object Storage provides an HTTP REST gateway to objects, which means a unique URL over HTTP is available for every object. Once your static site is built, making it available publicly over the Internet is as easy uploading files to an Object Storage bucket.

      Object Storage Hosting Workflow

      At a high-level, the required steps to host a static site using Object Storage are:

      1. Install the static site generator of your choice to your local computer.

      2. Create the desired content and build the site (using your static site generator).

      3. Upload the static files to your Object Storage bucket to make the content publicly available over the Internet.

      This guide will use Hugo to demonstrate how to create a static site and host it on Linode Object Storage. However, there are many other static site generators to choose from–Jekyll and Gatsby are popular choices, and the general steps outlined in this guide could be adapted to them. For more information on choosing a static site generator, see the How to Choose a Static Site Generator guide.

      Before You Begin

      1. Read the How to Use Linode Object Storage guide to familiarize yourself with Object Storage on Linode. Specifically, be sure that you have:

        • Created your Object Storage access and secret keys.
        • Installed and configure the s3cmd tool.
      2. Install and configure Git on your local computer.

      Install the Hugo Static Site Generator

      Hugo is written in Go and is known for being extremely fast to compile sites, even very large ones. It is well-supported, well-documented, and has an active community. Some useful Hugo features include shortcodes, which are an easy way to include predefined templates inside of your Markdown, and built-in LiveReload web server, which allows you to preview your site changes locally as you make them.

      1. Install Hugo on your computer:

        macOS:

        Linux/Ubuntu:

        • Determine your Linux kernel’s architecture:

          uname -r
          

          Your output will resemble the following:

            
          4.9.0-8-amd64
          
          
        • Navigate to Hugo’s GitHub releases page and download the appropriate version for your platform. This example command downloads version 0.55, but a newer release may be available:

          wget https://github.com/gohugoio/hugo/releases/download/v0.55.0/hugo_0.55.0_Linux-64bit.deb
          
        • Install the package using dpkg:

          sudo dpkg -i hugo*.deb
          
      2. Verify that Hugo is installed. You should see output indicating your installed Hugo’s version number:

        hugo version
        

      Create a Hugo Site

      In this section, you will use the Hugo CLI (command line interface) to create your Hugo site, initialize a Hugo theme, and add content to your site. Hugo’s CLI provides several useful commands for common tasks needed to build, configure, and interact with your Hugo site.

      1. Create a new Hugo site on your local computer. This command will create a folder named example-site and scaffold Hugo’s directory structure inside it:

        hugo new site example-site
        
      2. Move into your Hugo site’s root directory:

        cd example-site
        

        Note

        All commands in this section of the guide should be issued from your site’s root directory.

      3. You will use Git to add a theme to your Hugo site’s directory. Initialize your Hugo site’s directory as a Git repository:

        git init
        
      4. Install the Ananke theme as a submodule of your Hugo site’s Git repository. Git submodules allow one Git repository to be stored as a subdirectory of another Git repository, while still being able to maintain each repository’s version control information separately. The Ananke theme’s repository will be located in the ~/example-site/themes/ananke directory of your Hugo site.

        git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
        

        Note

        Hugo has many available themes that can be installed as a submodule of your Hugo site’s directory.
      5. Add the theme to your Hugo site’s configuration file. The configuration file (config.toml) is located at the root of your Hugo site’s directory.

        echo 'theme = "ananke"' >> config.toml
        
      6. Create a new content file for your site. This command will generate a Markdown file with an auto-populated date and title:

        hugo new posts/my-first-post.md
        
      7. You should see a similar output. Note that the file is located in the content/posts/ directory of your Hugo site:

          
        /home/username/example-site/content/posts/my-first-post.md created
        
        
      8. Open the Markdown file in the text editor of your choice to begin modifying its content; you can copy and paste the example snippet into your file, which contains an updated front matter section at the top and some example Markdown body text.

        Set your desired value for title. Then, set the draft state to false and add your content below the --- in Markdown syntax, if desired:

        /home/username/example-site/content/posts/my-first-post.md
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        
        ---
        title: "My First Post"
        date: 2019-04-11T11:25:11-04:00
        draft: false
        ---
        
        # Host a Static Site on Linode Object Storage
        
        There are many benefits to using a static site generator. Here is a list of a few of them:
        
        - Run your own website without having to manage a Linode.
        - You don't need to worry about running a web server like Apache or NGINX.
        - Static website performance is typically very fast.
        - Use Git to version control your static website's content.


        About front matter

        Front matter is a collection of metadata about your content, and it is embedded at the top of your file within opening and closing --- delimiters.

        Front matter is a powerful Hugo feature that provides a mechanism for passing data that is attached to a specific piece of content to Hugo’s rendering engine. Hugo accepts front matter in TOML, YAML, and JSON formats. In the example snippet, there is YAML front matter for the title, date, and draft state of the Markdown file. These variables will be referenced and displayed by your Hugo theme.

      9. Once you have added your content, you can preview your changes by building and serving the site using Hugo’s built-in webserver:

        hugo server
        
      10. You will see a similar output:

          
        &nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp| EN
        +------------------+----+
          Pages&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp| 11
          Paginator pages&nbsp&nbsp&nbsp&nbsp|  0
          Non-page files&nbsp&nbsp&nbsp&nbsp&nbsp|  0
          Static files&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  3
          Processed images&nbsp&nbsp&nbsp|  0
          Aliases&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  1
          Sitemaps&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  1
          Cleaned&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  0
        
        Total in 7 ms
        Watching for changes in /home/username/example-site/{content,data,layouts,static,themes}
        Watching for config changes in /home/username/example-site/config.toml
        Serving pages from memory
        Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
        Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
        Press Ctrl+C to stop
        
        
      11. The output will provide a URL to preview your site. Copy and paste the URL into a browser to access the site. In the above example Hugo’s web server URL is http://localhost:1313/.

      12. When you are happy with your site’s content you can build your site:

        hugo -v
        

        Hugo will generate your site’s static HTML files and store them in a public directory that it will create inside your project. The static files that are generated by Hugo are the files that you will upload to your Object Storage bucket to make your site accessible via the Internet.

      13. View the contents of your site’s public directory:

        ls public
        

        Your output should resemble the following example. When you built the site, the Markdown file you created and edited in steps 6 and 7 was used to generate its corresponding static HTML file in the public/posts/my-first-post/index.html directory.

          
          404.html    categories  dist        images      index.html  index.xml   posts       sitemap.xml tags
            
        


        Track your Static Site Files with Git

        It’s not necessary to version control your site files in order to host them on Object Storage, but we still recommended that you do so:

        1. Display the state of your current working directory (root of your Hugo site):

          git status
          
        2. Stage all your files to be committed:

          git add -A
          
        3. Commit all your changes and add a meaningful commit message:

          git commit -m 'Add my first post.'
          

        Once you have used Git to track your local Hugo site files, you can easily push them to a remote Git repository, like GitHub or GitLab. Storing your static site files on a remote Git repository opens up many possibilities for collaboration and automating your static site’s deployment to Linode Object Storage. To learn more about Git, see the Getting Started with Git guide.

      Upload your Static Site to Linode Object Storage

      Before proceeding with this section ensure that you have already created your Object Storage access and secret keys and have installed the s3cmd tool.

      1. Create a new Object Storage bucket; prepend s3:// to the beginning of the bucket’s name:

        s3cmd mb s3://my-bucket
        

        Note

        Buckets names must be unique within the Object Storage cluster. You might find the bucket name my-bucket is already in use by another Linode customer, in which case you will need to choose a new bucket name.

      2. Initialize your Object Storage bucket as a website. You must tell your bucket which files to serve as the index page and the error page for your static site. This is done with the --ws-index and --ws-error options:

        s3cmd ws-create --ws-index=index.html --ws-error=404.html s3://my-bucket
        

        In our Hugo example, the site’s index file is index.html and the error file is 404.html. Whenever a user visits your static site’s URL, the Object Storage service will serve the index.html page. If a site visitor tries to access an invalid path, they will be presented with the 404.html page.

      3. The command will return the following message:

          
            Bucket 's3://my-bucket/': website configuration created.
              
        
      4. Display information about your Object Storage’s website configuration to obtain your site’s URL:

        s3cmd ws-info s3://my-bucket
        
      5. You should see a similar output. Be sure to take note of your Object Storage bucket’s URL:

          
              Bucket s3://my-bucket/: Website configuration
        Website endpoint: http://website-us-east-1.linodeobjects.com/
        Index document:   index.html
        Error document:   404.html
            
        

        Note

        The Linode Object Storage early access Beta provides SSL enabled by default. This means you can access your Object Storage bucket using https, as well.

      6. Use s3cmd’s sync command to upload the contents of your static site’s public directory to your Object Storage bucket. This step will make your site available publicly on the Internet. Ensure you are in your site’s root directory on your computer (e.g. /home/username/example-site):

        s3cmd --no-mime-magic --acl-public --delete-removed --delete-after sync public/ s3://my-bucket
        
        Option                          Description
        no-mime-magic Tells Object Storage not to use file signatures when guessing the object’s MIME-type.
        acl-public Sets the access level control of the objects to public.
        delete-removed Deletes any destination objects with no corresponding source file.
        delete-after Deletes destination files that are no longer found at the source after all files are uploaded to the bucket.
      7. Use a browser to navigate to your Object Storage bucket’s URL to view your Hugo site:

        Hugo Index Page

        Note

        It may take a minute or two after your s3cmd sync completes for the page to appear at your bucket’s website URL.

      8. If needed, you can continue to update your static site locally and upload any changes using s3cmd’s sync command from step 3 of this section.

      (Optional) Next Steps

      After uploading your static site to Linode Object Storage, you may want to use a custom domain for your site. To do this, you can add a CNAME entry to your domain’s DNS records that aliases it to your Object Storage bucket’s website URL. To learn about managing DNS records on Linode, see the DNS Manager and DNS Records: An Introduction guides.

      As noted before, it’s possible to trigger automatic deployments to the Object Storage service when you push new content updates to GitHub or GitLab. This is done by leveraging a CI/CD (continuous integration/continuous delivery) tool like Travis CI. Essentially, you would build your Hugo site within the Travis environment and then run the s3cmd sync command from it to your bucket.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      How to Use Linode Object Storage


      Updated by Linode

      Contributed by

      Linode

      Note

      Object Storage is currently in a closed early access Beta, and you may not have access to Object Storage through the Cloud Manager or other tools. To gain access to the Early Access Program (EAP), open up a Customer Support ticket noting that you’d like to be included in the program, or e-mail objbeta@linode.com – beta access is completely free.

      Additionally, because Object Storage is in Beta, there may be breaking changes to how you access and manage Object Storage. This guide will be updated to reflect these changes if and when they occur.

      Linode’s Object Storage is a globally-available, S3- and Swift-compatible method for storing and accessing data. Object Storage differs from traditional hierarchical data storage (as in a Linode’s disk) and Block Storage Volumes. Under Object Storage, files (also called objects) are stored in flat data structures (referred to as buckets) alongside their own rich metadata.

      Additionally, Object Storage does not require the use of a Linode. Instead, Object Storage gives each object a unique URL with which you can access your data. An object can be publicly accessible, or you can set it to be private and only visible to you. This makes Object Storage great for sharing and storing unstructured data like images, documents, archives, streaming media assets, and file backups, and the amount of data you store can range from small collections of files up to massive libraries of information. Lastly, Linode Object Storage has the built-in ability to host a static site.

      Below you will find instructions on how to connect to Object Storage, and how to upload and access objects:

      1. First, you’ll need to create a key pair to access the service.

      2. Then, you’ll use choose from a variety of available first-party and third-party tools to access and use the service.

      Object Storage Key Pair

      The first step towards using Object Storage is to create a pair of keys for the service. This pair is composed of an access key and a secret key:

      • The access key allows you to access any objects that you set to have private read permissions.

        Note

        To use your access key when viewing a private object, you first need to generate a signed URL for the object. The signed URL is much like the standard URL for your object, but some extra URL parameters are appended to it, including the access key. Instructions for generating a signed URL can be found for each of the tools outlined in this guide.

      • Your secret key is used together with your access key to authenticate the various Object Storage tools with your Linode account. You should not share the secret key.

        Note

        Each Object Storage key pair on your Linode account has complete access to all of the buckets on your account.

      Generate a Key Pair

      1. Log in to the Linode Cloud Manager.

        Note

        Object Storage is not available in the Linode Classic Manager.

      2. Click on the Object Storage link in the sidebar, click the Access Keys tab, and then click the Create an Access Key link.

        Click on the 'Access Keys' tab.

      3. The Create an Access Key menu will appear.

        The 'Create an Access Key' menu.

      4. Enter a label for the key pair. This label will be how you reference your key pair in the Linode Cloud Manager. Then, click Submit.

      5. A window will appear that contains your access key and your secret key. Write these down somewhere secure. The access key will be visible in the Linode Cloud Manager, but you will not be able to retrieve your secret key again once you close the window.

        Your access key and secret key.

        You now have the credentials needed to connect to Linode Object Storage.

      There are a number of tools that are available to help manage Linode Object Storage. This guide explains how to install and use the following options:

      • The Linode Cloud Manager can be used to create buckets (you are currently not able to upload objects to a bucket from the Cloud Manager).

      • s3cmd is a powerful command line utility that can be used with any S3-compatible object storage service, including Linode’s. s3cmd can be used to create and remove buckets, add and remove objects, convert a bucket into a static site from the command line, plus other functions like syncing entire directories up to a bucket.

      • Cyberduck is a graphical utility available for Windows and macOS and is a great option if you prefer a GUI tool.

      Cloud Manager

      Create a Bucket

      The Cloud Manager provides a web interface for creating buckets. To create a bucket:

      1. If you have not already, log in the Linode Cloud Manager.

      2. Click on the Object Storage link in the sidebar, and then click on Add a Bucket.

        The Object Storage menu.

      3. The Create a Bucket menu will appear.

        The Create a Bucket menu.

      4. Add a label for your bucket. A bucket’s label needs to be unique within the cluster that it lives in, and this includes buckets of the same name on different Linode accounts. If the label you enter is already in use, you will have to choose a different label.

      5. Choose a cluster location for the bucket to reside in.

      6. Click Submit. You are now ready to upload objects to your bucket using one of the other tools outlined in this guide.

      s3cmd

      s3cmd is a command line utility that you can use for any S3-compatible Object Storage.

      Install and Configure s3cmd

      1. s3cmd can be downloaded using apt on Debian and Ubuntu, and Homebrew on macOS. To download s3cmd using Homebrew, run the following command:

        brew install s3cmd
        

        Note

        On macOS, s3cmd might fail to install if you do not have XCode command line tools installed. If that is the case, run the following command:

        xcode-select --install
        

        You will be prompted to agree to the terms and conditions.

        To install s3cmd on Debian or Ubuntu, run the following command:

        apt install s3cmd
        
      2. Once s3cmd has been installed, you will need to configure it:

        s3cmd --configure
        

        You will be presented with a number of questions. To accept the default answer that appears within the brackets, press enter. Here is an example of the answers you will need to provide:

        Access Key: 4TQ5CJGZS92LLEQHLXB3
        Secret Key: enteryoursecretkeyhere
        Default Region: US
        S3 Endpoint: us-east-1.linodeobjects.com
        DNS-style bucket+hostname:port template for accessing a bucket: us-east-1.linodeobjects.com
        Encryption password: YOUR_GPG_KEY
        Path to GPG program: /usr/local/bin/gpg
        Use HTTPS protocol: False
        HTTP Proxy server name:
        HTTP Proxy server port: 0
        

        Note

        It is not necessary to supply a GPG key when configuring s3cmd, though it will allow you to store and retrieve encrypted files. If you do not wish to configure GPG encryption, you can leave the Encryption password and Path to GPG program fields blank.

      3. When you are done, enter Y to save your configuration.

        Note

        s3cmd offers a number of additional configuration options that are not presented as prompts by the s3cmd --configure command. One of those options is website_endpoint, which instructs s3cmd on how to construct an appropriate URL for a bucket that is hosting a static site, similar to the S3 Endpoint in the above configuration. This step is optional, but will ensure that any commands that contain your static site’s URL will output the right text. To edit this configuration file, open the ~/s3.cfg file on your local computer:

        nano ~/.s3cfg
        

        Scroll down until you find the website_endpoint, then add the following value:

        http://%(bucket)s.website-us-east-1.linodeobjects.com/
        

      You are now ready to use s3cmd to create a bucket in Object Storage.

      Create a Bucket with s3cmd

      You can create a bucket with s3cmd issuing the following mb command, replacing my-example-bucket with the name of the bucket you would like to create. Bucket names need to be unique within the same cluster, including buckets on other Linode accounts. If you choose a name for your bucket that someone else has already created, you will have to choose a different name:

      s3cmd mb s3://my-example-bucket
      

      To remove a bucket, you can use the rb command:

      s3cmd rb s3://my-example-bucket
      

      Caution

      To delete a bucket that has files in it, include the --recursive (or -r) option and the --force (or -f) option. Use caution when using this command:

      s3cmd rb -r -f s3://my-example-bucket/
      

      Upload, Download, and Delete an Object with s3cmd

      1. As an example object, create a text file and fill it with some example text.

        echo 'Hello World!' > example.txt
        
      2. Now, transfer the text file object to your bucket using s3cmd’s put command, replacing my-example-bucket with the label of the bucket you gave in the last section:

        s3cmd put example.txt s3://my-example-bucket -P
        

        Note

        The -P flag at the end of the command instructs s3cmd to make the object public. To make the object private, which means you will only be able to access it from a tool such as s3cmd, simply leave the ‘-P’ flag out of the command.

        Note

        If you chose to enable encryption when configuring s3cmd, you can store encrypted objects by supplying the -e flag:

        s3cmd put -e encrypted_example.txt s3://my-example-bucket
        
      3. The object will be uploaded to your bucket, and s3cmd will provide a public URL for the object:

        upload: 'example.txt' -> 's3://my-example-bucket/example.txt'  [1 of 1]
        13 of 13   100% in    0s   485.49 B/s  done
        Public URL of the object is: http://us-east-1.linodeobjects.com/my-example-bucket/example.txt
        

        Note

        The URL for the object that s3cmd provides is one of two valid ways to access your object. The first, which s3cmd provides, places the name of your bucket after the domain name. You can also access your object by affixing your bucket name as a subdomain: http://my-example-bucket.us-east-1.linodeobjects.com/example.txt. The latter URL is generally favored.

      4. To retrieve a file, issue the get command:

        s3cmd get s3://my-example-bucket/example.txt
        

        If the file you are attempting to retrieve is encrypted, you can retrieve it using the -e flag:

        s3cmd get -e s3://my-example-bucket/encrypted_example.txt
        
      5. To delete a file, you can issue the rm command:

         s3cmd rm s3://my-example-bucket/example.txt
        

        Caution

        To delete all files in a bucket, include the --recursive (or -r) option and the --force (or -f) option. Use caution when using this command:

        s3cmd rm -r -f s3://my-example-bucket/
        
      6. To list all available buckets, issue the ls command:

        s3cmd ls
        
      7. To list all objects in a bucket, issue the ls command and supply a bucket:

        s3cmd ls s3://my-example-bucket
        

      Create a Static Site with s3cmd

      You can also create a static website using Object Storage and s3cmd:

      1. To create a website from a bucket, issue the ws-create command:

        s3cmd ws-create --ws-index=index.html --ws-error=404.html s3://my-example-bucket
        

        The --ws-index and --ws-error flags specify which objects the bucket should use to serve the static site’s index page and error page, respectively.

      2. You will need to separately upload the index.html and 404.html files (or however you have named the index and error pages) to your bucket:

        echo 'Index page' > index.html
        echo 'Error page' > 404.html
        s3cmd put index.html 404.html s3://my-example-bucket
        
      3. Your static site is accessed from a different URL than the generic URL for your Object Storage bucket. Static sites are available at the website-us-east-1 subdomain. Using my-example-bucket as an example, you would navigate to http://my-example-bucket.website-us-east-1.linodeobjects.com.

      For more information on hosting a static website with Object Storage, read our Host a Static Site using Linode Object Storage guide.

      Other s3cmd Commands

      To upload an entire directory of files, you can use the the sync command, which will automatically sync all new or changed files. Navigate to the directory you would like to sync, then enter the following:

      s3cmd sync . s3://my-example-bucket -P
      

      This can be useful for uploading the contents of a static site to your bucket.

      Note

      The period in the above command instructs s3cmd to upload the current directory. If you do not want to first navigate to the directory you wish to upload, you can supply a path to the directory instead of the period.

      Cyberduck

      Cyberduck is a desktop application that facilitates file transfer over FTP, SFTP, and a number of other protocols, including S3.

      Install and Configure Cyberduck

      1. Download Cyberduck by visiting their website.

      2. Once you have Cyberduck installed, open the program and click on Open Connection.

      3. At the top of the Open Connection dialog, select Amazon S3 from the dropdown menu.

        Open Cyberduck and click on 'Open Connection' to open the connection menu.

      4. For the Server address, enter us-east-1.linodeobjects.com.

      5. Enter your access key in the Access Key ID field, and your secret key in the Secret Access Key field.

      6. Click Connect.

      You are now ready to create a bucket in Object Storage.

      Create a Bucket with Cyberduck

      To create a bucket in Cyberduck:

      1. Right click within the window frame, or click Action, then click New Folder:

        Right click or click 'Action', then click 'New Folder'

      2. Enter your bucket’s name and then click Create. Bucket names need to be unique within the same cluster, including buckets on other Linode accounts. If the name of your bucket is already in use, you will have to choose a different name.

      To delete the bucket using Cyberduck, right click on the bucket and select Delete.

      Upload, Download, and Delete an Object with Cyberduck

      1. To upload objects with Cyberduck, you can simply drag and drop the object, or directory of objects, to the bucket you’d like to upload them to, and Cyberduck will do the rest. Alternatively, you can click on the Action button and select Upload from the menu:

        Click on the 'Action' button to use the file upload dialog.

      2. To make your objects publicly accessible, meaning that you can access them from the object’s URL, you need to set the proper READ permissions. Right click on the object and select Info.

      3. Click on the Permissions tab.

      4. Click the gear icon at the bottom of the window and select Everyone.

        Open the file permissions prompt by right clicking on the file and selecting.

      5. A new entry for Everyone will appear in the Access Control List. Next to Everyone, under Permissions column heading, select READ from the drop down menu.

        Set the permissions for 'Everyone' to READ.

        Your object is now accessible via the internet, at the URL http://my-example-bucket.us-east-1.linodeobjects.com/example.txt, where my-example-bucket is the name of your bucket, and example.txt is the name of your object.

      6. To download an object, right click on the object and select Download, or click Download As if you’d like to specify the location of the download.

      7. To delete an object, right click the object name and select Delete.

      Create a Static Site with Cyberduck

      To create a static site from your bucket:

      1. Select a bucket, then right click on that bucket or select the Action button at the top of the menu.

      2. Click on Info, and then select the Distribution (CDN) tab.

      3. Check the box that reads Enable Website Configuration (HTTP) Distribution:

        Check the box labeled 'Enable Website Configuration (HTTP) Distribution'

      4. You will need to separately upload the index.html and 404.html files (or however you have named the index and error pages) to your bucket. Follow the instructions from the Upload, Download, and Delete an Object with Cyberduck section to upload these files.

      5. Your static site is accessed from a different URL than the generic URL for your Object Storage bucket. Static sites are available at the website-us-east-1 subdomain. Using my-example-bucket as an example, you would navigate to http://my-example-bucket.website-us-east-1.linodeobjects.com.

        For more information on hosting a static website with Object Storage, read our Host a Static Site using Linode Object Storage guide.

      Next Steps

      There are S3 bindings available for a number of programming languages, including the popular Boto library for Python, that allow you to interact with Object Storage programmatically.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Deploy Persistent Volume Claims with the Linode Block Storage CSI Driver


      Updated by Linode Written by Linode Community

      What is the Linode Block Storage CSI Driver?

      The Container Storage Interface (CSI) defines a standard that storage providers can use to expose block and file storage systems to container orchestration systems. Linode’s Block Storage CSI driver follows this specification to allow container orchestration systems, like Kubernetes, to use Block Storage Volumes to persist data despite a Pod’s lifecycle. A Block Storage Volume can be attached to any Linode to provide additional storage.

      Before You Begin

      • This guide assumes you have a working Kubernetes cluster running on Linode. You can deploy a Kubernetes cluster on Linode in the following ways:

        1. Use Linode’s k8s-alpha CLI to deploy a Kubernetes cluster via the command line.

        2. Deploy a cluster using Terraform and the Linode Kubernetes Terraform installer.

        3. Use kubeadm to manually deploy a Kubernetes cluster on Linode. You can follow the Getting Started with Kubernetes: Use kubeadm to Deploy a Cluster on Linode guide to do this.

        Note

        • If using the k8s-alpha CLI or the Linode Kubernetes Terraform installer methods to deploy a cluster, you can skip the Installing the CSI Driver section of this guide, since it will be automatically installed when you deploy a cluster.

          Move on to the Attach a Pod to the Persistent Volume Claim section to learn how to consume a Block Storage volume as part of your deployment.

      • The Block Storage CSI supports Kubernetes version 1.13 or higher. To check the version of Kubernetes you are running, you can issue the following command:

        kubectl version
        

      Installing the CSI Driver

      Create a Kubernetes Secret

      A secret in Kubernetes is any token, password, or credential that you want Kubernetes to store for you. In the case of the Block Storage CSI, you’ll want to store an API token, and for convenience, the region you would like your Block Storage Volume to be placed in.

      Note

      Your Block Storage Volume must be in the same data center as your Kubernetes cluster.

      To create an API token:

      1. Log into the Linode Cloud Manager.

      2. Navigate to your account profile by clicking on your username at the top of the page and selecting My Profile. On mobile screen resolutions, this link is in the sidebar navigation.

      3. Click on the API Tokens tab.

      4. Click on Add a Personal Access Token. The Add Personal Access Token menu appears.

      5. Provide a label for the token. This is how you will reference your token within the Cloud Manager.

      6. Set an expiration date for the token with the Expiry dropdown.

      7. Set your permissions for the token. You will need Read/Write access for Volumes, and Read/Write access for Linodes.

      8. Click Submit.

      Your access token will appear on the screen. Copy this down somewhere safe, as once you click OK you will not be able to retrieve the token again, and will need to create a new one.

      Once you have your API token, it’s time to create your secret.

      1. Run the following command to enter your token into memory:

        read -s -p "Linode API Access Token: " LINODE_TOKEN
        

        Press enter, and then paste in your API token.

      2. Run the following command to enter your region into memory:

        read -p "Linode Region of Cluster: " LINODE_REGION
        

        You can retrieve a full list of regions by using the Linode CLI:

        linode-cli regions list
        

        For example, if you want to use the Newark, NJ, USA data center, you would use us-east as your region.

      3. Create the secret by piping in the following secret manifest to the kubectl create command. Issue the following here document:

        cat <<EOF | kubectl create -f -
        
      4. Now, paste in the following manifest and press enter:

        apiVersion: v1
        kind: Secret
        metadata:
          name: linode
          namespace: kube-system
        stringData:
          token: "$LINODE_TOKEN"
          region: "$LINODE_REGION"
        EOF
        

      You can check to see if the command was successful by running the get secrets command in the kube-system namespaces and looking for linode in the NAME column of the output:

      kubectl -n kube-system get secrets
      

      You should see output similar to the following:

      NAME                                             TYPE                                  DATA   AGE
      ...
      job-controller-token-6zzkw                       kubernetes.io/service-account-token   3      43h
      kube-proxy-token-td7k8                           kubernetes.io/service-account-token   3      43h
      linode                                           Opaque                                2      42h
      ...
      

      You are now ready to install the Block Storage CSI driver.

      Apply CSI Driver to your Cluster

      To install the Block Storage CSI driver, use the apply command and specify the following URL:

      kubectl apply -f https://raw.githubusercontent.com/linode/linode-blockstorage-csi-driver/master/pkg/linode-bs/deploy/releases/linode-blockstorage-csi-driver-v0.0.3.yaml
      

      The above file concatenates a few files needed to run the Block Storage CSI driver, including the volume attachment, driver registration, and provisioning sidecars. To see these files individually, visit the project’s GitHub repository.

      Once you have the Block Storage CSI driver installed, you are ready to provision a Persistent Volume Claim.

      Create a Persistent Volume Claim

      Caution

      The instructions in this section will create a Block Storage volume billable resource on your Linode account. A single volume can range from 10 GiB to 10,000 GiB in size and costs $0.10/GiB per month or $0.00015/GiB per hour. If you do not want to keep using the Block Storage volume that you create, be sure to delete it when you have finished the guide.

      If you remove the resources afterward, you will only be billed for the hour(s) that the resources were present on your account. Consult the Billing and Payments guide for detailed information about how hourly billing works and for a table of plan pricing.

      A Persistent Volume Claim (PVC) consumes a Block Storage Volume. To create a PVC, create a manifest file with the following YAML:

      pvc.yaml
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: pvc-example
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: linode-block-storage

      This PVC represents a Block Storage Volume. Because Block Storage Volumes have a minimum size of 10 gigabytes, the storage has been set to 10Gi. If you choose a size smaller than 10 gigabytes, the PVC will default to 10 gigabytes.

      Currently the only mode supported by the Linode Block Storage CSI driver is ReadWriteOnce, meaning that it can only be connected to one Kubernetes node at a time.

      To create the PVC in Kubernetes, issue the create command and pass in the pvc.yaml file:

      kubectl create -f pvc.yaml
      

      After a few moments your Block Storage Volume will be provisioned and your Persistent Volume Claim will be ready to use.

      You can check the status of your PVC by issuing the following command:

      kubectl get pvc
      

      You should see output like the following:

      NAME          STATUS   VOLUME                 CAPACITY   ACCESS MODES   STORAGECLASS           AGE
      pvc-example   Bound    pvc-0e95b811652111e9   10Gi       RWO            linode-block-storage   2m
      

      Now that you have a PVC, you can attach it to a Pod.

      Attach a Pod to the Persistent Volume Claim

      Now you need to instruct a Pod to use the Persistent Volume Claim. For this example, you will create a Pod that is running an ownCloud container, which will use the PVC.

      To create a pod that will use the PVC:

      1. Create a manifest file for the Pod and give it the following YAML:

        owncloud-pod.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        
        apiVersion: v1
        kind: Pod
        metadata:
          name: owncloud
          labels:
            app: owncloud
        spec:
          containers:
            - name: owncloud
              image: owncloud/server
              ports:
                - containerPort: 8080
              volumeMounts:
              - mountPath: "/mnt/data/files"
                name: pvc-example
          volumes:
            - name: pvc-example
              persistentVolumeClaim:
                claimName: pvc-example

        This Pod will run the owncloud/server Docker container image. Because ownCloud stores its files in the /mnt/data/files directory, this owncloud-pod.yaml manifest instructs the ownCloud container to create a mount point at that file path for your PVC.

        In the volumes section of the owncloud-pod.yaml, it is important to set the claimName to the exact name you’ve given your PersistentVolumeClaim in its manifest’s metadata. In this case, the name is pvc-example.

      2. Use the create command to create the ownCloud Pod:

        kubectl create -f owncloud-pod.yaml
        
      3. After a few moments your Pod should be up and running. To see the status of your Pod, issue the get pods command:

        kubectl get pods
        

        You should see output like the following:

        NAME       READY   STATUS    RESTARTS   AGE
        owncloud   1/1     Running   0          2m
        
      4. To list the contents of the /mnt/data/files directory within the container, which is the mount point for your PVC, issue the following command on your container:

        kubectl exec -it owncloud -- /bin/sh -c "ls /mnt/data/files"
        

        You should see output similar to the following:

        admin  avatars  files_external  index.html  owncloud.db  owncloud.log
        

        These files are created by ownCloud, and those files now live on your Block Storage Volume. The admin directory is the directory for the default user, and any files you upload to the admin account will appear in this folder.

      To complete the example, you should be able to access the ownCloud Pod via your browser. To accomplish this task, you will need to create a Service.

      1. Create a Service manifest file and copy in the following YAML:

        owncloud-service.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        
        kind: Service
        apiVersion: v1
        metadata:
          name: owncloud
        spec:
          selector:
            app: owncloud
          ports:
          - protocol: TCP
            port: 80
            targetPort: 8080
          type: NodePort

        Note

        The service manifest file will use the NodePort method to get external traffic to the ownCloud service. NodePort opens a specific port on all cluster Nodes and any traffic that is sent to this port is forwarded to the service. Kubernetes will choose the port to open on the nodes if you do not provide one in your service manifest file. It is recommended to let Kubernetes handle the assignment. Kubernetes will choose a port in the default range, 30000-32767.

        Alternatively, you could use the LoadBalancer service type, instead of NodePort, which will create Linode NodeBalancers that will direct traffic to the ownCloud Pods. Linode’s Cloud Controller Manager (CCM) is responsible for provisioning the Linode NodeBalancers. For more details, see the Kubernetes Cloud Controller Manager for Linode repository.

      2. Create the service in Kubernetes by using the create command and passing in the owncloud-service.yaml file you created in the previous step:

        kubectl create -f owncloud-service.yaml
        
      3. To retrieve the port that the ownCloud Pod is listening on, use the describe command on the newly created Service:

        kubectl describe service owncloud
        

        You should see output like the following:

        Name:                     owncloud
        Namespace:                default
        Labels:                   <none>
        Annotations:              <none>
        Selector:                 app=owncloud
        Type:                     NodePort
        IP:                       10.106.101.155
        Port:                     <unset>  80/TCP
        TargetPort:               8080/TCP
        NodePort:                 <unset>  30068/TCP
        Endpoints:                10.244.1.17:8080
        Session Affinity:         None
        External Traffic Policy:  Cluster
        Events:                   <none>
        

        Find the NodePort. In this example the port is 30068.

      4. Now you need to find out which Node your Pod is running on. Use the describe command on the Pod to find the IP address of the Node:

        kubectl describe pod owncloud
        

        You should see output like the following:

        Name:               owncloud
        Namespace:          default
        Priority:           0
        PriorityClassName:  <none>
        Node:               kube-node/192.0.2.155
        Start Time:         Mon, 22 Apr 2019 17:07:20 +0000
        Labels:             app=owncloud
        Annotations:        <none>
        Status:             Running
        IP:                 10.244.1.17
        

        The IP address of the Node in this example is 192.0.2.155. Your ownCloud Pod in this example would be accessible from http://192.9.2.155:30068.

      5. Navigate to the URL of the Node, including the NodePort you looked up in a previous step. You will be presented with the ownCloud log in page. You can log in with the username admin and the password admin.

      6. Upload a file. You will use this file to test the Persistent Volume Claim.

      7. The Persistent Storage Claim has been created and is using your Block Storage Volume. To prove this point, you can delete the ownCloud Pod and recreate it, and the Persistent Storage Claim will continue to house your data:

        kubectl delete pod owncloud
        
        kubectl create -f owncloud-pod.yaml
        

        Once the Pod has finished provisioning you can log back in to ownCloud and view the file you previously uploaded.

      You have successfully You have successfully created a Block Storage Volume tied to a Persistent Volume Claim and have mounted it with a container in a Pod.

      Delete a Persistent Volume Claim

      To delete the Block Storage volume created in this guide:

      1. First, delete the ownCloud Pod:

        kubectl delete pods owncloud
        
      2. Then, delete the persistent volume claim:

        kubectl delete pvc pvc-example
        

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link