One place for hosting & domains

      Linode

      Use Cases for Linode Object Storage


      Updated by Linode

      Contributed by

      Linode

      What is Object Storage?

      Object Storage is a method of storing data that differs in a number of ways from Block Storage. Block Storage splits files into small blocks of data. Minimal file metadata is stored alongside this data and, in general, descriptive metadata must be stored in a separate file or database. In order to use a Block Storage volume it must be attached to a host server, where it acts like a hard drive.

      In contrast, Object Storage stores data, called objects, in containers, called buckets, and each object is given a unique identifier with which it is accessed. In this way, the physical location of the object does not need to be known. The objects are stored alongside rich, configurable metadata that can be used to describe any number of arbitrary properties about the object. Each object has its own URL, so accessing the data is often as simple as issuing an HTTP request, either by visiting the object in a browser or retrieving it through the command line.

      Benefits and Limitations

      Object Storage scales easily because all the objects are stored in a flat, scalable name space. Object Storage does not require a host server in order to be used, meaning many different clients can read from it or write to it.

      With that said, there are limitations to Object Storage. Objects in Object Storage cannot be modified at the block level, as with Block Storage, and must be rewritten in their entirety every time a change is made. This makes any scenario with many successive read/write operations – such as the needs of databases or transactional data – a poor choice for Object Storage. Additionally, Object Storage traffic runs over HTTP, so it does not benefit from the I/O speeds of a mounted Block Storage volume. As a rule of thumb, Object Storage shines when files do not need to be updated frequently.

      Below are some of the more popular use cases for Object Storage.

      Use Cases

      Static Site Hosting

      Because Object Storage buckets provide HTTP access to objects, it’s easy to set up a bucket to serve static websites. A static website is a website that does not require a server-side processing language like PHP to render content. And because a static site does not require each page to be processed with every request, they are usually very quick to load. For more information on setting up a static site on Object Storage, read our Host a Static Site on Linode Object Storage guide. For more on static site generators, visit our How to Choose a Static Site Generator guide.

      Website Files

      If you don’t want to host your entire site on Object Storage (for example: you plan to use a CMS like WordPress), you can still choose to host some of your site’s assets, like images and downloads, with Object Storage. This will save disk space on your server and can help reduce your costs.

      Software Storage and Downloads

      Similar to hosting website files, hosting software applications on Object Storage is a great use case for developers looking to give quick access to their products. Simply upload the file to a bucket and share its URL.

      Unstructured Data

      Unstructured data is any data that does not fit into a traditional database. Object Storage excels at storing large amounts of unstructured data. With the ability to configure custom metadata for each piece of unstructured data, it is easy to extrapolate useful information from each object and to retrieve objects with similar metadata. Examples of unstructured data include images, video, audio, documents, and Big Data.

      Images, Video, Audio, and Documents

      Multimedia assets like images, videos, audio files, and documents are a perfect match for Object Storage. In general these types of files do not change frequently, so there is no need to store them on Block Storage volumes. Because each file has its own URL, streaming the content of these files or embedding them in another program or website is simple and does not require the use of a server.

      Big Data

      Big Data typically describes data sets that are so large and so diverse that it takes specialized tooling to analyze them. In many cases the data that comprises Big Data is considered unstructured and does not fit neatly into a database, making it a great candidate for Object Storage.

      Artifact Storage

      As more and more of the development life cycle becomes automated and tested, more and more artifacts are generated in the process. Object Storage is a great solution for developers looking to store these artifacts, such as the bulk collection of logs. Sharing stored artifacts is as simple as sharing a URL. And if you’d rather your artifacts stay private, you can distribute an access key.

      Cold Storage

      Object Storage is, in the majority of cases, significantly cheaper than Block Storage. While Object Storage can incur a cost when retrieving data, the cost benefit for infrequently accessed data can provide you with an overall cost reduction when compared to similar methods.

      Similarly, Object Storage has benefits over tape storage. Tape storage is frequently used for archival purposes, but the read times that come with tape storage are many times more than what you’ll find with Object Storage. Special considerations have to be made when transferring tape drive data, such as the ability to ship drives safely across long distances. With Object Storage, this data is available through HTTP from anywhere in the world.

      Note

      The outbound data transfer for Linode Object Storage is part of your Linode account’s total transfer pool, which will reduce or completely eliminate transfer costs for Object Storage if you are also running Linode instances. If you expend your allotted transfer pool, you will be billed at a rate of $0.02 per GB for outbound transfers.

      Backups

      Databases and other critical data can be backed up to Object Storage with little effort using a command line client for easier automation. Objects within Object Storage are normally replicated three times, providing resiliency should an error occur with the underlying hardware. Additionally, buckets can be versioned so you never lose access to older backups.

      Private File Storage

      Objects can be made private and only accessible with a key. By default, all new objects in a bucket are set to private, so they are inaccessible by normal HTTP requests (though it’s easy to set public permissions on objects if you’d like). This makes it easy to store secure data.

      Next Steps

      If you’re curious about how to use Object Storage, you can read our guide on How to Use Linode Object Storage for detailed instructions on creating buckets and uploading objects. Read our Host a Static Site using Linode Object Storage to get started with hosting your own site on Object Storage.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Linode Managed


      Updated by Linode Written by Linode

      Linode Managed is a 24/7 incident response service. This robust, multi-homed monitoring system distributes monitoring checks to ensure that your servers remain online and available at all times. Linode Managed can monitor any service or software stack reachable over TCP or HTTP. Once you add a service to Linode Managed, Linode will monitor it for connectivity, response, and total request time. This guide shows you how to start monitoring your services with Linode Managed.

      Note

      Linode Managed applies to all Linodes on an account and will be billed at a rate of $100 per month for each Linode. If this service is not needed for all Linodes, a second account can be created to separate Managed Linodes from non-Managed Linodes. If needed, Linodes can be transferred by opening up a Support ticket from both accounts and requesting the transfer.

      Getting Started

      To sign up for Linode Managed:

      1. Log into the Linode Cloud Manager and click the Managed link in the sidebar.

      2. Click the Upgrade to Managed button on this page, and a confirmation dialog will appear. This dialog describes the total cost of the service for your account.

      3. Click the Enable Managed button in this confirmation dialog to enable the service for your account.

      Initial Configuration Checklist

      After you’ve signed up for the service, you’ll next want to let Support know about the websites and services you run, how to log into them, and who to contact if needed.

      Note

      Your Managed service dashboard will be blank and the Linode Support Team won’t receive health-check notifications for your services until these steps are completed. Please contact Linode Support if you have any questions when working through this checklist.

      1. Verify that your servers are responding to ping (ICMP echo) requests.
      2. Install Linode’s public SSH key on all of your servers. This will be the primary method that Linode Support will use to log into your servers.
      3. Add any service-specific credentials you may have. For example, if you run a WordPress site that communicates with a MySQL database, you should provide the MySQL username and password if you would like us to troubleshoot it in the event of outage for your site.
      4. Specify contacts and contact groups. If Linode Support has any questions while troubleshooting your services, they will reach out to the contacts you provide.
      5. Add services to be monitored. Services can be websites, databases, or any other custom application. You can even add checks to make sure specific strings of text appear on your monitored website or in the response body of your service.

      Credentials

      To investigate any alerts received for your services, the Linode Support team will need access to your servers and the applications you run on them. There are two primary ways that they’ll authenticate with your servers and applications:

      • The Linode Support Team uses SSH to connect to your servers. Specifically, they use public key authentication for the connection. To support this, they generate a unique public/private keypair for your Managed service, and they ask that you upload the public key to your servers.

      • If you run any applications which require a password to administer, you can supply those credentials to us in the Managed dashboard. They’ll then refer to your list of credentials when troubleshooting.

      Adding the Public Key

      Linode generates and assigns a unique public/private keypair to your account’s Managed service. This keypair will be used by Linode Support to log into your server whenever an issue needs to be investigated. You can locate your Linode Managed public key through the following steps:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Navigate to the SSH Access tab.

      4. Linode’s public key is displayed under this tab, and a Copy to clipboard button will appear next to it. Click on this button to copy it.

        Linode Managed Public SSH Key

      5. After you’ve copied the key, you can install it on your server, either under your root user or under another user. If you install it under another user, be sure to also update your Linode’s Managed SSH settings to reflect that user.

      Installing as Root

      Installing the public SSH key for the root user is the easiest way to add Linode’s public key to your server. However, if your server’s SSH configuration doesn’t allow root login, you may want to skip to the next section to add the public key to another user’s account.

      To install Linode’s SSH key for the root user:

      1. Open a terminal window and log into your Linode via SSH.

      2. Log in as root:

        su
        

        Note

        If you followed the instructions in the Securing Your Server guide to disable root login via SSH, you will need to reenable that feature to install the public key for the root user. Follow these instructions to edit the sshd_config file and reenable root login via SSH.
      3. Open the authorized_keys file in a text editor (for example, nano):

        nano /root/.ssh/authorized_keys
        

        Note

        The /root/.ssh/ directory may not exist yet. If this is the case, you must create the directory using the following command before opening the authorized_keys file:

         mkdir /root/.ssh/
        
      4. Paste Linode’s public key in a new line in the file.

      5. Save the changes to the authorized_keys file and exit your text editor. In the nano editor, enter Control-X, and then Y to confirm.

      6. You have successfully added Linode’s public key for the root user. If you’d like, you can open a support ticket to have the Linode Support team confirm that they have access. Repeat this process on every Linode you want to monitor with Linode Managed.

      Installing as Another User

      You can also install Linode’s public SSH key for another non-root user. This allows you to disable SSH root login and still allow our support staff to log into your servers.

      To install Linode’s SSH key as a non-root user:

      1. Log into your Linode via SSH.

      2. If you haven’t already created a non-root user on your server, you should do so now. See Adding a New User for instructions.

        Important: providing sudo access

        After logging into your Linode as a non-root user, the Linode Support team will generally need sudo privileges to run troubleshooting commands, so you should make sure to give your user sudo privileges.

        The Adding a New User guide shows how to add your user to the sudo group (or wheel or admin group, depending on your distribution), which will grant this privilege. When your user is in this group, your system will ask for the user’s password whenever a sudo command is run. Because of this, you will also need to tell us your Linux user and password by following the Adding Service Credentials section.

        Alternatively, you can set up passwordless sudo for your user, which means that your user’s password won’t be requested by the system when running sudo commands. To do this:

        1. Open the sudoers file for editing (this command will open the Vim text editor):

          sudo visudo
          
        2. Type i to enter insert mode in Vim.

        3. In the User privilege specification section of the file, add the following line, replacing example_user with your user name:

          
          example_user ALL=(ALL) NOPASSWD: ALL
          
          
        4. Press Esc to exit insert mode.

        5. Make sure this user is not in any other groups listed in /etc/sudoers, as this may override the passwordless sudo setting.

        6. Type :wq and press Enter to save and quit the file.

      3. Open the authorized_keys file for your user in a text editor (for example, nano):

        nano /home/example_user/.ssh/authorized_keys
        

        Note

        The /home/example_user/.ssh/ directory may not exist yet. If so, create it with the following command before opening the authorized_keys file:

        mkdir -p /home/example_user/.ssh/
        
      4. Paste Linode’s public key in a new line in the file.

      5. Save the changes to the authorized_keys file and exit your text editor. In the nano editor, enter Control-X, and then Y to confirm.

      6. You have successfully added Linode’s public key for the root user. If you’d like, you can open a support ticket to have the Linode Support team confirm that they have access. Repeat this process on every Linode you want to monitor with Linode Managed.

      Specifying Linode SSH Settings

      You can customize Linode Managed’s SSH settings for each of your Linodes:

      • The Linux user that Support should attempt to log in as.
      • Which of your Linode’s IP addresses Support should connect via, if your Linode has more than one IP address.
      • The port that Support should connect to, if your SSH service is listening on a non-standard port.

      As well, you can disable Managed SSH access for one or more of your servers. This does not turn off the SSH service on your Linode or uninstall our public key from it. Instead the Linode Support Team’s client software will simply not attempt connections if you have disabled access.

      Note

      If you want us to investigate issues, it’s important to keep SSH access enabled, which is the default.

      To access these settings:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the SSH Access tab. Your Linodes and their Managed SSH settings will appear in the table below:

        Linode Managed SSH Settings

      4. To enable or disable Managed SSH access for a Linode, click on the more options ellipsis that corresponds to the Linode and then select the Enable or Disable option from the menu that appears.

      5. To change other Managed SSH settings for a Linode, click on the more options ellipsis that corresponds to the Linode and then select the Edit option. A form will appear with fields for each setting.

      Adding Service Credentials

      Many of the applications running on your servers can only be accessed with the appropriate username and password combination. To provide Linode’s staff with access to those applications, you should upload credentials for them to the Linode Cloud Manager. Once uploaded, you can also link credentials to specific Managed services to communicate which ones Linode Support should use when troubleshooting the service.

      Note

      All credentials are securely stored in our encrypted database.

      To add a credential:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the Credentials tab, then click on the Add Credentials link under this tab.

        Linode Managed Credentials

        Note

        The table that lists your uploaded credentials will feature a Last Decrypted column. This column will show the most recent date/time that a member of Linode Support viewed each credential.

      4. A form for your new credentials will appear with these fields:

        Field Name Description
        Credential Label A descriptive name for the credential that tells the Linode Support Team what it should be used for. For example, if you are entering the MySQL root password, you might label it as MySQL Root.
        Username (optional) The username to supply when authenticating with the application.
        Password/Passphrase The password or passphrase to supply when authenticating with the application.
      5. Fill out the form and click the Add button.

      Linode Managed allows you to specify contacts that will be contacted if an issue is detected with one of your services. Contacts are categorized in groups and can be linked to specific Managed services. You can create separate groups for the administrators responsible for your different services and systems. For example, you might add all of your database administrators to a group named DBAs and then link that group to your monitored MySQL service. When Linode Managed detects an issue with MySQL, all of your database administrators will be notified at the same time.

      To add a contact to Linode Managed:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the Contacts tab, then click on the Add a Contact link under this tab.

        Linode Managed Contacts

      4. A form for your new contact will appear. The form accepts a primary and secondary phone number, which Linode Support may call if they need additional information to troubleshoot an issue on your servers.

        In the Group field, enter a group name. As described at the beginning of this section, groups can hold multiple contacts. Ideally, you’ll combine all of the individuals responsible for a particular service or system into one group.

        Note

        A Managed service can only be linked to a contact group and not to individual contacts, so you should create at least one group, even if it only contains one contact.

      5. Fill out the form and click the Save button.

      Services

      Linode Managed monitors the services running on your Linodes. Setting up services is an essential step in the configuration process–Linode Managed can’t monitor anything until you add services to be monitored.

      Adding a New Service

      To add a new monitored service to Linode Managed:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the Monitors tab, then click on the Add a Monitor link under this tab.

        Linode Managed Monitored Services

      4. A form for your new monitored service will appear with these fields:

        Field Name Description
        Monitor Label A descriptive label for the service that will help the support team quickly identify and fix the service if it needs troubleshooting.
        Contact Group (optional) One of the contact groups that you’ve registered with Linode Managed. Linode will contact the members of this group if they need help fixing the service.
        Monitor Type Select URL to monitor a website or TCP Connection to monitor any other service running on your Linode.
        Response Timeout The time (in seconds) for Linode Managed’s requests to timeout if they do not receive a response.
        URL If you selected URL for the Monitor Type field, enter a URL for a webpage that you’d like to monitor. If you selected TCP Connection, enter the domain or IP address and, optionally, a port number (separated by a colon) in the TCP field.
        Response Body Match (optional) A string that Linode Managed will check for in the URL or TCP response. If not present, then an alert will be generated for the service.
        Instructions/Notes Any notes or additional information about this service. The more information Linode Support has about the service and how it’s configured, the more quickly they will be able to resolve any issues that may arise.
        Credentials (optional) Any credentials that may need to be used to troubleshoot the service. You can select and save more than one credential for a service.
      5. Fill out the form and click the Add button.

      6. Once added, the monitored service will initially appear as Pending. Linode Support will start monitoring the service in a few minutes–if the service is available, the dashboard will then indicate that the service’s status is Verified.

      Temporarily Disabling Service Monitoring

      Service monitoring can be temporarily disabled:

      Note

      Linode Managed continuously monitors your services for availability. If you are going to perform scheduled maintenance on a service that temporarily makes it unavailable, you will receive a support ticket from Linode when they are alerted of the service’s outage. You can disable monitoring for that service during the maintenance to avoid receiving these reports.

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the Monitors tab. A table which lists your monitored services will appear below the tab.

      4. Find the service that you want to temporarily disable and click on the corresponding more options ellipsis. Select the Disable option from the menu that appears.

        Disable Monitoring for a Service

      The service is now temporarily disabled. Linode Managed won’t resume monitoring this service until you reenable monitoring, as described in the next section.

      Reenabling Disabled Services

      To reenable monitoring for a service:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the Monitors tab. A table which lists your monitored services will appear below the tab.

      4. Find the service that you want to temporarily disable and click on the corresponding more options ellipsis. Select the Enable option from the menu that appears.

      Linode Managed is now monitoring the service again. The service will be checked in a few minutes–if it’s available, the dashboard will then indicate that the service’s status is Verified.

      Removing Services

      If you decide to remove or stop using a monitored service on your Linode, you should also remove the service from Linode Managed. For example, you should remove the Apache service from Linode Managed if you decide to start using Linode as a dedicated database server. To remove a monitored service:

      1. Log into the Linode Cloud Manager.

      2. Click the Managed link in the sidebar.

      3. Select the Monitors tab. A table which lists your monitored services will appear below the tab.

      4. Find the service that you want to remove and click on the corresponding more options ellipsis. Select the Delete option from the menu that appears.

      5. A confirmation dialog will appear. Click the Delete button on this dialog to confirm.

      Linode Managed will stop monitoring the service.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Use Cases for Linode Dedicated CPU Instances


      Updated by Linode

      Written by Ryan Syracuse

      Why Dedicated CPU

      Dedicated CPU Linodes offer a complement to CPU intensive tasks, and have the potential to significantly reduce issues that arise from shared cloud hosting environments. Normally, when creating a Linode via our standard plan, you are paying for access to virtualized CPU cores, which are allocated to you from a host’s shared physical CPU. While a standard plan is designed to maximize performance, the reality of a shared virtualized environment is that your processes are scheduled to use the same physical CPU cores as other customers. This can produce a level of competition that results in CPU steal, or a higher wait time from the underlying hypervisor to the physical CPU.

      CPU Steal can be defined more strictly as a measure of expected CPU cycles against actual CPU cycles as your virtualized environment is scheduled access to the physical CPU. Although this number is generally small enough that it does not heavily impact standard workloads and use cases, if you are expecting high and constant consumption of CPU resources, you are at risk of being negatively impacted by CPU Steal.

      Dedicated CPU Linodes have private access to entire physical CPU cores, meaning no other Linodes will have any processes on the same cores you’re using. Dedicated CPUs are therefore exempt from any competition for CPU resources and the potential problems that could arise because of CPU steal. Depending on your workload, you can experience an improvement in performance by using Dedicated CPU.

      Dedicated CPU Use Cases

      While a standard plan is usually a good fit for most use cases, a Dedicated CPU Linode may be recommended for a number of workloads related to high and constant CPU processing. Such examples include:

      CI/CD Toolchains and Build Servers

      CI and CD are abbreviations for Continuous Integration and Continuous Delivery, respectively, and refer to an active approach to DevOps that reduces overall workloads by automatically testing and regularly implementing small changes. This can help to prevent last-minute conflicts and bugs, and keeps tasks on schedule. For more information on the specifics of CI and CD, see our Introduction to CI/CD Guide.

      In many cases, the CI/CD pipeline can become resource-intensive if many new code changes are built and tested against your build server. When a Linode is used as a remote server and is expected to be regularly active, a Dedicated CPU Linode can add an additional layer of speed and reliability to your toolchain.

      Game Servers

      Depending on the intensity of demands they place on your Linode, game servers may benefit from a Dedicated CPU. Modern multiplayer games need to coordinate with a high number of clients, and require syncing entire game worlds for each player. If CPU resources are not available, then players will experience issues like stuttering and lag. Below is a short list of popular games that may benefit from a Dedicated CPU:

      Audio and Video Transcoding

      Audio and Video Transcoding (AKA Video/Audio Encoding) is the process of taking a video or audio file from its original or source format and converting it to another format for use with a different device or tool. Because this is often a time-consuming and resource-intensive task, a Dedicated CPU or Dedicated GPU Linode are suggested to maximize performance. FFmpeg is a popular open source tool used specifically for the manipulation of audio and video, and is recommended for a wide variety of encoding tasks.

      Big Data and Data Analysis

      Big Data and Data Analysis is the process of analyzing and extracting meaningful insights from datasets so large they often require specialized software and hardware. Big data is most easily recognized with the the “three V’s” of big data:

      • Volume: Generally, if you are working with terabytes, petabytes, exabytes, or more amounts of information you are in the realm of big data.
      • Velocity: With Big Data, you are using data that is being created, called, moved, and interacted with at a high velocity. One example is the real time data generated on social media platforms by their users.
      • Variety: Variety refers to the many different types of data formats with which you may need to interact. Photos, video, audio, and documents can all be written and saved in a number of different formats. It is important to consider the variety of data that you will collect in order to appropriately categorize it.

      Processing big data is often especially hardware-dependent. A Dedicated CPU can give you access to the isolated resources often required to complete these tasks.

      The following tools can be extremely useful when working with big data:

      • Hadoop – an Apache project for the creation of parallel processing applications on large data sets, distributed across networked nodes.

      • Apache Spark – a unified analytics engine for large-scale data processing designed with speed and ease of use in mind.

      • Apache Storm – a distributed computation system that processes streaming data in real time.

      Scientific Computing

      Scientific Computing is a term used to describe the process of using computing power to solve complex scientific problems that are either impossible, dangerous, or otherwise inconvenient to solve via traditional means. Often considered the “Third Pillar” of modern science behind Theoretical Analysis and Experimentation, Scientific Computing has quickly become a prevalent tool in scientific spaces.

      Scientific Computing involves many intersecting skills and tools for a wide array of more specific use cases, though solving complex mathematical formulas dependent on significant computing power is considered to be standard. While there are a large number of open source software tools available, below are two general purpose tools we can recommend to get started with Scientific Computing.

      It’s worth keeping in mind that, beyond general use cases, there are many more example of tools and software available and often designed for individual fields of science.

      Machine Learning

      Machine learning is a powerful approach to data science that uses large sets of data to build prediction algorithms. These prediction algorithms are commonly used in “recommendation” features on many popular music and video applications, online shops, and search engines. When you receive intelligent recommendations tailored to your own tastes, machine learning is often responsible. Other areas where you might find machine learning being used are in self-driving cars, process automation, security, marketing analytics, and health care.

      Below is a list of common tools used for machine learning and AI that can be installed on a Linode CPU instance:

      • TensorFlow – a free, open-source, machine learning framework and deep learning library. Tensorflow was originally developed by Google for internal use and later fully released to the public under the Apache License.

      • PyTorch – a machine learning library for Python that uses the popular GPU-optimized Torch framework.

      • Apache Mahout – a scalable library of machine learning algorithms and distributed linear algebra framework designed to let mathematicians, statisticians, and data scientists quickly implement their own algorithms.

      Where to Go From Here

      If you’re ready to get started with a Dedicated CPU Linode, our Getting Started With Dedicated CPU guide will walk you through the process of an initial installation. Additionally, see our Pricing Page for a rundown of both hourly and monthly costs.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link