One place for hosting & domains

      Use Cases for Linode Object Storage


      Updated by Linode

      Contributed by

      Linode

      What is Object Storage?

      Object Storage is a method of storing data that differs in a number of ways from Block Storage. Block Storage splits files into small blocks of data. Minimal file metadata is stored alongside this data and, in general, descriptive metadata must be stored in a separate file or database. In order to use a Block Storage volume it must be attached to a host server, where it acts like a hard drive.

      In contrast, Object Storage stores data, called objects, in containers, called buckets, and each object is given a unique identifier with which it is accessed. In this way, the physical location of the object does not need to be known. The objects are stored alongside rich, configurable metadata that can be used to describe any number of arbitrary properties about the object. Each object has its own URL, so accessing the data is often as simple as issuing an HTTP request, either by visiting the object in a browser or retrieving it through the command line.

      Benefits and Limitations

      Object Storage scales easily because all the objects are stored in a flat, scalable name space. Object Storage does not require a host server in order to be used, meaning many different clients can read from it or write to it.

      With that said, there are limitations to Object Storage. Objects in Object Storage cannot be modified at the block level, as with Block Storage, and must be rewritten in their entirety every time a change is made. This makes any scenario with many successive read/write operations – such as the needs of databases or transactional data – a poor choice for Object Storage. Additionally, Object Storage traffic runs over HTTP, so it does not benefit from the I/O speeds of a mounted Block Storage volume. As a rule of thumb, Object Storage shines when files do not need to be updated frequently.

      Below are some of the more popular use cases for Object Storage.

      Use Cases

      Static Site Hosting

      Because Object Storage buckets provide HTTP access to objects, it’s easy to set up a bucket to serve static websites. A static website is a website that does not require a server-side processing language like PHP to render content. And because a static site does not require each page to be processed with every request, they are usually very quick to load. For more information on setting up a static site on Object Storage, read our Host a Static Site on Linode Object Storage guide. For more on static site generators, visit our How to Choose a Static Site Generator guide.

      Website Files

      If you don’t want to host your entire site on Object Storage (for example: you plan to use a CMS like WordPress), you can still choose to host some of your site’s assets, like images and downloads, with Object Storage. This will save disk space on your server and can help reduce your costs.

      Software Storage and Downloads

      Similar to hosting website files, hosting software applications on Object Storage is a great use case for developers looking to give quick access to their products. Simply upload the file to a bucket and share its URL.

      Unstructured Data

      Unstructured data is any data that does not fit into a traditional database. Object Storage excels at storing large amounts of unstructured data. With the ability to configure custom metadata for each piece of unstructured data, it is easy to extrapolate useful information from each object and to retrieve objects with similar metadata. Examples of unstructured data include images, video, audio, documents, and Big Data.

      Images, Video, Audio, and Documents

      Multimedia assets like images, videos, audio files, and documents are a perfect match for Object Storage. In general these types of files do not change frequently, so there is no need to store them on Block Storage volumes. Because each file has its own URL, streaming the content of these files or embedding them in another program or website is simple and does not require the use of a server.

      Big Data

      Big Data typically describes data sets that are so large and so diverse that it takes specialized tooling to analyze them. In many cases the data that comprises Big Data is considered unstructured and does not fit neatly into a database, making it a great candidate for Object Storage.

      Artifact Storage

      As more and more of the development life cycle becomes automated and tested, more and more artifacts are generated in the process. Object Storage is a great solution for developers looking to store these artifacts, such as the bulk collection of logs. Sharing stored artifacts is as simple as sharing a URL. And if you’d rather your artifacts stay private, you can distribute an access key.

      Cold Storage

      Object Storage is, in the majority of cases, significantly cheaper than Block Storage. While Object Storage can incur a cost when retrieving data, the cost benefit for infrequently accessed data can provide you with an overall cost reduction when compared to similar methods.

      Similarly, Object Storage has benefits over tape storage. Tape storage is frequently used for archival purposes, but the read times that come with tape storage are many times more than what you’ll find with Object Storage. Special considerations have to be made when transferring tape drive data, such as the ability to ship drives safely across long distances. With Object Storage, this data is available through HTTP from anywhere in the world.

      Note

      The outbound data transfer for Linode Object Storage is part of your Linode account’s total transfer pool, which will reduce or completely eliminate transfer costs for Object Storage if you are also running Linode instances. If you expend your allotted transfer pool, you will be billed at a rate of $0.02 per GB for outbound transfers.

      Backups

      Databases and other critical data can be backed up to Object Storage with little effort using a command line client for easier automation. Objects within Object Storage are normally replicated three times, providing resiliency should an error occur with the underlying hardware. Additionally, buckets can be versioned so you never lose access to older backups.

      Private File Storage

      Objects can be made private and only accessible with a key. By default, all new objects in a bucket are set to private, so they are inaccessible by normal HTTP requests (though it’s easy to set public permissions on objects if you’d like). This makes it easy to store secure data.

      Next Steps

      If you’re curious about how to use Object Storage, you can read our guide on How to Use Linode Object Storage for detailed instructions on creating buckets and uploading objects. Read our Host a Static Site using Linode Object Storage to get started with hosting your own site on Object Storage.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Use Cases for Block Storage


      Updated by Linode

      Contributed by

      Linode

      What is Block Storage

      Block Storage is a type of persistent cloud data storage that is similar to a traditional block device, like the hard drive in a PC. With Block Storage, your data is divided into blocks, which are the small, discrete units that Block Storage can read from and write to. These blocks are assigned unique identifiers, but these are generally not human-readable, so a filesystem is usually installed which maps your files to the underlying blocks they correspond to. This relationship is also analogous to your PC’s filesystem and hard drive.

      A Block Storage Volume houses these blocks of data. Volumes can be attached to a cloud computing instance, which makes its data and filesystem available to the instance. If your instance is running Linux, then mounting a Volume’s filesystem is just like mounting any other filesystem.

      Volumes are stored separately from your cloud instances, but inside the same data center, and they are attached via the data center’s private networking. A Volume can be detached from a cloud instance and its data will persist, even if the cloud instance is deleted. The Volume can also be re-attached to a different instance (though only one attachment at a time is possible). Volumes can also be increased in size at any time, independent of an instance’s built-in storage.

      Benefits and Limitations

      A Block Storage Volume augments the raw storage capacity of a cloud instance, which can be useful if your storage needs are greater than your computing demands. Because a Volume is scalable, it can adapt as your data grows in size. Additionally, all data stored with Linode Block Storage is replicated three times, so your Volumes are highly available and fault tolerant.

      Note

      While the health and uptime of Linode Block Storage is closely monitored by Linode Support, we still recommend making separate backups of your Volumes.

      Because Volumes are directly connected to an instance, their I/O speeds are much faster than those of an alternative storage solution like Object Storage. As well, the nature of Block Storage allows you to read and write small parts of your data, which means that you can incrementally update your files. This is in contrast to Object Storage, which requires a full re-upload of a file to update it.

      Some aspects of Block Storage lead to natural limitations. In particular, a Volume needs to be attached to a cloud instance for its data to be accessible. In comparison, a file stored in Object Storage can be downloaded by any internet connected client at any time.

      Below you will find some of the more popular use cases for Block Storage.

      Use Cases

      Databases

      Databases require quick read/write operations, and Block Storage Volumes are mounted directly to a cloud instance’s filesystem, so there is a minimal delay in writing and retrieving data. Growing businesses often have increasing data storage demands, so it’s also important for a database to be scalable. A Volume that stores your database can be resized to meet your storage needs.

      Persistent Storage

      Block Storage Volumes can be detached from a cloud instance just as easily as they are attached, meaning that it’s possible to create hot-swappable drives with Block Storage. This is useful if you need to perform the same kind of tasks across a fleet of instances with the same data.

      Caution

      While the Block Storage service has full support for hot swapping, it is important to follow the detachment instructions outlined in our Using Block Storage guide. If a Volume is not safely detached, there is a risk of data loss for the Volume.

      Container Storage

      Containers, like those created with Docker or inside Kubernetes Pods, can benefit from having some type of persistent storage. This helps to keep a container’s size down and makes it easy to maintain data outside of the normal lifecycle of the container or Pod.

      If you are using Docker, you can use the Docker Volume Driver for Linode to create a Docker volume from a Block Storage Volume. Similarly, if you are using Kubernetes you can use the Container Storage Interface (CSI) Driver for Linode Block Storage to create a Persistent Volume Claim that’s backed by a Block Storage Volume.

      Running Cloud Software

      In a climate where ownership over one’s data is an important need for many individuals and organizations, hosting your own cloud software is a great use case for Block Storage. Create and mount a Block Storage Volume, install an application like OwnCloud, and point its data folder to a location on your Volume. If you ever run out of space on your Volume you can always increase its size.

      Storage for Media Library Applications

      There are a few media library applications, most notably Plex, that offer media streaming functionality to internet enabled devices. The media libraries these applications serve can quickly grow in size, depending on the number of movie and audio files they contain. Using a Block Storage Volume can provide you with storage capable of growing with the needs of your library.

      Note

      Ephemeral Storage

      Various stages of the software development lifecycle can create large amounts of temporary data, such as buffers, builds, and cache and session data. While this data might only exist for a short period of time, it requires and utilizes storage space. Creating a Block Storage Volume just for ephemeral data is a good use case for times when the storage supplied with your instance is not enough, or for when you need extra space for a short period of time.

      Data Backups

      Having backups of your data is always a good idea, and Block Storage Volumes make for scalable and quickly accessible backup mediums. Store anything that you might need to quickly transfer to another instance, or anything that you might need at a moment’s notice.

      Boot Disks

      You can boot from disk images installed to a Block Storage Volume. This provides a cost effective means of maintaining an image that can be attached to a new Linode. For example, you could save money by creating and removing on-demand Linode instances that boot from a Volume. As well, you can boot from a Volume to access and recover an instance whose normal operating system may not be running as expected.

      Note

      Linode provides a built-in Rescue Mode feature, but maintaining your own rescue Volume can allow you to include the recovery tools you prefer to use.

      Next Steps

      For more information on how to use Block Storage, consult our How to Use Block Storage with Your Linode guide.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Use Cases for Linode Dedicated CPU Instances


      Updated by Linode

      Written by Ryan Syracuse

      Why Dedicated CPU

      Dedicated CPU Linodes offer a complement to CPU intensive tasks, and have the potential to significantly reduce issues that arise from shared cloud hosting environments. Normally, when creating a Linode via our standard plan, you are paying for access to virtualized CPU cores, which are allocated to you from a host’s shared physical CPU. While a standard plan is designed to maximize performance, the reality of a shared virtualized environment is that your processes are scheduled to use the same physical CPU cores as other customers. This can produce a level of competition that results in CPU steal, or a higher wait time from the underlying hypervisor to the physical CPU.

      CPU Steal can be defined more strictly as a measure of expected CPU cycles against actual CPU cycles as your virtualized environment is scheduled access to the physical CPU. Although this number is generally small enough that it does not heavily impact standard workloads and use cases, if you are expecting high and constant consumption of CPU resources, you are at risk of being negatively impacted by CPU Steal.

      Dedicated CPU Linodes have private access to entire physical CPU cores, meaning no other Linodes will have any processes on the same cores you’re using. Dedicated CPUs are therefore exempt from any competition for CPU resources and the potential problems that could arise because of CPU steal. Depending on your workload, you can experience an improvement in performance by using Dedicated CPU.

      Dedicated CPU Use Cases

      While a standard plan is usually a good fit for most use cases, a Dedicated CPU Linode may be recommended for a number of workloads related to high and constant CPU processing. Such examples include:

      CI/CD Toolchains and Build Servers

      CI and CD are abbreviations for Continuous Integration and Continuous Delivery, respectively, and refer to an active approach to DevOps that reduces overall workloads by automatically testing and regularly implementing small changes. This can help to prevent last-minute conflicts and bugs, and keeps tasks on schedule. For more information on the specifics of CI and CD, see our Introduction to CI/CD Guide.

      In many cases, the CI/CD pipeline can become resource-intensive if many new code changes are built and tested against your build server. When a Linode is used as a remote server and is expected to be regularly active, a Dedicated CPU Linode can add an additional layer of speed and reliability to your toolchain.

      Game Servers

      Depending on the intensity of demands they place on your Linode, game servers may benefit from a Dedicated CPU. Modern multiplayer games need to coordinate with a high number of clients, and require syncing entire game worlds for each player. If CPU resources are not available, then players will experience issues like stuttering and lag. Below is a short list of popular games that may benefit from a Dedicated CPU:

      Audio and Video Transcoding

      Audio and Video Transcoding (AKA Video/Audio Encoding) is the process of taking a video or audio file from its original or source format and converting it to another format for use with a different device or tool. Because this is often a time-consuming and resource-intensive task, a Dedicated CPU or Dedicated GPU Linode are suggested to maximize performance. FFmpeg is a popular open source tool used specifically for the manipulation of audio and video, and is recommended for a wide variety of encoding tasks.

      Big Data and Data Analysis

      Big Data and Data Analysis is the process of analyzing and extracting meaningful insights from datasets so large they often require specialized software and hardware. Big data is most easily recognized with the the “three V’s” of big data:

      • Volume: Generally, if you are working with terabytes, petabytes, exabytes, or more amounts of information you are in the realm of big data.
      • Velocity: With Big Data, you are using data that is being created, called, moved, and interacted with at a high velocity. One example is the real time data generated on social media platforms by their users.
      • Variety: Variety refers to the many different types of data formats with which you may need to interact. Photos, video, audio, and documents can all be written and saved in a number of different formats. It is important to consider the variety of data that you will collect in order to appropriately categorize it.

      Processing big data is often especially hardware-dependent. A Dedicated CPU can give you access to the isolated resources often required to complete these tasks.

      The following tools can be extremely useful when working with big data:

      • Hadoop – an Apache project for the creation of parallel processing applications on large data sets, distributed across networked nodes.

      • Apache Spark – a unified analytics engine for large-scale data processing designed with speed and ease of use in mind.

      • Apache Storm – a distributed computation system that processes streaming data in real time.

      Scientific Computing

      Scientific Computing is a term used to describe the process of using computing power to solve complex scientific problems that are either impossible, dangerous, or otherwise inconvenient to solve via traditional means. Often considered the “Third Pillar” of modern science behind Theoretical Analysis and Experimentation, Scientific Computing has quickly become a prevalent tool in scientific spaces.

      Scientific Computing involves many intersecting skills and tools for a wide array of more specific use cases, though solving complex mathematical formulas dependent on significant computing power is considered to be standard. While there are a large number of open source software tools available, below are two general purpose tools we can recommend to get started with Scientific Computing.

      It’s worth keeping in mind that, beyond general use cases, there are many more example of tools and software available and often designed for individual fields of science.

      Machine Learning

      Machine learning is a powerful approach to data science that uses large sets of data to build prediction algorithms. These prediction algorithms are commonly used in “recommendation” features on many popular music and video applications, online shops, and search engines. When you receive intelligent recommendations tailored to your own tastes, machine learning is often responsible. Other areas where you might find machine learning being used are in self-driving cars, process automation, security, marketing analytics, and health care.

      Below is a list of common tools used for machine learning and AI that can be installed on a Linode CPU instance:

      • TensorFlow – a free, open-source, machine learning framework and deep learning library. Tensorflow was originally developed by Google for internal use and later fully released to the public under the Apache License.

      • PyTorch – a machine learning library for Python that uses the popular GPU-optimized Torch framework.

      • Apache Mahout – a scalable library of machine learning algorithms and distributed linear algebra framework designed to let mathematicians, statisticians, and data scientists quickly implement their own algorithms.

      Where to Go From Here

      If you’re ready to get started with a Dedicated CPU Linode, our Getting Started With Dedicated CPU guide will walk you through the process of an initial installation. Additionally, see our Pricing Page for a rundown of both hourly and monthly costs.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link