Scaling Strategies for Reliable and Scalable Systems

Scaling Strategies for Reliable and Scalable Systems

"Every software engineer should know" System Design for Software Engineers

System design is the process of defining the architecture, modules, interfaces, and data for a system to satisfy specified requirements. It involves analyzing and understanding the requirements, breaking down the system into smaller parts, identifying constraints, and developing a high-level solution. The goal is to create a comprehensive and efficient design that meets the needs of the user.

Scaling is a crucial aspect in system design, as it determines the system's ability to handle increased demand from the internet.

There are two types of scaling: vertical and horizontal.

Vertical Scaling

On a virtual machine, vertical scaling is achieved by stopping an instance and resizing it to an instance type that has more RAM, CPU, I/O, or networking capabilities. Scaling vertically can eventually hit a limit, as a single virtual machine can only grow as large as the underlying hardware will allow. Vertical scaling operations can incur downtime, which decreases availability. However, vertical scaling is very easy to implement and can be sufficient for many use cases, especially in the short term.

Advantages of Vertical Scaling:-

  • Cost-Effective: Adding resources to an existing server is trivially inexpensive, especially in a virtualized environment such as the cloud.

  • Less Complex System Functions: When a single server (monolith) handles all services, it does not have to synchronize and communicate with other servers.

  • Less Complex Operations/Maintenance: Maintenance costs are lower because there are fewer servers to manage.

  • Simpler Software: Software can be developed as a monolith and does not need to be refactored into a distributed architecture.

Disadvantages of Vertical Scaling:-

  • Increased Downtime: Single servers that are taken down for patching or upgrades create service outages.

  • Single Point of Failure: Single servers increase the risk of losing data in the case of a hardware or software failure.

  • Hardware Limitations: There is a limit to the resources that can be added to a single server. Every machine has its threshold for RAM, storage, and processing power.

Horizontal Scaling

Horizontal scaling is the optimal way to build applications to leverage cloud computing but requires applications to be developed in a manner that permits a distributed architecture. Systems that cannot be architected to distribute their workloads across multiple resources must be scaled vertically.

Advantages of Horizontal Scaling:-

  • Ease of Implementation: Horizontal scaling is easier from a hardware perspective because to scale out you just add additional resources to your current pool.

  • High Availability / Redundancy: A highly available system is fault-tolerant and can withstand the failure of an individual or multiple components (e.g., hard disks, servers, network links, etc.). The elimination of single points of failure increases resilience and creates high levels of service availability.

  • Less Downtime: There is no need to take the system down when load changes. You simply add or remove resources from the cluster pool as demand dictates.

  • Geographic Distribution: Improved performance is achieved through reduced latency for global systems.

  • Improved Performance: A distributed system can change capacity quickly without downtime to match demand, enabling the system to maintain a consistent level of performance and availability.

Disadvantages of Horizontal Scaling:-

  • Increased System Complexity: A distributed/clustered architecture is more complex than a single server. Distributed systems must be engineered to synchronize and communicate with other servers.

  • Increased Complexity Operations/Maintenance: A cluster of multiple servers is harder to maintain than a single server. This increases the number of systems that need to be managed and maintained. This architecture requires additional technologies to manage the cluster, such as load balancing, replication, and virtualization.‍