Scaling Subclusters

Accelerator offers two options for automatically scaling subclusters: elastic autoscaling and schedule-based autoscaling. You also have the option of Idle Shutdown, which shuts down the entire database if there are no running queries or jobs for a specified amount of time.

Elastic Autoscaling

Elastic autoscaling manages the automatic utilization of secondary subclusters based on the total number of running sessions in a database.

When the number of running sessions in a database exceeds set threshold levels, corresponding secondary subclusters will be started and running sessions will be distributed among all the active subclusters to improve performance. Elastic autoscaling further helps in managing EC2 instance usage by stopping secondary subclusters when the number of running sessions falls below the set threshold values.

Note: Elastic autoscaling will never start or stop a primary subcluster.

Initially, all sessions will be run concurrently on the primary subcluster. As the total number of running sessions in the database exceeds a threshold value the corresponding secondary subcluster will be started and sessions will be distributed in a round-robin format across all active subclusters.

Subclusters will be stopped in reverse order as and when total running sessions drop below threshold values. Once a successful decision to scale up or down has been made, the application will wait for additional 5 minutes for all active subclusters to complete the execution of running sessions before scaling up or down.

Elastic autoscaling will need to be enabled database-by-database. Refer the Domain Name System in Vertica Accelerator section in the Subclusters chapter for information on Elastic Autoscaling DNS.

Note: You cannot use schedule-based autoscaling when elastic autoscaling is enabled.

Prerequisites

• You will need at least one secondary subcluster to enable elastic autoscaling.

• All secondary subclusters should be added to the database before configuring elastic autoscaling.

• You will have to manually disable elastic autoscaling while making topological changes like adding a new secondary, editing EC2 instance types, resizing a subcluster, or changing the CIDR access for that database.

• You will have to manually disable elastic autoscaling while creating backups or restoring tables from backups.

• Your primary subcluster will always need to be UP for elastic autoscaling to function.

• The order in which subclusters are started and stopped depends on the order in which you add your subclusters to the database. Vertica advises that you set the instance types for your subclusters accordingly before enabling elastic autoscaling.

Steps

1. Navigate to the Home page and select More menu (more_vert) for your database.

2. Select Autoscaling Policy. A new screen opens.

3. Select Elastic Autoscaling.

4. Input your desired step and period values. If unsure of the

   step value, you can start with the recommended value and adjust it based on the performance.

Step: The value which when multiplied 1x,2x, and 3x, represents the threshold values to trigger the start or stop of an additional secondary subcluster. For example, a database with a step value of 56 will be triggered if total running sessions exceeds 56, 112, or 168 for the duration of the set period.

Period: The period value represents the duration of time in which the total running sessions must remain above or below the set threshold before triggering a start or stop. Subclusters will be monitored every 5 minutes, meaning that if the period value is 15, subclusters will be monitored 4 times at 0,5,10, and 15 minutes. A secondary subcluster will be started/stopped when total number of running sessions remains increased/decreased for the duration of the specified period.

Recommended value: Vertica recommends a step value based on the sum of all vCPUs in that database. For example, if the database has a primary subcluster of i3.4xlarge instance type (16 vCPUs), and 2 secondary subclusters of types i3.2xlarge (8 vCPUs) and i3.8xlarge (32 vCPUs), the recommended step value will be 16+8+32=56.

5. Select Confirm.

Schedule-Based Autoscaling

Schedule-based autoscaling allows you to schedule automatic starts and stops for your subclusters. You can schedule one-time and recurring actions to scale your database up or down depending on anticipated workloads.

Schedule-based autoscaling cannot be enabled while elastic autoscaling is enabled.

Note: Turning on a secondary subcluster will also turn on the primary subcluster.

Turning off a primary subcluster will turn off all secondary subclusters in that database.

Prerequisites

• Elastic Autoscaling must be disabled to schedule subcluster actions.

• The primary subcluster needs to be UP to activate Schedule-based Autoscaling.

Steps

1. Navigate to the Home page and find your database.

2. Select Autoscaling Policy from the More menu (more_vert) Iconon the right-top corner of the

   database. A new window opens.

3. Select Schedule-based autoscaling in the ‘Database autoscaling’

   section.

4. Select Schedule one-time action or Schedule recurring action as per

   your requirement.

5. Configure your scheduled action:

   a. For one-time actions:

   i.  Select your Action, Subcluster, Date, and Time.
   
   ii. Select **Schedule** to schedule your one-time action.
   

   b. For recurring actions

   i.  Select your Action, Subcluster, Recurrence, and Time.
   
   ii. Select **Schedule** to schedule your recurring action.
   

Note: Recurring actions can be toggled on and off from the Autoscaling Policy page.

Currently, Accelerator does not support editing of scheduled jobs. Instead, you can delete and schedule a new job.

Idle Shutdown

Idle shutdown shuts down the entire database (i.e., stops the primary and all secondary subclusters) if there are no running queries or jobs for a specified amount of time. This includes:

• Refreshing/accessing the Home page

• Refreshing/accessing the Monitor page

• Refreshing the database card

• Updating database configuration, subcluster size, subcluster EC2 instance type

• Creating backups or restoring backups

• Adding/dropping subcluster(s)

• Starting/stopping subcluster(s)

Enabling idle shutdown can help you automate resource conservation and cut down on costs.

Prerequisites

The primary subcluster needs to be UP to activate Idle Shutdown.

Steps

1. Navigate to the Home page and find your database.

2. Select Autoscaling Policy from the More menu(more_vert) on the right-top corner of the database. A new window opens.

3. Toggle on Idle shutdown.

4. Choose your idle shutdown time.

Idle shutdown time: It is the duration for which your database must be idle before being automatically shut down. Subclusters will be monitored every 5 minutes, meaning that if the Idle shutdown time is 15, subclusters will be monitored 4 times at 0,5,10, and 15 minutes.

Subclusters in Eon Mode

Every Vertica Accelerator database supports 2 types of subclusters: Primary and Secondary.

• Primary subclusters form the core of your database. Your primary subcluster should be running for all job or query executions on that database. They are best suited for tasks such as running DDL statements and data loading, as they can perform these tasks more efficiently than secondary subclusters.

• Secondary subclusters are designed for dynamic scaling: you add and remove or start and stop these subclusters based on your analytic needs. They are optimized for running queries, rather than DDL or data loading workloads.

The nodes in the subcluster inherit their primary or secondary status from the subcluster that contains them; primary subclusters contain primary nodes and secondary subclusters contain secondary nodes.

Every node in your Eon Mode database must belong to a subcluster. This requirement means your database must always have at least one subcluster. When you create a new database, Vertica creates a subcluster named default_subcluster that contains the nodes you create on database creation. This is the database’s primary subcluster.

Note: Currently, Vertica Accelerator does not support designating another subcluster as the default subcluster.

Subcluster names cannot be changed after creation. Renaming using SSH command will lead to an error making the subcluster unreadable to Accelerator.

You can scale a Vertica Accelerator database by adding/dropping secondary subclusters, or by changing the number of nodes per subcluster. Scalability is usually more important in cloud environments where you are paying by the hour for each node in your database. If your database isn't busy, there is no reason to have underused nodes costing you money. You can reduce the number of nodes in your database during quiet times (weekends and holidays, for example) to save money.

Note: The primary subcluster cannot be dropped. Alternatively, you can terminate or delete the database.