A summary about hbase 89-fb online configuration.

1. 0.89 -
Online Configuration
brandboat
1

2. ❏ Add a utility to reload configurations in Region Server and HMaster
➢ Create a utility that can talk to the Region Servers and make them reload configurations on
disk.This however, does not make other classes reload, which have their own copies of conf
property values.
❏ Allow individual classes to get notified when Configuration is reloaded
➢ The purpose of this feature is to let individual classes who are interested in observing online
configuration changes, to be notified when the Conf is reloaded from disk.
Online configuration upgrade feature
2

3. Add a utility to reload configurations in
Region Server [HBASE-8544]
❏ How to run (see the usage):
❏ Flow :
1. get HRegionServer information
2.conf.reloadConfiguration()
3.getConfigurationManager.notifyAllObserver
s(conf)
${HBASE_HOME}/bin/hbase org.jruby.Main ${HBASE_HOME}/bin/reload_rs_config.rb usage
3

4. Conti. (Flow)
4
HBaseAdmin HRegionServer Configuration
updateConfiguration(address)
updateConfiguration()
// Update configuration for region server at this address.
public void updateConfiguration(HServerAddress address)
throws IOException {
HRegionInterface server =
connection.getHRegionConnection(address);
server.updateConfiguration();
}
// Reload the configuration from disk.
@Override
public void updateConfiguration() {
LOG.info("Reloading the configuration from disk.");
// Reload the configuration from disk.
conf.reloadConfiguration();
// Notify all the observers that the configuration has
// changed.
configurationManager.notifyAllObservers(conf);
}

5. conf.reloadConfiguration()
/**
* Reload configuration from previously added resources.
*
* This method will clear all the configuration read from the added
* resources, and final parameters. This will make the resources to
* be read again before accessing the values. Values that are added
* via set methods will overlay values read from the resources.
*/
public synchronized void reloadConfiguration() {
properties = null; // trigger reload
finalParameters.clear(); // clear site-limits
}
5

6. Conti. properties = null ?
private synchronized Properties getProps() {
if (properties == null) {
properties = new Properties();
loadResources(properties, resources, quietmode);
if (overlay!= null)
properties.putAll(overlay);
}
return properties;
}
6
public String get(String name)
public String getRaw(String name)
public void set(String name, String value)
…… has getProps()
/**
* Return the number of keys in the configuration.
*
* @return number of keys in the configuration.
*/
public int size() {
return getProps().size();
}

7. Added a utility to make the HMaster reload
its configuration from disk [HBASE-8587]
❏ How to run :
❏ Flow :
1. get HMaster information
2.conf.reloadConfiguration()
3.getConfigurationManager.notifyAllObserver
s(conf)
${HBASE_HOME}/bin/hbase org.jruby.Main ${HBASE_HOME}/bin/reload_master_config.rb
7

8. Allow individual classes to get notified when
Configuration is reloaded [HBASE-8576]
<ConfigurationManager.java> <ConfigurationObserver.java>
❖ If a class has configuration properties which you would like to be able to change on-
the-fly, do the following :
1. Implement the ConfigurationObserver interface.
2. Register the appropriate instance of the class with the ConfigurationManager instance,
using the ConfigurationManager#registerObserver(ConfigurationObserver)
method.
3. Deregister the instance using the
ConfigurationManager#deregisterObserver(ConfigurationObserver) method
when it is going out of scope.
8

9. ConfigurationManager.java
// The set of Configuration Observers. These classes would like to get
// notified when the configuration is reloaded from disk
private Set<ConfigurationObserver> configurationObservers =
Collections.newSetFromMap(new WeakHashMap<ConfigurationObserver,Boolean>());
// Register an observer class
public void registerObserver(ConfigurationObserver observer)
// Deregister an observer class
public void deregisterObserver(ConfigurationObserver observer)
// The conf object has been repopulated from disk, and we have to notify
// all the observers that are expressed interest to do that.
public void notifyAllObservers(Configuration conf)
// Return the number of observers.
public int getNumObservers()
9

10. ConfigurationObserver.java
/**
* Every class that wants to observe changes in Configuration properties,
* must implement interface (and also, register itself with the
* <code>ConfigurationManager</code> object.
*/
public interface ConfigurationObserver {
/**
* This method would be called by the <code>ConfigurationManager</code>
* object when the <code>Configuration</code> object is reloaded from disk.
*/
void notifyOnChange(Configuration conf);
}
10

11. Conti. notifyAllObservers()
// ConfigurationManager.java
public void notifyAllObservers(Configuration conf) {
synchronized (configurationObservers) {
for (ConfigurationObserver observer : configurationObservers) {
try {
observer.notifyOnChange(conf);
} catch (NullPointerException e) {
............
11

12. Ex. CacheConfig
// HRegionServer.java
public static final ConfigurationManager configurationManager =
new ConfigurationManager();
configurationManager.registerObserver(cacheConfig);
-------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
// CacheConfig.java
public class CacheConfig implements ConfigurationObserver { . . . . . . .
@Override
public void notifyOnChange(Configuration conf) {
new CacheConfigBuilder(conf).update(this);
}
12
configurationManager.notifyAllObservers(conf);

13. Others
Online Configuration
13

14. Number of compaction threads &
CompactionConfiguration class [HBASE-8805]
 Compaction Ratio
 OffPeak Compaction Ratio
 Throttle Point
 OffPeak Start Hour
 OffPeak End Hour
 Major Compaction Period
 Major Compaction Jitter
 Min Files to Compact
 Max Files to Compact
 Min Compact Size
 Max Compact Size
 Should Exclude Bulk
 Should Delete Expired
❏ Making the number of large/small compaction threads and the
properties in the CompactionConfiguration class online-
configurable.
❏ Properties :
14

15. Make L2 cache online configurable
❏ Allows the L2 cache to be disabled or L2 cache policies
to be modified without restarting the RS. (This is a
preliminary diff ,not fully done testing.)
❏ Properties (CacheConfig.java)
 L2_CACHE_BLOCKS_ON_FLUSH_KEY
 L2_EVICT_ON_PROMOTION_KEY (Not yet implemented!)
 L2_EVICT_ON_CLOSE_KEY
 L2_BUCKET_CACHE_SIZE_KEY
15

16. Update memstore flush threads in online
fashion
❏ The number of flush thread can be updated in
an online fashion.
❏ hbase.regionserver.flusher.count
16

17. Online configuration change for loaded
coprocessors
❏ Trying out if online config works when new
coprocessors are added, so far so good.
17

18. Allow number of SplitLogWorker threads to
be online-configurable
❏ the number of worker threads to use < the number of
workers in use, extraneous workers are stopped.
❏ the number of worker threads to use > the number of
workers in use, the appropriate number of additional
SplitLogWorker threads are added.
❏ hbase.hregionserver.hlog.split.workers.num
18

19. Make server side profiling online
configurable
❏ Make the parameter to enable/disable server side
profiling configurable online.
19

20. Hbase 1.0 Online Config Change
[HBASE-12147]
❏ HBASE-8805 HBASE-8544 in 89-fb.
❏ Improves operational efficiency in managing clusters
that are serving production traffic.
❏ The idea is to have a central configuration which can
manage notifying the configuration observers.
❏ The observers in turn should update their local state
from the latest config. Minor caveats where
configuration variables are corelated should be taken
care of with additional care.
20