diff --git a/modules/learn/pages/buckets-memory-and-storage/storage-engines.adoc b/modules/learn/pages/buckets-memory-and-storage/storage-engines.adoc index 83e6dbd97f..ea09e895af 100644 --- a/modules/learn/pages/buckets-memory-and-storage/storage-engines.adoc +++ b/modules/learn/pages/buckets-memory-and-storage/storage-engines.adoc @@ -18,17 +18,20 @@ If you have a small data set that can fit in memory, then you should consider us [.edition]#{enterprise}# Magma is designed for high performance with very large datasets that do not fit in memory. -It is ideal for use cases that rely primarily on disk access. +It's ideal for use cases that rely primarily on disk access. The performance of disk access will be as good as the underlying disk sub-systems -- for example, using NVMe SSDs will give higher performance. -In order to get maximum performance from Magma for disk-oriented workloads, it is recommended to set the Writer Threads to `Disk i/o optimized`. This setting will ensure there are enough threads to sustain high write rates. - -To learn more about Writer Thread settings, see xref:manage:manage-settings/general-settings.adoc#data-settings[Data Settings] - Magma can work with very low amounts of memory for large datasets: a minimum memory-to-data ratio of 1% is required. For example, if a node is holding 5{nbsp}TB of data, Magma can be used with only 64{nbsp}GB RAM. +Magma has several thread settings to tune performance for different workloads. +To get maximum performance from Magma for disk-oriented workloads, set the Writer Threads to `Disk i/o optimized`. This setting allocates more threads to sustain high write rates. +See xref:manage:manage-settings/general-settings.adoc#data-settings[Data Settings] for more information about this setting. + +You can also tune the number and allocation of threads that Magma uses to compact and flush data to disk. +You may choose to change the ratio of threads that compact data to the threads that flush data if you notice that Magma's data compaction spikes CPU use. + .Magma Supported Services |=== diff --git a/modules/learn/pages/buckets-memory-and-storage/storage-settings.adoc b/modules/learn/pages/buckets-memory-and-storage/storage-settings.adoc index e3b731b5c1..d24f278418 100644 --- a/modules/learn/pages/buckets-memory-and-storage/storage-settings.adoc +++ b/modules/learn/pages/buckets-memory-and-storage/storage-settings.adoc @@ -31,6 +31,11 @@ For illustrations of how Couchbase Server saves new and updates existing Couchba [#threading] == Threading +Couchbase Server uses multiple threads when reading an writing data. +It offers several configuration settings you can change to optimize performance for your workload and hardware. + +=== Reader and Writer Threads + Synchronized, multithreaded _readers_ and _writers_ provide simultaneous, high-performance operations for data on disk. Conflicts are avoided by assigning each thread (reader or writer) a specific subset of the 1024 vBuckets for each Couchbase bucket. @@ -59,7 +64,35 @@ Again, the maximum thread-allocation that can be specified for each is _64_, the Thread-status can be viewed, by means of the [.cmd]`cbstats` command, specified with the [.param]`raw workload` option. See xref:cli:cbstats-intro.adoc[cbstats] for information. -For information on using the REST API to manage thread counts, see xref:rest-api:rest-reader-writer-thread-config.adoc[Setting Thread Allocations]. +For information about using the REST API to manage thread counts, see xref:rest-api:rest-reader-writer-thread-config.adoc[Setting Thread Allocations]. + +[#magma-flushing-and-compaction-threads] +=== Magma Flushing and Compaction Threads + +Couchbase Server compacts the data it writes to disk for xref:learn:buckets-memory-and-storage/storage-engines.adoc#storage-engine-magma[Magma] buckets. +It allocates a thread pool (containing 20 threads by default) for background compaction and flushing operations for these buckets. +You can change the number of threads in this pool using the `num_storage_threads` setting of the xref:rest-api:rest-reader-writer-thread-config.adoc[thread allocation REST API]. + +Two types of threads share the Magma thread pool: compactor threads that compact data and flusher threads that write data to disk. +By default, Couchbase Server allocates 20% of the threads to flushing data and 80% to compacting data. +With the default thread pool size and the default flusher allocation, Couchbase Server uses 4 threads to flush data and 16 threads to compact data for Magma buckets. +You can also change the ratio of flusher to compactor threads using the `magma_flusher_thread_percentage` setting of the xref:rest-api:rest-reader-writer-thread-config.adoc[thread allocation REST API] + +For most workloads, the default thread pool size and flusher allocation percentage work well. +If you notice CPU use spikes during heavy data mutation workloads, you may want to investigate whether the compactor threads are the cause. +You can monitor the compaction and flushing activity using the `kv_magma_compactions` metric. +This metric counts the number of compactions Couchbase Server has performed. +You can view this metric via the xref:rest-api:rest-statistics.adoc[] REST API or through Prometheus if you have configured it to collect Couchbase Server metrics. +See xref:manage:monitor/set-up-prometheus-for-monitoring.adoc[] for more information about using Prometheus with Couchbase Server. + +If you see the CPU use of the `memcached` Linux process on your nodes spike while the `kv_magma_compactions` count is increasing, the compactor threads may be the cause of the spike. +In this case, you may want to reduce the number of compactor threads by increasing the percentage of flusher threads. +This reduction limits the compactor's ability to spike CPU use. +However, reducing the number of compactor threads may lead to higher latency before Couchbase Server writes data to disk. + +NOTE: The number of threads in the compactor and flusher pool and the percentages of flusher threads are advanced settings. +Contact Couchbase Support before making changes to them. +Support can help you determine the best settings for your workload and hardware. [#deletion] == Deletion diff --git a/modules/rest-api/pages/rest-reader-writer-thread-config.adoc b/modules/rest-api/pages/rest-reader-writer-thread-config.adoc index 3ad07c56c3..f3ebd3c008 100644 --- a/modules/rest-api/pages/rest-reader-writer-thread-config.adoc +++ b/modules/rest-api/pages/rest-reader-writer-thread-config.adoc @@ -1,5 +1,5 @@ = Setting Thread Allocations -:description: The number of threads allocated per node is configurable. +:description: Couchbase Server has several settings that let you change how it allocates and uses threads for the entire cluster. :page-topic-type: reference [abstract] @@ -7,6 +7,14 @@ == HTTP method and URI +.Get Global Thread Settings +[source,url] +---- +GET /pools/default/settings/memcached/global +---- + +.Set Global Thread Allocations +[source] ---- POST /pools/default/settings/memcached/global ---- @@ -14,69 +22,216 @@ POST /pools/default/settings/memcached/global [#description] == Description -Couchbase Server allows the number of threads allocated per node for _reading_, _writing_, and for _NonIO_, and _AuxIO_ thread-pools to be configured by the administrator. +Couchbase Server lets you set the number of threads per node for reading, writing, NonIO, and AuxIO thread pools. +You can also change the number of threads that Magma buckets use for writing data to disk. +Increasing thread allocation can improve performance on systems with a large number of CPU cores. +For example, more writer threads can optimize durable write performance. +See xref:learn:data/durability.adoc[Durability] for details. +However, allocating too many threads can reduce performance on nodes with limited resources. +Test any changes to thread allocation before applying them to production systems. + + +== Getting Current Thread Settings + +To get the current global thread settings for Couchbase Server, use the `GET /pools/default/settings/memcached/global` endpoint. + +[#get-curl-syntax] +=== curl Syntax + +[source,bash] +---- +curl -X GET http[s]://{HOST}:{PORT}/pools/default/settings/memcached/global + -u $USER:$PASSWORD +---- + + +==== Path Parameters +:required-privileges: required-privileges-get +include::partial$user_pwd_host_port_params.adoc[] + +[#required-privileges-get] +=== Required Privileges + +Your account must have at least 1 of the following roles to make a GET request from this endpoint: + +* xref:learn:security/roles.adoc#full-admin[Full Admin] +* xref:learn:security/roles.adoc#cluster-admin[Cluster Admin] +* xref:learn:security/roles.adoc#external-user-security-admin[External User Security Admin] +* xref:learn:security/roles.adoc#local-user-security-admin[Local User Security Admin] + +=== Responses + +200 OK:: +The request was successful. +Returns a JSON object containing the current global thread settings that have been set explicitly. + +400 Bad Request:: +The request could not be understood. +Also returns a JSON object containing an error message. + +=== Example + +The following example shows how to get the current global thread settings for Couchbase Server: + +[source,bash] +---- +curl -X GET -u Administrator:password \ +http://localhost:8091/pools/default/settings/memcached/global | jq +---- + +If successful, the call returns a JSON object containing the any overrides to the default global thread settings: + +[source,json] +---- +{ + "magma_flusher_thread_percentage": 30, + "num_storage_threads": 25 +} +---- -A high thread-allocation may improve performance on systems whose hardware-resources are commensurately supportive (for example, where the number of CPU cores is high). -In particular, a high number of _writer_ threads on such systems may significantly optimize the performance of _durable writes_: see xref:learn:data/durability.adoc[Durability], for information. +== Setting Global Thread Allocations -Note, however, that a high thread-allocation might _impair_ some aspects of performance on less appropriately resourced nodes. -Consequently, changes to the default thread-allocation should not be made to production systems without prior testing. +To set the global thread allocations for Couchbase Server, perform a `POST` ro the `/pools/default/settings/memcached/global` endpoint. [#curl-syntax] -== Curl Syntax +=== curl Syntax +[source,bash] ---- -curl -X POST -d hostname=:8091 - -d num_reader_threads= - -d num_writer_threads= - -d num_nonio_threads= - -d num_auxio_threads= - -d password= - -u : - http://:/pools/default/settings/memcached/global +curl -X POST http[s]://{HOST}:{PORT}/pools/default/settings/memcached/global + [-d num_reader_threads=] + [-d num_writer_threads=] + [-d num_nonio_threads=] + [-d num_auxio_threads=] + [-d num_storage_threads=] + [-d magma_flusher_thread_percentage=] + -u $USER:$PASSWORD ---- -Each key is specified optionally. + + +==== Path Parameters +:required-privileges: required-privileges-put +include::partial$user_pwd_host_port_params.adoc[] + +==== Parameters + +`num_reader_threads`:: +(optional) Sets the number threads Couchbase Server uses to read data. + +`num_writer_threads`:: +(optional) Sets the number threads Couchbase Server uses to write data. + +`num_nonio_threads`:: +(optional) Sets the number of NonIO threads to be used by Couchbase Server. +The NonIO thread pool runs in-memory tasks. +For example, the durability timeout task uses these threads. +Valid values are: -The keys `num_reader_threads` and `num_writer_threads` specify the number of threads that are to be used for _reading_ and _writing_, respectively. ++ +* An integer from `1` to `64`: explicitly sets the number of threads. +* `default`: Couchbase Server calculates and applies an appropriate number of threads. -The keys `num_nonio_threads` and `num_auxio_threads` specify the number of threads that are to be used for _NonIO_ and _AuxIO_ thread pools, respectively. -Note that: +`num_auxio_threads`:: +(optional) Sets the number of AuxIO threads to be used by Couchbase Server. +The AuxIO thread pool runs auxiliary I/O tasks -- for example, the access log task. +Valid values are: -* The _NonIO_ thread pool is used to run _in memory_ tasks -- for example, the _durability timeout_ task. ++ +* An integer from `1` to `64`: explicitly sets the number of threads. +* `default`: Couchbase Server determines the number of threads. -* The _AuxIO_ thread pool is used to run _auxiliary I/O_ tasks -- for example, the _access log_ task. +`num_storage_threads`:: +(optional) Sets the number of threads the Magma storage engine can use when compacting and writing data to disk. +This thread pool is divided into 2 types of threads: flusher threads and compactor threads. +See xref:learn:buckets-memory-and-storage/storage-settings.adoc#magma-flushing-and-compaction-threads[Magma Flushing and Compaction Threads] for more information. +The default value is `20`. + +`magma_flusher_thread_percentage`:: +(optional) Sets the percentage of Magma storage engine threads to allocate to flushing compacted data to disk. +The remaining threads in the thread pool are allocated to compacting data. +For example, if you set `num_storage_threads` to `20` and set `magma_flusher_thread_percentage` to `50`, Couchbase Server uses `10` threads to flush data and `10` threads to compact data. +Valid values for this setting are integers from `0` to `100`. +The default setting is `20` which allocates 20% of threads to flushing data. + +NOTE: `num_storage_threads` and `magma_flusher_thread_percentage` are advanced settings. +Contact Couchbase Support before making changes to them. +Support can help you determine the best settings for your workload and hardware. + +[#required-privileges-put] +=== Required Privileges + +Your account must have at least 1 of the following roles to make a POST request to this endpoint: + +* xref:learn:security/roles.adoc#full-admin[Full Admin] +* xref:learn:security/roles.adoc#cluster-admin[Cluster Admin] -In all cases, the value of `int` should be an integer between `1` and `64`, inclusive. -If in any case the value `default` is specified (for example, `-d num_reader_threads=default`), Couchbase Server itself calculates and applies an appropriate number of threads. [#responses] -== Responses +=== Responses + +202 Accepted:: +The request was successful. +This result also returns a JSON object containing all non-default thread settings. + +400 Bad Request:: +The request had errors or invalid values. +This result also returns a JSON object containing an error message. + Success returns an object whose values confirm the settings that have been made. [#examples] -== Examples +=== Examples To set the numbers of reader, writer, NonIO, and AuxIO threads for Couchbase Server, use the `POST /pools/default/settings/memcached/global` http method and endpoint as follows: +[source,bash] ---- -curl -v -X POST -u Administrator:password \ -http://10.144.220.101:8091/pools/default/settings/memcached/global \ +curl -X POST -u Administrator:password \ +http://localhost:8091/pools/default/settings/memcached/global \ -d num_reader_threads=12 \ -d num_writer_threads=8 \ -d num_nonio_threads=6 \ --d num_auxio_threads=6 +-d num_auxio_threads=6 | jq +---- + +This sets the number of `reader` threads to `12`, the number of `writer` threads to `8`, and the numbers of `NonIO` and `AuxIO` threads each to `6`. +If successful, the call returns an object that confirms the settings you specified: + +[source,json] +---- +{ + "num_reader_threads": 12, + "num_writer_threads": 8, + "num_auxio_threads": 6, + "num_nonio_threads": 6 +} ---- -This sets the number of _reader_ threads to `12`, the number of _writer_ threads to `8`, and the numbers of _NonIO_ and _AuxIO_ threads each to `6`. -If successful, the call returns an object whose values confirm the settings that have been made: +The following example increases the Magma storage engine's thread pool to 30 threads and allocates 25% of the threads to flushing data to disk: +[source,bash] ---- -{"num_reader_threads":12,"num_writer_threads":8,"num_auxio_threads":6,"num_nonio_threads":6} +curl -X POST -u Administrator:password \ + http://localhost:8091/pools/default/settings/memcached/global \ + -d num_storage_threads=30 \ + -d magma_flusher_thread_percentage=25 | jq ---- +If successful, the call returns an object confirming your new settings: + +[source,json] +---- +{ + "magma_flusher_thread_percentage": 25, + "num_storage_threads": 30 +} +---- + + + [#see-also] == See Also -See xref:learn:data/durability.adoc[Durability], for information on durable writes. -See xref:learn:buckets-memory-and-storage/storage-settings.adoc#threading[Threading] for an overview of threads. +* See xref:learn:data/durability.adoc[Durability], for information about durable writes. +* See xref:learn:buckets-memory-and-storage/storage-settings.adoc#threading[Threading] for an overview of threads. diff --git a/modules/rest-api/partials/user_pwd_host_port_params.adoc b/modules/rest-api/partials/user_pwd_host_port_params.adoc new file mode 100644 index 0000000000..06a3959e93 --- /dev/null +++ b/modules/rest-api/partials/user_pwd_host_port_params.adoc @@ -0,0 +1,15 @@ +// Be sure to set the 'required-priviledges' attribute before including this partial. +// It must be the anchor to the Required Privileges section for the specific endpoint. +`HOST`:: +Host name or IP address of a Couchbase Server. + +`PORT`:: +Port number for the REST API. +Defaults are 8091 for unencrypted and 18901 for encrypted connections. + +`USER`:: + +The name of a user who has at least 1 of the roles listed in xref:{required-priviledges}[Required Privileges]. + +`PASSWORD`:: +The password for the user.