...

A customer reported longer the than expected fetch times and frequent Postgres timeouts. With a server configuration of 256 GB RAM and 32 CPU cores, the daily fetch rate was expected to be well above 100,000, 100K documents but was less than 7070K. 000. This led to us to change the Postgres memory usage configuration as described below.

Config Changes

Four parameters were changed in the configuration:

• effective_cache_size estimates how much memory is available for disk caching.

  • Value The value was changed to 12GB (2GB is the default).

• maintenance_work_mem specifies maximum amount of memory to be used by maintenance operations.*

  • Value The value was changed to 1GB (256MB is the default and 2GB seems to be the limit on Windows).

• shared_buffers sets the amount of memory dedicated by the server for caching data.

  • Value The value was changed to 4GB (512MB is the default).

• work_mem sets the amount of memory available for every sort, group or hash operations.*

  • Value The value was changed to 100MB (50MB is the default ).

* Please note that maintenance_work_mem and work_mem appear to be limited to 2GB on Windows.

...

• • and 2GB seems to be the limit on Windows).

These changes are done in the file D:\Program Files\ayfie\saga\data\postgresql\data in Saga installation folder\custom.conf (assuming recommended install directory)

The changes above were done on a beefy host. Be careful to not configure beyond the available resources.

More in depth explanation for of these parameters can be found in this Postgres documentation.

Settings can be viewed using running these database queries:

• show all to display provides a simple view with human readable valuevalues

• select * from pg_settings to display displays a more complex view with additional columns and more information

Results

As a result of the changes that were made, we noticed The configuration changes produced a positive change in server performance. The daily document fetch rate went increased to above 100,000 and there were no more Postgres timeouts being reported. The values set above were used are based on the recommended values while at the same time also taking into consideration that Postgres was not running on a dedicated server.

...