Versions Compared

Old Version 8

changes.mady.by.user Roar Hansen

Saved on

compared with

New Version 9

changes.mady.by.user Roar Hansen

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Maintaining Locator Internal Systems

Some components of the Locator platform require maintenance on a periodic basis to keep the system running smoothly.  The following components need to be monitored and maintained:
  • Internal Database (PostgreSQL or MSSQL)
  • Solr
  • Internet Information Services (IIS)

The following steps should be performed:

1. Determining if the System is Running Correctly

Windows Services

Verify the following services are running on the Locator server:

●Locator Database Service

●Locator Index Builder Service

●Locator Index Service

●Locator Licensing Service

●"Fetch" service for each connector, such as the following: 

  • ayfie FileServer Fetch Service
  • ayfie SharePoint Fetch Service
  • ayfie  Exchange Fetch Service

Internet Information Services

From the IIS Manager, click on "Application Pools" under "Connections", and verify ViaWorksAppPool is running.

Processes

From the Windows Task Manager, on the "Processes" tab, verify that processes starting with Via.xxxxx in their name are running and memory levels are at acceptable levels.   Most processes should not take up more than 500Kb memory.

2. Determining if the System Needs Attention

Checking Locator error logs

The logs folder is located wherever you installed the program data path during installation.  By default it’s located in c:\programdata\VirtualWorks\ViaWorks\logs

 ●    Look in file "error.log" or "fatal.log".  Errors in these files could point to potential problems in the system.

Solr Admin tool

Access from the local machine only http://localhost:8983/solr/#/

The solr admin tool is a good diagnostic tool to help determine the number of documents in the index, the amount of memory used and more.


Dashboard 

The dashboard will show you the version, JVM information and arguments along with JVM memory usage.

Core Admin 

The Core admin will show you the directories configured as well as the index size and other index state.


3. Components that need to be Backed up

The database needs to be backed up at regular intervals.  The interval is determined by the customer and should be based on the rate of new documents going into the system, the time required to fetch that document volume, and the customer's user community requirements for how soon new and changed documents should be indexed and searchable.

4. Creating a Backup

Overview

The steps in this section describe how to perform a complete backup and restore of both the ayfie Locator index and database.  Options include restoring to the same or to a different server. Restoring to the same server only requires PostgreSQL / MSSQL changes.  Restoring to a different server requires PostgreSQL / MSSQL and Locator changes. This document refers to the server being backed up as the Primary server, and if the database is being restored to a different server, that is referred to as the Backup server.

For PostgreSQL installations, tt is possible to perform database backup/restore operations from the command line using the PostgreSQL pg_dump and pg_restore commands, as well as from using the pgAdmin UI. However, we strongly recommend using the method described below.


Assumptions

  • There is already a primary ayfie Locator server up and running, configured with connectors, and licensed
  • Locator is installed and configured for its default file locations
  • You have an account with local administrator rights on both the primary and backup servers
  • You have a Locator License for the backup server (if restoring to a different server)
  • You have installed all of the custom connectors on the backup server (if restoring to a different server)
  • This is a point-in-time backup solution. If you continue to use the primary after taking the backup, there will be differences between the real-time data and the backup. The longer the time between the backup and the restore, the larger the gap of information missing.Please note that this characteristic is not unique to this backup solution.

Planning

  • You will need enough free space to make the backup of your database. You can check the size of the PostgreSQL database by getting the size of the C:\ProgramData\VirtualWorks\ViaWorks\Database directory and files. To determine the size of a MSSQL database, open SQL Server Management Studio, expand Databases, right click on the Locator database, select Reports > Standard Reports > Disk Usage.

Making a Backup

Solr Index

  • Stop all of the Locator Services
  • Make a directory on one of the server’s local drives and name it “Locator _Backup” (e.g. C:\Locator _Backup)
    • Make three sub-directories under C:\Locator _Backup
      • DB
      • Solr
      • ZooKeeper
  • From an elevated command prompt, change directory to, for version 2.8 or older: C:\ProgramData\VirtualWorks\ViaWorks\Solr\  or, for version 2.9 and newer: C:\ProgramData\ayfie\Locator\Solr\ 
    • For a standard backup (all files and folder structure is visible like any other folder on your hard drive) copy the files directly to the backup directory : xcopy . C:\Locator_Backup\Solr /s/e
    • For a zip format backup use a compression program (like 7zip) to copy the files to a zip file in the backup directory: 7z a -tzip c:\Locator_Backup\SOLR\SOLR.zip -r
  • From an elevated command prompt, change directory to C:\ProgramData\Virtualworks\ViaWorks\ZooKeeper or C:\ProgramData\ayfie\Locator\ZooKeeper
    • For a standard backup (all file and folder structure is visible like any other folder on your hard drive) copy the files directly to the backup directory : xcopy. C:\Locator_Backup\ZooKeeper /s/e 
    • For a zip format backup use a compression program (like 7zip) to copy the files to a zip file in the backup directory: 7z a -tzip C:\Locator_Backup\ZooKeeper\ZooKeeper.zip -r 

PostgreSQL

Primary Locator Server

  • PostgreSQL database changes (Required only for ViaWorks Server versions prior to 2.4.  The following changes will have already been applied starting with the 2.4 release)
    • Edit the Locator postgresql.conf located at C:\ProgramData\VirtualWorks\ViaWorks\Databaseand make the following changes:

      # ---------------------------------------------------------------------
      # BEGIN - ViaWorks settings for Backup/Restore
      # ---------------------------------------------------------------------
      wal_level = archive
      max_wal_senders = 2
      # ---------------------------------------------------------------------
      # END - ViaWorks settings for Backup/Restore
      # ---------------------------------------------------------------------

    • In the same directory, edit pg_hba.conf and remove the comment prefix ("#") from the second line highlighted below:

      #local    DATABASE        USER            ADDRESS            METHOD
      # host    replication     postgres        127.0.0.1/32       trust

  • Start the Locator Database service
  • From an elevated command prompt, change directory to C:\Program Files\VirtualWorks\ViaWorks\PostgreSQL\bin\ or C:\Program Files\ayfie\Locator\PostgreSQL\bin\
    • For a standard backup (all files and folder structure is visible like any other folder on your hard drive) run: 
      • pg_basebackup.exe -D "C:\ViaWorks_Backup\DB" --xlog --progress --host=127.0.0.1 --username=postgres --no-password
    • For a gzip format backup run: 
      • pg_basebackup.exe -D "C:\ViaWorks_Backup\DB" --format=t --xlog --gzip --progress --host=127.0.0.1 --username=postgres --no-password
        • If you choose to use the gzip format, then be aware that you need a zip application installed/portable to unzip it (7-zip), and it will take some time and space. Because gzip has another .tar inside that needs to be unpacked.
  • Regardless of which format you pick, the backup will begin and display the progress. It will look something like: 
    • 25124/31938 kB (78%), 1/1 tablespace 
  • ...as it backs up the database. Eventually it will reach 100%
  • Start the rest of the Locator Services

Backup Server (required steps if you will be restoring to a backup server)

  • Copy the backup folder (e.g. C:\Locator _Backup) to a network share that is accessible from the backup server
  • Install the same version of Locator as the Primary server, but do not configure connectors or add any licenses
  • On the backup server, open an elevated command prompt
    • Type in “set computer” and hit enter
    • Use the value displayed for “COMPUTERNAME” in the SQL in the Backup Server section below

...

  • Stop all of the Locator Services
  • Delete all of the files and sub-directories in C:\ProgramData\VirtualWorks\ViaWorks\Solr and C:\ProgramData\VirtualWorks\ViaWorks\ZooKeeper
  • If you chose the standard method for your backup, copy the files from the backup directory (e.g. C:\ViaWorks_Backup\Solr on same server or the network share location if a different server) to C:\ProgramData\VirtualWorks\ViaWorks\Solr and C:\ProgramData\VirtualWorks\ViaWorks\ZooKeeper respectively
  • If you chose the zip format for your backup, copy the files from the backup directory (e.g. C:\ViaWorks_Backup\Solr or the network share location if a different server) and unzip the backup file to C:\ProgramData\VirtualWorks\ViaWorks\Solr and C:\ProgramData\VirtualWorks\ViaWorks\ZooKeeper respectively

PostgreSQL

  • Delete all of the files and directories in C:\ProgramData\VirtualWorks\ViaWorks\Database
  • If you chose the standard method for your backup, copy the files from the backup directory (e.g. C:\ViaWorks_Backup on same server, the network share location if a different server) to C:\ProgramData\VirtualWorks\ViaWorks\Database
  • If you chose the gzip format for your backup, unzip the backup file to C:\ProgramData\VirtualWorks\ViaWorks\Database
  • Start the Locator Services

Restoring to a Backup Server

  • Run ”C:\Program Files\VirtualWorks\ViaWorks\PostgreSQL\bin\pgAdmin3” or C:\Program Files\ayfie\Locator\PostgreSQL\bin\
  • Select the localhost database connection
  • Select the SQL button to open the SQL window
  • Run the following SQL script on the database...

    update config.via_works_server set computer_name = 'Backup server name', dns_name='Backup server DNS name' where computer_name = 'Primary server name'

    ...You should see a result with a message similar to: 

    Query returned successfully: one row affected

Microsoft SQL Server

  • If migrating the MSSQL database and Locator, the database should be restored to the new SQL server before installing Locator on the new server
    • If only the SQL database is being migrated, the existing Locator installation can be updated to point to the new database location by editing \Program Files\VirtualWorks\ViaWorks\Config\DataAccess.config or for version 2.9 and newer - \Program Files\ayfie\Locator\Config\DataAccess.configand updating the ViaDatabaseName and ViaDatabaseServer values. Restart each service after saving the changes
  • To setup the new Locator server, run the Locator installer using the Custom install path
  • On the Database Information page, choose Microsoft SQL Server and fill in the fields for the SQL database
  • On the Component page, choose which connectors to install
  • Once installation is complete, follow the steps above for restoring the Solr index


Locator (if you are restoring to a backup server)

  • Run the Locator management console
    • Add a new license for the backup server
    • Remove the old, invalid license left over from the primary server
  • Restart the Locator IIS app pool
  • Restart/Start all of the Locator services.
  • If necessary, update your DNS record to point client DNS requests to the secondary server.

Your Locator server should now be up and running. Go to your search page, sign in, perform a search and verify results.