Versions Compared

Old Version 1

changes.mady.by.user Steinar Skjerven

Saved on

compared with

New Version 2

changes.mady.by.user svc_ayfie_onedrive@ayfie.com

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Markdown
- [Introduction](#introduction)
- [Considerations before Starting](#considerations-before-starting)
- [Prerequisites](#prerequisites)
- [Pre Migration](#pre-migration)
  - [How to Perform Postgres Database Transfer](#how-to-perform-postgres-database-transfer)
  - [How to Perform Microsoft SQL Database Clone](#how-to-perform-microsoft-sql-database-clone)
- [Migration](#migration)
  - [Install Saga](#install-saga)
  - [Database Migration](#database-migration)
    - [Postgres Database Migration](#postgres-database-migration)
    - [Microsoft SQL Database Migration](#microsoft-sql-database-migration)
  - [Replace License](#replace-license)
  - [Install Connectors](#install-connectors)
  - [Assign Connections to New Server](#assign-connections-to-new-server)
  - [Remove Registered Server from Database](#remove-registered-server-from-database)
    - [Postgres Database Remove Registered Server](#postgres-database-remove-registered-server)
    - [Microsoft SQL Database Remove Registered Server](#microsoft-sql-database-remove-registered-server)
  - [Connector Migration Steps](#connector-migration-steps)
  - [Enable Connections](#enable-connections)
  - [Rebuild the Index](#rebuild-the-index)
  - [Assign Roles](#assign-roles)
  - [Search Contexts](#search-contexts)
  - [Content Handler](#content-handler)
- [Known Issues](#known-issues)
  - [Duplicate Users in Database Table](#duplicate-users-in-database-table)
- [Removing the Old Environment](#removing-the-major-version-2-environment)

# Introduction

This guide describes the steps for migrating from Locator 2.x and/or Supervisor 2.x to the latest version of the Saga platform on which the two products reside.

The easiest approach is to not migrate at all, but simply install the latest version of Saga from scratch. However, there are reasons why migrating might still be the preferred option:
- No need to re-configure all the connections.
- No need to re-fetch all the source data.
- No need to re-add any custom rules, any custom system scopes or any promoted results.
- The users will keep their settings, private search scopes and private tags.
- The analytics data will be retained.

The main aspect of the migration is the migration of the 2.x database to the new platform. Before starting, a transfer or clone of the 2.x database is required. The procedure is dependent on the type of database:
- **Postgres**: Do a transfer of the 2.x database to the new server.
- **Microsoft SQL**: Clone the 2.x database. The new platform will connect to the clone.

Once the transfer or the cloning has been completed, one can perform the migration on the new server independently of the existing 2.x installation. The 2.x installation can continue to serve the users until the migration is completed.

# Considerations before Starting
- Is one affected by any of the limitations in the new platform? See the *Limitations* section of the Install Guide.
- Is one affected by any of the deprecated features in the new platform? See the *Major Version Upgrades* section of the Install Guide.
- Are all connectors available on the Saga platform?
- Which type of authentication should one configure in the Authority Service?
- Are the connectors that are to be deployed compatible with the type of authentication for which the Authority Service is to be configured?
- Does the 2.x installation have any customizations? Migrating customizations is out of scope for this guide.
- In case of Postgres, how is the database to be transferred to the new server?

# Prerequisites
- The Locator 2.x installation must be upgraded to the latest 2.11 service release before starting on the migration to 5.x. At the time of this writing, that is Locator 2.11 SR10.
- Any Supervisor 2.x installation must be upgraded to the latest 2.4 service release. At the time of this writing, that is Supervisor 2.4 SR5.
- Everything listed in the Prerequisites section of the Install Guide, including a new license compatible with the Saga platform.

# Pre Migration
Before starting [Migration](#migration) a transfer (Postgres) or clone (Microsoft SQL) of the 2.x database is required, consult either [How to Perform Postgres Database Transfer](#how-to-perform-postgres-database-transfer) or [How to Perform Microsoft SQL Database Clone](#how-to-perform-microsoft-sql-database-clone).

## How to Perform Postgres Database Transfer
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Postgres database. Locator and Supervisor will be unavailable to the users while doing these steps.

On the 2.x server, perform the following steps:
- Obtain the 2.x install and data directories. If not known, then they can be retrieved by running these two PowerShell commands respectively.
  ```
  (Get-ItemProperty ("Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Virtual Works\ViaWorks") -Name "InstallDir")."InstallDir"
  (Get-ItemProperty ("Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Virtual Works\ViaWorks") -Name "DataDir")."DataDir"
  ```
- Obtain the database name from the variable *ViaDatabaseName* in the file *C:\Program Files\ayfie\Locator\Config* (assumes default install directory). It will be required later.
- Open the Management Console and disable all connections.
- Stop Default Web Site in the Internet Information Services (IIS) Manager.
- Store the code snippet below in the file *stop-services.ps1* and run it in PowerShell. The script will stop the services in the correct order.
  ```
  $services= Get-Service | Where { $_.DisplayName -like "*ViaWorks*Fetch*" -or $_.DisplayName -like "*Locator*Fetch*" -or $_.DisplayName -like "*ayfie*Fetch*"}
  $serviceNames= @(
    "Via.Authority.Service",
    "Via.ReportEngine.Service",
    "Via.Scheduler.Service"
    "Via.Notification.Service",
    "Via.Resource.Service",
    "ViaWorksIndexBuilder",
    "ViaWorksLingo",
    "Via.Index.Service",
    "Via.ZooKeeper.Service",
    "Via.Licensing.Service"
  )
  foreach ($serviceName in $serviceNames) {
    $service = Get-Service -Name $serviceName -ErrorAction SilentlyContinue
    if ($service.Length -gt 0) {
      $services += $service
    }
  }
  $services| ForEach-Object {
    Stop-Service $_.Name -Verbose
  }
  Write-Host Waiting 30 seconds before stopping database service...
  Start-Sleep -Seconds 30
  Get-Service -Name "ViaWorksDatabaseService" | Stop-Service -Verbose
  ```
- Transfer all files and sub-folders in *C:\ProgramData\ayfie\Locator\Database* (assumes default data directory) to the new server. They will be required later.
- Store the code snippet below in the file *start-services.ps1* and run it in PowerShell. The script will start the services in the correct order.
  ```
  $services= @()
  $serviceNames= @(
    "ViaWorksDatabaseService",
    "Via.Licensing.Service",
    "Via.ZooKeeper.Service",
    "Via.Index.Service",
    "ViaWorksLingo",
    "ViaWorksIndexBuilder",
    "Via.Resource.Service",
    "Via.Notification.Service",
    "Via.Scheduler.Service",
    "Via.ReportEngine.Service",
    "Via.Authority.Service"
  )
  foreach ($serviceName in $serviceNames) {
    $service = Get-Service -Name $serviceName -ErrorAction SilentlyContinue
    if ($service.Length -gt 0) {
      $services += $service
    }
  }
  $services += Get-Service | Where { $_.DisplayName -like "*ViaWorks*Fetch*" -or $_.DisplayName -like "*Locator*Fetch*" -or $_.DisplayName -like "*ayfie*Fetch*"}
  $services| ForEach-Object {
    Start-Service $_.Name -Verbose
  }
  ```
- Start the services again and the Default Web Site in the Internet Information Services (IIS) Manager.
- Open the Management Console and enable all connections.

## How to Perform Microsoft SQL Database Clone
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Microsoft SQL Server database. Locator and Supervisor will be unavailable for the users while doing these steps.

On the 2.x server, perform the following steps:
- Open the Management Console and disable all connections.
- Stop *Default Web Site* in the Internet Information Services (IIS) Manager.
- Clone the Microsoft SQL database. This requires database administrator privileges.
- Grant permissions for the gMSA account to the cloned database:
  - Open Microsoft SQL Management Studio and connect to the database server
  - Select the cloned database
  - Run the following commands after having replaced all instances of \<gMSA Account\> with the the gMSA account:
    ```
      IF NOT EXISTS (SELECT * FROM dbo.syslogins WHERE name = N'<gMSA Account>')
      BEGIN
          CREATE LOGIN [<gMSA Account>] FROM WINDOWS;
      END
      GO

      IF NOT EXISTS (SELECT * FROM [sys].[database_principals] WHERE lower([name]) = lower(N'<gMSA Account>'))
      BEGIN
          CREATE USER [<gMSA Account>] FOR LOGIN [<gMSA Account>];
          EXEC sp_addrolemember 'db_executor', '<gMSA Account>'
          EXEC sp_addrolemember 'db_ddladmin', '<gMSA Account>'
          EXEC sp_addrolemember 'db_datareader', '<gMSA Account>'
          EXEC sp_addrolemember 'db_datawriter', '<gMSA Account>'
      END
      GO
    ```
- Start the services again and the *Default Web Site* in the Internet Information Services (IIS) Manager.
- Open the Management Console and enable all connections.

# Migration
This section covers all migration steps on the new server.

## Install Saga

The first step in migration from 2.x to the new platform is configuring the new Saga platform on a new Windows Server 2019, consult the Install Guide. This can be done using the default Postgres database, even if the 2.x installation is using Microsoft SQL. The default Postgres database will be replaced by 2.x database in a later migration step. If 2.x is using Microsoft SQL database one should configure gMSA, as it will be required later. One should not install any connectors at this stage, that will be done in a later migration step. If Supervisor is installed in 2.x, one should enable the Report Engine.

One can continue with the [Database Migration](#database-migration) when one can successfully authenticate and access the search UI in the new Saga platform.

## Database Migration
The 2.x database must be migrated to the new platform. Depending on which database the 2.x installation is using, one should either do [Postgres Database Migration](#postgrs-database-migration) or [Microsoft SQL Database Migration](#microsoft-sql-database-migration)

### Postgres Database Migration
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Postgres database. If not, consult [Microsoft SQL Database Migration](#microsoft-sql-database-migration).

This section explains how to migrate the database from the 2.x server to the new server using the method described in [Upgrading Data via pg_upgrade](https://www.postgresql.org/docs/current/upgrading.html). The upgrade is required since 2.x is running Postgres version 9.6 and the Saga platform is running Postgres 12.

On the Saga platform server, do the following:
- Stop Saga
- Move the transferred sub-folders and files from 2.x server to folder *D:\Program Files\ayfie\saga\data\postgresql\migration\data*.
- Delete all files and sub-folders in *D:\Program Files\ayfie\saga\data\postgresql\data*. This will delete the database that was created during installation.
- In PowerShell, change directory to *D:\Program Files\ayfie\saga\docker*.
- In PowerShell, run the command below to perform the database migration:
  ```
  docker-compose -f docker-compose.postgres.migration.yml up ayfie-saga-db
  ```
- The migration process has completed when the message `PostgreSQL migration complete.` appears in the PowerShell console. This might take some time depending on the size of the database.
- Exit the container by pressing *Ctrl+C* after the migration process has completed.
- In PowerShell, change directory to *D:\Program Files\ayfie\saga*.
- In PowerShell, start Saga with the configure parameter by running the following command:
  `.\saga.ps1 -configure`
- In the install script menu, make sure that the database name is the same as in 2.x.
- *Exit* the install script menu.
- Wait for Saga to start successfully
- Review the *Via.Deploy.PostInstall.exe.log* to confirm that the database migration scripts ran successfully after Saga started successfully.
- Delete all files and sub-folder from folder *D:\Program Files\ayfie\saga\data\postgresql\migration\data*

### Microsoft SQL Database Migration
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Microsoft SQL Server database. If that is not the case, consult [Postgres Database Migration](#postgres-database-migration) or continue to [Migration](#migration).

On the Saga platform server, do the following:
- Stop Saga
- Delete all files and sub-folders in *D:\Program Files\ayfie\saga\data\postgresql\data*. This will delete the database that was created during installation.
- In PowerShell, start Saga with the configure parameter by running the following command:
  `.\saga.ps1 -configure`
- In the install script menu, configure Saga to use the cloned Microsoft SQL database.
- *Exit* the install script menu.
- Wait for Saga to start successfully
- Review the *Via.Deploy.PostInstall.exe.log* to confirm that the database migration scripts ran successfully after Saga started successfully.

## Replace License
- Start the Management Console and go to *Operations*->*License*.
- Replace the old 2.x license(s) with new Saga platform license(s).
- Make sure the new license has the *OmniPage OCR* converter included.

## Install Connectors
- Install all connectors from 2.x installation, see section *Installing Connectors* of the Install Guide.

## Assign Connections to New Server
- Start the Management Console and go to *Connections*->*Scheduling*
- For Each *Connector*->*Connection*
    - Click *Add Server Assignment* for *AYFIE-CONNECTOR*.
    - Click *Save*.
    - Enable *Perform Discovery* for *AYFIE-CONNECTOR* and disable *Perform Discovery* for the 2.x server.
    - Click *Save*.
    - Click *Delete* for all 2.x servers.
    - Click *Save*.

## Remove Registered Server from Database
Depending on which database the installation is using, one should either do [Postgres Database Remove Registered Server](#postgres-database-remove-registered-server) or [Microsoft SQL Database Remove Registered Server](#microsoft-sql-database-remove-registered-server).

### Postgres Database Remove Registered Server

If using Postgres database, perform the following steps to remove the registered 2.x server(s) from the database:
- Log in to pgAdmin, select the *Locator* database and open the Query Tool.
- Run Query
  ```
  SELECT server_id, computer_name, dns_name FROM config.via_works_server
  ```
- Note the server id(s) of the 2.x server(s), the server id(s) will be required for the next step (2.x server(s) are all servers not named *AYFIE-CONNECTOR*).
- For each 2.x server, unregister the 2.x server in database, by replacing \<server_id\> in the script and running the following query in the Query Tool in pgAdmin:
  ```
  DO $$
  DECLARE serverId integer;
  BEGIN
    select <server_id> into serverId;

    delete from config.via_works_windows_service
    where server_id=serverId;

    delete from config.converter_setting
    where converter_id in (
      select converter_id
      from config.converter
      where server_id=serverId
    );

    delete from config.converter
    where server_id=serverId;

    delete from config.connector_status
    where connector_id in (
      select connector_id
      from config.connector
      where server_id=serverId
    );

    delete from config.connector
    where server_id=serverId;

    delete from config.server_server_role
    where server_id=serverId;

    delete from config.via_works_server
    where server_id=serverId;

  END $$;
  ```

### Microsoft SQL Database Remove Registered Server
If using Microsoft SQL database, perform the following steps to remove the registered 2.x server(s) from the database (2.x server(s) are all servers not named *AYFIE-CONNECTOR*):
- Open Microsoft SQL Server Management Studio
- Select *Locator* database and open a *New Query*
- Run Query
  ```
  select server_id, computer_name, dns_name FROM [config].via_works_server
  ```
- Note the server id(s) of the 2.x server(s), the server id(s) will be required for the next step.
- For each 2.x server, unregister the 2.x server in the database, by replacing \<server_id\> in the script and running the following query in the Query Tool in pgAdmin:
  ```
  declare @serverId int;
  set @serverId = <server_id>;

  delete from [config].via_works_windows_service
  where server_id=@serverId;

  delete from [config].converter_setting
  where converter_id in (
    select converter_id
    from [config].converter
    where server_id=@serverId
  );

  delete from [config].converter
  where server_id=@serverId;

  delete from [config].connector_status
  where connector_id in (
    select connector_id
    from [config].connector
    where server_id=@serverId
  );

  delete from [config].connector
  where server_id=@serverId;

  delete from [config].server_server_role
  where server_id=@serverId;

  delete from [config].via_works_server
  where server_id=@serverId;
  ```

## Connector Migration Steps
- Replace the server name with the FQDN for all connections that are communicating with Active Directory. That includes:
  - The FileServer connector.
  - All database connectors indexing data from a Microsoft SQL database.
- If applicable, perform steps 10 and 11 from section *Upgrading from major version 4 to 5* of the Install Guide.

## Enable Connections
- Start Management Console and go to *Connections*
- For each connector, enable all connections.

## Rebuild the Index
- In PowerShell, rebuild the index by running the following command:
  `docker exec -it (docker ps -qf "name=^ayfie-locator$") c:\Locator\Tools\Via.Repository.exe reindex /all`

## Assign Roles
- The Authority Service has been replaced in the new platform. Any Supervisor roles assigned in 2.x, needs to be re-assigned in the new platform, see section *Managing Users and Roles* in the Install Guide.

## Search Contexts
On the new server, open URL http://localhost/Dashboard/Configuration/SearchContextList?se=1 and check if any search contexts have been configured. If yes, then add the *Timeline* refiner to all search contexts.

## Content Handler
If the Content Handler is required, then the Content Handler must be deployed across all clients. Deploying the Content Handler will automatically remove the Document Handler (the 2.x equivalent of the Content Handler). Hence, this step must be timed with going live on the new platform. How to deploy the Content Handler is described in the *Prerequisite* section of the Install Guide.

# Known Issues

## Duplicate Users in Database Table
The occurrence of the error message *Database upgrade script '50005.sql' failed* in the log file *Via.Deploy.PostInstall.exe.log* indicates that there are duplicated user(s) in the *search.user* table. This is a known bug in the Azure AD connector when Azure AD is configured as the primary authentication realm. To fix this, manually remove the duplicated user(s) from the database.

# Removing the Old Environment
Once the Saga platform is live, one can take down the 2.x environment:

- The 2.x server(s) can be decommissioned
- If Saga is using a Microsoft SQL database that started out as a 2.x clone, then the original 2.x database can now be deleted.