Versions Compared

Old Version 12

changes.mady.by.user svc_ayfie_onedrive@ayfie.com

Saved on

compared with

New Version 13

changes.mady.by.user svc_ayfie_onedrive@ayfie.com

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Markdown
- [Introduction](#introduction)
- [Considerations before Starting](#considerations-before-starting)
- [Prerequisites](#prerequisites)
- [Pre Migration](#pre-migration)
  - [How to Perform Postgres Database Transfer](#how-to-perform-postgres-database-transfer)
  - [How to Perform Microsoft SQL Database Clone](#how-to-perform-microsoft-sql-database-clone)
- [Migration](#migration)
  - [Install Saga](#install-saga)
  - [Database Migration](#database-migration)
    - [Postgres Database Migration](#postgres-database-migration)
    - [Microsoft SQL Database Migration](#microsoft-sql-database-migration)
  - [Replace License](#replace-license)
  - [Install Connectors](#install-connectors)
  - [Assign Connections to New Server](#assign-connections-to-new-server)
  - [Remove Registered Server from Database](#remove-registered-server-from-database)
    - [Postgres Database Remove Registered Server](#postgres-database-remove-registered-server)
    - [Microsoft SQL Database Remove Registered Server](#microsoft-sql-database-remove-registered-server)
  - [Connector Migration Steps](#connector-migration-steps)
  - [Enable Connections](#enable-connections)
  - [Rebuild the Index](#rebuild-the-index)
  - [Assign Roles](#assign-roles)
  - [Search Contexts](#search-contexts)
  - [Content Handler](#content-handler)
- [Known Issues](#known-issues)
  - [Duplicate Users in Database Table](#duplicate-users-in-database-table)
- [Removing the Old Environment](#removing-the-major-version-2-environment)

# Introduction

This guide describes how to migrate from Locator 2.x and/or Supervisor 2.x to the new Saga platform on a Windows 2019 Server, the only OS currently supported by Saga. However, if 2.x is already running on a Windows 2019 Server, one can opt to perform the migration on that very same server.

The reasons why migrating is the preferred option compared to installing from scratch are:
- No need to re-configure all the connections.
- No need to re-fetch all the source data.
- No need to re-add any custom rules, any custom system scopes or any promoted results.
- The users will keep their settings, private search scopes and private tags.
- The analytics data will be retained.
- The Supervisor reports and dashboards will be retained.

The main aspect of the migration is the migration of the 2.x database to the new platform. Before starting, a transfer or clone of the 2.x database is required. The procedure is dependent on the type of database:
- **Postgres**: Do a transfer of the 2.x database to the new server.
- **Microsoft SQL**: Clone the 2.x database. The new platform will connect to the clone.

Once the transfer or the cloning has been completed, one can perform the migration on the new server independently of the existing 2.x installation. The 2.x installation can continue to serve the users until the migration is completed.

# Considerations before Starting
- Is one affected by any of the limitations in the new platform?

  See the *Limitations* section of the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide).
- Is one affected by any of the deprecated features in the new platform?

  See the *Major Version Upgrades* section of the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide).
- Are all connectors available on the Saga platform?

  Verify that all installed connectors are listed in the [Data Sheets](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2928246816/Ayfie+Connector+Data+Sheets).
- Which type of authentication should one configure in the Authority Service?

  If one has Active Directory authentication in 2.x, one should opt for Active Directory in the Authority Service. If one has Azure Active Directory, on should opt for Azure Active Directory in the Authority Service.
- Does the 2.x installation have any customizations?

  Migrating customizations is out of scope for this guide.
- In case of Postgres, how is the database to be transferred to the new server?

# Prerequisites
- The Locator 2.x installation must be upgraded to the latest 2.11 service release before starting on the migration to 5.x. At the time of this writing, that is Locator 2.11 SR10.
- Any Supervisor 2.x installation must be upgraded to the latest 2.4 service release. At the time of this writing, that is Supervisor 2.4 SR5.
- Everything listed in the Prerequisites section of the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide), including a new license compatible with the Saga platform.

# Pre Migration
Before starting [Migration](#migration) a transfer (Postgres) or clone (Microsoft SQL) of the 2.x database is required, consult either [How to Perform Postgres Database Transfer](#how-to-perform-postgres-database-transfer) or [How to Perform Microsoft SQL Database Clone](#how-to-perform-microsoft-sql-database-clone).

## How to Perform Postgres Database Transfer
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Postgres database. Locator and Supervisor will be unavailable to the users while doing these steps.

On the 2.x server, perform the following steps:
- Obtain the 2.x install and data directories. If not known, then they can be retrieved by running these two PowerShell commands respectively.
  ```
  (Get-ItemProperty ("Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Virtual Works\ViaWorks") -Name "InstallDir")."InstallDir"
  (Get-ItemProperty ("Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Virtual Works\ViaWorks") -Name "DataDir")."DataDir"
  ```
- Obtain the database name from the variable *ViaDatabaseName* in the file *C:\Program Files\ayfie\Locator\Config* (assumes default install directory). It will be required later.
- Open the Management Console and disable all connections.
- Stop Default Web Site in the Internet Information Services (IIS) Manager.
- Store the code snippet below in the file *stop-services.ps1* and run it in PowerShell. The script will stop the services in the correct order.
  ```
  $services= Get-Service | Where { $_.DisplayName -like "*ViaWorks*Fetch*" -or $_.DisplayName -like "*Locator*Fetch*" -or $_.DisplayName -like "*ayfie*Fetch*"}
  $serviceNames= @(
    "Via.Authority.Service",
    "Via.ReportEngine.Service",
    "Via.Scheduler.Service"
    "Via.Notification.Service",
    "Via.Resource.Service",
    "ViaWorksIndexBuilder",
    "ViaWorksLingo",
    "Via.Index.Service",
    "Via.ZooKeeper.Service",
    "Via.Licensing.Service"
  )
  foreach ($serviceName in $serviceNames) {
    $service = Get-Service -Name $serviceName -ErrorAction SilentlyContinue
    if ($service.Length -gt 0) {
      $services += $service
    }
  }
  $services| ForEach-Object {
    Stop-Service $_.Name -Verbose
  }
  Write-Host Waiting 30 seconds before stopping database service...
  Start-Sleep -Seconds 30
  Get-Service -Name "ViaWorksDatabaseService" | Stop-Service -Verbose
  ```
- Transfer all files and sub-folders in *C:\ProgramData\ayfie\Locator\Database* (assumes default data directory) to the new server. They will be required later.
- Store the code snippet below in the file *start-services.ps1* and run it in PowerShell. The script will start the services in the correct order.
  ```
  $services= @()
  $serviceNames= @(
    "ViaWorksDatabaseService",
    "Via.Licensing.Service",
    "Via.ZooKeeper.Service",
    "Via.Index.Service",
    "ViaWorksLingo",
    "ViaWorksIndexBuilder",
    "Via.Resource.Service",
    "Via.Notification.Service",
    "Via.Scheduler.Service",
    "Via.ReportEngine.Service",
    "Via.Authority.Service"
  )
  foreach ($serviceName in $serviceNames) {
    $service = Get-Service -Name $serviceName -ErrorAction SilentlyContinue
    if ($service.Length -gt 0) {
      $services += $service
    }
  }
  $services += Get-Service | Where { $_.DisplayName -like "*ViaWorks*Fetch*" -or $_.DisplayName -like "*Locator*Fetch*" -or $_.DisplayName -like "*ayfie*Fetch*"}
  $services| ForEach-Object {
    Start-Service $_.Name -Verbose
  }
  ```
- Start the services again and the Default Web Site in the Internet Information Services (IIS) Manager.
- Open the Management Console and enable all connections.

## How to Perform Microsoft SQL Database Clone
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Microsoft SQL Server database. Locator and Supervisor will be unavailable for the users while doing these steps.

On the 2.x server, perform the following steps:
- Open the Management Console and disable all connections.
- Stop *Default Web Site* in the Internet Information Services (IIS) Manager.
- Clone the Microsoft SQL database. This requires database administrator privileges.
- Grant permissions for the gMSA account to the cloned database:
  - Open Microsoft SQL Management Studio and connect to the database server
  - Select the cloned database
  - Run the following commands after having replaced all instances of \<gMSA Account\> with the the gMSA account:
    ```
      IF NOT EXISTS (SELECT * FROM dbo.syslogins WHERE name = N'<gMSA Account>')
      BEGIN
          CREATE LOGIN [<gMSA Account>] FROM WINDOWS;
      END
      GO

      IF NOT EXISTS (SELECT * FROM [sys].[database_principals] WHERE lower([name]) = lower(N'<gMSA Account>'))
      BEGIN
          CREATE USER [<gMSA Account>] FOR LOGIN [<gMSA Account>];
          EXEC sp_addrolemember 'db_executor', '<gMSA Account>'
          EXEC sp_addrolemember 'db_ddladmin', '<gMSA Account>'
          EXEC sp_addrolemember 'db_datareader', '<gMSA Account>'
          EXEC sp_addrolemember 'db_datawriter', '<gMSA Account>'
      END
      GO
    ```
- Start the services again and the *Default Web Site* in the Internet Information Services (IIS) Manager.
- Open the Management Console and enable all connections.

# Migration
This section covers all migration steps on the new server.

## Install Saga

The first step in migration from 2.x to the new platform is configuring the new Saga platform on a new Windows Server 2019server, consult the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide). This can be done using the default Postgres database, even if the 2.x installation is using Microsoft SQL. The default Postgres database will be replaced by 2.x database in a later migration step. If 2.x is using Microsoft SQL database one should configure gMSA, as it will be required later. One should not install any connectors at this stage, that will be done in a later migration step. If Supervisor is installed in 2.x, one should enable the Report Engine.

One can continue with the [Database Migration](#database-migration) once the Saga platform has successfully been started and is fully up and running. One can either do the configuration of HTTPS and the Authority Service at this time or postpone it until after the migration has been completed.

## Database Migration
The 2.x database must be migrated to the new platform. Depending on which database the 2.x installation is using, one should either do [Postgres Database Migration](#postgrs-database-migration) or [Microsoft SQL Database Migration](#microsoft-sql-database-migration)

### Postgres Database Migration
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Postgres database. If not, consult [Microsoft SQL Database Migration](#microsoft-sql-database-migration).

This section explains how to migrate the database from the 2.x server to the new server using the method described in [Upgrading Data via pg_upgrade](https://www.postgresql.org/docs/current/upgrading.html). The upgrade is required since 2.x is running Postgres version 9.6 and the Saga platform is running Postgres 12.

On the Saga platform server, do the following:
- Stop Saga
- Move the transferred sub-folders and files from 2.x server to folder *D:\Program Files\ayfie\saga\data\postgresql\migration\data*.
- Delete all files and sub-folders in *D:\Program Files\ayfie\saga\data\postgresql\data*. This will delete the database that was created during installation.
- In PowerShell, change directory to *D:\Program Files\ayfie\saga\docker*.
- In PowerShell, run the command below to perform the database migration:
  ```
  docker-compose -f docker-compose.postgres.migration.yml up ayfie-saga-db
  ```
- The migration process has completed when the message `PostgreSQL migration complete.` appears in the PowerShell console. This might take some time depending on the size of the database.
- Exit the container by pressing *Ctrl+C* after the migration process has completed.
- In PowerShell, change directory to *D:\Program Files\ayfie\saga*.
- In PowerShell, start Saga with the configure parameter by running the following command:
  `.\saga.ps1 -configure`
- In the install script menu, make sure that the database name is the same as in 2.x.
- *Exit* the install script menu.
- Wait for Saga to start successfully
- Review the *Ayfie.Saga.DatabaseTool.log* to confirm that the database migration scripts ran successfully after Saga started successfully.
- Delete all files and sub-folder from folder *D:\Program Files\ayfie\saga\data\postgresql\migration\data*

### Microsoft SQL Database Migration
This section covers the case of Locator 2.11 and Supervisor 2.4 using the Microsoft SQL Server database. If that is not the case, consult [Postgres Database Migration](#postgres-database-migration) or continue to [Migration](#migration).

On the Saga platform server, do the following:
- Stop Saga
- Delete all files and sub-folders in *D:\Program Files\ayfie\saga\data\postgresql\data*. This will delete the database that was created during installation.
- In PowerShell, start Saga with the configure parameter by running the following command:
  `.\saga.ps1 -configure`
- In the install script menu, configure Saga to use the cloned Microsoft SQL database.
- *Exit* the install script menu.
- Wait for Saga to start successfully
- Review the *Ayfie.Saga.DatabaseTool.log* to confirm that the database migration scripts ran successfully after Saga started successfully.

## Replace License
- Start the Management Console and go to *Operations*->*License*.
- Replace the old 2.x license(s) with new Saga platform license(s).
- Make sure the new license has the *OmniPage OCR* converter included.

## Install Connectors
- Install all connectors from 2.x installation, see section *Installing Connectors* of the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide).

## Assign Connections to New Server
- Start the Management Console and go to *Connections*->*Scheduling*
- For Each *Connector*->*Connection*
    - Click *Add Server Assignment* for *AYFIE-CONNECTOR*.
    - Click *Save*.
    - Enable *Perform Discovery* for *AYFIE-CONNECTOR* and disable *Perform Discovery* for the 2.x server.
    - Click *Save*.
    - Click *Delete* for all 2.x servers.
    - Click *Save*.

## Remove Registered Server from Database
Depending on which database the installation is using, one should either do [Postgres Database Remove Registered Server](#postgres-database-remove-registered-server) or [Microsoft SQL Database Remove Registered Server](#microsoft-sql-database-remove-registered-server).

### Postgres Database Remove Registered Server

If using Postgres database, perform the following steps to remove the registered 2.x server(s) from the database:
- Log in to pgAdmin, select the *Locator* database and open the Query Tool.
- Run Query
  ```
  delete from config.via_works_windows_service
  where server_id not in (select server_id from config.via_works_server where computer_name = 'AYFIE-CONNECTOR');

  delete from config.connector_status
  where connector_id in (
    select connector_id
    from config.connector
    where server_id not in (select server_id from config.via_works_server where computer_name = 'AYFIE-CONNECTOR')
  );

  delete from config.connector
  where server_id not in (select server_id from config.via_works_server where computer_name = 'AYFIE-CONNECTOR');

  delete from config.server_server_role
  where server_id not in (select server_id from config.via_works_server where computer_name = 'AYFIE-CONNECTOR');

  delete from config.via_works_server
  where server_id not in (select server_id from config.via_works_server where computer_name = 'AYFIE-CONNECTOR');
  ```

### Microsoft SQL Database Remove Registered Server
If using Microsoft SQL database, perform the following steps to remove the registered 2.x server(s) from the database (2.x server(s) are all servers not named *AYFIE-CONNECTOR*):
- Open Microsoft SQL Server Management Studio
- Select *Locator* database and open a *New Query*
- Run Query
  ```
  delete from [config].via_works_windows_service
  where server_id not in (select server_id from [config].via_works_server where computer_name = 'AYFIE-CONNECTOR');

  delete from [config].connector_status
  where connector_id in (
    select connector_id
    from [config].connector
    where server_id not in (select server_id from [config].via_works_server where computer_name = 'AYFIE-CONNECTOR')
  );

  delete from [config].connector
  where server_id not in (select server_id from [config].via_works_server where computer_name = 'AYFIE-CONNECTOR');

  delete from [config].server_server_role
  where server_id not in (select server_id from [config].via_works_server where computer_name = 'AYFIE-CONNECTOR');

  delete from [config].via_works_server
  where server_id not in (select server_id from [config].via_works_server where computer_name = 'AYFIE-CONNECTOR');
  ```

## Connector Migration Steps
- Replace the server name with the FQDN for all connections that are communicating with Active Directory. That includes:
  - The FileServer connector.
  - All database connectors indexing data from a Microsoft SQL database.
- If applicable, perform steps 10 and 11 from section *Upgrading from major version 4 to 5* of the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide).

## Enable Connections
- Start Management Console and go to *Connections*
- For each connector, enable all connections.

## Rebuild the Index
- In PowerShell, rebuild the index by running the following command:
  `docker exec -it (docker ps -qf "name=^ayfie-locator$") c:\Locator\Tools\Via.Repository.exe reindex /all`

## Assign Roles
- The Authority Service has been replaced in the new platform. Any Supervisor roles assigned in 2.x, needs to be re-assigned in the new platform, see section *Managing Users and Roles* in the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide).

## Search Contexts
On the new server, open URL http://localhost/Dashboard/Configuration/SearchContextList?se=1 and check if any search contexts have been configured. If yes, then add the *Timeline* refiner to all search contexts.

## Content Handler
If the Content Handler is required, then the Content Handler must be deployed across all clients. Deploying the Content Handler will automatically remove the Document Handler (the 2.x equivalent of the Content Handler). Hence, this step must be timed with going live on the new platform. How to deploy the Content Handler is described in the *Prerequisite* section of the [Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2400714758/Ayfie+Locator+Installation+Guide).

# Known Issues

## Duplicate Users in Database Table
The occurrence of the error message *Database upgrade script '50005.sql' failed* in the log file *Ayfie.Saga.DatabaseTool.log* indicates that there are duplicated user(s) in the *search.user* table. This is a known bug in the Azure AD connector when Azure AD is configured as the primary authentication realm. To fix this, manually remove the duplicated user(s) from the *search.user* table in the database and restart Saga. Consult support for assistance if necessary.

# Removing the Old Environment
Once the Saga platform is live, one can take down the 2.x environment:

- The 2.x server(s) can be decommissioned
- If Saga is using a Microsoft SQL database that started out as a 2.x clone, then the original 2.x database can now be deleted.