| Markdown |
|---|
# The Locator 6.1.0.2 Installation Guide - [Introduction](#introduction) - [Target Audience](#target-audience) - [Architectural Overview](#architectural-overview) - [Limitations](#limitations) - [Browser Support](#browser-support) - [End User License Agreement](#end-user-license-agreement) - [Prerequisites](#prerequisites) - [Gateway Hostname](#gateway-hostname) - [Hardware Requirements](#hardware-requirements) - [Operating System](#operating-system) - [Content Handler](#content-handler) - [Anti-Virus Programs](#anti-virus-programs) - [Host Server Ports](#host-server-ports) - [License](#license) - [HTTPS](#https) - [gMSA Account](#gmsa-account) - [User Authentication](#user-authentication) - [Deciding Authority Service Authentication Method](#deciding-authority-service-authentication-method) - [Active Directory Prerequisites](#active-directory-prerequisites) - [Azure Active Directory Prerequisites](#azure-active-directory-prerequisites) - [Microsoft SQL Server](#microsoft-sql-server) - [Firewall Openings](#firewall-openings) - [IP Blocking](#ip-blocking) - [Prerequisites for Optional Features](#prerequisites-for-optional-features) - [Installing](#installing) - [Installation and Restart Time](#installation-and-restart-time) - [Installation Drives and Directories](#installation-drives-and-directories) - [Running the Install Script](#running-the-install-script) - [The Install Script Menu](#the-install-script-menu) - [Saga](#saga) - [Database](#database) - [Gateway](#gateway) - [Languages](#languages) - [Solr](#solr) - [Report Engine](#report-engine) - [Lingo](#lingo) - [Metrics](#metrics) - [Personal Assistant](#personal-assistant) - [Smart Classifier](#smart-classifier) - [Post-Install Configuration](#post-install-configuration) - [Configuring the Authority Service](#configuring-the-authority-service) - [Active Directory](#active-directory) - [Azure Active Directory](#azure-active-directory) - [Managing Users and Roles](#managing-users-and-roles) - [Connector Management](#connector-management) - [Installing Connectors](#installing-connectors) - [Configuring Connectors](#configuring-connectors) - [Upgrading Connectors](#upgrading-connectors) - [Uninstalling Connectors](#uninstalling-connectors) - [Secondary User Authentication](#secondary-user-authentication) - [Custom Branding](#custom-branding) - [Private Certificate Authority](#private-certificate-authority) - [Major Version Upgrades](#major-version-upgrades) - [Upgrading from major version 5 to 6](#upgrading-from-major-version-5-to-6) - [Upgrading from major version 4 to 5](#upgrading-from-major-version-4-to-5) - [Upgrading from major version 3 to 4](#upgrading-from-major-version-3-to-4) - [Starting and Stopping](#starting-and-stopping) - [Stopping](#stopping) - [Starting](#starting) - [Uninstall](#uninstall) - [User Interfaces](#user-interfaces) - [Search UI](#search-ui) - [Report Engine UI](#report-engine-ui) - [The Admin Dashboard](#the-admin-dashboard) - [The Management Console](#the-management-console) - [The Authority Service Admin Console](#the-authority-service-admin-console) - [The Solr Dashboard](#the-solr-dashboard) - [The Grafana Dashboard](#the-grafana-dashboard) - [The PostgreSQL pgAdmin Dashboard](#the-postgresql-pgadmin-dashboard) - [RestService API](#restservice-api) - [Known Issues](#known-issues) - [Losing Connection to Docker Daemon](#losing-connection-to-docker-daemon) - [Saga Not Starting Due to docker-compose Upgrade Failure](#saga-not-starting-due-to-docker-compose-upgrade-failure) - [Issue When Uninstalling Major Version 3 Connectors](#issue-when-uninstalling-major-version-3-connectors) - [Duplicated Client Mappers in Authority Service](#duplicated-client-mappers-in-authority-service) - [Firewall Blocks Communication between Containers](#firewall-blocks-communication-between-containers) # Introduction Locator is an enterprise search engine able to retrieve and index a large variety of office type documents and make these searchable. The Locator product runs on top of the Saga Platform. The name Saga may thus sometimes appear where one otherwise would expect the name Locator. Compared to Locator 2.x, the 6.1.0.2 version of Locator has many new customer facing features. However, the most significant change is that Locator 6.x, as all versions since 3.x, is running on top of Docker. ## Target Audience This guide is directed towards system administrators that are to install Locator 6.1.0.2. ## Architectural Overview For an architectural overview, consult [Saga Architectural Overview](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2180251662/Ayfie+Saga+Architectural+Overview) ## Limitations Not all features and use cases supported in 2.x are yet supported. Locator 6.1.0.2 only supports... - ...new installations and upgrades from 3.x and later versions (consult [Migration from Version 2.x to the Latest](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3312254977/Migration+from+Version+2.x+to+the+Latest) for how to manually migrate to the new platform) - ...single node installations. - ...online installations. It is not possible to neither install nor restart Locator if it is not online. - ...some of the many connectors that were supported in 2.11 (but new connectors are released all the time). - ...no support for integrating the search UI components into other web-based software. - ...default Solr configuration, no custom Solr *schema* or *solrconfig* overrides. ## Browser Support Locator works with all up to date modern browsers. Microsoft IE is not supported. ## End User License Agreement By installing the software one is accepting the [End User License Agreement](https://www.ayfie.com/hubfs/terms/eula.pdf). # Prerequisites ## Gateway Hostname The first prerequisite to complete is to decide on a Gateway Hostname as some prerequisites depend on knowing the Gateway Hostname. The Gateway Hostname is the FQDN used to access Saga applications: * https://\<Gateway Hostname\>/search * https://\<Gateway Hostname\>/report-engine These prerequisites/configurations depend on knowing the Gateway Hostname: * [HTTPS](#https) * [gMSA Account](#gmsa-account) * [User Authentication](#user-authentication) The installer will by default use the host server's FQDN as the Gateway Hostname. One can however change the Gateway Hostname to some other FQDN with a CNAME, an ALIAS or an A Record type DNS entry. ## Hardware Requirements Locator has the following resource consumption characteristics: - Indexing and search are disk I/O intensive operations that require a fast disk. It is recommended using an NVMe SSD disk for the D:\ drive where Locator is installed. - Data conversion (OCR in particular), entity extraction (Lingo) and indexing are all CPU intensive operations that will benefit greatly from fast CPUs. - Data retrieved from the data sources and converted to a textual format is first stored in an internal (PostgreSQL) or external (SQL Server) database before being indexed and becoming part of the searchable index (Solr). This means that Locator stores retrieved data twice, first in the database and then in the index. The use of an internal vs. an external database will thus have a significant impact on how much disk space that is required by the Locator host. The table below assumes the use of an internal PostgreSQL database and the amount of required disk space will be significantly less if one instead uses an external SQL Server database. - Locator is multi-threaded with many memory demanding processes running in parallel. Having enough memory is therefore essential. Initial indexing is more resource demanding than later operations. The table below aims at covering the peak resource demand. The C: drive, that holds both the Windows OS and the Docker framework, must have a minimum size of 150 GB. The Locator application should be installed on a separate drive (normally the D: drive). Its minimum disk size requirement is given by the Locator Drive Size column in the table below. |Documents |CPU Speed|CPU Cores |RAM |Locator Drive Size| |:-----------------------:|:-------:|:----------:|:------:|:------------------------:| |0-10,000 | ~2 GHz| 4| 16 GB| 50 GB| |10,000-100,000 | ~2.5 GHz| 4| 16 GB| 100 GB| |100,000-500,000 | ~3 GHz| 4| 24 GB| 100-250 GB| |500,000-2,000,000 | 3+ GHz| 8| 40 GB| 250-500 GB| |2,000,000-10,000,000 | 3+ GHz| 16| 64 GB| 500-1000 GB| |>10,000,000 | 3+ GHz| 16| 88+ GB| 1+ TB| By default Locator operates with 1 Lingo thread. Each Lingo thread has a cost of up to 8 GB of RAM. If one were to adjust the number of these threads or adjust the memory limit on a thread (or disables Lingo altogether), the resulting change in RAM usage should be taken into consideration. ## Operating System Windows Server 2019 ## Content Handler The Content Handler is a Windows only utility, used to open some search result links in their native applications on the client. These are links referencing data retrieved by any of the connectors listed below. Without the Content Handler installed, these links will have limited opening functionality. * eDOCS DM * Exchange (on-premises version only) * FileServer * WorkSite The Content Handler can either be downloaded by each individual user from the Locator search UI, in which case there are no prerequisites. Alternatively a system administrator can download an [MSI installer](https://nexus.ayfie.dev/repository/Raw-Hosted/contenthandler-releases/5/ContentHandler.msi) and roll the Content Handler out to all users in one operation. The system administrator can distribute the MSI installer using this command: ``` msiexec /quiet /i ContentHandler.msi APPLICATIONFOLDER="C:\Program Files (x86)\ayfie\Content Handler" ``` Note that the installer will install .NET Framework 4.7.2 if not already installed at the client. ## Anti-Virus Programs Anti-virus programs should preferably be stopped. If that is not possible, at a very minimum all Docker and Locator related directories should be exempted. Assuming a default Docker installation and recommended Locator installation, these are the directories (include sub-directories) that should be excluded: - D:\Program Files\ayfie\saga - C:\ProgramData\docker There are some anti-virus programs that are completely incompatible with Locator and should not be run on the host machine under any circumstances. Here is a non-exhaustive list of such programs: - Trend Micro Antivirus - Cynet - WebRoot SecureAnywhere - BitDefender ## Host Server Ports The host server's ports 80 and 443 are used by Saga. Any other application or service using those ports (for instance IIS) will interrupt Saga and thus need to be stopped or reconfigured. ## License A valid license is required to enable features and capabilities once the initial 30 day free trial period expires. Licenses are added using the [Management Console](#the-management-console). The features that depend on the license are: * Search UI - New users can log in as long as the license has free user slots. * Index - Documents will be added to the index as long as the document volume limit is not exceeded. * Converter - Requires the DocFilter and OmniPage OCR capability. * Lingo - Requires either Lingo Standard or Lingo GDPR (PII) capability, see [Lingo](#lingo). * Report Engine - Requires the Report Engine capability. * Connectors - Each connector have their own capability. This capability is required for the connector to start. * Personal Assistant - Requires the Personal Assistant capability. * Smart Classifier - Requires the Smart Classifier capability. ## HTTPS The Search and Report Engine UIs require the use of the HTTPS protocol, To facilitate that, Locator needs to be set up with an SSL certificate. The prerequisite for this depends on how the certificate is obtained: - Received from a commercial Certificate Authority (CA) such as for instance IdenTrust, DigiCert, Sectigo, GoDaddy, etc. - Automatically generated by Let’s Encrypt during the Locator installation The prerequisites in regard to the first option are: - two PEM formatted files, \<name\>.crt and \<name\>.key. - the \<name\>.crt has [Gateway Hostname](#gateway-hostname) set in the Subject Alternative Name attribute. This applies even if there technically are no alternative names. - the \<name\>.key file has an unencrypted private key. See [Saga SSL Certificate Management](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2987327489/Saga+SSL+Certificate+Management) for details. The two files are to be placed into directory _D:\Program Files\ayfie\saga\volumes\Traefik\certs_ (the given path assumes the recommended install directory) during installation. The prerequisites in regard to the second option of using Let’s Encrypt's are all network related: - One must be using an approved public Top Level Domain (TLD) - The Locator server must be exposed to the public Internet - Port 80 of the Locator host's firewall must be open - There must be a DNS A record entry for the Locator host FQDN and its public IP ## gMSA Account A *group Managed Service Account (gMSA)* is required if **any** of the following is true: - Saga is set up with a [Microsoft SQL Server](#microsoft-sql-server) database. - Saga is set up with one or more connectors that access resources over the Windows-based network (for instance by reading files from the disk of another server). - Authority service is set up with [Active Directory](#active-directory) user authentication. - Authority service is set up with [Azure Active Directory](#azure-active-directory) user authentication and the [Saga AD AAD Sync](#saga) setting is enabled in the Saga install script. For instructions on how to create the gMSA account, see [Create gMSA Account](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3343745025/Create+gMSA+Account) ## User Authentication Once Saga has been installed one needs to configure the Authority Service (Keycloak) with the authentication method that is to be used as instructed in the *Post-Install Configuration* section further down. The two options are Active Directory and Azure Active Directory, each option with a different set of prerequisites (see the two sections). ### Deciding Authority Service Authentication Method The recommended authentication method to use in the Authority Service is Azure Active Directory for the following reasons: - Easier to configure. - Enables 2 Factor Authentication (2FA). However, it is not always possible to use the recommended authentication method as some connectors are not compatible with that approach. One should for that reason consult the [User Authentication Method Matrix](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3327950849/User+Authentication+Method+Matrix) before determining which authentication method to configure. ### Active Directory Prerequisites These are the Active Directory prerequisites: - The LDAP connection URL - The full distinguished name (DN) of the LDAP tree where users are stored. - An Active Directory service account named *svcSagaKeycloak* with Domain User permissions - The service account Bind DN string See the Prerequisites section of [Configure Active Directory user federation in Authority Service](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578241/Configure+Active+Directory+user+federation+in+Authority+Service) for more details. ### Azure Active Directory Prerequisites These are the Azure Active Directory prerequisites: - An Azure Active Directory client ID - The corresponding Azure Active Directory client secret - The OpenID Connect metadata document URL See the Prerequisites section of [Configure Azure Active Directory identity provider in Authority Service](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578248/Configure+Azure+Active+Directory+identity+provider+in+Authority+Service) for more details. ## Microsoft SQL Server All data retrieved by the connectors is converted to text and stored in a database before being indexed. Saga has a built-in PostgreSQL database that is used by default. However, an external Microsoft SQL Server can be used instead. The pros and cons of using either of the two options are discussed in [Built in PostgreSQL vs. External Microsoft SQL Server](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2941812766/Built-in+PostgreSQL+vs.+External+Microsoft+SQL+Server). If one opts for using an external Microsoft SQL Server database, see [Configure with an external Microsoft SQL Database Server](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2374336517/Configure+with+an+external+Microsoft+SQL+Database+Server) for how to prepare the database before installing Saga. In addition, as the [Management Console](#the-management-console) depends on the database for it functions, users that are to operate it must be assigned the *db_datareader* and *db_datawriter* roles for the Locator database. ## Firewall Openings It is not possible to install and operate Locator without access to the Internet over port 443 (TCP protocol). It is however possible to reduce the external access from the host to only sites being used by Locator. Consult the full list of required [firewall openings list of required [firewall openings](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3075112967/Ayfie+Saga+Firewall+Openings). ## IP Blocking It is not possible to do IP blocking if one uses an SSL certificate from Let's Encrypt. ## Prerequisites for Optional Features The following optional features have their own required prerequisites: - **Connectors** - Each individual connector has its own set of prerequisites that can be looked up at [Connector Data Sheets](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2928246816/Ayfie+Connector+Data+Sheets). - **Personal Assistant** - See [Configure Personal Assistant](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3410165780/Configure+Personal+Assistant). - **Smart Classifier** - See [Smart Classifier ML Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/30751129673228499969/Ayfie+Smart+Classifier+SagaML+FirewallInstall+OpeningsGuide). ## IP Blocking It is not possible to do IP blocking if one uses an SSL certificate from Let's Encrypt. # Installing New versions of the Saga platform are released using the continuous delivery approach. The Saga platform is versioned using semantic versioning. When a new minor versions is released, running the saga.ps1 script will automatically result in an upgrade to that new version. When a new major version is released, a new version of saga.ps1 must be downloaded to replace the existing saga.ps1, and be executed according to the [upgrade instructions](#major-version-upgrades) given for that new major version. ## Installation and Restart Time Locator runs as a set of Docker containers that are instantiations of Docker images that are downloaded the first time Locator starts up. The time to download these images will vary depending on the amount of changes, and is likely to take some 15-20 minutes the first time (and "hang" for a few minutes after reporting that all images have been downloaded). ## Installation Drives and Directories Locator is a Docker based application and Locator installations consist of two parts, the Docker framework and the Locator application running on top of it. The Docker framework can only be installed on the C: drive (it can later be moved to another drive but that is outside the scope of this documentation). The Locator application should be installed on a different drive than the Windows OS. The D: drive is what is normally used. The minimum size of the two drives is given in [Hardware Requirements](#hardware-requirements). Any directory can be used to house the Locator application. However, the convention is to use the same directory structure as used by most other Windows installations: ``` D:\Program Files\ayfie\saga ``` In the rest of this documentation we will assume that the directory given above is the one used for the installation. ## Running the Install Script To install Locator, run the command lines shown below one by one from within an elevated PowerShell shell (do **not** run them from within PowerShell ISE): ``` > mkdir "D:\Program Files\ayfie\saga" > cd "D:\Program Files\ayfie\saga" > Invoke-WebRequest -Uri https://nexus.ayfie.dev/repository/Raw-Hosted/locator-releases/6/saga.ps1 -OutFile saga.ps1 > Get-ChildItem saga.ps1 | Unblock-File > .\saga.ps1 ``` Above we see the install directory being entered, the *saga.ps1* installation script being downloaded, digitally signed and then finally started. One will be prompted to accept the [End User License Agreement](https://www.ayfie.com/hubfs/terms/eula.pdf) to which one has to respond *Y* in order to continue: ``` The Saga End User License Agreement (EULA) is found at: https://www.ayfie.com/hubfs/terms/eula.pdf Do you accept the EULA? [Y/n]: ``` If Docker is not already installed, then the server will be restarted upon the install script having installed it. Run the script all over again once the server is back up. A successful installation will have the installer script terminate by showing the graphic below: ```___ / ___/____ _____ _____ _ \__ \/ __ `/ __ `/ __ `/ Welcome to Saga ___/ / /_/ / /_/ / /_/ / Start time: YYYY-MM-DD HH:MM:SS /____/\__,_/\__, /\__,_/ /____/ ``` Running the *docker ps* command should show *healthy* (and not *starting* or *unhealthy*) in the *STATUS* column (if the container has that type of information at all). ## The Install Script Menu The *saga.ps1* script is not only used as an install script. It can also be used for changing some of the Locator configurations at any time after the installation. Only the first time the script is run will the menu appear automatically. However, at any later time one can use the *-Configure* flag for the menu to be shown: ``` > .\saga.ps1 -Configure ``` All configurations performed using the menu are persisted in the file *D:\Program Files\ayfie\saga\docker\custom.env*. Below we see the different sections of the installation/configuration menu displayed during installation. Looking at the menu further below, notice how the first word of each line defines the various sections: - **Saga** - The Locator solution runs on top of the Saga Platform. In this section one finds parameters that does not naturally fit into any of the other categories below. - **Database** - Settings related to the database used by Locator to store configurations and retrieved and converted data that is to be indexed. - **Gateway** - Settings related to the hostname and HTTPS that is required by the UIs. - **Languages** - Settings related to language functionality. - **Solr** - Settings related to the search index. - **Lingo** - Settings related to the linguistic functionality. - **Report Engine** - Settings related to the Report Engine. - **Metrics** - Settings related to the collection and display of Locator performance and health data. - **Personal Assistant** - Settings related to the Personal Assistant functionality. - **Smart Classifier** - Settings related to the Smart Classifier functionality. - **Advanced** - Advanced Locator configuration menu, changes in advanced setting should only be done after contacting support. In the following sections we will cover these menu sections one by one. The full menu looks like this: ``` Your customized configuration values. Please choose from the list below to change. Saga AD Service Account: Saga AD AAD Sync: false Saga Auto Restart Saga Branding: ayfie Saga PreReleaseTag: Database Type: POSTGRES Database Server: ayfie-saga-db Database Port: 5432 Database Name: Locator Gateway Host Name: Gateway Certificate Name: Gateway Certificates Resolver: Gateway Acme Email: Languages Converter: en;nb Languages Index: en;nb Solr Java Memory Setting: -Xms4g -Xmx4g Lingo Disable: False Lingo Extraction Image Id: nb Lingo Pipeline Pool Size: 2 Report Engine Disable: True Metrics Disable: False Personal Assistant Smart Classifier Advanced == Exit == ``` The values we see at the end of each line are default settings. A few are blank and some of those have to be set. The script will keep returning to the menu upon exiting it until all required fields are set. Use the arrow keys to move up and down to the field that one wants to set and then press enter to edit, or as in some cases, toggle the field value. ### Saga - **Saga AD Service Account:** Enter the gMSA account name with a dollar sign tagged to the end of it (corresponds to the SAM account name). That is, if the account name is *locator* as in our example above, then enter it as *locator$*. See section [gMSA Account](#gmsa-account) for further details. - **Saga AD AAD Sync:** Determines whether or not Active Directory authorization will be performed when the Authority Service is set up with Azure Active Directory. Set it to *true* if users are synchronized by [Azure AD Connect](https://learn.microsoft.com/en-us/azure/active-directory/hybrid/whatis-azure-ad-connect) or otherwise having matching *userPrincipalName* properties across Active Directory and Azure Active Directory; otherwise set it to *false*. - **Saga Auto Restart** - Menu used to set up automatic Locator restart and thus automatic upgrade to the latest minor or patch version. - **Daily/weekly** - Determines the time unit of the next input field - **Days/weeks between restarts** - Number of days/weeks between restarts - **Time of restart** - The time of the restart given as HH:MM - **Restart days** - Set which days to restart (appears only if "weekly" has been chosen above) - **User password** - The password of the user running the installer is required for later running the scheduled restart script - **== Remove ==** - Removes any existing restart schedule - **== Save ==** - Saves the schedule and then exits - **== Back ==** - Exits without saving - **Saga Branding:** Set to *custom* to change the branding theme. The default theme is *ayfie*. The changed setting must be accompanied by the execution of the steps described in [Custom Branding](#custom-branding). If these steps are carried out afterwards, a restart of Saga is required. - **Saga PreReleaseTag:** This parameter is for internal use leading up to a new release. ### Database All data retrieved by the Locator's connectors is stored in a database before being indexed. Locator has a built-in PostgreSQL database and the default settings relate to that database. By changing the settings one can instead use an external Microsoft SQL Server database. - **Database Type:** The type of database, select POSTGRES or MSSQL from the drop down. - **Database Server:** If POSTGRES, *ayfie-saga-db*. If MSSQL and a named database instance, set to *\<FQDN>\\<name>*. If not named, set to the FQDN of the databases's host server. - **Database Port:** The port that the database can be reached on. If MSSQL and a named database instance, set to 0. Otherwise, set the port number. - **Database Name:** The name of the database. ### Gateway The table below shows the field values to be set based on the type of certificate one is using (see section [HTTPS](#https) for further details): | | Certificate Authority provided | Let's Encrypt generated | |:-----------------------------------|:--------------------------------------|:---------------------------------------| | **Gateway Host Name:** | [Gateway Hostname](#gateway-hostname) | [Gateway Hostname](#gateway-hostname) | | **Gateway Certificate Name:** | the \<name\> of \<name\>.crt | | | **Gateway Certificates Resolver:** | | "letsencrypt" | | **Gateway Acme Email:** | | contact email | A quoted value means that it is to be used as-is (but without the quotes). Empty fields means to leave the menu field empty. ### Languages It's recommended that the same set of languages are configured for both Converter and Index Languages. The following language codes are supported: *bg* (Bulgarian), *cs* (Czech), *da* (Danish), *nl* (Dutch), *en* (English), *fi* (Finnish), *fr* (French), *de* (German), *it* (Italian), *nb* (Norwegian), *pl* (Polish), *pt* (Portuguese), *ru* (Russian), *es* (Spanish), *sv* (Swedish) and *uk* (Ukrainian). Changes to the language configuration will only apply to documents that are updated after the change. A full reindex is thus recommended when changing *Languages Index*. A full refetch can be considered when changing *Languages Converter*. - **Languages Converter:** The converter operates with language specific lists of characters that define the set of characters that can be detected during OCR. Specifying the languages of the data set will thus improve the accuracy of the OCR process. - **Languages Index:** The (Solr) index uses the configured languages to decide which languages that can be detected. A document's detected language is the bases for further linguistics analyses upon it such as stop word filters, stemming, etc. ### Solr Solr is the actual search engine within Locator. - **Solr Java Memory Setting:** In general, Java Virtual Machines (JVM) are typically not good at freeing up memory once it has allocated it. For that reason it is important that one set the Solr Java Memory limit low enough to avoid other components from running out of memory, while at the same ensuring that Solr has enough memory. The default value is 4 GB which is only enough for small indexes with relative few documents. The suggested memory limits in the table below are based on the number of indexed documents. These limits should be considered a rule of thumb and increased further if Solr were to run out of memory. |Documents | Solr Java Memory | |:-----------------------:|:----------------:| |0-100,000 | 2 GB| |100,000-500,000 | 4 GB| |500,000-2,000,000 | 8 GB| |2,000,000-10,000,000 | 16 GB| |>10,000,000 | 16+ GB| ### Lingo Lingo is a Natural Language Processing (NLP) component that extracts two types of entities from the incoming data, Personally Identifiable Information (PII) like passport numbers, social security numbers, driver's license number's, etc., and more general and less sensitive entities such as dates, names of persons, locations, organizations, etc. There are two types of images for these two types of entity extraction, the GDPR (PII) extraction image and the standard extraction images. Only one extraction image can be used at any given time. The Report Engine requires the GDPR extraction image. If one is not to use the Report Engine, then select a standard extraction image based on the predominant language of the data set. - **Lingo Disabled:** This parameter is the master switch. If toggled to *True*, Lingo will be disabled. By default the value is *False*. - **Lingo Extraction Image Id:** This parameter determines which Lingo extraction image to be used. That can be one of the standard extraction images: da (Danish), en (English), fr (French), de (German), it (Italian), nb (Norwegian), pl (Polish), pt (Portuguese), es (Spanish) and sv (Swedish), or the GDPR extraction image: PII. The language in parentheses, like English in *PII (English)*, indicates the language dependent date format to be used during extraction. Note that if the extraction image identifier is changed after install time, a full re-index would have to be performed. - **Lingo Pipeline Pool Size:** The pipeline pool size is the maximum number of pipelines that Lingo may use at any given time. A pipeline corresponds to a processing thread. Each pipeline normally consumes about 2 GB of memory and never more than the memory threshold explained next. ### Report Engine The purpose of the report engine is to create reports, typically data compliance reports that identifies the presence of any sensitive information within the data that has been indexed. For more information on the report engine, consult [this article](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2928640017/What+is+Ayfie+Supervisor). - **Report Engine Disable** If set to *True*, the Report Engine will be disabled. By default the value is *True*. ### Metrics Locator provides two ways of reporting performance and health metrics: - External cloud based reporting using Datadog. This option is referred to as Telemetry and is disabled by default. - Product internal reporting via the Grafana Dashboard. This option is referred to as Metrics and is enabled by default. None, one or both can be used at the same time. - **Metrics Disabled:** False by default. Toggle to disable. ### Personal Assistant Personal Assistant allows users to chat with large language models and to interrogate documents using these same models. To enable the Personal Assistant, three models have to be configured in Azure, consult [Configure Personal Assistant](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3410165780/Configure+Personal+Assistant) for details. The three models to be configured are: - **Main Model** - The default model used for chat. - **High Quality Model** - The high quality model used for chat. - **Embeddings Model** - The model for creating a numeric representation of the text. Only *text-embedding-ada-002* is currently supported. In the Personal Assistant menu one needs to configure the following: - **Personal Assistant Mode:** Defines the criteria that is used to determine which users that will have access to which flavor of the Personal Assistant feature. These are the three alternatives: - **off** - the default value. The Personal Assistant feature is disabled for everyone. - **limited** - enables the Personal Assistant only for users with the *search.personal_assistant.premium* role, see [Managing Users and Roles](#managing-users-and-roles). - **full** - allows all users access to the Personal Assistant. User without the *search.personal_assistant.premium* role will have access to the Main Model and users with the role will have access to both the Main Model and the High Quality Model and can switch between two in the user interface. - **Main Model Deployment:** The name of the main model deployment in Azure. - **Main Model Display Name: GPT-3.5** The display name of the main model in the Search UI. - **Main Model API Address:** The API Address of the main model deployment in Azure. - **Main Model API Key:** The API Key of the main model deployment in Azure. - **Main Model Token Size:** The maximum number of input tokens to be used with the main model. A model has a maximum total number of tokens that can be used across the input and the output. The input is made up of questions and uploaded documents and only what is not used in the input can be used for the output. It is thus important to select a value within the overall capacity of the model that still leaves room for the output. The *Recommended Token Size* in the table below is the recommendation for such a number. It is also worth noting that one is billed by the total number of tokens used in the input, so reducing the token size will reduce the operational cost of the feature. |Model |Max Tokens | Recommended Token Size | |:---------------:|:---------:|:----------------------:| |gpt-3.5-turbo | 4096| 3500| |gpt-35-turbo-16k | 16384| 15000| |gpt-4 | 8192| 7500| |gpt-4-32k | 32768| 30000| - **High Quality Model Deployment:** The name of the high quality model deployment in Azure. - **High Quality Model Display Name: GPT-4** The display name of the high quality model in the Search UI. - **High Quality API Address:** The API Address of the high quality model deployment in Azure. - **High Quality API Key:** The API Key of the high quality model deployment in Azure. - **High Quality Token Size:** The maximum number of input tokens of the high quality model deployment in Azure. The same information given regarding the Main Model Token Size above applies for this one as well. - **Embeddings Model Deployment: text-embedding-ada-002** The name of the embeddings model deployment in Azure. - **Embeddings Model API Address:** The API Address of the embeddings model deployment in Azure. - **Embeddings Model API Key:** The API Key of the embeddings model deployment in Azure. - **Data Expiration Time In Seconds: 360000** All uploaded data older than the expiration time will be removed when starting new session. The default is 360000 (10 hours). ### Smart Classifier The Smart Classifier is a feature for users to create, train and activate machine learning models to be used on sets of indexed documents. The Smart Classifier feature requires the Smart Classifier ML services, consult [Smart Classifier ML Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3228499969/Ayfie+Smart+Classifier+ML+Install+Guide). - **Smart Classifier Disable: True** Set to *False* to enable the Smart Classifier feature. - **Smart Classifier Worker Count: 4** Kafka Consumer count used by Smart Classifier. - **Smart Classifier ML Service Address:** The FQDN used to access the Smart Classifier ML services, https://\<Gateway Hostname\>, where Gateway Hostname is set in the Smart Classifier ML installer. - **Smart Classifier ML Basic Auth Password:** The *Gateway Basic Auth Password* set in the Smart Classifier ML installer. The Smart Classifier feature is only accessible by users with the appropriate role, see [Managing Users and Roles](#managing-users-and-roles). # Post-Install Configuration ## Configuring the Authority Service The Authority Service is a Saga user authentication and management solution built on the Keycloak platform. Administrative activities are performed in the Authority Service Admin Console. See [The Authority Service Admin Console](#the-authority-service-admin-console) for details on how to log in. The Authority Service authenticates and authorizes users across all products in the Saga platform. During authentication, the Authority Service looks the user up in its internal user database. If the user is not to be found there, the Authority Service will then search for the user across all user storage providers that it has been configured to use. There are two types of user storage providers: - **Federated user databases** are user databases that Saga communicates with using the LDAP protocol. Saga itself provides the login page and then passes user credentials on to the federated user database for authentication. [Active Directory](#active-directory) is, and will remain, the only supported user database of this type. - **Identity providers** are third party user storage providers that Saga communicates with using protocols such as OpenID Connect. Login is done by Saga passing the user on to the identity provider's login page to enter the user's credentials there. [Azure Active Directory](#azure-active-directory) is currently the only identity provider supported by Saga, but others are to follow in the future. In either case, users that are successfully authenticated are imported into the Authority Service and their user attributes are mapped based on either LDAP attributes or OpenId Connect claims, depending on the type of user storage provider. Once the user is imported into the Authority Service, one may assign additional roles to the user. See [Managing Users and Roles](#managing-users-and-roles). ### Active Directory To configure Active Directory user federation, consult [Configure Active Directory User Federation](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578241/Configure+Active+Directory+user+federation+in+Authority+Service). ### Azure Active Directory To configure Azure Active Directory identity provider, consult [Configure Azure Active Directory Identity Provider](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578248/Configure+Azure+Active+Directory+identity+provider+in+Authority+Service). ### Managing Users and Roles Once a user is imported into the Authority Service's internal user database, the user can be granted additional roles in the Saga platform by the administrator. By default the user will be allowed to use the search UI with the default feature set. Some features and products require additional role(s) to be granted to the user. To grant a role to a specific user do the following: 1. Log in to the [Authority Service Admin Console](#the-authority-service-admin-console) as the *saga_admin* user. 2. Click Users in the left menu bar. 3. Search for a specific user. 4. Select the user to grant a role. 5. Go to Role Mappings tab. 5. Click Assign Role. 6. In filter drop-down, select *Filter by clients*. 7. Search for *saga_implicit_client*. 7. Grant the appropriate role to the user. The available Locator roles are: - **search.public_tags.assign** - Role for assigning and unassigning public tags to documents in the search result. - **search.public_tags.admin** - Role for creating and deleting public tags (also includes assigning and unassigning them). - **search.smart-classifier** - Role required to manage Smart Classifiers. - **search.personal_assistant.premium** - Role required to get the premium Personal Assistant features such as the high quality model. The available Report Engine roles are: - **notification.admin** - Composite role that covers all notification roles. - **notification.read** - Role required for viewing notification configuration. - **notification.write_email** - Role required for sending emails. - **notification.write_smtp_configuration** - Role required for configuring SMTP. - **report_engine.create_report** - Role required for creating reports. - **report_engine.delete_report** - Role required for deleting reports. - **report_engine.export_report** - Role required for exporting reports - **report_engine.admin** - Composite role that covers all report_engine roles. - **report_engine.read** - Role required for viewing reports. - **report_engine.write_dashboard** - Role required for configuring dashboards. - **scheduler.admin** - Composite role that covers all scheduler roles. - **scheduler.create** - Role for creating schedules. - **scheduler.delete** - Role for deleting schedules. - **scheduler.pause** - Role for pausing schedules. - **scheduler.read** - Role for viewing schedules. ## Connector Management ### Installing Connectors To install a connector, do the following: - Find the connector's folder in the list of folders at https://nexus.ayfie.dev/#browse/browse:connector-installer. - Open the highest numbered 6.x.x sub-folder in the connector's folder to access the zip file containing the latest connector installer for Locator 6.1.0.2. - Double click the path in the window that now has opened up to the right of the folder tree. - Download the connector zip file as-is (remain zipped) to directory D:\Program Files\ayfie\saga\plugins. The connector will now be automatically installed once/if Locator is up and running. ### Configuring Connectors To configure a connector, do the following: - Open the [Management Console](#the-management-console) - Click *Connections* and then the connector one is to configure - Click *Add New Connection* and run through the wizard. Consult [Connector Data Sheets](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2928246816/Ayfie+Connector+Data+Sheets) for more details on how to configure each connector. ### Upgrading Connectors Connectors are automatically updated upon restart if a newer version within the same major version exists. ### Uninstalling Connectors To uninstall a connector, do the following: - Navigate to directory *D:\Program Files\ayfie\saga\plugins* - Copy the connector folder to *D:\Program Files\ayfie\saga\plugins\delete* - Wait for the connector folder to disappear from both *D:\Program Files\ayfie\saga\plugins\delete* and *D:\Program Files\ayfie\saga\plugins* If one is uninstalling a 3.x connector, please also see this [known issue](#issue-when-uninstalling-major-version-3-connectors). ### Secondary User Authentication Users are authorized across the [user authentication](#user-authentication) and secondary user authentication. Authorization gathers the users security identifiers, that are used to determine which search hits the user has access to. If all configured connectors are using the user authentication configured in the Authority Service, no secondary user authentication is required. If a configured connector uses some other user authentication, the user will be asked to provide credentials after login. Secondary user authentication is registered behind the scenes when connectors are installed. Consult Authentication and Identification in the [Connector Data Sheets](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2928246816/Ayfie+Connector+Data+Sheets) to determine the required set of user authentication. ## Custom Branding It is possible to divert from the default *ayfie* branding theme and do custom branding for the following components: - The Search UI - The Authority Service (login page) **Branding the Search UI** Perform the following steps: 1. In *D:\Program Files\ayfie\saga\volumes\search-ui\branding\custom* (the path assumes recommended installation path), rename *brand.config.example.json* to *brand.config.json*. 2. In *D:\Program Files\ayfie\saga\volumes\search-ui\branding\custom*, rename *styles.example.css* to *styles.css*. 3. Add a *favicon.ico* to *D:\Program Files\ayfie\saga\volumes\search-ui\branding\custom*. 4. Add a *logo.svg* to *D:\Program Files\ayfie\saga\volumes\search-ui\branding\custom (one can change the file name by changing the value of the *logoPath* variable in file *brand.config.json*. SVG is the recommended image format to use due to its ability to scale. However, other image formats can also be used.) 5. In the *custom* folder, do any of the following optional steps to customize the branding: - Adjust the color values in file *styles.css*. - Change the value of *productName* in file *brand.config.json*. For the changes to take effect, restart Saga with `stop-saga.ps1` followed by `saga.ps1 -configure`. The latter will bring up the install menu in which one is to set *Saga Branding* to *custom*. **Branding the Login Page** If one has configured the *Identity Provider Redirector* in the Authority Service, then one can skip branding the login page as it will not be displayed to users. See section [Configure Identity Provider Redirector](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578248/Configure+Azure+Active+Directory+Identity+Provider) for details. If the login page is to be displayed, then one is free to do any of the following optional steps to customize the branding: - In the *D:\Program Files\ayfie\saga\volumes\authority\docker-entrypoint.d\branding\custom\login\resources\css* folder, rename *styles.example.css* to *styles.css* and adjust the color values in the file. - Add a *logo.svg* to *D:\Program Files\ayfie\saga\volumes\authority\docker-entrypoint.d\branding\custom\login\resources\img* (requires *styles.css*. One can change the file name by changing the value of the *background* variable in the *kc-header-logo* section in file *styles.css*. SVG is the recommended image format to use due to its ability to scale. However, other image formats can also be used.) For the changes to take effect, restart Saga with `stop-saga.ps1` followed by `saga.ps1 -configure`. The latter will bring up the install menu in which one is to set *Saga Branding* to *custom*. ## Private Certificate Authority If Saga needs to communicate with services that are set up with SSL certificates not provided by a well-known Certificate Authority (CA) but instead by for instance the customer's own IT department, then that corresponding private CA's root certificate must be imported into the Docker containers. The situations for which one needs to import the private CA root certificate are: 1. If one plans to configure [Active Directory](#active-directory) in the Authority Service using _ldaps_ and the _ldaps_ SSL certificate is issued by a private CA. 2. If the Saga host server is configured to use a proxy for all traffic and the proxy certificate is issued by a private CA. To import a private CA root certificate do the following: - Enter *certlm.msc* in the program search box down in the left corner (or alternatively find the tool via the *Manage Computer Certificate* option in the *Control Panel*) - Select *Trusted Root Certification* from the list that appears in the left pane - In the right window, click on *Certificates* - Find the Root CA from the list of certificates - Click on the certificate to open it and then click the *Details* tab. - Select *Copy To File...* and then next the *DER encoded binary X.509 (.CER)* option in the export wizard. - Store the file under any name but with the required .cer extension - Copy the file to *D:\Program Files\ayfie\saga\volumes\certs* - Restart Saga ## Major Version Upgrades To upgrade to a new major version, a saga.ps1 install script for that new major version is required. Section [Running the Install Script](#running-the-install-script) describes how to obtain the installer for any given major version. ### Upgrading from major version 5 to 6 1. If one has enabled the Smart Classifier feature in 5.x, then upgrade the Smart Classifier ML component to 6.x on the Linux box before doing any upgrade related operations on the Windows server housing Saga. See [Ayfie Smart Classifier ML Install Guide](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/3228499969/Ayfie+Smart+Classifier+ML+Install+Guide) on how to do the upgrade. 2. Run the stop_saga.ps1 script to stop Saga. 3. One should perform the following cleanup (the paths assumes recommended install directory): - Delete file *D:\Program Files\ayfie\saga\Admin\copyexcludelist.txt* - Delete file *D:\Program Files\ayfie\saga\docker\docker-compose.report-engine.override.yml* - Delete file *D:\Program Files\ayfie\saga\volumes\authority\docker-entrypoint.d\standalone.xml* - Delete file *D:\Program Files\ayfie\saga\volumes\locator\docker-entrypoint.d\01-locator-init.ps1* - Delete file *D:\Program Files\ayfie\saga\volumes\locator\docker-entrypoint.d\saga-functions.psm1* - Delete file *D:\Program Files\ayfie\saga\volumes\PostgreSQL\docker-entrypoint.d\CreateDatabase.sql* - Delete folder *D:\Program Files\ayfie\saga\data\notification* - Delete folder *D:\Program Files\ayfie\saga\data\scheduler* - Delete folder *D:\Program Files\ayfie\saga\data\smart-refiner* - Remove *AYFIE_SAGA_INSTALLER_SKIP_PREREQUISITE_WINDOWS_FEATURE_CONTAINERS* variable from *D:\Program Files\ayfie\saga\docker\custom.env* if exists. 4. Check if Docker is installed from the DockerMsftProvider by running the following command from an elevated PowerShell shell: ``` > Get-Package -Name docker -ProviderName DockerMsftProvider ``` - If the command lists Docker as an installed package, uninstall Docker by running the following command from an elevated PowerShell shell: ``` > Uninstall-Package -Name docker -ProviderName DockerMsftProvider > Uninstall-Package -Name DockerMsftProvider ``` 5. Download and run the 6.x version of the saga.ps1 install script, see [Running the Install Script](#running-the-install-script). 6. The normalization framework has been deprecated in Saga 6.x. One should perform the following cleanup: - Open the [The Solr Dashboard](#the-solr-dashboard). - Go to *Collections* menu option. - Select the *Normalization* collection and click the *Delete collection* button. - Confirm the deletion by typing the name of the collection and click the *Delete* button. 7. The index has been updated to a new major version. Perform a reindex by running the command below from an elevated PowerShell shell: ``` > docker exec ayfie-locator c:/Locator/tools/via.repository.exe reindex /all ``` **Consequences of Migration** - Saga 6.x ships with a new major version of Keycloak, version 22. During the upgrade a new *saga_admin* user will be created in the *Saga* realm. The new password can be found in *D:\Program Files\ayfie\saga\docker\\.env* (assuming recommended installation path) as the *AYFIE_SAGA_AYFIE_USER_PASSWORD* environment variable. It is recommended to later change the generated password to something else in the Authority Service Admin Console. Note that there is a new URL to log into [The Authority Service Admin Console](#the-authority-service-admin-console). The previous *admin* and other existing *master* realm users will automatically be deleted during the upgrade. ### Upgrading from major version 4 to 5 One should make sure all 5.x prerequisites are in place before one starts upgrading from 4.x to 5.x. 1. Uninstall all connectors, see [Uninstalling Connectors](#uninstalling-connectors). This operation will not remove any connections or indexed data. 2. Run the stop_saga.ps1 script to stop Saga. 3. Download and run the 5.x version of the saga.ps1 install script, see [Running the Install Script](#running-the-install-script). 4. Install the 5.x version of the connectors that were uninstalled in step 1, see [Installing Connectors](#installing-connectors). 5. Modern Authentication is introduced in 5.x. This requires that [User Authentication](#user-authentication) is configured in the Authority service: - **Scenario 1**: Report Engine is not enabled and either [Active Directory](#active-directory) or [Azure Active Directory](#azure-active-directory) must be configured. - **Scenario 2**: Report Engine is enabled and [Active Directory](#active-directory) or [Azure Active Directory](#azure-active-directory) is thus already configured. There are additional configuration steps in [Configure Active Directory user federation in Authority Service](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578241/Configure+Active+Directory+user+federation+in+Authority+Service) and [Configure Azure Active Directory identity provider in Authority Service](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2893578248/Configure+Azure+Active+Directory+identity+provider+in+Authority+Service) for 5.x than what there were for 4.x. - If [Active Directory](#active-directory) is already configured, then these are the additional configuration steps for 5.x: - Go to the configured User Federation in Authority Service Admin Console: - Add mapper for *source*. - Add mapper for *user*. - Add mapper for *upn*. - If [Azure Active Directory](#azure-active-directory) is already configured, then these are the additional configuration steps for 5.x: - Go to the registered App in Azure Active Directory: - Perform the Token Configuration. - Go to the configured Identity Provider in Authority Service Admin Console: - Add mapper for *source*. - Add mapper for *user*. - Add mapper for *oid*. - Add mapper for *upn*. - Add mapper for *groups* - Add mapper for *tid*. - In Default Scopes, set to *openid profile*. - Make sure one isn't affected by [Duplicated Client Mappers in Authority Service](#duplicated-client-mappers-in-authority-service). 6. Configuration related to Single Sign-On has been removed from the Management Console and must now be configured in [Active Directory](#active-directory). 7. The internal user *ayfieuser* has been removed. The license slot of this user can optionally be cleared in the Management Console. 8. The Authority Proxy service has been removed. One should perform the following cleanup (the paths assumes recommended install directory): - Delete *D:\Program Files\ayfie\saga\volumes\authority-proxy* - Remove *OAUTH2_PROXY_* variables from *D:\Program Files\ayfie\saga\docker\custom.env* - Delete *docker-compose.authority-proxy.yml* from *D:\Program Files\ayfie\saga\docker* 9. Delete *D:\Program Files\ayfie\saga\volumes\authority\docker-entrypoint.d\saga-realm.json* if exists. 10. If one has configured a SharePoint Online connection, one should go to the Management Console, click Edit and run the wizard for all connections. In the page for *Office 365 Domain Account Details* one needs to select the appropriate value in the *The Azure AD connector is used to obtain end user Office credentials* checkbox for ones setup. Please consult [Microsoft SharePoint Connector Data Sheet](https://ayfie-dev.atlassian.net/wiki/spaces/SAGA/pages/2937454598/Microsoft+SharePoint+Connector+Data+Sheet) for details. 11. If one had Azure AD configured as the primary authentication realm before upgrading to 5.x, one should go to the Management Console, click Edit and run the wizard without any changes for all connections of connectors in the list below: - Microsoft Dynamics CRM - Microsoft Exchange - Microsoft Teams The features listed below have been removed in 5.x. Replace them as suggested if they were in use in 4.x: - **Trusted Authentication** - Trusted authentication is only used by clients who have implemented a custom UI using the RestService API. Trusted authentication is in 5.x instead achieved by using Modern Authentication. See [Configuring the Authority Service](#configuring-the-authority-service). - **Access Control** - Access Control was a feature that could be enabled in the Management Console to restrict which users had access to the Search UI based on Active Directory groups. Access Control can in 5.x be achieved by using *Custom User LDAP Filter* when configuring [Active Directory](#active-directory) or by doing [Restrict your Azure AD app to a set of users in an Azure AD tenant](https://learn.microsoft.com/en-us/azure/active-directory/develop/howto-restrict-your-app-to-a-set-of-users) when configuring [Azure Active Directory](#azure-active-directory). ### Upgrading from major version 3 to 4 1. The _customer id_ and the _license key_ will be required in the last step. One may want to have the two ready before starting this procedure (a 30 days grace period is granted though). 2. Uninstall all connectors, see [Uninstalling Connectors](#uninstalling-connectors). This operation will not remove any connections or indexed data. 3. Run the stop_saga.ps1 script to stop Saga. 4. Unregister the 3.x package source by running the following command from an elevated PowerShell shell: ``` > Unregister-PackageSource AyfieDevNuget ``` 5. Download and run the 4.x version of the saga.ps1 install script, see [Running the Install Script](#running-the-install-script). 6. Install the latest version of the connectors that were uninstalled in step 1, see [Installing Connectors](#installing-connectors). 7. The license service has been redesigned and any license used with 3.x has to be re-added. Run the start_management_console.ps1 script, then click Operations, then License, then Add License to re-enter license info. # Starting and Stopping ## Stopping Use this command to stop Locator: ``` > .\stop-saga.ps1 ``` Do not use any alternative *docker-compose* command to stop Locator as that will not give the index time to shut down in a graceful manner. ## Starting ``` > .\saga.ps1 ``` The *saga.ps1* script checks for updates, and if any compatible newer version is detected, the script will perform the upgrade before starting Locator. # Uninstall To delete everything related to an Locator installation (including Docker itself), run these command lines one by one from within an elevated PowershellPowerShell command prompt: ``` > cd "D:\Program Files\ayfie\saga" > .\stop-saga.ps1 > docker rmi $(docker images ayfiehub/* -qa) > cd .. > Remove-Item –path saga -recurse ``` The command lines above will delete all Locator images (docker resources related to any still running docker application will not be effected). By now Locator is fully uninstalled. To also uninstall Docker, run these commands: ``` > Uninstall-Package docker -ProviderName DockerMsftProvider > Uninstall-Module DockerMsftProvider > Get-HNSNetwork | Remove-HNSNetwork > Remove-Item "C:\ProgramData\Docker" -Recurse ``` The last line assumes that the default Docker output directory is being used. # User Interfaces ## Search UI Accessing the URL below in a browser after replacing *\<Gateway Hostname\>* will allow one to log in to the search UI: ``` https://<Gateway Hostname> ``` The *Gateway Hostname* is as set during installation. The search UI will prompt for user name and password. If integrated with AD, one may log in using any valid domain account. ## Report Engine UI Accessing the URL below in a browser after replacing *\<Gateway Hostname\>* will allow one to log in to the Report Engine UI: ``` https://<Gateway Hostname>/report-engine ``` The Report Engine UI will only be available if the Report Engine is enabled in the installer. To grant user access to the Report Engine, access the Authority Service as described in [Configuring the Authority Service](#configuring-the-authority-service) and then grant roles as described in [Managing User and Roles](#managing-users-and-roles). ## The Admin Dashboard Accessing the URL below in a browser running locally on the Locator server, will take one to the Locator Dashboard: ``` http://localhost/dashboard ``` ## The Management Console Running the script below from within an elevated PowershellPowerShell command prompt will open the Management Console: ``` > cd "D:\Program Files\ayfie\saga" > .\start-management-console.ps1 ``` If one is using a Microsoft SQL Server database, then the user that is to run the script above to start the Management Console must have been assigned the *db_datareader* and *db_datawriter* roles for the Locator database. ## The Authority Service Admin Console Accessing the URL below in a browser after replacing *\<Gateway Hostname\>* will allow one to log in to the Authority Service Admin Console: ``` https://<Gateway Hostname>/auth/admin/saga/console ``` The admin account's username is *saga_admin* and the generated password is found as the value of variable AYFIE_SAGA_AYFIE_USER_PASSWORD in the file *D:\Program Files\ayfie\saga\docker\\.env* (the path assumes recommended install directory). It is recommended that one changes the generated password. To change the password, go to the Authority Service Admin Console and enter the credentials for the administrative account. In the top right menu, click *Manage account*. Select *Signing In* in *Account Security* and update the password. ## The Solr Dashboard Accessing the URL below in a browser running locally on the Locator server, will take one to the Solr Dashboard: ``` http://localhost/solr ``` ## The Grafana Dashboard Accessing the URL below in a browser running locally on the Locator server, will take one to the Grafana Dashboard: ``` http://localhost/grafana ``` Use user name *admin* and password *admin* to log in and then click the Home link up in the left corner to get to the dashboards. The Grafana Dashboard will only be available if metrics is enabled in the installer. ## The PostgreSQL pgAdmin Dashboard Accessing the URL below in a browser running locally on the Locator server, will take one to the PostgreSQL pgAdmin Dashboard: ``` http://localhost/pgadmin ``` Use user *admin@ayfie.com* and password *Ayfi3Ru13s!* to log into the pgAdmin and then connect to the database by right clicking Servers and select Register -> Server: - In the *General* tab set the *Name* to some name, for instance *locator-db* - In the *Connection* tab, set the *Host name/address* field to *ayfie-saga-db* and *Username* to *postgres* - Click *Save* The Locator database should now be displayed. The PostgreSQL pgAdmin Dashboard will only be available if Postgres is set at the database type in the installer. ## RestService API Accessing the URL below in a browser after replacing *\<Gateway Hostname\>* will take one to the RestService API documentation: ``` https://<Gateway Hostname>/RestService/swagger ``` # Known Issues ## Losing Connection to Docker Daemon As described in [Github issue 5044](https://github.com/docker/for-win/issues/5044), there is an issue with VMWare Tools and Docker that makes Locator lose its connection to the Docker daemon after some 10-15 minutes. The fix is to upgrade VMWare Tools to version 11.0.6 or higher. ## Saga Not Starting Due to docker-compose Upgrade Failure During Saga startup, docker-compose updates to 1.29.2 if currently installed version is older. It is installed in "$Env:ProgramFiles\Docker\" directory. If environment variables are configured in such way, that docker-compose.exe is used from another directory, then it should be either reconfigured, or docker-compose should be manually upgraded in that directory. ## Issue When Uninstalling Major Version 3 Connectors There is a known issue when uninstalling 3.x connectors. The value of COMPOSE_FILE on the first line of the file <install_dir>\docker\plugins\\.env ends up with extra semicolons that for now have to be removed manually. Here is a typical example of such a case. Notice the first semicolon at the beginning of the value and the two in the middle next to each other: ``` COMPOSE_FILE=;docker-compose.connector-exchange.ad.override.yml;;docker-compose.connector-worksite.ad.override.yml ``` Such extra semicolons should be removed (notice how two semicolon from above have been removed below): ``` COMPOSE_FILE=docker-compose.connector-exchange.ad.override.yml;docker-compose.connector-worksite.ad.override.yml ``` The correction should be done after any connector has been uninstalled and before any use of stop_saga.ps1 or any new connector installation. The following line in saga.ps1 or stop_saga.ps1 output, is an indication of this issue: ``` ERROR: .FileNotFoundError: [Errno 2] No such file or directory: '.\' ``` ## Duplicated Client Mappers in Authority Service This issue can affect customers who used Report Engine with Saga 4.1.x and then upgraded to version 4.2.x or later. To fix this, manually delete duplicated mappers in Authority Service Admin Console for *ayfie-service-account-client* client by following the steps below: - Go to Clients menu option in Authority Service Admin Console. - Click *ayfie-service-account-client* client. - Go to the Mappers tab. - Delete any duplicated mapper. - Restart Saga ## Firewall Blocks Communication between Containers The firewall can be set up to block some or all incoming and outgoing network traffic. Docker containers behave like virtual machines and have their own IP addresses on a virtual network named *vEthernet (nat)* as the *InterfaceAlias* (the PowerShell command *Get-NetConnectionProfile* can be used to list all networks). The containers running on an internal virtual network are normally not affected by firewall blocking. However, if one were to discover that the containers are unable to communicating with each other, then that could be explained by the firewall blocking. In that case one should investigate the firewall settings related to the virtual network. An indication that one is experiencing this issue could for instance be that the *ayfie-locator* container is unable to connect to an otherwise healthy Postgres database during startup. To look into this one could use the PowerShell command below from inside the *ayfie-locator* container to see if one is able to reach the *ayfie-saga-db* container on its operational port 5432: ``` Test-NetConnection -port 5432 ayfie-saga-db ``` |
...