To extract and index documents stored in Microsoft SharePoint, the Locator SharePoint connector needs to be configured with a user that has access to all the documents that are to be extracted. This user is referred to as the index user.

Since version 2.9.7.0 of the SharePoint Connector Azure AD Application can be used to extract and index documents in SharePoint Online. If you decide to use Azure AD Application instead of index user you should skip to Setting up the Windows Azure Graph API for Locator and then go to Adding Permissions to the Azure AD Application for SharePoint Online indexing.

• Important Note: Using Azure AD Application for indexing SharePoint Online documents requires Azure AD connector installed with connection configured to the same tenant.

Establishing Access for the "Index User"

Within the Office 365 organization, create a dedicated Office 365 user account (e.g. "ex_index").  This user account will be used to establish a Locator SharePoint Online connection. This will be referred to as the "index user". An existing user account may be used, but a dedicated user account is recommended.

Make sure that the index user account password is set to never expire. Follow the instructions in the Microsoft document Setup user passwords to never expire.

Required permissions quick guide

Required permissions for each connection type

Required permissions for index user are different for each connection type. Table below provides quick guide on minimal permissions required by all connection types.

* This can used when at least on one of these: Include Office 365 Personal Blogs, Include Office 365 Group Sites, Include Office 365 Communication Sites or Include other site templates is selected on Connection Configuration Page.

** If index user isn't site collection administrator then the connections won't be able to build document security correctly for groups that have "Who can view the membership of the group?" set to "Group Members" and index user isn't member of that group. This means some users might not get search hits for documents they have access to.

Additional permissions for included features

Some features provided by SharePoint connector may require some requirements.

Permissions for additional users

SharePoint connector allows to provide additional accounts used by the connector.

Access Rights

Important: Before you decide how to configure the index user's permissions:

Note 1: If you plan to use the SharePoint "change set" feature, which provides for a more efficient method for Locator to obtain changes to documents within SharePoint,
the index user will be required to access the SiteData web service.  This requires the index user to be a site collection administrator for every site collection that will be indexed.

• If installed connector version < 2.9.0.0 then site collection administrator for every site collection is required

• If installed connector version => 2.9.0.0 then site collection administrator for every site collection is not required

Note 2: If you plan to use the "Index All Site Collections" option, which allows for simple setup for all existing and future site collections,
the index user requires either to have custom permission level set on all site collection or to be part within primary or secondary site collection administrator. Additionally, it must be added as a SharePoint Administrator. 
However if you want to avoid giving SharePoint Admin role just for this purpose, this is not needed if:

• If any of the "Include Office 365..."-checkboxes in wizard is used for specific site collection templates

• If "Include other site templates" is checked in the wizard and some of the templates were selected.

Note 3: Global admins and SharePoint admins don't have automatic access to Group Sites. That means they can not manage permissions inside Group Sites. However global admins still have option to add members and owners to Group Sites.

Custom "Read for index user" permission level

SharePoint Online differs from the on-premises SharePoint server in that there isn't a Central Administration from which you can assign the "index user" with "Full Read" access to each SharePoint Web Application to which a connection will be established.  Therefore, you must provide the index user with the required permissions for each site whose documents you want included in the index.  

The SharePoint built-in "Read" permission level provides insufficient permissions for the index user, therefore you will need to create a custom permission level with the appropriate permissions.  This new custom permission level that you will create must include all of the List Permissions and Site Permissions that are included with the built-in "Read" permission level, plus one additional List Permission: "Manage Lists", and two additional Site Permissions: "Browse Directories" and "Enumerate Permissions". This requirements also applies to other types of sites like Group Sites, Personal Blogs, Team Sites and Communication Sites. To add custom permission level to Team Site you have to open it in SharePoint as it is not possible from Microsoft Teams.

From the top level site in each of your SharePoint Online site collections, perform the following:

1. Navigate to the top most site to which the index user will be given access.  This is the site that you will specify when presented with the “Enter the address of MS SharePoint server site you want to make searchable.” in the Locator Connector Wizard.

2. Click on the gears icon  at the top right of the page, and from the drop-down list click on "Site settings".

3. Under "Users and Permissions" click on "Site permissions".

4. From the Permissions tab click on "Permission Levels".

5. From the "Permissions > Permission Levels" page, click "Add a Permission Level"

6. Provide a name (e.g. “Read for Index user”) and select the following permissions:

   • List Permissions:

     • Manage Lists

     • View Items

     • Open Items

     • View Versions

     • Create Alerts

     • View Application Pages

   • Site Permissions

     • Add and Customize Pages

     • Browse Directories

     • Use Self-Service Site Creation

     • View Pages

     • Enumerate Permissions

     • Browse User Information

     • Use Remote Interfaces

     • Use Client Integration Features

     • Open

7. Click the “Create” button

8. Go back to Site Permissions by clicking on “Permissions”

9. Click on “Create Group”

10. Provide a name for this group (it will be used to apply to the index user)

11. Under “Choose the permission level group members get on this site:...” check the box for the new permission level you added in a prior step (e.g. "Read for Index user").

12. Click the “Create” button.

13. You will now see the new group

14. Click on “New” and with "Invite people" highlighted, enter the name of the index user, and click “Share”.

Note: There is no permission that would allow index user to retrieve group membership for groups that have "Who can view the membership of the group?" set to "Group Members". Even "Full Control" is not enought for that scenario. The only way for the index user to build the document security correctly is to either add index user as a member of that group or set them as site collection administrator.

Adding Index User as Site Collection Administrator

To index all site collections the index user requires either to have custom permission level set on all site collection or to be part within primary or secondary site collection administrator. You can use the Set-AdminOnSites.ps1 PowerShell script to add index user as secondary site collection administrator to all site collections in your tenant, excluding personal sites.

Important notes:

1. To execute the Set-AdminOnSites.ps1 the SharePont Online Client Components SDK is required >> SharePointOnlineClientComponentsSDK

2. You can run this script with IndexUser parameter set to index user's LoginName or Email. It is important to add i:0#.f|membership| prefix if LoginName is used.

3. Multiple index users can be provided in the IndexUser parameter..

4. Each time a new site collection is added to SharePoint, the SharePoint administrator will need to rerun this script or manually add index user permissions on newly created site collections.

5. If you run this script without IndexUser parameter then this script will only preview the changes it would have made.

6. You can provide list of site collections to be excluded in the SkipSiteCollections parameter. It works with both full and relative URLs.

7. If you run this script with Remove switch then it will remove index user from Site Collection Administrator on each site collection.

8. This script will prompt you for credentials, you have to provide SharePoint Administrator credentials otherwise the script will fail.

9. At the end you will have option to perform retry on all site collections the script had failed. AutoRetry switch can be used to do this automatically.

Adding Permissions to the Index User for MySites, OneDrive for Business and Delve Blogs

When adding a connection to SharePoint Online, in order to index information contained within individual users MySites and OneDrive for Business, the index user must be added to the list of site collection owners.  This must be performed for each user and can be accomplished by either of the following methods:

• Manually editing each user profile using the SharePoint Admin User Interface:

  1. From your browser, navigate to your office 365 SharePoint admin center.

  2. Click on "Admin" from the ribbon bar at the top right and select "SharePoint".

  3. Click on "user profiles" from the list on the left.

  4. Under "People", click on "Manage User Profiles".

  5. The "Total number of profiles" will be displayed. The following steps (#6 through #9) will need to be performed for each of the existing user profiles. 

  6. In the "Find profiles" entry field, type the user name and click the "Find" button.

  7. Position the mouse over the "Account name", right click and select "Manage site collection owners".

  8. In the "site collection owners window, enter the name of the index user in the entry box for "Site Collection Administrators".

  9. Click OK.

• Running the Set-MySiteIndexUser.ps1 PowerShell script.  This script will read all users from SharePoint online and add the index user to the "Site Collection Administrators" list, for each user's personal site, if it exists.

NOTE: To execute the Set-MySiteIndexUser.ps1 the SharePont Online Client Components SDK is required >> SharePointOnlineClientComponentsSDK

NOTE: You can run this script with IndexUser parameter set to index user's LoginName or Email. It is important to add "i:0#.f|membership|" prefix if LoginName is used.

NOTE:  Each time a new user is added to SharePoint, the SharePoint administrator will need to follow the previously mentioned steps to add permission to the index user for the user's MySites and OneDrive for Business. When indexing OneDrive, the index user must have a provisioned OneDrive set up, or else the script and the setting of the required permissions will fail.

• To index user’s Delve Blogs, run the Set-PersonalBlogIndexUser.ps1 PowerShell script. This script will read all users from SharePoint Online and add the index user to the “Site Collection Administrators” list, for each user’s personal Delve Blog. Alternatively, the -Contributors parameter can be used to only grant the index user “Contributor” permissions to the Delve Blogs.

• Note: Delve Blogs are only created once the user visits the Delve Blog site for the first time. The administrator will need to run this script again to index newly created Delve Blogs.

NOTE: If your SharePoint Administrator account uses Multi-Factor Authentication then you have to use Set-MySiteIndexUserMFA.ps1 version of the script instead. There are few differences compared to Set-MySiteIndexUser.ps1 script. There is additional prerequisite: you have to install module SharePoint Patterns and Practices PowerShell Cmdlets for SharePoint Online. This script is also slower than Set-MySiteIndexUser.ps1. There are few suggestions on how to use this script:

• It is recommended to use this script without -UseWebLogin switch. This switch changes the calls to Connect-PnPOnline to use -UseWebLogin instead of -SPOManagementShell. And in most cases -SPOManagementShell is recommended.

• When you use this script you will be prompted to login into SharePoint in separate window. If you are not prompted to do so then it means that it is using cached credentials. You can call the script with -ClearTokenCacheOnFirstConnection switch to use different account.

• If you experienced any Unauthorized or Forbidden errors that shouldn't be happening then you should try to run this script again, this time without the -ClearTokenCacheOnFirstConnection switch.

• If the previous step doesn't help, you should try using the script -UseWebLogin switch. Keep in mind that if you use this switch you will be prompted to login to Admin Center, MySites, and then once for each user's OneDrive. However you can always decide to stay signed in. In that case you will not be asked to login again, but you will see new windows appear and automatically disappear shortly after. That is why using this switch is not recommended unless you are experiencing issues without it.

Configuring Windows Azure Active Directory

Locator uses the Windows Azure Graph API.  The Office 365 Global Administrator will need to configure an application, called a "service principal" in Windows Azure terminology, to be authorized to read Windows Azure Active Directory information.

Setting up the Windows Azure Graph API for Locator 

Setting up the Windows Azure Graph API for Locator is required in order to configure either an Exchange Online or a SharePoint online connection.  This need only be done one time, as the client ID and secret key obtained through these steps can be used for both the Exchange Online connector and the SharePoint Online connector.

To create and obtain an Azure AD Client App Id and a Client Secret, sign into https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps using the global administrator account for your Office 365 organization and do the steps indicated in red below. The two values that are to be extracted are circled in green:

1. Select New registration.
   
   
   
   
   

2.  Set Name and set Supported account types to Accounts in this organizational directory only (Default Directory only - Single tenant) then click Register.
   
   

3. On the Overview for Locator Azure AD App (App Name), copy value of Application (client) ID and Directory (tenant) ID and save it, you will need it for configuration of SharePoint Online connection.
   
   

4. On Certificates & secrets, create New client secret and set Description and Expires date, then save secret's Value (Client Secret ID), you will need it for configuration of SharePoint Online connection.
   
   

5. On API permissions page, Add a permission from Microsoft Graph
   
   
   
   

6. Select Application permissions (Microsoft Graph), select and checked permission: Directory.Read.All (Read directory data)
   
   

7. Grant admin consent for Default Directory (Default directory - name of directory)
   
   

8. After the granting the consent for permissions, permission statuses will be changed to granted.
   
   
   

   Open

   

   
   
   
   

Adding Permissions to the Azure AD Application for SharePoint Online indexing

You will need Azure AD Application with client secret set to Never Expires. You can use the same application you have created in the Graph API in the Setting up the Windows Azure Graph API for Locator section or create new one.
Go to https://[tenant]-admin.sharepoint.com/_layouts/15/appinv.aspx (replace [tenant] with your Office365 tenant name). This page can be accessed only by SharePoint administrator.

Open

On this page perform following:

1. Under App Id: type your Application (client) ID and press the Lookup button.

2. Under App Domain: you can type anything, for example localhost

3. Redirect URL you can leave empty.

4. Under Permission Request XML: paste this:

   Permission Request XML

   <AppPermissionRequests AllowAppOnlyPolicy="true"> <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" /> <AppPermissionRequest Scope="http://sharepoint/taxonomy" Right="Read" /> <AppPermissionRequest Scope="http://sharepoint/social/tenant" Right="Read" /> </AppPermissionRequests>

   
   
   

5. Now you will have to confirm the changes. Press Trust It.

After those steps your applications can be used to index all sites, user profiles or OneDrives in your Office365 tenant.