Windows Search Index is a powerful tool to allow your users find files on your file servers and their shares engaging full text searches and getting results quickly.. Unfortunately if you are using DFS (distributed file system) you quickly will find out that this does not work as expected.
DFS can not use the Windows Search Index. This is a fact. Having said this, I was able to find a solution and allow users to search throughout the DFS namespaces and all resources behind while engaging Windows Search Index.
The issue:
DFS provides a single entry point (or multiple) to various file shares while providing a so to say simulated entry point. Lets make this more clear on a simple structure example:
You have a DFS root share \\files.domain.local\dfs or \\domain.local\dfs – in this case the share DFS is simply the main entry point. Let us assume user have mapped this DFS share as N:\ drive.
For the full picture, you can provide this DFS share domain integrated or independent from your domain – eventually it does not matter how you accomplished and designed this.
Underneath DFS are various namespaces. Those point to other namespaces and/or servers that hold the shares. Let’s say we have this:
- \\domain.local\DFS\Departments
- \\domain.local\DFS\Other
Departments and Other are both shares on either the same server or additional servers. They are primarily namespaces. Behind those can be a single target or even targets to multiple servers that hold and synchronize the content within this share. You might have local servers at each major site of your network (like New Your, London, Los Angeles, etc..) or single targets that do not synchronize the content.
If a users goes on the N:\ drive and types in a keyword in the upper right search box Windows Explorer provides, he will experience a very slow old school search throughout the drive – by default only looking for file and folder names that would match.
Assuming you started Windows Search Indexing on this target file server, you would expect a quick search with results including the engaged full text index that the server build. But you quickly awaken and realize that this does not work.
Root cause is, that the clients Windows Explorer asks the root DFS share server for files in the DFS share. This share likely is empty and you don’t get any results back, naturally.
The Windows client does not know that he would need to ask the target file-server behind the namespace for files – even if he would – you would face various issues like is the index created and replicated to each possible member that holds the same share. Microsoft simply does not support this and Windows can’t handle this as of today, the year 2019 – including current Windows Versions Windows Server 2016.
Windows Search further can’t index network shares. There actually was a temporary solution in the x86 / 32 bit world of Windows 7 that allowed this – but there is no solution as of today where everything is 64 bit. There are reason for this – but the explanation surely would exhaust the purpose of this website.
Your only official work-a-round is to connect the end user directly to the file server share that hold the Windows Search Index for the targeted namespace and have the user search there. This of course bypasses DFS and defeats the whole purpose of DFS.
The solution
What you will find here is a solution to overcome this challenge, with some minor buts.
We faced this situation:
- a DFS namespace with about 150 entry namespaces
- the namespaces targeted about 10 huge file servers
- some of them had file servers synchronizing the same share over multiple locations – local file servers that users accessed depending on their location respective on which one was active and which one is passive
- a folder structure within the DFS that exceeds 7 million folders and 20 million files (roughly)
Step one:
We enabled the Windows Search Index on one server within each namespace – all servers that had the index enabled resided on the main site, so they where close to each other. To give you some numbers, the indexing took more then a week on some of those servers generating index databases that summoned up throughout those servers to over 500 GB in total. We talk about a total used space of 10 TB or more.
Step two:
We developed a web service that runs in IIS (Internet Information Services) and provides the users a single entry point at the same side as those file servers reside. We engaged the Windows Federated Search engine that use using the OpenSearch protocol. This integrates in to the end users Windows Explorer as a single (or multiple) entry points.
Now, the federated search won’t give you any results, we had to develop a web service that actually knows the DFS shares and targets those servers that hold a Windows Search Index. We then had to tell the web service to ask those very servers for their Windows Search Index database and get results from the very same. Those then had to be translated back to the OpenSearch protocol (a XML result list) and posted through the IIS back to the Windows client.
The client then list the search results in this search provider as the user would expect it – as a simple file and folder search result that he can directly interact with.
The huge challenge with IIS and permissions
Our engaged web site and the application pool behind made this challenging. We had the issue that we could not use the ASP.NET impersonation – while the impersonation works fine directly on the web server it quickly became clear that this did not work as expected when targeting the actual target file servers through the IIS. Impersonation itself theoretically could still work, to go in to more detail – the issue is that you would need to enable and configure DELEGATION throughout your network to pass the client side user credentials not just to the IIS web site, rather then all the way to the target file server. There are one or two ways to accomplish this, but the security risks and changes within the network are simply not worth it.
Overcoming the challenge
We then decide we create a user account that runs the IIS application pool for this search web site. This user account further got LIST and READ permission on each possible file server path. Additionally it was allowed to READ SECURITY PERMISSIONS. This allowed us to list all files on those file shares and therefor Windows Search Index reported complete search results back to our web service.
Cause the users wouldn’t be able to access all those files as well, we instructed the web service to check if the user that is logged on to the Windows client actually can access the found file and only report it back to it if this is the case.
All of this is a pretty intense calculation that has many moving parts in it. But guess what – we just made it possible for our end users to search within our DFS namespaces throughout our file servers and get results from the fulltext index. The only downside was that users had to execute the search while using this search provider in Windows Explorer rather then directly engaging the search on the DFS network drive. This eventually only is a learning curve and can be overcome.
The benefits of the whole solution did outweigh the minor issues of it quickly.
Stay put for further details as I am about to publish the whole solution soon.