Searching LDAP with pagination

Searching LDAP with pagination

Originally published: 2023-07-05T13:53:00.003+03:00 · Original English article

When searching a large LDAP directory, the server may limit how many entries it returns in one response. Active Directory’s documented default MaxPageSize is 1,000 objects, not 20,000. The effective limit depends on the server’s query policy; a page limit is not the total number of entries that can be retrieved across a paged search. See Microsoft’s LDAP policies.

Use the LDAP simple paged-results request and response controls to retrieve successive pages. Start with an empty cookie. See RFC 2696.

Send the requested page size and then reuse the opaque cookie from the most recent response in the next request. Keep the search parameters unchanged across pages. An empty response cookie marks completion. A missing response control or a failed search must not silently be treated as successful completion. Paging does not guarantee an unchanging snapshot while directory contents are changing. See RFC 2696, client-server interaction.

Here is an application-level example using LdapForNet. It assumes an already established, appropriately protected connection, a base DN, requested attributes, a result collection and a logger. It keeps the original objectClass=user filter and sAMAccountName sort, but checks search, paging and sort results before accepting each page. In Active Directory, that filter can also match computer objects; use a filter appropriate to the objects you need, such as (&(objectCategory=person)(objectClass=user)) for user objects. See ADSI search filters and the .NET AD query implementation.

byte[] cookie = Array.Empty<byte>();

while (true)
{
    SearchRequest searchRequest = new(ldapSource.BaseDN,
        "(objectClass=user)", LdapSearchScope.LDAP_SCOPE_SUB, attributes);

    searchRequest.Controls.Add(new PageResultRequestControl(1000)
    {
        Cookie = cookie,
        IsCritical = true
    });
    searchRequest.Controls.Add(new SortRequestControl(
        new SortKey { AttributeName = "sAMAccountName" }));

    SearchResponse searchResponse =
        (SearchResponse)ldapConnection.SendRequest(searchRequest);

    if ((int)searchResponse.ResultCode != 0)
        throw new InvalidOperationException("LDAP search did not succeed.");
    if (searchResponse.References.Count != 0 || searchResponse.Referral.Length != 0)
        throw new InvalidOperationException("LDAP referral handling is required.");

    PageResultResponseControl page = searchResponse.Controls
        .OfType<PageResultResponseControl>().FirstOrDefault()
        ?? throw new InvalidOperationException("Missing paged-results response.");
    SortResponseControl sort = searchResponse.Controls
        .OfType<SortResponseControl>().FirstOrDefault()
        ?? throw new InvalidOperationException("Missing sort response.");
    if ((int)sort.Result != 0)
        throw new InvalidOperationException("LDAP server-side sorting failed.");

    ldapEntries.AddRange(searchResponse.Entries.Select(e => e.ToLdapEntry()));
    cookie = page.Cookie;
    _logger.LogDebug("Collected entries: {ldapEntries}", ldapEntries.Count);

    if (cookie.Length == 0)
        break;
}

Server-side sorting is a separate control, not a requirement of simple paging; when requesting it, inspect its result as shown. This example fails if referrals need handling rather than reporting a partial result as complete. Add application-specific cancellation, failure handling and a deliberate referral policy before using it in a production workflow. The example is not a tested, self-contained program. Sources: RFC 2891, LdapForNet controls and response types.

The .NET System.DirectoryServices.AccountManagement.PrincipalSearcher used below enables paging for an AD-backed context in the implementation inspected. This does not mean every .NET LDAP API automatically pages: a directly created DirectorySearcher defaults to PageSize = 0, which disables paging. The current PrincipalSearcher source sets 256 for AD-backed contexts; this is a number of entries, not kilobytes. See DirectorySearcher.PageSize.

using PrincipalContext ctx = new(
    ContextType.Domain, "[hostname]:389", username, password);
using UserPrincipal userPrincipal = new(ctx);
using PrincipalSearcher searcher = new(userPrincipal);
using PrincipalSearchResult<Principal> result = searcher.FindAll();
// Enumerate result and finish processing before these objects are disposed.

The hostname and credential variables are placeholders supplied by the application, not a complete connection-security configuration. The using declarations keep the context, query and result lifetime explicit. Enumerate the results while these objects are alive; select connection protection and authentication suitable for your environment.