Tuesday, April 15, 2014

Using Spring LDAP to page large search results

I have a need to publish xml representations of specific ldap accounts to a a REST API.  This post focuses on how I used features of Spring LDAP to page the large result (200,000)  using:

Also see the section of the Spring LDAP User Guide on this topic.

Problem:
The Directory Server (sometimes called an LDAP in everyday conversation) limits search results to 1,000 entries.  For example Oracle OID has such a limit by default
04: LDAP_SIZELIMIT_EXCEEDED - More entries match the search query than the size limit specified. If you have not specified a size limit for the search, Oracle Internet Directory uses a default size limit of 1000.

Code:
   public List<StaffPerson> getAllPersonNamesPaged() {  
     final SearchControls searchControls = new SearchControls();  
     searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE);  
     final PagedResultsDirContextProcessor processor = new PagedResultsDirContextProcessor(PAGE_SIZE);  
     return SingleContextSource.doWithSingleContext(  
         contextSource, new LdapOperationsCallback<List<StaffPerson>>() {  
       @Override  
       public List doWithLdapOperations(LdapOperations operations) {  
         List<StaffPerson> result = new LinkedList<StaffPerson>();  
         boolean isFinished = false;  
         while (!isFinished) {  
           List<StaffPerson> oneResult = operations.search(  
               "cn=users,dc=nysed,dc=gov",  
               "(staffID=*)",  
               searchControls,  
               new AttributesMapper<StaffPerson>() {  
                 public StaffPerson mapFromAttributes(Attributes attrs)  
                     throws NamingException {  
                   StaffPerson staffPerson = new StaffPerson();  
                   staffPerson.setCn(getAttributeValue("cn", attrs));  
                   staffPerson.setTeachID(Long.parseLong(getAttributeValue("staffID", attrs)));  
                   staffPerson.setMail(getAttributeValue("mail", attrs));  
                   return staffPerson;  
                 }  
               },processor  
           );  
           result.addAll(oneResult);  
           if (processor.hasMore()) {  
             println "processor has more: Paging Result Size ${oneResult.size()} Total Size is ${result.size()}"  
           } else {  
             isFinished = true;  
           }  
         };  
         return result;  
       }  
     }  
     );  
   }