AEM 6 SP2 - Simple Dispatcher Configuration (Author -> Publish -> Dispatcher)

Goal

Create simple Author -> Publish -> Dispatcher configuration on developer's box (not production). Assuming Author aem is running on port 4502 and Publish aem on port 4503, this post is on adding & configuring dispatcher module on Windows Apache 2.2 Http Server running on port 80. For product documentation on configuring dispatcher check this page and this page


Author - Create Page

1) Start by creating a template on author or download this sample package (contains template Basic Template); installing it creates the following structure (/apps/samples) in CRX (http://localhost:4502/crx/de)

2) Create a page of type Basic Template and add a text component (for information on creating CQ template components check this post). Here is the page created - http://localhost:4502/editor.html/content/experience-aem/english.html

3) The page node structure in author CRX

Publish - View Page

1) Try to view the page on publish, created on author above. As the page isn't published yet, accessing publish http://localhost:4503/content/experience-aem/english.html results in 404

2) Go back to author for publishing the page. Before page is published, package containing the template component needs to be replicated. Access CRXDE package manager of author (http://localhost:4502/crx/packmgr/index.jsp) and replicate the package basictemplate.zip

3) When replication is successful, the necessary /apps/samples node structure should have been created in Publish CRX (http://localhost:4503/crx/de/index.jsp#/apps/samples)

4) Publish the page on author. The node /content/experience-aem/english gets created in publish crx confirming page publish

5) Access the published page on http://localhost:4503/content/experience-aem/english.html

6) User now successfully authored and published  page /content/experience-aem/english.html


Dispatcher - Cache page

1) Using a Web Server and CQ Dispatcher to cache published pages, brings great performance by serving static html and limiting requests sent to publish server for recreating the same pages over and over for each client request

2) Assuming Apache 2.2 Http Server installed, download the dispatcher module. Dispatcher releases are independent of AEM releases, so find the right dispatcher for your OS (Windows), Web Server (Apache) and not AEM - dispatcher-apache2.2-windows-x86-4.1.9.zip (if this dispatcher version was updated, link may not work, download it from the page)

3) Unzip the file dispatcher-apache2.2-windows-x86-4.1.9.zip and copy disp_apache2.2.dll to apache modules folder (eg. C:\dev\code\install\Apache2.2\modules)

4) Open apache httpd.conf file (eg. C:\dev\code\install\Apache2.2\conf\httpd.conf) and add the following line to include dispatcher configuration (added in next steps)

Include conf/dispatcher.conf

4) Create file dispatcher.conf in conf folder (eg. C:\dev\code\install\Apache2.2\conf\dispatcher.conf) and add the following configuration. For more details on available parameters like DocumentRoot check this adobe doc and apache http server documentation

LoadModule dispatcher_module modules\disp_apache2.2.dll

<IfModule disp_apache2.c>
  DispatcherConfig conf/dispatcher.any
  DispatcherLog    logs/dispatcher.log
  DispatcherLogLevel 3
  DispatcherNoServerHeader 0
  DispatcherDeclineRoot 0
  DispatcherUseProcessedURL 0
  DispatcherPassError 0
</IfModule>

<Directory />
  <IfModule disp_apache2.c>
    SetHandler dispatcher-handler
    ModMimeUsePathInfo On
  </IfModule>

  Options FollowSymLinks
  AllowOverride None
</Directory>

5) #1 loads the dispatcher module, #4 specifies the dispatcher rules file (created in next step)

6) Create file dispatcher.any in conf folder (eg. C:\dev\code\install\Apache2.2\conf\dispatcher.any) with the following configuration

/farms {
 /experience-aem {   
  /clientheaders {
   "*"
  }
  /virtualhosts {
   "*"
  }
  /renders {
   /rend01 {
    /hostname "127.0.0.1"
    /port "4503"
   }
  }   
  /filter {
   /0001 { 
    /type "deny" /glob "*" 
   }    
   /0002 { 
    /type "allow" /url "/content*" 
   }
   /0003 { 
    /type "allow" /url "/etc/designs*" 
   }
   /0004 { 
    /type "allow" /url "/etc/clientlibs*" 
   }
  }
  /cache {
   /docroot "C:/dev/code/install/Apache2.2/dispatcher/cache"
   /rules {
    /0000 {
     /glob "*"
     /type "allow"
    }
   }
  }  
 }
}

7) #2 is name of farm which could be any, #10 render is the publish instance serving pages to be cached (1:1 dispatcher to publish mapping recommended). #17 first denies access to every resource and making exceptions at #20 to allow resources under /content, #23 for allowing site css, #26 for js files, minimally needed for typical websites. #30 specifies the cache folder where static content generated from publish servers is cached for any future requests until any cached content is invalidated (next sections). Rule at #32 directs dispatcher to cache all content. For more indepth dispatcher configuration check adobe documentation

8) With the above configuration in place, restart apache server, open a new browser instance and access the sample page on dispatcher running on default port 80 - http://localhost/content/experience-aem/english.html

9) The following statements should haven be logged in logs/dispatcher.log (eg. C:\dev\code\install\Apache2.2\logs\dispatcher.log) and cached content available in dispatcher/cache folder (eg. C:\dev\code\install\Apache2.2\dispatcher\cache)

Dispatcher - why is content not cached?

1) If the page requested from dispatcher (http://localhost/content/experience-aem/english.html) returns a 200 and available, but not cached in dispatcher; the reason could be login-token cookie present in browser. So you are logged into AEM in one tab and try to access the page on dispatcher in another tab; an AEM session is available and the following statement is logged in dispatcher.log

2) The statement above request contains authorization is logged when there is a login-token cookie sent with request (in this configuration, author AEM and dispatcher both have domain localhost). Using any REST client (chrome extension POSTMAN) to view the request headers and cookies...

3) Try accessing http://localhost/content/experience-aem/english.html by logging out of AEM or opening a new browser instance or deleting the login-token cookie from browser or configure /allowAuthorized in dispatcher.any (for more information on caching requests containing authentication information check this adobe documentation) and the page html should be cached

           Request sent with no login-token cookie (firefox with no AEM session)

           Log statement for create cache

           Cached file on file system

   

Dispatcher - Cache Invalidation 

1) Invalidating dispatcher cache is required to make sure it doesn't serve stale content. Auto invalidation or sending cache invalidate requests - /dispatcher/invalidate.cache are ways to delete  (invalidate) dispatcher cache

2) For publish instance to send dispatcher invalidate cache requests when there is content update on a page, the Dispatcher Flush Agent needs to be configured

3) Access http://localhost:4503/etc/replication/agents.publish.html, click on Dispatcher Flush (flush) agent -> Settings Edit and enable it (if you are not logged into publish instance, a 404 is returned; login to publish CRX http://localhost:4503/crx/de)

4) Enter the invalidate cache url of dispatcher running on port 80 - http://localhost:80/dispatcher/invalidate.cache

5) Here is the dispatcher flush agent on publish enabled

6) Publish is now configured to send invalidate cache requests to dispatcher; lets modify the page on author and publish it

7) When the page gets replicated (published) from author to publish the following statements should be logged in publish\crx-quickstart\logs\error.log confirming the page flush from dispatcher cache

8) Dispatcher flush request received and logged, in dispatcher.log as shown in picture below. The page should have been deleted from file system (eg. from C:\dev\code\install\Apache2.2\dispatcher\cache\content\experience-aem)

9) Access the page again to see updated content (and cache it)

10) A manual cache invalidation request can also be sent using any rest client (or curl) by adding the following custom headers. To send a cache invalidation request on page /content/experience-aem/english.html, add the following headers to request url http://localhost:80/dispatcher/invalidate.cache

                CQ-Action: Activate
                CQ-Handle: /content/experience-aem/english
                CQ-Path: /content/experience-aem/english

11) Dispatcher should be protected from unauthorized cache invalidate requests (hackers). On production systems /allowedClients should be configured in dipsatcher.any with the publish server ips sending cache invalidate requests, check adobe documentation