AEM 61 - Offloading DAM Upload Asset Workflow Process

Goal


Create a Master/Worker Author AEM Topology for offloading the asset upload post process, to a worker instance. So the worker instance is specifically configured to execute DAM Update Asset Workflow steps on assets uploaded to master instance.

1) Master is configured with DAM Update Asset Offloading Workflow
2) Users never see the worker author and always upload assets to master author
3) When assets are uploaded, a replication agent on master sends the asset package to worker
4) Worker executes DAM Update Asset Workflow on asset
5) Using reverse replication, the package with processed asset and renditions is replicated to master from worker

Clearly, processing too many heavy DAM assets in the author AEM instance (also used for authoring web pages) can drastically bring down performance, so offloading DAM Update Asset Workflow in such cases is considered wise....

Check adobe documentation for more information on Offloading Jobs & Sling Discovery features. The following steps are for local setup, so offloading replication agents use admin credentials and not a dedicated replication user.

Demo

Solution


1) Assuming there are two AEM author instances running on two separate machines (windows & mac) in same network...

                         Host                      IP Address                  OS                        Type                        URL

                         nalabotu-w7          192.168.0.3                  Windows              Master                       http://nalabotu-w7:4502/welcome
                         nalabotu-osx         192.168.0.8                   Macintosh            Worker                      http://nalabotu-osx:4502/welcome
                       
2) The first step is to add worker (nalabout-osx) in master (nalabotu-w7) topology. Here is the topology configuration on master before adding a worker; one instance exists with sling id ab3f05cd-1669-4c8a-b538-e24f6b37cfdf

                            http://nalabotu-w7:4502/system/console/topology

3) Access the worker topology management console. It has one instance with sling id da632281-8b9b-4aad-bfe9-89abdff4d471

                            http://nalabotu-osx:4502/system/console/topology



4) In the worker topology, configure sling discovery service with topology connector url (the toplogy connector running on master instance)

                            http://nalabotu-osx:4502/system/console/configMgr/org.apache.sling.discovery.impl.Config

                            Topology Connector URL: http://nalabotu-w7:4502/libs/sling/topology/connector



5) Add the worker hostname and ip address in master discovery service white list configuration.

                            http://nalabotu-w7:4502/system/console/configMgr/org.apache.sling.discovery.impl.Config

                            Topology Connector Whitelist: nalabotu-osx, 192.168.0.8



6) The topology after successfully adding a worker


7) The system should have created necessary replication agents on master. Access the replication agents console on master author. The replication agent responsible for sending asset input payload to worker is named offloading_<worker-sling-id>, and the reverse replication agent for getting processed package back into master is named offloading_reverse_<worker-sling-id>

                            http://nalabotu-w7:4502/etc/replication/agents.author.html



8) Granite offloading console shows the consumers for each topic, when a worker joins topology the consumers for offloading topic com/adobe/granite/workflow/offloading are available and active on both master and worker. To avoid master also processing the uploaded asset (only worker should), disable the consumer on master. In the screenshot below consumer on master (nalabotu-w7 192.168.0.3) is disabled, shown with red button

                            http://nalabotu-w7:4502/libs/granite/offloading/content/view.html



9) The next step is to configure DAM Update Asset Workflow Launchers on Master (nalabotu-w7). This step is to initiate, offloading workflow and not ootb regular update asset. Access Launcher console and change DAM Update Asset to DAM Update Asset Offloading workflow

                            http://nalabotu-w7:4502/libs/cq/workflow/content/console.html


                            Before Update



                            After Updating to DAM Update Asset Offloading



10) Access worker Launcher console and disable the DAM Update Asset Workflow. The assets are not uploaded to worker, but offloaded from master; so DAM Update Asset Workflow should remain disabled on worker



11) Upload any assets to master http://nalabotu-w7:4502/assets.html and the DAM Update Asset process happens on worker http://nalabotu-osx:4502/assets.html. Check the replication agent log



1 comment: