Workflow Logs API and Elastic Search

We understand that good workflow requires rich and comprehensive workflow logs to be valuable.   A workflow engine performs work according to a predefined process map. If you choose to use ProcessMaker I/O, you will need a way to understand exactly how your workflows were executed.  We also think that you will want your logs to be fully searchable.

In the most recent update of ProcessMaker.io, we added workflow logs to our API.

Logs API endpoints

We have added two endpoints to our API.  Both use a GET Request.  Here are the endpoints:

get listLogs - https://CHANGEME.api.processmaker.io/api/v1/logs/

get logSummary - https://CHANGEME.api.processmaker.io/api/v1/logs/summary

The Importance of Audit Logs and Good Search for Workflow Logs

Today’s international regulatory environment makes workflow logs even more important. GDPR legislation will go into effect in May 2018 throughout Europe. This new legislation will mean a lot of changes for software vendors and data handlers. From a workflow perspective, it will mean even more requirements for strong workflow event logging.

Management Interface

With our most recent update to ProcessMaker I/O we have also improved our instance management interface. If you follow our getting started blog for developers to deploy some of our sample Github code, you will see that we have made it easier to manage users and see running processes. You will also notice a new section called workflow logs.

We are logging everything in ProcessMaker I/O. Our workflow logs includes the following level of detail:

• Time stamps – In the workflow logs API you will see that the logs include full time stamping for token flow down to the thousandth of a millisecond.  If this is not enough, let us know.
• Action – The actions include everything that happens in the engine.  What happens when an event is triggered, what happens on the arrival of a token, what happens when the engine is processing a token, what happens when it finishes processing the token…the list goes on.
• Details – The detail will show each item in the json string

Here is an example log record:

2017-11-17 15:58:33.467
DEBUG
onTokenArrival

{"instance_id":4,"token_id":4,"token_uid":"2f484386-85bd-41d2-af23-a37cc1d7a53d","target_id":3,"target_type":"event","type":"END","name":"End"}

Searching Your Workflow Logs

All of this log data is great.  However, with so much detail, you really need powerful and lightening fast search if you want to take advantage of the logs.  Well, the good news is that we are storing all of the logs in Elastic Search in our cloud.  This means that you can do quick searches through the ProcessMaker I/O interface.

Coming Next

Our engineering team is currently working on a number of interesting new features for ProcessMaker I/O.  Expect to see a release soon that will update the workflow engine to provide support for Sub-Process Execution based on the BPMN 2.0 standard.

Also, we will be adding a way to track your token transitions inside the admin console.  Our initial pricing is based on token transitions, so we understand that it is important to provide visibility into the token transitions.  The log of token transitions will also act as an activity log so that it is clear when and to whom tokens (aka cases) were routed inside your engine.