Workflow

Design Proposal

Application designers should be able to create a set of custom workflows.

A workflow consists of a number of different states and transitions. Each state and each transition has a script/template associated with it.

A workflow is defined by setting the 'user.workflow.html' template. Default this template will do nothing. A set of standard workflows will be defined that can be enabled by calling them from this template.

Whenever an object is saved, the save() function of that object will check the current workflow set, and based on the type of the current object will run the control template. If the control template finds that all the conditions for a transition are there, it will then perform user defined actions for that transition. The default workflow may divide this up into a transition-from template for the current state and a transition-to template for the new state.

The workflow template and all templates called from it, may not in any way call a save function on the current object, since that would create a loop. Any changes in the current objects data should simply be set, and save() should be left to actually save it. This way the conditioncheck-transition is automatically atomic: nothing will be saved, or everything. This does mean that any changes in other objects are not automatically atomic, the developer must  take care that no undefined state is reached in this way.

The save function must check that no config data is changed by the workflow templates, since they are user definable in pinp via the layout grant, and could therefor be misused to circumvent security settings. The only way to do this is if the configuration data is seperate from the normal data and can be saved before the workflow call and reset after the call. This means that the change to set all configuration data in $data->config must be implemented for the workflow system to work.

The workflow template must also have the ability to add properties to the list of properties to save. One way to do this is to return the list of properties from the template. Upon an error, the value returned will be 0 (or false):

The code in pobject::save() would then look something like:

Each object will keep track of it's current status. It will also keep track of the user or group (or script) to which it is currently assigned, if any.

Furthermore, upon each save, a log is kept for each object, which contains the date, user and state change, with perhaps some comments by users.

The state as described above will only be used in the workflow setting. It is not used to define if an object is published on a website or not. For this purpose extra properties must be defined, which will be set or unset by the transition actions (or templates).

This state must be kept in the objects data, and also as a property, to allow for reports. Each state has an identifier.

Actions to take, operator (who or what needs to do something) and comments should be kept distinct from this state identifier, as this is something that the workflow transition code should fill in, depending on object type and user defined workflow.

So in the object a new data field for workflow is needed, like:

Each workflow template may add extra fields to the workflow section, if it needs any, e.g.

which signifies the next user / group /  program to perform an action, or

which add comments for the next operator

 The workflow state must also be kept as a property. For that only the current state of the object is needed. All extra workflow fields should use their own

properties if they need any. Thus:

For the extra operator field, a new property must be defined.

One problem with this approach is that you may want to generate overviews with objects in many different states visible. This means that you must add a new property, which is set by the transition-to step for those states. Combined with the operator, this is usefull enough to create a property set for this by default, which we'll add to the workflow property:

This property allows you to easily generate overviews per operator of 'incoming' items and their status, etc.

---------------

> A workflow consists of a number of different states and transitions.
> Each state and each transition has a script/template associated with it.

Perhaps it is good to clarify some definitions we have made in our discussions.

Workflow support in our view means having a way to have objects (i.e. documents) go through transitions based on certain criteria;

- state: the current state of the object (is it being edited, being reviewed, published ?)

- conditioncheck: test for a set of criteria that identify when an object must go to the next state

- transition: the act of moving an object from one state to another

- action: activity required or done by a person (or potentially an automated system) to allow for the next test to succeed

Verbal example: A document is written for a website. The website is programmed so only 'published' documents are visible:

- Once the writer is finished he marks it 'up for review' (i.e. tick a checkbox) and save the document

- The system identifies a test has succeeded (is the 'up for review' box checked ?) and starts the transition

- The transition will now modify the document so that the state reflects the next state and required action: someone must review the document.

- The reviewer sees the document and read is. If he agrees he can tick the 'publish' box and save it. Or he could possibly send it back with comments.

- Again, the system identifies a test has succeeded (is the 'publish' box checked ?) and starts the transition

- The transition will now modify the document so that the state reflects the next state: published and possibly required action: review or update document again in 3 months...

Functionally (backend) this can be done as Auke described:

> The workflow template and all templates called from it, may not in any way call
> a save function on the current object, since that would create a loop. Any
> changes in the current objects data should simply be set, and save() should be
> left to actually save it. This way the conditioncheck-transition is
> automatically atomic: nothing will be saved, or everything. This does mean that
> any changes in other objects are not automatically atomic, the developer must
> take care that no undefined state is reached in this way.

<rant>For the template developer this is an issue, but even more so for the workflow-designer. It is a serious case of BAD PRACTICE if a workflow is designed in such a way that not all conditions are checked combined; i.e. if the workflow breaks because your site-designer decided to update some spiffy script, it is really the workflow-designer who is to blame!</rant>

Hm, let me add a note here. I considered that in some cases it will be possible for a workflow design to have all criteria met, but still have faulty data. I.e. one might have assigned the wrong person for the next action. Let me pose another option for you all:

Would it make sense to not operate when the object is saved() but when it is closed(). I.e. consider all objects in a workflow situation to be locked for operation, and start testing criteria and doing transitions just before the lock is removed ? I think this would be a safer way, both for programmers and for users. It also blocks the road to potential conflicts when an object is assigned to a group, and 2 users from this group simultaneously attempt to work on it. Again, this also has to do with good workflow design, but it might be a good idea to deal with this from a systems point of view.

> The save function must check that no config data is changed by the workflow
> templates, since they are user definable in pinp via the layout grant, and
> could therefor be misused to circumvent security settings. The only way to
> do this is if the configuration data is seperate from the normal data and
> can be saved before the workflow call and reset after the call. This means that
> the change to set all configuration data in $data->config must be implemented
> for the workflow system to work.

However, workflow may need to set user grants to allow and deny access to documents ! (Ofcourse this is also the easiest way to arrange or know if documents will be visible on the website: No grants for Joe Public)

> Furthermore, upon each save, a log is kept for each object, which contains
> the date, user and state change, with perhaps some comments by users.
>
> The state as described above will only be used in the workflow setting. It
> is not used to define if an object is published on a website or not. For this
> purpose extra properties must be defined, which will be set or unset by the
> transition actions (or templates).

Must it ? (grants idea above)

> This property allows you to easily generate overviews per operator of
> 'incoming' items and their status, etc.

Also, it will be wise to allow a 'supervisor' for any workflow process, who must monitor possible due-dates for objects, and who can intervene if objects are not processed 'quick enough'. The administrator could do such a thing ofcourse, but if we consider large businesses, this would for instance be a manager, so it should be a delegateable position :-)

-----

>I fail to see the use of a transition-from template. I feel this will
>make things only more non-obvious.
>Other than doing sanity-checks which are needed for all state-exits,
>what is the purpose ? I think a better way is to keep it simple and just
>allow the workflow-programmer to call his own functiontemplates from the
>transition-template.

You don't have to use a transition-from and -to template, you have full  control over what the user.workflow.html template does. It just seemed to me that it makes sense to add standard building blocks which can be used independant of a specific workflow. This means that if you define a default set of states which may be used in user defined workflows, you should also create the transition from and to steps, so that they can be used as well. In this case the transition-from template should unset any properties specific for the state it left.

Again, this is all optional, but it might be a good approach to allow programmers to easily create a more or less standard workflow, with states with a defined meaning, which allows seperate systems to work together.

><rant>For the template developer this is an issue, but even more so for
>the workflow-designer. It is a serious case of BAD PRACTICE if a workflow is
>designed in such a way that not all conditions are checked combined;
>i.e. if the workflow breaks because your site-designer decided to update
>some spiffy script, it is really the workflow-designer who is to
>blame!</rant>
>
>Hm, let me add a note here. I considered that in some cases it will be
>possible for a workflow design to have all criteria met, but still have
>faulty data. I.e. one might have assigned the wrong person for the next
>action. Let me pose another option for you all:
>Would it make sense to not operate when the object is saved() but when
>it is closed(). I.e. consider all objects in a workflow situation to be
>locked for operation, and start testing criteria and doing transitions
>just before the lock is removed ? I think this would be a safer way,
>both for programmers and for users. It also blocks the road to potential
>conflicts when an object is assigned to a group, and 2 users from this
>group simultaneously attempt to work on it. Again, this also has to do
>with good workflow design, but it might be a good idea to deal with this
>from a systems point of view.

Hm, if I understand you correctly, you want to make sure that after the  workflow, nothing else is done to the object, correct? That is what your point about closed() instead of saved() is about? Unfortunately, there is currently no close() operation for objects. I'm not sure I'm willing to introduce it either. Ariadne is optimized for read access, since that is the most frequent activity needed for websites by orders of magnitude. If a close() operation is introduced for objects, and it must in some way check state, should that close() only be called when an object has been saved in the processing of the current request? If so, how should one keep track of all objects which have been saved? If not, what should the close() function do?

In short, I think the semantics of a save() operation are clear and easily understood, the semantics of a close() operation are only clear when something associated with the object is open, like a connection (in the case of a database connector) and in that case it is not at all clear that close should do anything in regard to workflow.

(Hm, I just noticed that I've just made the case for a close() operator... oh well.. we'll get to that eventually :)

Now, if a site designer decides to update a 'spiffy' script, used in the workflow process, designed by someone else, that site designer is to blame. We could start defining new template sets with new grants to seperate layout and workflow, but that seems overkill to me. Scripts that aren't used in the workflow process, cannot influence it. So if the workflow designer decides to create or update other objects, he/she should be aware that by default these actions are not atomic. If the user aborts halfway in the proces, some objects may have been saved and others not, while the 'root' object isn't saved.

If anyone has a practical solution for this, I'd like to hear :)

>However, workflow may need to set user grants to allow and deny access
>to documents ! (Ofcourse this is also the easiest way to arrange or know
>if documents will be visible on the website: No grants for Joe Public)

Hm, you are right, but changing the config data is not the solution, since grants are to be kept in the object and also in the user or group. There are templates defined to set grants, and they should be used. But in this case that isn't sufficient either, since the template would change the object in the store, not in memory, and upon saving the object in memory, it would break one side of the grants system.

The only way to do this, is to let the user.workflow.html template return or set a value, based on which the save() function knows what to do, and then let it set grants, etc.

This means that the workflow template should not simply return a list of properties, but a hash with information for save().

>> The state as described above will only be used in the workflow
>> setting. It is not used to define if an object is published on a
>> website or not. For this purpose extra properties must be defined,
>> which will be set or unset by the transition actions (or templates).
>Must it ? (grants idea above)

Well, if you keep the workflow states seperate from any other semantics, you do not risk overburdening them. e.g. using a workflow state to check whether or not an object is published. Since it is very much possible to have multiple workflow states in which the object should be 'published'. This should prevent people from trying to 'fix' the workflow, when all they want is to change the 'effect' (the object is publically visible or not).

>>
>> $workflow["group"]["string"]=16;
>> $workflow["operator"]["string"]=128;
>> $store->create_property("workflow",$workflow);
>>
>> This property allows you to easily generate overviews per operator of
>> 'incoming' items and their status, etc.
>
>Also, it will be wise to allow a 'supervisor' for any workflow process,
>who must monitor possible due-dates for objects, and who can intervene
>if objects are not processed 'quick enough'. The administrator could do
>such a thing ofcourse, but if we consider large businesses, this would
>for instance be a manager, so it should be a delegateable position :-)

Hm, I'm not sure this needs to be built in to the workflow properties table, but then again I'm not sure how otherwise to do it either. I'll have to think about it :)

-----------

From: "Arnaud Siminski" <asiminski@##########>
Subject: Re: Ariadne backend

Hi all !

I read the design proposal for workflow managment that Auke wrote on 09/20. That sounds great! Is there some specific technical problems of making it "nls compatible", I mean... having the possibility to set different workflows to the same object that depend on the language of the data ?

Let's say an article could not be publish before validation, we can easily imagine that the french version of the document could be released and published by feb. 3rd and the spanish one by feb. 4th. We can as well imagine that the validation process could be different because the personn (users) who are going to validate the article (datas) are not going to be the same (2 people, probably from 2 differents users' group, acting on 2 different datas ... means 2 workflows right ?)

It would be really nice to get that done for Ariadne, do you have any updates of the design proposal ? I would be very interested to read it.

Have a good day,
Arnaud Siminski
www.RollingCost.com