participants

Participants are handed workitems by the ruote engine and are expected to perform a task with them. Usually, some piece of information found in the payload of the workitem defines what/which task should be performed.

              Ruote.process_definition :name => 'accident investigation case' do
                sequence do
                  concurrence do
                    participant 'reporter 1', :task => 'gather info on accident site'
                    participant 'reporter 2', :task => 'gather info about customer'
                  end
                  participant 'hq', :task => 'handle case'
                end
              end
            

In this process definition example, the 2 reporter participants are handed each a workitem by the engine and once their work got done, a merged single workitem continues in the flow and is presented to the ‘hq’ participant.

Note that no information about the actual details of the participants are leaking to the process definition. By reading the definition, the only clue we get is that participant replies are expected in order for the process run to be productive.

Nothing was specified about how workitems are to be dispatched to participants. Does it occur via SMTP ? Via a webhook ? Is the workitem placed in a database for consultation by the real participants ? Is the workitem printed and sent tied to a pigeon carrier ? The process definition yields no clue about that.

How this page flows :


registering participants

Process definitions are in most cases pointed at to the engine right at the launch time. The engine parses the definition and creates a process instance.

Participant implementations are usually bound in the engine when it is started. The term is to “register a participant in the engine”. The engine is then equipped with a directory of participants that it looks up by name when a participant expression is applied, signifying that its workitem has to be despatched to a real participant.

A bit of history, initially OpenWFE had no participant registration. The participant list was an XML document mapping regular expression to participant implementation. When porting OpenWFE to ruby, I went for the simple Engine#register_participant method, with a participant list held in memory.

Now since ruote 2.1, its multiple workers and engines sharing a storage, the participant list is persisted as well. That means that registering participant once is sufficient, that also means you could have “leftovers”, participants registered but not more needed.

Two techniques against leftovers : register only the first time (when the participant list is empty) or register all at once (see Engine#participant_list= a few paragraphs below)

Here are the methods available to register participants :

method notes warning
engine#register_participant registers one by one
engine#register registers in one go, clears previous list (unless :clear => false)
engine#participant_list= registers in one go, clears previous list low-level

Probably, if you have a “static” list of participants, engine#register is the best way. If you add participants here and there engine#register_participant might be OK, unless you want more control, implement some business logic for adding participants that wraps engine#participant_list=.

how participants are resolved (looked up)

The participants are registered in a list.

When ruote has to deliver work to a participant, it looks it up in the participant list, starting with the first entry whose regular expression matches the participant name.

              engine.register_participant 'user_alice', Acme::ThisParticipant
              engine.register_participant /^user_/, Acme::ThatParticipant
            
              pdef = Ruote.process_definition do
                user_alice
                user_bob
              end
            
              engine.launch(pdef)
            

Thus user_alice will be handled by Acme::ThisParticipant, while user_bob will be handled by Acme::ThatParticipant.

The accept? method of the participant, if present, plays its role. Thus

              class Acme::ThisParticipant
                include Ruote::LocalParticipant
                def consume(workitem)
                  # ...
                end
                def accept?(workitem)
                  workitem.fields['domain'] == 'marketing'
                end
              end
            

Alice workitems will only be handled by Acme::ThisParticipant if they are for the ‘marketing’ domain, if not they are handled by Acme::ThatParticipant (as specified when registering).

engine#register_participant

The first way to register participants in ruote is to call the engine’s register_participant method.

              engine.register_participant 'reporter 1', Ruote::StorageParticipant
            

Here, the participant named “reporter 1” is registered in the engine. We trust the engine with the actual instantiation and thus pass only the class : Ruote::StorageParticipant (this participant implementation places workitems in the storage).

In order to avoid writing too much code for binding participants, it’s OK to leverage regular expressions. Here is something that can cope with ‘reporter 1’ and ‘reporter 2’ and many more :

              engine.register_participant /^reporter /, Ruote::StorageParticipant
            

Don’t worry about the participant implementation mixing the workitems,

              concurrence do
                participant 'reporter 1', :task => 'gather info on accident site'
                participant 'reporter 2', :task => 'gather info about customer'
              end
            

in our example, the attribute ‘participant’ of the top workitem while be ‘reporter 1’ vs ‘reporter 2’ for the bottom one, though the same participant implementation consumed the workitem.

NOTE : by default, engine#register_participant will delete any previously registered participant with the same regex and place the new participant at the end of the list.

Thus

              engine.register_participant 'engineering', Ruote::StorageParticipant
              engine.register_participant 'marketing', Ruote::StorageParticipant
            
              # and then later ...
            
              engine.register_participant 'engineering', Acme::EngineerParticipant
            

results in a participant list with 2 participants, ‘marketing’ then ‘engineering’.

The :position option may help :

              engine.register_participant 'alice', Acme::Participant
            
              engine.register_participant 'bob', Acme::Participant, :position => 'first'
                # will register bob before alice
            
              engine.register_participant 'charly', Acme::Participant, :pos => 1
                # will register charly between bob and alice
            

If you don’t want to override previous participants with the same regex, you can use the :override => false option or :position => ‘after’ or ‘before’

              engine.register_participant 'alice', Acme::ThisParticipant
              engine.register_participant 'alice', Acme::ThatParticipant, :override => false
                # will result in a participant list with two participants registerd under
                # /^alice$/
            
              engine.register_participant 'alice', Acme::ThisParticipant
              engine.register_participant 'alice', Acme::ThatParticipant, :pos => 'before'
                # will register the second /^alice$/ before the first one
                # (the Acme::ThisParticipant one)
            
              engine.register_participant 'alice', Acme::ThereParticipant, :pos => 'after'
                # will register the third /^alice$/ after the last one of the alices
                # (in this case Acme::ThisParticipant one)
            

Having multiple registrations for the same regex is useful when using the #accept? method of participant implementations.

It’s probably simpler to register all the participants at once with engine#register instead of fiddling with engine#register_participant and its options.

engine#register(&block)

Thanks to the work of Torsten, it’s possible to register participants in nice blocks :

              engine.register do
            
                notify MyApp::Participants::RemoteNotification, 'flavour' => 'normal'
                alarm MyApp::Participants::RemoteNotification, 'flavour' => 'alarm'
                  # two participants for notification and alarms
            
                catchall Ruote::StorageParticipant
                  # all the workitems for participants that are neither 'notify' or 'alarm'
                  # are caught by a storage participant
              end
            

Note that simply stating “catchall” will redirect to a/the storage participant :

              engine.register do
                catchall
              end
            

is thus equivalent to

              engine.register do
                catchall Ruote::StorageParticipant
              end
            

The first example above assumes the participant name is the first word encountered (the ‘method’ name in fact). If you want to pass a regular expression, use the ‘participant’ notation :

              engine.register do
                participant 'user-.+', Ruote::StorageParticipant
                participant 'action-.+', MyApp::RemoteActionParticipant
              end
            

NOTE : since ruote 2.3.0 register(&block) will clear the participant list by default, thus after

              engine.register do
                participant 'engineering', Ruote::StorageParticipant
                participant 'marketing', Acme::StorageParticipant
              end
            

one ends up with 2 participants listed, ‘engineering’ and ‘marketing’, whatever participants you had previously.

One can prevent this clearing by doing

              engine.register :clear => false do
                participant 'engineering', Ruote::StorageParticipant
                participant 'marketing', Acme::StorageParticipant
              end
            

engine#register(&block) and “catchall”

“catchall” when used, is placed last in a registration block:

              engine.register do
                participant 'user-.+', MyApp::InboxParticipant
                participant 'action-.+', MyApp::RemoteActionParticipant
                catchall
              end
            

This block is equivalent to:

              engine.register do
                participant 'user-.+', MyApp::InboxParticipant
                participant 'action-.+', MyApp::RemoteActionParticipant
                participant '.+', Ruote::StorageParticipant
              end
            

It catches all the workitems. Placing further participant registrations after a catchall makes thus no sense (they’ll never be used).

“catchall” can be used with participant classes (and options), like in:

                catchall Acme::DefaultParticipant
            

or:

                catchall(
                  Acme::SmtpParticipant,
                  :email => 'admin@example.com', :msg => 'unknown participant')
            

engine#participant_list=

Starting with ruote 2.1.11, instead of using register_participant and register, you can pass the whole list of participants at once.

              engine.participant_list = [
                [ /^user-.+$/, 'Ruote::StorageParticipant', {} ],
                [ /^action-.+$/, MyApp::RemoteActionParticipant, { 'server' => 'main' } ]
              ]
            

Note that it’s OK to pass the class name as an instance of Class (second line) or as a string (first line).

Checking the current content of the participant list looks like :

              engine.participant_list.each { |pe| puts pe.to_s }
                #
                # /^user-.+$/ ==> Ruote::StorageParticipant {}
                # /^action-.+$/ ==> MyApp::RemoteActionParticipipant { 'server' => 'main' }
            

participants and threads

As you may have gleaned from this doc or from a blog post, ruote 2.1 tries to have only 1 thread per / for the worker, handling all the workflow activity (direct launch/apply orders or triggering schedules). So you end up with 1 extra thread per worker. Note that if you only have an engine (and let the worker run in another runtime, there is no extra thread used by ruote).

(There is an exception with a worker bound to ruote-couch, it uses one extra thread to ‘observe’ CouchDB (instead of polling it))

Since, most of the time, participant are IO bound, having the dispatching work performed in the worker thread would mean that each delivery to a participant monopolizes the worker. That’s why, by default, ruote does each participant#consume call in a new thread.

(Maybe we should have a switch to disable participant threading “en masse”, it could be OK for small organization deployments, ping me on the ML or on #ruote if you need this)

A participant instance may inform ruote that it doesn’t want/need to have its consume method called in a new thread each time. It does that by making sure it has a do_not_thread method and that this method returns true. Since it’s a method, depending on its implementation, it might not always return true.

As an illustration, the Ruote::StorageParticipant class returns true for do_not_thread (all the time). (trusting you not to have put your storage on Mars while your worker is on Earth).

Some ruote users, have their own implementation of the dispatch pool that enforces an upper threshold for the thread count.

The “dispatch” thread is discarded as soon as the delivery to the participant (or the participant consumption and reply) is done.

see also