Simulation not in the BPMS please

The efforts of BPMS vendors to deliver simulation of processes have not been impressive ... even the best is more like a visualisation of the assumptions made in the process design rather than a model  that can usefully explore the real world factors that may influence future process design.
Bruce Silver has been able to consider this from the perspective of the process designer and concludes , among other things, that "Simulation should not be embedded in a BPMS". 
I agree and have argued before that the simulation should be handled by a dedicated simulation tool (after all these have been around longer than BPMS). see this post.
Fortunately we now have a formalised model of the business process in BPMN 2 that can be transformed into elements useful for a simulation engine. Similarly, the control blocks and flows that characterise a simulation model could be transformed into the starting point of a BPMN. The relationship between the simulation and the process model is 2-way.
Choose your BPMS toolset on its ability to delivery process design and implementation. Choose your simulation tool for its ability to support the analysis of the real world process under investigation.
Although I sympathise with Bruce's view that "Simulation should not require programming",  there will be occasions when the complexity and value of the model will merit more effort and experience. I suggest that the ideal simulation tool should have full logic development capability (programming) in addition to a simple wiring diagram approach.

Dynamic BPM Success Story

Sandy Kemsley brought us some good news here with

• a government department going from legislation to operation system in 12 months;
• agile methods in a government program of work;
• (dynamic) BPM underlying case management

What a pity the growth 'industry' is the UK's Insolvency Service.

BPMN only part of the solution

Scott Francis takes up the pen in the the ongoing and tedious debate about BPMN ( enough, too much, too hard, too rigid, too imprecise ....) and makes the excellent point that "too many people think that BPM starts and stops with BPMN!".

With a very small set of symbols, BPMN allows a range of expression of process definition from a whiteboard-quality overview to the precision of a computer algorithm for calculating Pi. Even with a palette of 1007667 English words, it is very difficult to get the necessary precision for specifying a business process. Working on the basis of the average reader using only a small subset of those possible words (about 2000) it is clear that we need something more formal than text to get to commitment on what a process is or should be.

It would be really scary if those responsible for the operation of multimillion dollar enterprises can't take on the meaning of a set of symbols that can be put on a small wallchart.

The sad thing is that there is not a similar set of symbols that could be used to encapsulate other aspects of the business canvas so that discussions on whiteboards, or around coffee-bar napkins could be more enduring and useful communication of requirements.

BPMN 2.0 Best Practice

The LinkedIn Group for BPMN has a discussion on the need for a best practice for BPMN 2.0.
People come at BPMN from a wide spectrum of use from initial discussion of a business process to the IT developer perspective and a matching wide variety of views of best practice will evolve.
BPMN supports a design to execution path.
Tom DeBevois makes a similar point ...

“Certainly, BPMN enables a powerful requirements gathering tool and it can be a part of the over-the-wall requirement. Still, the central focus of BPMN is to support an agile executing environment and promote visible active, responsive change.”

This focus in the notation is in my view highly desirable. A side benefit at BPMN 2.0 has been the tightening up of syntax and meaning. This should help in the process development by making it relatively easy to critique specifications, as Tom was able to do ...

“For instance, consider the car sales process example (Figure 5-5 in the Method and Style Book). At one point in the process, sales enters an order from the factory and a message is shown from the subprocess to the Factory process. This seems correct; however, when I review an actual BPMN process diagram, it suggests to me that the definition of the process might be insufficient. In reality, what happens in most corporations is a salesman takes the paper order form and punches in the car order into another system.”

This is not a defect in the example, or in the style of BPMN 2.0. It merely represents the stage reached in the process design (it is to be hoped, before critical review).
BPMN does not in itself do the design. Transforming thoughts and words into the formality of BPMN 2.0 reduces the ambiguity of common spoken language. Coming from the technical disciplines of IT , I am attracted by the concept of an execution specification of the process that relates directly to an expression of the apparent problem being addressed. I anticipate a few round trips between business process specification and execution design. That is, it is still best for there to be an ongoing conversation between the person originating the problem and the provider of the solution rather than throwing arbitrary specifications over a wall.
Jakob Freund provided a clear example of the happy path through the design process in BPMS. I particularly liked the transition between viewing the process as flowing between departments of an organisation (level 1) to those departments carrying out independent and complex activities communicating through messages (level 2).


This is an example of what could be described as a “best practice” as ..

• the process is decomposed by responsibility
• information required by the various responsibilities is clarified (it is the message content)
• organisational change (eg outsource development) does not necessarily force process change

It would be useful for organisations adopting BPMN to be able to point to practice notes. Those “best practices” must be appropriate to the organisation and to the task in hand.
I suggest that a reasonable approach at this stage in the life of BPMN the best approach is to take one or more of the publications like Bruce Silver’s and go through it crossing out and highlighting depending whether you feel the ‘rule’ is appropriate. This then becomes the initial quality manual for BPMN.

I do feel that any BPMN symbols used in the early stages should be used correctly.
Things important to the expression of the business requirement should be included (however complicated the diagramatic representation) but avoid the temptation to express unnecessary “how to” in the model. I do not feel that the symbol palette for level 1 or 2 of BPMN 2.0 should be a constraint on the modeller. It is more appropriate as a compliance statement for tool providers.

BPM not appropriate here?

Keith Swenson used the example of a major amalgamation to indicate an area where BPM would not be applied in his post Where did I put that major airline merger process definition?
Of course that may be true in that particular case, but it is poor argument for decision-making about how you are to manage a process, or processes in general.
I accept that there are things you just do (let's have lunch today...) and that some things come about despite having no obvious plan or process (UK government, eventually) but dispute the premise that BPM should not be playing a part.
One problem with BPM, is the assumption that the process cannot be changed in flight. It seems entirely reasonable that

1. a process instance should be followed ;
   
2. have a “I’m sorry, Dave. I’m afraid I can’t do that.” moment;
   
3. and be changed at that point.
   

Visualising the state in a BPMN diagram allows a view of what has been followed up to a state and what is expected to follow. The executable BPMN engines of today will struggle with a change to in-flight process instances but there is still plenty of development mileage to run there.

Incidentally, scheduling lunch or any other meeting often does need to follow a process while personal assistants juggle diaries. An advantage of formalising those processes is that all the players understand what is going on and play their part in a choreographed fashion. There are a number of function-specific tools that hide the complexity of process involved in some common activities but that does not mean the disciplines of BPM should be ignored.

Keep the coding out of BPM!

Phil Ayres writes on experiences in the refreshingly un-politically-correct world of Mexico City. I particularly liked the learning point:

if you can avoid coding (and in my opinion HTML, Javascript and all that proprietary script inside a BPM tool is code) do so. Code makes BPM solutions brittle, and hard for business users to update in the future.

The developers of BPM solutions have done a good job of providing process design tools and execution engines around the process flow. Unfortunately, the likes of Intalio, clutter their core idea with attempts at being a total application development system and fail. Even HelloWorld examples get cluttered with XPATH, SQL, XFORMS and Javascript.
BPM solutions seem to have been accidentally painted as an alternative to disciplined development and lessons painfully learned in IT-shops are having to be relearned.

It would be nice to see BPM solutions architected with clear separation between presentation, data-handling, business decisions, rules and process flow logic. Coding is required somewhere along the way but we should all understand where and who is responsible for it.

Right answer in selecting a BPMS - three BPMS???

Janelle Hill, Gartner Research VP,  was quoted by Mike Gammage in The BPMS Finds Its Home

'The right answer in selecting a BPMS is often three BPMSs, based on the particular projects' needs.'  

An immediate reaction to this is "what twaddle, doing the same thing 3 ways ..." but this may just be a characteristic of our time.

A need for end to end process definitions  was established (and it would be nice for them to be easily transformed into execution). IT solution vendors are just catching up by delivering products or relabelling earlier ideas as BPMS.

Governance practices over introduction of new technology are heavily weighted to buy rather than build, for very good reasons. Unfortunately, the Business Process and its management is only a component of the solution to any practical business problem.

Pure-play BPMS vendors emerged providing an attractive solution to the technical need for a means of execution of business process specifications but a very sketchy idea of how solutions could be delivered. Grafting on Business Rules Engines, rudimentary web user interfaces and database may make the product look attractive to a small development shop but tends not to tick the boxes for a project focussed on getting the business working quickly. A reasonable judgment call would be to accept the need for standard expressions of business process (BPMN, etc) and simply require the bought solutions to be compliant. This may result in multiple products but at least the short-term needs are being met.

If the BPMS bit of the technology portfolio looks to be an area where rationalisation to a single implementation is a priority, we actually need to be able to design to use BPMS as a component of the total solution and not expect the BPMS component or vendor to be the driver for every solution. As Mike Gammage says,

Out in the real world, business processes are only partly automated, and they need to be made visible and managed and improved end-to-end.  If BPM is seen as being focused on just one part of the CIO's portfolio of automation capabilities, then we've surely lost the plot.

So in our Enterprise Architecture efforts we need the business to be formalised in Business Rules, Business Process, and Decisions as well as having technology pieces that map to these formal constructs of the Business Operating Model.

Demo vs. Production BPM-based Systems

In his article Demo vs. Production BPM-based Systems Anatoly Belychook sounds a warning that the habit of BPMS solution vendors of presenting their suite as a complete application development solution through 'demo' applications can lead to serious problems for users when they try to product ionise even a single process. I have quoted Anatoly's points in full and annotated them with my thoughts.

1. The user portal - web application that starts processes, displays the list of tasks assigned to the user, manage activity forms for these tasks, monitors and administers processes. It will have different design in production and most likely different functionality too. If you’re lucky you will be able to customize out-of-the-box portal but be prepared to rewrite it from scratch at some point. Or to get away from a standalone BPM portal completely and wire process functionality into corporate applications. The reason: users typically do not accept BPMS supplier’s opinion that BPM should be the center of user’s universe. How true. It may be unfair to label the provision of a BPMS-centric portal or application as a fault or failure of the manufacturer ... it would be very hard to sell the product without being able to show an end-end model of implementation. The real failure is for the BPMS solutions to be presented without any clear directions of how they fit into common enterprise technical architectures and then black-boxing the behaviours of the BPMS.
   
2. In particular, you should eventually get rid of “start process” button. From user’s perspective, he doesn’t “start a process” but do something real e.g. accepts the incoming order or submits a request for vacation. The system must start the appropriate process transparently. Insightful ... just like the rest of the IT/IS components the BPMS is a tool that transparently handles a bit of business communication there is no need to bring the terms of bpms (message, process, instance ...) into the end user lexicon.
   
3. Be prepared that activity forms generated by BPMS in few mouse clicks will no longer meet the functionality, usability and design requirements at some point. So it’s better to have an idea how will you eventually develop these forms in terms of tools, labor force and
   costs. The importance of this issue can not be overestimated: what good is that the process scheme is depicted in two days if forms development for this process then takes say two months? (I do not play down the importance of rapid prototyping of screen interfaces - it’s the must for BPM, one won’t even come close to production without it.) By the way you probably would like to use the same tools to rewrite the BPM portal. BPMS implementations do not spring into a virgin site, consider how the enterprise views its technical architecture. Perhaps the architecture is layered, with presentation separate from the 'application code' or 'business service" . Then you will want to devise your working surface around the user actions and communicate with the BPMS with messages at the appropriate point. Of course, those messages may contain standard (within the enterprise) objects so choosing a BPMS that forces definition of the message from a form layout will not be a good idea.
   
4. Similarly you will no longer be satisfied with out-of-the-box reporting and monitoring tools at some point. Remember that this is about business requirements and a fancy dashboard updated every second showing messages per second by type is not useful beyond the server room. First catch your business requirement, then match it to the offering of the supplier. The most likely solutions will be found in generic reporting mechanisms but you will need to understand the operation of the BPMS to integrate its information about queues, instances in progress etc with information from other enterprise sources (stock on hand etc).
   
5. Demo and pilot processes typically store all data in process attributes, process variables or operands (different systems use different terminology) but only relatively insufficient and/or temporary information will be stored this way in production. Most data will go into a traditional database and only the primary key of the corresponding record will be stored within the process. Considering the process of client purchase order negotiation as an example, the information about the client and the order items are likely to be stored in a database while customer and order identifiers will remain in process attributes together with the deadline date for the call to the client. The reason to act this way is obvious: data which may be of interest after the process instance has ended must be stored so that
   they could be accessed independently from the process instance. This also means a separate user interface to this data independent from process screen forms. As for the process screen forms, they should access both process attributes via BPMS API and database fields via DBMS API. This actually requires some significant design consideration. In general, a process is a long running and non-ACID transaction. There may be instances where the information in the process instance represents a (useful) past state of data in a corporate database.
   
6. Building on the previous item, most likely the part of the long-term information (but usually not all) already have a room at your existing enterprise applications. Accordingly, the process attributes will store only the identifiers of the appropriate business objects and
   process screen forms will access the data stored within the application. (The latter isn’t an absolute requirement - the total integration is often very time-consuming so partial integration may be more justified.)
7. Similarly, while a demo or pilot most likely will store related documents (usually Word or Excel files) as attachments to a process instance, you’ll have to consider something more solid for production. The reason is the same: if the document may be of interest after the
   process instance has ended, then it must be kept independently from the process instances and user access to it must be provided independently from the user interface to the process. However you don’t need the full-blown ECM system: because BPMS takes care about the workflow you need only documents storage functionality with basic interfaces (user’s and programming) and services (search, archiving, security). If you are considering technical architecture components as a whole you have the opportunity of avoiding top-end document management systems which might be chosen for their version control and workflow capabilities because these can be realised in the generic BPMS toolset. However, just because a document management system has workflow capabilities, assuming that every process can be managed through its documents will provide worse problems than assuming that a BPMS suite can generate all the business application solutions.
   
8. Users authentication and authorization in a demo or pilot is usually done via independent LDAP directory, database or even a static list stored in the XML file. It is obvious that production system should utilize your existing user directory. But a bad surprise may be
   the amount of effort it requires. To start with there are usually several such directories. A typical example: an Active Directory, a separate authorization system within the legacy accounting system and a database keeping the users of remote offices and partner companies. As the project evolves additional requirements may arise e.g. the planned
   absence and automatic rerouting of the tasks. It is known that for a company having about a hundred of users Active Directory implementation alone is a non-trivial project and now we are facing more difficult task. As a result as much as 50% of total BPM project costs are spent on authorization and authentication issues at some projects. Imagine for a moment that it happened in your project and you didn’t take it into account in project schedule: you are out of schedule and budget for as much as 100%! The bottom line here is that the BPMS should be considered in terms of the ease or difficulty to implement within a separately chosen Identification and Authentication solution. As the world does not stand still the interfaces for authorization and authentication should be expressed in terms of standards rather than supported products.
   
9. For obvious reasons not the most complex business processes are taken for demos and pilot projects. That would be all right but worse than that, they are usually technically implemented as a single process thread. But in reality even the relatively simple employee onboarding process technically consists of several processes communicating with
   each other (it’s enough to notice that processing the incoming resumes is not directly related to the publication of vacancies). This is even more true for end-to-end processes that are of greatest interest in terms of business (see “End-to-end Process Orchestration” antipattern and “Internal Order” pattern). Accordingly, you will need more functionality from your BPMS pretty soon - not only the orchestration but also choreography. Modern BPMS are fine with that but if a rudimentary workflow and/or document management built into your accounting system is all you have then you may be in trouble.
10. And finally, a production system differs from a pilot by reliability, performance, security … but these are standard requirements not specific to BPM. Failures of the BPMS service and the infrastructure that support it have to be handled in the same way as any other operational service (separately from failures of the business process). Recovery from failure is complicated because the state of database that supports the BPMS operation is generally not synchronised other corporate databases and certainly not with invoked services.
    

BPMS solutions do offer a way addressing IT/IS delivery issues but they do not eliminate the need for the basic requirements, design and systems management that apply to all business solutions.

Drools + BPMN 2.0

An encouraging announcement from DROOLS indicating commitment to BPMN 2.0 at notation and xml representation level. This is how standards adoption should work ... do not wait for the tedious ratification and adoption votes; commit to the standard (whatever it will be); implement as it gels so that people can get used to it; fine tune implementation to standard as it gets to the ratified state.
I would put in a plea for a formal compliance document, stating what part of BPMN 2.0 is not yet implemented, so that early adopters do not waste time trying to decide whether it is the implementation or user's dumb specification of a business process that is at fault.

Heading into the clouds

I am taking my first steps into the Intalio Cloud. As an individual and for my small business, I like the concepts of cloud computing and use the Google family of services as part of my normal daily operation. This started as a Google Doc. Having an interest in the use of BPMS to formalise operations within and between organisations, I have been following the Intalio BPMS offering for some time and, despite some ragged edges, I find it a good approach.

Intalio Cloud is an interesting prospect. Where do you place it in the taxonomy of the cloud? Is it providing storage, compute power, platform, value added services ... ? Not only all of the above but apparently a range of hardware and services so that you can be your own cloud provider. An interesting scalability equation, I can operate somewhere in a blackbox datacentre with 2 users for free ; expand into a productive organisation at $x per user per month; form my own cloud service datacentre (using surplus power and cooling capacity of NZ South Island) all without changing the operational business processes.

So I will be doing a bit of tyre kicking and a much thinking about the security and other risks associated with putting the fundamentals of business operation out into the cloud. One problem with adopting a business solution in the cloud is that you may not pay much attention to what is going on to give you the results. You may trust that availability of the underlying components in the black-box data centre will be sufficient for your needs as you grow, and that your operation is secure. That last one is a bit problematic ... out of the box Intalio has you logging on with userid/password across http rather than use encryption (even https would be a great advance).

BPMN - Why and how of Signal

Thanks to Rick Geneva , I do not have to describe 'What' a Signal Intermediate Event is used for in a business process. He has provided the use cases for this useful element of BPMN. I responded quickly - now all we need is standard implementations in products like Intalio, pointing out the lack of implementation of signal in tools that support BPMN through to an executable. Of course, we can work around any lack of implementation of signal within a BPMS toolset but the meaning behind signal is not like anything else so a real implementation or common patterns of workaround would help.

You can draw the signal event in Intalio so at least the business process designer can start with a proper description of the use case even if the level-3 technical implementation diagram will differ in shape.

The BPMN specification says
A signal is a generic, simple form of communication

• Within pools (same participant)
• Across pools (different participants)
• Across Diagrams

It is communication in its simplest sense ...

• shouting out not knowing if anyone is listening, or has heard
  
• and listening for shouts unaware of any other listeners

Signalling within pools

This very important as BPMN deliberately restricts message flows to communcation between pools so if you have parallel flows within a pool and you wish to communicate an event between the flows, a signal is the only BPMN mechanism available. For example, a simple synchronisation of parallel flows would be represented like this in BPMN.

Without signal, the workaround is to use a message via another pool like this ...

This does have the merit of working in Intalio but clearly deviates
from the simple expression of the business process. Adding more
listenerswould seriously obscure the meaning of the business model with implementation artifacts.

Signalling across pools

In communication across pools, the signal has a single sender and one or more listeners while, with messages each sender is connected to a listener. Here is a simple synchronisation between parallel processes.

Replacing the signals with messages even in this simple case loses the clarity of expression of the business process

In this case the single activity of signal in master process is replaced by two message event throws and it gets progressively more complex as more processes are introduced. In addition, for every participant introduced, a change has to be made to the model of the process doing the signalling although in practice no change is made to the real business process.

Across Diagrams

The lack of a signal implementation is especially felt where the business is modelled across a number of diagrams and there is a common need for communication. An over-riding interrupt perhaps. Each separate diagram may represent a division of the organisation by end-to-end process or department with separate reactions to the communication (like a fire alarm). Implementing without signal has similar problems to the above but the separate diagrams make the implementation and business models even harder to relate together. The link element is a candidate for solution here, but implementation is missing in Intalio. So we really need a full publish - subscribe service. RSS might be a practical solution to explore.

Conclusion

There is a real need

1. for an implementation of signal events within tools that develop the executable from the BPMN model.
2. in the absence of the implementation of signal events, a well understood implementation method or pattern is required for each of the uses of signal.
   

Project management vs. process manage...

Ayalew Kassahun raised an interesting question on the LinkedIn BP Group

For many it may be weird to imagine a world in which no distinction is
made between projects and processes. However, I think that every
process instance is a mini project. ... Just for sake of discussion I would suggest that it is more appropriate
to make no distinction between project and process management. In such
a world what will then be the consequences on management,
specifications and tools?

Firstly, I think that distinctions between process and project are hard to make outside the toolsets that attempt to specialise.

My interest is in business process management toolsets and I have been following much of the discussion about BPMN, BPEL and how the process model should end up being 'executed'. I am convinced that there is a need for a specification of the process that endures from the business idea through to technical execution of an instance of the process.

If we merge the concepts of process and project, I can see some real benefits in the process design world. For example, some analysis and modelling of the process can be done at the instance level rather than engineering a highly complex general model which attempts to deal with every possible eventuality. This alone has the benefits of redeucing impact of process design bottleneck and allows for a common way of dealing with business activities leading to common reporting, management. We would no longer have to treat activities differently if they were being handled through a PMO or through BAU.

However, in the BPM world, the toolsets are not very mature and some radical rethinking would have to take place.

Because the instance of a process/project could change during its life, then tools that do not have a tight relationship between model definition and execution will really struggle to deliver. The executing process will need to be changed through the business expression and not through some programmer intervention. So I would expect the run time BPM system to be executing a development of BPMN rather than a transformation in BPEL,JAVA or whatever.

BPMN - Hard to Code???

Bruce Silver continues the valuable discussion on BPMN semantics and challenges a perception that BPMN has vague semantics.
In the example chosen, I agree with Bruce that

• having flows from downstream activities BPMN is not best practice (in that it can lead to misunderstanding by the casual reader who may more be familiar with basic flowcharts)
• there is only one reasonable interpretation of the required execution of the BPMN and therefore it is not an example of "vagueness".

Looking at the definition of the Inclusive Gateway in BPMN 1.2 , I might accept a criticism that it is hard to read in English and may benefit from formalising.

9.5.3.2 Sequence Flow Connections
This section extends the basic Gateway Sequence Flow connection rules as defined in “Common Gateway Sequence Flow Connections” on page 72. See Section 8.4.1, “Sequence Flow Rules,” on page 30 for the entire set of objects and how they may be source or targets of Sequence Flow.

• To define the inclusive nature of this Gateway’s behavior for converging Sequence Flow:

If there are multiple incoming Sequence Flow, one or more of them will be used to continue the flow of the Process. That is,

• Process flow SHALL continue when the signals (Tokens) arrive from all of the incoming Sequence Flow that are expecting a signal based on the upstream structure of the Process (e.g., an upstream Inclusive Decision).

• Some of the incoming Sequence Flow will not have signals and the pattern of which Sequence Flow will have signals may change for different instantiations of the Process.

Note – Incoming Sequence Flow that have a source that is a downstream activity (that is, is part of a loop) will be treated differently than those that have an upstream source. They will be considered as part of a different set of Sequence Flow from those Sequence Flow that have a source that is an upstream activity (my emphasis).

However it is clear from the phrase "... expecting a signal based on the upstream..." and the note in the section above that the inclusive gateway in the example has no (merge) function in the event that a sequence flow from the downstream exclusive gateway is processed. The token from the loop back is the only one that can be 'expected' at the inclusive gateway.

It will only be hard to code, if the developers have started from a point of view that each node (activity or gateway) can be transformed into an executable form (or executed directly) in isolation. The developer must consider the source of the signals (Tokens) and the structure of the process as a whole.

BPM high-wire act

Intalio appears to be eating its own dogfood very publicly, and on a highwire.
I wish Ismael and his team well with this venture as it will demonstrate that armies of strange developer types are not the key to success in getting value from a business process-centric view of solution delivery.

They are working with real business processes and have very quickly demonstrated two completely different approaches to BPM solutions with the same product.

• A Customer Support Process is presented in the early stages of BPMN expression (Level 2??) and is full of clearly recognisable business steps (like 'assign to support team..').
• The Marketting Process allows the process instance to be configured from a basic template at runtime rather than design time and the BPMN diagram is far more abstract. Perhaps there are some real developers around after all!

The template approach allows the business process to be finalised for each instance in a configuration table and should lead to a stable technical implementation without analysing the business process to death. In doing this, some of the benefits of BPM may be lost. If you look at an instance of the Marketting process through a reporting tool you will see that you are at Step N within a loop of Steps but with no real sense of flow. In contrast, looking at a customer support process instance will show where you are, and how you got there. Without wishing to fan the flames of the executable BPMN v BPEL debate, tailoring the pattern at the BPMN level and executing the result, retains the business-level communication of process requirement throughout the lifecycle.

I look forward to seeing the final implementation.

BPM is not Software Engineering

I agree with Keith Swenson that BPM is not Software Engineering
However, I think that some of the arguments are a little broad. Ultimately we need to have software construction and business process design working toward the same set of goals in an organisation.

Because business people do not have time to learn how to be a Software Engineer, a BPM system need to provide a representation of the process which is meaningful to a business person. It will have the aspects of the system that are important to them, and consequently it may not have some things that a Software Engineer would like. The diagram must not be cluttered by aspects which are “implementation oriented” as opposed to business oriented.

That is reasonable and familiar to anyone who has gone through the tortuous process of delivering software to meet a business need that is crystal clear in the head of the business person at the sharp end but leaves a whole lot of stuff for the developer to make up. Too often, the business requirements whether in a BPMN diagram or good old fashioned English paragraphs are missing simple things like what happens when the process does not work (business-wise rather than software crash).

Once a business person draws a diagram, that is the diagram that is executed. It must not be transformed to a different form for the convenience of the Software Engineer.

This sounds good as well and is often used to rubbish the use of an established form of software solution as a means of delivery for the new method of expressing the business problem ... "Can't use BPEL because it does not look like BPMN" . This misses one good reason for using transformations or compilations of one logical specification into another, progressively until we end up with instructions for the hardware ... at each level of transformation considerable work has been done to ensure that the forms of expressions are formally correct.
In practice, I think it is sufficient for the business person to draw the diagram defining the process and for all the elements of the diagram to be visible in the execution regardless of transform. So if you freeze the action of an execution, the business person would be able to see that there are instances of processes in play; the current state of any instance can be related to the processing state of activities etc.

The history and analytic reports need to match the original diagram to support the business user in evaluating the performance of the organization, not for the programmer to tell how well the program is running.

Exactly!

By attempting to include all the Software Engineering features in with the BPM (business person) features, the result can be something that is not useful for either. You have people today still believing that BPEL is the ultimate way to implement business processes. BPEL only provides a way to send, receive, and transform — these are Software Engineering requirements, not business requirements. A Software Engineer will tell you that with these primitives you can implement anything, probably even a spreadsheet, but that misses the whole point about why we have spreadsheets and BPM in the first place: because they are not Software Engineering.

While BPEL is not the most capable of languages it does have the merit of having a clear specification of how it should execute. In the same way as a spreadsheet program can be seen as a presentation or interaction layer for a particular class of user connected to a backend computational engine which is tried and tested, a business process diagramming tool with a BPMN or similar presentation form can be implemented through connection to a tried and tested executable engine. If that can be BPEL, who cares?

The problem really arises when the software engineer escapes from the cubical and starts expressing the business process requirements in BPEL and then translating that into a subset of BPMN (for example, with no backward flows) because it can be done.

It really is not helpful to end up with the business process definitions being done in the closed world of "IT" because only they could possibly understand the limitations and technicalities of the "system".

Business process designers (who should not be a specialist breed) need to be able to express the activities, flows, events and decisions of a business process in a way that makes sense to them and the proverbial man on the Clapham omnibus . This means that backward flows, arbitrary cycles, synchronising activities across the process all need to be allowed.

If a software specialist following extensive training and principles in the dark arts of software engineering can create executable code from this business process diagram, then it will be a small step to create software that replaces the software engineer in the development process. At present, in the BPMN/BPEL debate the proponents of BPEL as an intermediate step between BPMN and execution seem short of the patterns that demonstrate that everything that can be expressed in BPMN can be executed in BPEL. Similarly, the proponents of directly executable BPMN are short of the formal definition of the execution rules associated with BPMN patterns.

Executable BPMN via BPEL

In a short burst of discussion about whether BPEL was a viable execution mechanism for BPMN, Keith Swenson provided an excellent example of a valid BPMN expression that could not be expressed directly in BPEL with a follow up in his blog showing that the BPMN could be directly executed using his product.

This was followed up by a spirited exchange between factions with assertions that BPEL could indeed express the business process but no definitive BPEL as far as I can tell. Alex Boisvert acknowledged a bug in the BPEL produced by Intalio Designer v5.2 but unfortunately did not describe the code that should have been produced. [[Update:ActiveVOS provides an answer in its tutorial material ]]

However, the BPMN 1.1 specification provides a couple of items for thinking around this problem.

The use of a new signal element to provide synchronisation behaviour across processes covers exactly the intent of Keith's example.

There may be situations within a Process where the flow is affected by or dependent on the activity that occurs in another
Process. These events or conditions can be referred to as milestones. The process model must be able to identify and react
to the milestone.That is, the continuation of a Process may be triggered by Signal Events, which pass the flow between
processes (see Figure 10.48 reproduced below). The type of Workflow Pattern called a Milestone.

In para 10.2.1.11 Avoiding Illegal Models and Unexpected Behavior of the BPMN specification, the authors touch on the root cause of the issue.

The situation where alternative paths cross the implicit boundary of a group of parallel paths can cause an invalid model.

If there is to be a future for BPMN to BPEL, we need an indication of what transformation needs to be done to otherwise valid BPMN to produce the intended executable.

The use of BPMN signal elements suggests that the bright people at Intalio could replace the doubtful synchronisation with whatever mechanism they are going to employ to implement signal intermediate events (not implemented in v5.2!). That is, implement it as though the BPMN looks like this ...

This then gives us a manual work around until the transformation rules can be codified into the product.

Which raises the question ... does the catch event need to be reached before the throw can be effective? Should I draw the BPMN like this? ...

Executable BPMN call for action

In BPMI.org Redux Ismael Ghalimi of Intalio calls for participation in a group to sort out a true executable BPMN. The tentatively titled BPMNLab looks to be a good practical approach to the issues.

• What consitututes executable BPMN?
• Are there useful BPMN flow patterns that cannot be translated into WS-BPEL? If so, how are they best dealt with to produce an executable?

After what has been a bit of sniping from the sidelines on my part, I would be happy to be involved in this effort.

The transformation to BPEL approach seems to me pretty sound. Here is a working language definition appropriate for the BPM domain that is capable of withstanding rigorous assurance testing. Why attempt to define another?

A few BPMN constructs may produce really ugly BPEL code, but who cares except for a few developers in Intalio and the like who have to develop the transformation code. The last thing that anyone in the real world should be doing is fiddling with the generated code. I am reminded of a very successful Unisys product LINC which developed totally unreadable and arguably inefficient COBOL code. In years of use, we had no more reason to look at the generated intermediate COBOL than the final machine level code.

Comply with the specification for BPMN

Bruce Silver in BPMN-BPEL in Perspective responds to the same article as I did here and raises the issue of compliance.

OMG could define a compliance spec for BPMN 2.0 - the elements,
attributes, and flow patterns that MUST be supported to claim BPMN
support. They have made some murky noises about it, but I haven’t seen
anything concrete yet. Since IBM and Oracle are driving the bus there,
it would make sense that that subset would have a defined mapping to
BPEL, but it would allow non-BPEL engines to be compliant as well.

That would be a healthy thing for the industry. But I’m not holding my breath.

I may be more used to specifications for computer languages (like FORTRAN, COBOL) than business process but I suggest that the specification for BPMN 2.0 itself should be what vendors comply with and not a separate compliance statement. Compliance statements from vendors will then effectively state what they have not achieved (yet). Purchasers and consultants can then judge whether the level of compliance of a particular product is sufficient for there purpose.

As BPMN is unlikely to stop at 2.0, a really supportive vendor should not only be compliant with the currently approved specification but also support proposals for future versions, so that practitioners can work with the new ideas while they work through the acceptance process of a standard.

Execution is a different issue. A vendor could opt for a BPEL execution engine (as Intalio uses at present) and reflect that decision by flagging bits of the model as not-executable. An organisation working from the compliant BPMN model could than decide on a method for implementation ...

• use a tool that will generate an executable directly from the model, or
  
• hand the model to an IT developer as a requirement specification

I will be looking for solutions in the first category but we must not lose sight of the fact that a vast number of organisations like to keep a developer shop and consultants busy re-inventing processes in Java or whatever language is the flavour of the month.

Even for Intalio, the execution engine could be replaced by something easier to map to BPMN without changing the user view.

To BPEL or not to BPEL

In Why BPEL is not the holy grail for BPM, Pierre Vigneras thoroughly covers the issues of using BPEL as the implementation layer of a business process solution. From the article conclusions ...

• Therefore, we consider BPMN notation as the only currently viable solution for Business Analysts
• Transformation from BPMN to (readable) BPEL is quite hard to implement, and produces --- when correct --- hardly readable code.
• Therefore, we may wonder why BPMN is transformed to BPEL since there
  exist a graph-based standard that maps directly BPMN constructs ---
  namely XPDL v2.0
• Of course, one may claim that XPDL 2.0 lacks some execution specifications that makes him (sic) unsuitable for direct execution.

Although I am generally a fan of Intalio, Pierre provides a good example of the problems they get when BPMN is used as though it is a convenient graphical expression for BPEL. In particular, a real business analyst can express the process in valid BPMN which is then transformed by Intalio into BPEL which bears little relation to the designer's intention. Worse, the Intalio developers have removed valid BPMN constructs from the business analyst, probably because they do not readily fit with the BPEL implementation. To add insult to injury, there is a general impression that if your valid BPMN does not work, you must have designed your process against some mystic best practice rather than expressed real business events and activities.

I am less concerned about the round-trip issue. If the BPMN is transformed by a product into an execution language which then remains untouched by human developer hand why we should care about the readability of that executable code? The answer is that the maturity of product development in BPM does not give users much faith that the generated code will do what we expect.

If we concentrate on developing BPMN into the way of real business analysts express the business processes and assume an automatic transform into a executable, the choice of implementation paths is only of concern to product developers. Out in the real world we can judge their products by the ease with which we can describe the processes that we find (as-is) and design (to-be).

BPMN Modeler Eclipse Sub-project

Intalio has announced a move to raise the status of the BPMN Modeler component to
a sub-project under the top level project Eclipse SOA Tools
Platform (STP).

The BPMN sub-project will focus on building an usable and extensible
graphical modeler by leveraging the tools provided by the Graphical
Modeling Framework and the EMF Validation Framework.

• Provide a first class user experience when it comes to designing processes, using graphical assistants.
• Provide a graphical notation according to the BPMN standard.
• Provide ways to extend the modeler to integrate it within commercial products.

Great stuff.
Let us avoid the situation where people describing their view of the real world are constrained by a developer's pain of how the real world may have to be transformed into code.
I would really like to see a mandate to allow modelling in line with major proposals in BPMN rather than limited to published specifications. This seems to be a real benefit of extensibility. Perhaps it is a bit late for BPMN 2.0 but it would be good to be able to explore major changes within the normal modelling toolset.