SOA and WS-BPEL - here for review

I was kindly invited by Packt Publishing to review Yuli Vasiliev's new book SOA and WS-BPEL. The review copy just arrived today in time for a quick scan. I'm immediately drawn by the fact that this is one book that clearly goes beyond smarchitectural blurgh and gets down to concrete details. The focus is on showing how you can create solutions with an open source or freely available toolset - specifically PHP and ActiveBPEL. And when it comes the database of course you expect MySQL, but I am very pleased to see that Yuli also gives full coverage of using (freely available) Oracle Database XE. Looks like a few days of fun ahead as I work through the book in more detail. I'll be sure to post my review when I'm done.

Pipes - Web 2.0 Wake-up Call for BPEL?

Recently I've spent some time playing with Yahoo Pipes. I was tipped off by comments to Chris Muir's blog on Drowning in Oracle Blogs Aggregators.

So I've had some fun with Pipes, using it to aggregate all my Oracle news for example. Pipes is still beta and has a fairly modest objective of giving end-users a way to craft their own data feeds on the net, but I think there are some exciting implications.

My Little Experiment

I did a simple experiment with a Perl CGI that issues a "quote of the day" in JSON format, consumed it in Pipes, and then subscribed to an RSS version of the result in FeedReader.

I could have used an existing data source supported by Pipes (RSS, JSON, CSV or XML) but I wanted to experiement with formats. Hence the little bit of coding ..

use strict;
use CGI;
use JSON;

print header();

my $obj = {
  title   => 'Quote of the Day',
  quote   => 'the only dependency we have on the proof of concept is whether it works or not'
};

print objToJson($obj);
1;

(btw, that's one of my favourite quotes from a collegue during a particularly long and weary evening;-)

The Pipes editor makes it a piece of cake to get tricky ... like add a flickr lookup that will roll in a suitable photo to go along with the quote:

All fairly straight-forward, but what grabbed my attention is how Pipes provides an extremely intuitive graphical editor that makes it simple for an end-user to transform and consume data in a hosted model that would conventionally require server-side programming to arrange. It even has an integrated "debugger" that lets you see the output at any stage of the pipe you select.

Pipes - a Web 2.0 Wake-up Call for BPEL?

There are a few things that Pipes can't do (yet), like handle SOAP Web Services, incorporate human workflow or asynchronous events, secure published pipes, and of course reach data sources or systems that are not accessible from the Internet.

But I'm thinking it wouldn't take too much enhancement along these lines and we'd see Pipes rapidly encroaching the BPEL domain (albeit without the standards support). This would be a perfect example of the Web 2.0 challenge to Enterprise IT.

• Pipes - the funky UI that lets anyone create and publish their "flow"


• BPEL - a robust standards-based engine that runs "processes" that have been designed, tested and deployed by experts

If these two extremes don't come crashing together soon, I think we are not trying hard enough!

I'm not sure if Yahoo could do this without a rip'n'replace of the Pipes back-end, and rethink the whole hosting model. More likely that we see someone re-think our approach to BPEL, and the set of use cases it assumes.

I don't hear or see much movement in this direction from the existing BPEL players however. That includes the commercial offerings like Oracle BPEL Process Manager and also the free/open source kind like ActiveBPEL and OpenBPEL.

I suspect we need to challenge a few key assumptions implicit in current BPEL offerings. That processes are (a) largely a conversation between systems, and (b) built by a developer (or at least a skilled business analyst).

While that can and will remain true for core business processes, I think Pipes is beginning to demonstrate that there is a bigger picture we have yet to address.

Put it this way. I'd like to take a robust BPEL engine with...

• BPEL standards support of course


• Web Services (WSDL + SOAP) and WSIF adapter model


• Sophisticated human workflow and asynchronous task support


• Something I can deploy behind the firewall if needed (to access enterprise goodies)


• A decent security model

Then add some of the Pipes goodness:

• Oh-so-Web-2.0 hosted design tools - made for end-users.


• "Publish" instead of "Deploy". Implies automatically maintained (and accessible) registry of published pipes/processes.


• Support for RSS, JSON, CSV and plain XML "web sources"


• Inheritance by chaining or cloning existing pipes


• Smart output rendering. Need RSS, JSON or SOAP? If it makes sense, ask for it and the correct format should be delivered. Transformation taken care of by the infrastructure, not the process definition.

As an enterprise user, this means I'd expect to be able to launch my company's BPELPipes site and see the pipes published by IT. If I need something a bit different, I just go ahead and create it (or clone an existing service). And once I've done creating my pipe, I can use it immediately as maybe an RSS feed or somthing I can embed in my portal. And share it with others if I like..

Perhaps here lies the future of Enterprise 2.0 Workflow (WikiWorkflow?!), finally bringing SOA to the people rather than having it left stuck somewhere in the middle tier.

Diving for SOAP Perls

Antony Reynolds' recent Diving for Perls with WSIF post gave a great example of how you can use HTTP bindings to call perl CGI scripts from Oracle BPEL Process Manager.

If your perl code is not already available to be called in this way, then what to do? Certainly the "ideal" would be make it available as a native Web Service and do away with any special binding. Thanks to the SOAP::Lite module, this is actually quite easy to do.

I'm going to walk through an example of how to take some aribitrary perl code, wrap it as a Web Service, and then call it from a BPEL process. See the diagram:

The Perl Code

In this example, there's really only one bit of code that "matters" ... a helloWorld function. I'm going to start with this wrapped in a perl class module called HelloWorld.pm. As you'll see shortly, wrapping the business functionality in a class is a good idea because it allows automatic dispatching from the Web Services interface.

$ cat HelloWorld.pm
#!/usr/bin/perl -w
use strict;
package HelloWorld;
our (@ISA, @EXPORT, $VERSION);
use Exporter;
$VERSION = 1.00;
@ISA = qw(Exporter);
@EXPORT = qw( helloWorld );

sub helloWorld {
my ($self,$foo) = @_;
return 'Hello ' . $foo;
}
1;

Important to note that while the code here contains some of the module niceties, it doesn't make any reference to SOAP, CGI or BPEL. It's plain perl. We can prove that with a little perl test program:

$ cat helloWorld.pl
#!/usr/bin/perl -w
use strict;
use HelloWorld;
print HelloWorld->helloWorld( 'Sunshine' );

$ perl helloWorld.pl
Hello Sunshine
$

The SOAP Interface

The dynamic typing of perl and flexibility of the SOAP::Lite module really live up to the make simple things easy motto. In three lines of code we have a SOAP CGI server for our HelloWorld class (that's why I made it a class;)

$ cat HelloWorld.cgi
#!/usr/bin/perl -w
use HelloWorld;
use SOAP::Transport::HTTP;
SOAP::Transport::HTTP::CGI
->dispatch_to('HelloWorld')
->handle;

That was so easy, there must be a catch right? Well yes, one comes to mind: the reply message elements will necessarily have some generated names (like "s-gensym3") since there is nothing in our code to provide any guidance for things like the "name" of function return value elements.

Testing SOAP Client-Server

After dropping HelloWorld.cgi and HelloWorld.pm into my apache cgi-bin, I'm ready to test the SOAP service over HTTP. We can whip up a client in no time:

$ cat HelloWorldWSClient.pl
#!/usr/bin/perl –w
use SOAP::Lite;

my $soap = SOAP::Lite
->readable(1)
->uri('urn:HelloWorld')
->proxy('http://localhost:8000/cgi-bin/HelloWorld.cgi');

my $som = $soap->helloWorld(
SOAP::Data->name('name' => 'Sunshine')
);
print "The response from the server was:\n".$som->result."\n";

$ perl HelloWorldWSClient.pl
The response from the server was:
Hello Sunshine
$

If we sniff the network or route this request via a tool like org.apache.axis.utils.tcpmon, we can see the outbound request and incoming reply:

Creating a WSDL file

Alas, perl's flexibility means that automatically generating a WSDL for our SOAP service is easier said than done. Unlike in strongly-typed languages, perl methods can take an arbitrary number of parameters of arbitrary type ... whereas of course a Web Service should have a very clearly defined interface.

I think one of the best approaches at present for generating WSDL in perl is the Pod::WSDL module. I'll perhaps leave that for another blog entry. For now lets just assume we'll manually create a WSDL for our service:

$ cat HelloWorld.wsdl
<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions targetNamespace="http://localhost:8000/HelloWorld" xmlns:impl="http://localhost:8000/HelloWorld" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns1="http://localhost:8000/HelloWorld">

<wsdl:message name="helloWorldRequest">
<wsdl:part name="name" type="xsd:string" />
</wsdl:message>

<wsdl:message name="helloWorldResponse">
<wsdl:part name="s-gensym3" type="xsd:string" />
</wsdl:message>

<wsdl:portType name="HelloWorldHandler">
<wsdl:operation name="helloWorld" parameterOrder="name">
<wsdl:input message="impl:helloWorldRequest" name="helloWorldRequest" />
<wsdl:output message="impl:helloWorldResponse" name="helloWorldResponse" />
</wsdl:operation>

</wsdl:portType>

<wsdl:binding name="HelloWorldSoapBinding" type="impl:HelloWorldHandler">
<wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http" />

<wsdl:operation name="helloWorld">
<wsdlsoap:operation soapAction="" />
<wsdl:input name="helloWorldRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://localhost:8000/HelloWorld" use="encoded" />
</wsdl:input>
<wsdl:output name="helloWorldResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://localhost:8000/HelloWorld" use="encoded" />
</wsdl:output>
</wsdl:operation>

</wsdl:binding>

<wsdl:service name="HelloWorldHandlerService">
<wsdl:port binding="impl:HelloWorldSoapBinding" name="HelloWorld">
<wsdlsoap:address location="http://localhost:8000/cgi-bin/HelloWorld.cgi" />
</wsdl:port>
</wsdl:service>

</wsdl:definitions>

Invocation from a BPEL Process

Now you have all the bits in place to invoke your Perl code as a fully-fledged Web Service from within BPEL. I won't go into this in detail here because it is the standard Web Service invocation process. Just add an "invoke" activity in your process and point it to a partner link defined based on the WSDL generated above.

Once you have deployed your process, you can test it from the BPEL Console. Here's an example of the invoke activity in one of my tests:

Conclusion?

Hopefully I've shown that exposing perl code as a Web Service is actually pretty simple. Once done, the code is then available for use by standards-based tools like Oracle BPEL Process Manager.

There are a couple of consideration to bear in mind though:

1. SOAP::Lite provides some great hooks for automatically generating a SOAP interface, however these come with the caveat that reply message elements will necessarily have some "generated" names
   
2. Automatic WSDL generation is confounded by perl's dynamic typing. Modules like Pod::WSDL provide some good solutions though.