Monday, August 20, 2007

Revisiting 11g Native Web Services

I've just moved from 11.1.0.5 to 11.1.0.6, and updated my post First Tests of 11g Native Web Services in a few areas:
  • The XDB_* role names have been corrected in 11.1.0.6 to match the documentation.
  • Confirmed that the auto-generated WSDL still assumes 'orawsv' as the URL pattern (see below). So for now, it's clear that you should stick with 'orawsv' as the servlet name and URL pattern for your native web services configuration.


Thanks to Marco Gralike and Christopher Burke for the chats we've had in the comments thread. Here's a summary of the links that came up in discussion:

Tuesday, August 14, 2007

+0.9

Launch is over, software's released, and now I've actually posted a real 11g blog entry, so I've decided "Tardate 10.2" deserves to grow up by +0.9. Now to be known as "Tardate 11.1"!

Monday, August 13, 2007

First Tests of 11g Native Web Services

I mentioned in Log Buffer #54 that "Patrick Wolf stumbled across an 11g feature that means DBA's may put Java/SOA guys out of work".

Well I finally got myself setup with an 11g test system and the Native Web Services was the first thing I jumped at testing.

Conclusions first?
  • Although perhaps not a blockbuster new feature, Native Web Services provide an easy solution for exposing data services into a SOAP-oriented application stack.

  • For 11.1.0.5 (only), see Metalink Note 444191.1 for updated installation instructions.

  • The auto-generated XMLSchema and WSDL have some goofy features (like hyphens in method names) that may break SOAP client toolkit smarts.

  • The is no real control over the generated XML payload, which will possibly limit the usefulness of Native Web Services for Enterprise SOA initiatives.

  • The security model is simple but effective. The underlying database user role model provides authentication and access control.

  • The ability to call arbitrary packages/procedures/functions is perhaps the most powerful feature. No incremental coding or configuration is required for each method. A good service interface is thus just a matter of careful procedure design and construction.

  • I am less convinced that the ability to execute arbitrary SQL via Native Web Services is a good thing. It's not SOA, and it's not a good replacement for JDBC, ODP etc. Seems like just an invitation for bad hacks and shortcuts..


So, after a couple of hours of playing, I think maybe the Java/SOA guys don't have to panic just yet.. ;)

Here's the blow-by-blow, where I describe the setup gotchas I hit, and my simple SOAP::Lite testing with Perl ..

Initial Scan..

It took me a few moments to orient myself in the documentation; rather than being a distinct major new feature, Native Web Services are an enhancement of XML DB, and so the requisite documentation is mainly found in the XML DB Developer's Guide.

The architecture of Native Web Services is quite simple. The main moving part is a servlet called 'orawsv' which brokers SOAP 1.1 requests and handles automatic WSDL generation. Out of the box, it is not enabled.

The feature works in two main ways:
  1. An arbitrary SQL DML request can be sent to the main 'oawsv' endpoint, and the results are returned as XML in the SOAP response.

  2. Procedures and functions may be invoked directly over SOAP.

First Up - Configuration

Configuring Native Web Services is simply a matter of enabling the orawsv servlet, and then granting user access through role assignment.

This is covered in Chapter 33 of the docs.

I first tried this with 11.1.0.5, and found that the role names mentionedd in the docs do not match (XDB_WEBSERVICES instead of XDBWEBSERVICES and so on). For an 11.1.0.5 install, refer to Metalink Note 444191.1 for corrected installation instructions. In 11.1.0.6, role names correctly match the docs.

The second thing I discovered (actually, a problem I created for myself then had to fix!) is that the servlet name does matter! The SERVLET_NAME (as in the setup code below) must be 'orawsv'.

Thirdly, although you can set the URL pattern to anything you wish (and calls work), the auto-generated WSDL always assumes 'orawsv' when generating endpoint addresses. Effectively this means you must use 'orawsv' as the URL pattern also.
NB: thanks to Christopher Burke for alerting me to the WSDL situation.
DECLARE
SERVLET_NAME VARCHAR2(32) := 'orawsv';
BEGIN
DBMS_XDB.deleteServletMapping(SERVLET_NAME);
DBMS_XDB.deleteServlet(SERVLET_NAME);
DBMS_XDB.addServlet(NAME => SERVLET_NAME,
LANGUAGE => 'C',
DISPNAME => 'Oracle Query Web Service',
DESCRIPT => 'Servlet for issuing queries as a Web Service',
SCHEMA => 'XDB');
DBMS_XDB.addServletSecRole(SERVNAME => SERVLET_NAME,
ROLENAME => 'XDB_WEBSERVICES',
ROLELINK => 'XDB_WEBSERVICES');
DBMS_XDB.addServletMapping(PATTERN => '/orawsv/*',
NAME => SERVLET_NAME);
END;
/
GRANT XDB_WEBSERVICES TO SCOTT;
GRANT XDB_WEBSERVICES_OVER_HTTP TO SCOTT;
GRANT XDB_WEBSERVICES_WITH_PUBLIC TO SCOTT;

I couldn't quite figure out exactly why the 'orawsv' SERVLET_NAME is significant. It apparently just is, providing an explicit reference to the required servlet. The DBMS_XDB.addServlet docs are unfortunately terse and unenlightening.

So that all took me an hour more than it should. But not to worry - I am now up and running!

Now What Can It Do? Testing a Function Call by Web Services..

I was particularly interested to see a procedural interface exposed via Web Services, so my first little test was to call a database function from Perl (using SOAP::Lite). I created this very simple function in SCOTT:
CREATE OR REPLACE FUNCTION empcount
RETURN NUMBER IS
emp_count number;
BEGIN
SELECT count(*) INTO emp_count FROM emp;
RETURN emp_count;
END;
/

This becomes available at the endpoint 'http://server:port/orawsv/SCOTT/EMPCOUNT', and the auto-generated WSDL is available at 'http://server:port/orawsv/SCOTT/EMPCOUNT?wsdl'.

Next I knocked up a bare-bones Perl client. Note the redefined get_basic_credentials method to provide the basic authentication credentials that are required.
#!/usr/bin/perl -w
#
use SOAP::Lite +trace => 'debug';
sub SOAP::Transport::HTTP::Client::get_basic_credentials {
return 'scott' => 'tiger';
}
my $serviceNs = 'http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT';
my $soap = SOAP::Lite
->proxy('http://localhost:8080/orawsv/SCOTT/EMPCOUNT');
my $som = $soap->call( SOAP::Data->name('SNUMBER-EMPCOUNTInput')->attr({'xmlns' => $serviceNs}) );
print "The response from the server was:" . $som->result . "\n";

And the answer is .. 14;) [see the end notes below for the request/reply transcript].

As you can see, pretty simple, but it should be simpler. One liners should be possible, like this:
print SOAP::Lite-> service('http://scott:tiger@localhost:8080/orawsv/SCOTT/EMPCOUNT?wsdl')->EMPCOUNT();

... but you'll notice that the 11g generated method names (like 'SNUMBER-EMPCOUNTInput') have annoying hyphens, which subverts the smart typing and dynamic binding that kits like SOAP::Lite are capable of. Doh!

Closest I can get is this:
print "The response from the server was: ";
print SOAP::Lite
->uri('http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT')
->proxy('http://localhost:8080/orawsv/SCOTT/EMPCOUNT')
->call ( 'SNUMBER-EMPCOUNTInput' )
->result;


Endnotes - EMPCOUNT SOAP Request and Response
{The request ... }
SOAP::Transport::HTTP::Client::send_receive: POST http://localhost:8080/orawsv/SCOTT/EMPCOUNT HTTP/1.1
Accept: text/xml
Accept: multipart/*
Accept: application/soap
Content-Length: 461
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT#SNUMBER-EMPCOUNTInput"

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<SNUMBER-EMPCOUNTInput xmlns="http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT" xsi:nil="true" />
</soap:Body>
</soap:Envelope>

{The response ... }
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Server: Oracle XML DB/Oracle Database
Content-Type: text/xml; charset=UTF-8
Client-Date: Mon, 13 Aug 2007 16:06:28 GMT
Client-Peer: 127.0.0.1:8080
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
DAV: 1,2,<http://www.oracle.com/xdb/webdav/props>
MS-Author-Via: DAV

<?xml version="1.0" ?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<EMPCOUNTOutput xmlns="http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT">
<RETURN>14</RETURN>
</EMPCOUNTOutput>
</soap:Body>
</soap:Envelope>

Saturday, August 11, 2007

The BI/EPM Revolution (..needed, please!)

Frank Buytendijk makes a good case for why generic BI/EPM frameworks are most probably pretty useless in practice.

What puzzles me deeply is that so much of the focus of BI remains concerned with the technology infrastructure and dashboard bells and whistles. This is seen most clearly when a new enterprise application or business process is being introduced, and you hear the almost dismissive assertion made that "it will integrate with you DWH", or that it will "plug in to your management dashboard". And "we'll scope for, let's say, 20 reports to be built".

I think we are all missing the point.

I'm going to suggest that on the whole, business is still struggling with two symptoms of IT growing pains ...
  • The IT profession has lost sight of the fact that reports, dashbords and the like are not end products in their own right. They typically have no intrinsic value. The value lies purely in the decisions and actions that may be taken as a consequence (the exception perhaps being for reports that are required by an external party such as shareholders, government or regulatory authority).

  • Equally, I think business management is still adapting to a world where information is plentiful. Even when fact-based decision making is practiced, it is all too common to find it be based on spreadsheets (which may or may not have had their original source data pulled from an enterprise system).

Every time I hear the ERP vendors boasting about enterprise adoption, I cynically imagine a Microsoft advertisement that could (probably truthfully) claim
"99.9% of Fortune 500 companies run their business on Excel!!"

I think its easy to understand how we got to this position. I remember working for a reinforcing mesh ("fabric") manufacturing company in the late 80's. We had factories filled with fabric machines like the one in the picture. They weren't networked or anything. The only information that the production manager had to work with were the production sheets filled out by the operator each shift, and the maintenance manager had to send his technicians around to check out the machines periodically.

We were working in an environment where information was scarce, yet the business imperitives were very much the same as today: production manager was concerned with productivity and profitability; and the maintenance manager cared about the maintenance costs and machine performance (netting to the "TCO" of the plant). We managed through a combination of MWA (Management by Walking Around, aka gemba / genchi genbutsu), MGF (Management by Gut Feel! .. some would say experience;-), and also selected special projects to capture data related to problem hypotheses [what I was most involved in].

But the world has turned in the last 20 years. Today, most organisations have more information sitting in various databases and enterprise systems than they know what to do with. I believe the challenge these days is firstly to make that information accessible, and then - most critically - to ensure that our management practices can make effective use of that information.


We actually have great technology these days for cracking that first challenge - making information accessible and actionable. I like to think in terms of three tiers (management, operational, infrastructure), as in the diagram.

Unfortunately, that's often as far as our thinking (and implementations) go.

However as Frank points out in his post, we need to address the strategic framework for BI. How BI impacts, changes or even transforms the business.

For me, the implications are clear for all decision-makers (which may or may not just mean "management")...
  • In the past we have excelled at managing in an information-poor environment. Now we need to go back to school, unlearn some of our assumptions and practices, and start exploiting the information that (should be) at our fingertips.

  • This means a renewed relevance of management science, particularly quality management (e.g. 6 Sigma) and key concepts such as kaizen/continuous improvement.

  • In true "Flat Earth" style - if you don't, someone else will and then you are history, mate.

And for IT, I think a few things to consider...
  • First, get real and get connected to your business. You are not delivering 20 reports, you are delivering information that will hopefully help your business users may really good decisions, impact the bottom line and keep your job safe.

  • Avoid the temptation to dumb-down and de-scope BI and reporting. It could be the most business-critical aspect of the whole project ... Promote monitoring and management to first-order use cases.

  • Consider all tiers of monitoring and management: from corporate performance, to operational management, to infrastructure.

So maybe this is an aspect of what Enterprise 2.0 is really all about?

Monday, August 06, 2007

Take the 2007 Perl Survey

Take part in the 2007 Perl Survey! The Perl Survey is an attempt to capture a picture of the Perl community in all its diversity. No matter what sort of Perl programmer you are, they'd love to hear from you.

The survey can be found at: http://perlsurvey.org/

It only takes about 5 minutes to complete, and will be open until September 30th, 2007.

My Reading Feed

Just a quick note to highlight a relatively new enhancement to blogspot .. the ability to get an RSS feed for a given label. I stumbled across the instructions while trying to figure out how to get a feed of all the book reviews I decided to put up at my Prata Life blog. The basic URL formulation is like this:

http://blogname.blogspot.com/feeds/posts/default/-/labelname
e.g. my reading feed is:
http://pratalife.blogspot.com/feeds/posts/default/-/Read
So now I have a little side-panel on the tardate blog showing the most recent books I've read, using the RSS widget. Cool!

Top of the list right now is The Ambler Warning by Robert Ludlum, which had some interesting, coincidental cross-overs to other books I'd read recently (blink, Softwar). Includes an interesting cameo(?) as the book reaches a climax, action centers on the World Economic Forum meeting at Davos..

"A couple of yards away from him, an older, rangy American billionaire - someone whose 'enterprise software' was an industry standard across the globe..." Hmmm, ring any bells? If you don't think Bill makes 'enterprise software', then I reckon it can only be one person ...

Sunday, August 05, 2007

OTN in China and the importance of aggregators

Justin reported the stunning success of the OTN lounge at OpenWorld Shanghai 2007. Although I never made it to the lounge, the buzz in the halls and corridors was great testament to the enthusiasm of the Oracle community in this part of the world.

My trip up to Shanghai also provided a pointed reminder of China's on-again-off-again dance with blogspot (currently "off-again") when I tried to update my blog.

Now I'm not going to argue that China needs to cease its attempts to censor the net since I respect it's authority to exercise such control, just as we do in the west but with different values. However I must admit I had never thought of the collateral damage inflicted on the Oracle community of users in China, since it seems quite a large number of bloggers on Oracle topics are using one of the blocked platforms.

Fortunately it seems that the various Oracle-topic aggregators have not been hit by any of the blocks (and presumably won't be because of the generally non-political content).

So I'd encourage the aggregators to note the importance of aggregating full-text feeds with attachments ... it may be the only chance that some people may have of reading the content (also works better for anyone with an offline reader anyway).

For Oracle, I'd suggest this is another reason why it would be great to open up to hosting non-employee blogs.

Friday, August 03, 2007

Innovation on show at OpenWorld Shanghai

I've just spent the past few days at Oracle OpenWorld Asia Pacific in Shanghai. Although of course a smaller scale to the one coming up in San Francisco in November, I think there were 8000 or so attendees and in true OOW style, each session slot presented the challenge "which of these 10 concurrent sessions do I want to attend?!"

I also got to present a session myself, which was great fun (thanks to those who filled up my session on the last afternoon of the last day!) More of that in future posts maybe ...

Innovation was one of the key themes of the conference and Charles Phillips' keynote. And I think there was no better illustration of this than the demo that the OARDC team were showing in the demogrounds. It got my "best in show" vote!

Their Second Life "Innovation+ Village" show-cased a solution concept bridging the real and the virtual worlds, and they brought some interesting ideas into play.

The scenario was a Second Life wine store. In the room you can examine the products and get some gratuitous live-video. But then you can make purchases (using Linden$), which are captured in iStore for delivery in "real life" (I hope .. still waiting for my case of Bollinger;-) BI dashboards track order volumes and status. Then back in Second Life, you provide feedback on the wine purchased. The feedback is not only collated in Oracle Database for analysis (more dashboards), but also fed into a blog of comments on the particular wine (WebCenter).

Overall, a great example of an "Enterprise Mashup". I don't think the team created any new technology pieces, just a novel solution using a combination of available components and APIs, and provided the content.

While you may doubt the financial viability of doing business in Second Life, it does emphasise the fact that creating Enterprise 2.0 value for your organisation does not necessarily involve a huge development investment. Just as a great artist can produce a masterpiece with a child's paintbox, real solutions are possible today if you have the imagination and creativity to combine the youthful "Web 2.0" component palette with your existing IT infrastructure.

Congrats to the crew from OARDC that put the solution concept together - Lennard Low, Teo Kian Hui, Wang Rong Rong, Zhang Rong (I believe this is a group photo below) and thanks Rong Rong for the demo;) Hope you get to show this in San Francisco too ... make sure you get a more prominent space in the DemoGrounds though, and don't have the screen facing away from the entrance!