The BI/EPM Revolution (..needed, please!)

Frank Buytendijk makes a good case for why generic BI/EPM frameworks are most probably pretty useless in practice.

What puzzles me deeply is that so much of the focus of BI remains concerned with the technology infrastructure and dashboard bells and whistles. This is seen most clearly when a new enterprise application or business process is being introduced, and you hear the almost dismissive assertion made that "it will integrate with you DWH", or that it will "plug in to your management dashboard". And "we'll scope for, let's say, 20 reports to be built".

I think we are all missing the point.

I'm going to suggest that on the whole, business is still struggling with two symptoms of IT growing pains ...

• The IT profession has lost sight of the fact that reports, dashbords and the like are not end products in their own right. They typically have no intrinsic value. The value lies purely in the decisions and actions that may be taken as a consequence (the exception perhaps being for reports that are required by an external party such as shareholders, government or regulatory authority).


• Equally, I think business management is still adapting to a world where information is plentiful. Even when fact-based decision making is practiced, it is all too common to find it be based on spreadsheets (which may or may not have had their original source data pulled from an enterprise system).

Every time I hear the ERP vendors boasting about enterprise adoption, I cynically imagine a Microsoft advertisement that could (probably truthfully) claim

"99.9% of Fortune 500 companies run their business on Excel!!"

I think its easy to understand how we got to this position. I remember working for a reinforcing mesh ("fabric") manufacturing company in the late 80's. We had factories filled with fabric machines like the one in the picture. They weren't networked or anything. The only information that the production manager had to work with were the production sheets filled out by the operator each shift, and the maintenance manager had to send his technicians around to check out the machines periodically.

We were working in an environment where information was scarce, yet the business imperitives were very much the same as today: production manager was concerned with productivity and profitability; and the maintenance manager cared about the maintenance costs and machine performance (netting to the "TCO" of the plant). We managed through a combination of MWA (Management by Walking Around, aka gemba / genchi genbutsu), MGF (Management by Gut Feel! .. some would say experience;-), and also selected special projects to capture data related to problem hypotheses [what I was most involved in].

But the world has turned in the last 20 years. Today, most organisations have more information sitting in various databases and enterprise systems than they know what to do with. I believe the challenge these days is firstly to make that information accessible, and then - most critically - to ensure that our management practices can make effective use of that information.


We actually have great technology these days for cracking that first challenge - making information accessible and actionable. I like to think in terms of three tiers (management, operational, infrastructure), as in the diagram.

Unfortunately, that's often as far as our thinking (and implementations) go.

However as Frank points out in his post, we need to address the strategic framework for BI. How BI impacts, changes or even transforms the business.

For me, the implications are clear for all decision-makers (which may or may not just mean "management")...

• In the past we have excelled at managing in an information-poor environment. Now we need to go back to school, unlearn some of our assumptions and practices, and start exploiting the information that (should be) at our fingertips.


• This means a renewed relevance of management science, particularly quality management (e.g. 6 Sigma) and key concepts such as kaizen/continuous improvement.


• In true "Flat Earth" style - if you don't, someone else will and then you are history, mate.

And for IT, I think a few things to consider...

• First, get real and get connected to your business. You are not delivering 20 reports, you are delivering information that will hopefully help your business users may really good decisions, impact the bottom line and keep your job safe.


• Avoid the temptation to dumb-down and de-scope BI and reporting. It could be the most business-critical aspect of the whole project ... Promote monitoring and management to first-order use cases.


• Consider all tiers of monitoring and management: from corporate performance, to operational management, to infrastructure.

So maybe this is an aspect of what Enterprise 2.0 is really all about?

Take the 2007 Perl Survey

Take part in the 2007 Perl Survey! The Perl Survey is an attempt to capture a picture of the Perl community in all its diversity. No matter what sort of Perl programmer you are, they'd love to hear from you.

The survey can be found at: http://perlsurvey.org/

It only takes about 5 minutes to complete, and will be open until September 30th, 2007.

My Reading Feed

Just a quick note to highlight a relatively new enhancement to blogspot .. the ability to get an RSS feed for a given label. I stumbled across the instructions while trying to figure out how to get a feed of all the book reviews I decided to put up at my Prata Life blog. The basic URL formulation is like this:

http://blogname.blogspot.com/feeds/posts/default/-/labelname
e.g. my reading feed is:
http://pratalife.blogspot.com/feeds/posts/default/-/Read

So now I have a little side-panel on the tardate blog showing the most recent books I've read, using the RSS widget. Cool!

Top of the list right now is The Ambler Warning by Robert Ludlum, which had some interesting, coincidental cross-overs to other books I'd read recently (blink, Softwar). Includes an interesting cameo(?) as the book reaches a climax, action centers on the World Economic Forum meeting at Davos..

"A couple of yards away from him, an older, rangy American billionaire - someone whose 'enterprise software' was an industry standard across the globe..." Hmmm, ring any bells? If you don't think Bill makes 'enterprise software', then I reckon it can only be one person ...

OTN in China and the importance of aggregators

Justin reported the stunning success of the OTN lounge at OpenWorld Shanghai 2007. Although I never made it to the lounge, the buzz in the halls and corridors was great testament to the enthusiasm of the Oracle community in this part of the world.

My trip up to Shanghai also provided a pointed reminder of China's on-again-off-again dance with blogspot (currently "off-again") when I tried to update my blog.

Now I'm not going to argue that China needs to cease its attempts to censor the net since I respect it's authority to exercise such control, just as we do in the west but with different values. However I must admit I had never thought of the collateral damage inflicted on the Oracle community of users in China, since it seems quite a large number of bloggers on Oracle topics are using one of the blocked platforms.

Fortunately it seems that the various Oracle-topic aggregators have not been hit by any of the blocks (and presumably won't be because of the generally non-political content).

So I'd encourage the aggregators to note the importance of aggregating full-text feeds with attachments ... it may be the only chance that some people may have of reading the content (also works better for anyone with an offline reader anyway).

For Oracle, I'd suggest this is another reason why it would be great to open up to hosting non-employee blogs.

Innovation on show at OpenWorld Shanghai

I've just spent the past few days at Oracle OpenWorld Asia Pacific in Shanghai. Although of course a smaller scale to the one coming up in San Francisco in November, I think there were 8000 or so attendees and in true OOW style, each session slot presented the challenge "which of these 10 concurrent sessions do I want to attend?!"

I also got to present a session myself, which was great fun (thanks to those who filled up my session on the last afternoon of the last day!) More of that in future posts maybe ...

Innovation was one of the key themes of the conference and Charles Phillips' keynote. And I think there was no better illustration of this than the demo that the OARDC team were showing in the demogrounds. It got my "best in show" vote!

Their Second Life "Innovation+ Village" show-cased a solution concept bridging the real and the virtual worlds, and they brought some interesting ideas into play.

The scenario was a Second Life wine store. In the room you can examine the products and get some gratuitous live-video. But then you can make purchases (using Linden$), which are captured in iStore for delivery in "real life" (I hope .. still waiting for my case of Bollinger;-) BI dashboards track order volumes and status. Then back in Second Life, you provide feedback on the wine purchased. The feedback is not only collated in Oracle Database for analysis (more dashboards), but also fed into a blog of comments on the particular wine (WebCenter).

Overall, a great example of an "Enterprise Mashup". I don't think the team created any new technology pieces, just a novel solution using a combination of available components and APIs, and provided the content.

While you may doubt the financial viability of doing business in Second Life, it does emphasise the fact that creating Enterprise 2.0 value for your organisation does not necessarily involve a huge development investment. Just as a great artist can produce a masterpiece with a child's paintbox, real solutions are possible today if you have the imagination and creativity to combine the youthful "Web 2.0" component palette with your existing IT infrastructure.

Congrats to the crew from OARDC that put the solution concept together - Lennard Low, Teo Kian Hui, Wang Rong Rong, Zhang Rong (I believe this is a group photo below) and thanks Rong Rong for the demo;) Hope you get to show this in San Francisco too ... make sure you get a more prominent space in the DemoGrounds though, and don't have the screen facing away from the entrance!

Playing with CAPTCHAs

Security Now! #101 with Steve Gibson and Leo Laporte covered the deceptively simple challenge of differentiating human from non-human automated clients (great podcast as always .. check it out). Commonly this is done with distorted text like the sample on the right.

The Official CAPTCHA Site has a wealth of information about this field, including discussion of the relay attack that has the potential to defeat any "are you a human?" test (because it enrolls unwitting human accomplices to do the work).

The coolest piece of work though is the reCAPTCHA project. This is a project of the School of Computer Science at Carnegie Mellon University, and it provides a public service for plugging-in a CAPTCHA to your site. But unlike other systems that are just wasting 10 seconds of your time, this system is actually digitizing public domain archives at the same time (getting you to fix the translation of words that have defied the best OCR software)!

Perl is one language that you can easily use reCAPTCHA with, using Andy Armstrong's Captcha::reCAPTCHA module.

After installing the module, it just takes a few minutes to register your site and setup a test page. I built a quick reCAPTCHA cgi in perl .. you can try it out here (sorry, currently may find it offline but get the script source here: myReCaptcha.pl. I think I spent 10 minutes setting this up, and then an hour playing with it;) All in the name of digitizing historical works...

But I'm afraid the humour archive has the best captcha ever ..

Fellow travellers on the road to Fusion..

At one point, the cynical would have said that Fusion is just a con to placate all the PeopleSoft, Siebel and JD Edwards customers. Others just complained about the con-fusion.

Of course, Fusion Applications are still a way off in the future, and I'm not sure if even Oracle Development really know in full cinematic detail what shipping the products will really entail. Its all still fairly big-picture stuff .. although in classic Oracle style, the user/user experience seems to have got lost along the way. But there's enough pressure and creativity that I think we will see the current Fusion formula grow an additional term at some point. i.e.

Fusion = Grid Computing + SOA + Enterprise Information Architecture
will become
Fusion = Grid Computing + SOA + Enterprise Information Architecture + Web 2.0

In other words, potentially the best darn definition of "Enterprise 2.0" in the market so far!

In the meantime however, some things are coming into clear focus. One is the critical importance of SOA and Security Fusion Middleware components (a.k.a. Oracle Application Server). This struck home for me when listening to the recent AppCast interview with Cliff Godwin who is now heading Oracle's Fusion Upgrade Program Office. What is the world coming to.. Oracle Applications folks selling technology?!!

The true test of fusion will be the extent to which it is embraced by customers and the user community. It was great to see Floyd Teter kick-off a new series on his blog where he will be covering his company's Detailed Roadmap to Fusion Applications. Hats off to Floyd for the courage to do this in the open, and to provide the necessary detail that will actually make this useful for others considering the same path.

Log Buffer #54: a Carnival of the Vanities for DBAs

This week its my pleasure to host Log Buffer #54, the Carnival of the Vanities for DBAs. Thanks to Dave Edwards at Pythian for the invite to put this together. Dave must be a very chilled sorta guy to be able to survive each Friday, not knowing what on earth his guest editor is going to churn out!!

Of course 11g continues to be a big topic this week.

Dan Norris adds his views on the best 11g new features to the flurry that came out last week. Christian Bilien's having a party to celebrate ASM fast mirror resync. Syed Jaffar Hussain discovers the alert log will now be XML. Personally I shudder; still not convinced that XML is appropriate for log files.

It strikes me that in this period between launch and release, there's more eyeballs on docs that at any other time in the product lifecycle;)

NB: There still seems to be some confusion whether we are meant to know when 11g will arrive or not. Steven Chan's older post almost convinced me there's a good reason why we don't hear dates;)

In the SQL Server world, Kalen Delaney blogs in wonderment having discovered the dialog box that time forgot. And Bob Beauchemin has some good SQL Server book reviews.

While on book reviews, Stewart Smith is impressed by O'Reilly's Backup & Recovery because it covers just about every way to back up and recover systems (I trust not this way).

So did PostgreSQL trounce Oracle or not? Kevin Closson exposed the lazy reporting that seems to have sparked this "non-story". But you know how pointless it is to try and correct the facts once they are out on the net, right? Its a pity for the controversy, because as Jay Pipes writes, the benchmark achievement is no mean feat for the PG dev team.

Lots of interseting stuff on mysql this week. Mats Kindahl takes mysql proxy for a decent test drive, concludes it has great potential and offers some constructive comments on gaps to be filled. The MySQL 5 HA with DRDB and Heartbeat guide is very well put together by Mark Schoonover (who blurs the line between blog post and technical reference guide!)

What do I like most about open source? Take something like Jay Pipes internals of MyISAM Concurrent insert (part 1 posted this week). Great example of hard core geek writings. Can you ever imagine Oracle, IBM or Microsoft getting down and dirty on internals like this? I love it!

Moving on to deeper thoughts...

K. Brian Kelley mulls over what it means to be a dba, but might be surprised that Patrick Wolf stumbled across an 11g feature that means DBA's may put Java/SOA guys out of work.

Doug Burns ponders just how many blogs a person should have, and in the process gave quite a few pause for thought.

Over in the Oracle Forums, activity seems to be at a high but one wonders about the signal to noise. I think Sidhu was remarkably restrained when I think my response may just have been jfgi!

... and that's pretty much a wrap for this week. Thanks for reading my first attempt at editing the Log Buffer. It just remains for me to leave you with my lame attempt at a bit of 11g humour. Keep on blogging!

Check LOCAL_LISTENER if you run RAC!

Had a case recently with a 10gR2 RAC install. Everything seemed to have been setup to spec, but we were seeing clients occasionally getting ORA-12545 errors and failing to connect, and things getting even worse during failover testing.

After investigating and solving this, it was painfuly obvious how easy the configuration issue can sneak into a RAC install, which prompts me to blog about it now. Bottom line: if you are installing RAC or are responsible for managing a RAC database, I strongly suggest you swivel over right now and 'show parameter LISTENER'!

So back to our case.... looking at a sqlnet client trace, it was apparent that the client was being redirected to the server hostname, not one of the RAC virtual addresses. Two problems:

1. the client couldn't resolve the server hostname since it wasn't in DNS or the client hosts file, and
   
2. the client shouldn't be connecting to the server hostname anyway!
   

The client tnsnames.ora and the servers' tnsnames.ora and listener.ora files had all been checked and were setup with only references the the RAC virtual addresses, so where was the reference to the 'physical' hostname coming from? Well the answer is the LOCAL_LISTENER server parameter.

You would think however that if your client connection descriptor (taken from tnsnames.ora for example) only referenced virtual addresses, you would be safe, right? Not the case, and having a solid understanding of how the listener works is critical to knowing why.

The problem stems from the way that in 10g instances automatically register with the listener, and it is very easy to fall into this trap if you haven't paid very close attention to section "9 Understanding the Oracle Real Application Clusters Installed Configuration" in the platform-specific cluster installation guides.

If you have a DEDICATED server config, then the LOCAL_LISTENER parameter is used for the instance registration with the listener. If you are using a default listener on port of 1521, then DBCA will not automatically set the LOCAL_LISTENER. Section "9.8 Configuring the Listener File (listener.ora)" describes how to manually set a correct LOCAL_LISTENER value, but if you haven't done that, it will default to a connection string that refers to the physical host address (not virtual address).

But you might think "connecting to the physcial host address can't be too bad, can it?". Well yes, there are two problems you can see:

1. Clients may not be able to resolve the host address if you don't have that in DNS, and more importantly
   
2. In a failover situation, clients will not follow the virtual IP.
   

Even then, you might think this would be a very rare problem, because client's tnsnames or other naming is always telling them to connect to the virtual address anyway.

Again, not so. It can be very common for the client to get a connection to the physical host address even if the tnsnames tells them to connect to the virtual address, because of RAC workload management and listener redirects.

Lets take an example of a RAC service called SVC with two instances SVC1 and SVC2 running on host1 and host2 (with virtual addresses host1_vip and host2_vip). The client tnsnames would look something like this:

SVC = 
  (DESCRIPTION = 
    (ADDRESS = (PROTOCOL = TCP)(HOST = host1_vip)(PORT = 1521)) 
    (ADDRESS = (PROTOCOL = TCP)(HOST = host2_vip)(PORT = 1521)) 
    (LOAD_BALANCE = yes) 
    (CONNECT_DATA = 
      (SERVER = DEDICATED) 
      (SERVICE_NAME = SVC)
      (FAILOVER_MODE=(TYPE=select)(METHOD=basic)(RETRIES=10)(DELAY=1)) 
    ) 
  )

The diagram shows how a connection is handled through the listener in the case where the LOCAL_LISTENER is not set correctly. The flows go like this:

1. The client selects a virtual host from the address list and attempts a connection, in this case to the listener on host1_vip.
   
2. The listener selects a preferred instance to handle the session for the service SVC. If the local SVC1 instance is down, or if it thinks instance SVC2 is better able to service the request it sends a listener redirect to the client. This redirect will be to the physical hostname (host2)
   
3. If the client is not able to resolve "host2", you will see an ORA-12545 at this point. If it can resolve the address, then the client establishes a connection to the listener on the host2 address. If SVC2 is running, you should now have a good connection to the database. However, consider now what happens if host2 fails. CRS will ensure that the host2_vip will shift over to host1, but the client is connected to host2 address and you will get into a TCP timout situation. Maybe your client will eventually detect the dead connection and attempt to reconnect (using VIPs), but at best the user application will have stuttered for a significant period of time (depending on your tcp and sqlnet settings).
   

Now consider what should happen, with a correctly configured LOCAL_LISTENER.

1. The client selects a virtual host from the address list and attempts a connection, in this case to the listener on host1_vip.
   
2. The listener selects a preferred instance to handle the session for the service SVC. If the local SVC1 instance is down, or if it thinks instance SVC2 is better able to service the request it sends a listener redirect to the client. This redirect will be to the virtual hostname (host2_vip)
   
3. We get a good connection established to SCV2 on host2_vip. Again, consider now what happens if host2 fails.
   
4. CRS will ensure that the host2_vip will shift over to host1, and the client connection follows this VIP. There will be a slight interruption to communication (depending on how quickly the VIP take-over occurs, and also other factors such if you are connecting through a NAT router, the NATting tables will need to update). Depending on how you have configured FAILOVER_MODE (session, select), you should find that very soon your database connection is alive and you can continue working.
   

Fortuntely there is an easy fix: update the LOCAL_LISTENER parameter to reference the virtual address.

Oracle have a couple of notes on the issue (342419.1, 333159.1) and how to setup LOCAL_LISTENER. Note however, a the time of writing this (and I'm trying to get it fixed), note 342419.1 does not exactly describe the fix correctly.

It mentions to set the LOCAL_LISTENER using the command like this (where server tnsnames.ora has an instance-connect string using VIP address called 'LISTENER_LXDB0036' ):
Alter system set LOCAL_LISTENER= 'LISTENER_LXDB0036' scope=both;

However, this sets a server parameter that would be picked up on all instances. So there are actually two choices:

1. change the tnsnames.ora file on each server to have a different, instance-specific definition of 'LISTENER_LXDB0036' , or
   
2. I think the preferred way: set a SID-specific parameter by adding the SID parameter to the alter system command (assuming the instance name is SVC1 in this case):
   Alter system set LOCAL_LISTENER= 'LISTENER_LXDB0036' scope=both SID='SVC1';
   

Note that the listener registration is is only a problem if the database was created with a lister on default port. Haven't seen it myself, but apparently if you use DBCA to create a database with a non-default listener, then the LOCAL_LISTENER entry is explicitly set with correct reference to VIP.

So the message for today. If you are installing RAC or are responsible for managing a RAC database, I strongly suggest you swivel over right now and 'show parameter LISTENER'. Make sure you don't have an HA problem lurking in your system just waiting to bite you and the very worst moment!

Postscript: I submitted a request for change, and note 342419.1 has now been updated to reflect a more correct solution as of 25-Jun-2007.

Postscript 2: This issue is apparently addressed in 11g. Thanks for the report on this karlarao.
Postscript 2b: As Mark Twain would have said, "The reports of a solution are greatly exaggerated" - apparently 11gR2 still does not set a FQDN by default. Time to crack out a test...

iX2007 day 2 pm - SOA track

Ended up catching the SOA track which was moderated and introduced by Ng Beng Lim from NCS. Oracle, Microsoft and BEA all had a failrly standard pitch with a heavy emphasis on governance (although slightly different ways of modelling the overall journey).


Jurgen Coppens (Accenture) did a refreshingly different approach with a focus on overall transformation (to be expected;) but in particular value realization. Presented some good models, like my sketch to the right, and also advocated a pragmatic approach of focusing on the quick wins in the early stage.

One impression I got from all the speakers however is that they all called for a rational architecture and governance process that is strongly tied to business strategy. In fact this is a fundamental assumption. I have this impression however that for many companies are not really ready to address the question of whether to adopt SOA or not, since they are still have not established the foundation architecture and governance competency. Baby steps.

Anyway that's a wrap for the conference. As I mentioned in my first post, this was the first that had caught my eye for some time, and it lived up to the promise.
ix2007

iX2007 day 2 am - some nice provocative points of view

Morning sessions for day 2 of iX2007 have wrapped. Some highlights...

Douglas Merrill, Google, delivered a provocative point of view on the implications on Enterprise IT. Talked of the changing expectations of users and what they will demand and use in the enterprise. Yes its Web 2.0, but also more broadly the impact of an increasingly IT-literate workforce.
For IT, it means embracing new ways of conceiving enterprise solutions. And if SMEs are running their own infrastructure and applications, they are probably wasting their money; they will be exploiting Software as a Service.

Bit disappointed in David Willis' (Gartner) presentation. Did a good job of summarising the trends but nothing earth-shattering.

Louis Broome gave us great insight into the benefits Microsoft are getting run production (podcasts, video) inhouse. Cost-savings are one aspect (75% or so). Amazing to hear that their latest studio only cost US$13k to setup, and their 4.5 headcount are producing close to 1000 hours of video a year (thats with post production, compositing and the like).

Jeremiah Owyang argued that the corporate website is rapidly becoming irrelevant, being sidelined with the rise of 'disruptive' social network tools (from a marketeer's perspective). Go to the corporate website for product specs etc, but if you want buying advice and opinion, you are more likely to trust your peer networks. Corporate employees are in a way contributing to this with the rise of blogging etc, where they blur the corporate boundary with informal communication (bidirectional).

A good morning. Now looking forward to the tracks; looks like I'll hop between SOA and Security, although I'd also like to catch eGovernment!
ix2007

iX2007 Singapore - day 1 deep thoughts

iX2007 started today here in Singapore. The first conference that really caught my eye for quite some time. Today was keynote day.

iN2015 Update - Optimism in the air?

Interesting to see Mr Chan Yeng Kit (iDA) presenting his update on Singapore's iN2015. When you look at the progress being made in key areas like digital infrastructure upgrade and undergraduate admissions in ICT programmes, I think it reflects accurately the "mood" in the industry, which I'd summarise as "optimistic, with increasing confidence". SARS, dot.bomb and the Asian financial crisis are still recent memories but slowly receeding into history ...

Business Forum - a Truely Interactive Panel Discussion!

The Business Forum ran as a panel discussion involving Pek Chew Yai (SiTF), Lim Chin Hu (Frontline), Craig Gledhill (Cisco) and Phey Teck Moh (Pacific Internet). Robert Chew (Accenture) did a great job as moderator by feeding the discussion with comments coming from the chatroom. I've got to say that this resulted in the most interactive and interesting panel discussion I've ever experienced in this part of the world. Amazing to see the realtime flow from keyboard to chatroom then having a direct impact on the discussion. I think The Digital Movement cashed in on a really good bet by setting up the chat (running 37 Signals Campfire) and providing SMS and free wireless access. We may be too paisei here to step up to a microphone, but in the chatroom the speech runs free and wild!

Great to see the conference assuming a web persona; Jeremiah Owyang has already blogged on his Day 1 thoughts.

Singapore - Producers or Consumers?

All this enthusiasm to use the technologies at hand to enhance the conference experience really underscored for me one of the topics that got a bit of an airing during the panel: are we clear on our aspirations as both consumers and producers? I think so much of the focus in iN2015 is geared towards the consumption ... improving our already-great infrastructure, ensuring we have an ICT-savy workforce. I think we are still struggling with the production aspect, the irony being that there appears to be a huge amount of energy and creativity pent up in Singapore just waiting to be unleashed on the production side. How many of the grads from SMU, NUS, NTU and the like are destined to face the choice of (a) stay in Singapore but end up in a less exciting role than they had hoped for, or (b) head overseas to pursue their dreams? I guess they can take comfort that going overseas is at least a realistic option once again!

Meaning no disrespect to any of the pioneers doing amazing work in Singapore, ICT and Digital Media production is still niche here. And I don't see too many signs this is changing. There seems to be a defacto assumption that its good enough to be a "hub". After all, that is what has made Singapore so successful in logistics, corporate and financial services. But I wonder if that necessarily has to be the case for ICT? Its almost like we are afraid to dream the big dream.

I'll throw a hypothetical out there:

• Singapore has a skilled workforce, an attractive base for foreign workers, great infrastructure, stable and transparent government (in most things;), cultural ties to India, China and the West, high proficiency in English in addition to many other languages, and lower salary costs than most western countries.
  
• So why don't we see, for example, large software companies basing development centres here?
  
• Sure, the prevailing view is that if its not in China or India, you must be crazy. But what would be so crazy, for example, of a US company maintaining corporate HQ in the US, development centres in China or India, but running R&D, product management and lead development out of Singapore?
  
• Personally I think the net result would be a huge benefit for the company concerned. I think you would also find managing offshore development in India and China from Singapore surprisingly effective.
  
• And as a result, all these creative, innovative technopreneurs being groomed in Singapore would find much more opportunity to do big things in their own backyard.

Solutions? Well of course, in true Singaporean style, we could say the gahmen needs to do more to set a bold vision, attract and support ventures of this nature. That means EDB, SiTF and iDA. But I think the real change needs to come from within the ICT community in Singapore (inlcuding those working in the multinationals). Be big, be bold, be best. Believe.

Tomorrow...

Anyway, looking forward to the track sessions tomorrow. My only problem is that I want to be in 3-4 places at the one time. Hard to decide between the 6 tracks: digital media, eGovernment, security, SOA, wireless and eLearning. I think I want to participate in them all ... now is there a technology solution for that?

ix2007

Diving for SOAP Perls

Antony Reynolds' recent Diving for Perls with WSIF post gave a great example of how you can use HTTP bindings to call perl CGI scripts from Oracle BPEL Process Manager.

If your perl code is not already available to be called in this way, then what to do? Certainly the "ideal" would be make it available as a native Web Service and do away with any special binding. Thanks to the SOAP::Lite module, this is actually quite easy to do.

I'm going to walk through an example of how to take some aribitrary perl code, wrap it as a Web Service, and then call it from a BPEL process. See the diagram:

The Perl Code

In this example, there's really only one bit of code that "matters" ... a helloWorld function. I'm going to start with this wrapped in a perl class module called HelloWorld.pm. As you'll see shortly, wrapping the business functionality in a class is a good idea because it allows automatic dispatching from the Web Services interface.

$ cat HelloWorld.pm
#!/usr/bin/perl -w
use strict;
package HelloWorld;
our (@ISA, @EXPORT, $VERSION);
use Exporter;
$VERSION = 1.00;
@ISA = qw(Exporter);
@EXPORT = qw( helloWorld );

sub helloWorld {
my ($self,$foo) = @_;
return 'Hello ' . $foo;
}
1;

Important to note that while the code here contains some of the module niceties, it doesn't make any reference to SOAP, CGI or BPEL. It's plain perl. We can prove that with a little perl test program:

$ cat helloWorld.pl
#!/usr/bin/perl -w
use strict;
use HelloWorld;
print HelloWorld->helloWorld( 'Sunshine' );

$ perl helloWorld.pl
Hello Sunshine
$

The SOAP Interface

The dynamic typing of perl and flexibility of the SOAP::Lite module really live up to the make simple things easy motto. In three lines of code we have a SOAP CGI server for our HelloWorld class (that's why I made it a class;)

$ cat HelloWorld.cgi
#!/usr/bin/perl -w
use HelloWorld;
use SOAP::Transport::HTTP;
SOAP::Transport::HTTP::CGI
->dispatch_to('HelloWorld')
->handle;

That was so easy, there must be a catch right? Well yes, one comes to mind: the reply message elements will necessarily have some generated names (like "s-gensym3") since there is nothing in our code to provide any guidance for things like the "name" of function return value elements.

Testing SOAP Client-Server

After dropping HelloWorld.cgi and HelloWorld.pm into my apache cgi-bin, I'm ready to test the SOAP service over HTTP. We can whip up a client in no time:

$ cat HelloWorldWSClient.pl
#!/usr/bin/perl –w
use SOAP::Lite;

my $soap = SOAP::Lite
->readable(1)
->uri('urn:HelloWorld')
->proxy('http://localhost:8000/cgi-bin/HelloWorld.cgi');

my $som = $soap->helloWorld(
SOAP::Data->name('name' => 'Sunshine')
);
print "The response from the server was:\n".$som->result."\n";

$ perl HelloWorldWSClient.pl
The response from the server was:
Hello Sunshine
$

If we sniff the network or route this request via a tool like org.apache.axis.utils.tcpmon, we can see the outbound request and incoming reply:

Creating a WSDL file

Alas, perl's flexibility means that automatically generating a WSDL for our SOAP service is easier said than done. Unlike in strongly-typed languages, perl methods can take an arbitrary number of parameters of arbitrary type ... whereas of course a Web Service should have a very clearly defined interface.

I think one of the best approaches at present for generating WSDL in perl is the Pod::WSDL module. I'll perhaps leave that for another blog entry. For now lets just assume we'll manually create a WSDL for our service:

$ cat HelloWorld.wsdl
<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions targetNamespace="http://localhost:8000/HelloWorld" xmlns:impl="http://localhost:8000/HelloWorld" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns1="http://localhost:8000/HelloWorld">

<wsdl:message name="helloWorldRequest">
<wsdl:part name="name" type="xsd:string" />
</wsdl:message>

<wsdl:message name="helloWorldResponse">
<wsdl:part name="s-gensym3" type="xsd:string" />
</wsdl:message>

<wsdl:portType name="HelloWorldHandler">
<wsdl:operation name="helloWorld" parameterOrder="name">
<wsdl:input message="impl:helloWorldRequest" name="helloWorldRequest" />
<wsdl:output message="impl:helloWorldResponse" name="helloWorldResponse" />
</wsdl:operation>

</wsdl:portType>

<wsdl:binding name="HelloWorldSoapBinding" type="impl:HelloWorldHandler">
<wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http" />

<wsdl:operation name="helloWorld">
<wsdlsoap:operation soapAction="" />
<wsdl:input name="helloWorldRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://localhost:8000/HelloWorld" use="encoded" />
</wsdl:input>
<wsdl:output name="helloWorldResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://localhost:8000/HelloWorld" use="encoded" />
</wsdl:output>
</wsdl:operation>

</wsdl:binding>

<wsdl:service name="HelloWorldHandlerService">
<wsdl:port binding="impl:HelloWorldSoapBinding" name="HelloWorld">
<wsdlsoap:address location="http://localhost:8000/cgi-bin/HelloWorld.cgi" />
</wsdl:port>
</wsdl:service>

</wsdl:definitions>

Invocation from a BPEL Process

Now you have all the bits in place to invoke your Perl code as a fully-fledged Web Service from within BPEL. I won't go into this in detail here because it is the standard Web Service invocation process. Just add an "invoke" activity in your process and point it to a partner link defined based on the WSDL generated above.

Once you have deployed your process, you can test it from the BPEL Console. Here's an example of the invoke activity in one of my tests:

Conclusion?

Hopefully I've shown that exposing perl code as a Web Service is actually pretty simple. Once done, the code is then available for use by standards-based tools like Oracle BPEL Process Manager.

There are a couple of consideration to bear in mind though:

1. SOAP::Lite provides some great hooks for automatically generating a SOAP interface, however these come with the caveat that reply message elements will necessarily have some "generated" names
   
2. Automatic WSDL generation is confounded by perl's dynamic typing. Modules like Pod::WSDL provide some good solutions though.
   

An MQ and OCCI Demo

A little while ago I got to dust off my C++ skills for a project that was to use Oracle Database (via OCCI) and also Websphere MQ. Oracle and IBM already make a range of demos available, but they are mostly all very closely scoped on one feature only. Since I didn't find anything that included all they key concepts in a full working demo, I put together a combined OCCI/MQ demo to do the job (available for download as a tar/gzip file here: occidemo.tgz, see the readme.txt for details).

A couple of key things demonstrated:

1. C++ (OCCI) Oracle database access
   
2. Transparent Application Failover (TAF) notifications in C++ (OCCI)
   
3. Building a C++ application with MQ and OCCI support
   
4. Using makefile flags to build either with full or a "stub" database library class
   

The demo is written for Linux (32 or 64 bit) and has been tested with Oracle Database 10g Server, Oracle 10g Instant Client, and IBM WebSphere MQ 6.0.

The diagrams below give a simple exposition of how the demo is structured. The executables "mqproducer" and "mqconsumer" are MQ clients shuttle messages back-and-forth via queues. For each message sent by "mqproducer", a reply is expected from "mqconsumer". The readme.txt in the archive contains fairly detailed coverage of how to run the demo.

If the sample is built with full database support, then a "dblibrary" is linked in that will persist each message to database (and the dblibrary_test program can be use to test the operation).
If the sample is built with a database "stub", then a dummy database library is substitued, and the programs a built without any Oracle Database support linked in at all. This can be useful when just wanting to focus on the MQ aspects in isolation.

The top 10 dead (or dying) computer skills

I try to avoid postings that just refer you to other blogs or articles, but I've succumbed. ComputerWorld's The top 10 dead (or dying) computer skills prompted a bit of nostalgia. I scored 91% [giving myself 1% for the time I bought a book on COBOL while at uni ... and had the good sense to take it no further than that!!].

OS/2 brings back memories, of which I was also reminded when I first checked out Google's code search and found some of my 1995 OS/2 code lying around! [NB: these days, I look at this code and shudder "Eek!... buffer overflow vulnerability!!" ... security just wasn't front of mind back then! ]. But it also reminds me of how much thought I put into the decision to adopt C++ on OS/2. It very much felt like "this is a decision that I'll live with for years". But 12 years later, in 2007, that decision-making process seems so naive and foreign. Now it is routine to dabble in a couple of scripting languages, some Java, even some C++. The right (or most fun) tool for the job, right?

If I could say "Programming Language Bigotry" is a skill (some people certainly practiced and honed it like it was), then boy am I glad it seems to be a thing of the past and perhaps it deserves to be #1 in this list!

After a brief post-dot-boom hiatus, the drammatic rate of evolution is certainly back, spurred on by Web 2.0 hype. The rate of technological change has indeed become so "normal" that a top 10 list hardly scratches the surface. Personally I would have voted for int 21h. I'm sure generations to come will have absolutely no idea what that means, but for me and presumably many others, I can sum up a year of computer science with that very phrase.

For many (myself included), To Be Alive is To Be Learning and vice versa. The new religion if you will. "Lifelong learning" or "learning for life" are too trite and miss the essential truth.

Other may say that to be continuously learning is to be in a perpetual state of childhood. Look at some of the toys we are learning about and maybe they have a point!

Postscript: I just re-listened to a WebDevRadio Episode 18 which reminded me that Coldfusion is not dead!! At least according to the guys at Mach ii..

Monitoring log files on Windows with Grid Control

The Oracle Grid Control agent for Windows (10.2.0.2) is missing the ability to monitor arbitrary log files. This was brought up recently in the OTN Forums. The problem seems to have been identified by Oracle earlier this year (Bug 6011228) with a fix coming in a future release.

So what to do in the meantime? Creating a user defined metric is one approach, but has its limitations.

I couldn't help thinking that the support already provided for log file monitoring in Linux must already be 80% of what's required to run under Windows. A little digging around confirmed that. What I'm going to share today is a little hack to enable log file monitoring for a Windows agent. First the disclaimers: the info here is purely from my own investigation; changes you make are probably unsupported; try it at your own risk; backup any files before you modify them etc etc!!

Now the correct way to get your log file monitoring working would be to request a backport of the fix from Oracle. But if you are brave enough to hack this yourself, read on...

First, let me describe the setup I'm testing with. I have a Windows 10.2.0.2 agent talking to a Linux 10.2.0.2 Management Server. Before you begin any customisation, make sure the standard agent is installed and operating correctly. Go to the host home page and click on the "Metric and Policy Settings" link - you should not see a "Log File Pattern Matched Line Count" metric listed (if you do, then you are using an installation that has already been fixed).

To get the log file monitoring working, there are basically 5 steps:

1. In the Windows agent deployment, add a <Metric NAME="LogFileMonitoring" TYPE="TABLE"> element to $AGENT_HOME\sysman\admin\metadata\host.xml
   


2. In the Windows agent deployment, add a <CollectionItem NAME="LogFileMonitoring"> element to $AGENT_HOME\sysman\admin\default_collection\host.xml
   


3. Fix a bug in $AGENT_HOME\sysman\admin\scripts\parse-log1.pl
   


4. Reload/restart the agent
   


5. In the OEM console, configure a rule and test it
   

Once you have done that, you'll be able monitor log files like you can with agents running on other host operating systems, and see errors reported in Grid Control like this:


So let's quickly cover the configuration steps.

Configuring metadata\host.xml
Insert the following in $AGENT_HOME\sysman\admin\metadata\host.xml on the Windows host. NB: this is actually copied this from the corresponding host.xml file used in a Linux agent deployment.

<Metric NAME="LogFileMonitoring" TYPE="TABLE">
<ValidMidTierVersions START_VER="10.2.0.0.0" />
<ValidIf>
<CategoryProp NAME="OS" CHOICES="Windows"/>
</ValidIf>
<Display>
<Label NLSID="log_file_monitoring">Log File Monitoring</Label>
</Display>
<TableDescriptor>
<ColumnDescriptor NAME="log_file_name" TYPE="STRING" IS_KEY="TRUE">
<Display>
<Label NLSID="host_log_file_name">Log File Name</Label>
</Display>
</ColumnDescriptor>
<ColumnDescriptor NAME="log_file_match_pattern" TYPE="STRING" IS_KEY="TRUE">
<Display>
<Label NLSID="host_log_file_match_pattern">Match Pattern in Perl</Label>
</Display>
</ColumnDescriptor>
<ColumnDescriptor NAME="log_file_ignore_pattern" TYPE="STRING" IS_KEY="TRUE">
<Display>
<Label NLSID="host_log_file_ignore_pattern">Ignore Pattern in Perl</Label>
</Display>
</ColumnDescriptor>
<ColumnDescriptor NAME="timestamp" TYPE="STRING" RENDERABLE="FALSE" IS_KEY="TRUE">
<Display>
<Label NLSID="host_time_stamp">Time Stamp</Label>
</Display>
</ColumnDescriptor>
<ColumnDescriptor NAME="log_file_match_count" TYPE="NUMBER" IS_KEY="FALSE" STATELESS_ALERTS="TRUE">
<Display>
<Label NLSID="host_log_file_match_count">Log File Pattern Matched Line Count</Label>
</Display>
</ColumnDescriptor>
<ColumnDescriptor NAME="log_file_message" TYPE="STRING" IS_KEY="FALSE" IS_LONG_TEXT="TRUE">
<Display>
<Label NLSID="host_log_file_message">Log File Pattern Matched Content</Label>
</Display>
</ColumnDescriptor>
</TableDescriptor>
<QueryDescriptor FETCHLET_ID="OSLineToken">
<Property NAME="scriptsDir" SCOPE="SYSTEMGLOBAL">scriptsDir</Property>
<Property NAME="perlBin" SCOPE="SYSTEMGLOBAL">perlBin</Property>
<Property NAME="command" SCOPE="GLOBAL">%perlBin%/perl</Property>
<Property NAME="script" SCOPE="GLOBAL">%scriptsDir%/parse-log1.pl</Property>
<Property NAME="startsWith" SCOPE="GLOBAL">em_result=</Property>
<Property NAME="delimiter" SCOPE="GLOBAL">|</Property>
<Property NAME="ENVEM_TARGET_GUID" SCOPE="INSTANCE">GUID</Property>
<Property NAME="NEED_CONDITION_CONTEXT" SCOPE="GLOBAL">TRUE</Property>
<Property NAME="warningStartsWith" SCOPE="GLOBAL">em_warning=</Property>
</QueryDescriptor>
</Metric>

In the top TargetMetadata, also increment the META_VER attribute (in my case, changed from "3.0" to "3.1").

Configuring default_collection\host.xml
Insert the following in $AGENT_HOME\sysman\admin\default_collection\host.xml on the Windows host. NB: this is actually copied this from the corresponding host.xml file used in a Linux agent deployment.

<CollectionItem NAME="LogFileMonitoring">
<Schedule>
<IntervalSchedule INTERVAL="15" TIME_UNIT = "Min"/>
</Schedule>
<MetricColl NAME="LogFileMonitoring">
<Condition COLUMN_NAME="log_file_match_count"
WARNING="0" CRITICAL="NotDefined" OPERATOR="GT"
NO_CLEAR_ON_NULL="TRUE"
MESSAGE="%log_file_message%. %log_file_match_count% crossed warning (%warning_threshold%) or critical (%critical_threshold%) threshold."
MESSAGE_NLSID="host_log_file_match_count_cond" />
</MetricColl>
</CollectionItem>

A bug in parse-log1.pl?
This may not be an issue in your deployment, but in mine I discovered that the script had a minor issue due to an unguarded use of the Perl symlink function (a feature not supported on Windows of course).

The original code around line 796 in $AGENT_HOME\sysman\admin\scripts\parse-log1.pl was:

...
my $file2 = "$file1".".ln";
symlink $file1, $file2 if (! -e $file2);
return 0 if (! -e $file2);
my $signature2 = getSignature($file2);
...

This I changed to:

...
my $file2 = "$file1".".ln";
return 0 if (! eval { symlink("",""); 1 } );
symlink $file1, $file2 if (! -e $file2);
return 0 if (! -e $file2);
my $signature2 = getSignature($file2);
...

Reload/restart the agent
After you've made the changes, restart your agent using the windows "services" control panel or "emctl reload agent" from the command line. Check the management console to make sure agent uploads have resumed properly, and then you should be ready to configure and test log file monitoring.

Validating Oracle SSO Configuration

A failing OC4J_SECURITY process recently had me digging out an old script I had put together to test Oracle Application Server Single-Sign-On (OSSO) configuration.

How and where the OSSO server keeps its configuration is a wierd and wonderful thing. The first few times I faced OSSO server issues I remember digging through a collection of metalink notes to piece together the story. It was after forgetting the details a second time that I committed the understanding to a script (validateSso.sh).

Appreciating the indirection used in the configuration is the key to understanding how it all really hangs together, which can really help if you are trying to fix a server config issue. Things basically hang together in a chain with 3 links:

1. Firstly, the SSO server uses a privileged connection to an OID server to retrieve the OSSO (database) schema password.

2. With that password, it can retrieve the SSO OID (ldap) server connection details from the OSSO (database) schema.

3. Thus the SSO Server finally has the information needed to connect to the OID server that contains the user credentials.

The validateSso.sh script I've provided here gives you a simple and non-destructive test of all these steps. The most common problem I've seen in practice is that the OSSO schema password stored in OID gets out of sync from the actual OSSO schema password. I think various causes of these problems, but the script will identity the exact point of failure in a jiffy.

Getting your Oracle Forum posts as RSS

In my last post I said one of my "Top 10" wishes for OTN was to be able to get an RSS feed of posts by a specified member to the Oracle Forums.

At first it may sound a bit narcissistic to have a feed that allows you to follow what you have written yourself!

It was my exploration of Jaiku that prompted the thought. Since "presence" is their big thing, I've been experimenting to see what's its like to have Jaiku aggregate all your web activity. So far it looks really cool - I love the interface. Must say that I'm not sure how useful Jaiku may turn out to be in the long run ... I suspect it works best if you have a whole lot of your friends also using it. NB: the Jaiku guys are particularly focused on mobile phones. Its not something I've tried yet because I think it would be a bit ex from where I live.

So Jaiku was the catalyst for me thinking about getting an RSS feed of my forum posts. Recently I've been trying to make an extra effort to contribute to the forums; frankly, they've always seemed a little quieter than I think they should be. So seeing any forums posts I make highlighted on Jaiku should be one of the neat indicators of my "web presence".

Problem is, while you can get a web page that lists your recent posts, and you can subscribe for email alerts when authors post, I wasn't able to find a way of getting an RSS feed for a specific author's posts.

So I created a little perl script that scrapes the HTML and generates an RSS feed (using XML::RSS::SimpleGen). I've packaged it as a CGI program on a server I have access to. That's what I registered on my experimental Jaiku site, and it works like a charm.

Until Oracle build this feature into the forums, feel free to take my oracleForumRSS.pl script and experiment away. Its pretty basic, but is generic for any forum user and ready to go. Sorry, but I'm not hosting it for direct use by others, so you'd have to find you own server with cgi or convert it to script that spits out a static rss xml page instead.

Post-script: Eddie Awad blogged on the "Easy Way" to do this using Dapper. Very cool, thanks Eddie!

OTN Semantic Web - first look

My last post on the state of the OTN community was probably a little long on rhetoric. So I thought since the move of Oracle Blogs! to the much vaunted Semantic Web seems well underway, it would be worth taking a first look. The old site has taken a bit of stick for being, let's say, less than engaging.

The big improvement by using the Semantic Web is of course all the slicing and dicing it allows to hone in on posts of interest, with the blogger tag cloud giving instant feedback (and access) to the bloggers most active on a given subject.

I got to say though it doesn't take long before you realise this site needs a serious design and usability facelift. Urgently!

Maybe its easy to be critical with something new. Actually, no "maybe" about it...

OK, I won't nitpick too much. The biggest problem is that we've paid for all the slice & dice flexibility in the worst possible way ... all the content has been squeezed off the page!

If Oracle do nothing else, they should push all the filtering and clouds out of the way to make a nice, big section of the page available for the star of the show - the content. Use AJAX to make all the filtering available in an instant, but avoid the clutter. At that point, we have a decent replacement for the old blog site.

But for OTN to truely find its Web 2.0 mojo, I think the Semantic Web is probably laying some important foundations but its just the beginning.

My Top 10 OTN/Semantic Web Wishlist
OK, so quick brainstorm, and here's a selection of things I'd really love to see happen on OTN:

1. Take Down This Wall ... between content types (not what Justin originally meant, I know)! The current layout of the Semantic Web page puts concrete delineation between content types (podcast, blogs, forums and other personally attributed content) at the top of the page. At first that may sound fine, but its locking us into a mindset and behaviour patterns that assume these are all distinct in terms of production and consumption. A destructive and divisive fallacy. Get rid of it, and make the content type just another filter.
   


2. Long live the forums! These are places where you can actually have a conversation (instead of trying to have a conversation in blog comments). OK, so they look a bit dated, and the level of participation can be a problem. Nothing a lick'a paint can't fix, and use the Semantic Web to drive usage.
   


3. Give Oracle's "celebrity" bloggers a personal forum that links neatly off their blog so that conversations can be usefully launched on the back of a blog post. What the heck, let everyone have a personal forum!
   


4. Let me take an RSS feed of my forum post history. Its the kind of thing I'd put on my Jaiku page. At the moment, we can just watch via email.
   


5. Let me take an RSS feed of any Semantic Web page I find/define (with all the current filtering etc).
   


6. Drop the barriers to participation. The Semantic Web makes concerns of the "quality" of bloggers irrelevent. If they never post, they get buried. OTN registration should include an opt-in for an Oracle-host blog, or a link to an externally hosted blog.
   


7. An OTN Community Site Badge. To promote participation, how about a logo/widget/javascripty thing that bloggers can put on their blog? It would both brand their blog as part of the OTN community, and also provide a link back to the OTN Semantic Web. It would be neat if it actually had a function (like clustermaps), but I can't think of anything useful right now.
   


8. Invert the Semantic Web. OK, we're getting used to coming top down. But what if I *start* from someone's blog post? It could be a really neat thing to be able to then explore the Semantic Web from that point out ... to discover related or linked pages. Worth an experiment I think.
   


9. Drop the dopomene theme. I'm in the Semantic Web, discovering fascinating information in ways that have never been possible before. This is really exciting! ... but the look and feel is blaring a very incongruous message that messes with your psyche. We need a better use of colour and graphics.
   


10. ... and a dozen other insanely cool Web 2.0 things that I can't even imagine right now, and if I could I'd be well on my way to my first $1b.
    

Wow, I'm actually getting excited now...

No respect! Should Justin care?

Justin "Dangerfield" Kestelyn launched one of the most lively discussions the Oracle community has ever had with his "I Don't Get It" post some weeks back.

"In particular," he states, "Oracle gets zero credit in this community for its rather aggressive support of blogging (by employees and nonemployees), despite the fact that a rather large blogging community exists and has for some time"

Strangely, there seems to be pretty unanimous agreement that there is a large number of bloggers out there, and some very good ones at that. Vincent McBurney's blog made special mention of Nishant Kaushik, Rob Smythe and Steven Chan for example.

And if we also consider the OTN Podcasts (my favourites being the ones that feature interviews with the "names" like Tom Kyte and Wim Coekaerts), it seems to me pretty evident that we actually have a pretty healthy community of content creators.

But when I look back at what Justin actually said, he was referring specifically to the lack of credit from the Web 2.0 community.

I think the OTN team - and Justin in particular - have been doing a fantastic job with the blogsphere and podcasts. But is that enough to make a stir in the Web 2.0 scene? Maybe a year or two ago it would have, but not any more. Sadly (for Justin) it is now just all too routine.

How many "Web 2.0 firsts" can OTN really claim? The harsh reality is that to make a splash and get some respect in the Web 2.0 community, Oracle needs to do much much more. And I don't think its about content or whether our blog etiquette is any good. Leadership and innovation is the name of the game in two important areas:

• How to build more effective social networks and find new and better ways for this to deliver real benefit to the community. At present, I'm not sure we even deserve the "community" moniker .. it feels more like a public swimming pool we all just happen to go to, rather than a forum (in the Roman sense) where we meet, discuss and debate.
  


• Invent and apply cutting-edge Web 2.0 techniques and technologies to support this goal. Yes, this IS about technology;) The Web 2.0 community is incredibly dynamic and creative at this point. Take blogs for example. They've been around for a while. Long enough for people to discover that for some things they are really good, but in other ways they suck (like trying to have a "conversation" in comments). So we now have sites like twitter, tumblr, virb and jaiku all experimenting with different approaches and trying to push the envelope in meaningful ways. Its this kind of creative experiementation that we haven't seen Oracle doing in the past ... with the one recent exception being the semantic web (hopefully an indicator of more great things to come). If Oracle really wants Web 2.0 street cred, OTN should be the playground where it is seen to be exploring the outer limits of what is possible - some of which may find its way back into the Fusion Middleware product line.
  

One notion we must definitely reject is that somehow we need to coach all the Oracle bloggers into becoming Web Celebs. To do so totally ignores (and destroys) the value of diversity in the community. Personally, I identify five "kinds" of web presence we should embrace:

1. Leadership and Product Management as a "conversation". These are the celebs and thought leaders engaging with the community, but very much with their corporate responsibility at the fore. Funny thing is, I had the impression Oracle was doing much better in this regard, but it doesn't hold up to inspection. Mark Wilcox is one of the few getting close. Perhaps commercial considerations actually make it a very hard thing to do without tipping the competition too much, or just sounding like a mouthpiece for marketing.
   


2. Web 2.0 as Shared Memory. I think one of the completely understated revolutions going on. As I've blogged before, and epitomised by the likes of Alejandro Vargas, this is all about using the web to finally Get Knowledge Management Right. These tend to be boring as hell to try and follow unless they are right in your niche. Scenario: One day, you'll be sweating a problem. Ask google, and thank your lucky stars that there are people around like Alejandro.
   


3. Living your professional life online. Probably the most common approach today on OTN. Its a diary, scrapbook and log. You may find some really good gems, but there's no harm in being obscure in this category... you're just one of the community and its often done more for your own personal benefit.
   


4. The personal/social presence. And yes there is room for all those who are part of the community (because they work at Oracle for example) but just want to talk about baseball!
   


5. The audience. Let's not forget the vast majority of people who are searching and reading, but will never do much more that perhaps post a question to a forum or maybe a comment on a blog. For a whole range of reasons there's no value or motivation for them to go further. Don't try and make them blog. It won't work. But should we do everything possible to make sure they are well served by the community ... yes!! Numerically, they ARE the community.
   

Justin finished his initial post with a somewhat flippant "...maybe I shouldn't even care!". But perhaps he unwittingly hit the nail on the head.

It's a truism in business that if you forget who your customers are, you are doomed. Similarly, if OTN becomes preoccupied with impressing the Web 2.0 community as its primary mission, I'm pretty sure they will find success "inexplicably" elusive (and prove that all of Justin's denials of it being a PR conspiracy are lies!!).

Success will come most easily if OTN focuses on serving its real constituency first - the Oracle community of employees, users and developers. Do that well, and if OTN is indeed pushing the boundaries, then the Web 2.0 cred will be the just reward.

I guess in a way its like being cool. Try to be cool and you'll fail. You just are (or not, as the case may be).

Do Oracle temp tables behave correctly under DBI?

Andon Tschauschev recently posted on perl.dbi.users concerning an apparent problem with temp tables "disappearing" between statements in the same session. He was using SQL Server via ODBC support.

The discussion and investigation continues, but it made me think to test if there's any similar strange behaviour with Oracle via DBI.

The temporary table model is somewhat different in Oracle, and centers around the "CREATE GLOBAL TEMPORARY TABLE.." statement. Temp table definitions are always global, but data is always private to the session, and whether data persists over a commit depends on whether the qualification "on commit preserve rows" or "on commit delete rows" is specified.

testOraTempTables.pl is a simple test script to check out the behaviour. Good news is that all seems to be a-ok. The temporary table definition is persistent across sessions, but data is not, and importantly (..the point of Andon's investigation..) data is preserved across DBI calls within the same session as expected.

Sample output from the test program:

C:\MyDocs\Testers2\perl\dbi>perl testOraTempTables.pl orcl scott tiger
[1st connection] connect to orcl {AutoCommit => 1}:
[1st connection] create global temp table:
create global temporary table t1 (x varchar2(10)) on commit preserve rows
[1st connection] insert 3 rows of data into it: insert into t1 values (?)
[1st connection] should be 3 rows because we have "on commit preserve rows" set:
select count(*) from t1 = 3
[2nd connection] connect to orcl:
[2nd connection] should be 0 rows because while the table definition is shared, the data is not:
select count(*) from t1 = 0
[2nd connection] disconnect:
[1st connection] disconnect:
[1st connection] reconnect {AutoCommit => 0}:
[1st connection] should be 0 rows because this is a new session:
select count(*) from t1 = 0
[1st connection] drop the temp table: drop table t1
[1st connection] create global temp table:
create global temporary table t1 (x varchar2(10)) on commit delete rows
[1st connection] insert 3 rows of data into it: insert into t1 values (?)
[1st connection] should be 3 rows because we have autocommit off and not committed yet:
select count(*) from t1 = 3
[1st connection] should be 0 rows because now we have committed:
select count(*) from t1 = 0
[1st connection] disconnect:
[1st connection] reconnect {AutoCommit => 1}:
[1st connection] insert 3 rows of data into it: insert into t1 values (?)
[1st connection] should be 0 rows because we have autocommit on and "on commit delete rows" defined:
select count(*) from t1 = 0
[1st connection] disconnect:
[1st connection] reconnect {AutoCommit => 0}:
[1st connection] drop the temp table: drop table t1
[1st connection] disconnect:

Burning DVDs with unicode filename support

I have quite an eclectic music collection, which grew considerably last year when I spent quite a bit of time in Tokyo. My favourite Sunday afternoon haunt was HMV @ Times Square, checking out the Shunjuku South indie chart. A suitcase of CDs later, I finally got around to ripping my entire CD collection and adding it my old ripped vinyl collection. All nicely organised, named and categorised in iTunes.

Feeling very pleased with myself, I wanted to archive all my hard work onto a set of DVDs, only to find that my (Windows-based) recording software baulked at the Japanese, Korean and Chinese characters in the folder and filenames.

My first thought .. should just need to change the file system format? But it didn't take long to discover it wasn't so simple. In fact, of the half a dozen different disk burning softwares I ended up trying out (including most of the "major" names), all but one failed to handle my babel of disks correctly.

So I'd like to spread the word. VSO CopyToDVD was the ONLY product that worked. It costs a few bucks to buy, but I just downloaded the limited-time trial and it passed the unicode test with flying colours. I didn't really poke around all its features, but it seems to pack the lot. And the fact that I could download it and bang out a few DVDs in short order pretty much sums things up.

I'll be buying it, and if you also need to burn disks with unicode file/folder names then I can recommend you check it out too.

Getting environment variables on the Oracle database server

Say you have a connection to a remote Oracle Database server and want to get the ORACLE_HOME setting. Or any other environment variable for that matter. As far as I can see, Oracle doesn't provide any direct, supported way to do this.
In 10g however, there's an interesting procedure DBMS_SYSTEM.GET_ENV available which does the job:

set autoprint on
var ORACLE_HOME varchar2(255)
exec dbms_system.get_env('ORACLE_HOME',:ORACLE_HOME)

PL/SQL procedure successfully completed.

ORACLE_HOME
-----------------------------------------
D:\oracle\product\10.2.0\db_1

DBMS_SYSTEM is an undocumented/unsupported package. It mainly seems to be an internal utility function for debugging and event monitoring. The package itself is obfusticated, but we can discover a little about it from the data dictionary. The USER_PROCEDURES view lists the individual procedures available in the package:

select PROCEDURE_NAME from USER_PROCEDURES where OBJECT_NAME = 'DBMS_SYSTEM';
PROCEDURE_NAME
------------------------------
DIST_TXN_SYNC
GET_ENV
KCFRMS
KSDDDT
KSDFLS
KSDIND
KSDWRT
READ_EV
SET_BOOL_PARAM_IN_SESSION
SET_EV
SET_INT_PARAM_IN_SESSION
SET_SQL_TRACE_IN_SESSION
WAIT_FOR_EVENT

And USER_ARGUMENTS can tell us about the parameters. For example:

select OBJECT_NAME,ARGUMENT_NAME,POSITION,DATA_TYPE,IN_OUT
from USER_ARGUMENTS
where PACKAGE_NAME='DBMS_SYSTEM' and OBJECT_NAME='GET_ENV'
order by POSITION;

OBJECT_NAME ARGUMENT_NAME POSITION DATA_TYPE IN_OUT
------------- -------------- ---------- --------- ------
GET_ENV VAR 1 VARCHAR2 IN
GET_ENV VAL 2 VARCHAR2 OUT

Given an environment variable name (VAR), GET_ENV returns its value (VAL). These values are coming from the system environment that belongs to the Oracle server process. If you have a dedicated server config, the environment is inherited from the tnslsnr process that spawned the server process. If shared server, then the environment is inherited from whatever process (PMON? PSP0?) that started the shared server process.
So an interesting poke around in some Oracle internals, but there are lots of reasons why you shouldn't use this trick in any production situation!

• It is undocumented and unsuppported. The "get_env" method seems to have appeared in 10g, but there's also no guarantee it will be present in any future versions.


• There are better solutions. SQL client code shouldn't directly depend on server environment variables.


• Remember it is instance specific, and may be misleading in a RAC environment.

Synchronising two directory trees - update

I've just released an update to tree-sync-2.2.pl on CPAN. This fixes a few bugs with special character handling in filenames.

I've currently three main uses for the script:

• On my Windows XP laptop, I use it to backup my working files to an external drive. For this I use the "fwdonly" mode so that the external drive copy is a perfect copy of what I have on the laptop.
• Also on my laptop, I have a collection of files that I sync with the external drive in "full" sync mode (default). This means changes will be exchanged bidirectionally. This is useful because sometimes I will drop files into this area with the external drive connected to another machine.
• Thirdly, I use it on my server to sync a collection of files from cvs into a web-visible area.

PS: As of Oct-2008, the tree-sync project is now on github. Use this if you want to contribute to development. Of course, releases will still be distributed for use on CPAN.

Find and tail the Oracle alert log

"Where's the alert log?" .. usually the first thing you want to know when looking at a new database.

Oracle's Grid Control solves this problem very well with it's web interface. But not always available.

Or in my case recently, where I had 6 databases setup on a single machine for various testing scenarios, I was getting tired of cd'ing all over the place, forgetting paths, or ending up with too many windows open for my own good. Getting well sick of this, I did a quick hunt for scripts to help but surprisingly didn't find much. So diving in, I created oraAlertLog.sh and now I'm happy;)

This script is for running on the database server. The Oracle environment must be set, and - at least for the first call - the database must be available so that the "background_dump_dest" parameter can be obtained. The script will cache the alert log location so that it will still work if the database happens to be down.

After getting this running I thought "duh!", should have been in perl so it would be possible to run on any platform supported by Oracle. Here's a version in Perl: oraAlertLog.pl. It requires Perl with DBI and DBD::Oracle .. the Perl distribution included with Oracle Database is fine for this (you just need to make sure the environment is properly configured).