JDeveloper Filter Add-in Now on Sourceforge

I introduced the JDeveloper IDE External Filter Add-in project last weekend. Happy to announce that the project's registration on sourceforge is complete, so full source and downloads are now available from the sourceforge jdevfilter project page.

Tux gets his break with OS/2

Staying with my OS/2 reminiscence for a moment, there's a trove of old gems up on YouTube (including the famous quote from Bill "we believe OS/2 is the platform for the 90's").

While IBM ads in the US defied the limits of corny, in Japan it is all so kawaii.

Of course! The correct way to market an operating system is with a TV star/model (Yamaguchi Tomoko 山口智子), a bear and a penguin.

At least we know what Tux was up to before he got his big break with Linux in 96...taking gigs playing "DOS" and living it up in Ropongi!

OS/2 Still Kicking..!?

I was pretty amazed to see IBM responding as recently as 16-Jan-2008 to the requests by the OS/2 online community to open source OS/2.

I've admitted before to being an OS/2 fanatic for a few years, but seriously, I think we should let it die and just savour the memories!

One of the great things about the OS/2 community way-back-when is that it was a total geek-out (memories of the OS/2 Devcon in 1992 - still have the bag tag!). Linux has I think long taken that mantle.



It's a bit of a laugh to find some of my OS/2 detritus still kicking around the Internet, like:

• My article in EDM/2: REXX - The Developer's Best Friend
• A few tools still up in the os2 collection at hobbes:
• xeol for text file format conversion
• A Workplace Shell Command Shell Starter
• INISED - a utility to manipulate Windows-style profile (INI) files
• varc - Visual archive utility front-end, requires VREXX
• dsize - Display file/directory space usage on drives (REXX)
• shlong - Lists non-FAT names on an HPFS drive (REXX)
• .. mostly developed using REXX or my favourite compiler of the time: Borland C++ for OS/2!

I had fun with a couple of other projects that never made it to release (in both cases beaten to the post by the big boys. But dang it, no M&A windfalls in those days!):

• LNDW - Lotus Notes Doorway. I wrote a web server in C++ that provided gateway access to Lotus Notes database under OS/2. Tricky adaptation of the Notes C API to C++ and getting it to work with BC++. Then came Domino...
• OS2IP - a project with a long-lost collegue Bryan Ryan (if you are out there Bryan - drop me a line!). At the time, there was no free TCP/IP stack for OS/2 Warp, so we set about writing one. Got as far as an NDIS packet driver, and a user mode frame interface before IBM finally got their act together and made us obsolete! Nevertheless, good fun hacking around with the Driver Dev Kit in assembler, and bridging to a C++ user-mode API. I don't think it was until then that I really understood Comer in detail. When you are working at that level, it is amazing how exciting exchanging just a few bytes of data in a chat application can be!

JDeveloper Filter Add-in

I hinted at a project to develop a simple filter add-in in my last post on Code Generation with JDeveloper. So that turned into my latest weekend project;-)

I'm registering the project on sourceforge, but in the meantime you can get the full source kit here, or if you prefer just the add-in jar here (which just needs to be dropped into your ${jdev.home}/jdev/extensions folder). [30-Jan-2008: the project is now available on sourceforge]

What does it do?
It is a simple idea. Allow you to invoke an external program to filter any text you have selected in the JDeveloper Editor. Of course, it is up to you to define what filter means - limited by your imagination only. And by externalizing the filter process, it means you can use the tool or scripting language of your choice to implement and make on-the-fly changes to your filter.

To be precise, the filter operates like operating system pipes:

• Selected text is sent to the standard-input of the filter process.
• The standard-output of the filter process is written back to replace the selected text.

Configuring the Filter
Configuration is done via the JDeveloper Tools | Preferences menu, and is very simple at present. It only supports a single filter process definition. As you can see in the screenshot, you just need to enter a valid command line to invoke the filter. A nice enhancement would be the support for multiple filter definitions.


This is where the flexibility comes in. As an example, I've provided a rot13 encoder written in ruby [samples/myrot13filter.rb in the source kit].

puts $stdin.readlines.to_s.tr("A-Za-z", "N-ZA-Mn-za-m")

Ruby, Perl, php, vbscript/WSH, bash ... use whatever you prefer. Rather than needing to develop and deploy an add-in in Java, you can just script the filter.

Applying the filter is simply a matter of selecting some text in the IDE, and choose the Custom Text Filter item from the right-click pop-up menu

The Development View
The truth is that I developed this add-in as more an investigation of the Extensions SDK, and the filter idea appeared to be unexplored territory.

As Brian Duff writes, there are some major improvements in the support for JDeveloper Extensions in 11g. Since it is still in preview, I decided to stick with Extension SDK 10.1.3 however.

There are four classes of significance. since this was an educational exercise, I spent a bit of time to make sure I really understood how it all worked, and tried to reflect that with liberal comments in the source.

• CodePaneAddin - implements the pop-up menu hook and applies the filter
• ExecShell - wraps the invocation of the filter process iwth stdin/out handling. It is independent of the extension API and may be of interest in its own right
• ConfigPanelData - handles preferences panel configuration
• ConfigData - bean to wrap the configuration data

The source kit comes with a JDeveloper project and also ant build file. If you want to build and deploy the add-in to JDeveloper, simplest is to first check the environment-specific settings in build.properties and then simply run

C:\MyDocs\MyDev2\JDevFilter>ant deploy
Buildfile: build.xml

init:

prepare:

compile:

jar:

deploy:
[copy] Copying 1 file to C:\oracle\jdevstudio10131\jdev\extensions

BUILD SUCCESSFUL
Total time: 0 seconds

Interested? Head on over to sourceforge to download the kit.

Code Generation with JDeveloper

I've written about Jack Herrington's Code Generation in Action before; it is a great book for stimulating your thinking on how you go about producing code.

JDeveloper has evolved into one of the strongest IDEs when it comes to code generation. It's not explicitly mentioned in the book however. That's not a criticism of the book, but it does beg a review of how JDeveloper stacks up against the concepts covered.

Integrating code generation into your IDE

Writing a code generator can be difficult-but convincing your fellow engineers to use [it] can be even more difficult. The key .. is to make it as simple as possible to use.

IDE integration is one approach to address this problem, and the common techniques supported are external tool invocation and filtering.

JDeveloper only directly supports external tool invocation (see the Tools menu). For instance, you can call simple Ruby scripts like the following to perform file-level processing (this example simply comments each line of the file that is currently open in the editor pane).

fh = File.open( ARGV[0] )
text = fh.read()
fh.close
text.gsub!( /^/, "// " )
File.open( ARGV[0], "w" ).write( text )

NB: in the 11.1 preview, it seems the "Availability" conditions are still a little broken and it is not possible to make commands available only when a file (or certain type of file) is open.

Filters are not directly supported by JDeveloper, however implementing a filter is a fairly straight-forward project using the JDeveloper Extension API.

JDeveloper Extension API

Herrington does not directly address the topic of IDE APIs, which is understandable given the scope of his book. However if you are looking to implement complex and very specific code generation approaches you will probably want to look into this topic further.

JDeveloper comes with a comprehensive Java-based extension API which was significantly revamped in (I think) 10.1.3 to bring it into line with the standardisation efforts of JSR-198.

I must say documentation of the extension development process is a little thin. And it is only in 11.1 that JDeveloper has explicit project support for extension development (although I find it a little broken and mysterious in the preview edition). The best approach at present seems to be good ol' copy-and-modify the extensive set of samples provided by Oracle at the Extensions Exchange. There is a growing number of 3rd party open source extensions that are even available through the JDeveloper Check for updates menu.

Still, no filter add-in is available, and you are of course stuck with developing in Java. It of course puts me in mind of a little project: a simple filter add-in that then allows you to invoke a filter process developed in the language of your choice such as perl, ruby or python. Maybe I'll be able to follow-up this post with an example;-) PS: yes I did; you can read about it here.

JDeveloper's Generation Credentials

I've been hedging around the fact that JDeveloper is very significantly geared towards code generation itself. It is one of the main drawcards for why you would use it over your favourite text editor.

In the back cover of the book, Herrington includes a very illustrative Generator Tree which provides a taxonomy of generators, with of course references back to what is covered in the book. I thought I'd experiement with mapping JDeveloper into the taxonomy. I doubt if I'll manage to cover all of JDeveloper's features (or get it all right), but here goes:

Class	Platform	Technology	Herrington	JDeveloper
Database Access	MS	ASP	ASP Generator	n/a
		ASP.NET	ASP.NET Generator	n/a
	Java	EJB	EJB Generator	EJB Entities from Tables
		JDBC	JDBC Generator	ADF Business Components
		JDBC	n/a	Toplink O-R Mapping
	PHP	PHP	PHP Generator	n/a
	Perl	DBI	Perl DBI Generator	n/a
User Interface	Java	JSP	JSP Generator	JSP/struts/JSF wizards
		Swing	Swing Generator	Swing wizards; ADF Swing wizards
	MS	MFC	MFC Generator	n/a
Documentation	SQL	SQL	SQL Documentation Generator	n/a
	Java	Java	JavaDoc	JavaDoc
Unit Tests	C	Tests	Augmented C Unit Test Generator	n/a
		Ordered Tests	Ordered Test Generator	n/a
	Test Data	n/a	Test data generator	n/a
	Test Robots	n/a	Test robot generator	n/a
File Formats	Java	CSV	CSV file Reading Java Generator	File Adapter
	Java	Data Adapter	Data Adapter Generator	(arguably) File Adapter; Toplink
	Java	Binary	Binary File Reader Generator	n/a
Web Services	Java	XML-RPC	XML-RPC Generator	n/a?
	Java	SOAP	SOAP Generator	Java Web Service from WSDL
		PL/SQL	n/a	PL/SQL Web Service
		JMS	n/a	JMS Web Service
		Various	n/a	Technology Adapters
Business Logic	Java	Math	Equation Generator	n/a
	Java	Reports	Report Logic and Interface Generator	n/a
External Libraries*	C++	DLL	DLL Wrapper Generator	n/a
	Scripting Languages	C++	External Language Wrapper Generator	n/a
Configuration*	Firewall	any	Firewall Configuration Generator	n/a
	Reference Data*	any	Lookup Function Generator	n/a
		any	Macro Lookup Table Generator	n/a

*I've reclassified these items from the original table in the book.
NB: I generated the table above using an OpenOffice Calc spreadsheet [generator-taxonomy.ods]. Good example of dog-fooding perhaps;-) I've always found Excel/Calc to be really useful tools for rapidly generating scripts, SQL or HTML fragments like this.

With so much capability in JDeveloper, you might expect instant adoption. But there are of course two classic issues still to be addressed:

1. There's quite a steep learning curve until you get comfortable with what features are available, and when they are appropriate to use. The rapid enhancement of JDeveloper and all the technology over the past few years means that your average dev shop probably doesn't have the luxury of many gurus who can coach the team either
   
2. It seems to be an inherent trait of developers (arguably a good one), that generators and wizards are not fully trusted and accepted until you've had the chance to dig down under the covers. That's simply a lot of work for all of JDeveloper's features, and there isn't a great deal of information available to help your short-cut the process.

The natural conclusion is that to optimise adoption, I'd exhort the JDev team (and the community of users) to put as much effort as possible into producing resources for JDeveloper's code generation (wizards etc) and extension SDK. Importantly, to put the stress on explaining the how and why these things work (not just step-by-step tutorials).

Postscript:

• Shay Shmeltzer just posted a useful summary of How Do I Start Learning JDeveloper and ADF?


• Olaf Heimburger has started a new blog series demonstrating ADF In Action
• Eduardo Rodrigues has a great post on his favourite JDeveloper extensions with some good recommendations of what you should install or at least have a closer look at.

2007 In Review

Picking up the idea to present "my blog-year in review", by taking just the first complete sentence each month's post (remembering not to make this the first post for Jan;-)

January	Apache Axis2 is the core engine for Web services.
February	I've been working a bit with Oracle's OCCI (the C++ API for Oracle Database), and stateless connection pools in particular.
March	Recently wanted to track down the details of the process that had a specific port open.
April	"Where's the alert log?" .. usually the first thing you want to know when looking at a new database.
May	Say you have a connection to a remote Oracle Database server and want to get the ORACLE_HOME setting.
June	A little while ago I got to dust off my C++ skills for a project that was to use Oracle Database (via OCCI) and also Websphere MQ
July	This week its my pleasure to host Log Buffer #54, the Carnival of the Vanities for DBAs.
August	I've just spent the past few days at Oracle OpenWorld Asia Pacific in Shanghai.
September	I've blogged previously about playing with the reCAPTCHA service in Perl.
October	I had the privilege of listening to Kevin Walsh's "30/30" presentation last week.
November	Well, today is my first as a member of the Oracle Alumni
December	Laurent Schneider just asked what LinkedIn is good for?.

Hmm, very geek. What's not listed is the post that generated the swiftest, strongest reaction: Ballmer Peak and the Programmers' Paradox, which probably goes to show that frat humour is the surest way to internet fame;-)

During the year I also started blogging quick reviews of all the books I read over at PrataLife. So as a variation on the meme, here's my Book-Year In Review:

July	The Extraordinary Adventures of Arsène Lupin, Gentleman-Burglar (audiobook)
August	The Ambler Warning
September	A Different Point of View (audiobook)
October	The Warden (audiobook)
November	Code Generation In Action
December	Quarter Share (audiobook)

The number of audiobooks is a bit misleading, as I'm sure I actually read at least as many on paper (although Neal Stephenson is slowing me down a bit at the moment!). And ye gads, only one tech book! But this is to be expected: with fiction its easy to read and do a quick blog entry, whereas technical non-fiction is generally not suited to blogging an instantaneous reaction. They'll tend to stay on my desk/shelf for a probationary period;-)

Backup your WebCenter Wiki. Yikes!

Previously I wrote about installing Oracle WebCenter Wiki, and also had my 2c worth.

One of the surprises is that it uses file storage by default. Database storage seems to be a work-in-progress (Toplink, hibernate and HSQLDB are all involved somehow), but not a documented option at this point.

Now if you are going to run WebCenter Wiki for real, that means as a dilligent system admin you want to be pretty sure you really understand how the file storage is working so you can have good backup and recovery procedures in place.

The documentation on Backing Up and Restoring Wiki Content sounds straight-forward. Just make a backup of the OC4J_HOME/applications/application_name/owc_wiki/pages folder.

But BEWARE!

Gotcha #1: Don't Forget Your Toothbrush!
A little poking around, and it is clear that this is NOT all you need to backup. I've noted content modifications in at least the following locations also:

OC4J_HOME/applications/application_name/owc_wiki/attachments [any uploaded attachments]
OC4J_HOME/applications/application_name/owc_wiki/templates [added/changed page templates]

The jzwiki backup/restore instructions also tell you to backup WEB-INF/classes/yawikiDB.script (WebCenter wiki is based on jzwiki)

So to be sure you haven't left anything behind, I'd suggest you probably want to be backing up the entire OC4J_HOME/applications/application_name/owc_wiki directory structure no matter what the doc says, and practice some restores!

Gotcha #2: Beware of EAR Re-deployments!
If you've worked with OracleAS/OC4J deployment before, and also noticed that the wiki content is stored in the application directory structure, I hope it rang an alarm!

Yes, if you touch or replace the owc_wiki.ear, then say goodbye to your wiki content and pray your backup is good and not too old! OC4J will redeploy the application and give you a nice fresh install, deleting all your wiki content and customisations;-)

I have always had a sneaking suspicion that there can be other circumstances in which OracleAS/OC4J may choose to redeploy an EAR. I can't really substantiate this, but I have had situations in production where EARs are redeployed for no immediately apparent reason.

This is of course very hairy for any non-trivial production deployment. The obvious first step would be to configure wiki content storage to some other location. This isn't documented however, and I'm not sure if its even possible with the product as it is.

So for now, it seems just backup or rsync deep and often, and enjoy the view as you walk the edge of the abyss;-)

WebCenter Wiki Future?
So far, there's no real news on what is planned for WebCenter Wiki in 11g. If you've read my posts, it will come as no surprise that my view is the 10g wiki is far from production-ready.

Which makes the question of what is in store for 11g so interesting. Maybe a significant revamp of the 10g (jzwiki) wiki? Maybe an internally developed replacement based on Content DB or UCM?

Or maybe something completely different? A collegue pointed me to Jive Clearspace today .. "discussions, blogs, files, instant messaging, and wiki documents under one unified umbrella" .. and since Oracle already have Jive Forums integrated in WebCenter it raises interesting possibilities!

Ringing in the New Year with a timely reminder from xkcd

I've refrained from reposting xkcd for a while, but this is just too good! So where did you leave your key-ring last night, huh?


Happy New Year!