Handling range parameters in Bash

I've come across a few occasions where I wanted to specify a "range parameter" for Bash scripts. Like "1..4" meaning do for 1,2,3 and 4.

Here's a simple trick that uses the (relatively obscure) variable expansion and substring replacement capabilities of the shell.

#!/bin/bash
v_range="1..3"         # or you could have taken it as a script parameter
v_start=${v_range%%.*} # chomp everything from the left-most matching "."
v_end=${v_range##*.}   # chomp everything up to the right-most matching "."

The repeated %% and ## basically mean you will get a "greedy" match, so you can say "1..4" or "1....5"; it doesn't matter how many repeats you have of the range delimiter. Of course, you can choose other delimiters such as a hyphen, as in "5-10", if you wish.

Now that you have extracted the start and end indices, you can then loop or whatever to your hearts content:

for ((a=v_start; a <= v_end ; a++))
do
   echo "Looping with a=$a"
done

For more information on variable expansion, see 9.3 in the Advanced Bash Scripting Guide.

Postscript 5-Feb-2008:
We live and learn! Hat tip to buford for alerting me to the seq utility, which simplifies the iteration over a range, as in:

for a in `seq 1 10`
do
  echo "Looping with a=$a"
done 

You still need to determine the start and end values of the range, which is the whole point of the variable expansions approach posted here.

Generating a temp filename in Bash

Here's a function that simplifies the process of generating temporary filenames in shell scripts (I use bash). This function will generate a temporary filename given optional path and filename prefix parameters. If the file happens to already exist (a small chance), it recurses to try a new filename.

It doesn't cleanup temp files or anything fancy like that, but it does have the behaviour that if you never write to the temp file, it is not created. If you don't like that idea, touch the file before returning the name.

This procedure demonstrates the use of variable expansion modifiers to parse the parameters, and the $RANDOM builtin variable.

function gettmpfile()
{
  local tmppath=${1:-/tmp}
  local tmpfile=$2${2:+-}
  tmpfile=$tmppath/$tmpfile$RANDOM$RANDOM.tmp
  if [ -e $tmpfile ]
  then
    # if file already exists, recurse and try again
    tmpfile=$(gettmpfile $1 $2)
  fi
  echo $tmpfile
}

By default (with no parameters specified), this function will return a random filename in the /tmp directory:

[prompt]$ echo $(gettmpfile)
/tmp/324003570.tmp

If you specify a path and filename prefix, these are used to generate the name:

[prompt]$ echo $(gettmpfile /my_tmp_path app-tmpfile)
/my_tmp_path/app-tmpfile-276051579.tmp

Why we love Tom

I just picked up my copy of Effective Oracle by Design to check some facts. It just took a few moments for it to remind me how much I enjoy and learn from Tom Kyte's work.

Why do we love Tom? I have a theory on this! It comes down to two factors:

• Substance. In an industry overwhelmed by candyfloss powerpoints, its refreshing to read someone who lives in the details, yet manages to retain sparkling clarity. It's apple pie for the technically oriented: database administrators, developers, and perhaps even a few architects(!).
• Uncompromising rigour. Never one to let an assumption go unchallenged or untested. If there was an IT version of MythBusters, he would be your Jamie to Dvorak's Adam (or vice versa if you like). Highly opinionated, his opinions are (annoyingly) based on hard facts. Sometimes you just want to jump on the keyboard and hack away to try and prove him wrong ... But that's good! He's got you thinking.

Indexing with Oracle TDE

Oracle Transparent Data Encryption allows data encryption to be declared in a database schema, meaning that anything persisted on disk is protected from prying eyes.

It is a simple matter to setup TDE by bascially configuring a wallet location in sqlnet.ora, setting the key, and then opening the wallet after database startup. There's a good tutorial for this on OTN.

Declaring data encryption is then simply a matter of using the ENCRYPT keyword in your schema defintion, for example:

CREATE TABLE T1
(SEQ NUMBER(15),
CCNUMBER CHAR(16) ENCRYPT USING 'AES128' NO SALT);

Indexing encrypted columns is covered in the Advanced Security Guide. It mentions specifically that you cannot create an index on a column that has been encrypted with salt (hence the 'NO SALT' above). There is another restriction that you cannot use encrypted columns in functional indexes, but I've yet to find this covered explicitly in the doco. You may be surprised to find out that this also means you get caught if you try to create an index with descending values, such as:

CREATE INDEX T1_AK1 ON T1 (CCNUMBER, SEQ DESC);

This will fail with the error "ORA-28337: the specified index may not be defined on an encrypted column". The reason for this is that the use of "descending" will be treated as a functional index.

Removing the "descending" qualifier allows a valid index to be built:

CREATE INDEX T1_AK2 ON T1 (CCNUMBER, SEQ);

Exercising Regular Expressions and Arrays in Javascript

I've been working a bit with AJAX, and that soon prompted me to brush up on my Javascript. In the past I'd never used it to do more than a little validation or form "glue". Now revisiting the language from a true programmer's perspective, I recognise that it is pretty capable!

Speak Good Singlish! is a sample that gives regular expression and multi-dimensional array handling a good workout. It's a Singlish translator implemented in Javascript. I can't remember what sparked the idea to use this as an example, but somehow I got thinking back to ole "Jive" and "Valley girl" translators that have been floating around the net for many years.

The RegExp object usage in the sample is fairly standard, but the array handling is not.

I'm using a 2D array for the "lexicon" - basically a list of regex matches and replacements. For example the following fragment. You'll note that the second dimension can contain either a simple string or an array (as the replacement element):

var slexicon = [
[ '\\benglish\\b', 'Singlish' ] ,
[ '\\bdo\\b', [ 'do', 'do until sian', 'do sure can one' ] ]
];

There's a core routine that iterates through the array and performs the substitutions. Where an array substitution is present, it uses a funtion inline to pick a random element from the array to perform the substitution:

// do the translation
for (var i = 0; i < slexicon.length ; i++) {
var slexiconRow = slexicon[i];
var theRegex = new RegExp( slexiconRow[0], "gim" );
var theReplacement = slexiconRow[1];
if (theReplacement instanceof Array ) {
dataOut = dataOut.replace( theRegex, function (match) {return randomElement(theReplacement)} );
} else {
dataOut = dataOut.replace( theRegex, theReplacement );
}
}

The randomElement function is simplicity itself:

// returns random element of an array
var randomElement = new Function("x", "return x[Math.floor(Math.random() * (x.length))]");

The HTML page contains the whole script inline, so it is easy to review in-place with a "view source". Enjoy!

Simple Perl clients for Axis2 with SOAP::Lite

SOAP::Lite is an incredibly powerful little module that makes using Perl for SOAP clients a piece of cake (groan).

True to the spirit of Perl, it makes simple things simple. This is all you need to do to interrogate the "Version" Axis (1+2) service:

#!/usr/bin/perl -w
use SOAP::Lite;
print SOAP::Lite
-> proxy('http://localhost/axis2/services/Version')
-> getVersion()
-> result;

The output:

Hello I am Axis2 version service , My version is 1.1

The SOAP request that this generates is as basic as it gets, and as you can see most of it is dedicated to the envelope definition:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
xmlns:xsi=
"http://www.w3.org/1999/XMLSchema-instance"
xmlns:SOAP-ENC=
"http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV=
"http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd=
"http://www.w3.org/1999/XMLSchema"
SOAP-ENV:encodingStyle=
"http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Body><getVersion/></SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The example does illustrate one very important capability of SOAP::Lite - the automatic marshaling of the response into a usable Perl structure. In this case, its just a string.

One of the difficulties with SOAP::Lite at present is the dearth of good documentation and examples. This does tend to make doing more complex operations feel like you are living on the edge, but that's where the fun begins for some I guess;) I will look at some more complex request and response topics in a further post.

AXIS Revisited on OC4j - AXIS 2

Apache Axis2 is the core engine for Web services. It is a complete re-design and re-write of the widely used Apache Axis SOAP stack to build on the lessons learnt from Apache Axis. Version 1.1 was released in Nov'06. My first simple test was to update my deployment on Oracle OC4J.
Happily, it deployed just as easily as the original Axis, as I reported in a previous post.

Deployment of the Axis2 webapp is a little different, but just as straightforward. This is what I did:

1. Download the Standard Binary Distribution and explode into a local folder
2. In the $AXIS2_HOME/webapp directory, use ant to build the war file: ant create.war
3. This creates the $AXIS2_HOME/dist/axis2.war file
4. edit $ORACLE_HOME/j2ee/home/config/application.xml to add <web-module id="axis2" path="file:/d:/axis2/dist/axis2.war"/>
5. edit $ORACLE_HOME/j2ee/home/config/http-web-site.xml to add <web-app application="default" name="axis2" root="/axis2"/>

And that's it. Axis2 up and running in my OC4J home at http://localhost/axis2/