Sunday, May 25, 2008

Config Files for Windows - an Enhancement

In a previous post, I described a general scheme for managing config files for Windows.

Well, there's a simple enhancement that I've added concerning how to locate the config file for the script in question.

The issue is that if I run a script, you always wanted to look for the config file relative to the script location. This can be easily done using Windows Batch Parameter Expansion. %~dp0 expands to the path of the current script, and %~n0 is the name of the script file. Hence, to name the appropriate config file for a script, we generate the name as follows:

set LOCALCONF=%~dp0conf\%~n0-%COMPUTERNAME%.cmd

Blog Stats

I want a t-shirt just like this for my birthday, except I'd change it to "..than MY blog" and add my url;-).



Originally posted on Prata Life..

Business Networking that Doesn't Suck

I really like Geoffrey Grosenbach's ruby on rails podcast, especially because it's not just about rails (even though Rails is a keen interest of mine).

On the latest issue, I was excited to learn about biznik.com. "Business Networking that Doesn't Suck" as they say.

I immediately subscribed, but was disappointed to find that there's nothing happening in my local area yet (Singapore), at least not within 6000km.


Something that we should change soon, especially given the attention paid to entrepreneurship in Singapore. Take the efforts of SPRING for example, or the investment that the government is making in Interactive Digital Media.

I love the idea of this site, and hope to find some local compatriots up there soon!

Saturday, May 24, 2008

Monday, May 19, 2008

Tu Plang, Unit and the Green Papaya


I've had Regurgitator's Tu-Plang and Unit since when I was working in Sydney around 98/99. These albums are classics. I love their sound, and the lyrics are smart. They still get a regular listen, and that's not going to stop.

In a twist, I saw recently on the Food Lover's Guide to Australia that Quan's mother established a Vietnamese restaurant in Brisbane called Green Papaya, and it has great reviews.


It was Hanoi-born Lien Yeomans's dream come true - opening her own restaurant. At Brisbane's Green Papaya she cooks dishes from North Vietnam. She says she has her rock star son Quan to thank for her fame (he’s with the band Regurgitator) but we think it’s her recipes, each one linked to a personal story from her incredible life.



So next time I'm in Brisbane...


View Larger Map

Sunday, May 18, 2008

Calendar


Day to day, our time is marked by weeks, months and years. We take it for granted that no matter where you are on earth, you always know where you are in time.

The calendar is such a routine concept that it takes a book like David Ewing Duncan's excellent and most engaging Calendar to make you stop and think.

A 7-day week is completely arbitrary for instance, apparently originating circa 700 BC in Babylon, and probably influenced by the seven known planets (at the time).

Months are confusion personified. Inspired by the lunar cycle, one is left with the dilemma of trying to fit the cycles within the solar year. The chart below (produced with the open source lcap program) is probably the easiest way to see how the lunar months drift in relation to the solar year. Some, like the Greek astronomer Metron, attempt to rationalise the relationship with great precision (the 19-year Metronic cycle has 7 years of 13 lunar months followed by 12 years of 12 lunar months. Even that is not completely accurate). To most calendarists however, months scarcely retained a notional relationship to the moon, becoming little more than convenient units of time upon which to hang the names of Gods and Emperors.





NB: the difficulty in tracking the moon also came up in relation to the challenge of measuring longitude. Accurate moon charts would have allowed mariners to determine their position even in the absence of accurate time-keeping devices. Predicting the moon's course in the end proved to be much more challenging than building a clock that could survive a nautical adventure and still keep good time. See Dava Sobel's Longitude.

The idea of the solar year - the complete cycle of the seasons from one solstice to the next - is fundamental to how we conceive the passage of time. But we don't often think about the difference between a sidereal year (time for the earth to complete one complete orbit of the sun), and the tropical year (time from one vernal [spring/March] equinox to the next). Or the fact that the year is so very slightly different if you measure between the June or December solstices or the September equinox. And then consider that the Earth is a huge object hurtling through space and subject to varying gravitational forces and other bumps and jitters.

Once you have boiled all this down, it seems a "day" is about the only fundamental unit of calendar time that makes sense. Everything else is just convention and approximation!

Considering the difficulty of measurement, and the many social, political and religious factors involved in common convention, it is quite extra-ordinary that we even have a universally accepted calendar today.

It is easy to forget just how recent a phenomenon this is, and how long its introduction was in the making.

The introduction of the Gegorian calendar required 11 days to be lost for ever!The Gregorian calendar we use today was officially launched by Pope Gregory in 1582, but it wasn't introduced in Britain until 1752 with many countries not adopting it until the 20th century (for example, Russia in 1918 and Greece in 1924). Many quote Mao Zedong declaration of the People's Republic of China in October 1949, which included a point on the adoption of the Gregorian calendar, as the final triumph for a universal calendar (although many cultures still maintain parallel systems such as the Chinese lunar calendar). But do not forget that the flaws in Julius Ceasar's previous calendar had been know long before - the cause of much consternation at the Council at Nicaea in AD325 over the scheduling of Easter, and the subject of the monk Roger Bacon's most distressed entreaty to Pope Clement in 1266 to:
.. apply excellent remedies in this particular .. If then this glorious work should be performed in your Holiness' time, one of the greatest, best, and finest things ever attempted in the Church of God would be consummated.

All of this is of course the background to an epic tale that finds a thread through all peoples over all time.

Which is really the genius of David Duncan's work. He is a great explainer, and in telling the story of the calendar, he tells the story of much of human civilization. And where other "histories of the world" tend to struggle to fit a narrative thread across time and place, the story of the calendar provides Duncan with a perfect segue. The result is the most enjoyable, understandable and effective history lesson I have ever had.

If you have just the slightest interset in history or how the calendar came about as we know it, I couldn't recommend this book more highly.

Other interesting facts:

  • In Britain, New Years Day was celebrated on March 25, the Feast of the Assumption, until it was changed to January 1 in 1752. There are over 30 definitions of New Year listed on wikipedia.

  • Until the modern era of atomic clocks, the most accurate measurement of the year was by Abu Allah Mohammed Ibn Jabir al-Battani in AD 882 (22 seconds short). al-Battani's work came from the same period of Arab innovation that produced modern algebra and added "zero" to the nine Indian symbols that eventually became our familiar number system.


Calendar for December from a French Book of Hours, written around 1470:
Calendar for December from a French Book of Hours, written around 1470
Easter table, England, 1073:
Easter table, England, 1073
Originally posted on Pratalife

Tuesday, May 06, 2008

Configuration Files - a pattern for windows batch scripts

It's a very common practice in perl and linux/unix shell scripting to use an external configuration file (.conf or .properties), keeping all the script's necessary settings organised tidily in one place. This makes it easier to configure in the first place (because you don't have to hunt through the whole script to find what needs tweaking, or which environment variables need to be set), and also makes maintenance of the script so much easier (because you don't need to reconfigure after installing an updated script).

For some reason though, this practice never seemed to get widely adopted in the Windows batch file world. Either Windows script-kiddies just never learned to think like a sysadmin, or we never expected the DOS shell to be capable enough! Or maybe we never bothered, because surely windows batch files should be extinct by now?

I guess the dominant approaches to configuring batch files on Windows are:

  • Command line arguments [painful if you have too many, annoying if they are invariant for your setup, and may entice you to write a wrapper script just to save all the typing]

  • Environment variables [can lead to pollution of your environment, but also not usually backed up along with your scripts]

  • Good ol' "just edit the batch file and fill in the settings" approach


Well, it is possible to take advantage of the elegance of external configuration files. For some time I've been using a technique that I adapted from what I'd normally do with perl and bash scripts. I'm sure this was invented decades ago, but it surprises me that I haven't yet found a good example or tutorial on the net.

So I thought maybe time to polish up the "pattern" and share it here.

The Pattern

It's pretty simple! The flowchart on the right provides an overview.

To make this work with simple windows batch files, we use a configuration file that it itself a batch file (.bat or .cmd) and it is called from the main script. The configuration file would usually set a range of environment variables

A nice usability touch is that when the script is first executed (i.e. no configuration file present), it will generate a default configuration file, give the user some helpful instructions on how to edit the configuration file, and exit.

This makes your script "safe" for users who are liable to run it just to find out what it does(!).

But it also means that you can maintain the default configuration file structure - with as much built-in documentation as you like - inside the script itself (therefore only one file to maintain).

The Script Template

Here's a script template that includes the configuration file handling (also available for download here). As you can see, not rocket science!

If you have multiple scripts that all need to share a common configuration, you can extract the config file handling as a separate script and include (call) that from all the others. See the scriptTemplate.cmd and config.cmd in this kit of examples.
@echo off
REM $Id$

setlocal

echo Welcome to %0

REM Initialise the config flag [CONFIGSET] and config file [LOCALCONF]
REM You must name the config file .bat or .cmd to ensure Windows can execute it cleanly
set CONFIGSET=NO
set LOCALCONF=%~dp0conf\%~n0-%COMPUTERNAME%.cmd

REM ===============================================================
REM Configuration section
REM ---------------------------------------------------------------
if "%LOCALCONF%"=="" goto config_help
goto config_do


:config_help
echo This is a configuration help script
echo Call from another script with first parameter being the config file name
echo This script will set the variable CONFIGSET
echo CONFIGSET=NO in the case of error or undefined configuration
echo CONFIGSET=YES in the case where configuration has been successfully read
goto config_exit


:config_do
REM handle configuration file
IF EXIST %LOCALCONF% goto config_cont

REM generate default setting file
REM adapt this to you needs. Here are some samples
echo REM configuration file> %LOCALCONF%
echo set JAVA_HOME=C:\bin\jdk1.6.0_03>> %LOCALCONF%
echo set TMPFILE=c:\temp\mytemp.txt>> %LOCALCONF%
echo set SUBJECT=A subject line>> %LOCALCONF%
echo set DBUID=dbusername>> %LOCALCONF%
echo set DBPWD=dbpassword>> %LOCALCONF%

echo #
echo # Local configuration not yet set.
echo # A default configuration file (%LOCALCONF%) has been created.
echo # Review and edit this file, then run this process again.
echo #
goto config_exit


:config_cont
call %LOCALCONF%
set CONFIGSET=YES


:config_exit
if "%CONFIGSET%"=="YES" goto config_ok
echo Configuration is not set
goto exit
:config_ok
REM ---------------------------------------------------------------
REM Configuration section ends
REM ===============================================================


echo The main script starts from here.

echo The following configuration is set:
echo JAVA_HOME=%JAVA_HOME%
echo TMPFILE=%TMPFILE%
echo SUBJECT=%SUBJECT%
echo DBUID=%DBUID%
echo DBPWD=%DBPWD%


:exit
endlocal


Sample Output

Here's an example of running the script above. The first time through, no configuration file is detected, so it creates one and exits.

At that point, the user could edit the configuration file if necessary.

Subsequent runs pick up the settings in the configuration file.

D:\MyDocs>scriptWithConfigTemplate.cmd
#
# Local configuration not yet set.
# A default configuration file (scriptWithConfigTemplate-conf.bat) has been created.
# Review and edit this file, then run this process again.
#
Configuration is not set

D:\MyDocs>scriptWithConfigTemplate.cmd
The main script starts from here.
The following configuration is set:
JAVA_HOME=C:\bin\jdk1.6.0_03
TMPFILE=c:\temp\mytemp.txt
SUBJECT=A subject line
DBUID=dbusername
DBPWD=dbpassword

D:\MyDocs>

Sunday, May 04, 2008

Complex ACL filters with OVD

Access Control Rules are a very important aspect to get right when setting up Oracle Virtual Directory. But at the same time, I guess they are probably the most complex configuration option, especially if you haven't had some hard core LDAP experience before.

In Using OVD Filtered Directories for LDAP Authentication, I talked about using ACLs to restrict access to certain parts of a directory: each Access Control Rule has a filter setting, which is an LDAP filter that specifies which directory entries the rule applies to.

However I also discovered that using complex (multiple term) Access Control Rule filters is a great way to kill the OVD server process.
NB:it bombs with a java.lang.NullPointerException, and it is necessary to go and manually remove the offending filter from the $OVD_HOME/conf/acls.os_xml file on the server before it will startup again.


Say, for example, you wanted to grant access to entries for the ITGroup and HR departments. In RFC 4515 LDAP search filter terms, we can write this simply as:
(|(departmentNumber=ITGroup)(departmentNumber=HR))



But as mentioned, this will kill your OVD server (OVD 10.1.4.0.1 build 06.07.19).

There is a simple work-around though, by reverting to simple set theory and write multiple rules. In other words, granting (|(departmentNumber=ITGroup)(departmentNumber=HR)) is equivalent to two rules applied in sequence:
  • Grant: (departmentNumber=ITGroup)
  • Grant: (departmentNumber=HR)