Sunday, February 11, 2007

Complex SOAP::Lite requests - my rules for SOAP::Sanity!

Previously, I mentioned I'd come back to more complex request and response structures with SOAP::Lite.

Frankly, I haven't posted because I can't avoid the feeling that there's still a little more to unravel. Did I mention that good documentation is sparse? ;) Byrne and others have posted some good stuff (see for example the majordojo soaplite archive and this hands-on tour at builder.com), but mostly you'll find the experiences shared go along lines like "...after hacking around for a bit, I found that this worked...".

But is it possible to try and pin down at least a couple of guidelines for using SOAP::Data? Well, so far I can't claim to solving it all, but I am able to share a few anchors I've tried to plant for my own sanity!

My Rules for SOAP::Sanity


In the following "rules", $soap is a pre-initialised SOAP::Lite object, as in:
my $soap = SOAP::Lite->uri ( $serviceNs ) -> proxy ( $serviceUrl );

1. The value of a SOAP::Data element becomes the content of the XML entity.


It may seem bleeding obvious. Nevertheless, get this idea fixed in you head and it will help for more complex structures.

So if we are calling a "getHoroscope" service with the following request structure:
<getHoroscope>
<sign>Aries</sign>
</getHoroscope>

"Aries" is the value, i.e. the content, of the XML entity called "sign". Thus our request will look like this:
$data = SOAP::Data->name("sign" => 'Aries');
$som = $soap->getHoroscope( $data );

2. To create a hiearchy of entities, use references to a SOAP::Data structure.


In (1), the content of the entity was a simple string ("Aries"). Here we consider the case where we need the content to encapsulate more XML elements rather than just a string. For example a request with this structure:
<getHoroscope>
<astrology>
<sign>Aries</sign>
</astrology>
</getHoroscope>

Here "astrology" has an XML child element rather than a string value.
To achieve this, we set the value of the "astrology" element as a reference to the "sign" SOAP::Data object:
$data = SOAP::Data->name("astrology" =>
\SOAP::Data->name("sign" => 'Aries')
);
$som = $soap->getHoroscope( $data );


3. To handle multiple child entities, encapsuate as reference to a SOAP::Data collection.


In this case, we need our "astrology" element to have multiple children, for example:
<getHoroscope>
<astrology>
<sign>Aries</sign>
<sign>Pisces</sign>
</astrology>
</getHoroscope>

So a simple variation on (2). To achieve this, we collect the "Aries" and "Pisces" elements as a collection within an anonymous SOAP::Data object. We pass a reference to this object as the value of the "astrology" item.
$data = SOAP::Data->name("astrology" => 
\SOAP::Data->value(
SOAP::Data->name("sign" => 'Aries'),
SOAP::Data->name("sign" => 'Pisces')
)
);
$som = $soap->getHoroscope( $data );

4. Clearly distinguish method name structures from data.


This is perhaps just a style and clarity consideration. In the examples above, the method has been implicitly dispatched ("getHoroscope").
If you prefer (or need) to pass the method information to a SOAP::Lite call, I like to keep the method information distinct from the method data.

So for example, item (3) can be re-written (including some additional namespace handling) as:
$data = SOAP::Data->name("astrology" => 
\SOAP::Data->value(
SOAP::Data->name("sign" => 'Aries'),
SOAP::Data->name("sign" => 'Pisces')
)
);
$som = $soap->call(
SOAP::Data->name('x:getHoroscope')->attr({'xmlns:x' => $serviceNs})
=> $data
);

I prefer to read this than have it all mangled together.

That brings me to the end of my list of rules! I am by no means confident that there aren't more useful guidelines to be added, or that in fact the ones I have proposed above will even stand the test of time.

Nevertheless, with these four ideas clearly in mind, I find I have a fair chance of sitting down to write a complex SOAP::Lite call correctly the first time, rather than the trial and error approach I used to be very familiar with!

33 comments:

kernelkole said...

I'd like to post a comment (I need some advice), but the Blogger doesn't like something - either the XML or the Perl - I'm including.

Is there a way to "escape" some charachters, or to attach the XML protions to the post?

Paul said...

Hi! To post code you probably need to HTML-encode it first (for example, using http://www.string-functions.com/htmlencode.aspx)

kernelkole said...

The SOAP message that I want to end up with is something like this:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/
">
<SOAP-ENV:Body
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<ts:Transform>
<source xsi:type="ts:IOSpec">
<spec xsi:type="ts:stringData">
<str xsi:type="xsd:string">c:\Microsoft_word.doc</str>
<charset xsi:type="ts:CharacterSetEnum"> windows-1252</charset>
<base64 xsi:type="xsd:boolean">false</base64>
</spec>
<specType xsi:type="xsd:string">path</specType>
</source>
<sink xsi:type="ts:IOSpec">
<spec xsi:type="ts:stringData">
<str xsi:type="xsd:string"></str>
<charset xsi:type="ts:CharacterSetEnum">ISO-8859-1</charset>
<base64 xsi:type="xsd:boolean">false</base64>
</spec>
<specType xsi:type="xsd:string"></specType>
</sink>
<outputFormat xsi:type="xsd:string">search-text</outputFormat>
<optionSet xsi:type="xsd:string"></optionSet>
<options xsi:type="SOAP-ENC:Array" SOAP-ENC:arrayType="ts:Option[0]">
</options>
</ts:Transform>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

So far, I have not been able to create this message, using your examples. Concentrating on the <sink> element, the child

<specType> occcurs outside the element.

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" SOAP-

ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema">
<SOAP-ENV:Body><namesp1:Transform xmlns:namesp1="http://www.outsideinsdk.com/transformation_server/transform/1/0/">
<source xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/" xsi:type="ts:IOSpec">
<spec xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/" xsi:type="ts:stringData">
<str xsi:type="xsd:string">C:\Simple document.doc</str>
<charType xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/"

xsi:type="ts:CharacterSetEnum">windows-1252</charType>
<base64 xsi:type="xsd:boolean">false</base64>
</spec>
</source>
<specType xsi:type="xsd:string">path</specType>
...

Here is my code:

my $inputiospec = SOAP::Data
->attr({'xmlns:ts' => $serviceNs})
->type("ts:IOSpec")
->value(
\SOAP::Data->name("spec" => $inputstringdata),
SOAP::Data->name("specType" => 'path')
)
;

my $inputstringdata = SOAP::Data
->attr({'xmlns:ts' => $serviceNs})
->type("ts:stringData")
->name("spec" =>
\SOAP::Data->value(
SOAP::Data->name("str" => $inputfile),
SOAP::Data->name("charType" => 'windows-1252')
->attr({'xmlns:ts' => $serviceNs})
->type("ts:CharacterSetEnum"),
SOAP::Data->name("base64" => 'false')
->type("xsd:boolean")
)
)
;

But the result is:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" SOAP-

ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema">
<SOAP-ENV:Body><namesp1:Transform xmlns:namesp1="http://www.outsideinsdk.com/transformation_server/transform/1/0/">
<source xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/" xsi:type="ts:IOSpec">
<spec xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/" xsi:type="ts:stringData">
<str xsi:type="xsd:string">C:\Simple document.doc</str>
<charType xmlns:ts="http://www.outsideinsdk.com/transformation_server/transform/1/0/" xsi:type="ts:CharacterSetEnum">windows

-1252</charType>
<base64 xsi:type="xsd:boolean">false</base64>
</spec>
</source>
<specType xsi:type="xsd:string">path</specType>
...

kernelkole said...

one more problem: I need a data type that looks like:
<options xsi:nil="true" xsi:type="ns1:ArrayOfOption"/>

when I code this:

my $options = SOAP::Data
->name("options" => "")
->attr({'xmlns:ts' => $serviceNs})
->type("ts:ArrayOfOption")
->attr({'xsi:nil' => "true"})
;

I get this:

<options xsi:nil="true" xsi:type="ts:ArrayOfOption"/>

..where my namespace "ts" is not qualified. seems like I can't have two attr vaues in one Data

Paul said...

Hi there, let me try that again.
On your question regarding the specType appearing outside of the element, you can fix by using "rule 3" - wrap all of the source/sink items in an anonymous SOAP::Data element.
On the options question, think you can avoid this issue.

Here's an "almost-complete" suggestion, which covers most the things you need. helps?

my $sourcespec = SOAP::Data
->type("ts:IOSpec")
->name('source' =>
\SOAP::Data->value(
SOAP::Data->name("spec" =>
\SOAP::Data->value(
SOAP::Data->name("str" => 'c:\Microsoft_word.doc'),
SOAP::Data->name("charset" => 'windows-1252s')->type("ts:CharacterSetEnum"),
SOAP::Data->name("base64" => 'false')->type("xsd:boolean")
)
),
SOAP::Data->name("specType" => 'path')
)
);

my $sinkspec = SOAP::Data
->type("ts:IOSpec")
->name('sink' =>
\SOAP::Data->value(
SOAP::Data->name("spec" =>
\SOAP::Data->value(
SOAP::Data->name("str" => ''),
SOAP::Data->name("charset" => 'ISO-8859-1')->type("ts:CharacterSetEnum"),
SOAP::Data->name("base64" => 'false')->type("xsd:boolean")
)
),
SOAP::Data->name("specType" => '')
)
);

my $som = SOAP::Lite
->proxy( 'http://localhost:8000/blah/DummyService' )
->call(
SOAP::Data
->name('ts:Transform')
->attr({'xmlns:ts' => 'http://www.outsideinsdk.com/transformation_server/transform/1/0/'})
=> $sourcespec,
=> $sinkspec,
=> SOAP::Data->name('outputFormat' => 'search-text')
=> SOAP::Data->name('optionSet' => '')
=> SOAP::Data->name('options' => '')->type('SOAP-ENC:Array')
);

Anonymous said...

Can someone help me build this object:

<xsd:complexType name="UserData">
<xsd:sequence>
<xsd:element minOccurs="0" name="password" nillable="true" type="xsd:string"/>
<xsd:element minOccurs="0" name="user" nillable="true" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>

Paul said...

The XSD ComplexType is in effect defining a structure like this:

<UserData>
<password xsi:type="xsd:string"/>
<user xsi:type="xsd:string"/>
</UserData>

So, its actually quite straightforward to produce (rule #3), for example..

my $soap = SOAP::Lite
->uri ( $serviceNs )
->proxy ( $serviceProxy );

# define user data
my $UserData = SOAP::Data
->name('UserData' =>
\SOAP::Data->value(
SOAP::Data->name("password" => 'my-pwd'),
SOAP::Data->name("user" => 'my-uid')
)
);

my $som = $soap->call(
SOAP::Data
->name('thisfunction')
=> $UserData
);

Chris said...

Hi Paul- Thank you so much for your help.

I am now very close. I think that user and password have to be attached to a particular namespace. This xml below works in Soap UI. I just need to convert this to a SOAP::Lite friendly format. Any ideas? TIA -CK

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ser="http://service.openreports.efs.org" xmlns:sab="sabrixreports" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<sab:getReportGroups>
<sab:in0>
<ser:password>asdfkjaslfj</ser:password>
<ser:user>admin</ser:user>
</sab:in0>
</sab:getReportGroups>
</soap:Body>
</soap:Envelope>

Paul said...

Hi Chris,

Try something like this...

my $soap = SOAP::Lite
->proxy( 'http://localhost/blah/DummyService' );

my $serializer = $soap->serializer();
$serializer->register_ns( 'http://service.openreports.efs.org', 'ser' );
$serializer->register_ns( 'sabrixreports', 'sab' );

# define user data
my $UserData = SOAP::Data
->name('in0' =>
\SOAP::Data->value(
SOAP::Data->name("password" => 'asdfkjaslfj')->prefix('ser'),
SOAP::Data->name("user" => 'admin')->prefix('ser')
)
)->prefix('sab');

my $som = $soap->call(
SOAP::Data
->name('sab:getReportGroups')
=> $UserData
);

Anonymous said...

I've been trying to get a script of mine to talk to a .NET server and am not having much luck. I'm new to SOAP and .NET so I've been doing a lot of fiddling and cutting/pasting. I can get the script to successfully run from my CLI (Win2K) but not from a *nix server, which is where it needs to be. Have you some experience with the .NET <-> SOAP interoperability?

Paul said...

I'm sorry but I haven't done anything specifically with .NET <-> SOAP::Lite.

But since it sounds like you have the Perl script running ok on one system but not another, I'd recommend the usual basics - particularly checking the Perl and SOAP::Lite versions you are using in each place.

Feel free to post a more complete description of the problem (with code and error messages) here or on the SOAP::Lite mailing list if still a problem..

Anonymous said...

What good astrology websitesdo you know?

Benny said...

possible to help me build this complexType "TrafficClassifer" ?

<xsd:complexType name="TrafficClassifier">
<xsd:sequence>
<xsd:element name="networkAddress" nillable="false"
type="impl:NetworkAddress">
</xsd:element>
</xsd:sequence>
</xsd:complexType>

<xsd:complexType name="NetworkAddress">
<xsd:sequence>
<xsd:element name="value" nillable="false" type="xsd:string">
</xsd:element>
<xsd:element name="type" nillable="false" type="impl:NetworkAddressType">
</xsd:element>
</xsd:sequence>
</xsd:complexType>

<xsd:simpleType name="NetworkAddressType">
<xsd:restriction base="xsd:string">
<xsd:enumeration value="IPV4">
</xsd:enumeration>
<xsd:enumeration value="IPV6">
</xsd:enumeration>
</xsd:restriction>
</xsd:simpleType>

Anonymous said...

Totally new to all of this, and reading with great interest in the hope to find enlightenment - how do you output the generated SOAP?


Cheers

The said...

Paul,

Sorry to barge in on you like this. I'm maintaining a piece of perl-code that so far only needed to grab information from a proprietary application and populate and LDAP directory with additions/modifications, which works quite well. Now I've been tasked to push some of that information into an identity management solution via JMS, wrapping SPML into soap over http. Your blog-posts are about as close as it gets to useful information about using perl & SOAP::Lite for the type of thing I think I need. Realistically I had no previous experience with SOAP or JMS, so am stabbing in the dark. Would you mind if I contacted you via e-Mail to try and understand a few basics that I can't gather from SOAP::Lite's docu?
It's quite terse.


Cheers

Paul said...

@benny: just saw your question about generating TrafficClassifier data. Try this for size:

use strict;
use SOAP::Lite +trace => 'debug';

my $soap = SOAP::Lite
->proxy( 'http://localhost/blah/DummyService' );


# define TrafficClassifier
my $TrafficClassifier =
SOAP::Data->name('TrafficClassifier'
=> \SOAP::Data->value(
SOAP::Data->name("value" => '127.0.0.1')->type('xsd:string'),
SOAP::Data->name("type" => 'IPV4')->type('impl:NetworkAddressType')
)
);

my $som = $soap->call(
SOAP::Data
->name('thisfunction')
->attr({'xmlns:tns' => 'http://TrafficClassifier/'})
=> $TrafficClassifier
);

exit;


The generated SOAP message:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/1999/XMLSchema"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Body>
<thisfunction xmlns:tns="http://TrafficClassifier/">
<TrafficClassifier>
<value xsi:type="xsd:string">127.0.0.1</value>
<type xsi:type="impl:NetworkAddressType">IPV4</type>
</TrafficClassifier></thisfunction>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Paul said...

@anonymous: "how do you output the generated SOAP?"

For debugging:

use SOAP::Lite +trace => 'debug';

This will dump to console all the details of the request and response messages.

There are also some other tricks for inspecting and dumping SOAP objects that I've played with in a little tester called debugSoapData.pl

Paul said...

@The Tink: sure, but no primoses! you can find my email contact on my blogger profile

Anonymous said...

I need some serious help here. I'm going crazzzzzzyyyyyyyy.

WSDL file says ...
<'complexType name="ArrayOfString">
<'sequence>
<'element maxOccurs="unbounded" name="string" nillable="true" type="xsd:string" />
<'/sequence>
<'/complexType>

How do I create an object that will satisfy this on the client side?

Anonymous said...

Paul,

wanted to thank you for the package, first of all. Very difficult endeavor!

I've a question. How do I encode this XML envelope:

<?xml version = "1.0" encoding = "UTF-8"?>
<inputMessage>
<ns0:Order_Status_Input_Root xmlns:ns0 = "http://www.tibco.com/schemas/MER_OTC_GetCustomerAccountInfo/SharedResources/Schemas/Schema.xsd">
<ns0:Order_Status_Input_Row>
<ns0:Customer_Id>92214</ns0:Customer_Id>
</ns0:Order_Status_Input_Row>
</ns0:Order_Status_Input_Root>
</inputMessage>


?

Anonymous said...

better yet, is there a tool that would convert a given XML payload to an appropriate perl data structure?

Anonymous said...

Very helpful post. Thank you. I was totally perplexed on how to force nested items and couldn't find any examples in the POD.

Maybe you can get this document into the distribution and/or the SOAP::Lite site?

Paul said...

Thanks for the comments anonymous. I'm amazed how much traffic this post still gets .. I even use the post to remind myself of how to do stuff with SOAPLite!

It is obviously a 'hot' topic, and the suggestion of getting this kind of information into the docs is a good one. I'll look into that..

adverick said...

Thank you, very helpful.

Paul said...

Thanks adverick

lstandish said...

Thanks very much for this very clear and useful information. I'm just learning Soap:Lite and this is the first documentation I found.

Paul said...

@lstandish, happy to hear it helped. Good luck with it!

Andre Foddrill said...

Hello. Great job. I did not expect this on a Wednesday. This is a great story. Thanks!

Theo K said...

And one more question (they never end, do they?!?)

How do I encode this envelope?

Many, many Thanks!

Muhammad Azeem said...

This is a nice article..
Its very easy to understand ..
And this article is using to learn something about it..

c#, dot.net, php tutorial

Thanks a lot..!

Anonymous said...

Nice article. 1 question:

How to escape a URI with something like this:

service endpoint:
http://blah.com/Search?text=

web service = Search

because when I run SOAP::Lite it will complain Error 404 web service cannot be found.

I check the debug the the URI will be concatenate up to http://blah.com/Search

Thanks

Anonymous said...

Thanks. good rules... helped me alot.

pisces said...

Dear Sir,

Your guide is very useful to a newbie like me!

I wish to know how to encode the following if I have some attributes in XML tag?



You have good luck today


Yellow is your lucky color today



Besides, how to i 'decode' the coding into XML format in order to check I have formed the XML correctly?