2008-10-20 21:59:59 by abowman in Ravings of a Lunatic (3 comments) permalink
I’ve been teaching myself Ruby for a few years now. I first got interested in it when I was looking for something to do admin scripts for a project running on Linux. I coded the first version in Perl but was not satisfied with the error trapping and object orientation in Perl.
Ruby proved to be a pleasant change. It took me a while to get used to it–and I still am at times–but within a few weeks I had migrated all of the project’s code from Perl to Ruby. An unfortunate side effect of learning on the fly is that the Ruby code I wrote looks very much like the Perl code it replaced. I didn’t deviate from the existing structure and the logic. I believe now that it was a mistake to not go fully Ruby and use the best idioms available. I used some gimmies like iteration with “each.” It was too tasty to pass up. But I didn’t go fully object oriented and I didn’t use some of the advanced syntactic sugar that Ruby dishes up.
In the end the code works, but changing it and extending it involves all of the pain from the original Perl code.
Today I pushed a few more boundaries in Ruby. I started playing with the SOAP4R library. I’ve been working on a web service that one of my coworkers wrote for an emergency project. He churned it out in no time and it runs to spec. We’re having some issues with it though. It’s not the logic of the service, it’s the performance. At the beginning of our testing they gave us data that hauled a huge number of records back. We were very anxious because the service ran 1.5 to 3 minutes. We did some tweaks on the application server and sped it up some, but it was still far too slow. When they gave us better (read “more believable”) data the execution times dropped down into the acceptable range.
Everything was looking rosy until QA got involved. Their tests showed a rapid spike in execution times under load. The service pushed back up into the 30 second to 1.5 minute execution range.
This service really isn’t like the usual services we write. We usually write data access services and business orchestrations. This one is a data search service, but it has to perform it’s search across 37 versions of the same database. Each search is fast, but done serially it would take too long. So we set the service up to do the searches in parallel. The total search time is only as long as the longest search.
In theory this is great. Fast! But in practice doing the same search 37 times isn’t that great. When you do one process instance at a time its fast, but when you do 20 in one minute–like when you load test–you get around 700 searches. That’s a bit slower.
So the question is, what’s slow? Is it the code, the application server? The VM? The OS? The remote services? I think it’s the remote services, but to prove it I have to execute the same pattern of searches without the application server. That’ll give me a good comparison.
So I sat down this afternoon and started to cobble together a Ruby script to execute the web service in the same 37 by 20 invocations per minute fashion as the orchestration does. I used the SOAP4R library and generated the interface files for the service with wsdl2ruby.rb. Two hours later I was still struggling to get the damn auto generated client to work. I had to solve a problem relating to included libraries. That took a while. And then I had to figure out how to do complex web services in Ruby.
I have a complaint. It’s not just Ruby, but other languages too. There are plenty examples on how to invoke service, but all of them are trivial services that take one or two inputs and fire back a flat data structure. That’s not what I deal with. I deal with highly nested structures with anywhere from a dozen fields to hundreds and with repeating elements. Most of the input messages I deal with are more complex that the output messages in all of the examples.
So it’s very frustrating to try any kind of new SOAP library because the documentation usually sucks.
More later…