david sidlinger

Philip Carl was working on some XSL yesterday and noticed a fair amount that looked like this:

<table>
  <col…/>
  <col…/>
  <col…/>
  <x:apply-templates select="…" />
</table>

It was everywhere, and we all know to not repeat ourselves. But neither of us knew at the time how to wrap the execution of an XSL template in some boilerplate. Ladies and gentlemen, may I present the Wilson-Sidlinger Method†:

<x:template name="wrapWithTable">
  <x:param name="innards" />
  <table>
    <col…/>
    <col…/>
    <col…/>
    <x:copy-of select="$innards" />
  </table>
</x:template>


<x:template match="critter">
  …
  <x:call-template name="wrapWithTable">
    <x:with-param name="innards">
      <x:apply-templates select="kin" />
    </x:with-param>
  </x:call-template>
  …
</x:template>

<x:template match="you_uns">
  …
  <x:call-template name="wrapWithTable">
    <x:with-param name="innards">
      <x:apply-templates select="mama" />
      <x:apply-templates select="em" />
    </x:with-param>
  </x:call-template>
  …
</x:template>

We verified that execution time was not greatly affected. As a matter of fact, loading the stylesheet was a little faster with the reduced amount of code.

Is the apply-template being passed to the called template as a function or the result of a function? We may never know. The great thing about XSL, though, is it is a side-effect-free language that lets us care not one bit.

† This method of dealing with XSL repetition may have been used before, but we couldn’t find any mention of it.

Update: P. Carl corrected me. It’s copy-of in the “wrapWithTable” template.

Here’s another one for all you Greasemonkeys out there. The Snizzlator solves one of the world’s most pressing problems: How do I inject Snoop-speak into the web? I can’t take much credit for this one. One of the code monkeys I keep in a subterranean lair, P. Carl Wilson, recognized this need years ago and solved the problem with a JavaScript bookmarklet. My only contribution was to make it a Greasemonkey script. Like every good partnership, we’ve decided to split the fame, money, and groupies 50/50. You should definitely grab the script regularly. P to the C has found his calling in life and regularly updates the script. Those of you interested in what you can do with Greasemonkey should take a look under the hood. There’s all kinds of JavaScript goodies in there.

I posted another Greasemonkey script last night. This one puts a link to search Google Products on Amazon pages, based on the name of the item you’re viewing. It’s more complicated than the last one, but it didn’t take too much time to throw together. I realize that I’m pretty late to the Greasemonkey party, but I’m having fun, and isn’t that the point?

One thing that bugged the hell out of me when working on this script is that it seems like Amazon adds absolutely no semantic meaning to their product pages. Does the product name sit in an element with an ID clearly labeling it as such? No. Here’s the XPath I had to use to extract the name:

//div[@class='buying']/b[@class='sans']

Isn’t that cheesy? It feels really fragile. I suppose I could have parsed the page title to strip out extraneous information, but I wouldn’t feel much better.

Update:

Surprise, surprise! It is fragile. The different templates that Amazon uses based on product category (I think?) break the script. Oh, well, at least it doesn’t fail spectacularly.

Known good:

• Books
• Music
• Electronics

Known bad:

• DVD

I’ve created a Subversion repository to hold any Greasemonkey scripts I might come up with. The first one sets the focus to the search box when viewing Amazon pages. It will save me a bajillion tabs every time I hit the site. Is it worth talking about? Probably not. Am I disproportionately pleased with myself? Yes.

Over the years, I have been treated to a wide range of what I can only call perversions of technology. By far, the funniest I have seen is the Numbers Table. Why would you need a table of numbers, you ask? Because you need to count, of course.

The team of “developers” that came up with the Numbers table had heard that making a database call has overhead. Apparently, they had not heard about M.A. Jackson’s Rules of Optimization:

1. Don’t do it.
2. (for experts only) Don’t do it yet.

The system in question used pessimistic locking, another great story. There was a method to release multiple locks, which were stored in a database table. There was no way this performance critical code was going to call the LockRelease procedure for every lock. That would be insane! Presented here is the code that used the Numbers table. It’s reconstructed from memory, because I am too lazy to go digging through source control history. I have expanded a SQL function into the procedure definition and left out the irrelevant code.

First, the C# code:

public void ReleaseMultipleLocks(params Lock[] locks)
{
    StringBuilder builder = new StringBuilder();
    foreach(Lock lock in locks)
    {
        builder.Append(lock.LockId.ToString().PadLeft(11, "0"));
    }
    // Code to call the stored procedure with the (potentially) gigantic string
}

Now, the SQL:

CREATE PROCEDURE dbo.ReleaseMultipleLocks
(
    @LockIds nvarchar(4000)
)
AS

DECLARE @locks TABLE
(
    LockId bigint NOT NULL
)

INSERT
    @locks
SELECT
    CAST(SUBSTR(@LockIds, (11 * (Numbers.Number - 1)) + 1, 11) AS bigint)
FROM
    dbo.Numbers
WHERE
    Numbers.Number <= (LEN(@LockIds) / 11)
ORDER BY
    Numbers.Number

-- Use the @locks table to update the release datetime on
-- the specified lock records.  If I remember correctly, it 
-- used a cursor.

Wow.

So, in order to avoid those “expensive” database calls, we: build a string out of all those pretty, compact integers, pass that big string on the wire to the database server, create a table that holds (wait for it) integers, and use a huge table of numbers to do string parsing in SQL. Before you ask, this table was not used for anything else.

For your additional amusement:

• A long/bigint holds 19 digits, not 11.
• The Numbers table had hundreds of thousands of rows. You only need 364 numbers to parse the maximum string length that could be passed.
• There were mysterious gaps in the numbers table. None of them were in the range that would have affected this procedure, but…c’mon.
• The Numbers table was rebuilt every time the system started from hundreds of thousands of INSERT statements…in multiple databases.
• Why you need to use a double-byte character set for a string that holds only digits is beyond me.
• The team that came up with this was supposed to be our crack team of Enterprise Infrastructure Developers.

I’m posting this story and, soon, others I can bring myself to remember for a few reasons:

1. We have told them plenty of times to developers joining the team. Pointing them to a link will take a lot less time.
2. I want to remind myself how good I have it now, relatively speaking.
3. To, hopefully, stop myself from falling into the optimization trap.
4. Poking fun at this helps me get rid of the bad feelings associated with it.

Update: A co-worker corrected some of my code. SUBSTR is 1-based, and my WHERE clause was jacked up.

The latest on the reading list is Programming WCF Services from Juval Löwy. I’m having a real hard time maintaining interest in this book for a variety of reasons, but I’ll save the review for when I’m done.

I was only in the first paragraph of the preface when I hit a statement that sums up how I have been feeling about Microsoft development lately. In Juval’s own words:

Then, in July 2002, during a C# Strategic Design Review, the remoting program manager outlined in broad strokes plans to rework remoting into something that developers should actually use.

Now, Juval is not a Microsoft employee so much as a Microsoft shill; so, nothing he says can be taken as anything official from on high. However, I think that one statement has focused my frustration enough that I can articulate what has been bugging me. It shows a complete and utter disregard for the people that have been building their products on the .NET platform by Microsoft and their evangelists. This type of behavior can be seen in almost all facets of their development tools and technology stacks (e.g. distributed systems, data access, web applications.) They come out with yet another new solution to a problem only to turn around and rework it from the ground up and tell us what a piece of crap that last implementation was.

Outside of the standard disclaimers on alpha and beta software, I have never seen or heard Microsoft say that we shouldn’t be using their current technology stack. If remoting was so bad (I do think it was and still is,) why didn’t they caution us against using it? Why do they continually come out with shiny new toys that require the people driving their success to rewrite mountains of code?

The simplest answer is that most of what they put out does suck. While there have been some real “gems” thrown out there for people to risk their business on, I don’t think that completely explains it.

The second simplest answer is that Microsoft has shareholders and loads of salespeople and middlemen to appease. They cannot survive without a constant stream of revenue pouring in. They could accomplish that by being the hands-down best platform out there. Instead, they choose to feed the MSDN beast and get developers whipped into a frenzy about something that will require more upgrades, training, etc. Incremental, but substantial, improvements with a lot of thought put into maintaining compatibility with existing code do not keeping cash flowing into the maw.

A lot of people have written about this. Some of Microsoft’s own people have had moments of sanity. I just wanted to register my disgust. If anyone from Microsoft comes upon this, know that you’ve lost another “heart and mind.” I have unwittingly done my part to push your products and agendas in the past. I won’t be doing it anymore. Will I stop using the .NET platform altogether? No. Will I avoid subjecting my employers and clients to change for change’s sake? Absolutely. Will I make every reasonable effort to get them to consider alternatives? You better believe it.

Update: As with all things, I should have looked to Looney Toons for an appropriate analogy. We developers are Elmer Fudd, and whatever Redmond is pushing is Bugs Bunny in a dress. We just keep falling for it every time.

Rafe at rc3.org posted about an interview with Twitter developer Alex Payne where he discusses some of the problems with scaling Rails-based applications. The title of Rafe’s post, Twitter developer: Rails performance blows, was pretty inflammatory and provoked a strong response from interested parties.

The benefit, for me, is that a lot of interesting posts on how to scale Rails.

• Ryan Tomayko has weighed in with a great post on making use of multiple databases with ActiveRecord and scaling in general.
• Another Twitter developer, Blaine Cook, has let us know that the Twitter team has developed some tools to deal with the problem that may be released at some point. Blaine will also be giving a talk next weekend, partially about this topic.
• DHH has posted about a plug-in for Rails that addresses (somewhat) the multi-database issue.  It appears to be a work in progress, but at least the ball is rolling.

Do you think they would have let this bug escape if they cared just a little bit about unit testing?

Over the past few years, a lot of my work has been centered around Microsoft BizTalk Server. While I have a lot of good things to say about the product, there are a few things that really get under my skin. I had very high hopes that they would be addressed in BTS 2006, but it seems that there are some things the BizTalk team just isn’t ready and/or willing to address.

If I’m way out of line on any of these, please let me know. I would love nothing more than to be shown how to address these issues.

Building BizTalk Projects

Unless I have just really missed the boat on something, integrating BizTalk projects into any kind of automated build process is really a nightmare. When Visual Studio 2005 came out, it really looked like Microsoft was “getting it.” With the .NET Framework SDK installed on a machine, you could build your projects without Visual Studio or any other software installed. Project files are now just build scripts in disguise. Project files, that is, other than the ones for BizTalk projects. For some inexplicable reason, BizTalk projects are still in the same old Visual Studio 2003 format. This exempts BizTalk projects from things that are generally considered good development practices, like continuous integration builds.

Some of you (if there are any of you at all) will argue that there are workarounds and custom build actions that enable the building of BizTalk projects. This is accomplished by subverting the normal build process and shelling out to Visual Studio. There are at least a couple of problems with this. First, it means a Visual Studio/BizTalk installation on the build server. Not only does this mean licensing costs, it also feels a lot less clean. By ending around the “normal” build tool, I would imagine that logging and transparency are comprimised.

Another side effect of this is that developers without BizTalk installed can’t build the projects. Restricting their ability to edit the projects is one thing, but keeping them from building the project prevents them from determining if a code change will break the build when checked in.
Everything in the GAC

The GAC can be a very powerful tool. It can also cramp a developer’s style like nobody’s business. I can’t tell you how may times I’ve made a change to a component consumed by BizTalk only to wonder why I wasn’t seeing a change in the behavior. Is there not any other way that this could be approached? The workarounds for GACing your assemblies are almost as cumbersome. I’ve used the registry key for assembly resolution. I’ve tried editing the BTS AppDomain creation information to use a different base path. Nothing seems to stop the interruption to my normal development flow.

Another consequence of this is that providing configuration information to your BizTalk artifacts is hampered. Editing the BizTalk service configuration seems risky and, once again, just makes BizTalk development seem different than anything else you encounter in the .NET world.

Poor Facilities for Unit Testing

Has anyone actually come up with a workable solution to BizTalk unit testing? There are a few tools provided to execute pipelines, etc., but they don’t really integrate well with the MS Test framework. Testing an orchestration without jumping through the deployment and binding hoops is next to impossible. I was extremely excited when I heard about BizUnit. That lasted until I realized that the tool was not correctly named. BizUnit performs what I would consider acceptance or integration tests, but does not allow you to test BTS artifacts in isolation. Last time I checked, a unit test meant testing a unit of code and as little as possible of anything else.