Autobiography is only to be trusted when it reveals something disgraceful. A man who gives a good account of himself is probably lying, since any life when viewed from the inside is simply a series of defeats.

George Orwell

You may find your attempts to introduce technologies or ideas to be frustrating and the path littered with failures. The important thing, I think, is to remember that you are not alone in that feeling.

Also, if you find people who are absurdedly self-confident and self-aggrandizing, they are probably blustering to cover up some significant weaknesses.

Tony Morris blogs about his solution to the problem of validating ‘balanced pairs’ of tokens in a command line argument

Read both the article and the comments. Then come back here. Don’t worry, the browser will wait patiently for you.

My thoghts:

• Tony Morris believes that he demonstrates that Haskell has better functional composition than Java. But given the examples of others in the comments, he has only demonstrated that he does not know how to produce well-designed Java.
• His Haskell example has a high complexity/size ratio (technical density, if you will) and it would be very difficult for anyone but a Haskell expert to decode it.
• There are lots of examples written in the comments, in various languages, many of which are more elegant (easy to understand, yet effective in small spaces) than anything Tony wrote

It’s clear from the article that Tony wrote these in a hurry, and I don’t want to sound like I think he’s a bad programmer or a bad person because I disagree with him about the ‘lesson’.

What is interesting is that Tony wrote the code with a certain mental model of what ’success’ meant. And the comments exploded with people coming up with different definitions of success, and different ways to succeed.

As an Agile Project Manager, if I can define success the right way (for me): “I want a balanced token matcher, that’s easy to maintain“, then I can step out of the way, and let the developers figure out how to meet that goal.

Kudos to Tony for bringing up this interesting discussion.

(Hat Tip: Ragenwald)

I was able to write a working Monte Carlo Project Simulation prototype in less than 24 hours. Here’s a screenshot of the relevant portion:

Manifest:

• Grails
• IntelliJ
• JFreeChart (via Eastwood / Google Chart API)
• EatJ hosting

In the process of building a Monte Carlo simulation engine, I stumbled across Eastwood, and the Google Chart API.

Diving in, I’ve discovered that the Google Chart API is pretty solid for simple charts - easy to get started, and fairly simple to experiment with by manipulating the URL.

And, if you’re using Grails, Eastwood is a plugin that converts JFreeChart to use the Google Charts API. Very nice.

• Dealing with Negative numbers

Over the summer, I was tasked to put together a ‘Build Metrics Dashboard’ - a piece of software that could parse the XML reports from a bunch of common code and automated testing analysis tools, and produce a comprehensive report.

We’ve been sitting on it for a few weeks, but the website is up and running, and it’s time to announce: FuseMetrics

FuseMetrics is written in Groovy, and can parse the reports of many of the most common analysis tools:

• Junit
• TestNG
• JDepend
• Checkstyle
• PMD
• JavaNCSS
• FindBugs
• Simian
• Clover
• Cobertura

And, because of a basic plugin architecture, it can be extended to support just about any other tool, as long as the output is well-formed.

What does it do?  It produces summary metrics and graphs - sparkline graphs of various metrics-over-time, and histograms, which profile the population of a set of code files.    The histograms are more powerful with bigger codebases, but the sparklines are useful for just about anyone.

Here’s a screenshot of FuseMetrics running on my test data

It’s completely open source - you can get the code here.   Try it out, let me know what you think!

Back in June, someone hacked this site, and added a malware iframe.

I cleaned it up, upgraded Wordpress, and went on with my life, assuming that someone had just exploited a post.

Well, it was hacked again, the same way. Since I had upgraded Wordpress, I was more suspicious, and I delved into my website setup.

When I went to the User tab, I saw this:

Administrators(2)

(And I expected to only see one) But when I went to the adminstrators tab, I saw this:

Administrators(1)

hmmm. That’s suspicious. I looked at the page source, and sure enough, there were two entries, but the second one was buried in a morass of javascript and styles. Egads - someone has not only gained administrative access to my blog, but they have effectively prevented me from removing them via the website.

Luckily for me, I know enough about SQL to be dangerous, so I went into PhpMyAdmin and deleted all the other users except me. This worked because the only users were random bogus registrations trying to get around my spam filters.

I’ve since turned off new registrations, and I suggest everyone else do the following:

1. Check on your users - see if you have more administrators than you expect, and if so, delete them with extreme prejudice.
2. turn off new user registration

Bleah

I spent most of the last post talking about magic, but what I intended to talk about is the regular occurrence of the database getting out of sync with the domain objects in Grails.

In theory, based on using the Update directive in the database configuration, my database should be updated to match the domain objects. In my experience, however, Grails regularly fails to properly upgrade the database, and I’m forced to delete the whole thing in order to get the system sane again. This doesn’t happen all the time - generally only when I need to create a new relationship between tables or apply a new constraint (for example, allowing a column/field to be null).

Luckily, since I’m using Hypersonic SQL, this is straightforward. Shut down your grails app, and from your grails project top-level directory:

cd db
rm devDb.*      # or whatever the name of your database is

Then restart grails. It will regenerate the database (without data) with the latest constraints and structures you’ve made. You’ll need to repopulate, but there are tools for that.

In Rails, your domain model objects derive their structure and content from the database layout. In Grails, however, the database tables are created based on the fields and directives embedded in your .groovy files.

Of the two mechanisms, I find Grails’ approach to be more sensible for greenfield applications (i.e. no legacy database), and for legacy applications, it’s probably a wash, unless the database is already named in a Rails-friendly way.

I like the Grails approach because it does not violate the ‘Do No Magic’ directive.
What, pray tell, is the ‘Do No Magic’ directive?

Good question! Let’s consider a software thought experiment. You’re a novice programmer, and you’re assigned to work on an existing project. You understand the basics of software development, logging, etc, but you’re not an expert.

Your first job is to fix a bug. You have access to the source code, you know generally where the bug is, and you have log files that show the bug in action.

So you look at the source, and then look at the log files. Let’s say your source looks like this

    log.debug "About to do Step A"
    step_a()
    log.debug "Completed Step A"
    log.debug "Program Finished

And inside step_a() you see:

    def step_a() {
        log.debug "Starting Step A"
        // biz logic
        log.debug "Completing Step A"

But when you look at the log file, you see this:

1000 : About to do Step A
1001 : Completed Step A
1002 : Program Finished
1003 : Launching Thread
1005 : Activating Queue
1006 : ... other stuff...

How is this possible? Some of you may ask. My answer is - in this case - Aspects - which ‘intercepted’ step_a() and did something else instead.

But at no point in the code do you see the possibility of said interception. You have to know ahead of time that Aspects are running, in order to understand this code. To the uninitiated, this code is magic.

Rails has a similar problem - you have to look at the correct table in the database to discover how the domain objects are structured.    Again, to those who don’t know that Rails derives models from database columns, this is magic

Let’s not even speak of monkeypatching, which might more properly be called duck punching.

Hold Your Flamethrowers

To be clear: the use of database for domain objects in Rails is a relatively benign form of magic - it isn’t nearly as bad as aspect-injected  asynchronous thread launching and other such mischief.   But it is still more magical than Grails.

One of the interesting things I’ve had to work with on my Grails project is compilation.  Occasionally, I’ll get a .groovy file that seems to put Grails into an endless ‘compile-load-restart’ loop.   To discover which file is the culprit, I physically stop the server after the compilation, but before the restart, and then go into $HOME/.grails/<VERSION>/<Project_Name>/classes/  and look to see which file was most recently compiled.

This tells me a little about what’s going on, and I was able to remove the .groovy file in this case to stop the endless restarting.

One frustrating thing I’ve discovered with Grails is the way data is sent to the view pages.

In Rails, it looks a little like this:

@scenario = Secenario.new
@scenario.title = "New Scenario"
@scenario.description = "Add A Description Here"

render :view => 'create'

This snippet creates a new Scenario object, populates it, and tells Rails to render the create.rhtml file, with the @scenario object in the page’s model, helpfully known as @scenario

In Grails, it looks like this:

def scenario = new Scenario()
scenario.title = "New Scenario"
scenario.description = "Add A Description Here"

render( view: create, model: [ scenario: scenario ])

and, similarly, Grails knows to render the create.gsp file, and the scenario object is available to the page as scenario

Which isn’t terribly different, and fairly easy to use.

The problem comes in because of the way Grails uses closures to provide the action methods on the controllers (edit, create, delete, save, etc).

In Grails, you might have a action method as such:

def create = {  // closure of action method here }

Now, look at the previous Grails code. Do you see the issue? Yes - create is now overloaded, and in some cases, Grails will attempt to find a page named Controller_closure_blah_blah_blah.jsp, which is obviously completely wrong.

The fix is simple:

render( view: 'create', model: [ scenario: scenario ]) 

Basically, make sure you always use quotes for your string values, even though the Groovy language allows you to leave the quotes off. Otherwise, you’ll occasionally get frustrating and confusing results.

Interesting issue that left me scratching my head this morning…

using gsp tags, you can iterate over arrays:

   <g:each in="${myarray}">
      <p>${it.title}</p>
   </g:each>

but if you do something slightly more clever:

   <g:each in="${myarray}">
      <p><g:link action='myaction' id='${it.id}'>${it.title}</g:link></p>
   </g:each>

You’ll get null pointer exceptions.

Why?

Because the <g:link> tag creates its own instance of the special variable it inside its own ‘domain’.

Fix

add the parameter var=’myvar’ to the <g:each”> tag: <g:each in=”${myarray}” var=”myvar”>. Then, use ${myvar.id} and ${myvar.title} in your code:

   <g:each in="${myarray}" var="myvar">
      <p><g:link action='myaction' id='${myvar.id}'>${myvar.title}</g:link></p>
   </g:each>

And everything should work again.

If you’ve used Rails, you’re familiar with the flash, and all the nifty things you can do with it.

In Grails, you have flash, but you also have an entire tag library of <g:hasError>, <g:renderErrors> and so forth. None of which, as far as I can tell, care one whit about the flash. They’re all focused on domain objects, and the errors associated with processing them.

(In my opinion, this is one of the frustrations of Grails examples - the tendency to get the domain connected to the view via scaffolding, and then the author changes the subject)

But you can use the flash, in much the same way as Rails. For example, I put the following in my top-level layout template:

              <g:if test="${flash.error}">
                 <div class="errorbox">
                    ${flash.error}
                 </div>
              </g:if>

              <g:if test="${flash.message}">
                 <div class="messagebox">
                    ${flash.message}
                 </div>
              </g:if>

(note that you’ll have to define the messagebox and errorbox css classes)

Putting information into the flash is straightforward:

     flash.error = "Your error message here."

Hope this helps!

Can’t resist

Clif Bar Triathlon Start Commercial

Heheheh

My site was “partially” down for a week or two, because someone hacked a malware iframe into one of my posts, and Google flagged it.  This cascaded into Firefox 3 refusing to let me visit my site (bleh).  I was able to find the post, remove the offending iframe (and upgrade to the latest version of WordPress).

Still, more than a little frustrating to discover accidentally that my website was blocked.

8.3 miles running yesterday (1 hour, 40 minutes)
35.3 miles biking today (2 hours)

Yeah, I know I’m slow compared to all you uber athletes out there. I’m just working with what I have

An interesting comment on estimation:

1. Nobody ever does it. In fact, I don’t even know of a process to achieve this. Hollering at people who over/under estimate is not an improvement process.
2. It assumes you can make developer estimates better. More experienced developers estimate better, that I’ll take as a given, but can you accelerate this with novice/junior developers or testers?…
3. Software is NOT like mechanical engineering. It is a craft. … So our inability to accurately and precisely estimate shouldn’t be all that surprising.

Personally, I believe that these claims are false.

1. People estimate all the time - Velocity based on complexity, jellybeans, gummy bears and ideal hours are all fairly rigorous forms of estimation, when they’re done consistently.
2. There is plenty of evidence that mechanisms such as the Delphi Method do, in fact, make the general estimate far better, even when you include a mix of junior and senior developers. I use this extensively, and it has never done me harm yet. When you have five smart people discussing how hard a particular task is, you find out the different perspectives quite quickly.
3. While I agree that software is a craft of creativity, no one (that I know of) thinks that estimates have to be train-schedule accurate and/or precise. When I wear my project manager hat, I just want a general feel about how many tasks will fit into my two week iterations.

Insightful commentary on the future of spam.

He (or She) is right - Spam-management is likely to be one of the critical priorities for the future, before the entire Internet crumbles under the tragedy-of-the-commons Spam assault.

There is one “ideal world” answer, which would essentially stop spam in every possible permutation - the use of micropayments.

Specifically:

• A micropayment (by you) each time you post a comment
• A micropayment (by you) each time you link to a blog and it generates a trackback
• A micropayment (by you) to the recipient of every email you send
• A minipayment (by you) each time you sign up for an account, and then a micropayment each time you reach out to someone using that account - facebook wall messages, twitter posts, craigslist ads, /tells in MMOs, etc. The receiver will receive the micropayment.

I know, roll your eyes, micropayments are so played out. Well, I admit that they have been, and continue to be ahead of their time.

Captchas

Every other solution I’ve heard proposed involves more complex Captchas. Captchas are a hurdle, but, by definition, not an insurmountable one (because they have to be simple enough for below-average humans to pass them). Spammers continue to develop ever-increasingly-sophisticated Turing Machines, and are probably on the cutting edge of some forms of pattern recognition/AI. Each time we come up with a test, the spammers will eventually beat it. Unless you believe that computers “can’t ever” be smart enough to pass the Turing test, you have to assume that no Captcha will stand forever.

Essentially, Captchas are like building a wall in front of a horde of oncoming Mongols. They will eventually get over it. Once over it, it becomes a useless bit of architecture.

Micropayments

Micropayments, on the other hand, create an economic cost for every attempt at communication. And for most people, who receive communications about as often (roughly) as they send them, this is no big deal. If I send out 10,000 messages next year, at $0.001/message, that will cost me $10. And if I receive 10,000 messages from other people, that will reward me with $10 in credits. If I am a big talker, perhaps I spend a few dollars a year to keep my account balance up. But the point is, I don’t spend so much money that it becomes economically meaningful to me.

Spammers, on the other hand, if they send out 10 million messages, are looking at $10,000 in expenses. That’s a lot more money, and not something that can be done “lightly”.

The biggest problem in all of this is the starting-up problem. No one wants to be first website to demand micropayments, and hassle their users with an extra sign-up step. But referring back to the article - that - the generalized willingness to setup a micropayment solution - is the shape of “Web 4.0″

The Scenario

Here’s how I think it could go down - Google, Yahoo, EBay, Amazon, Microsoft and various other major players hash out a protocol for micropayment transfers. Then, they license the protocol to other organizations to implement, with the rule that every licensee has to accept and reciprocate payments with every other licensee.

My next thought is that once money (yours, mine, etc) goes into the system, it can never come out again - the micropayment universe is essentially a financial black hole - Why? To make it far less attractive as a target for hacking.

The Business Model

The Micropayment “Banks” would make money when you or I sign up our blogs to accept micropayments - a few dollars a year from a bunch of bloggers and smaller websites, a few more dollars from larger websites, etc, and you have a sustainable business model.

Accountability

The Licensees would be obliged to keep their books transparent - how much cash they received, how much they received in micropayments from others, how much they gave out in micropayments. That might even be part of the overall software specification. Someone (or many someones) can audit the books of the various Micropayment Banks, and verify that all the flows check out - that no one is claiming more “micropayment credits” than they actually have to give out. Standard financial audits would verify that the money coming in to each organization was legitimate and clean.

Commentary

This isn’t the most elegant solution in the world. Ideally there’s a solution that just involves the phrase “leave it up to the market”, but I can’t come up with one. That doesn’t mean it’s not out there…

Some feel that software products have to be essentially perfect in order to even have a ghost of a chance to succeed. People point to the iPhone and the iPod as examples of products that are practically perfect, down to the last detail, and are very successful.

Of course, one can point to a lot of counter-examples - projects that are less than perfect, but still quite successful (MySpace, Facebook, Google Mail, Google Docs, Twitter, etc). And, the huge pool of unknown projects - things that were never released because they could not achieve this desired perfection, and thus you are not even aware that they existed.

I think the gap here is the definition of “successful”. Some feel that a project is only successful if it has raving fans. Others focus on being useful to a lot of people, without worrying so much about perfection to the last detail.

I suspect (without statistical evidence) that you have the following kinds of results for projects that focus on perfection, vs. those that focus on “good enough”:

(Sorry the scale isn’t clear, the categories are “Never Released”, “Failure”, “Moderate Success” and “Huge Success”)

In reality, these scales are misleading, since the number of “Never Released” items is probably 10x as large (on both graphs). But in general, if you demand perfection, you have a slim increase in the chance that it will be a huge success, and a fairly significant increase in the chance that it will never be released.

Having said that, smooth functionality, elegant design and attention to detail are worthwhile. But they can be deceiving.

I gave up on Hani Sulemain when I realized that his modus operandi could be described in two steps:

1. Find something I don’t like
2. Announce with great certitude how clueless and incompetent everyone is who disagrees with me.

Alas, the chickens have come home to roost, I guess.

There’s a certain sense of “insecurity” in Hani’s post.  As if he’s suddenly lacking in confidence.  After all, someone fully confident of their opinions wouldn’t care what others thought.    I find that surprising in someone who was once so self-assured he could shower hate down on Google Code.   Now it’s almost like he is tired of being beaten up and just wants people to leave him alone.

Sorry Hani - this is the life you chose when you first decided to call someone else an idiot in public for supporting something  you thought was lame.