Secondary Actions in Web Forms

Just checked out this interesting article from LukeW on secondary actions on web forms. The sum of it is that people don't complete forms faster when you make distinctions between primary (submit) and secondary (cancel) buttons. Still, as Luke points out- hitting cancel after filling in a long form really is not good. Do you really need a button for that in all cases?

I really enjoy the data driven approach of using eye tracking data to help make decisions on form elements. Unfortunately he missed my current preferred option: Bolder font-weight on the text on the primary action button, lighter or normal font-weight on the secondary actions. Since we have CSS classes for those button types on my project, I was all ready to make some changes...looks like I won't be making any, but it's still worth the quick read.

Authentication in the database- revised

There are some people out there that believe there is one right way to build an application. The people that believe in best practices. The people that fall victim to Harwell's laws. (Particularly #1, "People always try to use their experience even if it doesn't apply to the current situation." and #3 "If a manager doesn't know how to improve an organization, then he/she will change it to look like the last organization that he/she remembers")

The particular issue I am dealing with has to do with web application security architecture, but it comes up again and again in architecture, particularly in web services. Where does authentication live and where does authorization live? And what trusts what. In particular, should a web application authenticate a user's x509 cert or Kerberos ticket, and then pass user information on to further systems, or should it pass the users security information on to further systems, such as databases. In other words, should authentication performed at each stage of a request that passes through multiple systems?

If you ask Oracle any question of the form "Should X be in the database?" the answer is yes. Apparently Microsoft is training their people to think the same way. It sure works great to induce vendor lock-in.

Somehow I am of the opinion that the first system a user comes in contact with should perform the authentication, that system should authenticate to other systems- as a system, not a user, and that their should be trust in the initial user authentication.

Having the database re-authenticate the user seems wasteful...and of no value.

Process Overload

I am completely sick of all of the processes that have to be constantly running on my Windows machines. The things that particularly annoy me are the "update check" software- I am looking at you menacingly iTunes, Java, Adobe and Windows Update. Why on earth do these programs have to run constantly? Can't they at least just have a scheduled task to run at a particular time? Or how about copying firefox and just checking for an update whenever I start your program. I think Quicktime asks for updates more often that I actually use the software. I don't need a background process running constantly to tell me a piece of software that I haven't run in two months needs to be updated.

Perhaps even more annoying are the "helpers" that run to make various software applications appear to start up faster. I suppose this is just keeping up with the various tricks that Microsoft employs to make their stuff startup faster, so it's fair to blame them for starting the arms race.

And the final annoyance is the difficulty in seeing which svchost.exe PID goes with which service. How hard could it be record the name of the service, so that I could in a glance see which one is sucking CPU or RAM? I suppose I could actually code a solution to that myself...but I shouldn't have to. In fact, someone already did:

Process Explorer- this is what task manager should have been...

A dozen readings later...

I guess I am finally reaching the "advanced" Rails tricks, since I just used this. (Although, I have to admit the origin was a ryanb tutorial on editing multiple models, not from repeated readings...)

from Agile Web Development with Rails

An Extension to Ruby Symbols

(This section describes an advanced feature of Ruby and can be safely skipped on the first dozen or so readings....)

We often use iterators where all the block does is invoke a method on its argument. We did this in our earlier group_by and index_by examples.

groups = posts.group_by {|post| post.author_id}

Rails has a shorthand notation for this. We could have written this code as

groups = posts.group_by(&:author_id)

Another kind of DVR

Over at boingboing there's an interesting piece about the future of streaming video. "It's a bummer to consider a future in which broadcasts -- which we can all see and record -- are replaced with geo-locked, streaming crippleware netcasts.....All that said: top marks to the first person to demonstrate a working, reliable solution for watching and recording the CBC Dr Who episodes from anywhere in the world."

Right now, I am living in a bit of a gilded Tivo age where I can record just about any signal that comes into my home by intercepting its output right before it goes into the TV (which is basically just a monitor, I have never actually used its built-in tuner). The hugely hacky piece of the whole setup is the IR blaster to control the cable box by pretending to be a remote- with my new Verizon FIOS Motorola set top box, it just doesn't work all that well. New TiVos support the Cable Card standard- at the price of losing control of stream + $800 and a monthly fee. If restrictions are put into place on those new TiVos, for example flags on HD content that make it disappear after a week, then we might actually have a situation where you need to TiVo your TiVo (by capturing its output) to get the content into an open and safe format.

What is really striking about all of this is that the IR remote control is the open interface by which I can control the output of the cable box. Without that, it's basically useless because I couldn't choose what to capture. The whole weirdness by which all IR remotes are similar in that most can imitate the others but just about every device comes with a bunch of pages in the back that allow you to program the remote to control a variety of other devices.

I wonder if we aren't going to see a solution where the output going full screen to your monitor is going to be intercepted in a similar fashion, and then dropped back in to a capture card. It's really insane. The part of the computer that is analogous to the IR interface of the remote control is the URL. It's nice to take a step back every now and then and appreciate the beauty of that- URL addressable content. Now if we could just be assured that we'd be able to look at it when and where we want to.

s3sync.rb

I am finally backing my life (and my company's data) offsite with Amazon S3. I have been diving into this full on and it's quite cool. It took me way way too long to get it all set up. Nearly 4 hours to get a 7 line shell script in a cron job, due to a serious of yak-shaving events

In any case, it was basically easy with s3sync (the Ruby version, I believe the Perl version is abandoned...) Here is a decent link on getting it done. I actually used Marcel Molina's s3sh shell program a bit, simply because the error messages it dumped were vastly better than the HTTP 403s I was getting from s3sync. I just didn't find s3sh as "scriptable" due to my lack of shell scripting expertise at the moment. I am just glad I have a place to practice.

It's definitely worth reading a bit on the main AWS site about S3 before diving in. I wasted quite a bit of time due to not understanding the flexibility of the key/bucket system's handling of directories. And the minor bit about buckets having to be unique across all users.

Persistence Scaling

Interesting interview with Michael Stonebraker (the Ingres guy) from the ACM.[via GLinden]

The basic thrust is the RDBMS is slowing fading in its place at the heart of IT systems everywhere, for a variety of reasons. I personally have seen a lot more in the embedded database and in-memory database category lately, but he's talking about the relational model itself failing for warehousing, streams, text, and list processing. I don't know if the database concepts behind his current company Streambase and stream data processing are that broadly applicable. Still, his depiction of the areas of the relational model where there are scaling or intelligibility problems seems right on. I am not sure I get the following, but it makes sense in a way too:

"It’s the same case in scientific and intelligence databases. Most of these clients have large arrays, so array data is much more popular than tabular data. If you have array data and use special-purpose technology that knows about arrays, you can clobber a system in which tables are used to simulate arrays."

Vector oriented programming?

Stonebraker makes a nice point about how the ActiveRecord component of Rails is in effect a cleaner way of interacting with data than SQL. It really is. That said, I wonder if LINQ is on his radar? Sidetracked by this great post on from 9 till 2 on the 'Rubenesque' (That's Paul Rubens, not Matz) status of c#, which says the following about LINQ:

"Historically, there have been more Microsoft ways to access the Northwind database than they are rows in the Customer table. OLEDB, ODBC, DAO, RDS, JRO, RDO, SQLXML, ADO, ADO.NET, Entity Services....I suspect that DLINQ will probably not be the last ever ever ever in this data accessing periodic series."

And tellingly:

"LINQ has that big benefit of being able to treat relational, XML hierarchical and in-memory data objects all with the same query syntax, allowing you to swap store types at a drop of a hat. The nagging doubt though is that this may be the same big benefit akin to not having to use SOAP over HTTP. How did that go again? Something wonderful for a demo, but something that may not actually be a real pressing 'need'. There is a school of thought that when you're working with a relational database rather than a collection of in-memory objects, then you should not lose track of the various nuances and advantages of the stores - abstraction to save typing can come back to bite you?"

Yeah, like if you're doing Rails finders without :include? Ooops, N queries just ran. Still, I like the idea of s3record. As Nutrun says: "I have occasionally participated in conversations around the subject of the database as a product with an expiry date, destined to eventually be replaced by highly distributed data storage models. Although S3's data storage and retrieval model looks presently better suited for larger units of data (e.g. media content), it would be interesting to investigate how it could be applied as an Object persistence service."

This is obviously the wrong implementation for such a scheme, but it points in one of the directions we're exploring. The concepts of streams, spaces, distribution are the future of scaling and persistence. It won't be a simple design decision whether a database is at the heart of your system or not. It's going to be interesting...as usual.

Java EE 6: profiles versus subtraction

The JCP has produced a Java Enterprise Edition spec that even "Mr. Spring" Rod Johnson can approve of. This is actually a really big deal, as Java containers have become bloated by implementing expensive pieces of the spec which are seldom used. What they are doing is allowing for compliance to subsets of the standard, with these subsets referred to as profiles. They are also allow for extensibility, with a standard for extensions to the spec.

The "profiles" approach can lead to much confusion, as anyone who has tried to figure out why their Bluetooth phone and Bluetooth car can only communicate on the most basic levels can attest. The OGC also has been using this approach for GML data standards, and seems like overkill when you are just talking about XML. However, in general it makes a lot of sense when you need to preserve the whole from a service implementation standpoint. But really, sometimes just subtracting things from standard is a great thing. (JSF anyone?)

I wonder if there will be profile for app servers to run JRuby on Rails?

CHAOS = BS

Anyone who has looked at software development methods, myself included, is probably familiar with the CHAOS report of The Standish Group. I have blogged before about Robert Glass's article that the numbers reported that group simply are not supported by other studies. I was happy to see that referenced on Herding Cats.

The effects of the data they report (189% cost overrun average) are negative for the IT industry. They are used to justify all means of odd process controls and contribute to a general excessive scepticism about the possibility of making a software project a success. The lack of context given for the data (actually, the data is kept secret) means that all of this is based on a very shaky quantitative foundation.

Having just finished "Fooled by Randomness", I am going to take another look at the Monte Carlo approach to estimation. I used it in a couple of cases to provide broad estimates for large projects, but all of the numbers came out quite low. I still the think the only responsible thing to do is to provide a range of estimates, especially when the nature of the work to be done is uncertain. And it usually is. The real benefit of the simulation approach, from the fooled by randomness perspective, is that they allow to take account of the black swans. If you are using a purely inductive approach, you can't account for things that have never happened. Like a 3 day power outage in your server room. Or system administrator that changes the name of your source control server and costs 10 person days of work. Or a new programmer that checks his whole c drive into the source code repository. Well, those 3 have actually happened to me, but they aren't the sort of thing I usually factor into my estimates, particularly when using the methods from "Planning Extreme Programming", which are purely based on yesterday's weather.

The real challenge here is estimating the shifting sands. Until you have a feel for the rate at which requirements change, you are only going to be able to estimate based upon what people thought they might want something they don't understand to do. Of course, if there is a reorganization in a business you are supporting, and there usually is at some point, the rate of change can shoot up. This is where the difference between methods really starts to kick in. If you are using a method that "embraces change" (the subtitle of the XP book), then you can leverage change to be the engine that carries the organization forward. If you try to keep the requirements frozen, you will be one of the projects reported when the Standish Group calls and says, "tell me about your failed projects".

Many Methods

Cognitive Dissonance development (CDD) - In any organization where there are two or more divergent beliefs on how software should be made. The tension between those beliefs, as it’s fought out in various meetings and individual decisions by players on both sides, defines the project more than any individual belief itself.
-Scott Berkun

I've been there...wondering whether it is better to give up the argument because the other person is so incredibly stubborn. And it would have been better for the actual product had I given up. Not because my idea was worse, but because trying to do it two ways at the same time was worse than any one way.

I've been on projects where people waste all of the time trying to pick a product. I now have the confidence to tell people that most products really are not that good. Particularly ones that are intended to make development easier. So many of these create frameworks that are more complicated than what one was trying to accomplish in the first place, but offer less flexibility.

I was on a project that was trying to replace an ArcView 3.1 data entry and analysis application with about 25k lines of custom scripts with a Java web application based on ArcIMS and ArcSDE. This was in 2001. With a primordial version of Netscape. When all ArcIMS did was push jpegs down the wire. Oh, and they wanted to host it at a remote site. Adding points in ArcIMS at the time was a serious challenge. Adding lines was nearly impossible. I quickly quit. The project kept growing until the first deployment was such a failure they had to hide the bodies.

Obviously a place where Agile would have made a big difference. "Architects" in the ivory tower of the CIO shop had decided that we must have web apps for everything. If they had just put the demo app in front of an actual user, the whole project could have been quickly canceled.

I was listening to Uncle Bob Martin's Agile Skypecast while mowing the lawn on Sunday. It's a must listen. Despite all of the BS and corporate maneuvering around the buzzword, agile has deep principles behind it. I suggest ditching the buzzword and sticking to the concepts.
Early and frequent delivery of working code. Daily customer feedback. Developer testing (I am still working on mastering that one.) And the big one: Embrace Change. Don't fight requirements volatility, leverage it. (of course the trick is keeping the requirements stable during the iteration...)

It is interesting how Scrum leaves a lot of these things out and is, as Bob says, more of a management process that doesn't have much to do with software.

My favorite part of the agile universe remains the XP planning game. Point budgets as a means of giving multiple customers a voice, realistic estimation feedback, and ever so slightly, possibly, a little wee smidgen of fun. We could all use that from time to time.

Anyway, Uncle Bob gives the principles a good run through, worth the time to listen to a podcast.