A dozen readings later...

I guess I am finally reaching the "advanced" Rails tricks, since I just used this. (Although, I have to admit the origin was a ryanb tutorial on editing multiple models, not from repeated readings...)

from Agile Web Development with Rails

An Extension to Ruby Symbols

(This section describes an advanced feature of Ruby and can be safely skipped on the first dozen or so readings....)

We often use iterators where all the block does is invoke a method on its argument. We did this in our earlier group_by and index_by examples.

groups = posts.group_by {|post| post.author_id}

Rails has a shorthand notation for this. We could have written this code as

groups = posts.group_by(&:author_id)

Another kind of DVR

Over at boingboing there's an interesting piece about the future of streaming video. "It's a bummer to consider a future in which broadcasts -- which we can all see and record -- are replaced with geo-locked, streaming crippleware netcasts.....All that said: top marks to the first person to demonstrate a working, reliable solution for watching and recording the CBC Dr Who episodes from anywhere in the world."

Right now, I am living in a bit of a gilded Tivo age where I can record just about any signal that comes into my home by intercepting its output right before it goes into the TV (which is basically just a monitor, I have never actually used its built-in tuner). The hugely hacky piece of the whole setup is the IR blaster to control the cable box by pretending to be a remote- with my new Verizon FIOS Motorola set top box, it just doesn't work all that well. New TiVos support the Cable Card standard- at the price of losing control of stream + $800 and a monthly fee. If restrictions are put into place on those new TiVos, for example flags on HD content that make it disappear after a week, then we might actually have a situation where you need to TiVo your TiVo (by capturing its output) to get the content into an open and safe format.

What is really striking about all of this is that the IR remote control is the open interface by which I can control the output of the cable box. Without that, it's basically useless because I couldn't choose what to capture. The whole weirdness by which all IR remotes are similar in that most can imitate the others but just about every device comes with a bunch of pages in the back that allow you to program the remote to control a variety of other devices.

I wonder if we aren't going to see a solution where the output going full screen to your monitor is going to be intercepted in a similar fashion, and then dropped back in to a capture card. It's really insane. The part of the computer that is analogous to the IR interface of the remote control is the URL. It's nice to take a step back every now and then and appreciate the beauty of that- URL addressable content. Now if we could just be assured that we'd be able to look at it when and where we want to.

s3sync.rb

I am finally backing my life (and my company's data) offsite with Amazon S3. I have been diving into this full on and it's quite cool. It took me way way too long to get it all set up. Nearly 4 hours to get a 7 line shell script in a cron job, due to a serious of yak-shaving events

In any case, it was basically easy with s3sync (the Ruby version, I believe the Perl version is abandoned...) Here is a decent link on getting it done. I actually used Marcel Molina's s3sh shell program a bit, simply because the error messages it dumped were vastly better than the HTTP 403s I was getting from s3sync. I just didn't find s3sh as "scriptable" due to my lack of shell scripting expertise at the moment. I am just glad I have a place to practice.

It's definitely worth reading a bit on the main AWS site about S3 before diving in. I wasted quite a bit of time due to not understanding the flexibility of the key/bucket system's handling of directories. And the minor bit about buckets having to be unique across all users.

Persistence Scaling

Interesting interview with Michael Stonebraker (the Ingres guy) from the ACM.[via GLinden]

The basic thrust is the RDBMS is slowing fading in its place at the heart of IT systems everywhere, for a variety of reasons. I personally have seen a lot more in the embedded database and in-memory database category lately, but he's talking about the relational model itself failing for warehousing, streams, text, and list processing. I don't know if the database concepts behind his current company Streambase and stream data processing are that broadly applicable. Still, his depiction of the areas of the relational model where there are scaling or intelligibility problems seems right on. I am not sure I get the following, but it makes sense in a way too:

"It’s the same case in scientific and intelligence databases. Most of these clients have large arrays, so array data is much more popular than tabular data. If you have array data and use special-purpose technology that knows about arrays, you can clobber a system in which tables are used to simulate arrays."

Vector oriented programming?

Stonebraker makes a nice point about how the ActiveRecord component of Rails is in effect a cleaner way of interacting with data than SQL. It really is. That said, I wonder if LINQ is on his radar? Sidetracked by this great post on from 9 till 2 on the 'Rubenesque' (That's Paul Rubens, not Matz) status of c#, which says the following about LINQ:

"Historically, there have been more Microsoft ways to access the Northwind database than they are rows in the Customer table. OLEDB, ODBC, DAO, RDS, JRO, RDO, SQLXML, ADO, ADO.NET, Entity Services....I suspect that DLINQ will probably not be the last ever ever ever in this data accessing periodic series."

And tellingly:

"LINQ has that big benefit of being able to treat relational, XML hierarchical and in-memory data objects all with the same query syntax, allowing you to swap store types at a drop of a hat. The nagging doubt though is that this may be the same big benefit akin to not having to use SOAP over HTTP. How did that go again? Something wonderful for a demo, but something that may not actually be a real pressing 'need'. There is a school of thought that when you're working with a relational database rather than a collection of in-memory objects, then you should not lose track of the various nuances and advantages of the stores - abstraction to save typing can come back to bite you?"

Yeah, like if you're doing Rails finders without :include? Ooops, N queries just ran. Still, I like the idea of s3record. As Nutrun says: "I have occasionally participated in conversations around the subject of the database as a product with an expiry date, destined to eventually be replaced by highly distributed data storage models. Although S3's data storage and retrieval model looks presently better suited for larger units of data (e.g. media content), it would be interesting to investigate how it could be applied as an Object persistence service."

This is obviously the wrong implementation for such a scheme, but it points in one of the directions we're exploring. The concepts of streams, spaces, distribution are the future of scaling and persistence. It won't be a simple design decision whether a database is at the heart of your system or not. It's going to be interesting...as usual.

Java EE 6: profiles versus subtraction

The JCP has produced a Java Enterprise Edition spec that even "Mr. Spring" Rod Johnson can approve of. This is actually a really big deal, as Java containers have become bloated by implementing expensive pieces of the spec which are seldom used. What they are doing is allowing for compliance to subsets of the standard, with these subsets referred to as profiles. They are also allow for extensibility, with a standard for extensions to the spec.

The "profiles" approach can lead to much confusion, as anyone who has tried to figure out why their Bluetooth phone and Bluetooth car can only communicate on the most basic levels can attest. The OGC also has been using this approach for GML data standards, and seems like overkill when you are just talking about XML. However, in general it makes a lot of sense when you need to preserve the whole from a service implementation standpoint. But really, sometimes just subtracting things from standard is a great thing. (JSF anyone?)

I wonder if there will be profile for app servers to run JRuby on Rails?