This post by Jeff Atwood really gets at why I don't blog much any more. There is a certain kind of pressure when people are actually or might be actually reading your blog that you don't get when it is just sitting there It's the pressure that what I am doing at the moment is not as interesting or useful to other people as the things so I don't want to waste people's time even skimming it to decide that themselves. I also ran into some incidents where idiots took everything I said in the way they wanted to, or used it to strategic advantage against me to supposedly reveal my hidden preferences or lack of commitment to their chosen path in The Neverending Battle over Which Technology to Use. And then there are the other things, the Twitters and Google+... anyway, maybe I'll find something useful to say again someday. Just reading through my old ideas, there aren't too many embarrassing ones that would prevent me from ever looking at this stuff again.
Saturday, March 01, 2014
Saturday, November 17, 2012
Felix Salmon dismisses the effect of loan subsidies on rising tuition costs by pointing out that the loans get repaid with interest, so that means it is not a subsidy. Besides being a very weak argument, whether or not it is labeled a subsidy is immaterial. It is credit which allows people to pay more. Without a cap, colleges wouldn't be able to charge as much, and hire so many useless administrators. The cost of college is out of control. Watching the online disruption is fun. Would love to find a way to get more involved....
Posted by Matt McKnight at 1:26 PM
Thursday, November 15, 2012
Reading this semi-widely shared piece by Damon Krukowski- I really think the analogy is wrong. Pandora is like having your song played on the radio, not like selling 7" of vinyl. When I buy a piece of vinyl, I can listen to it as many times as I want, or at least until the grooves run down. He tries to compare the royalty for 7,000 plays of a track on Pandora to 7,000 sales of a single?? It's more like a radio station with 7,000 listeners playing it once. How much is Galaxie 500 making in radio royalties? I haven't heard them on the radio for 15 years. They should be comparing their $1.21 to $0.00.
Posted by Matt McKnight at 11:37 PM
Friday, October 05, 2012
Should the domain model be separate from the serialization format? For example, if you are using something like Avro or Thrift as your serialization format, does it make sense to have all of the objects in your domain modeled as objects generated by these technologies or is better to just model them as Plain Old Java Objects and then find a way to serialize those?
This question has recently come up, and I find it reminiscent of the older question of whether your XML data model should be your domain model or whether your database model should your domain model. The basic difference effect of using this pure data model approach is that you end up with classes that do not contain any behavior- basically structs. In the other case, you end up with objects that have both data and behaviors that depend on that data. The additional wrinkle is that generating classes from schema, be it thrift, Avro, ProtocolBuffers or XML adds the additional wrinkle of working with generated code. You typically don't modify generated code, so this can make adding methods, functions, etc. to this data a little challenging.
There is a great body of knowledge out there that supports not putting a dependency in your code on a particular technology implementation. In particular, modeling your domain effectively and independent of other components follows a lot of solid principles, and lets you pursue a domain driven design. The opposite, an Anemic Domain Model has its limitations- a procedural design, generally missing out on OO.
The first "depends" I would consider is the programming paradigm. If you are working in a functional programming paradigm, not just a procedural one, for example in a language like Clojure, separating data model from behavior is a natural pattern. Many of the object oriented advantages to a rich domain model simply do not apply. If you are working in an object oriented paradigm, for example in a language like Java, it makes more sense to put the behavior with the data.
These are still generalities though. Generated code is less of a challenge in languages other than Java. Some languages, such as C#, allow partial classes, so you can combine generated code with code that you write in the same class. Dynamically typed languages like Ruby provide a lot of flexibility, so having model classes that are generated by a framework like ActiveRecord, or a pattern like Active Record)is not a problem, because the classes contain no code.
The question then comes up of things like the Hadoop MapReduce framework. While your code may be in Java, the programming paradigm of map reduce is functional. (Some would even say that it is giving us an opportunity to get away from the object oriented programming model- and that it is sad that we produce CompSci graduates that only know Java.) Thus the benefits of having behavior associated with your objects is greatly reduced.
However, in the particular case we are considering, we are not just working in MapReduce, we have a number of application layers that can use the domain object. We are working in Java, so we can't do a lot of cute things with partial classes or mixins. We do want to use Avro as a serialization format though, so there is some momentum for using that as the medium of our domain model. It forces a certain weakness into our domain model, and an awkwardness to our programming. It is not as noticeable when writing functional style Map Reduce code, but it is quite noticeable when we try to abstract behavior such as validation and mapping operations that naturally go on the model. We can get something close to the
Of course, if we go to a more pure, POJO domain model, it then becomes a little more tricky to go in reverse and make the serialization occur with Avro. Still thinking on this one...
Posted by Matt McKnight at 5:05 PM
Tuesday, August 21, 2012
The patent system, and intellectual property rights claims in general, are out of control. You may think I am referring to the current Apple v. Samsung case where the invention of the rectangle is in question. I am actually more concerned about the White v. Heinz case. We get no benefit from allowing these independent inventors to hold companies who came up with the same idea hostage...just as we get no benefit from more organized patent trolls. It's a game lawyers are playing to exploit the legal system to take money from people. I came home yesterday and my daughter had created a shop for Patented fashion designs, along with a licensing scheme if you actually wanted to make it. The sewing machine was sitting there unused. The designs were great though, but the inspiration was clear. I guess Everything is a Remix.
Posted by Matt McKnight at 5:00 PM
Saturday, December 25, 2010
It's been a busy year for me, but more troublingly, a year in which I have been full of 75% complete ideas that haven't been worthy of their own posts. I have done a fair bit of reading though. Here are some of my favorite books that I've read this year (regardless of whether they came out this year). Maybe you can use some of the % back from your Amazon Visa card to buy them.
Morville, Search Patterns: Design for Discovery
If you are working on application that includes search and discovery features, this book sets forth a great pattern language. There are tons of examples and ideas here.
I'll come right out and say it here publicly on the Internet: Scrum is not a good software development method. Scrum masters are not useful. I still like XP. Kanban and Lean are much more sensible methods to go with XP practices than Scrum. It's really way too much to get into here, but having spent the past 7 years reading on the Theory of Constraints, Toyota Way, Lean, etc. this is a method that makes so much sense to me I think I've already been doing it without having a name for it. Yes we Kanban!
Fried, Hannson, Rework
These guys just have no patience for all the non-value added aspects of work (shades of Lean and 'muda'). They also have a way to ensure that the work we do maximizes the opportunities for innovation, creativity, and even beauty, ultimately leading to get us towards our own happiness. Work should make you happy.
Matt Ridley, The Rational Optimist: How Prosperity Evolves
The author of the awesome The Red Queen comes back with treatise on why all the things that the whiners keep themselves busy whining about aren't really too bad. In all of the hand wringing about the minor issues of the day, we are vastly better off than people at any time in history. His thesis is that specialization and markets are the prime movers of progress, and the world might not falling apart at the seams as much as it sometimes seems. Actually made me hopeful about life on Earth, which is a rarity.
Sam Harris, The Moral Landscape: How Science Can Determine Human Values
This is a fascinating book. It's bit harsh on the approach for dealing with religious people, but it gets to the heart of where we can find areas of moral agreement, based on some simple starting points. I thought it would devolve into utilitarianism, but Harris takes it in another direction all together. He directly attacks the distinction between values and facts. It really made me question some of my "cultural relativism". I saw Harris speak earlier this year, and it was quite good. Plenty of his videos out on YouTube if you want the lite version. This will be offensive to many, but it's worth at least figuring it how to address these arguments if you disagree with them.
Ricky Gervais Presents: The World of Karl Pilkington
This is a bit funny. You probably have to heard the podcasts or show a couple of times to appreciate it, but it is really quite good with lots of laughs at the expense of the indefagitable Pilkington.
Lidwell, Holden, and Butler Universal Principles of Design, Revised and Updated: 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions, and Teach through Design
I love reading about design. I am in the middle of the new Frederick Brooks book on the design of design. I am still wondering how much reading about design it will take for me to eventually be a novice level design person.
Doug Lemov, Teach Like a Champion: 49 Techniques that Put Students on the Path to College
This is one thing missing from the teaching profession. In the manner of a software patterns book, this covers a set of techniques for running an effective classroom. Buy copies for your kids' teachers.
Jeffrey Pfeffer, Power: Why Some People Have It and Others Don't
Just be aware that I've read this book and have been trying to internalize it. At some point, you have to achieve a balance between getting what you want and being right. Even if you don't have the Will to Power, this will help you understand what those with the ambition are up to.
Ackoff, Beating the System: Using Creativity to Outsmart Bureaucracies
Some good ideas here.
That's just a quick sampler of what I can remember, which gets smaller every year. Enjoy!
Monday, October 26, 2009
The concept of a migration is that each change that is made to the structure of the database is captured in a version controlled script. Any time you want to create a table, add a field, change a field type, etc., this is done via a migration. Migrations are generated along with models, views, and controllers when a Rails scaffold is created to manage a resource. In addition, migrations are used to create a test database for each user which allows for rapid, automated quality checks on the application. If you have ever worked on a project where it is difficult to create a development "sandbox" database for each developer, where databases are copied via backup processes that leave weird permissions behind (I'm looking at you SQL Server), or where keeping the development database in sync with the application is something you even have to think about, you will easily see benefits from migrations.
Using Rails migrations can be a bit of change for teams. In some cases, developers are not very familiar with relational database concepts and object modeling principles and can create some overly complex and/or inefficient structures. Having the overall model reviewed by a database modeling expert and an object modeling expert can be quite helpful. I recommend Coad's "Java Modeling in Color with UML", Fowler's "Analysis Patterns" and David Hay's "Data Model Patterns" for getting yourself up to speed with this. However, it is also worth communicating with the rest of the team that may be touching the model.
Here are a few things we've done in the process of keeping migrations under control:
Don't edit an existing migration, create a new one.
Editing an existing migration defeats the whole point. You might be able to get away with it if no one has run it. One obvious exception to this is if someone creates a migration that doesn't run and checked in. They must get the pig.
Update and run new migrations before you check in a migration.
While this might seem obvious, in many cases, it is easy to check in a new migration file to the repository without seeing if someone else has created one that does something that conflicts with yours, since the names will be different. If you don't run the migration, you risk doing something with your migration that causes issues. If you have already run your own migration locally, but not checked it in, and you get an update of a new migration, you need to roll back your own migration and change the numbering on it so that it runs after the migrations you just received from your version control system.
Check in the models that go with the migration when you check in the migration
One key benefit of migrations is keeping the code changes in sync with the database changes. Add new tables and models in a single, logical commit.
Assign models to class owners
The feature of XP that I have been least successful with is the concept of common code ownership. It is one of the easiest practices to apply in theory, but without the sense of collective responsibility and the compensating practices of pair programming and continuous integration, it can be problematic. In larger teams, it often makes more sense to have people or subteams assigned to manage a particular model so that changes can be coordinated. In theory, there can be enough communication to make it work. In reality, many teams are full of introverts that would rather rewrite huge swaths of code (and tests) than ask a question.
Changes to the object model can impact the whole application. These changes should be communicated to the rest of the team so that the reasoning behind them can be better understood and the data can be used correctly. There is a risk of someone criticizing or trying to change your proposal, but this is ultimately a lower risk than building something that others fail to understand.
Roll up all migrations into a big file after a major release
Once you've been going in your project for a while and get around the 100-150 migration point, it is probably worth it to roll the migrations prior to the current point into a single migration.
Generate SQL to run on the production database
In many cases it is preferable to generate SQL to run on the production database, as opposed to running the migrate task directly against the database. This also allows for some more extensive code review on structural changes.
Don't forget to create constraints and indexes (indices?)
If you are using foreign keys in your database, it's often worth the cost to create a constraint on the column to prevent any data integrity issues. It's also a good practice to create an index on those columns to speed joins.
This is really just the tip of the iceberg, but it's a yummy water ice iceberg. Learning to use database migrations is a key skill, not just for Rails development, but for any agile development. A project which cannot create its database from scripts accessible to every developer is missing something important. By breaking those scripts down into small chunks and coordinating them with changes to the code, the situation where the application and database are out of sync simply does not exist.
Wednesday, October 07, 2009
There is sometimes a quandary between whose success we seek.
Acting as a consultant, you obviously want your own work to be considered successful, so that you will induce others to hire you. Getting there isn't so easy though. We often look downstream to the customer's customers. By helping meet the customer's customers' needs, we hope to make our immediate customers successful. The catch comes when something the immediate customer is doing is preventing them from satisfying their customer- and they don't want to change it. If we make the customer's customer successful at the cost of making the customer unsuccessful, we're probably not going to be successful.
Remember the secrets of consulting- you have to help people learn to solve their own problems, you can't always solve their problems for them. You can be satisfied with your accomplishments, regardless of whether they give you credit. The ideal form of influence is to help the customer see the problem more clearly, but let them decide. Finally, if you want to understand a dysfunctional system, follow the money!
Monday, September 28, 2009
Very nice addition to Rails as part of the security updates in 2.3.4- an easy way to load seed data.
Added db/seeds.rb as a default file for storing seed data for the database. Can be loaded with rake db:seed (or created alongside the db with db:setup). (This is also known as the "Stop Putting Gawd Damn Seed Data In Your Migrations" feature)
I like this on several accounts, not least of which is it invalidates my previous post on the subject.
In the meantime, there are several plugin based approaches to making this work, for example seed-fu.
Posted by Matt McKnight at 3:59 PM
Thursday, September 10, 2009
One of the things that has been bothering me in Rails since everything switched to resources is how we lost the easy way any public method in the controller became an accessible URL without having to write a rule for it. For example, in a person controller, I could define a search method like this:
def search results = Person.seach(params[:q]) endAnd it would be accessible, just like that. http://myapp/people/search?q=matt
I even wrote a simple router in Java to do the same thing, because it just makes it so easy to navigate the application code.
The poor design of map.resources messes this up. By adding the route pattern :controller/:id it starts with the default assumption that the search?q=matt part of my URL is an ID. Is it so hard to put an action name in there? In any case,
map.connect ':controller/:action'goes above all of the map.resources calls in my app. You might want to put it in your application too.