tag:blogger.com,1999:blog-153249112024-03-13T10:24:09.456-04:00Let's Push Things ForwardMaking things better.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comBlogger255125tag:blogger.com,1999:blog-15324911.post-74667862241484223122020-08-17T22:35:00.003-04:002020-08-17T22:35:49.586-04:00Accuracy vs. Explainability Tradeoff<a href="https://ryxcommar.com/2019/07/14/on-moving-from-statistics-to-machine-learning-the-final-stage-of-grief/">This fascinating article</a> attempting to explain machine learning for statisticians is very interesting. I don't know if I even properly understand it, and the tone is a bit negative. Here's the simplest way I can think of it.<br />
<br />
If I have:<br />
(b1 * x1) + (b2 * x2) + (b3 + x3) + a = y<br />
the statistician is trying to minimize error in the b values and the machine learning person is trying to minimize error in the y value?<div><br /></div><div><br /></div><div>If you get the stats "too correct", the ML guy will know you are overfitting, and model will do worse on new data that was not in the sample/training set.</div><div><br /></div><div>Is that it?<br />
<br />
<br />
<br /></div>Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-24250818418504569062020-08-15T11:41:00.006-04:002020-08-15T11:41:49.708-04:00The Twitter for blogs<p> Thinking about whether there is a way to beat Twitter with open standards. The day Twitter shut off RSS (so that people couldn't build apps that avoided their ads) we lost a huge function of it as a part of an aggregation system. However, tools like NewsBlur show it can still be scraped.</p><p>Spotify is trying to create a walled garden around podcasts. Again, shutting off the aggregation function.</p><p>Google Reader was very popular. With some additional tools to <b>create</b> self-hosted content, likes, and some additional discovery tools, it could have been a distributed twitter.</p><p>We may be able to do this.</p>Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-49563413311000111042020-07-06T15:01:00.001-04:002020-07-06T15:01:18.662-04:00Lowering the barI have another post sitting in the dock, but I can't really release it because I need to think about it a bit more. I've been trying to lower the bar yet again so that I can produce some shorter things, but I still don't want to put out things that are wrong!<br />
<br />
Fred Wilson at AVC <a href="https://avc.com/2020/07/short-and-sweet/">https://avc.com/2020/07/short-and-sweet/</a> has posted about how he has been getting shorter with his blog.<br />
<br />
<img src="https://avc.com/content/uploads/2020/07/avc-word-count-corrected.png" /><br />
<br />
<br />
If shorter is good enough for Fred, it's good enough for me.<br />
<br />
Short content seems like Twitter. I just am not finding Twitter rewarding lately. It seems like the Cultural Revolution on there, just people spitting at each other. Not pleasant in any way. I was trying to keep a list of just silly funny stuff in my approach to using lists as the way to consume Twitter, and only following people I know personally and am thus less likely to be sickened by. I just can't stick to it properly and keep following people.<br />
<br />
Anyway, things are so ugly on Twitter I don't even feel like linking up these posts there.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-89739529272486973112020-06-28T22:51:00.000-04:002020-07-07T20:54:22.011-04:00Thoughts on David Patterson and Lex FridmanI was watching the fascinating Lex Fridman discussion with David Patterson and found it very stimulating. It really made me think about the nature of computer science as a whole in a way that I haven't for a while. David Patterson is just so clear when he speaks about things that it makes you consider the basic nature of the endeavor. It reminded me of so many things I used to think about more frequently.<br />
<br />
<a href="https://www.youtube.com/watch?v=naed4C4hfAg">https://www.youtube.com/watch?v=naed4C4hfAg</a><br />
<br />
Some random notes or ideas that occured or recurred to me. Not necessarily in the discussion, but where my mind went...<br />
<br />
<ul>
<li>The concept of layers of abstraction in computer science is simply fascinating. It is really possible to keep diving down deeper. It reminded me of when I was in my first electronics class and I was asking about how AND and OR logic gates worked. That wasn't really the point of the class, it was more about figuring out how to assemble those parts to make an adder or do something useful. The teacher didn't even really know or understand my question (or why I was slowing down the class by asking it). Some other student volunteered that were must be some kind of thing inside of the gate that would close or open based on the total voltage. It took me a while to realize this word I had heard so many times, transistor, was really how these were made.</li>
<li>I was then drifting to thinking about the intersection between philosophy and computer science at that most basic level. Not only are the logic gates abstractions for the transistors used to create them, the gates themselves are assembled into things like adders, that compose arithmetic operations from boolean logic operations. This ties to (analytic) philosophy a bit, when you think about how Frege was talking about Peano's axioms, and Russell and Whitehead trying to derive mathematics from logic. It took me back to a semester when I studying Wittgenstein and Computer Architecture (two separate classes) and kept sliding to how Frege's types and type systems in programming languages are similar. I wonder what kind of programming language Wittgenstein would have created- maybe it would have a been a game...</li>
<li>Of course, compilers themselves are a form of translation from one language to another. That's always interesting to think about. I really didn't quite grasp how Intel x86 architecture translates from CISC to RISC in the chip, just in time.</li>
<li>The idea of compilers then got me thinking about the idea of interfaces as mediating between layers of abstraction. The layer of abstraction can be thought of as an interface. It defines how to interact with that layer, but provides this additional opaque box capability of being able to swap out the implementation without changing the other layers. </li>
<li>Some more prosaic thoughts on the idea of the best technology not always being the market winner. How much I hated Elasticsearch from a pure attitude and marketing perspective, their deceptive ways How I felt vindicated when they went away from open source. How much I hate their current market position and hope AWS crushes them. </li>
<li>Just the discussion of how ARM has so effectively pursued RISC, and how Apple is making the software stack integrated with the hardware so effectively, I wonder when there will be a true open architecture phone with mass adoption that can compete. Amazing that Apple is moving to this for their desktop operating system. It seems crazy, but the possibilities are huge.</li>
<li>Finally, the chat on mlperf was so interesting. I didn't realize Nervana had refused to release their mlperf scores prior to the acquisition by Intel and their tech has been abandoned. Meanwhile Intel acquired Habana Labs, partially because they did have good mlperf scores. Benchmarking is so important. It made me want to benchmark things I am doing!</li>
</ul>
<div>
Okay, that's about it so far, but I am going to try to start recording my impressions like this more often. Probably won't be able to keep it up. But who knows?</div>
Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-44514006892784558302020-06-28T01:01:00.001-04:002020-06-28T01:01:19.423-04:00Transitory technologiesMost of the JavaScript frameworks seem to be transitory technologies. They exist to fulfill some perceived gap in the web browsers' capabilities. Once those gaps.are filled, their reason for existing fades.<br />
<br />
Probably the two strongest pieces are Web assembly and Web components. We also have webgl. It seems hard to imagine that a virtual DOM and another event model on top of the event model that already exists, are the right way to deal with these technologies.<br />
<br />
The only question is how long it takes.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-23789261494503632362020-06-20T17:42:00.001-04:002020-06-20T17:42:18.940-04:00Text as DesignI am not opposed to Design. I do think there are a lot of advantages to text as opposed to image heavy design, but I see both sides of both sides. There was a trend for a couple of years to have website home pages be huge, highly detailed, saturated images, or even video. The idea there seemed to be create this feeling of lushness or richness. This has obvious costs- bandwidth, latency, etc., but there is also something decadent about it. We have all of this bandwidth, let's use it. A picture is worth a thousand words. However, these are fleeting feelings that leave me empty. I need the words to create a rational idea out of the picture.<br />
<br />
I need the command line.<br />
<br />
I love the episode of the Netflix series on design "Abstact" that featured Paula Scher. <a href="https://www.netflix.com/watch/80093802?trackId=14277283">https://www.netflix.com/watch/80093802?trackId=14277283</a> There is a lot you can do with typography. I do see evidence that people are coming back to text favored design. <a href="https://cheapskatesguide.org/articles/beauty-of-text.html">https://cheapskatesguide.org/articles/beauty-of-text.html</a>.<br />
<br />
I am sick of fighting over which icon to use on a toolbar when a word would do-- although this does bring another battle. Which word? Which language? We live in a world where we fight over words. Do symbols let us agree to disagree? Do they let us put our own interpretations into those symbols, while disagreeing under the surface about what they mean?<br />
<br />
I kinda want to switch from using blogger to get more fonts.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-29386502731762443882020-06-17T11:54:00.001-04:002020-06-17T11:54:12.180-04:00Purist ProgrammingProgramming is about making things. Purity is about adhering strictly to a set of rules. It seems that one should favor the programming over the purity. Abstractions should be minimal.<br />
<br />
Think about this thing from John Cook on "<a href="https://www.johndcook.com/blog/2020/05/15/pretending-oop-never-happened/">Pretending OOP Never Happened</a>".<br />
<blockquote class="tr_bq">
That has been my experience. I hardly ever write classes anymore; I write functions. But I don’t write functions quite the way I did before I spent years writing classes.<br />
And while I don’t often write classes, I do often use classes that come from libraries. Sometimes these objects seem like they’d be better off as bare functions, but I imagine the same libraries would be harder to use if no functions were wrapped in objects.</blockquote>
Cook then quotes James Hague's "Follow-up to "<a href="https://prog21.dadgum.com/55.html">Functional Programming Doesn't Work</a>":<br />
<blockquote class="tr_bq">
100% pure functional programing doesn’t work. Even 98% pure functional programming doesn’t work. But if the slider between functional purity and 1980s BASIC-style imperative messiness is kicked down a few notches — say to 85% — then it really does work. You get all the advantages of functional programming, but without the extreme mental effort and unmaintainability that increases as you get closer and closer to perfectly pure.</blockquote>
Getting to the point of it, the pure functional people are the most annoying people in the world. The world has state. You need state.<br />
<br />
I really get annoyed by the programming paradigms in place now on the web, particularly the front end paradigms. There is a very simple event driven model that works surprisingly well. The component is an object, with state, that responds to events...anyway probably worth exploring the world of JavaScript separately.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-8699365218544410542020-06-14T12:24:00.000-04:002020-06-14T12:24:04.375-04:00GPT-3I am not sure if people who haven't been doing machine learning can appreciate how weird GPT-3 is or what "few-shot learner" means.<br />
<br />
In normal NLP machine learning, you might start with a pre-trained language model that encodes the relationships between words of a language. You then build a training set of data, probably 1000s of items where you have labels applied to text, correct translations, answers to questions, and things like that. You then train the model to minimize error on that training set. Then, with the trained model, you send it new samples of text and it spits out a label, translation, or answer as appropriate.<br />
<br />
With GPT-3, a much bigger language model, trained to just predict the next word, you don't have to do any of that. You construct the whole task in the last bit, where you would normally be sending a trained model new samples of text. The trick is, you send in a description of the task in with the text. So you could send in:<br />
<blockquote class="tr_bq">
translate from English to French: hat => chapeau, cat => chat, hello => </blockquote>
and it would send back "bonjour".<br />
<br />
It learned enough about language to be able to have examples of what typically follows "translate from English to French" to be able to get good performance on that task. This wouldn't be surprising if it had been trained on that task, but there is no task specific training (aka fine-tuning). It was just trained to predict the next word. Having a big enough model (a mind boggling 175 billion parameters) it just picks up that whole task as a pattern.<br />
<br />
Read the <a href="https://arxiv.org/abs/2005.14165">paper</a>.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-10257966979537516932019-12-04T13:17:00.000-05:002019-12-04T13:17:14.672-05:00Decisions<h2>
Thinking about decisions</h2>
<div>
Check out this product from <a href="http://www.osparna.com/">Osparna that supports investment diligence and decision-making</a>. The idea is that we should take a systematic approach to decision making around investments. Rather than only base them on our intuition, we decide what the basis for a good decision is, and then gather the information. There's a lot more to it...</div>
Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-55446336299044339932019-08-29T11:42:00.002-04:002019-08-29T11:42:54.464-04:00Black box<a href="https://news.ycombinator.com/item?id=20826979">https://news.ycombinator.com/item?id=20826979</a><br />
<br />
<span style="background-color: #f6f6ef; font-family: "verdana" , "geneva" , sans-serif; font-size: 12px;">Unpopular quote from my image and video processing professor - “The only problem with machine learning is that the machine does the learning and you don’t.”</span><br />
<div style="background-color: #f6f6ef; font-family: Verdana, Geneva, sans-serif; font-size: 12px; margin-top: 8px;">
While I understand that is missing a lot of nuance, it has stuck with me over the past few years as I feel like I am missing out on the cool machine learning work going on out there.</div>
<div style="background-color: #f6f6ef; font-family: Verdana, Geneva, sans-serif; font-size: 12px; margin-top: 8px;">
There is a ton of learning about calculus, probability, and statistics when doing machine learning, but I can’t shake the fact that at the end of the day, the output is basically a black box. As you start toying with AI you realize that the only way to learn from your architecture and results is by tuning parameters and trial and error.</div>
<div style="background-color: #f6f6ef; font-family: Verdana, Geneva, sans-serif; font-size: 12px; margin-top: 8px;">
Of course there are many applications that only AI can solve, which is all good and well, but I’m curious to hear from some heavy machine learning practitioners - what is exciting to you about your work?</div>
<div style="background-color: #f6f6ef; font-family: Verdana, Geneva, sans-serif; font-size: 12px; margin-top: 8px;">
This is a serious inquiry because I want to know if it’s worth exploring again. In the past university AI classes I took, I just got bored writing tiny programs that leveraged AI libraries to classify images, do some simple predictions etc.</div>
<br />
The complaint that a neural network, or deep learning model, is a "black box", where you can't see why the model is making a particular output seems quite weak to me. Not being interested in it because of that is worse, because it is like saying that we shouldn't study neuroscience because we don't understand how memories are stored in the nervous system. The box is not black. It is just a very complicated box, with lots of weights and layers. You can look at the weights and inputs and layers and literally see exactly what simple math applied to the inputs led to the output. It might take you a while, but it is all there. We are building tools all of the time to make it easier to make this process more efficient, but, unlike a brain, no one is stopping you from reaching into the box and taking a look.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-65466199819746062722017-10-27T11:08:00.000-04:002017-10-27T11:08:19.013-04:00KerasI've been working hard to keep up to date with recent progress in machine learning. The progress in Deep Learning is exciting to me, especially because it builds on all of the things I learned about neural networks in 1995-99 in college in grad school (particularly some work with <a href="http://faculty.virginia.edu/levylab/">Dr. William Levy</a> at UVA), but never really had the opportunity to put into practice directly, except in some simple classifier type situations.<br />
<br />
In any case, last year I took a short course from <a href="https://minerkasch.com/">Miner and Kasch</a> taught by <a href="https://github.com/FlorianMuellerklein">Florian Muellerklein</a> that was quite good. You can check out the slides and code on <a href="https://github.com/MinerKasch/applied_deep_learning">github</a>. I'd previously been messing around with Torch, Caffe, TensorFlow, cuDNN and a variety of other libraries, while easier than the old days of finding eigenvalues in c++ or running out of MatLab, they required a lot of configuration and such. In the course, we jumped right into using <a href="https://github.com/fchollet/keras">keras</a>. Wow, so much easier. It's a bit like a Ruby on Rails for neural networks, giving you some sensible defaults and get going right away, minimizing common errors, but different in that it is just a simpler interface overlaying other libraries. If anyone is diving in this stuff, the keras path is the best path that I've tried.<br />
<br />
To help one out on this path, there are now a couple of books on keras. <a href="https://www.amazon.com/gp/product/1617294438/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1617294438&linkCode=as2&tag=letspushthing-20&linkId=d46c6bf90e9636fd207c3fe32fdd6209" target="_blank">Deep Learning with Python</a><img alt="" border="0" height="1" src="//ir-na.amazon-adsystem.com/e/ir?t=letspushthing-20&l=am2&o=1&a=1617294438" style="border: none !important; margin: 0px !important;" width="1" /> is the one that I recommend. It is written by <a href="https://twitter.com/fchollet">Francois Chollet</a>, who is the creator and maintainer of keras, and is now working for Google. Respective of that, I think it does the best job of communicating how the library is intended to be used, and puts things in the right context of experimentation with hyper-parameters and other topics that can take up all of your time.<br />
<br />
In any case, I was recently asked what I thought about the joint project <a href="https://aws.amazon.com/blogs/aws/introducing-gluon-a-new-library-for-machine-learning-from-aws-and-microsoft/">Gluon</a> from Amazon and Microsoft. I think it is trying to do the same thing as keras, and I am hoping that it doesn't mean more divergence in the stack. Already, the deep learning stack is pushing in the hardware direction. There were already a level of matrix math operations that were implemented in hardware, which is why Nvidia GPUs were such a early accelerator for progress based on similar needs in graphics processing, and AMD and Intel are not far behind, with Intel's acquisition of Nervana systems a key recent purchase in designing chips built for purpose. Google's TensorFlow Processing Units (TPUs) take this a step further, and are designed to push more of the TensorFlow code into hardware. Obviously Microsoft and Amazon don't want to be left behind, as they won't be buying TPUs for AWS or Azure. Even Tesla is looking at building their own chips for image analysis in cars.<br />
<br />
That leaves those of us at software level trying to find the right API to code against. Right now, the answer is for me keras, but keeping an eye on the whole stack is necessary to see what choice is the right one to make at the top of it.<br />
<br />Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-32149729619240615062017-07-23T18:51:00.000-04:002017-07-23T18:51:07.141-04:00Programming LanguagesWhy are some programming languages more popular than others?
The number one reason is libraries. If you are going to be building something for the iPhone, you would have used Objective-C and now Swift because they are supported the tools on that platform. If you are doing Microsoft development, you will probably have a similar preference for C#, because that's where the libraries needed to connect to various applications live. Python is successful in data science, because it has a great set of libraries for numerical analysis which have formed the basis for more complex. Ruby became quite popular because of the Rails framework. Java has a number of open source projects that have kept it alive, things like Lucene and Hadoop, so we can blame Doug Cutting for its continued existence (in addition to a virtual machine that supports a number of platforms). And, of course, we are all stuck with JavaScript because it is the only thing that runs in the web browsers- despite the numerous and ever changing set of libraries that JS developers have been rapidly moving through over the past five years.<br />
<br />
You'll notice that what is largely missing here are any features of the language, whether a language is functional or object-oriented, dynamic or compiled. In the post-modern programming world where things are built from other blocks, these kinds of features of the language tend to be dwarfed in importance, because the average enterprise project includes more code written elsewhere than it does code written specifically for the project. Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-70014172677359347722014-03-01T08:58:00.001-05:002017-07-23T18:41:13.574-04:00Another post about nothing<a href="http://blog.codinghorror.com/10-years-of-coding-horror/">This post</a> by Jeff Atwood really gets at why I don't blog much any more. There is a certain kind of pressure when people are actually or might be actually reading your blog that you don't get when it is just sitting there. It's the pressure that what I am doing at the moment is not as interesting or useful to other people as the things they are doing so I don't want to waste people's time even skimming it to decide that themselves. I also ran into some incidents where idiots took everything I said in the way they wanted to, or used it to strategic advantage against me to supposedly reveal my hidden preferences or lack of commitment to their chosen path in The Neverending Battle over Which Technology to Use. And then there are the other things, the Twitters and Google+... anyway, maybe I'll find something useful to say again someday. Just reading through my old ideas, there aren't too many embarrassing ones that would prevent me from ever looking at this stuff again. Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-58720455588158026842012-11-17T13:26:00.001-05:002012-11-17T13:26:37.181-05:00Subsidy<p dir="ltr"><a href="http://feeds.felixsalmon.com/~r/felix-all/~3/FOcvEnPAsnM/">Counterparties: The lessons of tuition inflation</a></p>
<p dir="ltr">Felix Salmon dismisses the effect of loan subsidies on rising tuition costs by pointing out that the loans get repaid with interest, so that means it is not a subsidy.  Besides being a very weak argument, whether or not it is labeled a subsidy is immaterial. It is credit which allows people to pay more. Without a cap, colleges wouldn't be able to charge as much, and hire so many useless administrators. The cost of college is out of control. Watching the online disruption is fun. Would love to find a way to get more involved....</p>
Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-43121645718161968072012-11-15T23:37:00.001-05:002012-11-15T23:37:21.193-05:00Money for MusicReading <a href="http://pitchfork.com/features/articles/8993-the-cloud/">this semi-widely shared piece</a> by Damon Krukowski- I really think the analogy is wrong. Pandora is like having your song played on the radio, not like selling 7" of vinyl. When I buy a piece of vinyl, I can listen to it as many times as I want, or at least until the grooves run down. He tries to compare the royalty for 7,000 plays of a track on Pandora to 7,000 sales of a single?? It's more like a radio station with 7,000 listeners playing it once.
How much is Galaxie 500 making in radio royalties? I haven't heard them on the radio for 15 years. They should be comparing their $1.21 to $0.00.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-89807090041395400742012-10-05T17:05:00.002-04:002019-08-29T11:43:56.448-04:00Domain ModelingShould the domain model be separate from the serialization format? For example, if you are using something like <a href="http://avro.apache.org/">Avro</a> or <a href="http://thrift.apache.org/">Thrift</a> as your serialization format, does it make sense to have all of the objects in your domain modeled as objects generated by these technologies or is it better to just model them as <a href="http://www.martinfowler.com/bliki/POJO.html">Plain Old Java Objects</a> and then find a way to serialize those?
<br />
<br />
This question has recently come up, and I find it reminiscent of the older question of whether your XML data model should be your domain model or whether your database model should your domain model. The basic difference effect of using this pure data model approach is that you end up with classes that do not contain any behavior- basically <a href="http://en.wikipedia.org/wiki/Struct_%28C_programming_language%29">structs</a>. In the other case, you end up with objects that have both data and behaviors that depend on that data. The additional wrinkle is that generating classes from schema, be it thrift, Avro, ProtocolBuffers or XML adds the additional wrinkle of working with generated code. You typically don't modify generated code, so this can make adding methods, functions, etc. to this data a little challenging.
<br />
<br />
There is a great body of knowledge out there that supports not putting a dependency in your code on a particular technology implementation. In particular, modeling your domain effectively and independent of other components follows a lot of <a href="http://en.wikipedia.org/wiki/SOLID_%28object-oriented_design%29">solid</a> principles, and lets you pursue a <a href="http://en.wikipedia.org/wiki/Domain_driven_design">domain driven design</a>. The opposite, an <a href="http://martinfowler.com/bliki/AnemicDomainModel.html">Anemic Domain Model</a> has its limitations- a procedural design, generally missing out on OO.
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://upload.wikimedia.org/wikipedia/commons/7/73/Maintaining_Model_Integrity.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://upload.wikimedia.org/wikipedia/commons/7/73/Maintaining_Model_Integrity.png" height="501" width="632" /></a></div>
So The unsatisfying answer, of course, is "it depends". To make this answer a bit more satisfying, we need to look at what it depends on, and how those dependencies affect the answer.
<br />
<br />
The first "depends" I would consider is the programming paradigm. If you are working in a functional programming paradigm, not just a procedural one, for example in a language like <a href="http://clojure.org/">Clojure</a>, separating data model from behavior is a natural pattern. Many of the object oriented advantages to a rich domain model simply do not apply. If you are working in an object oriented paradigm, for example in a language like Java, it makes more sense to put the behavior with the data.
<br />
<br />
These are still generalities though. Generated code is less of a challenge in languages other than Java. Some languages, such as C#, allow partial classes, so you can combine generated code with code that you write in the same class. Dynamically typed languages like Ruby provide a lot of flexibility, so having model classes that are generated by a framework like ActiveRecord, or a pattern like <a href="http://martinfowler.com/eaaCatalog/activeRecord.html">Active Record</a>)is not a problem, because the classes contain no code.
<br />
<br />
The question then comes up of things like the Hadoop MapReduce framework. While your code may be in Java, the programming paradigm of map reduce is functional. (Some would even say that it is giving us an opportunity to get away from the object oriented programming model- and that <a href="http://www.joelonsoftware.com/items/2006/08/01.html">it is sad that we produce CompSci graduates that only know Java</a>.) Thus the benefits of having behavior associated with your objects is greatly <i>reduced</i>.
<br />
<br />
However, in the particular case we are considering, we are not just working in MapReduce, we have a number of application layers that can use the domain object. We are working in Java, so we can't do a lot of cute things with partial classes or mixins. We do want to use Avro as a serialization format though, so there is some momentum for using that as the medium of our domain model. It forces a certain weakness into our domain model, and an awkwardness to our programming. It is not as noticeable when writing functional style Map Reduce code, but it is quite noticeable when we try to abstract behavior such as validation and mapping operations that naturally go on the model. We can get something close to the
<br />
<br />
Of course, if we go to a more pure, POJO domain model, it then becomes a little more tricky to go in reverse and make the serialization occur with Avro. Still thinking on this one...Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-20281767192817537932012-08-21T17:00:00.000-04:002012-08-21T17:00:04.385-04:00PatentsThe patent system, and intellectual property rights claims in general, are out of control. You may think I am referring to the current Apple v. Samsung case where the invention of the rectangle is in question. I am actually more concerned about the <a href="http://online.wsj.com/article/SB10000872396390444233104577593590556900030.html">White v. Heinz case</a>.
We get no benefit from allowing these independent inventors to hold companies who came up with the same idea hostage...just as we get no benefit from more organized patent trolls. It's a game lawyers are playing to exploit the legal system to take money from people.
I came home yesterday and my daughter had created a shop for Patented fashion designs, along with a licensing scheme if you actually wanted to make it. The sewing machine was sitting there unused. The designs were great though, but the inspiration was clear.
I guess <a href="http://www.allremix.ru/watch-the-series/">Everything is a Remix</a>.
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.heinzketchup.com/img/dslocator/ds_pack.png " imageanchor="1" style="clear:right; float:right; margin-left:1em; margin-bottom:1em"><img border="0" height="213" width="296" src="http://www.heinzketchup.com/img/dslocator/ds_pack.png " /></a></div>
Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-28843765211914424262010-12-25T01:13:00.002-05:002010-12-25T01:14:43.468-05:00BooksIt's been a busy year for me, but more troublingly, a year in which I have been full of 75% complete ideas that haven't been worthy of their own posts. I have done a fair bit of reading though. Here are some of my favorite books that I've read this year (regardless of whether they came out this year). Maybe you can use some of the % back from your Amazon Visa card to buy them.<br /><br /><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Webber, Paristaditis, Robinson <a href="http://www.amazon.com/gp/product/0596805829?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0596805829">REST in Practice: Hypermedia and Systems Architecture</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=0596805829" style="border-bottom-style: none !important; border-color: initial !important; border-left-style: none !important; border-right-style: none !important; border-top-style: none !important; border-width: initial !important; cursor: move; margin-bottom: 0px !important; margin-left: 0px !important; margin-right: 0px !important; margin-top: 0px !important;" width="1" /></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">I've been waiting a long time for this book (ever since <a href="http://www.infoq.com/articles/webber-rest-workflow">How to GET a Cup of Coffee</a> came out). Let's just say I have been going around for five years talking REST. I didn't actually know WTF I was talking about until I read this book and started to understand HATEOAS. It came out while I was struggling with a "RESTful" API that was singularly non user friendly. I still have a fundamental issue with REST where it seems like it is jamming too much into the six verbs. I was hoping this book would dissolve it, but it's actually written at too basic of a level. </div><br />Morville, <a href="http://www.amazon.com/gp/product/0596802277?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0596802277">Search Patterns: Design for Discovery</a><img src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=0596802277" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /><br />If you are working on application that includes search and discovery features, this book sets forth a great pattern language. There are tons of examples and ideas here. <br /><br />Anderson, <a href="http://www.amazon.com/gp/product/0984521402?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0984521402">Kanban</a><img src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=0984521402" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /><br />I'll come right out and say it here publicly on the Internet: Scrum is not a good software development method. Scrum masters are not useful. I still like XP. Kanban and Lean are much more sensible methods to go with XP practices than Scrum. It's really way too much to get into here, but having spent the past 7 years reading on the Theory of Constraints, Toyota Way, Lean, etc. this is a method that makes so much sense to me I think I've already been doing it without having a name for it. Yes we Kanban!<br /><br />Fried, Hannson, <a href="http://www.amazon.com/gp/product/0307463745?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0307463745">Rework</a><img src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=0307463745" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /><br />These guys just have no patience for all the non-value added aspects of work (shades of Lean and 'muda'). They also have a way to ensure that the work we do maximizes the opportunities for innovation, creativity, and even beauty, ultimately leading to get us towards our own happiness. Work should make you happy.<br /><br />Matt Ridley, <a href="http://www.amazon.com/gp/product/006145205X?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=006145205X">The Rational Optimist: How Prosperity Evolves</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=006145205X" style="border: none !important; margin: 0px !important;" width="1" /><br />The author of the awesome <a href="http://www.amazon.com/Red-Queen-Evolution-Human-Nature/dp/0060556579/ref=pd_sim_b_3">The Red Queen</a> comes back with treatise on why all the things that the whiners keep themselves busy whining about aren't really too bad. In all of the hand wringing about the minor issues of the day, we are vastly better off than people at any time in history. His thesis is that specialization and markets are the prime movers of progress, and the world might not falling apart at the seams as much as it sometimes seems. Actually made me hopeful about life on Earth, which is a rarity.<br /><br />Sam Harris, <a href="http://www.amazon.com/gp/product/1439171211?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1439171211">The Moral Landscape: How Science Can Determine Human Values</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=1439171211" style="border: none !important; margin: 0px !important;" width="1" /> <br />This is a fascinating book. It's bit harsh on the approach for dealing with religious people, but it gets to the heart of where we can find areas of moral agreement, based on some simple starting points. I thought it would devolve into utilitarianism, but Harris takes it in another direction all together. He directly attacks the distinction between values and facts. It really made me question some of my "cultural relativism". I saw Harris speak earlier this year, and it was quite good. Plenty of his <a href="http://www.youtube.com/watch?v=sTKf5cCm-9g">videos out on YouTube</a> if you want the lite version. This will be offensive to many, but it's worth at least figuring it how to address these arguments if you disagree with them.<br /><br /><a href="http://www.amazon.com/gp/product/1401303420?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1401303420">Ricky Gervais Presents: The World of Karl Pilkington</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=1401303420" style="border: none !important; margin: 0px !important;" width="1" /><br />This is a bit funny. You probably have to heard the podcasts or show a couple of times to appreciate it, but it is really quite good with lots of laughs at the expense of the indefagitable Pilkington.<br /><br />Lidwell, Holden, and Butler <a href="http://www.amazon.com/gp/product/1592535879?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1592535879">Universal Principles of Design, Revised and Updated: 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions, and Teach through Design</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=1592535879" style="border: none !important; margin: 0px !important;" width="1" /><br />I love reading about design. I am in the middle of the new Frederick Brooks book on the design of design. I am still wondering how much reading about design it will take for me to eventually be a novice level design person.<br /><br />Doug Lemov, <a href="http://www.amazon.com/gp/product/0470550473?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0470550473">Teach Like a Champion: 49 Techniques that Put Students on the Path to College</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=0470550473" style="border: none !important; margin: 0px !important;" width="1" /><br />This is one thing missing from the teaching profession. In the manner of a software patterns book, this covers a set of techniques for running an effective classroom. Buy copies for your kids' teachers. <br /><br />Jeffrey Pfeffer, <a href="http://www.amazon.com/gp/product/0061789089?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0061789089">Power: Why Some People Have It and Others Don't</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=0061789089" style="border: none !important; margin: 0px !important;" width="1" /><br />Just be aware that I've read this book and have been trying to internalize it. At some point, you have to achieve a balance between getting what you want and being right. Even if you don't have the <a href="http://www.amazon.com/gp/product/1420935011?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1420935011">Will to Power</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=1420935011" style="border: none !important; margin: 0px !important;" width="1" />, this will help you understand what those with the ambition are up to.<br /><br />Ackoff, <a href="http://www.amazon.com/gp/product/1576753301?ie=UTF8&tag=letspushthing-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1576753301">Beating the System: Using Creativity to Outsmart Bureaucracies</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=letspushthing-20&l=as2&o=1&a=1576753301" style="border: none !important; margin: 0px !important;" width="1" /><br />Some good ideas here. <br /><br />That's just a quick sampler of what I can remember, which gets smaller every year. Enjoy!Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-74082597992140856332009-10-26T23:01:00.001-04:002009-10-28T17:45:47.179-04:00Tips for using Rails migrations The concept of a migration is that each change that is made to the structure of the database is captured in a version controlled script. Any time you want to create a table, add a field, change a field type, etc., this is done via a migration. Migrations are generated along with models, views, and controllers when a Rails scaffold is created to manage a resource. In addition, migrations are used to create a test database for each user which allows for rapid, automated quality checks on the application. If you have ever worked on a project where it is difficult to create a development "sandbox" database for each developer, where databases are copied via backup processes that leave weird permissions behind (I'm looking at you SQL Server), or where keeping the development database in sync with the application is something you even have to think about, you will easily see benefits from migrations.<br />
<br />
Using Rails migrations can be a bit of change for teams. In some cases, developers are not very familiar with relational database concepts and object modeling principles and can create some overly complex and/or inefficient structures. Having the overall model reviewed by a database modeling expert and an object modeling expert can be quite helpful. I recommend Coad's "Java Modeling in Color with UML", Fowler's "Analysis Patterns" and David Hay's "Data Model Patterns" for getting yourself up to speed with this. However, it is also worth communicating with the rest of the team that may be touching the model. <br />
<br />
Here are a few things we've done in the process of keeping migrations under control:<br />
<br />
<b>Don't edit an existing migration, create a new one.</b><br />
Editing an existing migration defeats the whole point. You might be able to get away with it if no one has run it. One obvious exception to this is if someone creates a migration that doesn't run and checked in. They must get the pig.<b><br />
</b><br />
<br />
<b>Update and run new migrations before you check in a migration.</b><br />
While this might seem obvious, in many cases, it is easy to check in a new migration file to the repository without seeing if someone else has created one that does something that conflicts with yours, since the names will be different. If you don't run the migration, you risk doing something with your migration that causes issues. If you have already run your own migration locally, but not checked it in, and you get an update of a new migration, you need to roll back your own migration and change the numbering on it so that it runs after the migrations you just received from your version control system.<br />
<br />
<b>Check in the models that go with the migration when you check in the migration</b><br />
One key benefit of migrations is keeping the code changes in sync with the database changes. Add new tables and models in a single, logical commit.<b> </b><br />
<b><br />
</b><br />
<b>Assign models to class owners</b><br />
The feature of XP that I have been least successful with is the concept of common code ownership. It is one of the easiest practices to apply in theory, but without the sense of collective responsibility and the compensating practices of pair programming and continuous integration, it can be problematic. In larger teams, it often makes more sense to have people or subteams assigned to manage a particular model so that changes can be coordinated. In theory, there can be enough communication to make it work. In reality, many teams are full of introverts that would rather rewrite huge swaths of code (and tests) than ask a question.<br />
<br />
<b>Communicate</b><br />
Changes to the object model can impact the whole application. These changes should be communicated to the rest of the team so that the reasoning behind them can be better understood and the data can be used correctly. There is a risk of someone criticizing or trying to change your proposal, but this is ultimately a lower risk than building something that others fail to understand.<br />
<br />
<b>Roll up all migrations into a big file after a major release</b><br />
<b> </b>Once you've been going in your project for a while and get around the 100-150 migration point, it is probably worth it to roll the migrations prior to the current point into a single migration.<br />
<br />
<b>Generate SQL to run</b> <b>on the production database</b><br />
<b> </b>In many cases it is preferable to generate SQL to run on the production database, as opposed to running the migrate task directly against the database. This also allows for some more extensive code review on structural changes.<br />
<br />
<b>Don't forget to create constraints and indexes (indices?)</b><br />
If you are using foreign keys in your database, it's often worth the cost to create a constraint on the column to prevent any data integrity issues. <b> </b>It's also a good practice to create an index on those columns to speed joins.<br />
<br />
This is really just the tip of the iceberg, but it's a yummy water ice iceberg. Learning to use database migrations is a key skill, not just for Rails development, but for any agile development. A project which cannot create its database from scripts accessible to every developer is missing something important. By breaking those scripts down into small chunks and coordinating them with changes to the code, the situation where the application and database are out of sync simply does not exist.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-33062974990934387082009-10-07T00:00:00.000-04:002009-10-07T00:00:40.125-04:00Whose Success?There is sometimes a quandary between whose success we seek.<br />
<br />
Acting as a consultant, you obviously want your own work to be considered successful, so that you will induce others to hire you. Getting there isn't so easy though. We often look downstream to the customer's customers. By helping meet the customer's customers' needs, we hope to make our immediate customers successful. The catch comes when something the immediate customer is doing is preventing them from satisfying their customer- and they don't want to change it. If we make the customer's customer successful at the cost of making the customer unsuccessful, we're probably not going to be successful.<br />
<br />
Remember the <a href="http://www.amazon.com/Secrets-Consulting-Giving-Getting-Successfully/dp/0932633013">secrets of consulting</a>- you have to help people learn to solve their own problems, you can't always solve their problems for them. You can be satisfied with your accomplishments, regardless of whether they give you credit. The ideal form of influence is to help the customer see the problem more clearly, but let them decide. Finally, if you want to understand a dysfunctional system, follow the money!Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-33969240343448477102009-09-28T15:59:00.001-04:002009-09-28T16:01:11.349-04:00Seed data in Rails 2.3.4Very nice addition to Rails as part of the security updates in 2.3.4- an easy way to load seed data.<br />
<br />
<pre>Added db/seeds.rb as a default file for storing seed data for the database. Can
be loaded with rake db:seed (or created alongside the db with db:setup). (This
is also known as the "Stop Putting Gawd Damn Seed Data In Your Migrations"
feature)</pre><br />
I like this on several accounts, not least of which is it invalidates my <a href="http://lptf.blogspot.com/2009/07/rails-fixtures-and-seed-data.html">previous post</a> on the subject.<br />
<br />
In the meantime, there are several plugin based approaches to making this work, for example <a href="http://www.intridea.com/2008/4/19/seed-fu-simple-seed-data-for-rails?blog=company">seed-fu</a>.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-63619198762241180662009-09-10T11:20:00.004-04:002009-09-10T23:41:53.660-04:00Default route...One of the things that has been bothering me in Rails since everything switched to resources is how we lost the easy way any public method in the controller became an accessible URL without having to write a rule for it. For example, in a person controller, I could define a search method like this:<br />
<pre>def search
results = Person.seach(params[:q])
end</pre>And it would be accessible, just like that. <span style="font-family: ";">http://myapp/people/search?q=matt</span><br />
I even wrote a simple router in Java to do the same thing, because it just makes it so easy to navigate the application code.<br />
<br />
The poor design of<span style="font-family: ";"> map.resources<span style="font-family: inherit;"> messes this up. By adding the route pattern <span style="font-family: ";">:controller/:id <span style="font-family: Arial,Helvetica,sans-serif;">it starts with the default assumption that the </span></span></span></span><span style="font-family: ";">search?q=matt </span><span style="font-family: inherit;">part of my URL is an ID. Is it so hard to put an action name in there? In any case, </span><br />
<pre>map.connect ':controller/:action'</pre>goes above all of the<span style="font-family: ";"> <span style="font-family: courier new;">map.resources </span><span style="font-family: inherit;">calls in my app. You might want to put it in your application too.</span><br />
</span>Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-29593306075556801922009-07-11T22:58:00.001-04:002009-09-28T16:02:08.945-04:00Rails Fixtures and Seed DataOne of the things that makes Ruby on Rails a great web framework is the quantity of tools available that are focused around the task of making web applications easy to build and encourage good software engineering practices.<br />
<br />
Rails makes it really easy to test by building in a testing framework. One aspect of this is what are called "fixtures", data described in CSV or YML formats that is loaded into the database for each round of testing. Tests live at both the unit (model) and integration (controller) level. End to end tests that involve the view are traditionally run from an automated tool, such as Selenium or HPQC (ha). <br />
<br />
UPDATE ALERT... Rails 2.3.4 <a href="http://lptf.blogspot.com/2009/09/seed-data-in-rails-234.html">has a nice, new way to do this</a>.<br />
<br />
<br />
Fixtures also make it easy to load seed data into the database. You can do this with migrations, but it gets messy, especially if you use ActiveRecord to do it, and then you delete a model class sometime in the future, it forces you to do an edit of an old migration, which means you might as well roll them all up. I prefer:<br />
<div style="font-family: "Courier New",Courier,monospace;"></div><div style="font-family: "Courier New",Courier,monospace;">rake db:fixtures:load<br />
</div><div style="font-family: "Courier New",Courier,monospace;"></div><div style="font-family: "Courier New",Courier,monospace;"><span style="font-family: inherit;">Which runs:</span><br />
</div><pre>namespace :fixtures do
desc "Load fixtures into the current environment's database. Load specific fixtures using FIXTURES=x,y"
task :load => :environment do
require 'active_record/fixtures'
ActiveRecord::Base.establish_connection(RAILS_ENV.to_sym)
(ENV['FIXTURES'] ? ENV['FIXTURES'].split(/,/) : Dir.glob(File.join(RAILS_ROOT, 'test', 'fixtures', '*.{yml,csv}'))).each do |fixture_file|
Fixtures.create_fixtures('test/fixtures', File.basename(fixture_file, '.*'))
end
end
end
</pre><span style="font-family: inherit;">This will load all of your test data into the development database (or production, if that's what your RAILS_ENV is set to). Jeffrey Allan Hardy documented <a href="http://quotedprintable.com/2007/11/16/seed-data-in-rails">this</a> pretty well, and added a separate rake task that will do this from a separate directory from your test fixtures, which is a great idea. It goes like:</span><br />
<span style="font-family: inherit;"><br />
</span><br />
<pre><code class="ruby">namespace :db do
desc "Load seed fixtures (from db/fixtures) into the current environment's database."
task :seed => :environment do
require 'active_record/fixtures'
Dir.glob(RAILS_ROOT + '/db/fixtures/*.yml').each do |file|
Fixtures.create_fixtures('db/fixtures', File.basename(file, '.*'))
end
end
end</code></pre><div style="font-family: "Courier New",Courier,monospace;"></div><div style="font-family: "Courier New",Courier,monospace;"></div><br />
It's pretty easy and keeps this stuff out of the migrations.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-56716871520950048702009-06-02T23:26:00.000-04:002009-06-02T23:26:02.039-04:00Thank You for Agreeing<div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/_i3E7tAYvk0Y/SiXg2b1ucxI/AAAAAAAAARs/GJh_EE8UlcM/s1600-h/thank+you+for+arguing.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/_i3E7tAYvk0Y/SiXg2b1ucxI/AAAAAAAAARs/GJh_EE8UlcM/s320/thank+you+for+arguing.jpg" /></a></div>I've been reading "Thank You for Arguing" tonight, thanks to a tip from the excellent time sink known as <a href="http://news.ycombinator.com/">Hacker News</a> . The first page of the introduction has a pretty telling tip.<br />
<br />
"Try this in a meeting: Answer someone who expresses doubt over idea with, 'Okay, let's tweak it.' Now focus the argument on revising your idea as if the group had already accepted it. This move is a form of concession- rhetorical jujitsu that uses your opponent's move to your advantage."<br />
<br />
Heinrichs gets to some much deeper points as well. One key thing to focus on is getting the outcome you want from the discussion, and not on scoring points to "win" the argument.<br />
<br />
<a href="http://3.bp.blogspot.com/_i3E7tAYvk0Y/SiXjT72s5gI/AAAAAAAAAR0/R2RzvKbYVdM/s1600-h/mt-logo.gif" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://3.bp.blogspot.com/_i3E7tAYvk0Y/SiXjT72s5gI/AAAAAAAAAR0/R2RzvKbYVdM/s320/mt-logo.gif" /></a>I wanted to draw a tenuous connection here, between these rhetorical skills, to which Heinrichs gives a fabulous introduction, and the listening, or "receiving" skills that the <a href="http://www.manager-tools.com/">Manager Tools</a> guys have <a href="http://www.manager-tools.com/2008/06/how-to-coach-directs-on-interpersonal-skills-part-2">focused on</a> . They put communication skills at the foundation of developing interpersonal skills, and they put listening skills at the foundation of communication. In their characteristic way, Horstmann and Auzenne give some concrete examples of specific, measurable behaviors that we can use to evaluate and improve our communications, by strengthening the feeling in the person we are communicating with that we understand them. <br />
<br />
One of the items on their list of things to measure is using "Thank You" to begin a reply. (I said it was a tenuous connection.) Other things they recommend include "No interruptions", "Agreeing without qualification", "No buts or howevers", and my personal favorite, "smiling". By making simple gestures like this, the person that you are communicating with is shown that you respect them as a person. If we cut someone off, we are going to have a much harder time persuading them of anything, because they are likely to think that we don't understand their argument and they are also probably not going to like us very much.<br />
<br />
What makes this even more challenging, is that in many situations, the person that you are doing a poor job of listening to is not going to know exactly what it is you are doing that they don't like, but they are just going to think you are disagreeable or contrary. You may end up getting negative feedback from managers that doesn't make sense- vague comments on how you are perceived. This is where I like to tie in things like the Manager Tools material to figure out what good feedback for those situations would look like. Respect for people is a key aspect of successful systems, such as the <a href="http://www.toyotatr.com/eng/toyotaway.asp">Toyota Way</a>. However, it is also key in being persuasive. To really change someone's mind to what we think is an amazing thing. It means teaming up and getting on the same side.<br />
<br />
"Thank You for Arguing" is filled with little gems that you can use to show respect to others, by offering them a choice, instead of negating what they say directly. Simple phrases like, "On the other hand, it could be that..." create an option for the listener, instead of putting them in a box. It offers them the opportunity to admit that they aren't 100% sure about their view. Moving the argument into the future tense also affords the chance to remove some of the contentiousness about any concerns, while defining the issue in a way that makes your point clear. <br />
<br />
Rather than despise rhetoric for muddying the waters of rational debate, we should use it as a tool to achieve our goals, as long as we can keep it in a framework of respect for others- by combining it with the listening/receiving skills that are the foundation of communication.<br />
<br />
Other good books on similar subjects are "Influence" and "Yes!" by Cialdini, "Difficult Conversations" and "Getting to Yes" out of the Harvard Negotiation Project that was led by Fisher the last I looked into it, "Never Eat Alone" by Ferazzi, and the classic "How to Win Friends and Influence People" by Dale Carnegie. My wife read the last on her Kindle recently, and the chapter on married couples is well worth it.Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.comtag:blogger.com,1999:blog-15324911.post-47080823175234261332009-06-01T10:23:00.001-04:002009-06-01T10:28:10.231-04:00Passionate Programmer<div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/_i3E7tAYvk0Y/SiPkIM7KCMI/AAAAAAAAARk/szgNA0RRNjA/s1600-h/iloveprogrammers.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://4.bp.blogspot.com/_i3E7tAYvk0Y/SiPkIM7KCMI/AAAAAAAAARk/szgNA0RRNjA/s200/iloveprogrammers.jpg" /></a></div>Chad Fowler's <a href="http://www.amazon.com/Passionate-Programmer-Creating-Remarkable-Development/dp/1934356344">The Passionate Programmer</a> is a rewrite of <a href="http://www.amazon.com/Job-Went-India-Pragmatic-Programmers/dp/0976694018">his earlier book</a> on how to improve your career in software development, not a guide to finding love using Ruby scripts to mine craigslist. I read it anyway.<br /><br />Today I'd like to take a quick look at #21, "The Daily Hit". The concept is to, each day, "have some kind of outstanding accomplishment to report to [your] manager, some idea [you] had thought of or implemented that would make [your] department better."<br /><br />There are a couple of interesting aspects to this. First, is it forces you to break work down into small enough chunks that you can get something done in a day. The second is that improvement part. Thinking of, and implementing, small improvements every day is closely related to a Lean concept called <a href="http://en.wikipedia.org/wiki/Kaizen">Kaizen</a>. The third is that reporting and tracking these things really helps demonstrate your value to the organization. While it seems annoying, and perhaps a bit like boasting, to have to market yourself to your boss or customer or whomever, it actually creates a more positive relationship, because it forces communication about positive subjects. Having at least one of these ready for a meeting like a Daily Scrum is a good idea- it can even set the tone of accomplishment.<br /><br />Keeping it positive is really important. A ratio of nine positive to one negative feedback item is what the Manager Tools guys recommend. A key aspect of making Lean work is that it is not about blame. It is about admitting that things are never perfect, or even if they are, the context will change and make them imperfect. We face a lot of people challenges in the world of work. There are constant machinations of status and power, siphoning off of high profile work and dumping of boring work. We can't let this prevent us from doing good things. We need to make the assumption that there are more good people in the environment than bad, and building up a network of allies by providing them with support when needed, while also marketing the fact that we are providing that support, will make sure that there is someone out there who has our back. Help people get better.<br /><br />So, get your daily hit. Communicate what your up to. Do it again tomorrow...Matt McKnighthttp://www.blogger.com/profile/16098483018096096360noreply@blogger.com