Tegu in Action

I borrowed the idea of the Dreyfus Model from Andy Hunt’s Pragmatic Thinking and Learning. Since I keep asking people where they’re at on a particular skill on that model, I thought I’d outline it in a place where I can reference it. It’s probably good to know in general, because it tends to reduce frustration and focus learning styles by knowing what stage I am on each skill I need to have. The model as I remember it is basically:

• Novice: looking for recipes
• Advanced Beginner: able to work with the reference material nearby, not able to see the big picture
• Competent: Mostly able to work without having to re-architect very often
• Proficient: Able to work at a steady pace, able to self-correct when working
• Expert: People are seeking you out in this field

So, for what it’s worth, there it is.

I keep reminding myself about the path of mastery. The path is:

• from novice (looking for recipes)
• to advanced beginner (avoiding the big picture)
• to competent (able to get some work done)
• through proficient (able to self-correct)
• finally arriving at expert (someone whose advice is sought after)

This is the Dreyfus model that Andy Hunt talks about in his book, "Pragmatic Thinking and Learning":http://www.pragprog.com/titles/ahptl/pragmatic-thinking-and-learning.

I remind myself of this model, because I am awash in decades of ideas, all of them with merit, all of them begging to be tried out. I think I can say I'm generally leaving the advanced beginner stage, but I'm not really fully qualified as competent yet. I have written plenty of code using many ideas, but there is a general foundation that I keep piecing together from fresh ideas and obscure algorithms. Things like keeping a bloom filter handy for my larger algorithms, extracting methods into common libraries, building basic patterns and extending them with mixins and blocks. It's a fun world to be working in, but certainly an easy one to get lost in.

I think, to be fair to myself, that I'm going to write down the basic landmarks, and I'll see if they're recognizable over time.

Machine learning, artificial intelligence, collective intelligence, and other related terms all get to be called machine learning in my language. It's just easier that way. I say this, so you know that I'm speaking broadly and loosely.

There are people in different camps. Some people have assembled around a particular algorithm: neural networks or Bayesian inference. Some people have mastered basic concepts, like symbolic representations. There are hot new areas, such as building ontologies from the semantic web. There are old-school camps based in mathematical statistics, people using Principal Component Analysis to clean up their models. There are the systems science people, who sometimes get stuck looking at the forest and forgetting the joy of studying the DNA on some of the bark of some of the trees. A good example of this is René Thom's good ideas with catastrophe theory being over-used and loosely applied by Arnold, creating a bad name for an interesting topic.

There are good resources out there, available. I've enjoyed Stanford's "course on machine learning":http://see.stanford.edu/SEE/courseinfo.aspx?coll=348ca38a-3a6d-4052-937d-cb017338d7b1. Despite my religious affiliation with Ruby, I should probably have started out with R and Octave, and come back later to building libraries and uses for the subject in Ruby. For instance, the Gaussian Process takes a few minutes to implement in R, and probably not too much longer with Octave (I grabbed the code the other night, but 2 AM comes quickly sometimes and I left that project half-done). That's something I've spent hours on, getting ready to get ready with Ruby.

I really think that one of the more important works that I ever found was Judea Pearl's "Probabilistic Reasoning in Intelligent Systems":http://www.amazon.com/Probabilistic-Reasoning-Intelligent-Systems-Plausible/dp/1558604790/ref=pdbbssr_1?ie=UTF8&s=books&qid=1235113202&sr=8-1. I'll tell my grandkids some day about living in Portland, broke, spending my literal last dollar on that book. They won't appreciate it. Neither could you, unless you pick that book up and realize how carefully Pearl walks you through some very powerful ideas. I'm interested in looking up his Causality book some day when I can see daylight again.

I think that anyone interested in this stuff should use the systems science people as a compass: Jerry Weinberg, Herbert Simon, and Arthur Hall will create a very good canopy to help keep things going. Peter Senge's good for a lay-person's perspective (Senge isn't a lay person, but he's not teaching the computational models, just the big-picture archetypes and ways of thinking).

The more I revolve around this subject, the more I see Bayes as the Swiss Army knife, and an understanding of reinforcement learning as the cell phone, if I can mix metaphors here. All processes seem to at least tip their hats to both of these foundational concepts. Basic statistics are useful: Chi-square analysis, AIC, and understanding probability distributions. Linear Algebra is found everywhere, if you're not going to learn it, make sure you have something like linalg working in your Ruby environment. I have four or five linear algebra books lying around to bail me out every time I go into a subject where I have no business playing. I think Kline's "Calculus":http://www.amazon.com/Calculus-Intuitive-Physical-Approach-Second/dp/0486404536/ref=pdbbssr_1?ie=UTF8&s=books&qid=1235113656&sr=8-1 is the best book I could use to get a grasp on that. I'll see if my opinion stands after a deeper dive into the book.

Finally, I think that the Gaussian Process and Support Vector Machines are going to be my bread and butter for many things. I thought neural networks would do that for me. I'm not opposed to them, I've messed around a little with FANN, and I'm pretty sure that will make it into my AMI. But there is a voodoo science behind constructing an effective neural network that I don't have to mess with as much from SVM. And, since the GP gives me an explicit idea of how my regression and classification is doing at any given moment, I'm excited to keep that handy as well.

So, this may be more of an affirmation that I'm stuck in Advanced Beginner Land with this domain, but hopefully I'm moving along to a defensible competency soon.