Journey in Instruction

What is the best way to avoid analysis paralysis and ensure results are measured? Fail first. Work as fast as you can to fail in a measurable way. The focus initially should be on your ability to measure the failure – all the important aspects of it. With this understanding of what matters and a prototype, you can now begin making incremental improvement and testing it.

This is not meant to conflict with Philip Crosby’s recommendation to get it right the first time. The advice to fail first is meant for one situation and the advice to get it right the first time is meant for a different situation. If you already know what the requirements are (i.e., there is no need for analysis), then follow Crosby’s advice and conform to requirements. However, in many situations, it is not clear what the requirements are and it is in these situations that the fear of analysis paralysis comes about. In such situations, my advice is to focus on failing first.

In the #lrnchat last night, there was discussion about “selling” training to management. I saw this morning that there is an active discussion in the ASTD National LinkedIn group responding to a request for help in presenting on the Return on Investment (ROI) of training. I thought I’d take a step back and reflect.

Training professionals assume that they need to sell training proposals using ROI calculations. While the ROI calculation may feel like a familiar financial concept to managers, it is not a recommended technique for analyzing and evaluating investment alternatives in managerial accounting. This is because it can lead to incorrect investment decisions due to the fact that it ignores the time value of money.

The recommended technique for analyzing investment alternatives is the discounting technique. Two different methods fall under this technique. The first is the net present value (NPV) and the second is the internal rate of return (IRR). The NPV is the Holy Grail as it is theoretically superior to other techniques, but the IRR is easier to compute and interpret.

The typical capital budgeting techniques include: discounting, ROI, payback, and urgency. Payback is perhaps the simplest calculation to interpret: When do I get my money back? However, like ROI, payback ignores the time value of money. The urgency technique simply requires an estimation of potential costs, which are usually the focus of training interventions (e.g., compliance training) — stop the financial bleeding. An important observation is that all the capital budgeting techniques except the urgency technique require forecasts of future cash flows.

Beyond the urgency perspective

If a manager requests an ROI, this is essentially a request for a quantified cost / benefit analysis. If you can’t convert the benefits into a quantitative form, then another representation for those benefits is probably required that accounts for the intangibles (i.e., stories, examples, etc.).

If you want to propose a new technology project that will require a capital outlay, then a NPV calculation using discounted cash flows is recommended. These cash flows could be increased sales (easiest to interpret) or more likely expressions of productivity improvement (as is typical of technology). Quantifying these productivity increases is important for capital decision making, but expressing these increases in ways that users understand is perhaps more important (i.e., WIIFM). How will the new technology not add to the work the user already does? How will it save them time? Ideally, it would save enough time that some of that time could be given back to users while some of it could be reinvested by management into greater increased earnings.

From ideas to implementation plans

All of this quantified talk is really only important assuming a solid understanding of the problem that includes clear distinctions of its symptoms. The training project or performance improvement technology should be a recommended solution to the identified problem or opportunity. This is the first step. The typical strength, weakness, opportunity, and threat (SWOT) analysis ideally comes next. If the idea seems worthwhile based on the SWOT, a cost/benefit analysis is necessary to ground the idea. If it is still worth pursuing and involves significant capital, then a net present value based on discounted cash flows should be calculated. That is my theoretical model of how an idea turns into plan that is ready to be implemented.

As I read Bush and Mott’s 2009 article on the transformation of learning, specifically their words related to the learner-centricity principle, I can’t help but reflect on the YouTube videos I watched to learn about the neuroanatomy of the brain.  In particular, I can’t help but reflect on the comments I observed from students in neuroanatomy classes praising the video’s author for videos, which students concluded enabled them to complete their courses.  The video author, Walid Aziz Basharyar (or hyperhighs), is in fact a student himself.  He is also an artist who takes pride in his work.

The learner or viewer of Walid’s videos, which average about 4 minutes in length, have the ability to pause, seek (limited only by bandwidth), or skip to other videos.  The learners clearly enjoy watching these videos and attribute their success to them:

“Thank you so much for helping pass my last A & P test. Your illustrations were great and the explantions even better. I take A & P online and have never heard some of the terms spoken out loud. Thanks again! Debby.“

Learners also attribute a sense of intrinsic motivation to the videos, ”Can’t wait to take neuroanatomy next year! Lord knows this is facinating stuff! Combine this with genetics…whaoo!“, or reflect on their previous classroom experiences, “Neuro was very boring when I was at college.”

Walid clearly enjoys creating these videos.  This enjoyment, typical of an artist doing his work, seems to be indifferent to the praise he receives, “Very well done tutorials, thank you!” or “u are so artistically talented!”

Where are all the other artists who could benefit from these positive comments, which undoubtedly would feel better than the comments they might receive on facebook?  These videos are clearly contagious for viewers who have found them, but why aren’t they contagious for other artists to do similar with other subject matter?  The technology requirements seem to be affordable and available: a pen, graph paper, a camera, and music player.

Bush, M. D., & Mott, J. D. (2009). The transformation of learning with technology: Learner-centricity, content and tool malleability, and network effects. Educational Technology, 49(3), 3-20.

The child forced into premature concern with economic remote results may develop a surprising sharpening of wits in a particular direction, but this precocious specialization is always paid for by later apathy and dullness.

–John Dewey, How to Think, 1910, p. 219

Trying to keep track of papers and presentations (or anything else) is tricky.  Here’s my approach that seems to be working:

general form: 1stauthor_tag1_…tagn_journalorconferenceacronym_year
example: clark_motivating_performance_pi_1998.pdf

While the tags sometimes relate to the title, they sometimes don’t.  I keep filenames and titles separated.

What works for you?

I put a dollar in one of those change machines.  Nothing changed.  ~George Carlin

Seems everyone is being forced to change.  The world we live in is more connected than ever before and it seems uncertainty and unanticipated challenges are everywhere.  In the business world, Clark and Gottfredson (2008) believe that enduring competitive advantage is built on an organization’s ability to respond to adaptive challenge.  In the military, the ambiguous, changing nature of the operational environment requires the development of agile, adaptive leaders (Shadrick, Lussier, & Fultz, 2007).  So how are we developing adaptive thinking? It seemed to me like a topic uniquely suited for simulations that are able to provide virtual worlds.  However, the folks at the Army Research Institute (ARI) are a bit cautious on the ability of simulations to do this.

The Army Research Institute believes they’ve found a successful approach to  training adaptive thinking skills and it is not simply placing participants in a realistic, unstructured, whole-task environment using some form of costly, high-fidelity live, virtual, or constructive simulation (Shadrick, Lussier, & Fultz, 2007).  The training approach ARI is promoting is called Think Like a Commander (TLAC) and is based on a framework of deliberate practice.  TLAC is not your traditional software simulation.  It is a set of cognitive drills that present learners with a scenario in a 2-4 minute audio-video file.  After learners are introduced to the scenario, they are asked to list items that should be considered before making a decision.  The list the learner provides is then compared to a list previously produced by experts to determine a score.

I believe the research findings on TLAC are significant.  There are three studies concerning the TLAC training approach I reviewed.  The first study assessed 24 officers improvement in adaptive thinking using TLAC.  It used a repeated measures analysis of variance to confirm a significant linear trend (Shadrick & Lussier, 2004).  The second study examined the difference in scores between officers who had experience in Afghanistan and Iraq and those that didn’t.  Using a two-way analysis of variance, the authors found a significant difference in the scores of experienced officers vs. their non-experienced counterpart (Shadrick, Lussier, & Fultz, 2007).  The third study found that training received in TLAC transfers to other tasks with low surface similarity (Shadrick, Crabb, Lussier, & Burke, 2007).  I believe this research suggests something good about the approach being taken with TLAC.

While these studies may not be “strong” evidence for this approach (i.e., I didn’t see a clear statement that the groups were randomized and in study 1 there was only one group), the studies certainly seem to be “possible” evidence of effectiveness.  I’d encourage you to look at the studies, but if you only look at one thing, I’d look at the discussion of the second study (cf., Shadrick, Lussier, & Fultz, 2007).   In that discussion, the authors create a distinction between training and practice pointing out that most of our simulation efforts focus only on practice.  In that discussion, they provide a list of instructional principles to insure quality training for a simulator.

“a) identification of tasks, b) presentation of enabling knowledge, c) demonstration of how the task should be performed, d) the opportunity for the trainee to perform the task, e) provision for feedback to the trainee concerning task performance, and f) the opportunity to practice task to mastery under increasingly difficult, but realistic conditions (Black & Quinkert, 1994; Black, 1996; Holding, 1965, as cited in Black, 1996)” (p. 18).

Of these components of effective training, the authors suggest that simulations by themselves only enable the opportunity to perform and the opportunity to practice.  “Much additional exercise design work must be done to add the other components” (Shadrick, Lussier, & Fultz, 2007, p. 19).

References

Clark, T. R. & Gottfredson, C. A. (2008). In search of learning agility. Retrieved April 3, 2009, from  http://www.elearningguild.com/content.cfm?selection=doc.1054

Shadrick, S. B., Crabb, B. T., Lussier, J. W., Burke, T. J. (2007). Positive transfer of adaptive battlefield thinking skills (Research Report 1873). U.S. Army Research Institute for the Behavioral and Social Sciences. Retrieved April 3, 2009, from http://www.hqda.army.mil/ari/pdf/RR_1873.pdf

Shadrick, S. B., Lussier, J. W. , & Fultz, C. (2007). Accelerating the development of adaptive performance: Validating the think like a commander training (Research Report 1868). U.S. Army Research Institute for the Behavioral and Social Sciences. Retrieved April 3, 2009, from http://www.hqda.army.mil/ari/pdf/RR_1868.pdf

Shadrick, S. B. & Lussier, J. W. (2004). Assessment of the think like a commander training program (Research Report 1824). U.S. Army Research Institute for the Behavioral and Social Sciences.  Retrieved April 3, 2009, from http://www.hqda.army.mil/ari/pdf/RR_1824.pdf

While I watched the Sunday shows this week, the theme of globalization seemed to emerge.

Meet the Press

2:45 into the video below

David Gregory asked GM CEO, Fritz Henderson, “Do you expect and would you like to see President Obama encourage the country to buy American cars?”  Henderson responded:

“Um… no, actually.  I think consumers should buy exactly what kind of car they think meets their needs and that excites them. And as I look at it, it is our job to provide that, not necessarily have it mandated or otherwise encouraged.  I think we have fantastic cars and trucks and we’re going to win in the marketplace not just because we’re a U.S. company.”

Face the Nation

2:20 into the video below

Watch CBS Videos Online

Bob Shieffer asked the Treasury Secretary, Timothy Geithner, if he thought unemployment could go as high as 10%.  Geither responded:

“Well again, Bob, it depends how effective we are in moving and why it is so important that we move with countries around the world.  You saw the president this week in London move with countries around the world to make sure that they’re moving with us.  You know, in past recoveries, the world sort of depended on the American consumer to spend the world into recovery.  And that is not a healthy balanced way for us to do this.  We need countries around the world moving with us.”

This Week

The last 0:30 of the video

http://abcnews.go.com/video/playerIndex?id=7261357

Participating in George Stephanopoulos’s roundtable, Richard Haass, President of the Council on Foreign Relations, commented on the possible responses to the North Korean missile launch:

“We shouldn’t kid ourselves, George.  There is no solution to this — whether it’s sanctions or diplomacy or whether it’s missile defense.  Welcome to the 21^st century.  This is a world in which power, and in many cases in nasty forms like proliferation, will become more dispersed, more distributed.  This is the future of the world.  This is history coming to us now.  We can’t solve it.  We can perhaps manage it at times.  But this is going to be a messier, more difficult world that the United States at best can influence.  We can’t dominate it.  We certainly can’t control it.”

p.s. CBS wins for easiest video to embed.  NBC’s embed code appears to break if you switch between visual mode and HTML mode in Wordpress.  I didn’t see an easy way to embed ABC News.  Anyone have a legit method?

I just installed Twitter Tools for Wordpress.  I was amazed how easy it was to install and configure.  Now I just hope it actually works as intended.  If it does, there should be a tweet associated with this blog post.  I cool little yes/no checkbox appears on my Wordpress post page asking me: “Notify Twitter about this post?”

p.s., What is the difference between tags and categories when making a post in Wordpress?

I wanted to add my nifty little shortcut icon to pages associated with my Wordpress blog.  To do this, just edit the header.php template file to insert the shortcut link markup:

<link rel=”shortcut icon” href=”http://damonregan.com/favicon.ico” type=”image/x-icon” />

in the <head> section.  I put mine just above the <title>.