Lean startup methodology is brilliant but confusing

I just read two articles which beautifully illustrate the brilliance and challenges with the Lean Startup methodology.

First up was the story of Blue River Technology an agriculture robotics company whose first product is called LettuceBot. They were part of Steve Blanks Launchpad class at Stanford and followed lean principles to great effect. Their first idea was an autonomous lawn mower. In conversations with customers they discovered that was a bad idea, but also learned that farmers have a problem with weeding fields of carrots. Through more customer development they found that thinning lettuces is a bigger problem, and LettuceBot was born. They then sold their first LettuceBots off Powerpoint and had huge validation before they started building product. Brilliant.

Second up was Dan Kaplan’s critique of Peter Thiel’s critique of lean. I’m with Kaplan in thinking that Thiel is wrong in his critique of lean, but the interesting thing is that the problem stems from Thiel’s understanding, not from any fundamental issues with lean:

  • Misunderstanding 1: Lean is only good for making small changes to things that already exist
  • Misunderstanding 2: Customer development is nothing more than listening to what customers say they want
  • Misunderstanding 3: Identifying and testing hypotheses isn’t a planned process
  • Misunderstanding 4: MVPs are half baked products

Those are Thiel’s confusions as described by Kaplan. Then as a bonus Kaplan also points out that most people misunderstand the word pivot, mistakenly defining a pivot as when a company switches from one product or business idea to another (e.g. when Stuart Buttefield switched from the failed flash game Glitch to Slack) whereas Steve Blank defines a pivot as a smaller iteration of a business model or idea (e.g. a change of channel or customer segment).

To be fair Thiel’s critique of lean is bound up in a wider critique of incrementalism and a desire to see more step change thinking, but these points illustrate that lean methodologies are hard to understand and implement. It’s interesting to note that Steve Blank was in the classroom with Blue River Technology to help them with any misunderstandings and to stay disciplined. Most entrepreneurs don’t have direct access to Blank, and maybe that’s why they struggle. At one point Kaplan says:

if more entrepreneurs understood it [Lean] and applied it rigorously then fewer startups would fail

I agree with this. The high failure rate for startups is an unnecessary source of misery and loss. The opportunity now is to do something about it by helping entrepreneurs understand and apply lean. That means making it simpler and more practical.

You will see more on this subject from Forward Partners over the coming weeks.

  • My main is issue with Lean is that it’s closely associated with Fail Fast. There is a prevalent view that if the spaghetti doesn’t stick to the wall, then it’s time to move on. This may be true with “frivolous” ideas à la ChatRoulette, but if you’ve identified a genuine market need (and here’s where Blank’s customer development can really help) then perseverance becomes key to success.

  • It’s interesting that so much baggage has become associated with lean on such a short period of time.

    Probably because it’s difficult to understand.
    Teasing out the difference between not continuing with things that aren’t working and giving up too easily would be one way to improve clarity.

  • Rmicals

    I’m with Thiel. As a Trader by profession the Lean approach comes across as nothing more than a risk management strategy. An attempt at forming a rational framework to break down starting a company into a formula. The first example you gave on LettuceBots is more Market Research and Ideation which would be a common sense thing to do for any startup before building a product. As for ‘if more entrepreneurs understood it [Lean] and applied it rigorously then fewer startups would fail’ I disagree. Quite the contrary, more startups will fail but fail earlier. The question would become – how many of the potentially successful ones gave up. Not every product will develop momentum straight away for a host of reasons. Your hypothesis might not necessarily be invalidated.

    On the whole I think there is a small subset where the lean approach is useful but too many VC’s misunderstand it as a broadly applied viable way to build any business. Building companies is an inherently risky business if investors are not comfortable with it maybe they should look at high yield bonds or something.

  • Lean is a risk management strategy, but then startups and venture are all about taking measured risks and managing them…

    You are right that the LettuceBot isn’t a great example though. With the web products that we work with building product is part of the market research and ideation process. In fact one of the main points of lean is to efficiently evaluate the quality of the ideation and research process by getting people to buy cut down versions of the product (MVPs).

    As to more or less failures – what we’ve seen is that by doing effective customer research using MVPs companies more often build products that people want (although not necessarily first time) than with a research and then build mentality.

    I’m with Thiel in wanting radical innovations, but you can have them without losing the discipline of efficient testing and development. In fact, you can have more of them that way.

  • Rmicals

    If you wanted to build a web marketplace, aggregator or some software type company initially the lean approach is viable. But then it would probably get competitive as well as the barriers are lower. If you’re successful its likely a bunch of other guys will come in and then the most aggressive to scale wins (might not be you). So in the end have you really achieved a better risk/reward ratio? Its all about net expected return and not frequency of return or how small your failures are. Also, do you think think Aeroplanes, Railways or even game-changing companies like Theranos (in stealth for 10years) or the Hyperloop could be built within the lean approach? These have been high capital investments but in great markets with little competition and change. Also, you are absolutely right – having a constant feedback loop on your product I agree is one of the most crucial things but not specific to the lean approach. I genuinely feel the economics of web/software businesses have warped many investor’s perception of risk/reward.

  • Don’t equate lean with slow. The point is to quickly and efficiently get to a nuanced understanding of what’s going to really fly with customers.

    With something like the Hyperloop it is difficult to launch a minimum version of the product, but lean approaches to understanding customer requirements are still a big step forward from the traditional ‘build and hope’. And, perhaps more importantly, the fact that a large capital investment is required doesn’t make it a bad investment, it just ups the risk. So long as the reward is also higher, that’s fine.

    I guess I would ask ‘which bits of lean are you uncomfortable with?’ and ‘compared with what?’

  • Stuart Brameld

    Really interesting articles and debate. For me the key points are around staying disciplined and applying the process rigorously. The process and methodology is in principle pretty simple, perhaps even obvious (“it’s nothing more than listening to what customers want”). In practise, it’s really flipping hard when you have a team of people that just want to build and ship stuff. The customer perspective takes a back seat ….. fast.

  • I think that’s the challenge – the frameworks are very solid but the gap between theory and practice makes it hard to maintain discipline. Does that make sense to you?

    Everyone’s desire to simply get on with what they’re good at (usually building stuff, not talking with people) is one of the things that makes it hard.

  • Stuart Brameld

    Absolutely, though I think it’s probably 50% getting on with what you’re good at and 50% getting on with things where you can easily see/feel/measure/demonstrate achievements and progress.

    The tech/engineering focus of the startup world doesn’t help here, particular for early-stage companies (I realise Forward are, to an extent, bucking the trend here which is great).

    Go to a Startup Weekend for example and the focus is all about building, developing, designing and marketing. Nobody stands up on the Sunday evening and says “we didn’t do any design or development but have this excel sheet that documents 50 people we spoke with about this problem. We know their challenges, frustrations, how they work around this problem today and what that costs in terms of time and money. We learnt X, so the next step is to build Y and based on our conversations we think we could charge Z for this solution.”

  • Interesting. Thanks Stuart.

  • Rmicals

    I think it needs to be acknowledged that ‘lean’ attempts to cover the ‘should we build it’ but does not cover the ‘can we build it’ part till much later ie the execution risk. And I disagree with you that it helps develop a ‘nuanced’ understanding. It assumes as Thiel says that nothing can be known. Do you think Graham Bell should have polled the market while he invented the telephone? Should Steve jobs have asked customers before he built the mac? No customers weren’t aware how these products would change their lives till they used it. Secondly, a lot of VCs look at the approach and hunt for capital efficient models which favours web and software. Great if you find that magic model and it can happen but chances are you’ve got more competition with the low barriers. The most important thing is the entrepreneurs with the bold ideas get turned away and that should not happen. This is especially true in the UK startup scene.
    Thirdly it also seems many vcs and founders seem to take this approach to mean skimping on sales and marketing a crucial endeavour even if you have a killer product.

    So this is not a blanket approach to build any business.

    Also, I wouldn’t classify anyone who not following lean to be following a ‘build and hope’ strategy and lacking a constant feedback loop on his product or having lack of discipline in development.

    Also, you mentioned your own figures using lean but you really need to widen your data I feel to get a statistically significant result. Look at the larger startup space. There are many markets with higher barriers but less competition where there is a lot of potential.

  • Arguably Steve and Woz adopted a very lean approach, showing prototypes at the Homebrew Club and then selling 100 hand made kits.

    Lean doesn’t mean you can’t attempt big things. It means you should find the most efficient ways possible to attempt them.

    It’s very possible that investors are misunderstanding what lean is about and making bad investment decisions as a result. That doesn’t mean the methodology is wrong.

  • Rmicals

    True but I suspect that’s Jobs enticing people. Jobs approach was the opposite of lean. The Mac could not have packed all those amazing features by talking to customers. Customers wanted a better, cheaper, and faster Apple 2. The issue I have is that devotees of the church of lean seem to think that customers will iterate them to product/market fit. You need to have your own vision.
    I’m not saying the methodology is wrong I’m saying it applies to a subset and has blind spots. The book has a number of great tips but I find those to be the more intuitively obvious things. The danger is almost every vc you meet tries to sell it to you and it often results in underfunding and premature conclusions of invalidated hypotheses.
    If you truly want to get to the truth of lean ask yourself why you believe it works . Look at the the wider startup space apply some scientific rigour and come up with a statistically significant conclusion. Also Google Ben Horowitz’ and Marc Andreesens arguments on the limitations of lean in executing big ideas.

  • My sense is that we need to avoid looking at this in black and white terms i.e. Blank/Ries are right and Thiel is wrong, or vice versa. Should we not just accept that they both make some incredibly good points?

    The Lean Startup is variously described as a methodology, framework and/or approach. It is not a catch all prescriptive solution because “business is complicated”. Eric’s words, not mine.

    I’m shamelessly sharing a post (https://medium.com/@rhughesjones/another-post-about-whats-wrong-with-the-lean-startup-ae4fa93f3a8b) I wrote about this point which Eric himself shared on Twitter (which I take to be some form of validation, to use the venacular!) It stems from the idea put forward by FT columnist Andrew Hill that “and management theory is dangerous when applied inflexibly”.

  • Interesting conversation. Your comments have had me thinking and prompted conversations with others.

    I guess I haven’t seen the underfunding and premature conclusions you describe, at least not more than I see startup errors of other kinds.

    I’m reading and writing about lean at the moment because we see limitations too, so we’re not a million miles away. That said, I think the errors are much more of interpretation and implementation than of theory or methodology.

    On the subject of step changes, one of the big things that we do differently is advise entrepreneurs to spend time looking for a point of strong emotional connection before they start building. You might describe that as searching for the big leap forward before you start iterating.

  • They definitely both do make good points, although Thiel makes some bad ones too. But that’s a whole other topic…

    You are right that no management theory should be applied inflexibly, but I do think there’s space to help entrepreneurs by being more specific with elements of lean as they apply to early stage ecommerce. It’s crucial that we don’t forget the role of judgement though. One of our core beliefs is that there’s a great entrepreneur at the heart of every great business, and that’s because it takes real talent to build a startup, even with all the frameworks in the world.

  • Totally with you Nic. And for the record I’m all for lean, it just makes heaps of sense.
    The judgement point is an important one – it’s far harder to ‘teach’ than a methodology.

  • Rmicals

    That’s great. I think it’s a much needed debate given I’m an entrepreneur who almost moved to the valley this year after seeing an American company build a $bn dollar valuation raising the right amount of capital. London has a unique set of conditions to build huge companies -it has high quality diverse talent brought in by the universities, it’s a global media hub and has some great ideas. My experience is the lack of capital or risk appetite let’s it down. Vcs need to give their entrepreneurs permission to be bold. Happy to be part of any such debate.
    Interestingly you might find this article on vc returns useful. Exponential growth curves feel gradual and then sudden http://blakemasters.com/post/21869934240/peter-thiels-cs183-startup-class-7-notes-essay.

  • Rmicals

    Interestingly, following our discussion on risk below WSJ just wrote an article too http://www.wsj.com/articles/europe-is-struggling-to-foster-a-startup-culture-1431992065?mod=WSJ_article_EditorsPicks_4

  • Lean Startup is like many other business theories. It is a useful tool to navigate the real practical strategic issues and business faces. But every company needs to find its own answers so it is only a guide. Albeit a very good one. Big management models start to cause problems when people get religion about them and argue the one true path is the only option. Unfortunately, authors all write their books in this tone because the real value to them is the speaking and consulting fees. Startup management is a craft not a science. So read, learn from the ideas and do the best thing for your own business.

  • Hi Kenny – definitely agree. At the heart of every great startup there’s a great entrepreneur. Startup methodologies can be a massive help though.

  • Pingback: Lean Startup News | LaunchHack()

  • Graham Courtney

    It’s the discipline that is most pertinent to all of this. We all want our ideas to succeed and typically we have already invested ourselves in the idea before we even beginning any sort of disciplined validation. We all want to do something worthwhile and when you control what that is, it becomes potent. To the point that bias and clear no-go signs are ignored. So for me, lean startup is about the discipline of becoming a 3rd party to your idea, a scientific approach which does not guarantee success (wow, wouldn’t that be wonderful) but guarantees that you’ve taken a non-partisan approach to moving forward with your idea. The same number of ideas might fail but a lot less people would spend their lives chasing ideas that will never succeed.

  • Well said!