Estimating and Turkeys

November 14, 2019

Joshua and Kel revisit estimation! Why is estimation important? What do you need to get accurate estimates? How many turkeys are there in Turkey? (As it turns out, apparently there aren’t wild turkeys in Turkey…)

Be sure to check out our new Slack community to meet others who are facing the same things you are and share your journeys!

• 00:00 Joshua

  Hey folks, welcome to Getting Apps Done. A mostly non-technical podcast about building software. I am your host Joshua.

• 00:12 Kel

  And I'm Kel!

• 00:13 Joshua

  And on the show instead of talking about syntax and frameworks and technical things like that, uh, we like to talk about more practical things like getting a job as a developer or learning to be a developer in the first place. So in that vein, today we are going to talk about estimation because it's non-technical, but it's an extremely important part of being a developer and it's important to most skills in life. Realistically being able to estimate and figure out how big something is is very important in a lot of different disciplines. But definitely in software development it's a big deal that is often overlooked and sometimes kind of spurned if nothing else by developers as being a waste of time. And I think we've actually, we brought this up before we have done this episode in the past, but we thought it was so important and I see this so often that I thought it was worth coming back to again.

• 01:02 Kel

  Yeah, that's a pretty good introduction actually. I mean to start with and we should probably talk about what you're estimating, like where did these things come from that need a number attached and an estimate of how long it's going to take. And hopefully they came as a well-defined user story, a well-defined product vision, a, I don't know, a note from on high.

• 01:24 Joshua

  Yeah, I could just be a problem that came up and somebody came up with a solution and they hopefully they have defined it, but that's probably the first thing that we need to go through when we're talking about estimation because you cannot estimate something if you don't know what it is. I don't know why it's so hard to figure this out, but a lot of businesses really struggle with this concept. So one of the first things you need to know about estimation is how to help people understand what they need to give you because they can't just say, Oh, how long will it take to build a house? That's not enough information. You need to start to coax some information out of them. Okay. So if you want to build a house, what kind of house are you thinking about? Do you want brick? Do you want to four story mansion thing? Or do you just want a two bedroom flat? Or what are you looking for here? Just getting what we call a scope is what we're trying to do here to kind of hone in on what exactly is it that we need to build. So I know how long it's going to take.

• 02:19 Kel

  And there's actually some similarities of building your scope and estimating your scope because they kind of start with the same thing. It really helps to have an example of a something that exists already that is sort of what they're looking for because that'll get you a lot closer. It's a great starting point. So in your example of a house, if they bring in some pictures of houses they like, that'll give you a much better starting point than just words. Like the more information that is provided, the better you can kind of like get it starting point, uh, a shared context for communication, if you will. Um, and from there you can move on to estimations.

• 02:51 Joshua

  Absolutely. And one thing that I like to do practically for that is I like to start out before we even start estimating anything. I might do this on a whiteboard, I might do it with a napkin or post-its or something like that, but basically building out a little diagram or workflow of what it is we're trying to build because that starts getting them to ask some questions and to start to think about what that's going to be. When you put a piece of paper down and say, okay, so you got to your first screen, what does it do? What questions do you need to ask? Or where's it going to go? Okay, so you answered that question. What if they say yes, what happens then? Okay, now what if they say no? What happens then? And it starts to kind of tease out those details. What is it that we're trying to get to here? Why are we doing this? What is the flow of what we're trying to build?

• 03:33 Kel

  And you'll find that it's definitely a skill. Like that is absolutely a skill that I've had to deal with. Right? You go into the client, you go to a new, a new project, you start the new project and everyone is a little bit different. They have a different idea of how these things should go. Especially if you're not working on like the same type of web app or something. You're going into, Oh, this was a manufacturing plan, now I'm working in healthcare and these are two totally different solutions that need to be created. And so there's a bit of a skill you have to kind of like learn. You have to learn about your audience, you have to learn who you're talking to and learn the best ways of pulling that information out of them. And luckily there are a lot of things that you can do that have been predefined for those. You've probably heard the words user stories. That's a great way to start. They describe, they basically kind of describe what the person wants and the problem they're trying to solve rather than a solution

• 04:19 Joshua

  Absolutely. That kind of puts it in human terms. I mean we're all humans, hopefully. Um, and we can start to kind of think about things in the way that we explain what a problem is. All the steps that we would take to solve a problem. You can use analogies or whatever you need to do to try to figure out not just a user story but then a user journey. What journey does this user go through to start from, I have a problem, to my problem is solved. And it kind of, if you just go straight to the diagram, that can be quite scary or daunting for a lot of people who don't understand diagrams. Fair enough. Not everybody uses diagrams on a daily basis. So starting out with something simpler, just asking some questions about what would you do if this happened? What's your manual process? And quite often, you are replacing a process that already exists. So learning that process can be a really great way to get started. Okay. Let's map out this process. What's broken?

• 05:13 Kel

  To give an actual, like real world example I used to do, one of the things I used to do in healthcare was we would just print off paperwork for new patients coming in. And so when you'd come in, you know, you would register the person, you'd ask them their information and then you'd print out like a giant pile of paperwork that matched the reasons they were in there. And that was a real challenge. It was kind of a workflow type scenario of people trying to figure out, well what do we do? And the easiest way by far was to take, a face sheet, which is just an healthcare, just the general information about the patient, handed it to someone and ask them what are the pieces of paper you would pull out of a drawer and hand to somebody else.

• 05:48 Kel

  Like exactly what you're describing as we're going to be automating this process. But the first version is, well, how do you do it now? Like what is it that you do here given piece? All right. And then I would sit down and write down what they told me and that's where we start. That's always where we would start. And it was never quite perfect. Like each person had a slightly different way of going about it, which is part of the reason why we were there and automated and so. But it gave you a first, like a first a starting point. You can iterate from there. Once you have something in place you can go, okay, that's better than nothing. And from there we can tweak and fix and Oh like this person obviously doesn't need a form that says they're, you know, getting their leg removed. They're here for you know, a checkup.

• 06:29 Joshua

  That's actually a really good point that not everybody does it the same way. And that's one of the reasons why I ask more than one person. I will get one person to draw out that process with me and then I'll ask somebody else and then we'll compare them and we'll start to ask them questions. And you would be amazed how often that exposes something that either the first person didn't think of or they don't usually do because they're in a slightly different role or they're any number of reasons that they would have completely missed that if you just asked the one person, it's definitely worth asking more than one person.

• 06:58 Kel

  Exactly, and you'll see that like recommended all the time in agile methods when you start talking about getting feedback faster, sooner, it will also pretty specifically mention and not just one person. Like as many people as you can. You want data, you want statistics. If nothing else like you want to go, okay, the majority of folks think this is the best way to do it. We'll start here and then we can determine if the majority of folks are wrong, because that happens pretty regularly too.

• 07:24 Joshua

  It is amazing how often that happens. You will have one person who knows absolutely everything and they are right? Well a bunch of other people seem to think that this is the right way to do it, but they're completely wrong. It can go either way. That's why it's good to get different opinions, to map these things out and actually to try them out before you even get to the point that you're estimating anything.

• 07:44 Kel

  And this is why we do this prior to estimation, because what you'll find out if you just listened to the one person, they'll tell you to do something, you'll estimate that you'll build it, and then you'll come back and find out it was totally wrong and you're starting all over again from scratch. You've, your estimate was wrong in that case, cause now you get to do it twice.

• 08:00 Joshua

  Absolutely. I and I have seen practical he cases where exactly that happened. One person defined something, we went off, we built it. Two months later we came back, delivered it, and it wasn't at all what they wanted. So another person said, alright, well then let's change these things. Let's do this, do that. We went off, built it for another month, came back and another person said, Oh no, that's not at all what we wanted. In the end, it took 12 months to get to the stage that we'd built something that most of them agreed. Yeah, that's not, that's not, not too bad. That's pretty close to what we wanted. It's not the best way to build software. And that's why first off specification, which I think we are going to do another episode specifically about specifying and getting really into exactly what that is and making that process better. But also estimation are very important because it is about setting these expectations. It's about giving yourself finite deadlines and achievable deadlines so that you know what you're working toward and you can plan around that.

• 09:00 Kel

  And we can always assume like when we talk about these things where we need metrics, right? And you need a way of determining whether something was successful or not successful. And the very first metric that you're going to be dealing with is time, because it's the one that we all experience and everybody wants it sooner and faster. And so the very first thing you're always going to be measuring is time and being able to estimate, to be predicting how long something can take is a very useful skill. And you know, humans and surviving, but also when building software.

• 09:28 Joshua

  Well, the second thing you're going to look at is money, because I can tell you right now, when a plumber shows up to my door and they're telling me how much it's gonna cost or how much is going to be to fix something, that's the part that I care most about. I want to know how much is this going to cost me. And if one plumber says, well, it'll cost what it costs and it'll be done when it's done. And the next one says it's going to be exactly 200 pounds. I can tell you which one I'm going to go with. Even if the first one might've ended up being cheaper, I'm not willing to take that risk. And most businesses aren't either. For good reason because you don't, what did he want to gamble your entire business on a, Yeah. It'll take whatever it takes.

• 10:04 Kel

  Yeah. From a business perspective, they're doing the exact same thing, but they're including this in the things they're estimating. They're trying to predict the future. They're trying to estimate what they should do, why they should do it. They want it to be predictable. They want to know as much as they can and then make a decision at some point that will hopefully result for them and more money. That's their goals and so they want that predictability for them. It's better for it to be predictable than risky.

• 10:28 Joshua

  Absolutely. It's better for you as well because when it comes down to it, if you aren't estimating appropriately or you're underestimating and setting unrealistic expectations, that then falls on your shoulders at some stage. It's much better to know upfront so the business can plan around that. If they absolutely need this in 12 months and you think it's going to take 24 then they know they need to bring in another person to help you out so they aren't asking you to do 24 months of work in 12 months. Because it'll happen, or everybody will just be wildly upset at the end. So estimation isn't just important to the business is important to you as well as a developer, as a team. So while I can understand that it does take extra time and it is extra effort and it's something that a lot of people are really uncomfortable with, it is well worth learning. It's a skill that will help everybody involved.

• 11:18 Kel

  So let's talk about like why it's uncomfortable. Like one of the reasons why we don't like estimating is because we're all kind of crap at it. Like it's the future. Who knows what's going to happen in the future. Like I am not, you know, I am, I do not know what the future is. I am taking a guess. And that is why we call it an estimate.

• 11:34 Joshua

  Absolutely. But we can quite often gauge the future based on the past. When it comes down to it, if we look at history, it tends to repeat itself for better or for worse, it will. And that is one of the first things I'm going to do. Once I have a specification, the first thing I want to do is try to apply things that I know from the past to that new specification because they're going to be things that I've done before. One example that we were talking about before the, uh, show was planning some sort of a Google login that goes and fetches contacts. I've done this before. I know how long it took me in the past, and I know that each time I do it, it takes a little bit less time. So I've got a lot of knowns in there. But if you then shoving something that I don't know about, I've never connected to...

• 12:20 Kel

  Something you've never connected to.

• 12:22 Joshua

  I've connected to everything. I'm mean, come on. At some stage somebody wanted me to connect to everything and download contacts. But say there's a new tool out that I've never worked with before. I might still use my Google example because it's relative, it's related. I'm still downloading contacts. I still have to log into something. It's probably gonna use similar protocols. There's going to be API's involved and I'll have to have screens to allow them to click through things and all sorts of technical things. All these technical things that are going to be very similar but probably a little bit different so I can use that prior experience to decide, okay, well I know if it was going to take me two weeks to do it for Google, this new one is probably going to be two weeks plus a bit because there's some risk and risk is one of those things that I, that's where it starts to get scary.

• 13:09 Joshua

  That's why nobody wants to ask me things because there's risk involved, but you can actually take care of that just by assuming that there is some risk. If somebody asked me tomorrow to estimate how long it would take to do the Google contact download, the level of risk I would associate with that is quite low. So if I decided it would take me two weeks to do that, I would probably say two weeks in a few days because my certainty around that is quite high because I've done it several times before. If they asked me to do something completely new though, I'm going to internally assign that a very large amount of risk or a relatively large amount of risk, because I've never touched it before. I've taught similar, so it's not high, high risk, but it's something new. So I might add an extra week in there and say, Nope, that one's going to take me three weeks instead of two weeks because I don't really know everything.

• 13:57 Kel

  You'll hear pretty early, you'll hear people say, and this is kind of a, you know, just totally off the hip kind of estimation, but to double it, to quadruple it, and especially when you're starting, it's a starting point. You're going to be way off and you're going to always estimate under and that's a pretty normal thing to do because especially if it's things you're like, Oh yeah, I've done all of these things and that one took me these times. Yeah, it's like a week. Okay, let's just assume a month and then after you've done that, well then you can start using that as an estimate. Like, if you've never built a thing, just assume it's something ridiculous, try it and then come back and re-estimate. Uh, earlier we were talking about houses of like coming in with pictures of houses that somebody might want and an example of those.

• 14:39 Kel

  That can help you estimate it. Well I know that that house took a year, that house took six months. That house took about 10 days and then it fell down. And so like you have previous examples of timelines that you can use and that's the best way to estimate is to compare it against previous things that have happened. And at some point you'll get good enough as yourself to be able to estimate your own skills. And like how long, like Joshua saying here, how long it takes to implement a Google login. Eventually you just know this, it's kind of ingrained in your head and it will be a totally different number for different developer. Like I have not implemented Google logins that often, depending on what framework you asked me for, it might take me more than a week. I mean probably not, but it might.

• 15:19 Joshua

  Certainly as you get higher up, you do have to start to account for, okay, it might take me two weeks to do this, but how much is it going to take my teammate, that might be three weeks. So between us it's probably gonna be a week and a half or two weeks and a half or something. You have to start to be able to account for, not just your own abilities but also others. Uh, but certainly to start with it, just figuring out your own abilities and figuring out how to estimate your own time is very difficult. And as Kel says, the double it thing is a little bit loose, but you kind of have to be, at first you're assigning a fairly high level of risk because you don't have enough experience to know. Therefore, lack of experience is high risk.

• 16:03 Joshua

  The more you do these things and the more things that you can associate with it, the Google login versus another one to download contacts. Because I've done that as several times over with different API's, with different languages. My level of comfort to estimate a new project or a new service is going to be fairly high because most of them kinda work the same. I've got enough experience that I will be fairly accurate with my estimates, but if you ask me tomorrow to go build an AI that figures out exactly how many turkeys are in Turkey, I wouldn't even know where to start.

• 16:42 Kel

  Well, I mean I guess first would be Turkey... And that's actually the first part of estimation when we talk about it. So you received this thing, you've got a house, you've got a huge project to estimate all the turkeys in Turkey.

• 16:53 Joshua

  Turkeys in Turkey. Do they have turkeys in Turkey?

• 16:55 Kel

  I'm going to assume yes. Um, probably. It's November, everybody should have a Turkey. If nothing else. It's decorative. Um, so the very first step though is to break it apart. Like that's way too big of a problem. You just kind of look at it and go, eh, years. I don't know. And so the very first step is always just to start breaking it. What do I know how to do? Can I split that stuff out first and start breaking things into smaller and smaller chunks. Cause when we start talking about risk, the longer your timeline is, the farther out it goes, the more risk you're going to be just building up over time and you'll see that with project estimates and that's why a lot of software development is done on like two week intervals, one week intervals, day long intervals. It keeps the turnaround very short on those things. That way you can adjust your timelines as you're going. Um, but the first step is getting it into those bite sized chunks that you can then estimate with at least some vague certainty. You can compare one week project to a previous one week project and get a rough estimate of is this a reasonable week? Is this too much, is this too little?

• 18:00 Joshua

  Absolutely. Turkeys in Turkey. One of the first things I'd probably do is figure out, well how do I find out at all where that data comes from? One thing might be, I might look at social imagery and how many Turkey photos in Turkey are there on Instagram. I've worked within the Instagram API, so I actually have a pretty good idea how much time it would take me to build up something that can connect to Instagram and look for things tagged Turkey in the geographical location of Turkey.

• 18:29 Kel

  And to compare this to a real life too. It's also quite plausible that somebody already has this data and you do not need an AI for it.

• 18:36 Joshua

  Absolutely. Probably. Yeah, but it just, assuming we're trying to solve this problem from that point of view, breaking it down into these chunks of, okay, so where do I get the data? How do I connect to that? Where do I store that? What do I do with that data once I've got it, how do I collate it? How do I search it? What do I need to do to actually start to figure out are these pictures actually of Turkey or are they of something else? Those are the sorts of things that you can break down into small chunks and you can start to see that, okay, some of these things I've done before, so I know connecting to Instagram is going to be about this. I have no idea how to determine if a picture is a Turkey or not. So that's got some high risk involved. I'm going to have to figure something out with that so I might be able to estimate it, but by applying a high risk to it, I know that I need to make that estimate bigger because I don't know, there's some uncertainty there

• 19:24 Kel

  There's some negotiation at that point too. So like this example of a, trying to recognize a bird and an image and being able to identify correctly as a Turkey or not a Turkey is a challenge and it's one that I would have to go research. So somebody approached me with this as a project. The first thing that I would be, well give me, you know, a month and I'll get back to you with data or give me a week or give me, you know, give me some time to try this out so I can come back with a reasonable expectation that isn't somewhere between a tomorrow and heat death of the universe. Like sometimes you have to be, sometimes you have to invest essentially. And it is a gamble of I'm going to throw a week at this and it may be a total waste or it may be a gain and then I'll know how to do something or at least have a better estimate at it.

• 20:09 Joshua

  Absolutely, and that is quite often one of the first things that I will do with something that is very unknown to me. I will just flat out put in a time frame and say, Hmm, this time frame is to figure out what the time frame is for the actual thing you want. And it's really, again, we've talked about setting expectations fairly recently and that's exactly what this is. You need to set some expectations. If you tell them it's going to be a month and then two days into it realize, Oh no, this is more like a year. That makes you look really bad. Whereas if you'd just said, give me a week and I'll come back with a better estimate, that's going to look much better. Most people aren't going to be, unless they can find somebody else who knows exactly how long it's to take to identify all the turkeys in Turkey, they're going to be okay with that. They're going to, that expectation is set. They will either grant you the week or they won't, but you're not going to be in a position where you're having to answer a question that you just can't answer.

• 21:01 Kel

  And overall you'll find out, especially as you kind of grow as a programmer, that this becomes easier, because this is part of your day to day life as a programmer is breaking problems into smaller chunks and turning them into functions, turning them into capabilities of software and that maps very, very directly to how you estimate projects. Break it into smaller chunks, figure out which ones you know how to do, which ones you don't and then plow forward. And you'll find that that kind of starts leaking into your normal life and people look at you a little weird when you approach, you try to solve all of their problems that way. Um, but it is quite effective and it is one of the bonuses of learning how to be a programmer.

• 21:37 Joshua

  Absolutely. So next week we will come back and we will talk about more around specification and how to identify that specification and what exactly it is. So if we go back to turkeys and Turkey, figuring out exactly what classifies as a turkey and all these sorts of things that we need to know before we can actually provide an accurate estimate. Until then. I'll put some transcripts up that getting apps done.com please be sure to check out my website at joshuagraham.info and Kel's website at piffner.com. If you have any questions about estimation or you are struggling with estimation because that's very common, or you want some tips on a specific situation or something like that, join us on our Slack channel at gettingappsdone.com/slack. We'd love to hear about that. We'd like to share about these sorts of things because it's a common thing that a lot of different people run into, so your problems are probably somebody else's problems as well, so please be sure to share. We post every Thursday, so we'll be back next week until then thanks for listening.

• 22:31 Kel

  Cheers.