Friday, May 15, 2009

Carmel CA: Model Evaluation

Finally, here in Carmel, the limitations of our current decision model become clear. The current approach, which uses only 'black and white' assessments (or green and red), does not give us enough flexibility to address a complex decision like the one we are facing. It works fine for Legal Apprentice, where ultimately a defendant must be found guilty or not guilty, but not for the tradeoffs of picking a new home.

My brother Vern had suggested to me earlier that I could build this model using the 7 point scales we use for assertions in Legal Apprentice, and that would indeed have given us more variability than the current model. The problem with this is that eventually the 7 point scales get rolled up into a yes/no, black/white, red/green decision. What we really need for this exercise is a scoring algorithm that rolls up into a 'so many points out of 100' type of conclusion. The Carmel example makes this very clear.

Here is what the current model looks like for Carmel:

  • Carmel By The Sea (Decision)
    • Carmel is a good choice for our next home.
      • AND [1 of 5] : Carmel has a good climate.
        • AND [1 of 2] : Carmel has moderate winters.
          • OR [1 of 2] : The number of days each year at or below freezing should be 10 or less.
          • OR [2 of 2] : The average winter temperature should be at least 10 degrees warmer than Hudson OH.
            • True
        • AND [2 of 2] : Carmel has a good balance of sunshine and summer heat.
          • OR [1 of 2] : Carmel has moderate summers in exchange for lots of sunshine.
            • True
          • OR [2 of 2] : Carmel has lots of sunshine but hot summers.
      • AND [2 of 5] : Carmel has an affordable cost of living.
        • AND [1 of 4] : Carmel has health insurance available for less than $1500 a month.
          • True
        • AND [2 of 4] : Carmel has lower taxes than Hudson OH.
        • AND [3 of 4] : Carmel has a lower cost of living index than Hudson OH.
          • False
        • AND [4 of 4] : Carmel has good housing for less than $800,000.
          • False
      • AND [3 of 5] : Carmel has good work opportunities.
        • OR [1 of 2] : Carmel has good fulltime work opportunities.
          • False
        • OR [2 of 2] : Carmel has good freelance work opportunities.
      • AND [4 of 5] : Carmel provides a good quality of life.
        • AND [1 of 2] : Carmel can provide a healthy lifestyle.
          • OR [1 of 2] : Carmel has neighborhoods that provide a walkable urban lifestyle.
            • AND [1 of 2] : Carmel has at least one neighborhood with a walkscore greater than 90.
              • True
            • AND [2 of 2] : The neighborhood with a walkscore greater than 90 has a crime rate no greater than 125% of the national average.
              • True
          • OR [2 of 2] : Carmel has areas with affordable nature 'compounds'.
        • AND [2 of 2] : Carmel can provide an inspirational lifestyle.
          • OR [1 of 3] : Carmel has a high quality university.
          • OR [2 of 3] : Carmel has cultural diversity.
          • OR [3 of 3] : Carmel has beautiful scenery and views.
            • True
      • AND [5 of 5] : Carmel has good accessibility.
        • AND [1 of 2] : Carmel is within 2 hours of a major airport.
          • True
        • AND [2 of 2] : Carmel is within 30 minutes of high quality hospitals.
          • True


    It says that Carmel does not pass our test, yet we have looked at a couple of houses here and are seriously considering making an offer on one. Why? Because every town is a set of tradeoffs, and sometimes outstanding strengths on one criterion outweigh the drawbacks on another. A scoring model could account for this.

    Example: Carmel obviously does not pass the current test for affordable cost of living, but it scores so highly on quality of life, we are willing to consider that tradeoff. If we allowed scores for each of the subcomponents of the model, when we added them together or created a weighted average, or whatever the rollup scoring method is, it would be possible for Carmel to outscore locations like Austin and Dallas, which 'passed' the decision test. We need to enhance the model to allow this level of sophistication, and we are brainstorming now ways of doing this.

    2 comments:

    1. Hello Deb, sounds like your trip is going well and I have enjoyed reading your posts. THe problem you discuss with the model, reminds me of the problem we used to have with our store location models. When our methodology was applied blindly (as sometimes it was) we would basically run around approving or denying new sites as they were submitted for consideration. However, the real question that I tried to turn the modelling to address is "what is the best location for a new store?". This lends itself to a scoring or even a ranking methodology. At the time I was exiting that field, I had really felt that the purpose of the models had become to narrow the list to some small set of good candidates. At that point, we could then rely on (perhaps) more subjective (or at least not yet quantified ) factors. At that time we would want to use the model deficiencies to try to build a plan to "overcome" those areas. Since where you live is a very emotional/subjective choice...I think that this approach has some merits...the model works as a mechanism to curb emotions and forces you to plan how to overcome the deficiencies of a location without eliminating the very strong ratings/feelings on particular attributes. These are just some random thoughts...I have not really thought yet about how scoring/ranking lists could be translated into decision modeler type logic.

      Carmel/Monterrey would be a great choice.

      Steve A

      ReplyDelete
    2. Steve, we've decided to focus on rewriting the decision model, including scoring, as our project to keep us occupied on the long trip home. Four days at least, to reach Ohio from the Northwest....
      Deb

      ReplyDelete