November's "Blue Wave" -- a prediction

Depending on whose social media account / newsfeed you have looked at recently, you may believe that November's elections will either bring a massive "Blue Wave" of Democrats taking Republican seats in congress, or you believe that there be a "Red Tide" or no change at all.

In both cases, pundits propose that the election results will be a massive response to #MeToo, #KavanaughConfirmation, #BuildtheWall, the state of the US economy, a burgeoning trade war with much of the rest of the world, Trump's tax returns, "Grab her by the Pussy", "Mob Rule", etc. etc.  In a not-at-all-astonishing confirmation-bias-fueled assessment atmosphere, folks on the Left can look at this morass of bile and say "Trump is going to get a strong message in November" and at the same time folks on the Right can look at the same morass and say "Trump is going to get a strong message in November" and mean exactly the opposite thing.

So, if you never read my original prediction about Trump, here it is:
https://psyphifi.blogspot.com/2016/02/politics-and-market-segmentation-why-is.html
If you've read this far and never read that, go read it.  It's worth it.

Back then, I hypothesized that the "inside out" market segmentation being performed by the major political parties was largely miscategorizing the vast majority of the electorate by trying to cram them into "Left" and "Right" and failing to recognize that the electorate is made up of people who are more complex than that.  I predicted that a big surprise was coming, and sure enough there was one.

So now we see the same nimrods talking the same nonsense about November.  They've learned nothing.  In some areas (Florida's gubernatorial race, for example) we're starting to see more non-traditional candidates (a Trump-lover v. Bernie's fave), but in most places it's the same-old v. the same-old.  Party leadership is selecting people that they like without consideration to whether or not the electorate will like them, or whether or not the party is even representative of the electorate at all any more.

I found another interesting metaphor in vision.  Here is a chart of colors, drawn based on the sensation of the cones in the eye:

I think it's a good metaphor for the phenomenon I'm describing.  Traditional political analysis is based on just the red to blue continuum on the bottom.  Basically, it takes everybody in the rest of the spectrum and just flattens them down along the y-axis onto whatever is below them and completely misses the reality.  There is a lot more green, yellow, orange, etc. up there, but it is all ignored.

So now we have a system based on "politics of approbrium" where each highly non-representative party tries to convince all the greens up there that the other party is a horrible villainous group of monsters.  They don't even try to build an affinity in general, just try to create a sense of enmity with the opposing party.  The goal isn't to try to understand what people really want, just try to exploit biases and human fallacies to convince voters that they don't want what the other side is cooking (more Killary/Pelosi/Schumer v. more Drumpf/McConnell)

So will there be a "blue wave"?  No.

If there is anything resembling a wave of any kind, it will be a wave of non-traditional candidates taking places ordinarily held by stock party-line candidates.  People like Kyrsten Sinema in AZ who are known for their independent voting records (Kyrsten is very much up in that yellow space when it comes to her perspectives on most meaningful issues).

So which party has more candidates that hew away from the bottom of color scale?  Which candidates better represent their electorates?

Four big ways to avoid "Excusable failure"

How many times have you seen this?  A big group initiative kicks off and there are two paths:

< Insert Frost quote here, if you really feel the need >

One path is more likely to succeed, but everybody on the team is culpable if it fails.  It will take a lot of work and dedication, but if everybody pulls together success is almost certain.  If it the project fails, there's no mistaking why.

The other path appears solid, but is really more likely to fail because nobody on the team is culpable if it does fail.  Responsibility is diffuse.  Significant aspects of the project rely on people outside the project team (maybe even external vendors or consultants) who may be perceived as experts, but collect a check no matter what happens.  If the project fails, it's possible to point fingers and claim "we did everything we could".  Sometimes the participants even move on to bigger and better projects half-way through and leave the mess for their replacements.

I have seen this many, many times.  Enough where I started calling out "excusable failure" as decision factor.  Like many agency problems, excusable failure is the situation when the people who are most responsible for the success of the project are not held responsible in the event of its failure.

How can you beat it?

Make and keep meaningful commitments

This is a motto for all aspects of life, but especially matters when kicking off a new project.  Start up front with a set of valuable contributions your project will deliver, an achievable timeline for delivery and then put yourself in the critical path.

Don't substitute an explanation for an accomplishment

Every project should start out with identifying the various possible ways it will fail.  Pre-mortems, FMEA, etc. are all great tools.  Once you have a good list of possible failure mechanisms DON'T ACCEPT THEM.  The project is starting because you want to accomplish something.  Set the standard up front that only the projects goals will be acceptable--if any of those risks go wrong, we need to understand why, but we can't give ourselves a participation award, we FAILED.

Don't substitute an excuse for an explanation

There is a big difference between an excuse ("this thing happened, so you have to forgive me") and an explanation ("this thing happened, and therefore we can make sure it doesn't happen again").  Know which one you're searching for--always try to figure out what you could have done differently (there is always something you could have done differently).  If you find yourself starting to think about why things are going wrong, you had better also be thinking about how to stop that from happening.

Keep skin in the game

We have probably all seen a leader/manager "doodle jump" themselves away from their commitments to a new role before their failures caught up with them.  It works in the short run, but it leaves a trail of destruction behind and seems to always catch up at some point.  Don't do it. Keep your skin in the game and see things through to completion.

Allocations and Decisions

I frequently run into an issue with allocations.  Specifically, an issue with confusing accounting allocations for something that is useful in decision-making.  In accounting, allocations is:

the process of shifting overhead costs to cost objects, using a rational basis of allotment. Allocations are most commonly used to assign costs to produced goods, which then appear in the financial statements of a business in either the cost of goods sold or the inventory asset.

As an example, imagine this classic situation in engineering.  BigCo is developing a new product line.  My kid is 4 and obsessed with potty words, so we'll call it "Product Booger".  Product Booger will cost about $1,000 of RND spending to get ready for market, and once it goes to market, BigCo expects it will sell around 500 units at $20 each.  Each unit will cost $10 to manufacture.  Should BigCo do this project?

Here is the super-basic analysis

If BigCo has the $1000 to invest and doesn't have other investments with a better efficiency ratio, they should do this project.

Now imagine that a super-eager young person comes along with an idea for a product extension based on BigCo product Booger.  Product NoseGoblin is a derivative product that will cost $100 of additional RnD and should sell around 200 units at $16 each.  Each unit will cost $14 to manufacture.  It is safe to assume that NoseGoblin won't cannibalize any of Booger's sales or anything like that. Should BigCo do this project?

Here is this super-basic analysis:

So this NoseGoblin isn't as good as Booger, but it still has a positive return and recommends itself for investment if BigCo's alternatives don't have a better efficiency ratio than 1.0.

Then the accountants catch wind of NoseGoblin.  They decide that it needs to carry a portion of the shared investment from Booger.  Doing a classic revenue basis allocation--allocating the RND spending according to proportional revenue-- Nosegoblin picks up an extra allocation of $242 (at the same time, Booger looks $242 cheaper).  A few months later, a new financial analysis is performed and NoseGoblin's financials look like this:

It is now a negative financial return.  Kill it fast!  Super-eager person's heart is broken.

this what a GIS for "sad super eager person" finds for me

This is not a made up situation.  I have seen this happen.  It gets even crazier when BigCo decides that everybody needs to carry a share of overhead allocations in financial analyses, too.  Suddenly you see stuff like this:

Even though this project doesn't drive any additional SGnA, IT activities, shared technology development, new building construction, etc., GAAP rules require that these allocations get managed in the PNL so they tend to find their way into the financial analysis, too.  Load in things like depreciation, taxes (which are usually so convoluted that trying to estimate the real tax impact of a new product is fool-hardy) and you're ready to start shooting down loads of good projects that fail to meet a corporate hurdle that completely misses the point.

Accounting allocations being used for decision purposes leads to bad decisions.  When we do portfolio analyses, we keep them separate and link the spending through relationships and work hard to identify incremental spending.  Only spending that is truly incremental is relevant in the analysis.

organization and confusion

A company is organized the way it is for many reasons:

• execution
• strategy
• marketing
• executive ambitions
• legacy
• etc.

It's all a trade-off, not necessarily built around effective choice architecture.  Each individual org may help drive bad decisions for the company, but that are good for individual decision makers.  e.g. empire building, or agency problems or local optimum instead of global

It's sort of textbook, but I see it all the time, especially in market modeling.  Specifically it is very common to see groups that model markets in a fashion that mirrors their own organization structure, rather than reflecting how customers make purchasing decisions.  They practice inside-out market definition.

Good market intelligence requires understanding the buying behaviors of your potential customers.  Market segmentation means grouping those customers into categories based on salient similarities with regards to your product offerings.  For example, you might segment your customers according to price sensitivity.  For example, you might segment them according to how much they value certain different features your product offers.

This is called outside-in market segmentation.  It helps you to understand how well your products will play in the market.  Coupled with projections about the size of those market segments you can start to put together a business model.  With time, smart companies reorganize themselves so that their corporate structure reflects the reality of customer segmentation in the world.

If you try to cram your customers into groups that fit how you run your business instead of trying to run your business so that it fits the needs of your customers...

Scrapping the code and starting over... and also the Trump budget proposal

If you've ever developed a software product that needs to live through multiple versions and over many years, you've probably experienced this situation.  For that matter, if you've ever used one, you have experienced it.  That includes everybody, since Windows is a good example.

Your software starts out with a purpose.  It has a crystal-clear goal in mind.  If you're any good, the software does that so well, that people want to do related things, too.  So the software starts to expand.  Also, bugs happen and so you start fixing those bugs.  Bug fixes and new features start piling up.  Eventually you start getting unintended consequences.  A new feature here causes a bunch of bugs over there.  A bug fix here disables a feature over there.  Was anybody using that feature?  Who knows, let's ask.  Did we ask the right people?  Whose opinions matter when it comes to the bugs we fix first?  Who should we ask which features to add next?  We have to start deprecating some stuff because the code base is too big and too messy and has turned into a hairball.  What do we deprecate?  We need version control and a change committee and and agile process and now we have people whose jobs are just to manage those things that often have no idea what the software is even supposed to do.  It spirals out of control until you have Windows 10.  Or until you have a typical Fortune 50 company (this is one of my favorite books on this topic), or you have the US Government.

With software, you eventually have to scrap the whole thing and start again from scratch.  This really works great if you have the same team in place and they can apply all the learnings and wisdom they have accumulated while putting together the previous version.  This is why we usually start with a quick, disposable prototype for any new idea.  We know that our first pass will need to be scrapped, so we just start out trying to test out the concept and the rough structures.  The goal for a first draft is never to create an enterprise-level software product that will withstand the ages.  The goal is to see if it's worth building an enterprise-level software product at all.

So what does this have to do with Trump's budget, you ask?

CNN Article
USA Today Article
Bloomberg article

The Bloomberg article has this great infographic:

In it we can see that many government agencies are getting serious cuts.  In a very real way, many programs are getting scrapped.  Now, I understand that the administration's purpose here is most likely not "those were prototypes, let's take the learnings we have accumulated and build a better system from scratch" but that may still be the outcome.  We can only hope that the next administration will recognize that these endeavors (feeding the elderly, protecting clean air and water, PBS and the arts, school lunch programs, science, space, housing, etc.) are meaningful and important enough to build new, better, v2.0 agencies in a few years.  Maybe they will all be sort of like win95 compared to win 3.0!  maybe?  right??

It's sort of curious that some of the agencies that Trump has been most vocal in deriding --namely the VA and the military-- are the biggest beneficiaries of this shift.  It's a shame that these two groups will not have the opportunity to experience as radical restructuring and tabula rasa sort of rearchitecture.  As somebody who has experienced both of these organizations first-hand, I think that the opportunities for waste reduction are legion...

A quick data visualization, fat with implications

I whipped this up last night.  I will refrain from comment, but please tell me what you think...

A "natural leader"

What is a "natural leader"?

I got to thinking about this the other day while considering my two older boys.  They are just kids, and like anybody they are complicated and multi-faceted individuals.  For the sake of this example, I'm going to just treat them as simple stereotypes.

Jack is my older son.  He is naturally charismatic and has a large group of acquaintances and friends.  He charms children and adults with his easy style and effusive character.  Other kids naturally follow his example, want to play games that he devises and generally look to him for leadership.  At the same time, Jack is not very concerned about others.  He is often dismissive of others' feelings.

Ethan is my middle son (I have another, even younger son).  Ethan is very empathetic and quickly identifies the needs and emotional wants of others.  He feels deeply and shows great concern for the development and well-being of other kids (including his brothers).  Ethan does not display the same sort of outward and obvious charm that Jack does, he has his own (very unusual) sense of style and in general is not a "popular" sort of kid.  Other kids don't naturally flock to him, and he has very few acquaintances and just a few close friendships.

So, in this either-or scenario, who is the "natural leader"?  Jack is the one that people naturally want to follow, but Ethan is the one who possesses the emotional intelligence and selfless concern that characterizes great leaders.

The easy answer is to say that neither is really a natural leader, and that a natural leader is someone who possesses both characteristics.  I think that answer is wrong, though.  There is a lot of literature on leadership that seems to indicate that really successful leaders are often not the big keynote, charm-the-audience types, but more of the help-develop-your-people-though-honest-methods types.  We all remember Steve Jobs, but who ever really wanted to work for him?

Update:
I see this thing from time to time and it gets me to thinking:

At first glance it's very heart-warming and embraces diversity and has a cute girl standing akimbo with a soft-focus field and really seems like a great idea!  Except this... "Bossy" is not a leadership skill.  "Bossy" is a completely unnatural leader--bossy is a person who wants others to do what she says to do.  Bossy doesn't imply that others want to follow.  Bossy doesn't imply that you care about what's best for others, or that you're qualified to lead at all.  Bossy just means that you want to lead.

Really makes me wonder about Sheryl Sandberg that she doesn't seem to know the difference.


What do you think?

The parable in Tsar Saltan... "Gvidon's Squirrel"

I like to tell my 3 year old stories that are loosely based on Russian Folk Stories that I found in this book.  I take down the gore as much as possible and generally try to shift the jobs that people do to be a little more appropriate for modern times (and make the animals more fit for the desert, where we live).

On story I love to tell is based on "The Tale of Tsar Saltan".  In this story, a young man (Gvidon) ends up distant and unknown to his father (Tsar Saltan).  He helps out a swan, who turns out to be a magic princess and who falls in love with him and does him all sorts of favors.  Every time Gvidon sneaks back to his father's castle in the form of a fly, he comes back with a request for some new and crazy thing that he heard about there.  One example is this one:

 "After he flew back to the island, Gvidon told the swan the story he heard about the remarkable squirrel. Then the prince walked into his courtyard and, lo and behold, there was the singing squirrel, sitting under a fir tree, cracking golden nuts! The prince rejoiced at this and ordered that a crystal house be built for the little animal. He placed a guard there to stand watch and ordered a scribe to record every shell. Profit for the prince, honor for the squirrel!"

Basically, he found himself in possession of a magic squirrel/tree combination.  The tree produced golden nuts (each of which held a ruby inside) and the singing squirrel would go up and collect them, take them down and crack them and stack the riches up in piles.

The part that I love about this story is that Gvidon doesn't just take the riches, he then builds a completely non-value-added organization around those riches.  He establishes a completely superfluous crystal house for the squirrel, he organizes and accounting department to keep score and establishes a management structure that watches the whole thing happen.  

When I tell the story, I keep going.  The guards need management, so an executive structure is organized.  The accounting department eventually branches out into strategy and financial analysis and grows headcount.  Eventually there are organizations built up around selling the rubies and the gold on international markets.  The organization continues to get bigger and bigger and bigger until eventually the tree and the squirrel are at the bottom of the hierarchy, getting bossed around by MBAs and seasoned executives who have been in this business for years.  By the end, Gvidon's entire income is consumed by the organization.

How many times does this happen in the real world?  A start-up comes out with a great business concept and develops a product that is worth buying.  People value the services or the experience and the company starts to take off.  The need to scale and they take on investors, who want to track progress.  Before you know it, it's "Gvidon's Squirrel."

(if you're interested in reading the whole story, I found an online copy here, which is where I found that picture, too. 

If, when, and how often ... security flaws and fake news

When I was in middle school, my family moved from the sleepy rural town of Hurley, New York to the nearby city of Kingston.  While living in Hurley, my family had grown accustomed to the security standards of small communities where the nearest neighbors are quite a long distance from one another--that is to say no security standards.  My family almost never locked our doors, and my parents regularly left their keys in the car overnight.

Within a few weeks of moving to the comparatively populated Kingston, my father's car disappeared one morning.  He had left the car unlocked, with the keys in the ignition.  Some enterprising young man had noticed and taken advantage of the security hole.

A few weeks later we heard from the police at SUNY Albany.  Apparently, the guy had taken the car up to the campus and had been breaking into dorm rooms and storing stuff in the car (which he had also been living out of).  When the car came back, it was full of stolen stuff.  Amazingly, the SUNY police refused to deal with the stolen stuff because the car was originally stolen in Kingston and the Kingston PD refused to deal with the stolen stuff because the robberies happened on SUNY campus, so we were left with a whole bunch of stuff.  The guy who did all the stealing went to jail for a few months and my dad started taking his keys out of the ignition at night.

Amazingly, the day that the guy got out of jail, our car disappeared again!  Apparently, he had made himself a copy of the key.  That key was among his personal things when he was jailed and was among the personal things returned to him upon release.  He walked straight to our house, unlocked and started up the car and --creatively-- headed straight to SUNY Albany again and started robbing dorm rooms.  When the car came back the second time, Dad finally changed the locks.

I hope this is going somewhere...

When I was a teenager, I got involved with the early idea of online video games.  Back then, "online" meant dial-up, and specifically it meant direct dialing either a BBC or a friend's computer.  You would plug your phone line into the modem (external modem, of course) and then one of you would call the other one.  A few DOS commands later (actually a ton of inscrutably complex commands later) and you were DOOMing it up against one another.  In the '80s and early '90s, security considerations were a lot like Hurley... everything was unlocked and the keys were in the ignition.

Years later, at one of the first companies I started, I decided to host the email and web server for the company locally.  By "host locally" what I mean is that I got a static IP, stuck an old desktop in a closet and installed Apache.  Installing Apache was easy, but configuring security was complex and lengthy.  I thought back to the old BBC days and considered "what are the odds that somebody will discover this IP?"  The idea of learning about the security was interesting to me, though, and I put in a few days to educate myself, figure it all out, configure and install, etc. before activating the server.

What happened astonished me.  The first attack on the server came within a few seconds.  The next one just a few minutes after that.  By the end of the day, there had been dozens of intrusion attempts against this brand new server that was there to serve up a couple of lame web pages about a nobody video games company and to pass our completely uninteresting emails back and forth.  The pace of attacks didn't slow down, either.  It continued to increase as time went on.

At the time I started saying that security is serious business.  Security wasn't a question of IF you will get attacked or even WHEN, but HOW OFTEN will you get attacked?  This is probably more true now than ever before.

Over time, I have come to recognize that this equation is a function of network density.  The more dense a network of nodes are interconnected, the more interactions each node needs to expect.  These interactions can be anticipated, they can be serendipitous, or they can be hostile.  The more deeply and broadly we immerse ourselves as individuals into a more interconnected society, the more interactions we are exposed to.  The likelihood of "intrusion attempts" (hostile interactions) goes from "will it happen?" to "when will it happen?" to "how often will it happen?"

The next interesting question becomes "what will tomorrow's intrusion attempts look like?"  A decade ago everybody had at least a few Nigerian prince emails.  A few years ago, everybody was falling for phishing scams on Facebook.

This year was the year of fake news.  Possibly the most insidious of all hostile interactions, fake news plays on our human desire for an echo chamber.  Fake news spreads like a virus through the like-minded and hijacks our rational-thinking.  The Nigerian emails stole bank account numbers.  The phishing scams stole our identities.  Now fake news is stealing our ability to think.

The metaphor of the deer

Many moons ago (in college) I was driving on a small, country road in upstate New York ( somewhere around here, if I remember right ).  It was night and it was winter.  A dusting of snow was sprinkling down on the already frozen road, the sides of which were burmed with banks of tight-packed snow and ice with thick forest beyond.

This was college, and I was broke.  I was driving a Nissan Sentra Station Wagon, which was an absolute POS in case you're not familiar with it.

The POS in question

I was going around 60 and came over a hill to discover a deer hanging out in my lane, about half a mile away.  He was a big buck, facing to my left.  I seemed to have plenty of time to work out my options in my head, and it went like this:

a) Swerve around him to the left.  This would bring me into the empty oncoming lane, but I have always heard that a deer will jump forward if frightened, and that would mean he would jump right in front of my car.
b) Swerve around him to the right.  This would bring me right into the frozen snow bank.  Odds were really good that my junky little car would end up in the woods in the middle of nowhere.
c) Jump on my breaks.  The road was frozen and it seemed really likely that the bicycle tires on my car would lose traction and I would end up spinning out in the middle of the road.
d) Get off the gas, steer straight and hope for the best.

Now, it's worth mentioning that I was a truly terrible driver as a teenager.  These scenarios were not academic concepts to me, they were all experienced realities.  I had driven off the road several times (including a roll-over accident that crushed the roof of my car with a tree), I had lost control and entered into flat spins during bad weather, bad road conditions (and honestly sometimes under ideal conditions but bad choices).  I had wrecked 4 or 5 cars by this point IIRC.  I knew my way around crashing a car.

So, I did what they teach you to do in that circumstance.  Without any panicking, I took my foot off the gas and steered toward the right edge of the road.  I knew there wasn't room to get around the deer, but I was hoping he would jump out of the way.  In retrospect, I should have honked my horn, too.

About an hour later the local sheriff was there by the roadside, poking a buck with his foot and pronouncing "yup, he's dead alright.  Do you want to keep him?"

I spent the rest of the year driving a car with a completely smashed up front end, leaking Styrofoam from the bumper (did you know that cheap cars have Styrofoam bumpers??) and with a grill full of fur.

I tried a google image search for "furry bumper" and "fur in my bumper"... I don't recommend it.

There are lots of situations like this.  You can see them coming a mile away, but there isn't anything you can do about it.  In fact, anything you do is likely to make the situation worse.  When you look back at the outcome, it's easy to second-guess, but sometimes the right choice is to limit the downside--not to try to avoid it.

Bad stats -- "good enough"

I'm reading (or really listening to the audiobook during my commute) "Traffic: Why we drive the way we do" by Tom Vanderbilt.  It is a pretty good book and well researched and surprisingly interesting given the pretty dry nature of the material.

This morning, he was discussing what is a "safe road."  In particular, he was talking about small, curvy roads and whether they are made safer when the turns are labeled with reflective posts and recommended speed signs.

Proper signage is optional

Research has shown that the average speed on unlabeled roads is lower than on labeled roads.  The proposed reason is that the unlabeled road is observably dangerous, so drivers are more careful and drive more slowly.  The labeled road lulls drivers into a false sense of security and therefore they drive faster.  His conclusion is that the labeled roads are therefore more dangerous, as evidenced by the higher average speed.

Here's the thing though... average speed doesn't matter!

What matters is the occurrence of a driver entering a turn with a speed that exceeds the combination of his driving ability, the character of the turn (radius, traction, etc.) and the performance of his car.  If the "fly off the road threshold" (or put another way, "bad enough") isn't exceeded, the driver keeps right on going, no matter how close he might have gotten.  The conclusion drawn by the author is therefore baseless.

Further, if we imagine that the presence of the signage increases the average speed but decreases the standard deviation (or more specifically tightens up the distribution in one way or another), then we're probably less likely to hit the bad-enough threshold and have an accident, even though average speeds are higher.  This seems believable, since better information means that people don't have to guess how fast to go through the curve, and they should converge on something similar to the posted, recommended speed.

made with @Risk by Palisade

In the little simulation I did here, we have two distributions.  The blue represents the "no signs" scenario, where the driver has no idea what they're in for and tends to drive slower.  The red represents the good signs scenario, when everybody knows what's coming.  I set the "bad enough" threshold at 40 mph.  You can see that 0% of the good signs people crash, but 1.3% of the no signs people crash.

There is an analogous concept in marketing, called the "good enough" line.  The idea is that people have some threshold for purchase that depends on certain quality vectors.  In a car, for example, a driver might have a minimum requirement for horsepower, seating capacity, gas mileage, etc.  If your car doesn't meet that threshold, that person won't buy it.  If you exceed that threshold, they won't pay any more for the car and you don't monetize the value.  It's all or nothing.  This concept applies to lots of things.  While the car crashes when it exceeds the threshold, the customer buys something when their desire for the product exceeds the threshold.

So here's the problem.  Statistical process control (SPC) is all about removing variability.  Mass marketing and homogeneity of product offering is the way of the world.  What are the actual differences between different Android phones, for example?  As things move toward the red distribution, fewer and fewer outliers will exist in the tail, and fewer and fewer people will reach the purchase threshold for your product.

So the takeaway is that new products should desperately avoid the middle path.  If you're going to put out something new, put out something really new.  Expect most people to dislike it.  What you need is a few people who will like it, and you won't get there if you don't produce the wildly unpredictable product.

Growth and ungrowth - forcing the vote

Over the past 30 years the US economy has grown.

from http://data.worldbank.org/

At the same time, middle class incomes have stayed flat.

from: http://www.advisorperspectives.com/

I found a really great breakdown and analysis of earnings trends at this website.  It's interesting to note that the sources here are not political sources or think tanks.  This is the US census and a couple of investment advisers that have aggregated and charted the census data.

At the same time, there has been considerable growth in the income of middle class women, and some minorities.

http://www.bls.gov/opub/ted/2013/ted_20131104.htm

The majority of middle class men have not been realizing these gains.  Additionally, many recent college grads are not experiencing these gains.  Link to wikipedia on income

There is a significant body of research on happiness that points to the concept that an individual's sense of happiness is a function of how they feel they compare to their self-described set of peers.  When a person looks around at their friends they do a natural, precognitive comparison that establishes happiness on a sliding scale.  In the case of the economy, the rule is similar.  White men are still better off than most ethnographic groups in the US, but when they measure themselves based on growth, they feel left behind (they are left behind in growth).

Lastly, I'll leave this article on the slide of the American Middle class from the NYT.

Now I will make a logically unsubstantiated leap here and posit that a vast majority of Donald Trumps supports are accurately described by the above economic picture.  I believe that Trump's base is built on the frustrations of white men (and those who are close to them and care about them) who have legitimate economic grievances regarding their individual situation.

It is really a shame that Trump is their mouthpiece, however, because his root cause analysis is deeply flawed.  I don't think there are any credible economists out there who would blame illegal immigration for the trends I identified above.  Certainly there are no economists who would blame ISIS or gay rights or a woman's right to freedom from sexual assault or any of Trump's other defining positions.

The shame here is that he a) provides an unproductive distraction to the people who are getting screwed.  They're now getting screwed by him, too.  b) His angry rhetoric is producing a dangerous death spiral in the form of a reactionary populist echo chamber.  c) the result is that everybody who has been experiencing the gains in the economy assumes that those who are left behind are just racist crackpots.  The whole argument becomes illegitimate.

Ironically, the result is very similar to what you see with Black Lives Matter.  White men are not the subject of negative implicit bias and therefore are not systematically and consistently exposed to negative (and often lethal) encounters with the Police can't wrap their heads around the problem.  They look at the movement, find the worst in it, see some looters or rioters and decide the whole thing is bullshit.  Same story, different disenfranchised and marginalized set of experiences.

Metacognition, forecasts and estimating ability

tl;dr: Odds are pretty good that your opinions on everything are worse and more ill-informed than you think they are. Suspend judgment about the ability of others (positive or negative) because it is probably just an echo effect (they're brilliant if they agree with you and stupid if they don't). This is all especially true if you aren't an expert in the field under consideration. Don't apply this logic to others, apply it to yourself first.

Metacognition refers to the act of thinking about thinking. This is the same sort of concept as Freud's superego, or what phenomenologist philosophers call our consciousness. Kahneman might refer to it as "slow thinking".

In general, our brain has two methods for making a decision. By "decision" I mean just about any sort of conclusion about something. There is a fast method that runs ahead and draws its conclusions based on instinct, training, experience and mental shortcuts. The fast method is great for things that we are evolutionarily prepared for and for things where training is effective. Generally training is effective for things that require quick reflexes and that provide immediate, salient feedback. Fast thinking is gut feel and reflexes and is not something we are aware of consciously because it doesn't formulate in language -- it forms in action.

The slow method is what we think of as "thinking". It is logical, it handles new ideas and concepts. It is what we observe as "thought". It presents itself in our language and is shaped by the way our language works. Slow thinking spends a lot of its time confabulating a logical reason why the fast method came up with what it did. A lot of our thought is spent thinking about what we just did and justifying it to ourselves.

This where our ability to forecast comes in. Most of the time, when asked about the future, people immediately have a gut feel. That gut feel is hugely biased (too many forms of bias to go into now). As long as we don't get immediate and obvious feedback that the forecast was wrong, our brain has mechanisms in place to pat itself on the back for another great forecast. Even if the forecast is terrible, if the feedback is subtle or slow, our brain still thinks it did a great job and our metacognition formulates a story to back it up.

Unfortunately, most of the forecasting situations we face in the modern world don't provide immediate and obvious feedback.

This is the reason that we're all bad at estimating our ability. We are best at identifying problems when we have expertise with a subject. The better we are at something, the more capable we are of identifying when we screw it up. Conversely, the worse we are at something, the worse we are at identifying how bad we are. We blithely go along, thinking we're pretty awesome at the stuff we're worst at. Those who are the best at things are also the most aware of their shortcomings.

There are lots of examples of this in the world. I guarantee that Tiger Woods is far more critical of his golf game than a typical weekend duffer. Proof-reading the English language is another great example--if you don't know the rules of grammar very well, then you won't be aware of your errors. Driving is another example where people are notoriously bad at identifying their own skill (except race-car drivers, who are typically very slow drivers on city streets).

So what?

Well, this whole thing applies to business, sure. I think what's really interesting is how much it applies to politics. Look at how many people who know nothing about economics are judging the economic policy pronouncements of the Presidential candidates. Look at how many people who know nothing about police training are hypothesizing about their motives. Look at how many people who know nothing about life experience as a black American are denigrating their perspective. Science would tell us all those people are probably wrong and atribiliously wrong about it because they are even ignorant of their ignorance.

Mutual Admiration Society

Be careful what you hear, it may not have been what was said (or what was meant).

I just finished Malcolm Gladwell's wonderful podcast series "Revisionist History".  There are many interesting threads through the various episodes, but the big meta-theme that loomed large for me was the danger of confirmation bias.  I think it was the second episode  (the one about Saigon) that really hit on the issue.  It is worth the listen.

This problem is giant when you're building a business model.  It is very hard to keep somebody in the room who honestly thinks your idea is stupid, but that person is essential.  I'm not talking about some devil's advocate who will throw out half-baked objections before rolling over and giving you a false sense of accomplishment.  I mean somebody who really thinks the idea won't work and is willing to try hard to articulate why.

"Oh, you're so smart and your new business idea can't fail!"

What comes naturally is that you will find somebody else who is enthusiastic about your idea and will be supportive.  Often this is somebody you already know, and probably you're on friendly terms. After all, that's why you brought the idea to her in the first place.  She'll talk up how great your idea is and you'll recognize what a genius she is for liking your idea.  The two of you will add more friends until you've built up a real, proper mutual admiration society around self-love as represented in lavish praise for one another's brilliance.

"We're all so smart and funny and we're all going to be super-successful!!"

This is great when you're starting out!  Everybody except the most ruthlessly anti-social people need external approval of some kind.  The trick is knowing when it is time to stop bolstering your confidence and adding good news and start sorting out the potential problems, doing your pre-mortems with serious abandon and sorting out the real contingency plans.

At that point, it's essential to realize that sometimes the smartest people in the room are the ones who are willing to say "I don't get it" and your best friends are the ones who are willing to say "that's stupid."

Management Fail!

A developer in my org came to me with a technical question she was working on puzzling out around something we call "Decision Units" in portfolio.  Decision units itself is a topic worth its own post, and not really important here, but the gist of it boiled down to this:

Her: "We have a problem handling decision units that exist in multiple business units, where analysts in different groups aren't aware of all of the logic relationships that might exist for that project."

Me: "What's the problem?"

Her: "Well, the way we're handling things right now, if anybody makes a change to the logical relationships for a given project --if they add a dependency or some sort of optional relationship or anything-- it will wipe out all the stored values for that project.  Worse yet, the analyst who loaded them before won't know they were wiped out, and when they go back to load them again, there will be a bunch of new alternatives there that they never heard of!"

Me: "That is a problem!  What do you think we can do about it?"

She gave me a good explanation of the issue that indicated she understood both the technical character of the problem and the business character of the problem. 

She really had her head around the thing.  She then delivered a very technical, possible solution that involved setting some flags and having users determine several things in advance and some other business rules around the process flow.

Me: "I think we can do something simpler.  Here's what I think..."

And then I went on to sort of brainstorm out a couple of default states and business rules that eliminated most of the possible sources of trouble.  Lastly, I sketched out an approach that would handle the last situations elegantly and simply and without requiring the users to do anything they weren't already doing.

Her: "Wow! That was really great!  I'm glad I came to you.  Here I was, thinking of something super-complicated and all I needed was to talk to you and you had a simple answer right away!  Thanks!"

I felt pretty good about myself for a few minutes and then realized I had totally failed her as a manager.

With a little coaching I'm pretty sure that she could have come with the answer on her own.  Maybe a few questions, or at the very least I could have structured the conversation more collaboratively.  I sort of pontificated how I think it should work and we were done.  My developer was left maybe feeling good about my skills, but probably a little less confident in her own.  That's a big fail as a manager.

this is a clever metaphor!

I think this sort of thing is why it's very hard for managers to make those "Career Pipeline" transitions from independent contributor (IC) to manager.  We build up these technical skills and are rewarded for them, and often they become a source of pride and self-worth.  When problems come along that are in the sweet spot, we think immediately of a fix.  We miss the chance to do our new job, which is coach the person bringing the problem as they figure out a fix.

Intelligent Software Building: products, platforms, programs

On a regular basis I see folks who are trapped by the siren song of reusable code.

Now, don't get me wrong.  Reusing code is a wonderful thing, and it's a good goal, but it's important not to get carried away.  There are basically three levels of reuse:

1. Programs - this is when you reuse parts of your own code, on a personal sort of level.  You built these routines and snippets, documented them and like them.  They're something you've done before and know you want to do again, so you were kind and thoughtful to yourself and wrote it in a way that you would recognize.

2. Platforms - this is when you specifically plan for reuse when you're writing something.  Platforms refer to a broad area of capability that is important and that you'll want to build on.  Platforms require serious forethought.  They also require serious documentation and some support.  Typically this is something you will share with other members of your team.

3. Products - this refers to a software platform that you want to make broadly available.  When you create a software product you absolutely have to have clear documentation in and out of your code.  You have to plan for wide deployment and you have to have support people in place.  When changes are made, you have to have a thorough test apparatus in place to understand what the impact of your changes will be on others who have built their software with your product.

especially to yourself...

Be kind to the person who will inherit your code, because often that person is you. 

Many times I have cursed the me of weeks or months ago for leaving a mess for me.  When I'm kind to me, I always appreciate it.  Reuse of your programs is just good practice, and naturally happens when you're writing thoughtful and clean.

After you've experienced those benefits a few times, you'll start dabbling around with platform style reuse.  You'll think "hey!  that little block of code has been so useful to me, I bet others would like it!"  Before you know it, you have a whole set of interconnected things that you've put together and share.  You probably don't have any sort of obligation to support it, but it's fun and you're supporting your own work through the thing anyway, so it just makes sense. 

Ain't nothin' wrong with Platform style reuse.

Where you have to think long and hard is around the Product-style reuse.  This makes sense if your goal is to sell this code as a development tool.  This is something like D3, or Telerik, or the Unreal Engine, or Havok or anything like that.  More often than not, I see this when developers fall in love with their own, potentially reusable programs, assemble them into a platform and then decide the whole world needs it.  You will have tons of support work to do.  You will have to maintain forward and reverse compatibility.  

If this is some sort of exercise in self-love, you need to stop now and slowly back away.  Really, unless you're planning on making enough revenue from this (or if internal facing, saving enough effort) to justify several developers and a testing team, this is just a bad idea.

Sometimes a platform may naturally find demand in the market.  Maybe the platform you built is truly wonderful and well documented and bullet-proof and you really really want everybody talking about how awesome you are... Even then, don't do it unless you have the support staff.  The first time some package you're including gets depricated or MS rolls off some capability you're counting on or any other of a million possible things, you're going to have all those people who were singing your praises on your IM, demanding immediate fixes...

software III

Don't pee your pants for nobody

This one is pretty similar to the "you'll regret the hack" thing, and I see this a lot with green developers.  With deadlines looming and a rush to get things done, you may be tempted to degrade your work from "software product" to "proof of concept".  What this means is that you're no longer building a software platform you can add to later, instead you're building specific features to try and get somebody off your back.  It works out a lot like the old metaphor about peeing your pants for warmth.

you're warm for a while, anyway...

Essentially, you solve your short-term problem but wind up with a bigger, long-term problem.  This is bad when it's a team decision, and even worse when a developer does this without telling anybody.  I've personally been in the situation where one of my developers did this sort of thing and I wasn't close enough or careful enough to know that's what was happening.  Once we got past UAT (user acceptance testing) and I started rattling off the next steps and new features to add, he turned defensive and told me he needed a few months to rework everything because everything he had just done was purpose built to get past that first deadline.  He literally set us back 6 months and absolutely destroyed my trust in him.

Hire fast, check-in faster, fire fastest

Back in the games days I had an employment model that was built around the fact that I had no idea how to hire people.  In retrospect, I was probably ahead of my time, since research seems to indicate that interviewing is almost worthless, GPA is a poor predictor of work performance, and things like pedigree and credentials are over-rated.  This was before I had ever heard of a behavioral inteview, much less experiened or conducted one.  Since I worked in computer games, every job opening I had was flooded with candidates, and often these candidates were non-traditional.  They were often wiz-kids with no college or formal training.  Sometimes they were older folks with a passion for gaming.  Some were hobbyists who wanted some extra income (that was fine, I had plenty of contract-work jobs available, too).  The hard part wasn't finding people, the hard part was knowing which ones I could count on.

This was especially important for my projects with really short timelines and really tight budgets.  One really flaky developer could derail the entire project (and with it the budget and with the budget my paycheck for weeks or months).

The approach I came up with was "Hire fast, check in faster, fire fastest".

I googled "you're fired" and somehow this image came up.  It somehow seems better than anything else I could have put here.

Basically, I gave a job to anybody who wanted one.  Every new employee would get a single work product to deliver and two weeks to deliver it.  The work product was always something I actually needed and something that I knew could be done in two weeks.  Anybody who didn't get the work done on time was fired.  No questions asked, no conversation, no excuses.  What I had learned was that anybody who is willing to flub their very first impression will continue to disappoint thereafter.  Honestly, this got rid of around 80% of the people I hired.

I've applied this rule in a general sense in many ways since.  If I'm looking to hire a painter and he shows up late to discuss the quote, I take him out of the running.  I once had a buddy ask me to help him find a contractor to build a block wall around his house.  The guy showed up 30 minutes late and I told my friend not to hire him.  My friend did anyway and since I'm telling this story here, I'm sure you can imagine he never got his wall.

Hit the easy problems first

There's the rule when you're playing pool.  When one of your balls is sitting right on the pocket, you don't shoot that shot.  It will probably get knocked in on its own and you're really just wasting a shot.  That mindset is completely misplaced here.

a tempting but misleading metaphor!

Really this is probably just a good project management thing.  If you leave the little problems until they end, nothing good can come of it.  If they really are little, you'll just have a shit-ton of little problems to sort through and fatigue will build.  If any of them turn out not to be little problems (and some of them will end up a lot bigger than you originally thought) you suddenly have big problems that you weren't expecting, popping up right at the end of your project, when you don't have time to course-correct effectively.

Do those little, easy-looking things as early as you can.  Sprinkling in small wins helps to keep you from getting worn down on big projects.  Also, it gives you the chance to adjust your timelines or features if you wind up facing some big, unforeseen shitshow!

this thing again

software development II

You will always regret the hack

Every project gets hairy at some point.  Deadlines start to loom, customer requirements start to tighten, subcontractors and ingredient suppliers are late or flake out entirely.  The team gets stressed and the iron triangle reminds you of its immutable force:

the ol' iron triangle

Adding resources at this point never works.  Even if you have some ninja team of superhuman developers in your back pocket, they won't be able to parachute in at the last minute and do anything right away other than churn the project and introduce more problems.

Slipping the schedule is always tempting.  Reducing features is often what happens.  Sometimes we're smart about it and try to stage them together; reduce the features in the first release, with a schedule for releasing the remainder of the features in the future.

The worst thing that happens is when the decision is made to fight the triangle and keep the same schedule with the same features.  When that happens something else gives--Quality.

In software, when quality slips this usually means that the developers start to introduce hacks.

Magic numbers start to appear instead of declared variables.  Convoluted blocks of nested code get tucked into other functions instead of broken out into separate functions.  Blocks of code get commented out.  Blocks of code go undocumented.  Temporary variable names that make no sense get introduced.  Single purpose design takes over and expandable, scalable features fall by the way-side.  Unit tests and BAT tests go unwritten.

Your software product effectively gets downgraded to proof-of-concept.  When you need to build more later, you're instead spending time refactoring the junk you slapped together.  When users find errors it takes 5x as long to troubleshoot.  At the end of the day, you give up a lot more than you gained and always, always regret it.

Software Development

I've been doing professional software development of some kind or another since the mid-90's.  I wrote my first program on a Timex Sinclair when I was 10 years old, and taught myself Assembly to try to squeeze more into the 2K of RAM it came with.

pictured here with the 16K RAM upgrade!!  That changed everything!!

A lot has changed since then (thankfully).  Along the way I learned a lot of lessons--all of them the hard way.  Looking back at my first attempts at building a company, developing a brand, organizing a development methodology and cultivating people I realize I did things about as wrong as I could.

Here are some of those lessons.  Maybe you won't have to learn them the hard way, too...

Produce excellence, especially when you're not asked for it

My first commercial products were euphemistically called "Casual Games" or "Budget Extreme Sports Games" or more correctly "Shovelware".  Google "Extreme Paintbrawl" and read some of the reviews.  Enjoy the laughs and then come on back.

Extreme Paintbrawl and the games like it were precisely what our publisher hired us to do.  One important guy at a publisher told us "You could literally take a shit on the CD and I could sell it with the right licensing, just get them done as fast as possible."  So that's what we did.  We banged out junky games as fast as we could, because that's what we were hired to do.

This kept the lights on, and kept our customer satisfied, but it didn't lead anywhere.  Later on, when we tried to land bigger, higher-quality projects, nobody was interested.  We had a reputation for producing junk games at light speed.  Literally nobody would give us a shot at producing a quality game with more time and more budget.  In retrospect, they were probably right.  We had a shovelware culture and I'm not certain we could have turned the corner.

At the same time, a friend of mine with a very similar company and very similar starting point aligned his people around producing beautiful, compelling games.  He had the same customers, the same deadlines and the same budgets.  They were late all the time, the games were completely over-engineered relative to the spec, and they didn't make any money on those games, but they proved that his group could do really nice work.  They successfully segued into AAA games while I wound up throwing in the towel and leaving the industry.

Get a monkey on a football

On one of my early projects we had to develop a considerable amount of development infrastructure from scratch.  We were building a new rendering engine, new level design tools, new asset management tools and were working out of a basement.  Weeks went by with no visible progress to show the customer.  They started freaking out and threatened to cancel the contract.

I came up with a schedule of visible milestones to deliver to the customer.  For the first one I said "Look, I don't care if we render a monkey riding a flying football around the course, we have to show these guys something!!"  "A Monkey on a Football" became our catch-phrase for tangible customer deliverables that may not be completely required for the technical aspects of the development, but are completely required to make sure that your customers know you're moving forward.

It is a lesson I should have internalized from an earlier interaction with a guy named "Joris" in the Netherlands.  Joris had worked for us before and we hired him to write the AI for Santa in one of our early projects called "Nuclear Winter".  Joris kept telling us he was making progress and we kept believing him without demaning tangible proof.  When the time came for Joris to turn in his code, Joris disappeared.  Then he turned in corrupt zips, then disappeared again.  Eventually, Joris admitted that he hadn't done any work at all.  He had spent his entire advance on phonesex with some woman in Kansas that he was in love with and he needed more money from us and then he promised he would get right to work.  

So now, I always demand a monkey on a football from the folks who are working on my team, and I always give a monkey on a football to my stakeholders.

I have a few more, but my blogging time is up for this week... I'll post them another time.

options in the wild: part II

At the very beginning, I did a straightforward ROI type of analysis.  The decision seemed very clear from that perspective.  There was one line that took a little more work to get to, but was half as long and had twice as many windows operating.  It should have moved twice as fast.  Easy choice!

I hadn't considered the uncertainty in the model.

The people in line "d" were long a call option on all of the windows to the right.  If any of the windows from "e" through "h" opened up, they could easily exercise that option by walking over and moving to the new line.  I hadn't thought of that until it happened!

Once window "e" opened up, I sat there considering moving to line "e", thinking they might open up "f" or "g" or "h" and I didn't want to miss that opportunity, even though it meant moving to the back of the line.  I decided I was pot-committed to "b" when the other option in the mix made itself known...

The crew of any plane that happened by was long a call on window "a" and "b"!  Since I was in line "b", I was effectively the counterparty and short a call on those windows.  When the crew turned up and took over the windows, they exercised their call and my line came to a stop.

This story is a pretty good representation of the value of using a real options decision framework for a couple of reasons.  The most obvious reason is the importance of involving uncertainty in your decision.  If nothing had changed, my first choice was unquestionably the best.

The second reason is the importance of having a dynamic decision framework that can adapt to new information.  It's easy to make a straight-forward ROI type decision when you ignore the changing world.  One line was served by two windows and was half as long!  That's an idiot-proof decision!  A few minutes later, the decision was more complex.  I was at the end of a line that was roughly the same length as the others, but didn't have the advantage of more windows possibly opening up.  A few minutes after that, I was at a stand-still.  A robust, options-based decision framework that encapsulated the upcoming decisions would have helped.

From an executive decision making perspective, this sort of decision tool is almost never available.  What can be available, however, is some intelligence about what the inflection points are.  If I had any idea about the chances of another window opening up, or what the chances of a crew showing up, I could have produced some heuristics about comparative lengths of each line and made dynamic adjustments to my strategy when the real world changed in front of me.