EHR: It is like herding cats

I spent a summer in Weaverville, North Carolina, just outside of Asheville. (I couldn’t find it on the map either.) That summer, I was the head wrangler at Windy Gap, a summer camp for high school kids. I’m not sure I’d ever seen a horse, much less ridden one, so I guess that’s why they put me in charge. I thought that maybe if I dressed the part that would help. I bought a hat and borrowed a pair of cowboy boots from a friend; the boots were a half size too small, and I spent the better part of the first night stuffing sticks of butter down them trying to get them off my swollen feet.

The ranch’s full-time hand taught us how saddle the horses and little bit about how to ride. In the mornings we had to herd the horses from the fields, bring them into the corral, and saddle them. The other wranglers would ride out to the field to bring in the horses, while I being the least experience of the wranglers would race after them in my running shoes trying to coax them back to the barn. We would take the children for a breakfast ride halfway up a mountain path where we would let them rest and cook them a breakfast of sausage and scrambled eggs. One morning there were a group of 15 high school girls sitting on the fence of the corral. I walked up behind them carrying two saddle bags filled with the breakfast fare. I slung the saddlebags over the top rail of the fence, and hoping to make a good impression I placed one hand on the rail and vaulted myself over. I landed flat on my back smack dab in the middle of the pile of what horses produce when they’re done eating—so much for the good impression.  That earned me the nick-name, “Poop Wrangler.”

I brushed myself off and saddled my horse. The moment I gripped the reins the horse reared, made a dash for the fence and jumped it in one motion. I could tell the high school girls were impressed as I flew by them. Both of my arms were wrapped around the horse’s neck, and I had my hands locked in a death grip. I yelled, “whoa” and stop”, only to learn that the horse didn’t speak English. We raced the 200 yards to the dining hall, stopped on a dime, and raced back to the corral, as the girls continued to cheer. One final leap, and I was back where I started; on the ground, in the corral, looking up at the girls. I took a bow and quickly remounted my steed. The full-time ranch hand came over and instructed me rather loudly, “You can’t let the horse do that. You have to show the horse that you’re in charge.” After that piece of wisdom he grabbed my horse by its bit, pulled its head down, and bit a hole in my horse’s ear. I’m not sure what kind of in an impression it made on my horse. I guarantee you it made an impression on me.

Horses aren’t very intelligent, but they know when you don’t know what you’re doing, when you’re bluffing—dressing like a cowboy didn’t even fool the girls, much less my horse—I guess he hadn’t seen many westerns. Here we go—you had to know where this was headed.

Selecting and implementing an EHR will be the most complex project your hospital will undertake.  If you do it wrong, you may not look any better than I did laying on my back in the corral.  You won’t have girls laughing at you, but you also may be looking for another line of work.

You don’t want to read this, but if your projected spend exceeds ten million dollars, your chances of success, even if you do everything right, is less than fifty percent.  I define success as on time, on budget, functioning at the desired level, and accepted by the users.  That’s reasonable, correct?  We don’t need to talk percentages if you don’t do everything right.

These figures come from the Bull Report—that’s really the name, honest.

The main IT project failure criteria identified by the IT and project managers were:

missed deadlines (75%)
exceeded budget (55%)
poor communications (40%)
inability to meet project requirements (37%).

The main success criteria identified were :

meeting milestones (51%)
maintaining the required quality levels (32%)
meeting the budget (31%)

How is yours matching against these?  Given a choice, sometimes I’d rather be the horse.

The Cat in the Hat’s EHR Philosophy

My mind is a raging torrent, flooded with rivulets of thought cascading into a waterfall of creative alternatives. (Hedley Lamar—that’s Hedley)

Let’s see if we can tie this collection of thoughts into something that won’t waste your time or mine.

The sun did not shine.  It was too wet to play.  So I sat in the house all that cold, cold, wet day.  It was too wet to go out and too cold to play ball, so I sat in the house and did nothing at all. (Dr. Seuss)  It was around that time when my wife decided maybe this whole sitting around thing wasn’t optimizing my time, so she decided “we”—which can also be interpreted to mean “me”—should caulk the master shower.  Personally, I thought that why God invented the Yellow Pages, you know, the whole thing about, “Let your fingers do the walking.”

I notice we just blew through an entire paragraph without accomplishing anything.  Sorry.  I got my designer tool belt, the same one I’ve had for twenty years—still looks the same as the day I bought it.  Today’s Roemer Minute—the less you know about what you’re doing, the more important it is to dress the part.  (This does not seem to work with the people whom I’ve told that I’m studying cardiology.)

Tool belt.  Tools—caulk taker-outer, caulk puter-inner.  Paper towels—the need for these will become clear.  Worse case, this is a ten minute job, but if I finish too quickly, there will be additional assignments coming my way.  The trick with caulking is that the success or failure can all come down to how much of the plastic tip you circumcise (can I say that on TV?).  Too much and caulk is everywhere, not enough and it is nowhere.  I made the incision and started to lay down the first bead.  It was quickly apparent that I should have used clear caulk as the white stuff stared back at me like bleached bones—I try and add a little medical flavor wherever I can.

I’ve watched the same shows as you.  Sometimes people spread the caulk with a tool, others prefer a wet finger.  I am equally unskilled with both, so I went with the finger method, smoothing the caulk into the joint.  I wipe my sticky white finger on the paper towel, place the towel on the limestone tile, and return to work, only to notice that although the caulk looks good, my finger created to parallel lines of caulk on either side of the repair, sort of like a snow plow does.  I grab another piece of paper towel and begin the process of trying to remove the excess caulk, finally tossing the paper towel to the side.

Fast forward twenty minutes.  The caulking is done.  My hands are so white it looks like I am wearing a pair of Mickey’s gloves.  (That’s spelled M-O-U-S-E.)  As I wipe my hands with a used piece of towel—there are no more clean ones—I unknowingly step on one of the pieces.  The piece sticks to my shoe.  I retrieve the other pieces and notice that the caulk which had been on the paper towels is now spread all over the tile like someone had a food fight with smores.

Whatever I touched only exacerbated the problem.  I am immediately reminded of the Dr. Seuss book, “The cat in the hat comes back.”  In the book, the cat goes from good intentions to spreading a pink stain over everything—sort of like me with the caulk.

Sometimes good intentions don’t add up to much.  I’d wager that everyone in the EHR process has good intentions.  Sometimes it’s more important to pair good intentions with good skills.  Let’s call EHR one of those sometimes.  Good intentions are okay up to the point when you’re dealing with two or more commas on the cost side.

Here is today’s thought, the one reason you keep coming back to this site. Why did you implement EHR?  The answer will surprise you.  You did it for one simple reason, to get people, doctors and nurses, to do what you want them to do and to do it in a way you want them to do it.  EHR will modify their behavior.  I am sure yours did, but did it modify it in the way you wanted?

Most times it’s good to call a professional before you start tracking caulk across the floor.

When Good Projects Go Bad

What are the EHR fail-safe points?  The points of no return, beyond which you can’t recover without exploding the budget and the  schedule?

Although there are several, my take is that the most important one is the planning process.  Without the right plan, an organization is entering its permanent whitewater moment.

What plan do you have to rigorously evaluate the plan before you step off the EHR cliff?  Are you stepping off with a parachute or a bag of rocks?

My best – Paul

Step away from the computer

Our middle school child is in the middle of a family consumer science project (home economics) to organize one room in our home.  He has redefined the project so that he reorganizes during commercials, and he is seven hours into a project involving our walk-in closet.

While watching the news it occurred to me that something is missing from my life, I do not belong to a gang, not even a little one.  So, I have decided to start one, a white collar gang of consultants.  A rough and tumbled, manicured group of professionals.

Instead of gang emblem, I am thinking each member of the gang will have their own embossed business card.  We will come up with creative gang nicknames.  For myself I am vacillating between ‘Dr. Knowledge’ and ‘The Voice of Reason.’  Instead of Harleys, we will roll through town to our national rallies on monogrammed Segways, and instead of leathers we will dress in Armani.

Mothers will hide their children from us as we power noiselessly down Main Street at four miles an hour, and their CPA husbands will turn green with envy.  We might not win many fights, but we will have the satisfaction of knowing we are smarter than those who beat us to a bloody pulp.

Sounds too good to be true, doesn’t it?

There are days when it doesn’t pay to be a  serial malingerer, and when it does, the work is only part time, but I hear the benefits may be improving as I think I heard somebody mention healthcare is being reformed.

I don’t know if you are aware of it, but there are actually people who have taken an Alfred E. Newman, “What, me worry” attitude towards EHR.  For the youngsters in the crowd, Alfred was the poster child for Mad Magazine, not Mad Men.

Just to be contrarian for a moment–as though that’s out of character for me–most providers have no need to fear–does this happen to you?  You are writing aloud, trying to make a point, and the one thing that pops into your mind after, ‘there’s no need to fear’ is “Underdog is here.”

Anyway, since many providers haven’t begun the process, or even begun to understand the process, there is still time for them to lessen the risk of failure from an EHR perspective.  Many don’t want to talk about it, the risk of failure.

Here’s another data set worth a look (The Chaos Report).  They went a little PC on us calling them ‘Impaired” factors.  EHR impairment.  Step away from the computer if you are impaired, and take away your friend’s logon if they are.  These are failure factors.

Project Impaired Factors % of  the Responses
1. Incomplete Requirements 13.1%
2. Lack of User Involvement 12.4%
3. Lack of Resources 10.6%
4. Unrealistic Expectations 9.9%
5. Lack of Executive Support 9.3%
6. Changing Requirements & Specifications 8.7%
7. Lack of Planning 8.1%
8. Didn’t Need It Any Longer 7.5%
9. Lack of IT Management 6.2%
10. Technology Illiteracy 4.3%
11. Other 9.9%

My take on this is with overall “failures” so high, several respondents could have replied to “all of the above.”  Also of note is that these failure reasons differ from the ones listed previously.

Who knows, maybe if we multiply them by minus one we can call them success factors.

The relationship of EHR to mobile health

Why not see if we can muddy the waters even further?  Is this a true statement: EHR is to the N-HIN of HIT applications, as wireless health devices and their apps are to the HIEs?

The good news is that in the last two to three years more variety, flexibility, and adaptability have been demonstrated by these mobile devices and their related software applications than has come out of traditional healthcare systems since Mr. Gore built his first lockbox.

The bad news is that in the last two to three years…(ibid).

Healthcare IT meets Ben & Jerry’s

The idea for this blog came about after reading a PowerPoint presentation by Doctor Alberto Borges.  All mistakes can be attributed to me.

When one is witness to the number of external influencers trying to shape policy on healthcare, reform, and healthcare IT, the best one can hope for is that hidden somewhere under the pile is a pony.

But let’s be real—the pony has suffocated.

While it is okay to point the finger of blame at the usual suspects—payors, lobbyists, and the lawmakers—let us not forget to ensure to point out the role paid by the healthcare IT applications vendors.

“Who me?” You ask.

Decrease costs, increase quality, decrease errors.  I did not invent these words; they are written on your websites.

Prior to 2008 the value of EHR vendors’ stocks plummeted.  Look at them now.  How does one explain the difference?  Can the gains be attributed to vendors having rewritten their applications?  New technological innovations?  If not, what else could it be?

Meaningful Use.  Meaningful Use tied to Medicare payments and a twenty billion dollar incentive to get providers to do something they otherwise would not have done.  Could life be any better if you are sitting in the EHR Tower’s corner office?

What if we think about the issue this way?  Let us suppose all of the leading ice cream manufacturers lobbied Congress to push for including ice cream machines in all new cars starting in 2012…silly idea, but then again, so is Meaningful Use.  Not only do the ice cream machines have to be installed, but they have to be able to communicate with one another.  That way, if I happen to rent a car, the ice cream machine in my rental will already know what type of ice cream I like to eat.

Now we already know that no car buyers and no car builders will think much of being forced to buy or make cars with pre-installed ice cream makers.  But, perhaps there is a way around that.  Maybe in some self-serving way the Cookies and Cream lobby can convince Washington of the merits of pushing through their agenda.

Time passes, and still the idea is not getting much purchase.  What happens next?  The ice cream manufactures get Congress to pass the Ice Cream Tech Act—ICTA.  And as part of the ICTA Act, Ben and Jerrys, Baskin Robbins, and Haagen Dazs convince our friends to offer the auto manufacturers a twenty-billion dollar rebate for building cars with built-in ice cream makers—ICTA Initiatives.

Now, why would the Ben’s and Jerry’s do this?  Good question.  They will do this because they know that without offering a large financial incentive the car company executives will not do what they want them to do.  Now to insert ice cream makers, you can imagine that the car companies will have to go way off message, will have to change their strategy, and will have to incur all sorts of costs that have nothing to do with selling cars.

And that brings us back to the start of this story.  There is a reason why EHR vendors needed to lobby Congress to put forth more than twenty billion dollars of lottery money, and that reason is healthcare providers would not be doing EHR the Meaningful Use way unless there was a monetary reason to do it.  There certainly is no business reason to do it.

And for the most part, if providers calculate an ROI on EHR, even factoring in the incentive payments, there is still no reasonable financial argument that can be made.  In fact, when the cost to meet Meaningful Use is factored in, the financial argument worsens.

So, what will happen?  Here is what we know so far.  The Meaningful Use deadlines draw closer, meaning there is less time left to get the incentive dollars.  Implementations of EHR continue to falter, be redone, and under deliver.  The result is that the purchase of EHR systems will slow, and many EHRs will be uninstalled.  When there is no time left to get the incentive dollars, only then will EHR implementations be driven by the needs of the providers, and the government will no longer be driving the process.


AN EHR Vendor Selection Strategy–Vendor Darts

Is there a best Electronic Health Records system? Perhaps Cerner, EPIC, GE, or McKesson?  For those who have followed my writing, you’re probably thinking my answer is “None of the above.”

I’ll do one better, and I write this with the utmost sincerity—it does not really matter which vendor you select.  As the EHR vendors reading this pull themselves off the floor, permit me to explain why.  Researching the question this is very little information to support the notion that any of the major hospital EHR systems quantitatively stands out from the others.

There are a few sites that offer user assessments across a range of functions, but those have at most three opinions—not enough to consider statically significant.  There are plenty of EHR scorecards and comparison tools, just not many scores.  The vendors’ sites do a poor job of differentiating themselves from their competitors.  Vendors use superlatives and qualifiers in an attempt to differentiate themselves.  When one considers the basic functions that make an EHR an EHR, the top vendors all have them.  No vendor highlights major clinical or business problems that their solution solves that another vendor does not solve.  Instead, they state they do something better, easier, more flexibly—none of which can be measured by prospective clients.

Imagine, if you were an EHR vendor, and you knew that your product did things to benefit a hospital better than the other vendors, wouldn’t you have an independent competitive assessment, some sort of “Consumer Report” chart and evidence to support why you are better?  Of course you would.  You would highlight your superlatives.  I have not seen one that would be very helpful.  The only information I found that might be worth a read comes from Klas Research.  However, the names of the modules rated are vendor specific, and none of the vendors use the same names.  It will give you a feel for how a small sample rated features within a given vendor, but there is no data to suggest how those ratings compare among vendors.

Even if there was a good comparison, the other thing to learn from this is all the areas that aren’t listed imply that the vendor is either no better or perhaps worse than the competition.  Cream rises to the top—we are left to choose among brands of milk.

One vendor may have a better medical dictionary than another, yet that same vendor will lack rigor in decision support.  No single vendor seems to have their customers doing back flips in their testimonials.  Some score high in their ability to deliver a complete inpatient solution and fail in their ability to integrate with other vendors.  Others hurt themselves during the implementation, user support, response time, and the amount of navigation required to input data.  Some EHR vendors posit their systems as being better at meeting Meaningful Use or passing all of the Certification requirements.  Ask them to name a single installed client for which they have met these.

Why doesn’t matter which vendor a hospital selects?  The reasoning holds not because all hospitals are the same, rather, it holds because were one to perform a very detailed comparison of the leading EHR vendors with a Request for Proposal (RFP), they would prove to be quite similar.  You might find significant separation if you only compared ten functional requirements.  You would expect to find less separation by comparing several hundred, and quite a similarity if you compare a thousand or more requirements.  The more you look, the more they seem the same.

Although the vendors will differ with respect to individual requirements, when evaluated on their entire offering across a broad range of requirements I would expect each to score within one standard deviation of the other.  You may be equally served playing a round of Vendor Darts.  However, make sure you sharpen the heads of each vendor prior to throwing them to make sure they stick to the dart board.

Reason 2.  It is possible to find hospitals who will give outstanding references for each of the leading vendors.  It is equally possible to find users in hospitals who have implemented one of the “leading” vendors’ systems who will readily tell you that the purchasing the system is the worst business decision they ever seen.  More to the point, every vendor A has probably had at least one of its implementations uprooted and replaced by vendors B, C, or D.  The same can be said for vendors B, C, and D.

If this is a fair assessment, what accounts for the difference?  How can we account for why one hospital loves a given EHR system and another one hates the same system?  Chances are they both needed about the same solution.  Chances are they received about the same solution.

Here’s the difference.  The hospital who thinks they made a good choice:

  • Had a detailed strategy and implementation plan
  • Paid as much or more attention to process alignment, change management, and training as they did to the implementation
  • Managed the vendor instead of being managed by the vendor.

Simply put—the problem is not the EHR system.

One other thought.  “Pay no attention to the man behind the curtain—the Great Oz.”  Do not put your scarce capital into a solution just because it offers or promises either Certification or Meaningful Use.  Yes, there is much discussion about both of these.  The industry stops and holds its collective breath each time a new set of stone tablets are brought forth from the ONC or CMS.  You can meet Meaningful Use with a Certified system and still wind up with a system the users hate and that does not support your business model.

Here is something else I cannot explain.  For those hospitals replacing a one hundred million dollar EHR with another hundred million dollar EHR, why do they think the second system will be any better?  If the systems are not materially different, the only way to get a different result is by changing behavior, not changing systems.  Why make the same mistake twice?  What could be so wrong with the first implementation that an expenditure of far less than another hundred million could not solve?

What is the cost of EHR 2.0 not working?

EHR–what do you do when your vendor leaves?

The room was silent except for the humming noise made by the computers’ fans. It smelled of stale cigarettes and spilt hops.  The venetian blind the program manager had been wearing as a hula skirt lay bent and twisted next to the large aluminum trash can.  Other than the light coming from the smashed exit sign, the only other illumination came from the few remaining flat screen monitors.

I made my way across the floor of the EHR War Room, accidently kicking the empty bottle of Grey Goose.  I watched without interest as it spun around on the damp commercial carpeting as though it was playing a solo game of spin the bottle.

The ten page project plan hung in tatters on the far wall, itself the victim of a game of nacho-darts.  Of the thirty-five desks in the room, all but four were empty.  The empty desks sat barren; no computers, no user manuals—no scraps big enough for the other Whos’ mouces.

Friday’s party was a joint celebration of the project team for the EHR go-live.  The thirty-one members the vendor had supplied were in such a hurry to leave the project at the end of the party that two of them were almost trampled to death by their mates as they rushed the door.  The scene was reminiscent of the Running of the Bulls in Pamplona.

Sally’s desk sat next to the wall chart that displayed the daily decline in productivity.  Her head rested on her desk while her monitor’s coral reef screen saver displayed a single yellow tang swimming from left to right and back again.

Larry was staring aimlessly into a Styrofoam cup, using his index finger to stir what was left of a room temperature margarita.  “What now Boss?” He asked.  “We all know it does not do what it is supposed to do.  And, you know who they are going to blame; us.

“Well, at least we have the Meaningful Use money to look forward to,” chimed Sally.  “That should make them happy upstairs.”

“We spent more money on chips and salsa than we will see of the ARRA money,” I told her.  She slumped back to her desk.

What now indeed, I wondered.  What do we do once all the money has been spent and the subject matter experts leave?

“Maybe if we do not say anything nobody will know,” I offered.  “Let’s pretend we know something about ICD-10, keep our heads down, and try to look busy.”

What should we do?  What would you do if your mother asked you?

EHR’s Gordian knot

There were four of us, each wearing dark suits and sunglasses, uniformly walking down the street, pausing at a cross-walk labeled “consultants only”—I think it’s a trick because a lot of drivers seem to speed up when they see us. We looked like a bad outtake from the movie Reservoir Dogs. We look like that a lot.

Why do you consult, some ask? It beats sitting home listening to Michael Bolton or practicing my moves for, So You Think You Can Dance, I tell them.

Listening to the BBC World News on NPR whilst driving, there’s one thing I always come away with—they’re always so…so British. No matter the subject—war or recession—I feel like I should be having a proper pot of tea and little cucumber sandwiches with the crusts removed; no small feat while navigating the road.

Today’s conversation included a little homily about the Gordian knot with which the company Timberland is wrestling, questioning whether as a company Timberland should do well, or do good. (Alexander the Great attempted to untie such a knot, and discovered it had no end (sort of like a Möbius strip, a one-sided piece of paper–pictured above. (For the truly obtuse, among which I count myself, the piece of paper can be given a half twist in two directions; clockwise and counter-clockwise, thereby giving it handedness, making it chiral—when the narrative gets goofy enough, sooner or later the Word dictionary surrenders as it did with chiral.))) I’m done speaking in parentheses.

Should they do well or good? Knowing what little command some people have of the English language, those listeners must have wondered, why ask a redundant question. Why indeed? That’s why I love the English, no matter the circumstances they, they refuse to stoop to speaking American.

Back to Gordo and his knot. That was the point of the knot. One could not have both—sorry for the homonym. Alexander knew that since the knot had no end, the only way to untie it was to cut it. The Gordian knot is often used as a metaphor for an intractable problem, and the solution is called the “Alexandrian solution”.

To the question; Well or good. Good or evil. Are the two choices mutually exclusive? For an EHR? They need not be. The question raised by the BBC was revenue-focused (doing well) versus community or green-focused (doing good). My question to the reader is what happens if we view EHR with this issue as an implication, a la p→q.Let’s review a truth table:

if P equals if Q equals p→q is
define requirements increase revenues TRUE
play vendor darts increase revenues FALSE
ignore change management increase revenues FALSE
no connectivuty increase revenues FALSE
new EHR software increase revenues FALSE
change processes increase revenues TRUE
eliminate waste increase revenues TRUE
decrease redundancy increase revenues TRUE
Strong PMO increase revenues TRUE

From a healthcare provider’s perspective the answers can be surprising; EHR can be well and good, or not well and not good.  The Alexandrian solution for EHR is a Alexandrian PMO.

Have your people call my people–we’ll do lunch.


EHR Milestones, should that read Millstones?

If you like adventure, here’s a site to check,  This is an annual event whose origin came about during the cold war.  Fortunately for both of us, the entry date has already passed.  The thought behind the JFK fifty-mile hike/run was that because of the possibility of a nuclear attack, each American should be in good enough shape to cover fifty miles in a day.

I participated in the event twice—I wrote participated because to state that I ran the entire way would be misleading— and I can state with certainty that almost no Americans are close to being able to complete this.  The event is run in the fall starting in Boonsboro, Maryland.  It takes place along the Appalachian Trail and the C&O Canal and various other cold, rain soaked, and ice and leaf covered treacherous terrains.

We ran it in our late teens or early twenties, the time in your life when you are indestructible and too dumb to know any better.  One of my most vivid memories of the event was that on the dozen or so miles along the mountain trail, leaves covered the ground.  By default that meant they also covered the rocks along the trail, thus hiding them.  That we were running at elevation—isn’t everyone since you can’t not run at at least some elevation, (that may be the worst sentence every written) but you know what I mean—meant the prior night’s rain resulted in the leaf covered rocks being sheathed in black ice.  That provided a nice diversion, making us look like cows on roller skates—roller blades had yet to catch on outside of California.

There were several places along the trail where the trail seemed to fork—I’m not going to say and I took it—and it wasn’t clearly marked.  Runners could easily take the wrong fork (or should that be Tine?).  I think it would have been helpful had the race organizers installed signs like, “If you are here, you are lost.”  Hold on to that thought, as we may need it later.

Some number of hours after we began we reached the C&O Canal, twenty-six miles of flat terrain along the foot path.  It’s difficult to know how well I was doing in the fifty-mile race, in part because I had never run this distance and because there we no obvious mile markers, at least so I thought.  Then we noticed that about every five and a half to six minutes we would pass a numbered white marbled marker that was embedded along the towpath.  Mile stones.  At the pace we were running, we anticipated we would finish high in the rankings.  As fast as we were running, we were constantly being passed, something that made no sense.  That meant that a number of people were running five minute miles, which we knew they couldn’t do after running through the mountains, or…Or what?

The only thing we knew with any certainty at the end of the day was that the markers with which we used to determine our pace and measure how far we’d run were not mile markers.  We never figured out why they were there or how far apart they were, but we greatly underestimated their distance and hence our progress.

It doesn’t really matter whether you call them mile stones or milestones.  What matters is whether they serve a valid purpose.  If they don’t, milestones become millstones.  Milestones are only useful if they are valid, and if they are met.  Otherwise, they are should’ a, could’ a, would’ a—failure markers, cairns of missed goals and deliverables.

How are your milestones?  Are they valid?  What makes them valid?  Are they yours, or the vendors’?  All things to think about as you move forward.