On Identity POCs - From a Vendor's Perspective

I pinged Joe (Nobody?) on Twitter last week regarding Identity Management POCs. Joe put up a lengthy post on his blog regarding some of his thoughts from the perspective of the vendor (so it seems). It's always great to get thoughts on the topic from another vantage point...some great points, with my $.02 in-line:

"a POC is that they are a dangerous sales activity used against a vendor rather than for it (I used to be a customer and did just that)"

I've witnessed that before. So Joe, how do we make sure the POC stays on the right track?

"But a POC is should not be a repeat of a demo in a customer's environment. On the flip side, a POC should not be an installation exercise based on the customer's demands.
A POC should be a onsite installation to show at a minimum, key use cases for the defined phase 1 and 2. Self service, HR feeds, provisioning into the key systems and de-provisioning for exmaple. Which means Phase 1 and 2 should be defined prior. How do you know what to show if the customer doesn't know where they are going?"

Use cases. I like. Tell me more...

"Prove the concept. Prove the process. Prove the business improvements and solving of business needs rather than proving when you hit this button this technical thing happens."

Completely with you.

"Don't prove installation, don't prove configuration, don't prove how many components it takes to do it. "

Hmm....not so sure about this one. A POC should prove business use cases as well as allow the technical team understand how it works in order to judge integration efforts and supportability...no?

"Another reason why POCs are often an embarrassing cluster is the customer's environment. I generally require, based on the customer's hardware, that I have sterile servers, patched specifically, nothing else on them and require the pre-req software installed on them before I walk in the door...What GPOs are set that is locking down a service and takes you 2 days to find it. Regardless on the cause, any delay is bad impression on you and the product."

Fantastic point. I've seen POCs blow up because of a misconfigured DC, DNS problems, etc. And the vendors end up spending time troubleshooting environment problems rather than working on the actual POC.

"If, as a vendor, you drive the use case creation with the customer you will show your knowledge and leadership. You will have a controlled flow from start to finish they will make you look successful and show the customer their needs. Your time will be shorter and cost less for you. The success rate will be higher. You miss these things, the customer will push you into a hole of broken knowledge. We are the experts, not them."

Well said, Joe. Nobody.

How to Make a Better Identity Management POC

Mike Trachta responded to my previous post about lackluster POCs by highlighting the difficulty posed to SIs to live up to the fantastic show presented by the POC folks.

"...the customer remembers more about the POC than you think. They remember how pretty the screens were. They remember how seamlessly all the pieces fit together, and how quickly each task executes. What they don’t realize is that all of the data is massaged and simplified. Of course the POC could do these things! It has 3 users with 4 roles to choose from, and doesn’t include any of the “exceptions” often found in the customer’s environment. When it comes time to actually implement this feature with 30,000 users things can (and do) get quite a bit more complex."

Jeff Bohren put together an eloquent post as well, pointing to his experience as the developer in the background helping out both the POC folk, as well as the SI who has to make this happen for real. It's a great read and provides insight from both ends of the spectrum.

I have a few suggestions that might make your IdM POCs a little better. So here they are:

1. Couch your immediate goals in the context of a larger Identity Management initiative that ties it to your business objectives. Jeff already hit on this point in his post, "Instead of doing a POC of who has 'The Most Magical Bullet', enterprise would be better suited to craft a long term IdM strategy and chose a vendor whose product best aligns with it." This approach voids the notion that phase 1 of the project has to cover everything under the sun. A fantastic way to do this is to engage an SI that understands the game, and can walk you through an Identity Management workshop that speaks to both your business and technical objectives.

2. From your workshop findings, carve out a Phase 1 of the project that is attainable - the best way to do that is to write up a handful of detailed use cases that boils the expected deliverables down to non-technical language.

3. Highlight the top X number of use cases to focus on for a POC. Keep it limited (but representative) of your expected project deliverables. Identify the vendors who might be able to respond, and request a technical architecture document outlining their approach to solving the use cases you have identified and have a Q&A session with them. Filter out the vendors who don't make the cut (yes, I know that's a loaded sentence), and identify the top 1 or 2.

4. Prepare a POC lab. (Another loaded sentence).

5. Bring the vendors in, but don't allow them to touch anything! OK, this suggestion point might be a bit much but I suggest that, as much as possible, have the vendor's experts sit on their hands, next to your techies while your techies drive. If the vendor whips out a canned script, your guys will know it (and document it in the findings). If the vendor has to make a nasty directory schema change, your techies will know it. If the vendor has an ugly hack that inserts pages of code into the presentation layer of the "ultra configurable identity app", your techies will know it.

6. Have a structured way to present the findings.


While I know that the approach above requires a lot of background knowledge and support, it just seems to be a much more valuable experience that actually tells you something that a demo can't. If help is needed, supplement the team with an SI that has strong experience in the space. Either that, or save everyone time and effort and just make your decision based on sales demos and references.