The making of a smackdown

The Updating of a smackdown

I thought it might be a good idea to share my experience on how you update a Smackdown or a bakeoff. There are several of these documents out there with matrixes that compare different products, features and vendors. They are needed for vendor tend to believe us they will deliver anything.

So How do you proceed when you would like to update a smackdown or a bakeoff?

  1. Make sure you have enough time planned or free time to spend, these thing take time…
  2. Get access to the original word document
  3. Gather experts around you, you don’t know everything
  4. Trust others
  5. Connect with the vendors up front and let them know what you’re up to
  6. Invite the vendors to help along with a final review
  7. Create a management spreadsheet
  8. Track who does what, status, emails you send out, emails you received, parts that are finished


My goal was to get two people assigned to each product so that one is the editor (checking and changing) and the other one is the reviewer that will go through the changes and see if all is correct. This way any errors or misses should be kept to a minimum, I of course let the guys decide how they work but asked them to work together and reach an agreement on who does what.. so that one check mark is only touched once.

Or course for some products it is harder to find people than for others, we internally have experience with some products but not with all. So I turned to the community to find help. Only a fool thinks he can do everything himself.  For all products that would be in the matrix two experts were found.


I would introduce you to the team now but I will do that in a second post to give them enough attention… this article is about the process behind it.

Share the document

Once you have a group of people you want to make sure they can start working, the long you wait the less enthusiastic they will become. So I looked out for a way to share documents, there were a few requirements;

  • We need to be able to work together in a document
  • We need to be able to give access and take access
  • We need to be able to work together at the same time (or at least look together)

There are some options in the field and not all of them are available to me, so I h ad to pick from the following options:

  • Google drive
  • One drive
  • Dropbox
  • Sharepoint (internal only)

So first I tried Google Drive but I noticed soon that a lot of people didn’t have google drive and so I stopped that one. also the performance wasn’t that great. Next up was OneDrive from Microsoft. I looked good at the start until a couple of people connected and OneDrive suggested that I was generating too much traffic. My account got blocked and that was Onedrive. Not meant for professional services it seems.

Let’s skip dropbox for a moment, sharepoint was an option. We use it internally all the time with document. As long as you don’t move while typing it works fine.. not my favourite for I had several corrupted documents but it would have been the best option. Only issue is that you can’t share with externals or at least we can’t.

So that left my to dropbox, a service I closed down a while ago… dropbox seems to be the easiest here, you can edit and other can watch. you see what other have done, you see if other accessed the folders. and you can give access or deny access easily. So I placed everything in dropbox and send out the invitation for that.

For an impression on the folder, all these documents will go into one master document eventually…



To make sure we all started of with the same information I send out an Email to all experts. In the way to long Email I explained what the planning was and how we would proceed. What kind of topics are in the “welcome” mail

  • What is the goal
  • What is de planning, when do you need to reviews in
  • Hoe should they review, what if you find errors?
  • What is you need to add something
  • What is you have any question

Let me highlight a few points here that are so important, if something is reviewed/edited you want to be able to check the change they made. Of course in Microsoft Word you have change mode but change mode itself doesn’t tell you why it was changed, it just shows the change.

What you need is a comment that points to the change and in the comment either a link or a description of the change. From time to time you have a feature that is not that clear, one will say its a checkmark where others say it’s not. If there is a link added you know you are talking about the same information. Who is right in the end??? of course the author of the document but it’s not that black and white, it’s discussion and together you try to find a good solution. The goal is to get a perfect-as-possible document not for you to win a fight.

What if new items are found? Of course that is the most common thing, new operating systems are added etc. So I mailed everyone that they are free to add lines in the tables to add new features. My task is to alert other to check the same new feature with their product. This was something that didn’t go as planned, lesson learned for next time.


After a while you will receive the first updates, if you have dropbox you’ll see docs getting updated regularly… which feels good. After receiving a message that a review is done your monk task starts. You need to check the reviews and see if you have any questions.

What I did was copy all edits that I agreed with to a master document so that all the work documents stayed original. I thought that would be easier for a later check. during other updates we used to work with a smaller group and worked in one document, this time I thought it would be good/nice to work with more people…. and it is, loved it.


As mentioned before contact with the vendors is key, I don’t want to put a document on the market before the vendors have also looked at it. So in an early stage I contacted all of them, by Email, twitter or face to face. I went to RES and VMware for a meeting, had a conference call with Appsense and many discussions with others. Why do I have meetings with vendors? Well as I’m working on a UEM document that will be read for a while (not updating it every month) its good to hear what is coming and whether we need to add that in the document. I have a rule that only GA products will be listed so we had a meeting with VMware about that. UEM 9 is due Q1, so we discussed if it would make it in the document. As I know it would not be possible to release before April we agreed to add it in the document… and what is a week or so, it would be bad to keep it out, it’s a major player these days. better be correct I think.

Good contact with vendors is important, all of them have been very helpful. They have their ideas and I have mine…somewhere in the middle we come together.

Something new we did this version is that we allow the vendor to write a description free of edit our side. In the document I will warn everyone that that chapter is a marketplace and everything you read there is marketing.

Accepting changes

One of the reviewers was using a Surface and opening the document with all changes was just too much for the surface. The Fan would go high pitch and the surface seemed unresponsive. After turning of “Show format” the surface returned to normality. Just want to refer to the fact that there are thousands of changes in the master document. I went over them today (matrix ones) and accepted once that should be accepted. It took me over an hours to accept them. Next I added the new descriptions of products in the document.

The whole document has to be reviewed (done twice already) and partly rewritten ( also done) and now it needs to be checked again before I accept the changes made. This is a two man job, my co0worker Hayscen will help me with this, full screen we will walk through sentences and see if we like them.

What else?

Don’t underestimate the job you have, I need to work also and do this in the evenings and weekend. somehow I was lucky that I got the flu, after a 4 days of sleep I was able to work for several minutes at one time. I spend every minute that my headache and body allowed me on the document. How much time is required and where is time spend?

Time is spend in organising and managing the project. people are editing and you need to make sure you know who is doing what, when etc. That sounds like easy peasy but all communication at the start was more work than expected. Also communication with vendors takes time, fun time as with the experts, but time it takes. Conference calls, meetings, email conversations if yu don’t have time planned for it, you are spending your own time.

Is this negative? hell no, I love it!!!!! just make sure you know that you will need the time and tell your wife 🙂


I hope you got a little insight view on the making of a smackdown, in the next article  (when we release) I will introduce the experts that worked on the document. They deserve a wall of fame!



Leave a Reply