News and Events

News and Events: Interview Series - Issue 3

Keith Kidd is a Founding Member of NTAF and served as the President of the Board from October 2009 until    May 2011.  In December 2011, following the retirement of the most recent Board President, Keith was appointed to serve once again as a Director and was elected as President of the Board.

In his day job, Keith presently serves as Director of Packet Systems Testing within Verizon's Technology Organization. Throughout his career, Keith has managed IT Operations, System Development, Architecture teams, and various test organizations. At the time of NTAF's inception, Keith was the Director of Verizon's Business Tools team with responsibility for the Lab's Automation Strategy.

You were one of the first, maybe even the very first, instigator of the effort that became the Network Test Automation Forum. What was the original motivation behind that effort?

Verizon Labs has invested heavily in automation and depends on automation to help achieve testing objectives. Over time, we have come to realize that automation has become a significant delivery in scheduling testing for new services. When the test equipment vendors first called on the Labs, we were happy to plug in their solutions and check the automated box off on our To Do list. Over time, we have figured out how to dynamically connect components and share test devices as well as run scripts to execute what was once manual. When one considers the time to automate a task and then adds the novelty of new technology solutions coupled with the required new release of a test head products to accomplish what we now call automation, automating the test bed is a pretty daunting deliverable in itself. As Business Process Engineering practitioners, we constantly evaluate our methodologies to find faster, cheaper, more expedient ways of accomplishing the task at hand and that includes how we automate. I was agonizing with yet another change when I was invited to become a Customer Advisory Board Member of another organization that had been set up in the industry. I had concluded that waiting for others to recognize and fix this challenge was not going to get the job done and I decided that getting involved was the only way to make a difference.

While this organization did listen to its advisory board members and even changed its charter as a result of our input, I learned that there were other barriers that would prevent all those who needed to be a
part of this endeavor from joining. Within a few months I came to realize that more would need to be done.

You've been involved in creating standards before. Can you tell us a little about that previous experience and how your experience with the formation and running of NTAF compares?

I was involved with the DSL Forum which later became the Broadband Forum for quite some time and made contributions to standards there. In addition, the Technology Organization within Verizon of which I am a
member sits on many boards and is heavily involved in the creation and support of a number of standards that we believe improve the Telecommunications Industry. Through the years, I have witnessed improvements in development time, process improvement and most importantly, immense customer benefit from standards work. When we set out to create NTAF, the founders leveraged this knowledge to create a model that would be attractive for its members and would be as effective as possible in creating open standards by which we would improve network test automation. Some of my own "lessons learned" that guided me in my
vision for what NTAF needed to be are:

* A truly democratic organization where each member has identical powers of persuasion; our technical and marketing committees are staffed with volunteers from member companies and we encourage all
members to participate.

* Equal power from the smallest to the largest (one vote per member)

* Position the work to move quickly; keep the submissions small and manageable so that the contributions can move through the process quickly

* Anyone can have a good idea, but when you share with the likes of the NTAF membership, it matriculates into a phenomenal idea

Clearly, I am extremely proud of the work NTAF has accomplished in a short time. Had one or two companies tried to accomplish what we have done in one year, I do not think it could have been successful.


Can you talk about what it took to get the competitive vendor members to cooperate within the realms of NTAF?

Just as a forum benefits from its membership, the development of the organization itself was the brainchild of a group of industry visionaries. We started by getting a select group of competitors together in a room in February, 2009 and talking about the state of the network testing business and what we thought could be done to improve the business for everyone involved and of course, would they agree to work together on any level. We left at the end of the day with the common belief that something should be done and that we would continue
to work on a plan for what we should do to improve test automation. l next set up weekly calls that continued for months and culminated with the founding members getting together in Chicago in October, 2009 and
agreeing to form what became NTAF. I really cannot say how many hours went into this, but want to credit this group of visionaries for what NTAF has become.


Early on, someone recommended hiring a professional management firm to help us with the details and to keep us moving forward; as it turns out, this was likely the best advice I ever received. I have to admit that
even I was not mentally prepared for what would be required to legally launch a non-profit, but AMS, the professional management firm we hired, guided us through the process. The founding members and AMS spent
another 5 months of weekly meetings to review and amend the documents by which we govern.


What do you see as the benefits NTAF brings, or potentially brings to Verizon's test environment?

NTAF's standards help Verizon focus our resources on the critical task at hand; validating new product and services that meet Verizon's exacting standards of quality. I want every minute of test time to be
about validating the quality of new components, not figuring out what changes were implemented in tools for testing. Once the standards begin showing up in test tools, Verizon will be able to leverage the first of the standards to aid in discovery and setup of the test environment. Obviously, the less time we spend figuring out what the tools are and where they are, the faster we can get on with testing.


Earlier we talked about the value of the organization feeding new ideas to the forum. In late 2010, the Technical Committee put together a "road map" for 2011, which I think is a shining example of the power in
working together. One idea that came from a brain storming session was inventory control tied to the discovery process. The Technical Committee decided earlier this year when it would work on inventory discovery and is diligently deliberating the issue of inventory and how we leverage the already developed discovery mechanism to help us manage dynamic and static labs more efficiently. I believe any laboratory that has invested in test equipment will be able to translate this addition into improving their return on investment.



How does Verizon plan to implement NTAF components into their test lab environment?


This is another example of one taking advantage of being a forum member! I encouraged Brian Bonnett, our Technical Committee member to be an active participant in the Technical Committee. Truthfully, Brian really
didn't need much encouragement as he was an active participant from the get go. When the TC made the call for prototypes to be used in developing our initial standards work, Brian showed up with a VoIP phone app that followed the model. In conjunction with others who also developed prototypes, we were able to move along with the evolution of the model.


As a result of Brian's ingenuity, he was able to begin the design updates to our own internal tool and is ready for the standards when they start showing up in our NTAF member's tools later this year. Now, I have to acknowledge this as a starting place, we have other areas where third party test automation tools are leveraged to communicate with the test heads we use as well as our on Tcl libraries. For example, we have plans to implement the interface in our Tcl libraries at the point that suppliers offer the interface for the tools we have within the individual labs. Verizon has regular discussions at the Executive Level with our suppliers and all have been asked for the NTAF compliance road map regardless of whether they are NTAF members. Since NTAF has provisions to share our standards with anyone in the industry, Verizon has put the community on notice that we expect to enjoy the benefits of the NTAF standards later this year. As the standards emerge in products, the presence of the standards will become differentiators in Verizon's purchasing decisions with new tools.



Some might see NTAF about getting vendors to comply with a standard, but we saw in the plugfest last year that Verizon implemented the NTAF specification on some of its "home-grown" tools. Why?

Verizon's Business Tools Team is ultimately a supplier of automation tools to the organization and as such is under the same constraints as any third party tool provider we use. Leveraging the NTAF standards is
in my opinion, practicing what we preach; helps us leverage change faster. I cannot talk about specific tools, but I can tell you that I started reaping benefits from our work with NTAF very early on. The standards NTAF have developed have helped us find less expensive and faster alternatives for automation already. In fact, you could say that I have already realized savings that will pay Verizon's dues for several years to come!



How do you address the potential perception that NTAF is a force that will lead to commodification of tools and will reduce differentiation?


In my humble opinion, NTAF frees up valuable resources that should help these competitors differentiate their products. I'm not aware of any suppliers Verizon is using that are making money from developing the
interfaces we mandate to work with other tools in the portfolio. I personally believe that most suppliers end up spending a lot more precious development resources on making their customers happy than they
would desire. I don't mean to imply they begrudge the work, they certain do not. No tool supplier that has come into my office has ever tried to tell me that the benefit they bring to the table is that the work with another tool. In fact, I often tell perspective members that what we are achieving in NTAF frees them from the drudgery so that they can focus on developing new technologies. Would you rather spend 6 months of precious development time making your tool work with Keith's test solution or leverage that valuable resource to implement 6 months
of new features on your tool that separates you from your competitors?



Do you envision Verizon adding the requirement that test lab purchases include products based on the NTAF spec? If so, when?

Now! In fact, if a supplier has met with Chris Mayer, the VP of the Lab's within the last 6 months, he has asked them to show the roadmap for NTAF compliance. As we meet with those potential suppliers we are also making it clear that we intend to make NTAF compliance a differentiator.


Say I'm a tool vendor that's still on the fence about joining NTAF or implementing the spec. What would be in it for me?

Immediately and directly, you are going to be able to reduce your development costs significantly. There are at least 4 current members of NTAF that have added requirements for interface work that vendors must follow today. I know there are others, but cannot say how many there are who have a similar philosophy. This amounts to a lot of custom development work these vendors are doing today to accomplish a common set of tasks, all of which goes away with implementing the interoperability standards we are working on. Ultimately, joining NTAF
gives a tool vendor the ability to influence those emerging standards so that their interests are represented in the standard.


Similarly, what would be the incentive for companies which are focused on building test labs and using test tools, to join NTAF?

Having served as President and Chair of the Technical Committee in year one; I can attest to how much input Verizon has had on the standards. Sure, our suppliers have always listened to what we have to say, but
the dialogue we have had in the Technical Committee sessions is early in everyone's life cycle where I believe it to be most beneficial. At the same time I am absolutely awestruck at the talent our members have put
at NTAF's disposal. The value of those divergent opinions strengthen our submissions into something no individual member firm would have accomplished on their own. I now know from experience that the knowledge one gains from participating also impact the approach and solutions one applies to other business problems; in many ways it is a real time advanced degree for those who are part of the experience.
This experience has demonstrated that the labs and the test equipment suppliers are truly benefiting from the idea exchanges from one another.



What would you like the future of NTAF to include?

We really all have the same mission; helping our businesses get new products and services in the hands of those who need them faster. Two years ago, when we started NTAF, we were solely focused on a narrow
definition of interoperability. Our definition of interoperability expanded with each weekly call and led to a road map for 2011 that is still evolving. My fondest wish for NTAF would be to continue that
evolutionary path forward for automation.