Spy Chips – My question to John Chambers in 2010

There is continuing controversy as to whether the claims that certain motherboards manufactured in China were fitted with tiny microchips that are capable of compromising data. Details as to what exactly this means is less important than the answers to “what if” questions. The chip could potentially install malware or open up a port silently for the hackers to invade any of the systems using such a motherboard. It could also potentially inflict other major damages such as erasing all data or corrupt the data slowly (in some intelligent ways) so that even backups over a period of time makes it impossible to retrieve the data. And it may be programmed to be dormant and wake up in some future date. Who knows?

Planting such “easter eggs” is a common practice in software, but not necessarily with the intent of causing damage, but more as a joke or for fun. Programmers of video games or computer software planted these for fun. For example, TECO editor from TOPS-10 operating system (in the late 70s and early 80’s) used the command make to create a file. If you typed “make love” to create a file called love, it had an easter egg which would respond by saying “not war?“. Most recently, Google engineers have been creating a fair amount of easter eggs.

I want to tell you a story about my question on this subject some 8 years ago…

Cisco, a company that was started by a few folks from Stanford in the mid-80’s dominated the network hardware market early on and most of us in Higher Ed have continued this relationship, though, over a period of time, many others have emerged in the market.

My association with Cisco goes back to 1987. I got my first job at Wesleyan and moved there in 1986 and soon after, Professor David Beveridge (my PhD thesis mentor with whom I had moved there) received an NSF Grant to have Wesleyan join NSFNet (precursor to the current Internet). This was a way for his research group to access national supercomputer centers to be able to conduct their research on molecular simulations.

David Todd, the head of IT department at Wesleyan at that time, had just returned from a sabbatical at Stanford University and knew Sandy Lerner and Len Bosack who began Cisco with a few others (Apparently, Len stole some of the software from Stanford that was used to drive the early routers of Cisco, something you can read about here).  So, Wesleyan was one of the very first customers of Cisco when we bought a terminal server to connect computers in Beveridge lab and a few others and a router to connect to NSFNet. When we first connected NSFNet, it felt like we were landing in Moon!

Later, when I was working at Pace University, I was invited to attend a meeting with John Chambers, then the CEO of Cisco Systems. I was naturally very excited and accepted the invitation. Only 25 or so CIOs of Higher Eds were invited and the meeting followed by dinner was held in Le Cirque in Manhattan. It was a rainy day and I arrived a few minutes early for the meeting and as I entered the restaurant, the staff there gave me a strange look!

I was pretty upset because I felt that they thought I was in the wrong place and was not worthy of such expensive restaurants! Well, it turns out that there is a dress code for Le Cirque… and I was not wearing a suit. I cursed myself for ignoring that part when one of them offered me a jacket that they had for people like me 🙂

We were all chatting away when John entered the room. I was about introduce myself and John said “I know all about you” and greeted me simply as Ravi and went on to express his appreciation for my support from my days at Wesleyan. I was floored. And the man did this to every one of the 25 or so people! No paper in hand or looking up details on the phone, just from memory…

Then came the question time and I asked my question, which basically went like this “In this globalized economy, where your hardware is being manufactured all around the world and software being written in places like India, how do you manage quality control? How do you know that malicious hardware or software are not being installed in your systems that may trigger at some point in the future, crippling the internet?”

John was very articulate in his answer, but, ultimately, the answer was something like “We have our methods to make sure this doesn’t happen. I just can’t tell you the details.” I took it to mean “There is absolutely no way for us to make sure that this doesn’t happen”.

And I believe this to be true even today. Unfortunately, the “spy chips” we are talking about are not physical chips that one can see. They are likely to be microscopic and given that millions of these are produced every few days, it is impossible for proactively examining them. The same is true for software. They are complex pieces of code, millions of lines in some cases and no matter how much testing is done, hidden parts of software may not come into play with standard tests.

So, trusting the partners is about the only solution to this unless artificial intelligence can be put to use for discovering such things that cannot easily be discovered through other means.

I want to leave you with another related story. I had a Wesleyan student work for me as a programmer and I ended up hiring him as a staff member after graduation. He was a brilliant coder and  I used to say that his code looked and read beautifully, like a poem. He moved on to graduate school.

Several months later, we were getting a lot of emails from the users about some random quotes appearing in emails being sent out from our systems. We scrambled to find where this was coming from and it turned out to be from a piece of code that this person had written. He had planted an easter egg to send out these quotes (Thank God that they were nice ones!) on April Fool’s day!

Of course, I reached out to him to ask what else he planted… He laughed and said he couldn’t remember, but was pretty sure that this was about the only one. I trusted him and as far as I know, nothing else has popped up to my knowledge.

Leave a Reply