Tuesday, April 15, 2014

A few decades' progress in computers and electronics

I got my bachelors degree in EE in 1981 and went to work doing board-level design. Most circuit assembly was wire-wrap back then, and we had TTL and CMOS and 8-bit microcontrollers. Most of the part numbers I remember from my early career are still available at places like Digikey. The job of programming the microcontroller or microprocessor on a board frequently fell to the hardware designer. In some ways it was a wonderful time to be an engineer. Systems were trivially simple by modern standards, but they were mysterious to most people so you felt like a genius to be involved with them at all.

After about 15 years of hardware engineering with bits of software development on an as-needed basis, I moved into software engineering. The change was challenging but I intuited that the years to come would see huge innovation in software engineering, and I wanted to be there to see it.

Around that time, the web was a pretty recent phenomenon. People were learning to write static HTML pages and CGI scripts in Perl. One of my big hobby projects around that time was a Java applet. Some people talked about CGI scripts that talked to databases. When you wanted to search the web, you used Alta Vista. At one point I purchased a thick book of the websites known to exist at the time, I kid you not. Since many websites were run by individuals as hobbies, the typical lifespan of any given website was short.

Software development in the 80s and 90s was pretty hit-or-miss. Development schedules were almost completely unpredictable. Bugs were hard to diagnose. The worst bugs were the intermittent ones, things that usually worked but still failed often enough to be unacceptable. Reproducing bugs was tedious, and more than once I remember setting up logging systems to collect data about a failure that occurred during an overnight run. Some of the most annoying bugs involved race conditions and other concurrency issues.

Things are very different now. I've been fortunate to see an insane amount of improvement. These improvements are not accidents or mysteries. They are the results of hard work by thousands of engineers over many years. With several years of hindsight, I can detail with some confidence what we're doing right today that we did wrong in the past.

One simple thing is that we have an enormous body of open source code to draw upon, kernels and web servers and compilers and languages and applications for all kinds of tasks. These can be studied by students everywhere, and anybody can review and improve the code, and with rare exceptions they can be freely used by businesses. Vast new areas of territory open up every few years and are turned to profitable development.

In terms of concurrent programming, we've accumulated a huge amount of wisdom and experience. We know now what patterns work and what patterns fail, and when I forget, I can do a search on Stackoverflow or Google to remind myself. And we now embed that experience into the design of our languages, for instance, message queues as inter-thread communication in JavaScript.

Testing and QA is an area of huge progress over the last 20 years. Ad hoc random manual tests, usually written as an afterthought by the developer, were the norm when I began my career, and many managers frowned upon "excessive" testing that we would now consider barely adequate. Now we have solid widespread expertise about how to write and manage bug reports and organize workflows to resolve them. We have test-driven development and test automation and unit testing and continuous integration. If I check in bad code today, I break the build, suffer a bit of public humiliation, and fix it quickly so my co-workers can get on with their work.

I miss the simplicity of the past, and the feeling of membership in a priesthood, but it's still better to work in a field that can have a real positive impact on human life. In today's work environment that impact is enormously more feasible.

No comments: