Finally, the thing is printing

Tomorrow I start a new job with Formlabs, a 3D printer company, and I'm psyched about that. Unfortunately, however, one of the conditions of my employment there is that I cease development on my own 3D printer. I spoke with their attorney and it all makes sense, it's the right thing to do, because my printer is entirely open source and they are selling a proprietary product. If I were to continue developing my printer, it would be too easy to unintentionally include pieces of their technology. So tonight I am putting the finishing touches on the Github repository.

Today was the first time I made a successful print with this printer design. I had hoped to print four dodecahedra at once. But I had some crud on the surface of my build plate, and I hadn't stirred the resin before printing, so only one of the four came out really well. Another was misshapen, and two of them never came together at all.

 Here are the two that were at least coherent solids. I think with a little more learning and practice, I'll get to where I can make four that all look as good as the one on the right.

Here is the setup I'm using. If you've followed this blog, you'll recognize the stuff on the bucket. The box-like thing overhead was quickly cobbled together when I realized that my mirror wasn't reflecting enough UV light to make the resin cure properly, because the mirror's glass isn't transparent enough in the UV range. To remove the mirror from the optical path, I needed to put the projector directly over the bucket, pointing down.

NodeJS and JSHint on Fedora

Yesterday I blogged that it's a hassle to install these on Fedora. Apparently I was suffering from brain fog. It's not so bad once you do enough research and stumble across the right advice online.

First, if you're a bonehead and you've made a mess trying unsuccessfully to install v8/nodejs/npm/jshint eight or ten times already, clean things up:

sudo yum -y remove v8

The repository for picking up nodejs, npm, and v8 is http://nodejs.tchol.org/ which you can enable on your system as follows:

sudo yum localinstall --nogpgcheck \
  http://nodejs.tchol.org/repocfg/fedora/nodejs-stable-release.noarch.rpm

You want to avoid installing the wrong version of the V8 Javascript engine, so edit /etc/yum.repos.d/fedora-updates.repo and add the line

exclude=v8*

to the "[updates]" section.

Now you're ready to install everything:

sudo yum install npm

sudo npm install jshint -g

Of microcontrollers and operating systems

In recent weeks I've put some effort into working with a Beagleboard running Angstom Linux. The Beagleboard has an OMAP3530 processor, a ridiculously over-powered thing. It's cool to be running Linux on something you can attack with a soldering iron. But as I looked at my intended application and the price of the Beagleboard, and the hoops they jump through to manufacture the Beagleboard, I started to wonder if more conventional weapons might be sufficient to win the day.

I'd blogged in the past about the AT91SAM7, another family of ARM-based chips that are a little less over-powered, so I wondered, would they work for this? My first thought was to use Angstrom Linux on the SAM7. I found that nobody had done it, and digging deeper to find out why they hadn't, I was reminded that Linux requires a MMU and the SAM7 doesn't have one. Neither of these was a surprising piece of information, and I was probably dimly aware of both, but had never consciously connected them.

The reason Linux needs an MMU is because it runs multiple processes in separate memory spaces, so that one process can't crash another by overwriting its memory. This requires remapping from virtual memory addresses to physical addresses. That's most of what an MMU does.

It's shameful to admit, but I had unthinkingly assumed that 32-bit processors would necessarily run something like Linux, merely by virtue of being 32-bit processors. This was the result of having grown up with 8-bit processors and thinking of 32-bit processors as "big" and "complicated" and a little "scary". They are in fact all those things, but we still need to keep our wits in their presence.

Casting about for an operating system that might be more SAM7-friendly, I came across FreeRTOS. I started puttering around with a FreeRTOS port for the SAM7, and after banging on that a while it crossed my mind to think that there might be some other ARM7 chip for which a FreeRTOS port already existed so I wouldn't have to do the port myself. A little investigation in this direction led me to the LPC1768 (overview, datasheet, user's manual, Digikey listing) an inexpensive ARM7 chip with lots of flash and RAM, an Ethernet controller, USB controllers for both host mode and device mode, buckets and buckets of GPIO pins, and a comfortably higher number of MIPS than the SAM7 family. The LPC1768 has an ARM Cortex M3 core (overview, user's guide).

So what are our hardware development options here? Sparkfun provides a nice little board for only $50. It has tons of pins, sanely spaced at 0.1", and a JTAG connector on one end and a mini-USB on the other. It does require a power supply but that's not unreasonable. It has two pushbuttons (one a reset) and an LED. While I heartily encourage anybody to buy this board, I ended up buying a different board (which, time will tell, I may regret) because it was available on eBay.

I'm hoping to see the board in about a week, and I'll try to get FreeRTOS running on it with reasonable haste. Hopefully it will all work out nicely and I'll get to do a lot of blogging about it. The LPC1768 is really an interesting chip with a lot of on-chip peripherals, and I'd expect that would be a good amount of fun.

Beagleboard, OMAP, and Angstrom

I've been doing a lot lately at work with the BeagleBoard, shown at right. It uses an OMAP processor from Texas Instruments. The OMAP family is ARM-based and includes a DSP core, along with an intimidatingly rich set of on-chip peripherals. The OMAP is built with a package-on-package arrangement so that the RAM die sits right on top of the processor die.

The board typically costs about $150 in the United States. You'll need a few things to work with it: an SD card, a USB adapter to write the SD card, a USB-to-serial adapter to communicate with the board, a 5-volt power supply, and later (maybe sooner) you'll want a USB hub with a RJ-45 ethernet jack.

A minimal setup is shown at right. This is just enough to connect to the board over a serial port (115.2 kbaud, 8N1, no flow control) and verify that you get a working Linux shell. The Angstrom Linux distribution has a bit of a learning curve but it seems well thought out.

I'm thinking of trying Angstrom on one of the AT91SAM7S boards from Sparkfun when I get a little spare time. I think that would work, and it would really rock to see full-blown Linux running on a $36 board. I don't know how I'd handle networking in that kind of situation, though.

Update: I am reminded that the SAM7S lacks an MMU so it can't run Angstrom. There is a different Linux distribution called uCLinux (see uclinux.org) that would work, maybe I'll try that some day.

March 2011 trip to Shanghai, Suzhou

Last week I was in Suzhou, a city a little west of Shanghai, and took some photos. Very interesting place with a lot of rather ancient history. I liked very much the Humble Administrator's Garden. It's quite large, with several small buildings and waterways and paths, and very pretty as you can see here.

Suzhou is a very pleasant place. I felt quite safe walking around after dark. There are lots of little outdoor markets. I stopped at one to get some squid on a stick, which was spicy and tasty. My photo of the squid-on-a-stick guy is unfortunately a little blurry.

I'm still kind of tired with jet lag. When my energy level is a little higher I will add more stuff to this. Generally it left me with a very positive impression of mainland China, which was a surprise as I'd been told to expect it to be a bit backward culturally. We were there for electronics manufacturing and there was certainly plenty of that, and plenty of heavy industry in the Shanghai area. Lots of construction, lots of big cranes all over the place.

This Google-Verizon deal and Net Neutrality and all that...

Like everybody else, I'm disappointed with Google on this one. The stuff about the wired Internet is good, it's actually a stronger stance on net neutrality than has existed to date. But the wireless Internet is now supposed to be the Wild West of high tech, a lawless place where anybody big enough can do anything they want. Google should know better. But Google is not the important party in all this.

My feelings about Verizon are very different. Verizon paid for the network (having purchased it from its builders and/or previous owners) and now pays to maintain it. When the network in my neighborhood goes down, the trucks that come to fix it are Verizon trucks. It's fair and reasonable for Verizon to decide which packets its network will carry, and how those packets will be prioritized.

What would not be fair or reasonable would be to allow Verizon to block other efforts to build traffic-bearing networks.

I would love to see a parallel Internet built by hobbyists and local communities and small businesses. A few years back there was a wonderful book called Building Wireless Community Networks by Rob Flickinger. It seemed to me that Flickinger envisioned a nation-wide and perhaps world-wide community network. Maybe I was projecting my own hopes, but I like to think he might have shared that sentiment.

The right response to the Google-Verizon deal is not to complain about Google's duplicity. They are a publicly traded company, with all that entails. The right response is to start building a network that isn't supported by already-large corporations, where individuals and small new companies don't need to worry about policy decisions by the Googles and Verizons of the world.

Maybe this should replace Amateur Radio, which has been in decline since the Internet came along.

Website design, lessons learned, part 1

I recently got a fortune cookie that said something along these lines:

Skillful actions come from experience. Experience comes from unskillful actions.

Maybe I can share some experience and save somebody a little grief. I anticipate I'll post more things along these lines so I'm calling this "part 1".

I've spent some months building a Django website. The website has been growing more complex and the requirement has been a moving target, so I have been developing practices accordingly.

The first thing is automated testing. We all know it's good, but we don't always remember just how good. Plan your test strategy early.

Django's templating system works well with nested Python dictionaries, so a data structure like

  {"foo":

     {"bar":

         {"baz": "some content here"},

       ... }

   ... }

can be included in an HTML page with a notation like this.

  Let's put {{ foo.bar.baz }} in our web page.

Python dictionaries are essentially the same as JSON data structures. So all my view functions produce nested Python dictionaries, which can either be plugged into HTML templates, or returned as JSON if "json=1" is present in the HTTP request parameters. In the short term, the JSON output makes it very easy to write automated tests that don't have to scrape HTML to find the content. In the longer term, I'll want JSON when I move to AJAX some day.

The second topic is URL design. I've discovered a new way to write spaghetti code -- I've produced a profusion of URLs as I've grown the functionality rapidly. Each URL ("/foo/", "/bar/", "/profile/", etc) has an entry in urls.py and a function in views.py. If you're careless about planning, these tend to sprawl.

I think the right thing is to draw something like a state machine diagram for your website. The nodes are the URLs, each mapping to a HTML page. The edges are the actions users take to go from page to page, clicking buttons or controls or submitting HTML forms. Somewhere in there you need notations for the stuff happening in the back end, things being fetched from the DB or stored, various computations being done, various complex data structures being constructed. My thoughts on how to construct a proper state diagram are not yet complete.

Learning to live with software specifications

We software developers have a knee-jerk hatred of specifications. Rather than write a document describing work we plan to do, we would rather throw together a quick prototype and grow it into the final system. We sometimes feel like specs are for liberal-arts sissies and pointy-haired bosses. Our prehistoric brains want us to dismiss specifications as a waste of time or even an intentional misdirection of energy.

The truth of it is that specs build consensus between developers, testers, tech writers, managers, and customers. They make sure everybody agrees about what to build, how to test it, how to write a user manual for it, and what the priorities are.

The Agile guys talk about the exponentially increasing cost of fixing a bug. The later in the process you find that bug, the more troublesome and expensive it is to fix it. Fixing bugs in code is hard, even prototype code, and fixing text is easy.

Let's learn to trick our brains to work around our reluctance. The Head-First books always start with a great little explanation about how our prehistoric brain circuitry divvies up our attention, classifying things as interesting or boring, and determines what sticks in our memories. Sesame Street learned how to make stuff sticky by

• repetition
  
• lighting up more brain circuitry
  
• infusing the topic with emotional content
  
• relating it to things that were already sticky

One way to infuse your spec with emotional content would be to make it a turf war. That hooks into all our brain circuitry for tribes and feuds. But turf wars are traumatic and damaging to people and projects, so let's not do this.

To light up more brain circuitry, sketch out pieces of the spec on a big whiteboard. Draw a lot of pictures and diagrams. Use different colored markers. Get a few people together and generate consensus (not a turf war), and ask them to help identify issues that you forgot. That meeting is called a design review, like a code review for specs.

Who should write and own the spec?  Part three of Joel Spolsky's great four-part (1, 2, 3, 4) article answers this question, drawing on his experience at Microsoft. One person should write and own the spec, and the programmers should not report to that person. At Microsoft, that person is a program manager.

It's important to differentiate between

• a functional spec (what the user sees and experiences, what the customer wants) dealing with features, screens, dialog boxes, UI and UX, work flow
• and a technical spec (the stuff under the hood) dealing with system components, data structures and algorithms, communication protocols, database schemas, tools, languages, test methodologies, and external dependencies which may have hard-to-predict schedule impacts

Write the functional spec first, then the technical spec, then the code. If you love test-driven development then write the specs, then the tests, then the code.

Joel's article includes some great points on keeping the spec readable.

• Use humor. It helps people stay awake.
  
• Write simply, clearly, and briefly. Don't pontificate.
  
• Re-read your own spec, many times. Eat your own literary dogfood. If you can't stay awake, nobody else will either.
  
• Avoid working to a template unless politically necessary.

How do you know when the spec is done?

• The functional spec is done when the system can be designed, built, tested, and deployed without asking more questions about the user interface or user experience.
  
• The technical spec is done when each component of the system can be designed, built, tested, and deployed without asking more questions about the rest of the system.

This doesn't mean that these documents can never be updated or renegotiated. But the goal is to aim for as little subsequent change as possible.

I am still sorely tempted by the idea of a quick prototype, an "executable spec" that exposes bugs in design or logical consistency. Maybe it's OK to co-develop this with the spec, or tinker with it on one's own time, or consider it as a first phase of the coding. I'm still sorting this out. The basic rationale of a spec, that fixing bugs in text is easier and cheaper than fixing bugs in code, still needs to be observed.

All web app frameworks lead to Rome

Earlier I blogged about how it seemed like web app development had just zoomed past me. Since then, I've buckled down and actually started to study this stuff. My earlier posting only talked about the presentation layer, HTML, Javascript, and CSS. I still have more to learn about those, but the really interesting stuff happens on the server.

In December I went to a two-day session on Hibernate and Spring, and it was full of mysterious jargon that made me sleepy: dependency injection, inversion of control, aspects, object-relational mapping, convention over configuration, blah blah blah. I kept at it, though, looking at Rails and later Django. I'm now waist-deep in building a MySQL-backed Django site. What I learned is that (A) all these web app frameworks are remarkably similar to one another, and (B) those jargon terms are a lot simpler than they seem.

Inversion of control means that the framework makes calls into your app code, rather than you calling the framework from a main() function. Dependency injection is a set of tricks to minimize dependencies between different Java source files. Aspects are Java tricks that you can do by wrapping your methods in other methods with the same signatures, a lot like decorators in Python. Object-relational mapping is creating classes to represent your DB tables: each instance represents a row, each column is represented by a setter and getter. The MVC pattern gives the lay of the land for all these frameworks, and all the presentation stuff I talked about before is limited to the "view" piece.

As I find my footing in the basics, I start to notice where the interesting bits of more advanced topics pop up. If I put a Django app and a Mediawiki on the same server, can I do a single sign-on for both of them? I think I can, by writing an AuthPlugin extension to make the Mediawiki accept Django's authentication cookie.

Don't ask Django to serve a PHP page because it doesn't include a PHP interpreter (what mod_php does for Apache). Your Apache config file must deal with PHP files before routing to Django.

    AliasMatch /([^/]*\.php) ..../phpdir/$1
    WSGIScriptAlias / ..../djangodir/django.wsgi

One thing I haven't quite understood is why the Django community seems to love Prototype and hate jQuery. Is that just because Prototype is included in the standard Django package? Is it purely historical, with jQuery the abandoned but superior Betamax to Prototype's VHS?

Learning Ruby on Rails

I'm learning Ruby on Rails to help a friend with his website and to be able to put it on my resume, and keeping notes as I go.

I've gotten the thing to do typical CGI script stuff, and now I'm figuring out how database access works. One big surprise is that as Rails advanced to version 2.0, one of the basic commands for setting up database access changed. Google "rails 2.0 scaffolding" for details.

Honorable mention for BetterExplained.com website and its author, Kalid Azad

Kalid Azad's BetterExplained.com website has a lot of elegantly straightforward articles on interesting topics, many of them mathematical. He's doing some really interesting stuff, including a brilliant online calculator that you can use to embed calculations in web pages.

I'm not a Microsoft fanboy by any means, but I admire the video he made, using Windows 7 for humanitarian purposes.

In an article on happiness, Kalid includes a video of Steve Jobs's commencement address at Stanford. I'm really grateful that he included this.

Remember this is a guy who had a diagnosis of terminal cancer a year earlier.

Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven't found it yet, keep looking. Don't settle... All external expectations, all pride, all fear of embarrassment or failure -- these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.

I recently left a job where I wasn't following my heart. The money was good and the rest of the economy was bad, so I spent a lot of energy and effort trying to make it work, but there was no passion or fun or excitement. So this talk resonates for me now.