Professional Online Whiteboarding

My job involves explaining and discussing all sorts of things to people who are almost always situated remotely. Particularly, I like sketching and drawing - that's what we software people do at the end of the day: planing, discussing and implementing boxes and arrows between them.

Shortcomings Of Builtin Tools
For a while I tried to use the builtin whiteboard tools together with the mouse as drawing device. This was inconvenient for a couple of reasons:

• drawn lines where usually quite edgy and appeared not natural
• Microsoft Lync (nowedays Skype for Business) had some lag when I tried the whiteboard there
• I never managed to freehand draw nicely with the mouse

Inspired By A Worldwide Math Trainer
At the end, I never used the builtin drawing facilities because I felt so limited. To overcome this I had a look what other people where doing. I ended up with a similar setup like the Khan Academy:

• Wacom Bamboo Tablet
• SmoothDraw (Freeware for Windows) or as a better alternative
• Autodesk Sketchbook (Free version is enough)

Note: The Wacom Bamboo series of tablet is discontinued and is succeeded by the Intuos series.

Since I wasn't sure if this whole tablet drawing thing works out for me I went for a cheap used Wacom Bamboo One tablet which I bought for 15 € off ebay.

A Cheap Wacom Bamboo Tablet Is All To Get You Started

Using The Drawing Tools
Software wise I kicked off with SmoothDraw which is not a bad choice since it is free and it doesn't require admin rights for its installation.

These days I use the free edition of Autodesk Sketchbook more often. Compared to SmoothDraw the same (limited) drawings of mine look much nicer here. Also the editing tools are way better with Sketchbook. For example, they have a Lasso selection which lets you draw the selected area and than easily move, turn or resize it.

Pressing TAB toogles between the blank canvas and the tool bars. This is nice when presenting: To get the audiences full attention I usually present inside the blank canvas. Only when I need to change a tool or setting plus I can't remember the hotkey I quickly press TAB to get all the menus and then TAB again to return to the blank canvas.

Talking about hotkeys or keyboard shortcuts: To become a fluent presenter it is advisable to know how to change between tools and colors by only using the keyboard. When I present, I draw with my right hand and use the left hand for pressing the hotkeys.

Airbrush For Visual Quick Wins
Beside the pressure sensitive pen tip I really do love the airbrush tool. Once I am done with my boxes and arrows I highlight the important parts with a subtle colored airbrush. These makes to end result much more appealing and it is so simple to do.

Post Production With Gimp
No matter what drawing tool I am using, I never use the full size of the canvas. Actually, I like to get started with a large canvas just to be sure I don't need to limit myself later on.

When I want to share the results of a drawing session I usually export the images to png and then import it into Gimp. There I cut the canvas to the actual size of the image. Also I downscale the image to a size usable for online purposes.

For this I zoom the image to a satisfying size (in percent). I then go to "Image > Scale Image". There I change the size unit from px to percent. I then enter the desired percentage I previously found out by zooming into the image.

When I am done I usually use the "Export to original image" option (can't remember the right name) to overwrite my png with a version of the right size.

Conclusion
I started my drawing experiment over a year ago. When I got the tablet I spend some evenings at home listening to good music and drawing (not only) boxes and arrows. This is also an advice for you: drawing with a tablet needs some time to get used to it. For enough self confidence for your free drawing presentation you better spend some time training which by the way is good fun anyway.

For me these 15 € together with the evenings of good music and drawing training have paid off more than once. In a world of over designed Powerpoint presentations it makes a difference when you build up a topic just with the power of a pen.

Software Craftmenship In Leipzig Is Picking Up Pace

Last September I wrote about my attempt to host a local meetup for the software craftsmen in my home town of Leipzig/ Germany. After a slow start things are recently going well.

This week we had Alex of Grossweber over how gave a very compact but at the same time well structured introduction to git and its advanced topics like interactive rebase. Up to this point my git knowledge was very basic but after this evening my understanding of this topic is much clearer. Good work Alex.

git Session with Alex @ Makerspace Leipzig, 2016/02/17

We started to publish our events on two platform. Beside our native Softwerkskammmer page we started using meetup.com as well.  I suppose being a meetup.com event helped use to constantly increase our attendees. Last week we where almost 30 people and I had to limit the subscription to the event. A couple of month back we where always below 10 people - how quickly things change...

"pragma once" as a better alternative for guard clauses

An include guard is a very popular and incredibly useful hack:

... an #include guard, sometimes called a macro guard, is a particular construct used to avoid the problem of double inclusion when dealing with the include directive. (Link)

Example:

// person.h
#ifndef PERSON_H
#define PERSON_H

typedef struct {
    char* first_name;
    char* last_name;
    int age;
} person;
 
#endif /* PERSON_H*/

The idea is to let the C preprocessor only evaluate the guarded code if the global symbol PERSON_H is not defined. Since line 3 defines PERSON_H as the very first step the person struct is guaranteed to be seen only once at compilation time.

It is simple macro programming but so popular that an IDE like Eclipse CDT auto-generates the include guard for you. You can even choose different naming schemes.

But this technique also has some downsides:

• three lines of extra code
• potential name clashes if there is another person.h in an included project

The latter one can be worked around with improved naming schemes like adding the path to the symbol name (#define MY_PROJECT_SRC_PERSON_H) or using a simple random number (#define DF454FSKWDLD) but stop - this hack is getting worse and worse.

Luckily there is a solution for quite some time now called #pragma once. The above example looks rewritten like this:

// person.h
#pragma once

typedef struct {
    char* first_name;
    char* last_name;
    int age;
} person;

The ‘#pragma’ directive is the method specified by the C standard for providing additional information to the compiler, beyond what is conveyed in the language itself. [Link].

And this is what #pragma once does:
#pragma once is a non-standard but widely supported preprocessor directive designed to cause the current source file to be included only once in a single compilation. [Link]

The Wikipedia page which provides this quote gives also a list of compilers which provide this feature. If you're not forced to compile under Solaris studio you're fine.

Printing boolean values in C

Since the bool type in C is nothing else than an integer, a naive printout of the number will also just produce a number. This little trick gives you a text representation of the boolean value:

bool status = true;
printf("The status is %s", status ? "TRUE" : "FALSE");

The ternary operator is evaluated first. Since the value of status is true the string TRUE is the second argument of our printf.

So the output is:

This status is TRUE

Infrastructure As Code - Some Lessons Learned

I used the last days of 2015 to automate the installation of our C development environment. Here are some of the ingredients:

• Eclipse CDT
• Eclipse ProC - Extensions. Required to let Eclipse CDT understand ancient *.pc files
• Google Workspace mechanics plugin to offer team wide default settings in Eclipse
• some other Eclipse plugins like subclipse, Vrapper (vi keyboad layout or Eclipse)...
• Oracle Client including header files and pre-compiler
• ...

Due to this long list of required tools and plugins setting up our development environment is quite complex. After asking for some clarification on the right tool for the job I went for Vagrant.

10 days later the result was as expected. I am now able to say vagrant up and (if running for the first time) a basebox from an internal repository is downloaded and Vagrant is then running all the shell scripts I've written to install the above. This is called to provisioning step and takes place only once.

Coding the infrastructures I found my self dealing with some of the issues I only knew from ordinary coding so far.

External Dependencies

When installing software which was not provided in a nice repository (Oracle client and Eclipse ProC extension for example) I had to decide if the install scripts download some version of the software from the internet or if I should add a specific version to my Vagrant project and keep it.

I went for the latter. To reduce external dependencies (=download links at the Internet) I have the required archives and binaries locally under version control. If I want to update the software, I manually need to download the newer version and let it replace the older version.

Inside my scripts I always tried to use wildcards when it came down to file names. The goal was that a version update does not require an update of the provisioning shell scripts. A simple overwrite of the old version with the new version should do.

ECLIPSE_PURE_SDK="/vagrant/files/eclipse-SDK-*-linux-*_64.tar.gz"
...
tar xvzf ${ECLIPSE_PURE_SDK}

Feedback Loop

The general approach to codify my infrastructure setup was very similar to the way I usually go forward:

1. write new code or correct existing code
2. let it compile (optional, only required for compiled languages of course)
3. execute
4. find the error
5. start from beginning

Translated into the world of Vagrant this is:

1. write a new installation task or correct an erroneous existing one
2. let Vagrant provision (=execute) the installation tasks
3.  find the error in the installation tasks
4. start from beginning

Particularly the second step, provisioning the Vagrant box was painfully slow. It took about 5-7 minutes to finish this step. Going forward in small incremental steps this means a lot of 5-7 minutes forced breaks.

This time I accepted these waiting times. For the next bigger infrastructure coding job I will definitely try out one of the configuration management tools (Ansible, Puppet, Chef...). All of them give me something really helpful I was lacking this time - idempotence:

... operations [...], that can be applied multiple times without changing the result beyond the initial application. (Wikipedia)

My imaginary updated Vagrant cycle would then look like this:

1. write a new installation task or correct an erroneous existing one
2. execute all installation tasks, only the new or updated once run
3. find the error in the installation tasks
4. start from beginning

This should save me a lot of time since here only the differences of the configuration tasks are executed.

Thoughts On "SE-Radio Episode 242: Dave Thomas on Innovating Legacy Systems"

In episode 242 Software Engineering Radio interviewed Dave Thomas about how to deal with legacy systems. I liked the show so much that I had to do a sketchnote:

Controversial And Very Inspiring At The Same Time - SE Radio 242 with Dave Thomas

Actually I am a faithful follower of Working Effectively With Legacy Code : isolate the piece of code you want to change (dependency breaking), write tests for it and then modify the code using TDD. Over time I got quite good at it - even in C. However, it's a lot of effort - even when you're trained.

Dave sayed "Unit tests are a waste of time, focus on acceptance test" (end-to-end tests). The problem with end-to-end tests is that they are even harder to setup. Instead of mocking the objects around you, you have to provide all the external dependencies or at least good replacements:  test databases, test middleware, test clients...

Anyway, once you've managed all that and wrote your first end-to-end test, things are getting easier a lot. Covering "unhappy paths" with tests is now actually quite simple - drop a central database table, switch of the middleware, send faulty messages to your application and check what's going on.

With all this virtualization (docker as latest hype) and infrastructure as code (Puppet, Chef, ...) we now have got good tools to write end-to-end tests which are repeatable, automated and maintainable.
Surely this was not as simple in 2004 when "Working Effectively With Legacy Code" came out.

Dave's statements  reminded of the Golden Master approach which is quite similar. However, the initial end-to-end tests there is only meant to  provide the basic safety net towards a unit test coverage. The latter one is the actual goal of "Golden Master" testing.

So yes, maybe going from outside to inside is nowadays a better way of creating a safety net. I am still not convinced to ditch unit testing of old code completely but this is as always something you have to try out.

Something For The Toolshelf - Code Analysis Tools Used For Security Analysis Of Truecrypt

Recently the Bundesamt für Sicherheit in der Informationstechnik (BSI), an authority of the German government released a security analysis of Truecrypt. This analysis was carried out by the Fraunhofer-Institut für Sichere Informationstechnologie (SIT) in Darmstadt /Germany. This institute is part of the Frauenhofer society - a research organization spread across Germany.

From a software engineering perspective I was curious what approach the researches took to evaluate the code code base.

 

GOTO

Apparently also the Truecrypt authors liked their goto. The study on goto (my translation):

To implement exception handling the usage of goto is generally accepted since the language C does not offer an own feature for that. New research concludes that meanwhile programmers are predominantly using goto in a sensible way.

Die Verwendung von goto wird jedoch im Allgemeinen zur Umsetzung einer Ausnahmebehandlung akzeptiert, da die Sprache C kein eigenes Konstrukt hierfür kennt. Neuere Untersuchungen haben ergeben, dass Programmierer mittlerweile die goto-Anweisung überwiegend nur noch in sinnvoller Weise verwenden. (original)

On that topic the study quotes An empirical study of goto in C, a paper which was pre-released in February 2015 and which was subject of my previous post.

 

Complexity Of The Source Code

To measure complexity the authors of the study employed a tool called Lizard which can deal with a bunch of languages including C, C++, Java, Python and Javascript. 

Here is the feature list taken from the Github page of Lizard:

• the nloc (lines of code without comments),
• CCN (cyclomatic complexity number),
• token count of functions.
• parameter count of functions.

As their measure of complexity the study uses the cyclomatic complexity:

As a measure for the complexity of the flow of control especially the cyclomatic complexity is being used. Values higher than 15 are an indicator for potential refactoring. Values above 30 are usually accompanied by flawed code. (my translation)

Als Maß für die Kontrollflusskomplexität wird insbesondere die zyklomatische Komplexität verwendet. Werte größer 15 sind ein Indiz dafür, dass Refaktorierung sinnvoll ist. Werte über 30 gehen oft mit fehlerhaftem Code einher. (original)

Code Duplicates

To find identical pieces of source code the autors of the study use Duplo, a duplicate finder for C and C++. With its default settings the tools consideres three and more identical  lines of code as duplicates. 

 

Static Code Analysis

For this kind of analysis three tools where used: Coverity, Cppcheck and the Clang Static Analyzer. The interesting point here is that there where almost no overlaps in the errors found by the three tools. Which brings me to the conclusion that it is a sensible investment to integrate more than one static analyzer in the Continuous Integration chain.

Rehabilitating C's goto

I admit - I regularly write goto's. Actually almost all non-pure functions see at least one goto. For always the same reason: Handling errors and cleaning up resources. I already wrote about the technique 1 1/2 year ago.

Example For Error Handling And Cleanup using goto [1]

In my eyes the usage of goto for cleanup and error handling is a good thing. The flow of application logic is not unnecessarily cluttered with local error handling. Instead the function is divided into two parts: The upper part which contains the application logic and the lower part which contains the error handling and the cleanup of resources.

However, using these goto's always left me feel like doing something in the gray zone: There is an old ban from the 60ies (Letters to the Editor: Go To Statement Considered Harmful, Dijkstra, 1968) but without talking to much about it in public C programmers still carry on writing goto.

The paper An Empirical Study of goto in C Code  releases as a pre-print in February 2015 now takes an interesting second look at this old ban.

The international group of researches who was involved in the paper analyzed 2 million lines of C code collected from 11K Github repositories. I leave the reading of the entire paper up to you and jump directly to the important part of the conclusion:

...far from being maintenance nightmares, most usages of goto follow disciplined, well-designed usage patterns, that are handled by specific constructs in more modern languages. 
The most common pattern are error-handling and cleanup, for which exception handling constructs exist in most modern languages, but not in C. Even properties such as several goto statements jumping to the same label have benefits, such as reducing code duplication, in spite of the coordinate problem described by Dijkstra.

That sounds like good news to me - I eventually can exit the gray zone.

Slides For "Not Your Fathers C - C Application Development in 2015"

Before going on a too short vacation I attended the Developer Open Space conference in my home town Leipzig. As the conference name suggests this was an Open Space conference where the participants them self create the agenda for the day.

I held a session on modern C development which was a high level summary of my past posts here. The slides are now online:

Not Your Fathers C - C Application Development In 2015 from maiktoepfer

Software Craftsmanship in Leipzig

I started hosting the meeting of the Leipzig branch of the German software craftsmanship community Softwerkskammer. This is also a reason why I am short on blog posts here. Anyway, it's good fun. We try to team with the hardware nerd at Makerspace Leipzig by meeting us at there premises. We think this is a better fit than meeting in some companies (clean) office environment.

Markdown Presentations

I plan to do offer a session on modern C development called Not Your Fathers C at the Developer Open Space conference in Leipzig/ Germany. For that purpose I was looking for a way to prepare slides not in some proprietary format like Powerpoint but in pure text which is rendered to something nice.

 After one evenings search engine investigation for me the way to go forward is remark which lets you create Markdown based presentations which can also be run offline. Very neat.

Socrates 2015 - What the f**k

I'm a IT professional for over a decade now - but I've never attended something like this before:

A colleague and me took part at the Socrates 2015 conference. It's an Open Space conference which mean there is no agenda at the beginning. Just an empty flipchart which if filled with self-proposed topics of the participants at the beginning of each conference day.
No pop star like speakers but people like you and me how admit "I am not the expert but I've done something in that field and I want to share this knowledge with you." For me this is all you need - for everything more involved there are books.

While thinking about the conference I was astonished how much I've either learned or was pushed towards something. Here is my top 5:

1. Sketchnotes. This is for me "Doing something more with the whiteboard than the usual stuff without needing to be an artist." I believe in the area of Powerpoint spoilt organisations  it makes a huge difference to develop a topic just with a flipchart and a pen. It's amazing how much you can do. I love it.
2. Walking Skeleton. I never heard of this topic before. It's basically an approach to build up a system not from inside out but from outside to the inside, guided by so called end-to-end tests as well as unit tests. The book to read on that topic is Growing Object Oriented Software Guided By Tests. People refer to the approach as "London School Of TDD" (the authors of that book are based in London) in contrast to the "Chicago School Of TDD" which is TDD as we know it (Kent Beck, Uncle Bob). Really interesting.
3. Personal Kanban. I was once again pushed to think about my time management method. I attended a session about Knowledge Management but we soon found out the the key is actually Time Management. So one of the attendees offered a session about time management the next day. It all was about the guys Getting Things Done interpretation. Anyway, during the discussion I was reminded on the existence of Personal Kanban, and this is what I am trying out at the moment, all analog!. There will be a post with a nice picture somewhen later.
4. Legacy Code Retreat. I spent quite some time wadding into muddy legacy C code. I thought I am quite good at this topic but this guy who hosted a 2 hour session an how to attack lagacy code was somewhere else. Really impressive. I've done some handson training for a Golden Master Test. Also, training people to deal with legacy code can be simplified by using Adrians work.
5. I had a chat with somebody at dinner how was working for a software consultancy run as a cooperative. Basically, they have no boss and decide all together how they want the company to proceed. This was really thrilling, I was just about to hand in my CV but they are based in the wrong city ;-).

Overall the conference was much more than I expected. In my hometown Leipzig/ Germany there will be the Developer Open Space conference in some weeks, run also as an Open Space event - this time I plan to prepare something my self. Working title "Not Your Father's C".

Yes - there were also some hippies ;-)

Uncle Bob On Videos

For those of you who are even later than me: Uncle Bob, famous for his Clean Code book has produced a series of one hour videos which cover the topics of the book - but in a very funny and at the same time educational way: Clean Code Videos

Bash syntax check

When automating your administrative tasks also shell scripts are checked in to a version control system. The built-in command line argument -n allows a syntax check of the shell script without executing it:

$ bash -n myScript.sh
$ echo $?
0

If the syntax check is successful nothing is displayed and the exit code is 0. However, if there is an error (like in the following case a missing parenthesis) an error message is printed and the exit code is 2:

$ bash -n myScript.sh
./myScript.sh: line 23: syntax error: unexpected end of file
$ echo $?
2

When checking your scripts in that way inside your continuous delivery pipeline it is easy to spot simple type errors.

Dependency Breaking in bash

In times of Devops you might want to also (unit) test your shell scripts. A collegue of mine introduced me to a nice trick how to do dependency breaking with bash.

The technique is based on the declare -f command which allows you to print the definition of previously defined function:

$ sayHello() { echo "hello world";}
$ declare -f sayHello
sayHello () 
{ 
    echo "hello world"
}

The following script sketches how to utilize this feature to replace an original function with a stub at run time:

#!/bin/bash

doComplicatedThings() {
    echo "i am doing many complicated things"
    # ...
}

codeOfOriginalFunction=$(declare -f doComplicatedThings)

# stub out "doComplicatedThings"
function doComplicatedThings { :; }

# executes stub (does nothing)
doComplicatedThings

# re-establish orignal function
eval "${codeOfOriginalFunction}"

# executes the original function
doComplicatedThings

The function doComplicatedThings is the production function which might have been loaded from an external script file via the source command. The definition of doComplicatedThings is then saved as a string into codeOfOriginalFunction.

The replacement function is just an empty stub. When executing the stubbed version of doComplicatedThings than as expected nothing happens. Calling eval restores the original function so that the second call of doComplicatedThings prints out the original message.

My Favorite Things - A One Time Downloader

For this blog it's a quiet year so far. I'm busy with work related stuff. Also, I'm collecting material for a book a colleague of mine and me are planning to write, more on that later.

Since Google Code is closing down I had to move an other old project of mine to github.com. It's a plain old CGI applications which allows small Do It Yourself (DIY) vinyl record labels to create small download vouchers for their releases. Being a vinyl collector myself five years back most of the vinyl records I bought came without any legal possibility to obtain the music in digital format as well.

Downloading the MP3 files for "A very good Record"

I wanted to make the world a little better here. Over the years the software has been used around the world. I was occasionally in touch with users. Some of them where so happy about the tool that they've send my a bunch of records they recently released.

This project was my first time getting out to the world with some software I developed. Back then I decided to use Perl as implementation language. For the reasons see here.

Nowadays I still would choose a dynamically typed language but I guess it would be Python then.

A Node.js based PLC Emulator

I just uploaded an old project of mine to Github: https://github.com/samba2/ripley/

Many thanks to my girlfriend Franziska for drawing Mr. Ripley above.

Background
Some years back a tried to get my head around Javascript and Node.js in particular. The latter one was quite new and you would read about it everywhere.

Based on some ideas from work I started implementing an idea I had for a long time: In a production environment their are many Programmable Logic Controller (PLCs) which communicate with a background system. To test the background systems logic it is common to replace the PLC with an emulation software. This software is than acting like the PLC (sending messages, receiving messages...).

At the time I was in charge of the background system including the required programming. Since the emulation software is complex we always required a separate emulation programmer for testing. My idea was create an alternative emulator which was configurable by a config file and which could cover 80% of the usual emulation cases.

I had good fun implementing my idea, working with Node.js went reasonable well. However, by the time when I finished I had a much better idea which is in use now: instead of impersonating a PLC I found out is much easier to write integration scripts on the background system itself which would trigger the same actions as the PLC emulator would have triggered.

Those scripts follow the xUnit style of programing tests and are way more broader and flexible.

The github repository is where Ripley is now laid to rest.

Quick C Tricks - Assertions With Error Message

This quick post is the last little trick I got out of the book Patterns in C written by Adam Tornhill. The technique presents comments for assertions which also appear as an error message if failing.

Here is an example:

#include <assert.h>

int main() {
   assert(1 == 2 && "This is an error message");
}

Running the code gives the following result:

main: Assertion `1 == 2 && "This is an error message"' failed.

This trick works since the string This is an error message itself evaluates to True. The actual checking is taking place in the first part of the assertion, a second appended True does not influence this behavior.

You can argue that this is a hack (that's true) but giving the user some guidance in the event of failure is also not a bad idea.;

Review of "Patterns in C" by Adam Tornhill

Twenty one years ago the book Design patterns : elements of reusable object-oriented software changed the way object oriented software was written. Some month back I asked myself if somebody has tried to apply those ideas to plain C. Although C has no object orientation build in, using Abstract Datatypes (ADT) it is possible and also preferable to structure C code in an object oriented manner.

My research lead me to the online book "Patterns in C" by Adam Tornhill, available as PDF or epub from leanpub.com. This book combines a couple of blog posts the author did over the course of years. Not unexpected, the book is introducing ADT in C as prerequisite for the rest of the book.

All the design patterns presented in the book share a second similarity: They all gain their flexibility through the consequent  usage of C function pointers. I admit, I did barely touch them in the past. After having read this book I still believe you should not overuse them but they give you flexibility in your programming you would not expect to find in good old C.

But let's look at the first chapter. It deals with the implementation of a state machine. The author quickly illustrates the classic approaches being simple case statements and a table based approach. He then argues that those classic approaches are not scaling to the big which lets him introduce a C take on the STATE pattern. I personally found that example rather confusing but I would consult it a second time if a state machine is on my C development agenda.

The next chapter deals with the STRATEGY pattern. This is also my favorite chapter of the book. The example given shows how a customer record can be designed leaving certain calculations interchangeable. This is a good example of how modern C can look like and something which is already in my personal tool box.

The following chapter introduces an example implementation of the OBSERVER pattern. The idea is that an object can have one or more subscribers which get informed if the internal state (e.g. measuring data) of the object changes. The subscribers can then decide what to do with this information. It turns the object communication, excuse me, I mean the ADT communication upside down from polling ("Did something change at your side?") to pushing: "Attention, something changed at my side!"

As with the STRATEGY pattern before the main ingredients for the OBSERVER chapter are ADTs and function pointers. The implementation is pleasant to work through and surprisingly straight forward.

The last pattern discussed is the REACTOR pattern. Here, an event is received and dispatched to the appropriate event handler. Although the patterns logic is simple, I had my difficulties to see that logic happening in the code. Maybe reading not only snippets but the whole example would have helped.

The book closes with a couple of C tricks of which I already posted Simulate Keyword Arguments here. This last chapter also shows my general problem with the book: it is too verbose and also trying to be too intellectual. At the end we talk about established programming conventions in the OO world and their adaption in C. This is straight forward software engineering where a Kafka quote feels a bit out of turn.

Concluding, I learned that function pointers are an important tool towards a flexible design. Together with ADTs they make the implementation of established OO design patterns possible also in C. The book itself is not an easy read, though.

A software engineer does for a dime what any fool can do for a dollar - SICP Finished

Done! Done! Really, no kidding. It took longer than expected but eventually I'm done with my self study of the online material of Computer Science 61A, Spring 2014 edition. The course is a modernized version of the material in "Structure And Interpretation Of Computer Programs" (SICP).

For more informations on why I initially started the course see my post from March 2014.

This post is about my lessons learned, findings and experiences.

General Software Engineering Findings

A great quote from the lecture (for a geek T-shirt maybe?): A software engineer does for a dime what any fool can do for a dollar.  The key to such an efficiency is abstraction and separation of concerns.

This wasn't knew to me but it was good to see that those concepts were not invented by the "object-oriented people". They've made the idea popular but you also can achieve nice abstraction layers in your code by only using functional programing, see this section of the original SICP book for a good example.

Recursive Thinking

Before I started the course I probably never wrote a recursion - at least not by intention. In SICP recursion is an important concept and I had solve many different exercises on that topic. Over time I believe I can say I've adopted the recursive thinking.

The most important idea here is the leap of faith. This is a way of thinking which help dealing with recursion: Since it is hard to play a whole recursion through in your head (particularly for complicated tasks) develop your code with a simpler variant of the original problem in mind and believe that the recursion will also work for the more complex variants.

Read Eval Print Loop (REPL)

These days I wouldn't start a bigger programing project without having a REPL available. A REPL gives you an interactive way to try out some pieces of code before adding it to your actual program.

Ideally you can interact with your programming language line by line - like in Python: Type in python and you're inside the REPL where your can program interactively. For Javascript on the server side node.js offers something very similar. For client side Javascript just use the console of the browsers development mode (F12).

For Java I utilize the Eclipse Scrapbook, for C I sometimes use codepad.org. Those last two examples are no REPLs since they are not interactive but at least they provide some convenient way to try out code. For C I might use straight gdb next time - we'll see how well that goes.

Functional Programming

SICP is heavily relying on the power of higher order functions. That is due to the Scheme heritage of the course material. In the original SICP book (the wizard book) the possibility to assign values to a variable is mentioned somewhen in the middle of the book. This modern version of the course is not as puristic.

The further you work your way through the course material you see functional programming being used side by side with classic procedural and object oriented programming. In my opinion this is what makes Python such an interesting language: you pick the programing paradigm (the tool) which fits best for the job to be done.

I did some Javascript coding recently and used this occasion to re-read Eloquent Javascript. After having taking SICP this whole fuzz about Javascript functions is now much less confusing. The syntax could be more elegant though ;-)

Object Oriented Programing

Due to my efforts in structuring my work C code I was aware of Abstract Data Types (ADT) before. In the course we've pushed this idea further: We've been using Python to implement an own object model including inheritance. This object model stood beside the build-in OO-features of Python.

This was an impressive and eye opening experience. Now I eventually know how to give the bash OO-capabilities ;-)

To quote the teacher, Professor Hillfinger: Object oriented programing is just a disciplined way to structure your programs.

Declarative Programing

The last quarter of the course had a huge emphasis on declarative programing. We've learned the concept, discussed a simple interpreter of a small invented declarative language written in Python. This language was a mix of Prolog (the structure) and Scheme (the brackets).

I saw that this is a very different way to approach solving a problem and that you should be aware of it. However, for my taste we've spend too much time on coding exercises with this little academic teaching.

Python as The Courses Main Language Language

Many years back my first programing language was Perl which impressed me with the power of a couple of code lines. Python is for me Perl done right - a multi purpose high-level scripting language with an easy to read syntax. At least Python doesn't share Perl (bad) reputation for cryptic one-liners.

Writing some Java after having done Python makes me feel like "Is the handbrake still pulled?" - not to mention my emotions going from Python back to application programing in C.

Scheme - Your Fathers Brackets

The original material of the course was entirely held in Scheme. In the new version the main programming language was Python but Scheme was still very present.

After doing some Scheme syntax and coding exercises the last project was about writing an own Scheme interpreter in Python. I wouldn't say that I'm now an expert in designing programming language but if I would ever get the task to implement an own language I definitely have a clear idea of the basic concept - a constant (meaning recursive) evaluation and application of a stream of input tokens.

This concept is so fundamental that it made it on the originals book title:

Quelle: http://icampus.mit.edu/files/2011/12/xTutor1.jpg

I have to say that I haven't become a big Lisp/ Scheme enthusiast. I still found the syntax rather confusing. However, I can image that it could be fun coding a bigger project and thus spending more time with the language..

Time Management

I started the course when I was off on parental leave for two months. At the beginning of that time I appointed one full day per week for the course work, in the second month I increased to two days per week. Additionally, I was watching the lecture screencasts in the evenings at home.

To stay focused I tried to study at my local universities library as often as I could.

When I started working again things slowed down. I tried to shift my study time to the evenings but my performance is not the best after 8:30 pm, particularly when you're not "just coding" but trying to solve tricky lab tasks. Additionally, work (a lot of mighty C programming) was quite involving.

The trick was to believe in small steps and to celebrate every complete homework, discussion or lab assignment.

At the beginning I tried to do all homework tasks, also the extra credits. When I saw my pace going down I decided to be just an average student and to focus only the compulsory work. My pace increased again and I realized that to be average in my situation was a good decision. It was my trade off being a full time worker, committed family father and a remote student at the same time.

Self Paced Learning

The course wasn't designed as a self study course. However, the lecture screencasts where great, I watched them all, some of them more than once. I found out that my brain digests new ideas better if new material is first presented/ explained to me and then I start practicing on my own.

The study guide Composing Programs also helped a lot to approach a topic from a second side (first listen to the screencast, second read in the book).

When we've learned about environment diagrams, the Online Python Tutor was very helpful.  It's basically a graphical debugger for smaller code snippets which nicely illustrates the environment frames involved  and their content.

Most of the hands-on assignments (labs, homework, projects) came with unit tests which described the intended behaviour of the functionality to be implemented. This way I could always check my implementation. Solutions were also available (with the exception of the projects).

Maybe the biggest problem for me was the lack of support from somebody else. As an enrolled student there are assigned teaching assistants (TAs) to help you. If I had questions I only had the Google search on my side which wasn't always as effective.

Particularly when I got into writing recursive functions I was sometimes fighting for hours to finish a task. Of course I only could have looked up the solution but this wasn't my intent - the whole purpose of my studies was to deeply understand the material.

Bigger projects where intended to be work on by small teams. As a one man team I did all the tasks on my own. This was also one reason why I took my longer then planned to finish the course.

Next Steps And Follow Up Courses

Online courses are great. They have their limitation once you get into project or group work. But e.g for learning a new programming language or library sites like codeschool.com are nowdays first choice for me. Infact, I would rather take a week off and work my self through a quality online course than attending a real life training. Check out this AngularJS course as an example for what I mean.

I was a bit disappointed that CS169.1x had already started when I finished: CS169.1x teaches the fundamentals of software engineering using Agile techniques to develop Software as a Service using Ruby on Rails. They have now the archived material available so I might work my self through that course again - we'll see.

Conclusion

It was fun and I definitely would do it again. Also for me the modern Python version of the course was more appealing than the pure text book based study of the old Scheme book.