Posts mit dem Label C werden angezeigt. Alle Posts anzeigen
Posts mit dem Label C werden angezeigt. Alle Posts anzeigen

Dienstag, 12. April 2016

Avoiding Temporal Coupling - Part 2/2

In part one of this post I briefly introduced the problems which come with temporal coupling and "passing a block" as a technique to overcome those problems. In this last part of the post I want to demonstrate "passing a block" in pure C.

Lets start with a basic implementation:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

typedef void (*FileCommandFunc)(FILE*);

int withFile(const char* fileName, const char* fileAccessMode,
             FileCommandFunc func) {
    FILE *myFile = fopen(fileName, fileAccessMode);
    if (NULL == myFile) {
        fprintf(stderr, "Error opening file '%s': %s\n",
                fileName, strerror(errno));
        return EXIT_FAILURE;
    }
 
    func(myFile);

    fclose(myFile);
    return EXIT_SUCCESS;
}

void printFirstLine(FILE* file) {
    char buffer[1000] = {0};
    if (fgets(buffer, sizeof(buffer), file)!=NULL) {
        printf("%s", buffer); 
    }
}

void printWholeFile(FILE* file) {
    char buffer[1000] = {0};
    while (fgets(buffer, sizeof(buffer), file)!=NULL) {
        printf("%s", buffer); 
    }
}

int main() {
    withFile("demo.txt", "r", printFirstLine);
    withFile("demo.txt", "r", printWholeFile);
}
Function withFile takes three arguments: the name of the file to be opened, the access mode (read, write, append...) and a file command. The latter is a pointer to a function which accepts a file object as only argument.

printFirstLine and printWholeFile are two example commands. Both employ fgets to traverse the file line by line.

In main we see "passing a block" in action. File "demo.txt" is opened in read mode and the actions implemented in the command functions are executed against the file. Error handling and closing of the file handle takes place inside withFile - the user of the function doesn't have to bother with that.

When coding this first draft I realized that my solution is quite limited. Inside the command you can only work with the file passed in and global variables. Other variables aren't simply accessible.

With the GNU Compiler Collection gcc there is the nested functions  feature which nicely works around this limitation:
int main() {
    const char* header = "Content of demo.txt";

    void printWholeFile(FILE* file) {
        puts(header);
        char buffer[1000] = {0};
        while (fgets(buffer, sizeof(buffer), file)!=NULL) {
            printf("%s", buffer); 
        }
    }

    withFile("demo.txt", "r", printWholeFile);
}
I admit the example is a little bit constructed since you could easily print the header before the call of withFile but there you go. The idea is simple: Since printWholeFile is now defined inside main, printWholeFile has access to all global variables and the local variables of main as well.

The downside of this is that it only works in gcc. Here is a more general implementation which uses a technique you find for example in the glibc comparision functions.
The command function accepts a second argument which is a void pointer. This allows us to pass anything into the command. We only have to cast it to the right type before using it. This is basically an accepted disabling of C's type checking.

Here is the code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

// withFile.h
typedef void (*FileCommandFunc)(FILE*, void*);

typedef struct {
    const char* name;
    const char* accessMode;
    FileCommandFunc func;
    void* commandArgs;
} withFileArgs;

// withFile.c
int withFile(withFileArgs args) {
    FILE *myFile = fopen(args.name, args.accessMode);
    if (NULL == myFile) {
        fprintf(stderr, "Error opening file '%s': %s\n",
                args.name, strerror(errno));
        return EXIT_FAILURE;
    }
 
    args.func(myFile, args.commandArgs);

    fclose(myFile);
    return EXIT_SUCCESS;
}

// production code

// the commands
void printFirstLine(FILE* file, void* commandArgs) {
    const char* header = commandArgs;

    puts(header);
    char buffer[1000] = {0};
    if (fgets(buffer, sizeof(buffer), file)!=NULL) {
        printf("%s", buffer); 
    }
}

void printWholeFile(FILE* file, void* notUsed) {
    char buffer[1000] = {0};
    while (fgets(buffer, sizeof(buffer), file)!=NULL) {
        printf("%s", buffer); 
    }
}

// using the commands
int main() {
    withFile((withFileArgs){.name="demo.txt", 
                            .accessMode="r", 
                            .func=printFirstLine, 
                            .commandArgs="Content of demo.txt"});

    withFile((withFileArgs){.name="demo.txt", 
                            .accessMode="r", 
                            .func=printWholeFile});
}
For convinces I've also replaced the growing number of arguments of withFile with something you call configuration object in Javascript, in C it is the beautiful marriage of designated initializers and compound literals, both introduced in C99.

Also note that in function printFirstLine the cast from the void pointer commandArgs to the const char pointer header is implicit - no extra cast operation needed on the right hand side of the equals operator.

Montag, 4. April 2016

Avoiding Temporal Coupling - Part 1/2

Every now and then we try to watch Uncle Bobs cleancoders videos. In episode 4, "Function Structure" the issue of temporal coupling is discussed. Temporal coupling is something we encounter quite often: to work with a database you first need to connect to the database. Next you do your work with the database and as the last step you disconnect. The same applies to working with a file: first you open it, then you perform your actions on the file handle/ object and when your done you close it.

The order of this actions is important - you can't call the database disconnect method before you call the connect method. Also, those temporal couplings are often hidden in the background. Quite often some global init method created this database connection your are using for you, at least this is what you believe. When shutting down the application you are hoping that there is another magic method which closes the connection.

In order to deal better with temporal coupling Uncle Bob suggests a technique called "passing a block". The idea is to allocate and to release resources at the point where you need it. The following pseudo code illustrates this approach:
withRessource(ressource, command):
    allocate(ressource)
    command(ressource)
    release(ressource)
Python supports this pattern out of the box with its context manager . Also Java 7 and following has a similar construct.

When looking at the pattern I immediately thought that implementing "passing a block" in plain C isn't too hard.  Function pointers are your friend here. Wait for the second part of this post to see the implementation of withFile, a function which first opens a file, executes your desired actions and then closes this file.

Donnerstag, 18. Februar 2016

"pragma once" as a better alternative for guard clauses

An include guard is a very popular and incredibly useful hack:

... an #include guard, sometimes called a macro guard, is a particular construct used to avoid the problem of double inclusion when dealing with the include directive. (Link)

Example:
// person.h
#ifndef PERSON_H
#define PERSON_H

typedef struct {
    char* first_name;
    char* last_name;
    int age;
} person;
 
#endif /* PERSON_H*/
The idea is to let the C preprocessor only evaluate the guarded code if the global symbol PERSON_H is not defined. Since line 3 defines PERSON_H as the very first step the person struct is guaranteed to be seen only once at compilation time.

It is simple macro programming but so popular that an IDE like Eclipse CDT auto-generates the include guard for you. You can even choose different naming schemes.

But this technique also has some downsides:
  • three lines of extra code
  • potential name clashes if there is another person.h in an included project
The latter one can be worked around with improved naming schemes like adding the path to the symbol name (#define MY_PROJECT_SRC_PERSON_H) or using a simple random number (#define DF454FSKWDLD) but stop - this hack is getting worse and worse.

Luckily there is a solution for quite some time now called #pragma once. The above example looks rewritten like this:
// person.h
#pragma once

typedef struct {
    char* first_name;
    char* last_name;
    int age;
} person;
The ‘#pragma’ directive is the method specified by the C standard for providing additional information to the compiler, beyond what is conveyed in the language itself. [Link].

And this is what #pragma once does:
#pragma once is a non-standard but widely supported preprocessor directive designed to cause the current source file to be included only once in a single compilation. [Link]

The Wikipedia page which provides this quote gives also a list of compilers which provide this feature. If you're not forced to compile under Solaris studio you're fine.

Sonntag, 24. Januar 2016

Printing boolean values in C

Since the bool type in C is nothing else than an integer, a naive printout of the number will also just produce a number. This little trick gives you a text representation of the boolean value:
bool status = true;
printf("The status is %s", status ? "TRUE" : "FALSE");

The ternary operator is evaluated first. Since the value of status is true the string TRUE is the second argument of our printf.

So the output is:
This status is TRUE

Donnerstag, 21. Januar 2016

Infrastructure As Code - Some Lessons Learned

I used the last days of 2015 to automate the installation of our C development environment. Here are some of the ingredients:
Due to this long list of required tools and plugins setting up our development environment is quite complex. After asking for some clarification on the right tool for the job I went for Vagrant.

10 days later the result was as expected. I am now able to say vagrant up and (if running for the first time) a basebox from an internal repository is downloaded and Vagrant is then running all the shell scripts I've written to install the above. This is called to provisioning step and takes place only once.

Coding the infrastructures I found my self dealing with some of the issues I only knew from ordinary coding so far.

External Dependencies


When installing software which was not provided in a nice repository (Oracle client and Eclipse ProC extension for example) I had to decide if the install scripts download some version of the software from the internet or if I should add a specific version to my Vagrant project and keep it.

I went for the latter. To reduce external dependencies (=download links at the Internet) I have the required archives and binaries locally under version control. If I want to update the software, I manually need to download the newer version and let it replace the older version.

Inside my scripts I always tried to use wildcards when it came down to file names. The goal was that a version update does not require an update of the provisioning shell scripts. A simple overwrite of the old version with the new version should do.

ECLIPSE_PURE_SDK="/vagrant/files/eclipse-SDK-*-linux-*_64.tar.gz"
...
tar xvzf ${ECLIPSE_PURE_SDK}


Feedback Loop


The general approach to codify my infrastructure setup was very similar to the way I usually go forward:
  1. write new code or correct existing code
  2. let it compile (optional, only required for compiled languages of course)
  3. execute
  4. find the error
  5. start from beginning
Translated into the world of Vagrant this is:
  1. write a new installation task or correct an erroneous existing one
  2. let Vagrant provision (=execute) the installation tasks
  3.  find the error in the installation tasks
  4. start from beginning
Particularly the second step, provisioning the Vagrant box was painfully slow. It took about 5-7 minutes to finish this step. Going forward in small incremental steps this means a lot of 5-7 minutes forced breaks.

This time I accepted these waiting times. For the next bigger infrastructure coding job I will definitely try out one of the configuration management tools (Ansible, Puppet, Chef...). All of them give me something really helpful I was lacking this time - idempotence:

... operations [...], that can be applied multiple times without changing the result beyond the initial application. (Wikipedia)

My imaginary updated Vagrant cycle would then look like this:
  1. write a new installation task or correct an erroneous existing one
  2. execute all installation tasks, only the new or updated once run
  3. find the error in the installation tasks
  4. start from beginning
This should save me a lot of time since here only the differences of the configuration tasks are executed.

Mittwoch, 25. November 2015

Something For The Toolshelf - Code Analysis Tools Used For Security Analysis Of Truecrypt

Recently the Bundesamt für Sicherheit in der Informationstechnik (BSI), an authority of the German government released a security analysis of Truecrypt. This analysis was carried out by the Fraunhofer-Institut für Sichere Informationstechnologie (SIT) in Darmstadt /Germany. This institute is part of the Frauenhofer society - a research organization spread across Germany.

From a software engineering perspective I was curious what approach the researches took to evaluate the code code base.

 

GOTO

Apparently also the Truecrypt authors liked their goto. The study on goto (my translation):

To implement exception handling the usage of goto is generally accepted since the language C does not offer an own feature for that. New research concludes that meanwhile programmers are predominantly using goto in a sensible way.

Die Verwendung von goto wird jedoch im Allgemeinen zur Umsetzung einer Ausnahmebehandlung akzeptiert, da die Sprache C kein eigenes Konstrukt hierfür kennt. Neuere Untersuchungen haben ergeben, dass Programmierer mittlerweile die goto-Anweisung überwiegend nur noch in sinnvoller Weise verwenden. (original)

On that topic the study quotes An empirical study of goto in C, a paper which was pre-released in February 2015 and which was subject of my previous post.

 

Complexity Of The Source Code

To measure complexity the authors of the study employed a tool called Lizard which can deal with a bunch of languages including C, C++, Java, Python and Javascript

Here is the feature list taken from the Github page of Lizard:
  • the nloc (lines of code without comments),
  • CCN (cyclomatic complexity number),
  • token count of functions.
  • parameter count of functions.

As their measure of complexity the study uses the cyclomatic complexity:

As a measure for the complexity of the flow of control especially the cyclomatic complexity is being used. Values higher than 15 are an indicator for potential refactoring. Values above 30 are usually accompanied by flawed code. (my translation)

Als Maß für die Kontrollflusskomplexität wird insbesondere die zyklomatische Komplexität verwendet. Werte größer 15 sind ein Indiz dafür, dass Refaktorierung sinnvoll ist. Werte über 30 gehen oft mit fehlerhaftem Code einher. (original)

Code Duplicates

To find identical pieces of source code the autors of the study use Duplo, a duplicate finder for C and C++. With its default settings the tools consideres three and more identical  lines of code as duplicates. 

 

Static Code Analysis

For this kind of analysis three tools where used: Coverity, Cppcheck and the Clang Static Analyzer. The interesting point here is that there where almost no overlaps in the errors found by the three tools. Which brings me to the conclusion that it is a sensible investment to integrate more than one static analyzer in the Continuous Integration chain.

Montag, 23. November 2015

Rehabilitating C's goto

I admit - I regularly write goto's. Actually almost all non-pure functions see at least one goto. For always the same reason: Handling errors and cleaning up resources. I already wrote about the technique 1 1/2 year ago.

Example For Error Handling And Cleanup using goto [1]

In my eyes the usage of goto for cleanup and error handling is a good thing. The flow of application logic is not unnecessarily cluttered with local error handling. Instead the function is divided into two parts: The upper part which contains the application logic and the lower part which contains the error handling and the cleanup of resources.

However, using these goto's always left me feel like doing something in the gray zone: There is an old ban from the 60ies (Letters to the Editor: Go To Statement Considered Harmful, Dijkstra, 1968) but without talking to much about it in public C programmers still carry on writing goto.

The paper An Empirical Study of goto in C Code  releases as a pre-print in February 2015 now takes an interesting second look at this old ban.

The international group of researches who was involved in the paper analyzed 2 million lines of C code collected from 11K Github repositories. I leave the reading of the entire paper up to you and jump directly to the important part of the conclusion:

...far from being maintenance nightmares, most usages of goto follow disciplined, well-designed usage patterns, that are handled by specific constructs in more modern languages. 
The most common pattern are error-handling and cleanup, for which exception handling constructs exist in most modern languages, but not in C. Even properties such as several goto statements jumping to the same label have benefits, such as reducing code duplication, in spite of the coordinate problem described by Dijkstra.

That sounds like good news to me - I eventually can exit the gray zone.

Sonntag, 25. Oktober 2015

Slides For "Not Your Fathers C - C Application Development in 2015"

Before going on a too short vacation I attended the Developer Open Space conference in my home town Leipzig. As the conference name suggests this was an Open Space conference where the participants them self create the agenda for the day.

I held a session on modern C development which was a high level summary of my past posts here. The slides are now online:

Sonntag, 22. Februar 2015

Quick C Tricks - Assertions With Error Message

This quick post is the last little trick I got out of the book Patterns in C written by Adam Tornhill. The technique presents comments for assertions which also appear as an error message if failing.

Here is an example:
#include <assert.h>

int main() {
   assert(1 == 2 && "This is an error message");
}

Running the code gives the following result:

main: Assertion `1 == 2 && "This is an error message"' failed.

This trick works since the string This is an error message itself evaluates to True. The actual checking is taking place in the first part of the assertion, a second appended True does not influence this behavior.

You can argue that this is a hack (that's true) but giving the user some guidance in the event of failure is also not a bad idea.;

Mittwoch, 18. Februar 2015

Review of "Patterns in C" by Adam Tornhill

Twenty one years ago the book Design patterns : elements of reusable object-oriented software changed the way object oriented software was written. Some month back I asked myself if somebody has tried to apply those ideas to plain C. Although C has no object orientation build in, using Abstract Datatypes (ADT) it is possible and also preferable to structure C code in an object oriented manner.

My research lead me to the online book "Patterns in C" by Adam Tornhill, available as PDF or epub from leanpub.com. This book combines a couple of blog posts the author did over the course of years. Not unexpected, the book is introducing ADT in C as prerequisite for the rest of the book.

All the design patterns presented in the book share a second similarity: They all gain their flexibility through the consequent  usage of C function pointers. I admit, I did barely touch them in the past. After having read this book I still believe you should not overuse them but they give you flexibility in your programming you would not expect to find in good old C.

But let's look at the first chapter. It deals with the implementation of a state machine. The author quickly illustrates the classic approaches being simple case statements and a table based approach. He then argues that those classic approaches are not scaling to the big which lets him introduce a C take on the STATE pattern. I personally found that example rather confusing but I would consult it a second time if a state machine is on my C development agenda.

The next chapter deals with the STRATEGY pattern. This is also my favorite chapter of the book. The example given shows how a customer record can be designed leaving certain calculations interchangeable. This is a good example of how modern C can look like and something which is already in my personal tool box.

The following chapter introduces an example implementation of the OBSERVER pattern. The idea is that an object can have one or more subscribers which get informed if the internal state (e.g. measuring data) of the object changes. The subscribers can then decide what to do with this information. It turns the object communication, excuse me, I mean the ADT communication upside down from polling ("Did something change at your side?") to pushing: "Attention, something changed at my side!"

As with the STRATEGY pattern before the main ingredients for the OBSERVER chapter are ADTs and function pointers. The implementation is pleasant to work through and surprisingly straight forward.

The last pattern discussed is the REACTOR pattern. Here, an event is received and dispatched to the appropriate event handler. Although the patterns logic is simple, I had my difficulties to see that logic happening in the code. Maybe reading not only snippets but the whole example would have helped.

The book closes with a couple of C tricks of which I already posted Simulate Keyword Arguments here. This last chapter also shows my general problem with the book: it is too verbose and also trying to be too intellectual. At the end we talk about established programming conventions in the OO world and their adaption in C. This is straight forward software engineering where a Kafka quote feels a bit out of turn.

Concluding, I learned that function pointers are an important tool towards a flexible design. Together with ADTs they make the implementation of established OO design patterns possible also in C. The book itself is not an easy read, though.







Sonntag, 7. Dezember 2014

Quick C Tricks - Simulate Keyword Arguments

Here is a little trick I found in the book Patterns in C written by Adam Tornhill. It will give you a syntax which looks like there are keyword arguments in C - but in fact there aren't:
...
int a, b, result;
result = sum(a = 2, b = 3);
...
This trick just moves the assignment of a and b insight the functions parameter section. However, this code is identical to:
...
int a = 2, b = 3, result;
result = sum(a,b);
...
So it is just some eye candy but are there any useful applications for that? Well, yes. This technique helped me to write some easy to read unit test code where I wanted to be explicit what the arguments of the function under test stand for.

Beside my unit tests I believe I won't use this syntactic sugar that often. If there is the need to pass in a couple of arguments to a function I would use this technique which relies on compound literals and designated initializers.

Sonntag, 30. November 2014

Quick C Tricks - Structured Include Path

In a larger C project you might want to have your API (=the collection of your header files) more structured instead of throwing all the header files into one flat directory.

The cpputest project is using a nice and simple approach. Their software (a C and C++ unit test library we are using at work) consists out of core functions and extended functionality.

Here is a sketch of the include path layout:
/.../cpputest/include/
                      CppUTest/
                              MemoryLeakDetector.h
                              TestHarness.h
                                ...
                      CppUTestExt/
                              GMock.h
                              MockFailure.h
                              ...  
The compiler gets the main include directory -I/.../cpputest/include/. In your code (for example to start writing a unit test) you include the remaining path:
#include <CppUTest/TestHarness.h>
... 
Looking at this include line you instandly know that TestHarness.h belongs to CppUTest and (if you are familiar with the library) it is part of the core functionality (=not inside CppUTestExt).

This structuring approach gives you simple namespacing of your headers - which also your Java co-works will appriciate ;-)

Freitag, 28. November 2014

Quick C Tricks - Explicit Declaration Of String Terminator Byte

In C a string is an array of single characters terminated by the NUL-value ('\0'). I came across the following notation to clearly communicate this fact:
// declaring a 10 byte long string buffer which 
// will be later terminated by '\0'
char myString[10+1];
Of couse I could have written the above line like this:
// same thins as above but not that obvious
char myString[11];
It's a cosmetical thing - but they help to improve the readability of your code.

Sonntag, 2. November 2014

Reflections on SICP (Part 1) - Function dispatching in C

Yesterday I've committed the last work for the Scheme project of SICP. Work, namely a lot of mighty C coding is mainly responsible for not having finished the course yet. However, since I am in the last quarter of the course material I decided to recap from time to time - what are the things I've learned, what was good, what was surprising and so on.

One of my favorite learnings is the idea of a dispatch table. Instead of having a long list of if/elseif branches which direct to a certain function the idea is to have this dispatching being done in a hash table. Since the table is dynamic the dispatching can be changed at run time (late binding).

Although I appreciate the "get my quickly from A to B" qualities of Python most of my daily business is based on good old C and Java. In this post I want to sketch out how function dispatching could be implemented in C - as usual as high level as possible.

Here is a naive Python example of using a dispatch table. See the last code example of this post of a more Pythonic version. This following code will be the template of our transformation to C later:

#!/usr/bin/python3

def _sumAll(numberList):
    sum = 0
    for number in numberList:
        sum = sum + number
    
    return sum

def _sumEven(numberList):
    sum = 0
    for number in numberList:
        if number % 2 == 0:
            sum = sum + number

    return sum

def _multiplyAll(numberList):
    prod = 1
    for number in numberList:
        prod = prod * number
    
    return prod


dispatchTable = {
    'sumAll' : _sumAll,
    'sumEven' : _sumEven,
    'multiplyAll' : _multiplyAll
}

def accumulateList(method, numberList):
    return dispatchTable[method](numberList)

numbers = (2,4,5,6)

print(accumulateList('sumAll', numbers))
print(accumulateList('sumEven', numbers))
print(accumulateList('multiplyAll', numbers))

Dispatching takes place in accumulateList where the dictionary dispatchTable is being looked up for the entry method. The function returned  is directly executed, hence the numberList argument.

So let'S look at C. As a friend of high-level expressions I'm using as usual GLib data structures. GList for the number list and GHashtable for the dispatch table.
#include <stdio.h>
#include <glib-2.0/glib.h>

static GHashTable* dispatchTable;

int _sumAll(GList* numberList) {
    int sum = 0;
    GList *iter = numberList;

    while(iter != NULL) {
        sum = sum + GPOINTER_TO_INT(iter->data);
        iter = g_list_next(iter);
    }

    return sum;
}

int _sumEven(GList* numberList) {
    int sum = 0;
    GList *iter = numberList;

    while(iter != NULL) {
        int number = GPOINTER_TO_INT(iter->data);
        if (number % 2 == 0) {
            sum = sum + number;
        }
        iter = g_list_next(iter);
    }

    return sum;
}

int _multiplyAll(GList* numberList) {
    int prod = 1;
    GList *iter = numberList;

    while(iter != NULL) {
        prod = prod * GPOINTER_TO_INT(iter->data);
        iter = g_list_next(iter);
    }

    return prod;
}

void createDispatchTable() {
    dispatchTable = g_hash_table_new(g_str_hash, g_str_equal);

    g_hash_table_insert(dispatchTable, g_strdup("sumAll"), 
                        _sumAll);
    g_hash_table_insert(dispatchTable, g_strdup("sumEven"), 
                        _sumEven);
    g_hash_table_insert(dispatchTable, g_strdup("multiplyAll"), 
                        _multiplyAll);
}

int accumulateList(const char* method, GList* numberList) {
    int (*func)(GList*);
    func = g_hash_table_lookup(dispatchTable, method);

    return func(numberList);
}

void destroyDispatchTable() {
    g_hash_table_destroy(dispatchTable);
}

int main() {
    GList* numbers = NULL;
    createDispatchTable();

    numbers = g_list_append(numbers, GINT_TO_POINTER (2));
    numbers = g_list_append(numbers, GINT_TO_POINTER (4));
    numbers = g_list_append(numbers, GINT_TO_POINTER (5));
    numbers = g_list_append(numbers, GINT_TO_POINTER (6));

    printf("%i\n",accumulateList("sumAll", numbers));
    printf("%i\n",accumulateList("sumEven", numbers));
    printf("%i\n",accumulateList("multiplyAll", numbers));

    destroyDispatchTable();
    g_list_free(numbers);

    return 0;
}

The code structure is close to the Python example before. A remark for the strange GINT_TO_POINTER and GPOINTER_TO_INT macros: This is the GLib way of storing and retrieving integers in their data structures. For more details refer to the GLib manual.

createDispatchTable uses a GLib GHashtable to store string to function pointer pairs.

accumulateList is retrieving the right function pointer by looking up the dispatch table. As the next step the function is executed and it's result is directly returned.

Usually I would append a common prefix to the public functions createDispatchTable(),
accumulateLis()  and destroyDispatchTable() to indicate that they belong to one module (see here for details) but for this example I tried to keep it simple.

So what do we think about that. Well, of course we could add some macro magic to shrink certain function calls (like the g_list_append) but I tend to use the C language straight, falling back on the preprocessor only if I can't achieve my goals directly.

The reason for that is more related to (good) code style. A macro always adds a new level of indirection which needs to be resolved by a human brain when trying to understand the code.  Eclipse CDTs macro expansion feature is definitly helpful here but I rather live without personal syntactic sugar.

The matter is different little fellows like GINT_TO_POINTER and GPOINTER_TO_INT. They represent a feature which can't be accomplished by using the language straight - so their fine.

I personally would like to see the definition of the dispatch table somehow outside a function, more prominent and if I could choose defined by something like designated initializers.

The GHashtable approach does not support that so we would need to replace it by something like a struct (e.g. DispatchTableEntry) and an array of these DispatchTableEntry struct.  The accumulateList function would then iterate over this array of structs looking for the right entry.

I tried that approach but it has a couple of downsides. One thing is that for simple usage of designated initializers the dispatch table array ideally has a fixed size. My personal favorite, a NULL terminated array of unknown length would lead to a rather odd syntax. So there is a trade-off here.

Also, implementing my own lookup functionality in accumulateList isn't what I was looking for - I wanted to use this out of the shelf.

An other obvious approach would be to define enums for the function names and use them as an index of an ordinary C array where the value would be the according function pointer. The lookup would be just a simple array index access using the enum as well.

Beside the missing runtime flexibility (you can't add enums at runtime) this approach has the problem that you maintain data at two sections: a new dispatch route would mean an entry in enums (new function name) and an extra entry in the array (the actual dispatch entry).

To my knowledge the best solution for this double data maintenance issue are X macros whose philosophy dates back to the 1960s. When you're in a pure C environment (poor you) this would be my way to go.

After weighting my options I stuck with the C implementation above. To add a new dispatch route function createDispatchTable has to be extended. Everything else is using battle proof implementations (GLib functions) in favour of (once again) rolling my own stuff.

To summarize the C example: this is as good as it gets in (high-level) C.

To conclude I promised to provide a Pythonic version of our example:
#!/usr/bin/python3

from operator import add,mul
from functools import reduce

onlyEvenFilter = lambda x: x % 2 == 0

dispatchTable = {
    'sumAll' : lambda list: reduce(add, list),
    'sumEven' : lambda list: reduce(add, filter(onlyEvenFilter, list)),
    'multiplyAll' : lambda list: reduce(mul, list)
}

def accumulateList(method, numberList):
    return dispatchTable[method](numberList)

numbers = (2,4,5,6)

print(accumulateList('sumAll', numbers))
print(accumulateList('sumEven', numbers))
print(accumulateList('multiplyAll', numbers))

That is more then 80 line of code in C versus 21 in Python - the same functionality, expressed more clearer (in my opinion) in 1/4 of the code ;-)

Samstag, 15. März 2014

gcc's cleanup attribute instead of goto

While reading Dr. Dobbs article about Apples goto fail bug I stumbled across the gcc attribute cleanup as a potential replacement for the mighty goto.

Here is a quote from the gcc manual:

The cleanup attribute runs a function when the variable goes out of scope. This attribute can only be applied to auto function scope variables; it may not be applied to parameters or variables with static storage duration.

So instead of using goto to free resources locally, at declaration of a variable you also attach a cleanup function to it. As soon as the variable goes out of scope your cleanup code is being run.

If you search the internet for "gcc cleanup" you'll find many hits. My favorite one including a nice demonstration is here.

Nevertheless, for my current projects I won't make use of this feature. Mainly because I'm happy with the simple goto mechanism (as thousands of other C developers apperently are). Also, I want to keep the C code portable.

Samstag, 11. Januar 2014

Poor man's error handling with goto

I quite often deal with code which is build up on C functions which return the functions status as an integer. That way a parent function can check if its child function calls returned an error. The parent function then returns either "0" if everything ran well or otherwise any other number than "0". With this design you can build a call stack which gives the user (some logging provided) an idea where an error had happen.

This is a simple example:
// in some header:
#define OK 0
#define NOT_OK 1

int doCalculations()
{
    int rc = 0;

    rc = calcSomething();

    if ( rc != OK )
    {
        printf("function calcSomething had an error.\n");
        return NOT_OK;
    }

    rc = calcSomethingElse();

    if ( rc != OK )
    {
        printf("function calcSomethingElse had an error.\n");
        return NOT_OK;
    }

    return OK;
}
The problem here is that it is hard to follow the flow of the business logic since error handling is happening on the same stage.  To gain better readability I started to split the business logic and the error handling inside the function using goto:
// in some header:
#define OK 0
#define NOT_OK 1

int doCalculations()
{
    int rc = 0;

    rc = calcSomething();
    if ( rc != OK ) goto error_calcSomething;

    rc = calcSomethingElse();
    if ( rc != OK ) goto error_calcSomethingElse;

    return OK;


// error handling    
error_calcSomething:
    printf("function calcSomething had an error.\n");
    return NOT_OK;

error_calcSomethingElse:
    printf("function calcSomethingElse had an error.\n");
    return NOT_OK;
}
Now the upper part of the function deals mostly with the business logic (you can't get rid of the return code checking totally), the lower part is purly used for error handling. This technique is quite popular, I also spotted it inside the Linux kernel sources.

To my defense I need to state: I only use the evil goto for that one purpose: error handling and freeing of locally allocated heap data - nothing else.

By the way: An alternative way to print a backtrace is provided by the glibc with the backtrace function. However, this is not part of the ANSI C standard - if this is important for you.

Montag, 30. Dezember 2013

PuTTY Backend Handling - A Virtual Functions Example - Part 2/2

This is the second and the last part about my digging of the PuTTY sources. The first part introduced the Backend interface which is used to provide all applications of the PuTTY suite (PuTTY terminal GUI, pscp, plink...)  with an abstratction of the communication protocol (ssh, telnet...) choosen by the user.

After looking  at the way PuTTY is organizing the backends available, this post concludes with an example of how the Backend interface is used in application code.

Collecting Backends

 

The available backends are bundled in an array of Backends which is later accessed by convenience functions to select the right Backend for a connection.
// file BE_ALL_S.C (all backends):

Backend *backends[] = {
    &ssh_backend,
    &telnet_backend,
    &rlogin_backend,
    &raw_backend,
    &serial_backend,
    NULL
}
Not all applications in the PuTTY suite require all backends. Hence, there are similar files (modules) to the BE_ALL_S.C module above with just a shorter list of available backends.

The following module BE_SSH.C only contains the Backend for SSH (used in plink and psftp):
// file BE_SSH.C (used in pscp, psftp)

Backend *backends[] = {
    &ssh_backend,
    NULL
}
Applications which let you choose between different backends are mainly using the convenience functions backend_from_proto:
Backend *backend_from_proto(int proto)
{
    Backend **p;
    for (p = backends; *p != NULL; p++)
        if ((*p)->protocol == proto)
         return *p;
    return NULL;
}
This functions takes in a protocol identifier number and then tries to find the right backend struct in an array of available backends (see BE_ALL_S.C and BE_SSH.C ).
For that purpose it is comparing the protocol identifier number with the identifier number configured as second last member of the backend structs. In our examples RAW.C and SSH.C as well as in all other modules which implement a certain protocol, the id number is not coded directly but as more reader friendly constants (e.g. PROT_RAW and PROT_SSH).

A last remark to backend_from_proto: The way the function iterates through the backends is a nice example of pointer arithmetic: Iterator p is declared as the same type as the backends array (remember, arrays are pointers in C). By setting it to the value of the backends array, is pointing to the first element of that array. Incrementing p by one skips over to the next element of the backends array. Since the array is NULL terminated it is easy to check whether or not p has reached the end of the array.

Using The Backend


Now only the actual usage of the Backend interface inside the application code is missing. The next lines of code are snippets from windows.c, the PuTTY main program:
// window.c - example of using the flexible backend
// in application code

static Backend *back;

...

back = backend_from_proto(conf_get_int(conf, CONF_protocol));

...

error = back->init(NULL, &backhandle, conf,
                   conf_get_str(conf, CONF_host),
                   conf_get_int(conf, CONF_port),
                   &realhost,
                   conf_get_int(conf, CONF_tcp_nodelay),
                   conf_get_int(conf, CONF_tcp_keepalives));

back->provide_logctx(backhandle, logctx);

...

Line 3 declares variable back as a pointer to a Backend implementation (well, actually struct as we've seen before). Line 7 let the magic happen: it takes the users selection (ssh, telnet...) and loads via function conf_get_int ( user selection to protocol id number) and eventually function backend_from_proto the right implementation of the communication protocol (see above for details).
In line 11 and 18 contain two examples for working with the backend functions. As can be seen, this is totally independent from the users selection. back->init(...) on line 11 will be processed for an ssh connection as well as a telnet connection (and all other protocols available for user selection).

Here we harvest the fruits of our effort (can one say that in English ?! ;-)  ): The logic of the application is completely decoupled from the actual implementation of the communication protocols.

window.c above is part of the well known Windows version of PuTTY. There is also a lesser known Linux version. There, the implementation of the application (the GUI) part differs from the Windows implementation. However, the Windows and the Linux version of PuTTY share exactelly the same implementation of the communication protocols (our Backends).


Here, our digging of the PuTTY sources ends. I have to admit, the structure and implementation the PuTTY author(s) chose to implement the communication protocols really impressed me - and is now also part of my own arsenal ;-) .

Freitag, 27. Dezember 2013

PuTTY Backend Handling - A Virtual Functions Example - Part 1/2

In one of my past posts I already wrote about more general insights while studing the PuTTY sources. One ill baby and some Chrismas preparations later I'm glad to post two articles about my clues looking at the code of some really experienced guys (well, it is mostly Simon Tatham - the main author of PuTTY).

I particuarly liked the way different applications of the PuTTY suite  (the well known PuTTY SSH GUI is just one amongst others, there is also plink, pscp ...) are sharing the implementation for certain communication protocols. The next screenshot shows where to select those protocols in the PuTTY SSH GUI:

Choosing between different connection protocols in the PuTTY GUI
In this post and the next one I want to show you how nicely the PuTTY authors share the implementation of those communication protocols amongst different tools of the PuTTY suite.

Overview


Before looking at the code lets have a look at this little sketch: (made with Asciiflow)
The implementation of the communication protocols is shared amongst all applications of the PuTTY suite 

This overview should assist you while continue reading. As can be seen, the central header file PUTTY.H contains the definition of the Backend interface.

Interfaces


Short stop: The term interface is widely used in object oriented languages but not so often when it comes to C. However, books like "C Programming: A Modern Approach" or "Test Driven Development for Embedded C" are heavly promoting (particuarly the latter one) the concept of an interface ( a C header file) and its implementation ( a C module). It's nice to see that an old school language like C already contained everything required for this recommendable programming approach.

Back to PuTTY and its sources. From a technical side the Backend interface is a struct mainly consisting out of function pointers. The name of the function pointers are the ones which will be later implemented by the single backend modules (RAW.C, TELNET.C, ...)
// file PUTTY.H

// Defining the backend interface:
struct backend_tag {
    const char *(*init) (void *frontend_handle, void **backend_handle,
    Conf *conf, char *host, int port, char **realhost,
    int nodelay, int keepalive);
    void (*free) (void *handle);
    /* back->reconfig() passes in a replacement configuration. */
    void (*reconfig) (void *handle, Conf *conf);
    /* back->send() returns the current amount of buffered data. */
    int (*send) (void *handle, char *buf, int len);
    /* back->sendbuffer() does the same thing but without attempting a send */
    int (*sendbuffer) (void *handle);
    void (*size) (void *handle, int width, int height);
    void (*special) (void *handle, Telnet_Special code);
    const struct telnet_special *(*get_specials) (void *handle);
    int (*connected) (void *handle);
    int (*exitcode) (void *handle);
    /* If back->sendok() returns FALSE, data sent to it from the frontend
     * may be lost. */
    int (*sendok) (void *handle);
    int (*ldisc) (void *handle, int);
    void (*provide_ldisc) (void *handle, void *ldisc);
    void (*provide_logctx) (void *handle, void *logctx);
    /*
     * back->unthrottle() tells the back end that the front end
     * buffer is clearing.
     */
    void (*unthrottle) (void *handle, int);
    int (*cfg_info) (void *handle);
    char *name;
    int protocol;
    int default_port;
};

// create the "Backend" data type for struct backend_tag"
typedef struct backend_tag Backend;
typedef for a struct is usually combined with the definition of the struct tag but can ( like here ) of course be written separately.

Interface Implementations


Now that we've seen the interface of the Backend lets have a look inside one of the Backend implementations. For simplicity I've chosen module RAW.C.

At the header of the module the raw_backend struct ( I don't dare writting  "object" ;-) ) is declared and initialized with the function names which actually implement the interface. The functions themself are living in the same module file:
// RAW.C - declaring and initializing the raw_backend struct 
// with the names  of the modul's functions which 
// implement the interface in PUTTY.H plus some default values.
...
Backend raw_backend = {
    raw_init,
    raw_free,
    raw_reconfig,
    raw_send,
    raw_sendbuffer,
    raw_size,
    raw_special,
    raw_get_specials,
    raw_connected,
    raw_exitcode,
    raw_sendok,
    raw_ldisc,
    raw_provide_ldisc,
    raw_provide_logctx,
    raw_unthrottle,
    raw_cfg_info,
    "raw",
    PROT_RAW,
    0
}
...
After publishing the names of the functions, the actual implementation follows in the same file, for demonstration, I just give you the stubs of the first two, raw_init and raw_free.
// RAW.C - local module functions (static) containing 
// the implementation of the Backend interface.

static const char *raw_init(void *frontend_handle, void **backend_handle,
       Conf *conf,
       char *host, int port, char **realhost, int nodelay,
       int keepalive)
{
   ...
}


static void raw_free(void *handle)
{
   ...
}
You might share my excitement ;-) when we look at the implementation of the next protocol. This time we're brave and browse through SSH.C, the implementation of interface Backend for the SSH protocol.

First, again the declaration and initialization of the Backend struct:
// SSH.C - the implementation of interface "Backend" 
// for the SSH protocol

Backend ssh_backend = {
    ssh_init,
    ssh_free,
    ssh_reconfig,
    ssh_send,
    ssh_sendbuffer,
    ssh_size,
    ssh_special,
    ssh_get_specials,
    ssh_connected,
    ssh_return_exitcode,
    ssh_sendok,
    ssh_ldisc,
    ssh_provide_ldisc,
    ssh_provide_logctx,
    ssh_unthrottle,
    ssh_cfg_info,
    "ssh",
    PROT_SSH,
    22
}
Again, the actual functions containing the implementation reside in the same file (just the stubs of the first two):
// SSH.C - private functions implementing 
// the interface "Backend"

static const char *ssh_init(void *frontend_handle, void **backend_handle,
       Conf *conf, char *host, int port, char **realhost,
       int nodelay, int keepalive)
{
    ...
}

static void ssh_free(void *handle)
{
    ...
}
Since I'm just interested in the mechanism of interface vs. implementation in C, I omitted the content of the functions - but did you see how the two modules RAW.C and SSH.C are following the same pattern - specified by (again) the interface PUTTY.H.

Now that there is the implementation of the protocols we need to investigate how the actual applications are using them. This is something for part two of this little series.

Donnerstag, 28. November 2013

I've learned the hard way - TDD in C, 2nd round

A while ago I wrote about how to insert "spy" code in C. That is the process of overwriting an existing function definition at link time with an own implementation to basically write unit tests.

After practicing for a while I have to admit that this approach works but is unpredictable and tricky to set it up. Unpredictable since I found it hard to guess in which exact order the linker is resolving the needed symbols (in this case function symbols), tricky when you're trying to get legacy code with many, sometimes depending libraries under test (which I currently do).

After some recent frustration I gave the search engine an other go and ended up with my new favorite toy: the --wrap option of ld, the GNU linker.

Here is what the man page of ld tells about it:
--wrap=symbol
    Use a wrapper function for symbol.  Any undefined reference 
    to symbol will be resolved to "__wrap_symbol".  Any undefined
    reference to "__real_symbol" will be resolved to symbol.

    This can be used to provide a wrapper for a system function.  
    The wrapper function should be called "__wrap_symbol".  
    If it wishes to call the system function, it should 
    call "__real_symbol".

    Here is a trivial example:

            void *
            __wrap_malloc (size_t c)
            {
              printf ("malloc called with %zu\n", c);
              return __real_malloc (c);
            }

    If you link other code with this file using --wrap malloc, 
    then all calls to "malloc" will call the function "__wrap_malloc" 
    instead.  The call to "__real_malloc" in "__wrap_malloc" will call 
    the real "malloc" function.

    You may wish to provide a "__real_malloc" function as well, so that 
    links without the --wrap option will succeed.  If you do this, you 
    should not put the definition of "__real_malloc" in the same file 
    as "__wrap_malloc"; if you do, the assembler may resolve the call 
    before the linker has a chance to wrap it to "malloc".

This functionality can of course also be used to overwrite some function which stands in our way for unit testing which I'm going to show you now.

I'm using the same test source as last time. Here are our example files:
// file fav_music.h
void tellFavoriteMusic( void );


// file program.c
#include "fav_music.h"

int main() {
   tellFavoriteMusic(); 
}


// file fav_music_indie.c
#include <stdio.h>

void tellFavoriteMusic() {
    printf("I like Indie!!!\n");
}


// file fav_music_soul.c
#include <stdio.h>

void __wrap_tellFavoriteMusic() {
    printf("I like Soul!!!\n");
}

Notice that the name of function tellFavoriteMusic in fav_music_soul.c has been changed to __wrap_tellFavoriteMusic.

When the three files are compiled and linked together (for clarity I keep this as two separate commands although gcc could do it in one go) as expected tellFavoriteMusic in fav_music_indie.c is executed by main():
$ gcc -c program.c fav_music_indie.c fav_music_soul.c
$ ls *.o
fav_music_indie.o  fav_music_soul.o  program.o
$ gcc -o program.out fav_music_indie.o fav_music_soul.o program.o
$./program.out
I like Indie!!!
This is something we actually expected: all function names are unique and the one specified is executed.

Now let the magic happen and tell the linker that it should replace a function call to tellFavoriteMusic with a call to __wrap_tellFavoriteMusic:
$ gcc -Wl,--wrap=tellFavoriteMusic -o program.out fav_music_indie.o fav_music_soul.o program.o
$ ./program.out
I like Soul!!!
program.c still calls tellFavoriteMusic, but this time its replacement __wrap_tellFavoriteMusic is called.

A note to the parameters syntax -Wl,--wrap=tellFavoriteMusic. Since we're calling the linker ld via the GNU compiler we need to tell gcc to pass the --wrap parameter through to the linker. Here is what the gcc manual says:
-Wl,option
    Pass option as an option to the linker.  If option contains 
    commas, it is split into multiple options at the commas.  
    You can use this syntax to pass an argument to the option.  
    For example, -Wl,-Map,output.map passes -Map output.map 
    to the linker.  When using the GNU linker, you can also 
    get the same effect with -Wl,-Map=output.map.
Although it looks similar -Wl is different to the common warning switch -W...

Other approaches
While browsing the Internet I stumbled  across some other approaches for  replacing a function with an other one at link time. My favorite ones use the tool objcopy which is part of the binutils and is used to change object files (the result of a compiler run).

Mark function to overwrite as "weak"
Here the idea is to use the ----weaken-symbol parameter of objcopy on the object file of the original function. By marking the function to replace as weak the linker should prefer a second function implementation with the same name which hasn't got that flag. The multiple definition error should not appear.

Temporary remove the original function definition from the object file
With this approach you generate a temporary object file which does not contain the conflicting original implementation of the function. I assume objcopy and/ or objdump could do that.

I haven't tried the last two ideas myself since I'm happy with the --wrap parameter solution. However, some of you might find those thoughts helpful as a starting point of own tests.

Montag, 18. November 2013

Multi line strings in C

I'm still trying to find some time to finish my second article about the Putty sources (first one is here).

In the meantime I want to share a helpful little trick on how to handle multi line strings in C with you. See this example:
//multi line strings and multi line printf examples
#include <stdio.h>

int main( void ) {

    // multi line string definition
    char* string1 = "i am "
                    "a multiline string "
                    "which the compiler assembles "
                    "to one sting. No need for "
                    "concatination.";

    printf( "An ordinary string on one line: %s\n", string1 );


    // multi line string inside printf
    char* string2 = "a string";

    printf( "Multiline string works also "
            "inside a printf command. "
            "This is helpful if the string part of "
            "the command is too long to fit your maximum "
            "collumn size. Of course you can use format "
            "specifier like %s.\n", string2 );
}
The output is:
An ordinary string on one line: i am a multiline string which the compiler assembles to one sting. No need for concatination.
Multiline string works also inside a printf command. This is helpful if the string part of the command is too long to fit your maximum collumn size. Of course you can use format specifier like a string.
The examples are (hopefully) self-explainatory. The first one simply defines a multi line string which will be printed on one line. The second example demonstrates that the printf family of commands also understands the multi line syntax. Neat, innit?