a cosmological pedestrian
I’ve just released a new version of cmocka. cmocka is a unit testing framework for C with support for mock objects. Jakub and I finished the API documentation and added two new useful macros mock_type(#type) and mock_type_ptr(#type). The rest is some small bugfixes and a pkgconfig file.
You can download it here.
You’ve probably heard that a developer of an established software project writes an average of 100 lines of code (loc) a day. I can say that this applies to me. So if you write 100 loc per day, how many do you read? I would estimate that the amount of time you spend on reading and understanding code is significantly more than on writing code. You probably also spend quite some time debugging code.
If you spend so much more time on reading and debugging code than writing code, shouldn’t you put more effort in writing clean and debuggable code? The Samba codebase is pretty old, more than 15 years now. I would say we have some experience with bad code and we have started to write much better and cleaner code, because we have wasted so much time trying to understand and debug code. However there is still room for improvement. Lets take a look at the following C code snippet.
if (!a) {
return;
}
if (!b) {
return;
}
if (!c) {
return;
}
if (!*d) {
return;
}
Can you guess from the code above what types of variables a, b, c and d are? The answer is no? Ok, lets take a look at the following code:
if (a == NULL) {
return;
}
if (b == 0) {
return;
}
if (c == false) {
return;
}
if (d[0] == '\0') {
return;
}
If you look at the code now, you can probably guess what types they are. Well not exactly which type, but in which superset they are. ‘a’ is a pointer, ‘b’ is an integer type, ‘c’ is a bool and ‘d’ is a string (char array). If you write code the way shown above, you don’t have to scroll up to find out as which type the variable is defined. Most of the time it is enough to know what type of superset you are checking and why.
Think about this: If you spend just a bit more time on writing clean code now, you will spend less time on reading, understanding and debugging the code later if you have to find a bug.
Lets look at some more best practices we do in the Samba code:
bool ok;
int rc;
/*
* bool return codes should always have the name 'ok' or
* start with 'is_' or 'do_'
*/
ok = fn_returning_a_bool();
if (!ok) {
return;
}
/* We use rc or ret for an integer return code */
rc = do_something();
if (rc < 0) {
return;
}
You can see that we have variables for the return codes and check them with an if-clause. The reason is that it is easy on the eyes and in a debugger you can simply print the return code variable. If you write it like this: if (do_something() < 0) You have a hard time in the debugger to find out the actual return code. You have to step into and through the function to get it. We allow the !ok syntax for bool types, cause ok is by definition in our code a bool.
To be continued ...
I’m working on different Open Source projects and most of them have different coding style guidelines. Mostly spaces or tabs or different tabwidth. The easiest thing would be to store these information in the git config of the project. So here is a easy and secure way to have modelines in the git config.
So first I set the modelines (here for the Samba project):
git config --add vim.modeline "tabstop=8 shiftwidth=8 noexpandtab cindent"
Then copy this plugin into ~/.vim/plugin folder.
The modeline you defined in your git config will be appended to the :set command of vim. It only allows a limited set of set commands to be used. It will not execute any arbitrary code and you probably are the only person modifying the git config. You can change the allowed commands by adding the following to your ~/.vimrc file
let g:git_modelines_allowed_items = [
\ "textwidth", "tw",
\ "softtabstop", "sts",
\ "tabstop", "ts",
\ "shiftwidth", "sw",
\ "expandtab", "et", "noexpandtab", "noet",
\ "filetype", "ft",
\ "foldmethod", "fdm",
\ "readonly", "ro", "noreadonly", "noro",
\ "rightleft", "rl", "norightleft", "norl",
\ "cindent", "cin", "nocindent", "nocin",
\ "smartindent", "si", "nosmartindent", "nosi",
\ "autoindent", "ai", "noautoindent", "noai",
\ "spell",
\ "spelllang"
\ ]
UPDATE:
* New script which only allows a specified list
* Use sandbox command for set
* Added git repository
* Set only locally when reading the buffer
I’m a big fan of unit testing frameworks. When I developed csync, a bidirectional file synchronizer, I used check to write unit tests from the start. check was ok, but it were running valgrind on your testcases to find memleaks in your code the mode reports were about check. So I needed to add valgrind suppressions to get rid of them. When I started to work on libssh, a library implementing the SSH protocol, I wrote unit tests with check too. libssh is multi platform and also works on Windows and with Visual Studio. So we needed a new unit testing framework which is platform independent and has better code quality. I stumbled upon cmockery, a unit testing framework from Google. It was easy to use, the code looked good and it worked with Visual Studio. The build system sucked, so I added CMake support to produce a NSIS installer for Windows. I sent all my patches upstream but nothing happened. I fixed more bugs and added all patches people posted in their bug tracking system. I tried to talk with friends at Google, but in the end I needed to fork it.
cmocka is a fork and the successor of cmockery. I started to fix a lot of bugs, got all examples working and wrote API documentation with doxygen. The result is this first release version 0.2.0.
cmocka is a great unit testing framework with support for mock objects. Mock objects are simulated objects that mimic the behavior of real objects in a controlled way. Instead of calling the real objects, the tested object calls a mock object that merely asserts that the correct methods were called, with the expected parameters, in the correct order. It is really easy to write a unit test, take a look at the API an get started.
Example:
#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>
#include <cmocka.h>
/* A test case that does nothing and succeeds. */
static void null_test_success(void **state) {
(void) state; /* unused */
}
int main(void) {
const UnitTest tests[] = {
unit_test(null_test_success),
};
return run_tests(tests);
}
I recently fixed a bug resolving Domain Local groups in Winbind. I was asked how to reproduce it with a more complex setup, so I had to dig through the Winbind code to understand everything in more detail. I have documented my findings here, in order to retain what I’ve learned and to help others understand how Winbind works.
We have a forest with two AD domains: level1.discworld.site and level2.discworld.site. The two domains have a two way trust. User accounts are created on LEVEL1, groups and machine accounts are on LEVEL2. We have a Linux machine named ‘linux’, with Winbind joined to LEVEL2. I will describe everything from the perspective of Winbind, so LEVEL2 inside of Winbind is also referred to as ‘own domain’.
Users:
LEVEL1\ab
LEVEL1\asn
LEVEL1\gd
Groups:
LEVEL2\samba (members: LEVEL1\ab, LEVEL2\asn, LEVEL2\gd)
Machine Accounts:
LEVEL2\linux$
Lets assume we have successfully joined the machine ‘linux’ to LEVEL2 and then start Winbind. There is a parent Winbind process which delegates work to Winbind children. The parent forks a child for each logical domain, so in this setup there are 4 domain child processes: LEVEL1, LEVEL2, BUILTIN and SAMBA (local SAM). LEVEL1 and LEVEL2 will connect to their corresponding AD domain controllers.
If we want to obtain information about users or groups we have to query a Domain Controller. There are two ways to lookup this information. If the corresponding user is not logged in, then we queries for this information using the machine account. The machine account has limited permissions to query information, especially on Domain Controllers of trusted domains, so most of the time this information is incomplete, as we cannot provide more than what the AD domain controllers allow us to read. Often these queries are expensive, so caching is important to reduce the load on the domain controllers. Correct information about e.g. group memberships for a user is obtained when we authenticate as this user. The domain controller will then collect the information with the token of the user and send it to Winbind. In Winbind we cache this information. We have an issue here. If you get the information about a user with the machine account and cache it. Then authenticate as the user and get the
information again, we still return the information from the cache. Even if there is new or additional information, you will not see it until the cache expires and the authenticated user collects it again.
If Winbind authenticates a user there are normally two routes. It could do a normal samlogon or a samlogon with kerberos. If you want to authenticate a user using samlogon you can do this using ‘wbinfo -a
So if a login is initiated, the main Winbind process gets a pam authentication request. Depending to which domain the user belongs the auth is sent to the child handling the domain. So if LEVEL1+asn is trying to login, the Winbind child handling LEVEL1 will do a LogonSamLogon using the Netlogon PIPE to the domain controller. The domain controller is responsible for collecting all required information about the user and will return all information about group memberships in the info3 structure of the LogonSamLogon response.
If Kerberos is involved the Winbind child handling LEVEL1 will authenticate the user talking to the KDC of the domain controller. All information will be stored in the PAC (Privilege Attribute Certificate) of the Kerberos ticket (which is similar to the info3 structure in the LogonSamLogon response).
Information retrieval for ‘id’ or ‘getent’ will take a special route if we are only able to gather it using the machine account. The results collected with the machine account can differ from the results obtained during user authentication. Normally the information sent back from the Domain Controller during authentication is much more detailed and complete. It is possible the results differ between querying information as a user and as a machine account. With the limited resources of a machine account we can only try to get the basic information from the domain controller that the user is a member of. We will not contact trusted domains as enumeration is expensive and often not allowed with a machine account.
Lets look which functions will be called in Winbind if a user runs the command ‘id LEVEL1+asn’. For ‘id’ to be working we assume that nsswitch has been correctly configured to talk to Winbind. There is a libnss_winbind.so module which talks to the parent Winbind process over a unix pipe. The parent Winbind process handles all nsswitch function calls (POSIX functions) coming over the pipe asynchronously. We not discuss id mapping here, it will get too complex, we will just look on the flow of information.
‘id LEVEL1+asn’ will calls several POSIX functions which are sent over the UNIX pipe to the main Winbind process. These functions are getpwnam, getgrgid and getgroups. We assume that we have cold caches and need to handle these requests using machine account privileges.
The first thing ‘id’ calls is the getpwnam function. This will retrieve basic information about the user like the primary group id, the home directory and shell. The main Winbind process sends three queries to the LEVEL1 child for this. lookupname to get the SID of the user, a second lookupname to translate the SID to the username for verification, and finally a QueryUser call to get the basic information (primary gid, …). The first lookupname is a lsa_LookupNames call to the DC’s LSA server. The second is a LookupSids call to translate the SID to a name again. The QueryUser command is a LDAP query. Normally we always try to get the information with the fastest method and fall back to slower mechanisms if that fails.
After we collect all information and also store them in the cache (the child processes are responsible for the caches) we return the information to nsswitch. Now the ‘id’ command needs to get the name for the primary group and calls ‘getgrgid 1000000′ (1000000 being the gid of the primary group).
The parent Winbind will connect to the LEVEL1 domain child and call lookupname. LEVEL1 will then connect over RPC to the LSA pipe and call LookupSids3 to translate the SID to the name (the idmapping knows about the SID for the gid, the details are left out here).
As we have the important user information it is time to ask for additional group memberships of the user. This results in the following call:
The request is received by the main Winbind process, which needs to resolve the groups on three domains. It is always the same even if the machine is joined to a different domain the user is a member of.
a) The domain the user is a member of (LEVEL1)
b) The local SAM Authority (SAMBA)
c) The BUILTIN domain
We will need the SID of the user first so we ask the LEVEL1 child to resolve the name to a SID. Then for each of the domains we ask the corresponding child to do a LookupUserGroups. The LEVEL1 child will do a LDAP search to get a list of SIDs the user is a member of. Then it will talk to the DC LSA Server and call LookupSids3() to translate the SID into a name for each group. The information is sent back to the parent which will ask the local domain (SAMBA) if there are any aliases that the user is a member of. It will send a LookupUserAliases to the SAMBA child which will lookup the information using pdb. The final step is to talk to the BUILTIN domain for user aliases.
After all of the above POSIX calls were successful id will print the information it collected.
If you login as the user using kerberos first, then the information about the user are cached by the domain child serving the users domain. If you now call getpwnam then query will be filled with the information stored in the cache. Only the SID to name translation requires a LookupSids3 LSA call to the DC if it is not cached yet. The same for get getgrgid or getgroups call. We already got the information from the DC in the PAC which groups the user is a member of. We just need to translate the SIDs to names.
To be continued …
I’ve got a new gadget, a nice and small Android based smartphone, the HTC Wildfire S (WFS). The week before I got it alquez finished porting CyanogenMod 7 to the wfs. I’ve installed it and started to use it. After some time I was curios how to build the system. I’ve asked alquez how to set it up and I built it from source. Then I got interested in Android 4.0 and looked at CM9. After I managed to build it, it booted with the CM7 kernel and you could get a shell but that was it. So I’ve started to look into the Kernel and read CM9 code. Now after two weeks of work the device shows a UI. The questions if it will work in the end. Most of the stuff is Open Source but you rely on some binary libraries for OpenGL and maybe will not work out in the end. Android 4.0 relies on a lot of features of the 3.0 Kernel, new netfilter modules, updated graphics stuff etc.
If it will not work out in the end, at least I worked on the Kernel 
Three years ago Michael Adam created a nice library to easily read the Samba configuration or modify it if it is stored in the registry. Since we have a new build system it is much easier to create shared libraries, I’ve created a public smbconf library now. The library can be used to setup Samba or Winbind without touching any files. Ok, smb.conf needs one entry: config backend = registry. This library should be available with Samba 3.6 which will be released some time this year.
You can find the documentation for the new library here.
Maybe you’ve heard already of the privacy aware, personally controlled, open source social network Diaspora. I’ve wanted to try Diaspora so I’ve setup my own seed of Diaspora, they are called pods.
I don’t wanted to run Diaspora with thin so I decided to go with mod_passenger on apache2. I will describe what you need to do to set it up and get it running with mod_passenger.
…
# General setup for the virtual host
DocumentRoot "/path/to/diaspora/pod/public"
ServerName pod.example.com:443
ServerAdmin webmaster@example.com
ErrorLog /var/log/apache2/pod/error_log
TransferLog /var/log/apache2/pod/access_log
SetEnv RAILS_ENV production
# This enables mod_passenger
Include /etc/apache2/conf.d/mod_passenger.conf
<Directory "/path/to/diaspora/pod/public">
Options +FollowSymlinks -MultiViews
AllowOverride All
Order allow,deny
Allow from all
</Directory>
Feel free to ask questions, I will try to extend the howto.
…
It is possible to protect passenger with apparmor. You need to create the following wrapper:
passenger.c
#includeint main(int argc, char *argv[]) { return execv("/usr/bin/ruby", argv); }
I’ve compiled it with gcc -o passenger passenger.c and move it to /usr/local/bin/passenger. Then set the variable PassengerRuby “/usr/local/bin/passenger” and created an apparmor profile for it.
At the end of last year I’ve worked on Dogtag PKI, which is enterprise-class open source Certificate Authority.
The project is written in serveral different laguages (C, C++ and Java), so I’ve used CMake as the new build system. I know CMake very well and use it since a long time. To be able to build the java project I needed Java support in CMake. The current support is broken and after talking to Bill Hoffman he suggested to write CMake functions. So I’ve started to write functions to compile java files, find jar files, bundle jar files and generate javadoc.
My Java support for CMake should provide everything you need to build java and jni projects. You find the files here:
http://git.cryptomilk.org/projects/cmake-tools.git/tree/language/java
…
Find JNI libraries and headers. I think this is the file from CMake itself.
Find all needed Java tools like javac, javadoc, jar, etc.
This file provides all needed function to support creating java projects in CMake. Most of the function are documented in detail. There is some documentation missing. I will try to add it soon.
This is needed to find the class files in the build directory. One .java file can create multiple .class files. So you have to glob for these file. There is support to only look for certain .class files.
This is a helper to create symlinks for versioned jar files.
To checkout the tree use:
git clone git://git.cryptomilk.org/projects/cmake-tools.git
It would be nice if this would be included into the CMake distribution, but at
the moment there is still documentation missing. I hope that this will help
some people to get their Java project built with CMake.
Comments, suggestions and patches are welcome!
This work is upstream and will be available with CMake 2.8.6 (September 2011).