Microsoft Catalog Files and Digital Signatures decoded

TL;DR: Parse and print .cat files: parsemscat

Introduction

Günther Deschner and myself are looking into the new Microsoft Printing Protocol [MS-PAR]. Printing always means you have to deal with drivers. Microsoft package-aware v3 print drivers and v4 print drivers contain Microsoft Catalog files.

A Catalog file (.cat) is a digitally-signed file. To be more precise it is a PKCS7 certificate with embedded data. Before I started to look into the problem understanding them I’ve searched the web, if someone already decoded them. I found a post by Richard Hughes: Building a better catalog file. Richard described some of the things we already discovered and some new details. It looks like he gave up when it came down to understand the embedded data and write an ASN.1 description for it. I started to decode the myth of Catalog files the last two weeks and created a tool for parsing them and printing what they contain, in human readable form.

Details

The embedded data in the PKCS7 signature of a Microsoft Catalog is a Certificate Trust List (CTL). Nikos Mavrogiannopoulos taught me ASN.1 and helped to create an ASN.1 description for the CTL. With this description I was able to start parsing Catalog files.

CATALOG {}
DEFINITIONS IMPLICIT TAGS ::=

BEGIN

-- CATALOG_NAME_VALUE
CatalogNameValue ::= SEQUENCE {
    name       BMPString, -- UCS2-BE
    flags      INTEGER,
    value      OCTET STRING -- UCS2-LE
}

...

END

mscat.asn

The PKCS7 part of the .cat-file is the signature for the CTL. Nikos implemented support to get the embedded raw data from the PKCS7 Signature with GnuTLS. It is also possible to verify the signature using GnuTLS now!
The CTL includes members and attributes. A member holds information about file name included in the driver package, OS attributes and often a hash for the content of the file name, either SHA1 or SHA256. I’ve written abstracted function so it is possible to create a library and a simple command line tool called dumpmscat.

Here is an example of the output:

CATALOG MEMBER COUNT=1
CATALOG MEMBER
  CHECKSUM: E5221540DC4B974F54DB4E390BFF4132399C8037

  FILE: sambap1000.inf, FLAGS=0x10010001
  OSATTR: 2:6.0,2:6.1,2:6.4, FLAGS=0x10010001
  MAC: SHA1, DIGEST: E5221540DC4B974F54DB4E39BFF4132399C8037

In addition the CTL has normally a list of attributes. In those attributes are normally OS Flags, Version information and Hardware IDs.

CATALOG ATTRIBUTE COUNT=2
  NAME=OS, FLAGS=0x10010001, VALUE=VistaX86,7X86,10X86
  NAME=HWID1, FLAGS=0x10010001, VALUE=usb\\vid_0ff0&pid_ff00&mi_01

Currently the projects only has a command line tool called: dumpmscat. And it can only print the CTL for now. I plan to add options to verify the signature, dump only parts etc. When this is done I will create a library so it can easily be consumed by other software. If someone is interested and wants to contribute. Something like signtool.exe would be nice to have.

A new cmocka release version 1.1.0

It took more than a year but finally Jakub and I released a new version of cmocka today. If you don’t know it yet, cmocka is a unit testing framework for C with support for mock objects!

We set the version number to 1.1.0 because we have some new features:

  • Support to catch multiple exceptions
  • Support to verify call ordering (for mocking)
  • Support to pass initial data to test cases
  • A will_return_maybe() function for ignoring mock returns
  • Subtests for groups using TAP output
  • Support to write multiple XML output files if you have several groups in a test
  • and improved documentation

We have some more features we are working on. I hope it will not take such a long time to release them.

Testing PAM modules and PAM-aware applications in the Matrix

Jakub Hrozek and I are proud to announce the first release of pam_wrapper. This tool allows you to either simplify testing PAM modules or your application using PAM to authenticate users. PAM (Pluggable Authentication Modules) is a layer of abstraction on top of Unix authentication.

For testing PAM-aware applications we have written a simple PAM module called pam_matrix. If you plan to test a PAM module you can use the pamtest library we have implemented. It simplifies testing of modules. You can combine it with the cmocka unit testing framework or you can use the provided Python bindings to write tests for your module in Python.

Jakub and I have written an article for LWN.net to provide more details how to use it. You can find it here.

Now start testing your PAM module or application!

libssh is running in the Matrix now

Since I joined the libssh project we started to write tests to find regression and make development easier. This has been achieved using the a unit testing framework called cmocka which I maintain and develop. The problem is that to run these tests you need to modify the sshd configuration and setup a test user so that the tests can be successfully executed. This is something contributors normally don’t do so we need to rely on our testing infrastructure.

In 2013 I’ve started the cwrap project. cwrap is a set of tool to make full network server/client testing made easy. These tools are used to make it possible to run the Samba Testsuite easily on every machine without setting anything up. Some time ago I’ve started to use cwrap for libssh testing. Finally I found the time to finish the task.

libssh in the Matrix

Now a libssh client tests sets up an artificial test environment. We have a passwd, shadow and group file so we can use two users to authenticate (nss_wrapper). sshd is running as the user starting the testcase but as it is part of the Matrix it thinks it is root (uid_wrapper). The client and server think they communicate on a real network (socket_wrapper) but it is again the Matrix!

It took me a while to get it working and I needed to implement new feature to the wrapper libraries of cwrap. socket_wrapper needed support to report TCP_NODELAY in getsockopt(). nss_wrapper needed shadow file support for password authentication so I had to add support for getspnam(). And as sshd is paranoid uid_wrapper needed checks if if is privileged to actually change to the user. After it drops privileged it checks if it really can’t go back.

With all of this implemented and new releases of the wrappers, which I’m preparing at the moment, all you have to do is to install cmocka, socket_wrapper, nss_wrapper and uid_wrapper and run ‘make test’. The Matrix will be created and libssh tested. You can find the cwrap libssh branch here.

There is one test for a feature missing right now. We do not test keyboard-interactive authentication, but the cwrap project is working on a new wrapper to fix this. Stay tuned!

Hunting down a fd closing bug in Samba

In Samba I had a failing test suite. I have nss_wrapper compiled with debug messages turned on, so it showed me the following line:

NWRAP_ERROR(23052) - nwrap_he_parse_line: 3 Invalid line[TDB]: 'DB'

The file should parse a hosts file like /etc/hosts, but the debug line showed that it tried to parse a TDB (Trivial Database) file, Samba database backend. I’ve started to investigate it and wondered what was going on. This morning I called Michael Adam and we looked into the issue together. It was obvious that something closed the file descriptor for the hosts file of nss_wrapper and it was by Samba to open other files. The big question was, what the heck closes the fd. As socket_wrapper was loaded and it wraps the open() and close() call we started to add debug to the socket_wrapper code.

So first we added debug statements to the open() and close() calls to see when the fd was opened and closed. After that we wanted to see a stacktrace at the close() call to see what is the code path were it happens. Here is the code how to do this:

commit 6c632a4419b6712f975db390145419b008442865
Author:     Andreas Schneider 
AuthorDate: Thu Mar 26 11:07:38 2015 +0100
Commit:     Andreas Schneider 
CommitDate: Thu Mar 26 11:07:59 2015 +0100

    DEBUG stacktrace
---
 src/socket_wrapper.c | 37 +++++++++++++++++++++++++++++++++----
 1 file changed, 33 insertions(+), 4 deletions(-)

diff --git a/src/socket_wrapper.c b/src/socket_wrapper.c
index 1188c4e..cb73cf2 100644
--- a/src/socket_wrapper.c
+++ b/src/socket_wrapper.c
@@ -80,6 +80,8 @@
 #include <rpc/rpc.h>
 #endif
 
+#include <execinfo.h>
+
 enum swrap_dbglvl_e {
 	SWRAP_LOG_ERROR = 0,
 	SWRAP_LOG_WARN,
@@ -303,8 +305,8 @@ static void swrap_log(enum swrap_dbglvl_e dbglvl,
 		switch (dbglvl) {
 			case SWRAP_LOG_ERROR:
 				fprintf(stderr,
-					"SWRAP_ERROR(%d) - %s: %s\n",
-					(int)getpid(), func, buffer);
+					"SWRAP_ERROR(ppid=%d,pid=%d) - %s: %s\n",
+					(int)getppid(), (int)getpid(), func, buffer);
 				break;
 			case SWRAP_LOG_WARN:
 				fprintf(stderr,
@@ -565,10 +567,35 @@ static int libc_bind(int sockfd,
 	return swrap.fns.libc_bind(sockfd, addr, addrlen);
 }
 
+#define BACKTRACE_STACK_SIZE 64
 static int libc_close(int fd)
 {
 	swrap_load_lib_function(SWRAP_LIBC, close);
 
+	if (fd == 21) {
+		void *backtrace_stack[BACKTRACE_STACK_SIZE];
+		size_t backtrace_size;
+		char **backtrace_strings;
+
+		SWRAP_LOG(SWRAP_LOG_ERROR, "fd=%d", fd);
+
+		backtrace_size = backtrace(backtrace_stack,BACKTRACE_STACK_SIZE);
+		backtrace_strings = backtrace_symbols(backtrace_stack, backtrace_size);
+
+		SWRAP_LOG(SWRAP_LOG_ERROR,
+			  "BACKTRACE %lu stackframes",
+			  (unsigned long)backtrace_size);
+
+		if (backtrace_strings) {
+			size_t i;
+
+			for (i = 0; i < backtrace_size; i++) {
+				SWRAP_LOG(SWRAP_LOG_ERROR,
+					" #%lu %s", i, backtrace_strings[i]);
+			}
+		}
+	}
+
 	return swrap.fns.libc_close(fd);
 }
 
@@ -704,6 +731,8 @@ static int libc_vopen(const char *pathname, int flags, va_list ap)
 
 	fd = swrap.fns.libc_open(pathname, flags, (mode_t)mode);
 
+	SWRAP_LOG(SWRAP_LOG_ERROR, "path=%s, fd=%d", pathname, fd);
+
 	return fd;
 }
 

We found out that the code responsible for this created a pipe() to communitcate with the child and then forked. The child called close() on the second pipe file descriptor. So when another fork happend in the child, the close() on the pipe file descriptor was called again and we closed a fd of the process to a tdb, connection or something like that. So initializing the pipe fd array with -1 and only calling close() if we have a file description which is not -1, fixed the problem.

If you need a better stacktrace you should use libunwind. However socket_wrapper can be a nice little helper to find bugs with file descriptors 😉

BUG: Samba standard process model closes random files when forking more than once

cmocka 1.0

At the beginning of February I attended devconf.cz in Brno and the days before I had a hack week with Jakub Hrozek on cmocka. cmocka is a unit testing framework for C with support for mock objects.

We already rewrote the test runner last year and it was time to finish it and add support for several different message output formats. You are able to switch between cmocka standard output, Subunit, Test Anything Protocol and jUnit XML reports. In addition we we have a skip() function and test_realloc() to detect buffer overflows and memory leaks now.

You can find all other required information on the overhauled shiny new website: http://cmocka.org

New uid_wrapper with full threading support.

Today I’ve released a new version of uid_wrapper (1.1.0) with full threading support. Robin Hack a colleague of mine spent a lot of time improving the code and writing tests for it. It now survives funny things like forking in a thread. We also added two missing functions and fixed several bugs. uid_wrapper is a tool to help you writing tests for your application.

If you don’t know uid_wrapper and wonder what you can do with it, here is an example:

$ id
uid=1000(asn) gid=100(users) groups=100(users),478(docker)
$ LD_PRELOAD=libuid_wrapper.so UID_WRAPPER=1 UID_WRAPPER_ROOT=1 id
uid=0(root) gid=0(root) groups=0(root)

More details about uid_wrapper can be found on the cwrap project website, here.

Taking your bike on a plane

Here is a totally computer unrelated post! Several people asked me how do I protect my bike to transport it safely on a plane. One possibility is to use a bike box, but the issue with a box is that airline personal likes big boxes, because they can use it to pile a lot of suitcases on it. I prefer to just wrap it with cardboard! Normally I go to a supermarket and ask if they have some spare cardboard for me. Normally they are happy to get rid of some. What you need bring from home in addition is a piece of rope, zip ties, duct tape, an old towel and a multitool or cutter.

I prepare everything at the supermarket. Cut the cardboard for the different pieces of the bike, put holes in the cardboard for the zip ties (first put duct tape on the cardboard then make the hole trough the duct tape and the cardboard!). Make sure you can still push the bike, the wheels should spin. In the end I have a small package like this:

Bike on a plane, cardboard collection

This small package is easy to transport. Either on the back of the bike or on your back 😉

At the airport you remove the pedals and fix the crank. Put the towel over the saddle and fix it with duct tape or a piece of rope. Tie a rope from the saddle to the handle bar so you can carry it. This makes it also easier for the airport personal to lift it. Then cover the bike with cardboard. Some parts are fixed on the bike with zip ties so they can’t move. In the end it normally looks like this:

Bike on a plane, protected with cardboard

If you’re on a bike trip you normally have 4 bike panniers with you, but the airline only allows to carry one luggage. Either the airport offers to wrap them or you go to a grocery store and buy plastfoil for food. It is very thin so you need 60m. It is not really Eco-friendly but normally you get it everywhere and it is cheap.

First start to wrap the two biggest panniers:

bike on a plane, panniers.

I use a rope to connect them and make sure not to loose one. After the two are in a good shape (ca. 25m) put the smaller panniers on the side and start to wrap the whole thing:

bike on a plane, wrapped panniers

Another possibility is to use 2 Ikea bags.

Have a safe trip!