Microsoft Catalog Files and Digital Signatures decoded

TL;DR: Parse and print .cat files: parsemscat

Introduction

GĂźnther Deschner and myself are looking into the new Microsoft Printing Protocol [MS-PAR]. Printing always means you have to deal with drivers. Microsoft package-aware v3 print drivers and v4 print drivers contain Microsoft Catalog files.

A Catalog file (.cat) is a digitally-signed file. To be more precise it is a PKCS7 certificate with embedded data. Before I started to look into the problem understanding them I’ve searched the web, if someone already decoded them. I found a post by Richard Hughes: Building a better catalog file. Richard described some of the things we already discovered and some new details. It looks like he gave up when it came down to understand the embedded data and write an ASN.1 description for it. I started to decode the myth of Catalog files the last two weeks and created a tool for parsing them and printing what they contain, in human readable form.

Details

The embedded data in the PKCS7 signature of a Microsoft Catalog is a Certificate Trust List (CTL). Nikos Mavrogiannopoulos taught me ASN.1 and helped to create an ASN.1 description for the CTL. With this description I was able to start parsing Catalog files.

CATALOG {}
DEFINITIONS IMPLICIT TAGS ::=

BEGIN

-- CATALOG_NAME_VALUE
CatalogNameValue ::= SEQUENCE {
    name       BMPString, -- UCS2-BE
    flags      INTEGER,
    value      OCTET STRING -- UCS2-LE
}

...

END

mscat.asn

The PKCS7 part of the .cat-file is the signature for the CTL. Nikos implemented support to get the embedded raw data from the PKCS7 Signature with GnuTLS. It is also possible to verify the signature using GnuTLS now!
The CTL includes members and attributes. A member holds information about file name included in the driver package, OS attributes and often a hash for the content of the file name, either SHA1 or SHA256. I’ve written abstracted function so it is possible to create a library and a simple command line tool called dumpmscat.

Here is an example of the output:

CATALOG MEMBER COUNT=1
CATALOG MEMBER
  CHECKSUM: E5221540DC4B974F54DB4E390BFF4132399C8037

  FILE: sambap1000.inf, FLAGS=0x10010001
  OSATTR: 2:6.0,2:6.1,2:6.4, FLAGS=0x10010001
  MAC: SHA1, DIGEST: E5221540DC4B974F54DB4E39BFF4132399C8037

In addition the CTL has normally a list of attributes. In those attributes are normally OS Flags, Version information and Hardware IDs.

CATALOG ATTRIBUTE COUNT=2
  NAME=OS, FLAGS=0x10010001, VALUE=VistaX86,7X86,10X86
  NAME=HWID1, FLAGS=0x10010001, VALUE=usb\\vid_0ff0&pid_ff00&mi_01

Currently the projects only has a command line tool called: dumpmscat. And it can only print the CTL for now. I plan to add options to verify the signature, dump only parts etc. When this is done I will create a library so it can easily be consumed by other software. If someone is interested and wants to contribute. Something like signtool.exe would be nice to have.

Hunting down a fd closing bug in Samba

In Samba I had a failing test suite. I have nss_wrapper compiled with debug messages turned on, so it showed me the following line:

NWRAP_ERROR(23052) - nwrap_he_parse_line: 3 Invalid line[TDB]: 'DB'

The file should parse a hosts file like /etc/hosts, but the debug line showed that it tried to parse a TDB (Trivial Database) file, Samba database backend. I’ve started to investigate it and wondered what was going on. This morning I called Michael Adam and we looked into the issue together. It was obvious that something closed the file descriptor for the hosts file of nss_wrapper and the samba binary opend a different file getting the same fd number assigned. The big question was, what the heck closes the fd. As socket_wrapper was loaded and it wraps the open() and close() call we started to add debug to the socket_wrapper code.

So first we added debug statements to the open() and close() calls to see when the fd was opened and closed. After that we wanted to see a stacktrace at the close() call to see what is the code path were it happens. Here is the code how to do this:

commit 6c632a4419b6712f975db390145419b008442865
Author:     Andreas Schneider < gro.a1496043378bmas@1496043378nsa1496043378>
AuthorDate: Thu Mar 26 11:07:38 2015 +0100
Commit:     Andreas Schneider < gro.a1496043378bmas@1496043378nsa1496043378>
CommitDate: Thu Mar 26 11:07:59 2015 +0100

    DEBUG stacktrace
---
 src/socket_wrapper.c | 37 +++++++++++++++++++++++++++++++++----
 1 file changed, 33 insertions(+), 4 deletions(-)

diff --git a/src/socket_wrapper.c b/src/socket_wrapper.c
index 1188c4e..cb73cf2 100644
--- a/src/socket_wrapper.c
+++ b/src/socket_wrapper.c
@@ -80,6 +80,8 @@
 #include <rpc/rpc.h>
 #endif
 
+#include <execinfo.h>
+
 enum swrap_dbglvl_e {
 	SWRAP_LOG_ERROR = 0,
 	SWRAP_LOG_WARN,
@@ -303,8 +305,8 @@ static void swrap_log(enum swrap_dbglvl_e dbglvl,
 		switch (dbglvl) {
 			case SWRAP_LOG_ERROR:
 				fprintf(stderr,
-					"SWRAP_ERROR(%d) - %s: %s\n",
-					(int)getpid(), func, buffer);
+					"SWRAP_ERROR(ppid=%d,pid=%d) - %s: %s\n",
+					(int)getppid(), (int)getpid(), func, buffer);
 				break;
 			case SWRAP_LOG_WARN:
 				fprintf(stderr,
@@ -565,10 +567,35 @@ static int libc_bind(int sockfd,
 	return swrap.fns.libc_bind(sockfd, addr, addrlen);
 }
 
+#define BACKTRACE_STACK_SIZE 64
 static int libc_close(int fd)
 {
 	swrap_load_lib_function(SWRAP_LIBC, close);
 
+	if (fd == 21) {
+		void *backtrace_stack[BACKTRACE_STACK_SIZE];
+		size_t backtrace_size;
+		char **backtrace_strings;
+
+		SWRAP_LOG(SWRAP_LOG_ERROR, "fd=%d", fd);
+
+		backtrace_size = backtrace(backtrace_stack,BACKTRACE_STACK_SIZE);
+		backtrace_strings = backtrace_symbols(backtrace_stack, backtrace_size);
+
+		SWRAP_LOG(SWRAP_LOG_ERROR,
+			  "BACKTRACE %lu stackframes",
+			  (unsigned long)backtrace_size);
+
+		if (backtrace_strings) {
+			size_t i;
+
+			for (i = 0; i < backtrace_size; i++) {
+				SWRAP_LOG(SWRAP_LOG_ERROR,
+					" #%lu %s", i, backtrace_strings[i]);
+			}
+		}
+	}
+
 	return swrap.fns.libc_close(fd);
 }
 
@@ -704,6 +731,8 @@ static int libc_vopen(const char *pathname, int flags, va_list ap)
 
 	fd = swrap.fns.libc_open(pathname, flags, (mode_t)mode);
 
+	SWRAP_LOG(SWRAP_LOG_ERROR, "path=%s, fd=%d", pathname, fd);
+
 	return fd;
 }
 

We found out that the code responsible for this created a pipe() to communitcate with the child and then forked. The child called close() on the second pipe file descriptor. So when another fork happend in the child, the close() on the pipe file descriptor was called again and we closed a fd of the process to a tdb, connection or something like that. So initializing the pipe fd array with -1 and only calling close() if we have a file description which is not -1, fixed the problem.

If you need a better stacktrace you should use libunwind. However socket_wrapper can be a nice little helper to find bugs with file descriptors 😉

BUG: Samba standard process model closes random files when forking more than once

New uid_wrapper with full threading support.

Today I’ve released a new version of uid_wrapper (1.1.0) with full threading support. Robin Hack a colleague of mine spent a lot of time improving the code and writing tests for it. It now survives funny things like forking in a thread. We also added two missing functions and fixed several bugs. uid_wrapper is a tool to help you writing tests for your application.

If you don’t know uid_wrapper and wonder what you can do with it, here is an example:

$ id
uid=1000(asn) gid=100(users) groups=100(users),478(docker)
$ LD_PRELOAD=libuid_wrapper.so UID_WRAPPER=1 UID_WRAPPER_ROOT=1 id
uid=0(root) gid=0(root) groups=0(root)

More details about uid_wrapper can be found on the cwrap project website, here.

resolv_wrapper 1.0.0 – the new cwrap tool

I’ve released a new preloadable wrapper named resolv_wrapper which can be used for nameserver redirection or DNS response faking. It can be used in testing environment to route DNS queries to a real nameserver separate from resolv.conf or fake one with simple config file. We tested it on Linux, FreeBSD and Solaris. It should work on other UNIX flavors too.

resolv_wrapper

You can download resolv_wrapper here.

A talk about cwrap at LinuxCon Europe

Next week is the LinuxCon Europe in DĂźsseldorf, Germany. I will be there and give a talk about cwrap, a set of tools to make client/server testing easy on a single machine. Testing network applications correctly is hard. This talk will demonstrate how to create a fully isolated network environment for client and server testing on a single host, complete with synthetic account information, hostname resolution, and privilege separation.

I hope you will attend my talk if you are there. If you can’t attend the LinuxCon Europe, but you’re going to the Linux Plumbers Conference then say hello and lets talk about cwrap there!

At the LinuxCon Europe I will announce new cool stuff and the website will be updated. So you should check

http://cwrap.org/

next week!

cwrap talk

How to get real DNS resolving in ‘make test’?

As you might know I’m working (hacking) on Samba. Samba has a DNS implementation to easier integrate all the AD features. The problem is we would like to talk to the DNS server but /etc/resolv.conf points to a nameserver so your machine is correctly working in your network environment. For this Samba in our dns resolver library we implemented a way to setup a dns_hosts_file to fake DNS queries. This works well for binaries provided by Samba but not for 3rd party application. As GĂźnther Deschner and I are currently working on MIT Kerberos support the libkrb5 library always complained that it is not able to talk query the DNS server to find the KDC. So it was time to really fix this!

I’ve sat down and did some research how we get this working. After digging through the glibc code, first I thought we could redirect the fopen(“/etc/resolv.conf”) call. Well as this is called in a glibc internal function it directly calls _IO_fopen() which isn’t weak symbol. So I looked deeper and recognized that I have access to the resovler structure which holds the information to the nameserver. I could simply modify this!

It was time to implement another wrapper, resolv_wrapper. Currently it only wraps the functions required by Samba and MIT Kerberos, res_(n)init(), res_(n)close, res_(n)query and res_(n)search. With this I was able to run kinit which asks the DNS server for a SRV record to find the KDC and it worked. With Jakub Hrozek I cleaned up the code yesterday and we created a parser for a resolv.conf file.

Here is a tcpdump of the kinit tool talking to the DNS server with socket_wrapper over IPv6.

resolv_wrapper will be available on cwrap.org soon!

What is preloading?

by Jakub Hrozek and Andreas Schneider

The LD_PRELOAD trick!

Preloading is a feature of the dynamic linker (ld). It is a available on most Unix system and allows to load a user specified, shared library before all other shared libraries which are linked to an executable.

Library pre-loading is most commonly used when you need a custom version of a library function to be called. You might want to implement your own malloc(3) and free(3) functions that would perform a rudimentary leak checking or memory access control for example, or you might want to extend the I/O calls to dump data when reverse engineering a binary blob. In this case, the library to be preloaded would implement the functions you want to override with prelinking. Only functions of dynamically loaded libraries can be overridden. You’re not able to override a function the application implements by itself or links statically with.

The library to preload is defined by the environment variable LD_PRELOAD, such as LD_PRELOAD=libwurst.so. The symbols of the preloaded library are bound first, before other linked shared libraries.
Lets look into symbol binding in more details. If your application calls a function, then the linker looks if it is available in the application itself first. If the symbol is not found, the linker checks all preloaded libraries and only then all the libraries which have been linked to your application. The shared libraries are searched in the order which has been given during compilation and linking. You can find out the linking order by calling 'ldd /path/to/my/applicaton'. If you’re interested how the linker is searching for the symbols it needs or if you want do debug if the symbol of your preloaded library is used correctly, you can do that by enabling tracing in the linker.

A simple example would be 'LD_DEBUG=symbols ls'. You can find more details about debugging with the linker in the manpage: 'man ld.so'.

Example:

Your application uses the function open(2).

  • Your application doesn’t implement it.
  • LD_PRELOAD=libcwrap.so provides open(2).
  • The linked libc.so provides open(2).

=> The open(2) symbol from libcwrap.so gets bound!

The wrappers used for creating complex testing environments of the cwrap project use preloading to supply their own variants of several system or library calls suitable for unit testing of networked software or privilege separation. For example, one wrapper includes its version of most of the standard API used to communicate over sockets that routes the communication over local sockets.

New features in socket_wrapper 1.1.0

Maybe you already heard of the cwrap project. A set of tools to create a fully isolated network environment to test client/server components on a single host. socket_wrapper is a part of cwrap and I released version 1.1.0 today. In this release I worked together with Michael Adam and we implemented some nice new features like support for IP_PKTINFO for binding on UDP sockets, bindresvport() and more socket options via getsockopt(). This was mostly needed to be able to create a test environment for MIT Kerberos.

The upcoming features for the next version are support for passing file description between processes using a unix domain socket and sendmsg()/recvmsg() (SCM_RIGHTS). We would also like to make socket_wrapper thread-safe.