Simple POSIX Semaphore Library for Windows

dandenongs 1889

I had a recent need to control access of one of my programs to a hardware device – basically the hardware can only handle a certain number of simultaneous connections before it starts returning garbage. POSIX semaphores are a really elegant way of handling this as you just create a semaphore and then put a semi_wait call before every critical code section and a sem_post call at the end of the critical code. This way you can control the number of simultaneous calls of the critical code no matter how many parallel processes or threads are running.

The only problem with POSIX semaphores is they are not natively supported on Windows (Windows has a entirely different semaphore system). Since my program is cross-platform I wanted a simple Windows-compatible POSIX semaphore library I could just drop in on my Windows builds and avoid a whole series of #ifdef #else conditionals. I managed to find a full Windows POSIX thread library (libpthread), but it was way more complex than what I needed and it does not build on Windows XP (I unfortunately still need to support Windows XP/2003). I was able to use the library as a starting point for making a very simple POSIX semaphore library. My cut-down library provides a simple drop in replacement for all the POSIX semaphore functions on Windows and just requires the inclusion of the header file in any project you might want to use it in. It is not identical to semaphores on linux, but it does the job.

You can download the simple windows semaphore library from my github repository. I hope it is of help for someone else.

Tracking down global variables in C code

I recently had the fun job of making some complex legacy C code (200,000+ lines) thread safe. Of course there were globals spread through the code so finding them all turned into a massive easter egg hunt. The initial problem was how to track them all down. If you ever have this fun job here are the steps I used.

1. I searched for the word “global”. If the person who wrote the code took care then all the global variables will have been commented as such (ha ha). Maybe surprisingly this did managed to find around 50% of the globals – while not perfect, it was a good start. I also found a few more unlabelled globals by searching for “thread-safe” and “thread” since some of the non-thread safe code was actually commented as being not thread-safe (I wish it all had been).

2. I next opened the .map file (generated by Visual Studio in debug mode) and looked for any “common” class variables (these are found down towards the bottom of the .map file). This process managed to shake out another 40% of the globals.

3. The last 5% of globals I found by doing a manual search through the entire code base for the “static” keyword. I of course then had to then trawl through several thousand static function declarations, but the last of the globals were in here too.

The most interesting thing about this painful exercise is how often people use globals when they are not really needed. 90% of the globals were being used only two or three times and were able to be made thread safe with very little code change. Of course the last 10% were used in more than 100 locations spread throughout the entire code base so the whole thread safe conversion exercise was not easy.

To keep the required code change to a minimum (I am very reluctant to make major structural changes to complex code even if I have written it) I solved the problem by just packaging up the difficult globals into a thread-safe strut declared in the main entry function and passed the pointer to this strut into each of the functions that were using the globals. While not a trivial job, this approach avoided disturbing code I knew was working well and which would have taken months to re-write.

Letters to avoid in creating passwords for non-english keyboard layouts

I had an interesting issue arise where I sent a computer to a customer located in Germany. I had created a password for them, but they had problems logging in with it. It turns out that the password I had created contained the letters Z and Y. These two keys are swapped on German keyboards so when they plugged in their own German keyboard they ended up entering the wrong password as the computer was still set to the English keyboard layout.

I have looked into the most common keyboard layouts used in Europe (QWERTY, QWERTZ & AZERTY) and you can avoid key swapping problems like mine if you avoid using the letters Z, Y, A, Q, M & L in any password or user name.

Update. Here is a simple one liner to generate a random alpha numerical password of 8 characters that avoids these letters. You can of course change the password length by changing the -c switch on the head call. This should work on any *nix based system (including MacOS X) that has openssl installed. One warning is the password will visible at generation time to other local users on shared systems if the other users are watching for process list changes. This doesn’t matter to me too much as I generate the passwords on my Mac laptop, but it is something to keep in mind if you are on an shared system.

 openssl rand -base64 25 | tr -dc 'BCDEFGHIJKNOPRSTUVWXbcdefghijknoprstuvwx0123456789' | head -c8; echo ""

How to create a Windows program that works both as a GUI and console application

It would be very nice to be able to make Windows applications that can act as either a GUI or console application depending on how they are used (i.e. act as a GUI application if double clicked in Windows Explorer or as a console application if called from a cmd.exe window).

Unfortunately the way Windows work, each exe application has field in the PE header that specifies which subsystem it should run under. This is set in Visual Studio by using one of the linker SUBSYSTEM option (i.e. Windows or Console). The subsystem is used by the Windows kernel to set up the execution environment for the application. If the application is built using SUBSYSTEM/Console then the kernel will connect the application to the parent console and the application’s stdout, stderr and stdin will be redirected to the parent console. If the program is built as a GUI application then the application detaches from the parent console and all output to stdout and stderr are are lost – basically the program runs, but doesn’t output anything to the parent console window.

People have attempted various hacks over the years to solve this problem. One solution proposed was to compile the program as a Windows application and then edit the PE header to mark the program as using the Console subsystem. The downside of this approach is it flashes a Console window on the screen when run as a GUI application that looks pretty unprofessional. The second hack commonly used is to create two separate binaries, for example, a myuselessprogram.com and myuselessprogram.exe. The .com version is built as a Console application while the .exe is built as Windows application. When you run myuselessprogram the Windows probing rule runs the .com version first and if there is nothing on the command line then the .com version calls the .exe Windows version. While both these approaches work, they are to say less than ideal.

A better approach is use the WINAPI AttachConsole function to attach the application to the parent console and then redirect stdout, stdin and stderr back to the parent console. This actually works very well except that when the application exits the parent console can’t detect this and hence release the command prompt. The end result is the parent console just sits there until the user presses the “enter” key.

There is no really elegant solution to this problem, but as applications can simulate the keyboard being used, a simple solution is to call the SendInput API function with the “enter” key just before the application exits. This simulates the user pressing the enter key and hence releases the command prompt.

To show how this approach works I have written a small test application (see below). The main limitations is that AttachConsole is only available on Windows XP and above. It does works under Cygwin which is nice.

Update. I have added a check to make sure that the console window is in focus before sending the enter key. It is a good check to make if you are running your console program in a background script or else you will end up with lots of “enter” key presses in whatever application you actually have in focus – lots of fun if you are working on the documentation while a script runs in the background :)

Update 2. Contrary to what has been posted on Stack Overflow this approach works with STDIN too. I don’t have any need to capture STDIN with my application (just the command line parameters), but all you need to do to capture stdin is treat STDIN as is done for STDOUT (i.e just redirect STDIN to the console) in the attachOutputToConsole function.

Update 3. This is MIT licensed if this is important to you :)

Update 4. Microsoft has broken the old approach I was using in VS2015. I have updated the code to use a different way of attaching STDOUT and STRERR to the parent console. This code works with VS2015 and all earlier compliers I was able to test (VS2013, VS2008, VS2005).

/*

 Copyright (c) 2013, 2016 Daniel Tillett
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:
 
 1. Redistributions of source code must retain the above copyright notice, this
 list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright notice,
 this list of conditions and the following disclaimer in the documentation
 and/or other materials provided with the distribution.
 
 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
 ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

*/

#define WINVER 0x0501 // Allow use of features specific to Windows XP or later.
#define _WIN32_WINNT 0x0501
#define WIN32_LEAN_AND_MEAN
#include "windows.h"
#include "io.h"
#include "fcntl.h"
#include "stdio.h"
#include "stdlib.h"
#pragma comment(lib, "User32.lib")

// Attach output of application to parent console
static BOOL attachOutputToConsole(void) {
HANDLE consoleHandleOut, consoleHandleError;

if (AttachConsole(ATTACH_PARENT_PROCESS)) {
  // Redirect unbuffered STDOUT to the console
  consoleHandleOut = GetStdHandle(STD_OUTPUT_HANDLE);
  if (consoleHandleOut != INVALID_HANDLE_VALUE) {
    freopen("CONOUT$", "w", stdout);
    setvbuf(stdout, NULL, _IONBF, 0);
    }
  else {
    return FALSE;
    }
  // Redirect unbuffered STDERR to the console
  consoleHandleError = GetStdHandle(STD_ERROR_HANDLE);
  if (consoleHandleError != INVALID_HANDLE_VALUE) {
    freopen("CONOUT$", "w", stderr);
    setvbuf(stderr, NULL, _IONBF, 0);
    }
  else {
    return FALSE;
   }
  return TRUE;
  }
//Not a console application
return FALSE;
}

// Send the "enter" to the console to release the command prompt 
// on the parent console
static void sendEnterKey(void) {
 INPUT ip;
 // Set up a generic keyboard event.
 ip.type = INPUT_KEYBOARD;
 ip.ki.wScan = 0; // hardware scan code for key
 ip.ki.time = 0;
 ip.ki.dwExtraInfo = 0;

 // Send the "Enter" key
 ip.ki.wVk = 0x0D; // virtual-key code for the "Enter" key
 ip.ki.dwFlags = 0; // 0 for key press
 SendInput(1, &ip, sizeof(INPUT));

 // Release the "Enter" key
 ip.ki.dwFlags = KEYEVENTF_KEYUP; // KEYEVENTF_KEYUP for key release
 SendInput(1, &ip, sizeof(INPUT));
}

int WINAPI WinMain(HINSTANCE hInstance, 
HINSTANCE hPrevInstance, 
PSTR lpCmdLine, 
INT nCmdShow) {
int argc = __argc;
char **argv = __argv;
UNREFERENCED_PARAMETER(hInstance);
UNREFERENCED_PARAMETER(hPrevInstance);
UNREFERENCED_PARAMETER(lpCmdLine);
UNREFERENCED_PARAMETER(nCmdShow);
BOOL console;
int i;

//Is the program running as console or GUI application
console = attachOutputToConsole();

if (console) {
    // Print to stdout
    printf("Program running as console application\n");
    for (i = 0; i < argc; i++) {
         printf("argv[%d] %s\n", i, argv[i]);
         }

    // Print to stderr
    fprintf(stderr, "Output to stderr\n");
    }
else {
    MessageBox(NULL, "Program running as Windows GUI application",
    "Windows GUI Application", MB_OK | MB_SETFOREGROUND);
}

// Send "enter" to release application from the console
// This is a hack, but if not used the console doesn't know the application has
// returned. The "enter" key only sent if the console window is in focus.
if (console && (GetConsoleWindow() == GetForegroundWindow())){
    sendEnterKey();
    }
return 0;
}

Solved: Problem with Skype number always engaged (busy)

I decided to get a skype number as a replacement for my landline number. I found that no matter what skype setting I used on my mac laptop the phone number was always engaged (busy). It turns out that the problem is due to the fact that I also had skype installed on my windows box. It seems that if you are logged in on two different machines skype will always take the settings from windows over osx. Any changes you make on the osx side are overridden by whatever settings you have on the windows box. As soon as I changed the settings on the windows box everything started working.

Now all I need to do is figure out how to get the 2 hours of my life back that I wasted on working this out :)

Apple Bootcamp 2.1 and Windows XP SP3

There is a problem with Bootcamp 2.1 in that you can’t install the Apple drivers under Windows XP SP3. I recently bought a new MacBook Pro and installed Windows XP SP3 from an image and then tried to update the XP hardware drivers using bootcamp 2.1. This didn’t work an instead gave a nice cryptic error 2762. It seems it is not possible to install the drivers manually either as windows can’t install the drivers for the wireless network, iSight camera, and trackpad.

The solution is to install XP from a SP2 image (luckily I had kept one), then install the drivers, and then upgrade to SP3. I hope this saves someone a bit of time.

Update. This has apparently been fixed in snow leopard. I guess another reason to update.