This is the tale of how I reverse-engineered a Motorola CPS radio protocol to make it work on Linux. While this may have been of questionable legality and thus lost interest in the project, I learned a lot on how to reverse engineer. I’m writing this entry more than a year after I initially did this, so I may be a little rusty on the details, but this is the gist of it.

My father worked in radio communications so when he passed I inherited his old EOL Motorola XTS 3000. I got an FCC ham radio license and wanted to utilize this device in service of my fledgling new radio hobby. Turns out, this device was the Rolls Royce of radios in its day. It can operate within the ham bands, can do encryption, digital communication, P.25, D-Star, trunking, and pumps out a very clear signal. In short, this was a heavy-duty mission-radio.

So I purchased a new $30 lithium ion battery, a programming cable, I ran into a few unfortunate roadblocks. First two…

This device cannot be front-face programmed, which is a fancy way of saying you cannot just set in an arbitrary frequency on the fly. Kinda sucks if you want to change to another random frequency. This could only be done by a proprietary Motorola CPS (Computer Programming Software) – but this was trivially easy to download.

…All of that was trivial compared to the next bomb-shell…

You could only program this device using a hardware serial port running on native 32-bit Windows. This means no Windows 7/8/10, no Virtual Machines, no Linux, no USB-to-Serial port.

Radio Reference users lamented that they were forced to maintain an old Windows XP laptop with a serial port for programming their device. I personally went out and purchased a $75 computer off Craigslist. Damn!

Up to this point, here are my thoughts: A serial RS-232 port is a “dumb” port compared to modern USB or PCI devices. In fact, serial does not even have a single standard, its whatever the device programmers decide. RS-232 is as close to raw data over a wire as you can get. So why doesn’t this work in a VM? And, why can’t I just capture this data, replay it and have the device function as normal?

Chipping Away at the Secret

I questioned the assumptions and put in an FTDI cross-over cable I made. One end went into the Windows machine, the other end went into My Linux machine, a final serial to radio cable connected to the device. This way my computer was essentially doing a Man-in-the-Middle (MitM) attack, but for serial.

I whipped up some C code to relay messages between the Windows machine and device. When I initialized the connection, the radio beeped! And then nothing happened…the software timed out and complained that the connection to the radio failed. I captured the initialization bytes 0x01,0x02,0x01,0x40,0xF7 and replaying them clearly made the radio do something, but immediately stopped afterwards.

I tried this process several times, but it failed. Annoyed, I looked into purchasing an RS-232 snooping cable: a cable with two regular connections that transfer data as normal and two tap ports, one that taps pins 2 and another that taps pin 3. For whatever reason, well-built cables cost upwards of $100 online and proper hardware snooping devices cost $500, way above my budget. So I decided it was much cheaper to build my own damn cable. I have the program I whipped up to read the bits from the transfer.

And it worked!

I saw a flurry of bits fly across the terminal. In the mix, I noticed a subtle pattern: A pause in the transfer, 7 bytes sent back, echoed back, another pause, and then another flurry of bits. For example, I saw:

0xF5,0x11,0x20,0x00,0x00,0x00,0xD9

Later on I saw:

0xF5,0x11,0x20,0x00,0x00,0x20,0xB9

and then

0xF5,0x11,0x20,0x00,0x00,0x40,0x99

If you didn’t catch it, the last two bytes are increasing by 0x20 (32) while the last bit ends in 9. (Spoiler, the repeating 9 was coincidental). I interpreted this as a value increasing by 32, and the last byte being a checksum. This was actually a lucky half-guess, because I had no way to know that.

Again I tried to replay these same bits, I ran into the same failure.

I briefly attempted to run the program in IdaPro, GNU Debugger for Windows and Immunity Debugger, but this approach failed and I am still not certain why. For example, I found a string that was clearly only utilized in one place in the binary and set an IdaPro breakpoint when the binary accessed that memory address. But for whatever reason, it did not break. Moreover, I learned the hard-way that the Win32 GUI API controls the application flow and is far from linear, so I could not just break after, say, 30,000 GUI instructions and expect to reliably step into the desired code.

I also briefly tried to use some RS-232 snooping tools, but every tool I found relied on a modern .NET framework not available on Windows XP. Moving on…

Win32 API Monitor

A Windows XP zealot on IRC informed me of a Win32 API monitoring application that would monitor DLL files and produce a list of when they were executed and their arguments. That might be useful.

I spent some time reading up on how Windows communicates over Serial: It uses CreateFileA() to open the COM1 port, typically listed as “\.\COM1” and then use ReadFileA() call, similar to Unix’s read(2) syscall. I expected to see this in the API capture.

Nope! Instead, I saw that the binary used CreateFileA() against Commsb9. Next, I saw ReadFileA(), sending over 0xF5,0x01,0x02,0x01,0x40 but not the trailing 0xF7. Win32 API even told me driver was communicating to it using USB IOCTLs — not only is this device serial, USB was barely invented when this program was created. What’s going on here?

Reading the code, I identified that these Read/Write commands were taking place in VcomSB96.dll, and filtering by that DLL file, I saw that it was loading Commsb96.sys and Commsbep.sys. In my experience sys files are always driver files.

The Driver

Looks like we are working with a driver. With my limited background in writing a Linux USB driver, Microsoft’s excellent documentation and websites like this, I had an idea of what I needed to hunt for. The C code would look like this:

NTSTATUS DriverEntry( IN PDRIVER_OBJECT pDriverObject,
IN PUNICODE_STRING pRegistryPath )
{
NTSTATUS ntStatus = 0;
UNICODE_STRING deviceNameUnicodeString, deviceSymLinkUnicodeString;
...
pDriverObject->DriverUnload = OnUnload;
pDriverObject->MajorFunction[IRP_MJ_CREATE] = Function_IRP_MJ_CREATE;
pDriverObject->MajorFunction[IRP_MJ_CLOSE] = Function_IRP_MJ_CLOSE;
pDriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = Function_IRP_DEVICE_CONTROL;

...
}

This snippet is assigning the driver methods to the pDriverObject struct. I opened up the Commsb96.sys in IdaPro, expecting to hunt through, starting from the obligatorily exported DriverEntry symbol and trace where the DeviceIoControl initializes. To my surprise I saw this:

Wow, that was easy. Turns out, a now defunct company called Vireo Software, Inc produced this driver in 1997 and failed to strip out the kernel symbols, making it much easier to reverse. Looks like they used C++ and compiled with optimization. That produced assembly that is a bit difficult to follow, but which I eventually traced back to where the DeviceIoControl messages landed in the kernel, and from there tracked down the USB message that Win32 API Monitor detected.

I finally traced code that read 5 bytes form the stack, and ran them through a loop, calculated a single byte, then returned that byte. I wish I could claim to have written the code below, but it was actually done by another IRC contact. He had a (probably bootleg) professional version of IdaPro that produced the following C code.

unsigned char sbCRC(const unsigned char *msgbuf, const int len) {
     const unsigned char table[8] = {}; // REDACTED FROM THE BLOG TO AVOID LEGAL TROUBLE!

     unsigned char a, b, n = 0;
     int i = 0;

     while (i < len) {
          n = (unsigned char)*(msgbuf + i) ^ n;
          a = ((unsigned char)((signed char)n >> 1) >> 1) ^ n;
          b = a;
          a = ((signed char)a << 1) & 0xF0;
          b = (signed char)b >> 1;
          if (b & 0x80)
               b = ~b;
          n = (a + (b & 0x0F)) ^ table[n & 0x07];
          i++;
     }

     return n;
}

I tested this function against known values such as 0xF5,0x11,0x20,0x00,0x00,0x00 that I cited before, ran it through the function and it resulted in 0xD9. Boom! Checksum reversed!

The Missing Link – RS-232 Flow Control

Coming of age in the 2000s, I learned about effectively full-duplex buses such as USB or PCI. In modern buses, you can effectively asynchronously send data but the RS-232 often requires manual flow-control by the CTS, DSR, RTS and DTR pins.

Providentially, around this time I found the amazing tool 232Analyzer, which was the only tool of its sort that did not require a modern .NET framework. Had I found it earlier, this would have saved me a lot of time! But I learned so much along this process.

Putting it all Together

With that, I modified my python code to emulate the flow control, calculate the checksum and sent the resulting bytes over. And this time….it worked! I could replay messages from original CPS and the radio responded with the same flutter of meaningless data.

Asking around on forums, IRC and reading up on the Motorola hardware, I learned that these sorts of devices do not request or set specific values from the radio as an API abstraction layer might do. Instead, you request memory locations and interpreting those locations according to a pre-known memory map. I deduced that the 0xF5,0x11 bytes meant “read”, the 0x20 is some sort of divider (or maybe more?), the next 3 bytes are memory locations, and the final byte is a checksum.

Armed with this hypothesis, I found the memory location of the device serial number and my code could read serial. I tested this on my second radio and it resulted it correctly retrieved the serial number.

To find other values, I recorded reading the radio memory, made a single change, then read the radio memory again and compared the difference to isolate values. I was able to find frequency values, channel names, signal strength values, etc! With time, I could mapped out the entire address space! I even found the write-prefix, but was too scared to test it in fears of bricking my radio.

Anti-Climactic Ending

Somewhere along this, I wondered “wait…is this legal?” I contacted the EFF. They were extremely eager to get back to me and after a long conversation the lawyer suspected that because the CPS has a copyright notice and I did not…um…come into acquisition of it through a financial transaction (sure, that phrasing works), it was likely illegal to distribute the reverse-engineered code.

And mapping out the memory got really tedious and annoying. And I started watching The Walking Dead.

…

And right about here my journey ended.

::cough::

But I learned so much!

1. How Windows device drivers work
2. Windows API calls (turns out, they don’t suck)
3. How to reverse engineer code with IdaPro
4. How RS-232 traffic flow works
5. A buncha tools!