C++ code to dump memory in hex and ASCII

Every now and then I have the need to dump a block of memory as hex and ASCII in some C++ code I am working on. Each time I google for some code I can snaffle to do the trick. Each time I find something of very mediocre quality which will just about do for the occasion. Well, I finally got sick and tired of this and now I am publishing my own solution. I hope that people find it useful.

Many thanks to ECI training for this YouTube video on how to do file attachments in blogger.

TrueCrypt has gone! OMG!!!

My hard drive failed so this weekend I started to migrate my stuff over to a new machine which didn't have much stuff set up on it. One of the missing components was truecrypt so I thought I would just download and build from source. What a shock awaited me - truecrypt has gone! See the wikipedia page that describes how it went on 29th May 2014. There was also an article in the Guardian. I am writing this on Sunday 1st June 2014. This was not how I wanted to spend my Sunday!

Conspiracy theories to one side, the practical question remains, what does one do if one wishes to continue using truecrypt as it was? The answer seems to be to build from the source of the 7.1a. Download it from the final release repository. However, there is more to it than that. It doesn't build cleanly. I found someone else who was trying to do what I wanted to do, Reinhard Seiler. He blogged about his build experience. However, this was on a raspberry Pi and I had some different problems. Here's what I found:

• The build requires nasm, yasm won't do. No problem, I installed it via synaptic package manager.
• SecurityToken.cpp failed to compile due to missing PKCS11 header files. I followed Reinhard Seiler's instructions, placing the headers from ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211 into a sub-directory of my truecrypt source. This is so I could copy the entire directory if this ever happens to me again (i.e complete install on new machine needed).
• I got compilation errors due to missing macros such as CKR_NEW_PIN_MODE. Luckily, I found a blogger who had hit the same problem and posted a solution. Basically you ifdef out the offending lines. It is safe to do this since it is only error message handling.
• Once it got past the PKCS11 errors I found that it needs fuse. I installed libfuse-dev from synaptic package manager.
• The final compilation errors came from the GUI bits where it depends on wxWidgets. Synaptic to the rescue!
  
• Finally it built. But then I got an error at runtime along the lines of "Failed to communicate with kernel device mapped drive". I had done a rather large synaptic upgrade without bothering to reboot. Apparantly this kernel mode was affecting truecrypt so I was forced to reboot. Then it worked! Hurrah!

Once I had a working version of truecrypt I copied the entire build directory to my external USB backups directory, ready for the next time I need to install truecrypt on a new machine.

Now I will just put on my tinfoil hat briefly. I reckon that it is a conspiracy that truecrypt has gone. The developers say that the tool is not necessary now that Microsoft have BitLocker but this just doesn't wash. For a start I am on linux! And second, BitLocker is closed, secret, proprietary, so there is bound to be an NSA backdoor. Now I will remove my tinfoil hat and go and get a nice cup of coffee!

Software developer competency matrix

One of the ACCU mailing lists currently has a thread going where the challenges of hiring good C++ developers is being discussed. Someone posted a link to a software developer competency matrix. I thought it was rather good so decided to blog about so I didn't forget and so that I would have something to go back to for reference purposes.

Someone else in the thread posted that they had been inspired by another one but I found that was much more career/corporation oriented. The first one is much better for a techie to measure up against. It can be used to measure oneself or if you are involved the hiring game.

Reverse engineering a sybase database

The world seems to have decided on Oracle. But every now and then as a freelancer I come across a system that uses sybase. Usually it is an ancient legacy system that cannot be turned off for some reason, so sybase soldiers on. In such cases it is often useful to be able to reverse engineer the schema. This is where SchemaSpy comes in.

SchemaSpy requires graphviz, aka dot. The final generated output is a set of web pages and a set of dot files. I suspect that the dot is used to generate the image files used on the web pages, so dot is needed even if you never want to look at the dot files directly. Dot can be installed on windoze by merely unpacking the zip and adjusting PATH.

Some sybase installations have internationalisation set up in such a way that the URL needs to end in ?charset=iso_1. schemaSpy forms the URL based on a config it reads so the config must be extracted from the schemaSpy jar (a zip).

I first came across the charset issue when setting up database connections in DbVisualizer. I found
this web page which has the solution. This is also the solution for the same problem in schemaSpy.

Edit a copy of this config, appending ?charset=iso_1 to the name of the URL, then invoke schemaSpy with this command line (the sybase config filename is sybase.properties):

Note that the schema dbo must be specified, otherwise it tries to use the username as the schema name.

java -jar schemaSpy_5.0.0.jar -t sybase.properties -db <dbName> -u <userid> -p  <password> 
  -o <outputDir> -host <stringFromSybaseInterfaceFile> -port <portNumber>
  -dp <pathnameTojconn3.jar> -s dbo

a high-volume, low-latency market data processing system implemented with IBM middleware

Way back in 2003 I worked on a very interesting C++ CORBA project. It was to do with the near real-time distribution of market data to multiple subscribers. Most of the work was contracted out to that well known consultancy, Object Computing Inc. IMHO they did a very good job. One of the things that made it interesting was the way they used an XML configuration file to describe a pipeline of components through which the messages passed. This used the Configurator pattern described in POSA.

Back in January 2013 I came across a system that reminded me strongly of the one I worked on. I found an article in the Wiley Online Library (WOL). Unfortunately the WOL article is behind a paywall. Luckily, the authors also made it available in several other places including one where you get get it as a PDF.

This is a reminder to me that OCI aren't the only ones to develop a system like the one I saw and that there is very good write up about that other system. Here is what is says in the abstract:

A stock market data processing system that can handle high data volumes at low latencies is critical to market makers. Such systems play a critical role in algorithmic trading, risk analysis, market surveillance, and many other related areas. We show that such a system can be built with general-purpose middleware and run on commodity hardware. The middleware we use is IBM System S, which has been augmented with transport technology from IBM WebSphere MQ Low Latency Messaging. Using eight commodity x86 blades connected with Ethernet and Infiniband, this system can achieve 80 μsec average latency at 3 times the February 2008 options market data rate and 206 μsec average latency at 15 times the February 2008 rate.