Monitoring your CrashPlan home backup instance

On April 15, 2014, in GNU/Linux, Hacks, News, Programming, by Vegard Haugland

I take a lot of pictures and little is more important to me than making sure that all my photos are kept in safe storage. For this purpose, I use the online backup service from CrashPlan. With the CrashPlan backup client being written in Java, it is supported on many platforms, including GNU/Linux, and it even allows you to configure your CrashPlan server via SSH.

CrashPlan is a great “bang for buck”, costing only as little as $4/month (if you buy a 4 year subscription) and top you have unlimited online storage. What is the catch you ask? Well, if you ask me the main catch is that with the “Home” edition (for $4/month), you are not able to access its API. This is important to me as I use Nagios to monitor all my servers and also important services that I need to know is up and running 24/7.

Using Nagios to monitor CrashPlan

Using Nagios to monitor CrashPlan

I figured out a way to accomplish this as the CrashPlan server application produces a history log of whatever it is doing, called “history.log.0″. This file typically looks something like this:

history.log.0 sample: click to open SelectShow

In brief, there are three important data types we need to parse:

  1. The timestamp
  2. When the backup (or scan) job started
  3. When the backup (or scan) job completed

If you are used to parsing text and recognizing patterns, you can easily write some regular expressions to extract the types of text that this log contains. I wrote a small application in Python (crashplan-log-parser) that does exactly this, and allows you to print the latest job that finished, statistics and also the number of hours passed since the last job was completed.

2014-04-15-215100_1366x768_scrot

The above screenshot shows the command line options of logparser.py. The below screenshot however, shows the statistics command in use. Of a total of 110 jobs (found in the log file), 76 are scan jobs and 34 are backup jobs. The last backup was completed this morning 03:00 AM Norwegian time.

2014-04-15-215132_1366x768_scrot

Finally, for integration in Nagios, the following command is also available. This prints the number of hours since the last backup or scan job was completed (integer value).

2014-04-15-215143_1366x768_scrot

With this information, one could easily write a Nagios monitor script for polling this e.g. once per 30 minutes. e.g.

Nagios passive check script: for monitoring CrashPlan Home SelectShow
Tagged with:
 

Reverse Engineering an Unknown Savegame File Format

On September 24, 2012, in Games, Hacks, News, Programming, by Vegard Haugland

Lately, I’ve spent a few hours playing FTL: Faster than Light – the spaceship simulation, real-time, roguelike-like game. One of the most challenging aspects of this game is that when you’re dead, you’re *really* dead. There is no way to continue playing, so you have to start over. Considering this game is quite difficult to finish in the first place (actually, I haven’t even finished it yet), I started looking for ways to tip the odds in my favor.

In-game screenshot of FTL showing the spaceship type “Kestrel”.

FTL allows you to save your game using a single save-slot. I suspected that this had to be written to a file somewhere, and after doing some digging I found the savegame file called “continue.sav”. The contents of this file is binary encoded in little-endian. So, in order to understand which bytes to alter in order to make the game easier, I had to reverse engineer the file format. Reverse engineering can be quite rewarding. Sort of like mentally climbing a mountain. While you’re working at it, it’s tedious and it can drive you crazy. However, when you finally make it, it is usually worth the effort!

In order to edit a binary file, you cannot use a regular text editor, as these programs parse the bytes in the binary file as ASCII. This will make the contents look like gibberish, and altering any of the contents will probably result in crashing FTL once you try to load the modified savegame.

Instead, you need a hex editor, which allows you to see and modify the binary file at byte level. Once you get a hex editor, you can fire it up and load the savegame. At first, you probably won’t make any sense out what you are seeing. To make it easier for yourself, make a copy of the savegame. Then return to the game, modify a variable and save the game again. Then, by comparing the two savegames, you should easily spot any differences.

Using this method, I quickly managed to map one third of the data in the binary file. To show an example, I have highlighted some of the byte values in the image below. The first 4 bytes contains the current ship integrity level. Second is the fuel value, followed by the number of available missiles. Finally we find the the number of droids and the amount of available scrap.

Here we are see bytes in the savegame encoded in little-endian. Each value consists of 32 bits (or 4 bytes).

Once you have all this information, you can either manipulate the values directly in the hex editor, or you can create a savegame editor. The first is definately easiest, as you need to spend more time decoding the file format in order to make a savegame editor. Especially if the size of the savegame changes based on the content, which is the case with the FTL savegames. As an example, text strings are stored dynamically, meaning that each string in the savegame is preceded by 4 bytes telling the length of the string. Because of this, you need to decode every byte before you reach the correct offset where you can modify the ship integrity.

ftlsg – an FTL savegame editor. Currenty, you are limited to changing the basic (read: important) attributes. (note: the values shown in this screenshot differs from the hex dump screenshot)

When you have finished decoding all the bytes preceding the offsets you want to modify, you need to apply your programming skills and create code that parses the binary file, that allows you to make modifications and that allows you to save the modified savegame. I started on one such project, written in C, which I am currently hosting on GitHub. Check it out at: http://github.com/v3gard/ftlsg.

The savegame editor is currently limited to changing the 5 basic attributes; ship integrity, fuel, missiles, droids and scrap. However, as I have mapped out a rather large portion of the binary file, I plan on adding support for changing more attributes in future versions.

If you feel like giving reverse engineering a try, I can highly recommend getting started with the FTL savegames. Mapping out the attributes listed above is not too difficult, and it is always fun learning something new!

Tagged with:
 

Teensy 2.0 USB Development Board

On February 20, 2012, in Programming, by Vegard Haugland

Last week I received my shining new Teensy 2.0 USB Development Board. During the weekend, I decided to spend some time getting to learn how to program it.

My Teensy 2.0 USB "Keyboard"

The Teensy is a complete USB-based microcontroller development system, capable of implementing several types of projects. The microcontroller on the Teensy 2.0 is an Atmel Mega32U4. All the programming is done via the USB port. There is also a project called Teensyduino, which allows you to write code for your Teensy board using the Arduino IDE.

The guys who produce the Teensy board have already made available easy to use libraries, that lets you implement a lot of functionality with little code. I decided to try out their USB Keyboard library, and make a small program that allowed me to make a programmable “keyboard”. These libraries are written in pure C, and can be compiled with gcc.

The small project I did allowed my to enter some text into any given application on my OS (which the Teensy was connected to), based on the state of a register in the micro-controller.  In order to see the state of the register, I hooked it up to a small blue LED, as seen in the image on the right. You can also see two buttons there. The top-most button flicks the LED on and off (i.e. sets the register to 0×00 or 0xFF, respectively), and the bottom button transmits some hard-coded text into the OS.

Screendump from my laptop while trying out the Teensy board

The buttons are powered by the +5V VCC pin on the Teensy and connected to ground via a 1K Ohm pull-down resistor. When the top button is pressed, a signal is sent to the D2 pin, and when the bottom button is pressed, a signal is sent to the D3 pin. The LED received power from the  B1 pin, which is set when the D3 pin receives power.

When my program is compiled and uploaded to the Teensy, the OS kernel thinks that a USB keyboard is connected to the USB port and parses all input as keyboard events. So, when I press the button button, the text “The light is now off” is sent to the OS. If I first press the top button which switches on the LED, I can re-press the bottom button, and the text “The light is now on” is sent instead.

I wonder what I might be able to accomplish  if I issued the command to open up a terminal and enter some code…:-)

 

Why you should stay away from Thrust::VPS

On February 19, 2012, in News, Ramblings, by Vegard Haugland
Thrust::VPS

Thrust::VPS - Damn, that's crappy VPS!

I’ve been a customer with Thrust::VPS (aka Damn::VPS) for well over a year. Despite several minor issues I’ve had with them during this period, I continued my customer relationship with them due to their low prices and high bandwidth cap of 1TB each month. During the last couple of weeks however, I started experiencing a lot of problems on my VPS (Virtual Private Server). When I contacted their tech support to resolve these issues, the answers I received were vague and, in my honest opinion, very unprofessional. In fact, the customer treatment I received from them was appalling to be frank, and it ended up with me terminating the contract I had with them, and signing a contract with Linode instead. Here’s why.

Continue reading »

 

Downloading FLV streams using rtmpdump

On October 16, 2011, in Hacks, News, by Vegard Haugland

Many TV channels allow you to stream their shows from a flash plugin on their website. How cool wouldn’t it be if you were able to download these streams to a file on your computer, or even show the streams in your favorite movie player, like mplayer? Let me tell you one thing. This is indeed possible.

How?

In order for the flash plugin to show the streams on your computer, they need to be freely available on the interwebz. If we use a protocol analyser like Wireshark to dig into the information sent between the flash plugin that runs in your browser, and the server that provides the video streams, we often see that it is transmitted using the Real Time Messaging Protocol (RTMP). If you happen to use a GNU/Linux based operating system, you’re in luck. There exists a toolkit called ‘rtmpdump’ that allows you to interact with a server offering RTMP streams. All you need to display these streams on your computer, are a few parameters that are exchanged between the aforementioned flash plugin and the RTMP server.

Example? Pretty please!?!

I will now show you an example of how this is done by grabbing an episode of the Norwegian TV show “Robinson-ekspedisjonen” that runs on TV3. This is the Norwegian version of Survivor. These episodes are freely available on http://tv3play.no if you want to display them in your browser. If you are like me, and you’d like a local copy on your computer, follow these steps:

1. Navigate to the URL where the first episode of Season 11 is located (http://www.tv3play.no/play/252950/)

2. Fire up a terminal and enter the following command (and yes, regular expressions are awesome!):

ngrep -W byline | grep -Pio '(swfUrl\.{2}M\K.*?(?=\.{2}tc))|(pageUrl\.{2}"\K.*(?=\.{3}))|(tcUrl\.{3 }\K.*?(?=\.{2}))|(resetting \K.*?(?=\.{3}))'

3. Start the show in your browser. A commercial should now appear (that you should really watch in return of downloading the stream ;-)), and wait for the output that appears in the terminal. It should be something like:

1.http://flvplayer.viastream.viasat.tv/play/swf/player110516.s.wf?rnd=1318382997
2.rtmp://mtgfs.fplive.net/mtg
3.flash/norway/tv3/robinson2011/robinsonekspedisjonen_sd_1101-flash_500

4. Here follows a short description of the above output.

1. This is the URL of the flash plugin that runs in your browser. Since we are parsing raw network data, an extra “.” has been added in between the “swf” extension. This can safely be removed. The corresponding switch in rtmpdump is “-W”
2. This is the location where you access streams on the RTMP server. The corresponding switch in rtmpdump is “-r”.
3. Finally, this is the path to the TV show on the RTMP server. The corresponding switch in rtmpdump is “–playpath”.

5. We can now feed these variables to rtmpdump, and do whatever we want with the captured stream. Here are two examples:

Download:

rtmpdump -r "rtmp://mtgfs.fplive.net/mtg" --swfVfy --swfUrl "http://flvplayer.viastream.viasat.tv/play/swf/player110516.swf?rnd=1318382997" --playpath "flash/norway/tv3/robinson2011/robinsonekspedisjonen_sd_1101-flash_500" -o output.flv

Show in mplayer:

rtmpdump -r "rtmp://mtgfs.fplive.net:1935/mtg" -W "http://flvplayer.viastream.viasat.tv/play/swf/player110516.swf?rnd=1318382997" --playpath "flash/norway/tv3/dagens_mann/sesong_1/dagens_mann_101-flash_500" -live | mplayer -

6. Side note. If you want to watch/download the stream in higher quality, replace “flash_500″ with “flash_900″ in the playpath.

Is this legal?

The way I see it, we are not doing anything illegal here, as the video streams are made available from the publisher.

Sure, they are hidden such that “Average Joe” will have difficulties downloading them. Also, we do perform a neat hack when we extract the required variables from the network packet stream using a regular expression, but we still need to watch the commercial (or at least wait for it to finish) in order to get these variables.

As long as I (or you) keep the downloaded streams for yourself, and don’t distribute them directly, you are operating within any legal boundaries, at least if you live in Norway.

Tagged with:
 

During my spare time, I try to learn C. Often I find myself having to create arrays, and fill these with data manually. Since I hate, and let me emphasize, hate, doing repetitive tasks, I wanted to show you guys how to create an array with 6 elements as simple as possible.

Consider the following; you want to create an array of strings with 6 elements, where each element can hold a maximum of 25 characters. Hence, you type the following:

#include <stdio.h>

int main(int argc, char argv[])
{
char my_array[6][26];

strcpy(my_array[0], "Something");
strcpy(my_array[1], "Something else");
strcpy(my_array[2], "Something different");
// etcetera

return 1;
}

Now, that wasn’t very tiresome, but what if you had to enter 10 elements, or even 50? Typing strcpy(my_array[x], “blahblah”); that many times can even make the most patient person go nuts. If you’re a VIM user, this process can however be simplified quite a bit.

First, start VIM. Next, type ‘strcpy(my_array[x], “”‘);’, and yank this line by pressing ‘yy’. Now, to repeat this process 5 times, simply type “5p”. You should now have 6 lines containing:

strcpy(my_array[x], "");
strcpy(my_array[x], "");
strcpy(my_array[x], "");
strcpy(my_array[x], "");
strcpy(my_array[x], "");
strcpy(my_array[x], "");

The final step is now to replace all the x characters with a sequence of numbers, ranging from 0 to 5. By using the Nexus plugin, located here, you can do this with one neat command.

With the plugin installed, replace the first occurance of ‘x’ with 0. Then, enter Visual Block Mode (C v), and select the remaining x characters. Now, enter the command: :’< ,'>s/\%V./\=s1.next()/, and all the x characteres is replaced with a sequence of numbers, ranging from 1 to 5. You should now have:

strcpy(my_array[0], "");
strcpy(my_array[1], "");
strcpy(my_array[2], "");
strcpy(my_array[3], "");
strcpy(my_array[4], "");
strcpy(my_array[5], "");

Neat, huh? If you want to perform the task again, you will see that s1 continues from where it stopped in the above example. To reset the s1 variable, run ‘:call s1.reset()’.

Tagged with:
 

Configure sendmail as an MTA relay on Slackware

On February 22, 2011, in GNU/Linux, by Vegard Haugland

My landlord recently changed to a different ISP that blocks all outgoing requests on port 25, except towards their own SMTP-server. After getting my hands dirty by plowing throgh the sendmail documentation, located here, I figured out that it wasn’t very difficult to configure sendmail to relay all outgoing e-mails towards this server, thus solving the entire problem.

If you want all outgoing mail to go to a central relay site, define SMART_HOST.

I started by checking the sendmail configuration file, /etc/mail/sendmail.cf, that strictly specified “DO NOT EDIT THIS FILE!  Only edit the source .mc file.“. Following this advice, I had to figure out where this .mc file was located.

I did a quick search for all files containing sendmail and cf on my local system, and discovered “/usr/share/sendmail/cf/cf/sendmail-slackware.mc“. That was indeed the correct file, and I simply added “define(`SMART_HOST’, `smtp.online.no’)“. To see the entire configuration file, expand the spoiler below.

Spoiler Inside: modified sendmail-slackware.mc SelectShow

Once that was done, I had to compile the configuration file with “m4 /usr/share/sendmail/cf/cf/my-sendmail.slackware.mc > /etc/mail/sendmail.cf“, and restart inetd and sendmail by running /etc/rc.d/rc.inetd restart && /etc/rc.d/rc.sendmail restart.

Problem solved, and I’m receiving my emails again!

Tagged with:
 

Getting cmus to cooperate with conky

On January 26, 2011, in GNU/Linux, by Vegard Haugland

Screenshot of My Current Desktop

cmus is a small, fast and powerful console music player for Linux and *BSD. If you also happen to use conky, the light-weight system monitor, grabbing data from cmus and rendering it in conky is a piece of cake!

cmus comes with an application called cmus-remote that allows you to control the music player from a remote location. We’re now gonna use this app to grab the metadata and make it accessible to conky.

Since conky doesn’t have a plugin for accessing cmus directly, we need to write a simple shell script:

if [ ! -x /usr/bin/cmus-remote ];
then
echo "cmus is not installed."
exit
fi

ARTIST=$( cmus-remote -Q 2>/dev/null | grep artist | cut -d " " -f 3- )
TITLE=$( cmus-remote -Q 2>/dev/null | grep title | cut -d " " -f 3- )

if [ -z "$ARTIST" ];
then
echo "Nothing"
else
echo "$ARTIST - $TITLE"
fi

If you have a song currently being played in cmus, you can run the script shown above in your favorite terminal, and something like “Scar Symmetry – Trapezoid” will be returned.

To get the information displayed in your conky bar, you can add the following somewhere in your .conkyrc:

${execi 2 /path/to/script}

To see an example of how a rendered bar looks like, check out the screenshot of my current desktop. I’m using the i3 window manager, and the output from conky is rendered using a combination of i3-wsbar and dzen2.

My .conkyrc is available here, and you can download the cmus-conky-script here.

Tagged with:
 

Identifying skimming equipment

On September 11, 2010, in Security, by Vegard Haugland

Skimming

Illustration of someone attaching skimming equipment to an ATM. Image courtesy of BBC News.

Almost every week I make notice of articles in the national newspapers that someone has become victim of ATM fraud, or skimming. Usually, the article describes where the event took place and issues a warning that you should contact your bank if you suspect that you have been victim to such crimes.

One thing that bothers me about this, is that neither the media or even the banks here in Norway publish any information about how one can detect skimming equipment, so you can avoid having your VISA card scanned (or your pin code video taped) in the first place.

By doing some research, I found out that a bank in Australia (!) has published information on how to spot if an ATM has been modified, with images of how it looks like both with and without the skimming equipment attached. My first impression is that the material is both informative and easy to follow.

Once you read through the PDF, you should be more fit to avoid being victim of card skimming in the future! :-)

In addition, I would recommend that you examine the ATM every time before you use it. If anything seems out of the ordinary, see if you are able to move any loose panels around. You do not have anything to loose, except your money.

Tagged with:
 

Install Android 2.2 (Froyo) on your HTC Magic

On September 6, 2010, in Android, by Vegard Haugland
Image of the HTC Magic

HTC Magic – Image courtesy of HTC.

Did you know that it is possible to upgrade your HTC Magic to use the newest version of Android? Doing so significantly increases the phone’s performance and reliability – in addition to give you access to awesome applications like chrometophone.

Last year I bought a HTC Magic (aka Sapphire). At the time, the phone shipped with Android 1.5, while the most recent version of Android at the time was 1.6. Since there was no official update available from HTC, I decided to try out one of the community-based ROMs.

After conducting some research, I settled for a project called CyanogenMod – mainly because it offered increased reliability and performance. By following the necessary steps for upgrading the HTC Magic, I managed to install the ROM and have Android 1.6 available on my phone. It wasn’t very difficult either. You basically:

  • Download the CyanogenMod ROM and  the corresponding kernel image. You also need a recovery ROM (for installing CyanogenMod and the kernel)
  • Copy CyanogenMod and the kernel over to the phone’s SD card.
  • Use the Android SDK (or fastboot) to boot the custom recovery ROM you downloaded earlier.
  • (Backup your existing ROM with Nandroid)
  • Install CyanogenMod and the kernel from the SD card
  • Reboot
CyanogenMod Logo

CyanogenMod Logo

CyanogenMod held its promise. The phone became much more responsive and reliable than earlier, and with the phone rooted, I also had the option to install applications that was unavailable to me before.

Now, a year later, I have installed Android 2.2 (Froyo) on the phone (by following the same steps as described above). Compared to the initial ROM that shipped with the phone a year ago, the performance is MUCH better. You also have multitouch functionality (like pinching and zooming), and you have access to the Android Cloud to Device Messaging (C2DM) framework (which lets you install chrometophone).

If you have a HTC Magic (or a similar Android device), you should install CyanogenMod. You won’t regret it.

Tagged with: