Georgia Tech BORG Lab's Color Tagger (CTag)

About CTag
CTag Downloads
Requirements and Installation
- Linux Systems
  - Debian
  - RedHat
- Mac OS X
CTag Usage Documentation
Examples and Screenshots

About CTag

The Georgia Tech Color Tagger (CTag) is the first in a suite of tools designed to be an "off the shelf" vision-based object tracking system. CTag itself allows users to define targets for tracking, using frames from their own video data. Image frames are loaded into CTag, allowing the user to define a hierarchy of colors that the user wishes to track through the video. The color definitions can then be exported to a .txt file for use by the next two pieces of the tracking suite to both find and to track the motion of regions that match the pre-defined colors.

CTag was written in early 2003 by undergraduate students Amy Hurst and Robert McGuire, and is being offered under the GNU Public License.

CTag Downloads

You can download CTag in your favorite format:

Statically linked binaries for RedHat, Debian, and OS X to follow.

Requirements and Installation

CTag was written on Debian and RedHat x86 based systems, and has been tested on:

Debian 3.0 (Woody)
RedHat 7.3
Mac OS X 10.2.6

The program requires a few libraries, notably Gtk+-2.0, as well as libpng, and libjpeg. All but gtk should come with any Linux system. Gtk often comes with Linux distributions, or can be obtained from their website, or through one of the popular packaging systems.

Compiling CTag requires gcc, as well as GNU make.

In-program help requires that the Dillo web browser be installed. However, the help currently only points to this page.

Linux Systems

After the proper libraries and development headers are installed (see below for examples on installing the pre-requisites), download and decompress CTag, change to the directory created and run:

make

This should create the "ctag" binary. Install the program by moving or copying the "ctag" binary to the desired location, for example:

cp ctag /usr/local/bin

And run the program by invoking:

ctag

Installing Requirements Under Debian

Satisfying the requirements under Debian is fairly straightforward, using the apt-get utility. It is likely that one can install all of the necessary libraries using the following commands:

	apt-get install libgtk2.0-dev
	apt-get install libpng2-dev
	apt-get install libjpeg62-dev

And to optionally install the dillo web browser:

	apt-get install dillo

Installing Requirements Under RedHat

RPMs are probably available for most of the required libraries. Check rpmfind.net to find packages for your version of RedHat. Alternatively, libraries and programs can be installed from sources available at:

Gtk: http://www.gtk.org
libpng: ftp://swrinde.nde.swri.edu/pub/png/src/
libjpeg62: ftp://ftp.uu.net/graphics/jpeg/jpegsrc.v6b.tar.gz
Dillo (optional): http://www.dillo.org

Mac OS X Systems

To install and run CTag on OS X, you will need the following:

Apple Developer Kit: this has the compilers you need (i.e. Gcc etc)
You will need to install Fink (http://fink.sourceforge.net/). Fink is a tool that makes it easy to download UNIX programs and libraries. Simply download the disk image and then run it.
Once you have installed fink you need to have a graphical X-windows display. We suggest X-Darwin, which you can download with fink. Simply type:
```
fink install xfree86-rootless
```
Once you have installed the X windows system that will run on your Mac, you next need to install gtk+. Gtk is the graphical toolkit that CTag is written in. Simply type:
```
fink install  gtk+2-dev
```
Now you have installed everything you need to run CTag! Follow the same make instructions for Linux systems. Then, put the "ctag" binary where you like, or type
```
./ctag
```
to run the binary file.

Using CTag

Loading Sources

CTag currently only supports the loading of images as sources. The ability to load DV files should come in the future. In order to import images simply select "Load Images" from the Sources menu. After you select this menu option, the following popup will appear on your screen:

In order to select multiple files one at a time (like the example above, simply hold down the "Ctrl" button on your keyboard and click on the additional files you would like to load. However, to select a range of multiple files: from image1.jpg to image8.jpg, simply click on the first image, hold down the "Shift" key and then click on the last image.

Images can be generated from video files using almost any video import program. Chances are, if you are using DV, your DV import program will allow you to take snapshots. The preffered image types are JPEG and PNG.

Navigating Frames

Once you have loaded some sources into your project you will need to navigate through them in some way. Fortunately, there are two different ways to navigate these images. The image below displays the buttons needed to navigate through the frames.

Slider Bar This slider bar allows you to navigate through all the different frames that are currently loaded. The frames are ordered according to the order in which they were loaded (the first loaded images get the lower numbers). The buttons on either side allow you to traverse the frame sequence either forwards or backwards. In order to view the next frame simply click on the single ">" to the right of the slider. To view the previous frame click the "<" button. Likewise, to move 5 frames forward press ">>" and to move backwards 5 frames, "<<".
Text Box To the right of the slider is a small text box which displays which frame is currently being displayed, and displays the total number of frames which have been loaded "Frame: 4 of 0009". In order to view a higher numbered frame simply press the up button, and to view a lower numbered frame press the down button. If you know the exact frame number that you want displayed, you can simply enter it in the textbox and then press the enter button on your keyboard.
Notice that these navigation options are tied together and update each other: as you move the slider, the textbox updates itself and vice versa.

Colors and Regions

This section describes colors and color regions as well as all the different things you can do to each.

Colors versus Regions

This section explains the differences between defining a color and defining a color region.

The two main categories of selections are Colors and Color Regions. Colors have names and contain regions and have a highlight color. Color Regions are made up of many different selections and have an average color.

Color Properties

It can be very useful to categorize your date in some way, and the Colors group is an excellent way to do this. A color can contain any number of color regions and can have a descriptive name.

Color Name
One of the best ways to keep track of your different colors is to name them. There are two different ways to name or rename a color.

Edit Color name in Current Colors Panel This lets you change the name of the color in the Current Color Panel by simply double clicking on the color's name. When you double click on the name of the color, the name will appear in a textbox which you can type directly into. When you have entered the name you want, simply press the enter key on your keyboard.
Edit Color name in Color Properties PanelThe first item in the Color Properties panel is a textbox containing that color's name. To change this color simply click inside this textbox and edit the name. When you are done editing the name simply press enter on your keyboard.

Both of these methods produce the same result and each textbox updates the other.

ColorRegion Properties

This section describes the different properties of a color region, and how you can manipulate them.
Color regions are composed of a collection of pixel values which you the user define. In order to add a pixel value (or color) to your color region simply select the color that you want to add to (in the Current Color panel) and then click on the image. The pixel color that is directly under your mouse click will be the value that is added to your color region. The Color Properties panel on the right contains a box which holds all of the values in your color region. In order to remove a value from your color region simply click on that value with the mouse and say "yes" to the popup box. The color Region box also has a Highlight Color which you can use to represent your selections.

Average Color
Each color region is represented by an Average Color which is a rough estimate of all the values that are currently in it. This average color is simply something that is helpful when viewing the colors in the Current Color Panel. This average color changes as pixels are added and removed from this color region.

Adding and Deleting Colors and Color Regions

Adding and deleting colors is rather simple, and there are several different ways to do each.

Adding Colors and Color Regions There are two ways to add colors to your project. The first is to simply press the "Add" button on the Current Color panel and select either "Color" or "Region" in the popup menu. The second way to add a color or a region is through the Color menu on at top of the program. Simply select New Color to add a new color, or Add Region to add a new color region. When you add a color to your current color list it automatically adds an unnamed color with a black highlight color. Additionally, it gives it a default color region without any values in it and whose average color is black.
Deleting Colors and Color Regions There are also two different ways to delete a color or a color region; however both ways require that the color or the region to be selected by the mouse. Once you have selected to color or color region that you want to delete you can press the "delete" button on the current color box or select "Delete selected" from the Color menu.

Highlight Color

This sections describes what a highlight color is and how you can change it.

In order to preview your color selections you can use the "Highlight Color" feature. Each color has its own highlight color which can easily be set. The highlight color is the color square next to the color's name in the Current Colors panel on the left. In order to change the highlight color for a given color region (the default color is black) select that color region and look at the region properties panel on the right. The Highlight color information is located directly below the box containing the colors in the region you selected.

Changing Highlight Color
The current highlight color is displayed in a square box underneath the "Highlight Color" text. Immediately to the right of this colored box is the RGB value of this highlight color. In order to change the highlight color simply click on the "Change" button to the right of the RGB value and a color selection window automatically pops up. If you ever need to know what a different section of the color chooser does, just mouse over the part of the color chooser that you want more information about and a small box will appear with a description.
In the example above the current highlight color was the default, black, and it was set to be changed to a light orange.
This interface provides you with many different ways to select a color, the easiest way is to select one of the colors in the color palette. This palette contains a selection of standard or popular colors that might be useful higlight colors. To choose one of these, simply click on it and select "OK", and you will have set the highlight color.

If you don't like any of the colors in the palette then you can use the triangle in a circle to select a color. Simply place your mouse in the outer ring of the circle, press and hold the mouse button and then move your mouse around the circle to change the color. In order to change the brightness of the color, simply press and hold the small circle inside the triangle and move it around inside the triangle, release it when you have found a color that you like.

The rectangle located underneath the color circle/triangle displays the original highlight color and the current color that you have selected. As you change the color in the triangle it will show up in this box.

Viewing the Highlight Color
In order to see the highlight color on the actual image you are working on just click in the box "Highlight Matches in Image" and the colors defined by your color region will automatically appear in the image with the highlight color.

GaTech Color Tagger Usage Examples

This page will help walk you through some example uses of CTag.

Example 1: Three Colored Pool Balls
Example 2: More Pool Table Action
Example 3: Marked Bees

All of these examples are in the examples/ directory of the source. Images are included and loaded (if the relative file finding works. It is best to run CTag from the main source directory and load the examples from there). Just select "File/Open Existing Configuration" from the menu.

Example 1: Three Colored Pool Balls

Let's start with a fairly simple example, some video from a computer billiards program. You have three balls, one red, one green, and one blue that you wish to track across this video. Armed with a still image from the video (in jpg or png format), crank up CTag.

Here's the starting screen of CTag. There's not much to look at, but there are some things to do. For instance, we could start by creating a color to define.

This add button allows you to add colors, and add more regions to the colors you are defining. Either click this button or select "/Color/New Color" from the menu.

You should now see an item appear in the tree view on the left, expanding it will give you a view like this one. What you see is a new, unnamed color, with black (0,0,0) as the color to mark it with in the video. The color has one region for it, which is basically empty. In order to define the region for this color, we will need some kind of image source.
To do so, select "Source/Load Image(s)" from the menu.

This is the image loading window. Navigate to the directory where your example images reside. Select one of them (in this case "pooltable.png"), and click "OK".

You will now see a screen like this one, with the image you loaded in the center pane.

Let's define a color that matches the blue ball, first. Click on the "Unnamed" in the tree view. The right hand pane will change to show some details about this color. You can change the name of the color in the box at the top. In this case we chose "Blue Ball".

Now click on the (0,0,0) region for "Blue Ball" in the tree view. The right pane will change, again, this time showing details for this particular region. Selected pixels appear in the box at the top (there are none at present).

Let's change that by clicking on the blue ball. You will notice as you click that little buttons appear in the upper right box. These are the pixels that you have selected from the image. You will also see the average value and color of the region change in the tree view at the left. However, we don't know if we're really getting all of the blue. Let's change this by turning on the "Highlight Matches in Image" checkbox in the right hand pane.

You will now see that part of the blue ball has been replaced with black. This helps you see what part of the color space your region currently matches. Black is not a great color for this, so click the "Change" button next to the highlight check box to change it.

This is the selector for changing the highlight color. The box in the lower left shows the current color on the left, and the tentative color that has been selected on the right. Start by clicking on the color ring to pick a base color, then select within the triangle what brightness you wish the color to have. You can also pick from the colors on the bottom right, or define HSV, RGB, or even HTML-style colors, yourself. Choose a color and click "OK". We chose yellow for now.

Now you can see much better where the blue is being matched by your color space. To finish filling it in, simply keep clicking parts of the blue ball until all of the ball is yellow.

This is the final space for the blue ball. It has been filled in, entirely, meaning the tracker should be able to get a good match on the entire ball in a video.

Now let's define a color for the green ball. As before, either click "Add" in the left pane or select "/Color/Add New Color" from the menu. If you click "Add", you will be asked whether you wish to create a new color or a new region. Select color, in this case. Also, rename this color (we called it "Green Ball"), and give it a highlight color. Be sure to make the highlight color different from the one you chose for the blue ball (we chose magenta in this case).

Now do the same clicking on the green ball. You will see the ball become shaded more and more with your highlight color. When the whole ball is covered in your highlight color, you are done and the color should be well defined.

Now do the same for the red ball. We chose sort of a cyan color for the highlighting, and called the color "Red Ball", then clicked the red ball until it was cyan.

You can now save this configuration using the "/File/Save Configuration" menu option, this will write out all of your color and region definitions as well as what pixels were clicked to define them, and from what image these pixels came. This can be loaded in later using the "/File/Open Existing Configuration" menu option.

If you are satisfied with your definitions and wish to try them out, select "/File/Export Config for Tracking" from the menu. This will let you save a .txt file suitable for use with the separate tracking program. It is a simple text file, defining color names, highlight colors (which is why it is suggested that each color have it's own unique highlight color), and ranges.

Example 2: More Pool Table Action

What if you didn't care which ball was which, and just wanted to define a space that would match any of them? This is easily done by giving one color multiple regions to match.

As before, create a color (we called ours "Balls") and give it a highlight color (we chose yellow), be sure to turn on highlighting. Now click on any ball (we chose blue to start) until it is entirely filled with the highlight color.

It would now seem reasonable to continue clicking on all of the balls, to fill them with the highlight color, too. However, notice what happens when we click on a red pixel. By stretching our region definition to include both the blue and red balls, we have gotten greedy, and are now matching lots of colors in between. This would lead to lots of over-matching by the tracker, and is generally considered to be a bad thing.

First, we want to remove that bad pixel, so click on it in the box at the upper right. A dialog will pop up offering to remove that color. Go ahead and say yes to restore the region to it's previous goodness.

Now, select "Color/Add Region" from the menu, or click "Add" and select "Region" when prompted. Pick one of the balls and fill it in with the highlight color. Notice that the other balls aren't being highlighted at all.

If you now select "Balls" from the tree view on the left, you will see that the two balls you have defined regions for so far are both highlighted, meaning the tracker would find them in the video and consider them to be the same "color".

Create a third region and define it by clicking on the third ball.

The final color, with three different regions matching it. This defines all of the balls as being one "color".

To save this configuration for the tracker, select "File/Export Config for Tracking".

And here's the export save window. Just find a place to put your .txt file.

Example 3: Marked Bees

An example closer to the real world involves video of marked honeybees. We'll use a sequence of images taken from part of a larger video that we desire to examine.

Start by selecting "Source/Load Image(s)" from the menu, select all of the images in the sequence by holding Control while clicking each one or by clicking the first one, then holding Shift and clicking the last one. Click "OK" when all of the images are selected.

Now you will see the first image in the center pane. Scrollbars will appear if the image is too large for the pane. Notice the navigation panel at the bottom of the pane. The buttons, slider, and frame number text box can all be manipulated to change the frame from the image sequence you are viewing. When video is implemented, it will be navigated in the same way.

Create a color (ours is "Bee Green"), and give it a highlight color (we chose yellow), and turn on highlighting. Now start clicking on the green dots on all of the bees until all of them are filled in as much as possible. If other, non-green-dot, parts of the image start to be colored with the highlight color, remove the pixels that caused the spillover, as the false positives will mess up the tracker.

Notice that the highlighting is maintained between frames of the image sequence, so you can navigate back and forth to make sure the matching is consistent between frames.

Here's the starting screen of CTag. There's not much to look at, but there are some things to do. For instance, we could start by creating a color to define.
	This add button allows you to add colors, and add more regions to the colors you are defining. Either click this button or select "/Color/New Color" from the menu.
You should now see an item appear in the tree view on the left, expanding it will give you a view like this one. What you see is a new, unnamed color, with black (0,0,0) as the color to mark it with in the video. The color has one region for it, which is basically empty. In order to define the region for this color, we will need some kind of image source. To do so, select "Source/Load Image(s)" from the menu.
	This is the image loading window. Navigate to the directory where your example images reside. Select one of them (in this case "pooltable.png"), and click "OK".
You will now see a screen like this one, with the image you loaded in the center pane.
	Let's define a color that matches the blue ball, first. Click on the "Unnamed" in the tree view. The right hand pane will change to show some details about this color. You can change the name of the color in the box at the top. In this case we chose "Blue Ball".
Now click on the (0,0,0) region for "Blue Ball" in the tree view. The right pane will change, again, this time showing details for this particular region. Selected pixels appear in the box at the top (there are none at present).
	Let's change that by clicking on the blue ball. You will notice as you click that little buttons appear in the upper right box. These are the pixels that you have selected from the image. You will also see the average value and color of the region change in the tree view at the left. However, we don't know if we're really getting all of the blue. Let's change this by turning on the "Highlight Matches in Image" checkbox in the right hand pane.
You will now see that part of the blue ball has been replaced with black. This helps you see what part of the color space your region currently matches. Black is not a great color for this, so click the "Change" button next to the highlight check box to change it.
	This is the selector for changing the highlight color. The box in the lower left shows the current color on the left, and the tentative color that has been selected on the right. Start by clicking on the color ring to pick a base color, then select within the triangle what brightness you wish the color to have. You can also pick from the colors on the bottom right, or define HSV, RGB, or even HTML-style colors, yourself. Choose a color and click "OK". We chose yellow for now.
Now you can see much better where the blue is being matched by your color space. To finish filling it in, simply keep clicking parts of the blue ball until all of the ball is yellow.
	This is the final space for the blue ball. It has been filled in, entirely, meaning the tracker should be able to get a good match on the entire ball in a video.
Now let's define a color for the green ball. As before, either click "Add" in the left pane or select "/Color/Add New Color" from the menu. If you click "Add", you will be asked whether you wish to create a new color or a new region. Select color, in this case. Also, rename this color (we called it "Green Ball"), and give it a highlight color. Be sure to make the highlight color different from the one you chose for the blue ball (we chose magenta in this case).
	Now do the same clicking on the green ball. You will see the ball become shaded more and more with your highlight color. When the whole ball is covered in your highlight color, you are done and the color should be well defined.
Now do the same for the red ball. We chose sort of a cyan color for the highlighting, and called the color "Red Ball", then clicked the red ball until it was cyan.

	As before, create a color (we called ours "Balls") and give it a highlight color (we chose yellow), be sure to turn on highlighting. Now click on any ball (we chose blue to start) until it is entirely filled with the highlight color.
It would now seem reasonable to continue clicking on all of the balls, to fill them with the highlight color, too. However, notice what happens when we click on a red pixel. By stretching our region definition to include both the blue and red balls, we have gotten greedy, and are now matching lots of colors in between. This would lead to lots of over-matching by the tracker, and is generally considered to be a bad thing.
	First, we want to remove that bad pixel, so click on it in the box at the upper right. A dialog will pop up offering to remove that color. Go ahead and say yes to restore the region to it's previous goodness.
Now, select "Color/Add Region" from the menu, or click "Add" and select "Region" when prompted. Pick one of the balls and fill it in with the highlight color. Notice that the other balls aren't being highlighted at all.
	If you now select "Balls" from the tree view on the left, you will see that the two balls you have defined regions for so far are both highlighted, meaning the tracker would find them in the video and consider them to be the same "color".
Create a third region and define it by clicking on the third ball.
	The final color, with three different regions matching it. This defines all of the balls as being one "color".
To save this configuration for the tracker, select "File/Export Config for Tracking".
	And here's the export save window. Just find a place to put your .txt file.

	Start by selecting "Source/Load Image(s)" from the menu, select all of the images in the sequence by holding Control while clicking each one or by clicking the first one, then holding Shift and clicking the last one. Click "OK" when all of the images are selected.
Now you will see the first image in the center pane. Scrollbars will appear if the image is too large for the pane. Notice the navigation panel at the bottom of the pane. The buttons, slider, and frame number text box can all be manipulated to change the frame from the image sequence you are viewing. When video is implemented, it will be navigated in the same way.
	Create a color (ours is "Bee Green"), and give it a highlight color (we chose yellow), and turn on highlighting. Now start clicking on the green dots on all of the bees until all of them are filled in as much as possible. If other, non-green-dot, parts of the image start to be colored with the highlight color, remove the pixels that caused the spillover, as the false positives will mess up the tracker.
Notice that the highlighting is maintained between frames of the image sequence, so you can navigate back and forth to make sure the matching is consistent between frames.