TMSB Tutorial 4E: Multitemporal Processing

article by Ben Lincoln

This tutorial describes how to build a custom input configuration based on a request by Steve Smeed. Steve uses a technique he discovered on Bjørn Rørslett, which Bjørn and Steve refer to as "Multitemporal Photography", in which the same shot is taken multiple times from a tripod - but unlike multispectral photography, with the same filter each time - and then the channels from different shots are combined together.

For example, if the photos are of a moving river, then combining the red channel from one exposure, the green channel from a second exposure, and the blue channel from a third exposure will result in an image in which the still surroundings of the river appear normal, but the river itself takes on an oil slick appearance similar to classic "plasma" graphics demos.

Interestingly, this technique (described by Bjørn Rørslett in 2000-2001 based on the dates of his photos) has been in use for far longer. My aunt gave me a copy of David Busch's Digital Infrared Pro Secrets awhile ago, and one of the sections ("Multi-Color Magic") includes this exact process. That book is from 2007, but the author cites as his inspiration Bob Harris' Here's How Book of Photography, Volume VII, which was published in the 1970s. Harris' method (from the analogue world of film) was to create a vertical strip of three filters (red, green, and blue) and drop it through a slot mounted in front of the camera lens with the camera set to about 1/4 second exposure time - a clever approach in an era when combining channels from separate exposures would have been extremely time-consuming and laborious.

Multitemporal Example Images

Red, green, and blue channels from different exposures

The green channels of three different exposures mapped to red, green, and blue

Maximum - Median - Mininum [3C] across all three channels of four RGB exposures

Absolute difference between the green channel of one image and the blue channel of another.

A few possible variations of this technique, using the example image set from this page (a shot of a river in Oregon's Umpqua National Forest back in 2008).

Date Shot: 2008-08-29
Camera Body: Nikon D70 (Modified)
Lens: Nikon Series E 28mm(?)
Filters: LDP LLC CC1
Date Processed: 2011-04-17
Version: 1.0

Apparently some people have told Steve that the images he produced using this technique are "too colourful", and "look too much like tie-dye". I ask those people to look at the photos of me elsewhere on this site. If I can like this type of photo, so can you.

Using The Mirror's Surface Breaks in this way is a repurposing of the software, much like how I use Hugin to align multispectral exposures even though it's intended to create panoramas (or image stacks) made from images of the same type as each other. As a result, while you can obtain some very interesting images this way (especially when using a statistical processing configuration), you may also encounter some unexpected behaviour. For example, don't select a processing configuration that includes calculated images, because it will take several minutes just to calculate the number of permutations (which will usually number in the hundreds of thousands).

The example configuration you'll create is capable of handling up to five different RGB colour input files, although only two or more are necessary. There is an example image set available at the bottom of this page which includes four exposures.

These instructions assume that the reader has gone through all of the previous tutorials in this section, except for TMSB Tutorial 4D: Klaus D. Schmitt Custom Colour-Processing Matrix, which is intended for advanced users only.

Like all of the tutorial configuration files, complete (and tested) versions of the results below are included in the Tutorial subdirectory of TMSB_Config wherever you unpacked the software to (TMSB version 1.2 or later). These can be used to refer to if you run into trouble, or you can just copy them into your TMSB_Config directory if you would rather just use them instead of know how to create custom configurations.

Basic Configuration

These instructions will actually end up creating three related input configurations. First, the master/template configuration, which does not place restrictions on how the input channels can be combined:

In the TMSB GUI, switch to the Input Configuration tab.
Make sure something is selected in the drop-down menu (it doesn't matter what), then click the Export XML button.
In the File name field, enter Input-Multitemporal_RGB-5F-TIFF.xml, then click Save.
Launch your text editor and open the file you just saved.
In the <Metadata> section:
1. In the <Name> section, change the text to read Custom: Steve Smeed Multitemporal RGB (TIFF - Up To 5 Files).
2. In the <Description> section, change the text to This is an unconventional input configuration which accepts up to five input files ("RGB-01.TIF", "RGB-02.TIF", "RGB-03.TIF", "RGB-04.TIF", and "RGB-05.TIF") which represent the same shot, but taken at different times. Each channel of each of the five files is treated as a "spectral band". This configuration is recommended for use with the statistical processing configurations.
3. If there is already a <Warnings> section, replace its text with Using this configuration (especially with all five source files) may result in a surprisingly large number of output images. Otherwise, add this block after the <Description> block:
  
  <Warnings>
  
  Using this configuration (especially with all five source files) may result in a surprisingly large number of output images.
  
  </Warnings>
4. If you like, enter the date/time, in XML format in the <Created> and/or <LastModified> sections.
Delete everything between the <InputConfig> and </InputConfig> tags, but leave those two tags themselves in place.
Copy the following XML code, and paste it in-between those two tags. This lengthy block of code defines the red, green, and blue channels for each of five input files as a faux "spectral band" (for a total of fifteen). Note that for completeness, the first of each type is assigned the corresponding <SpecialBandName> value (Red, Blue, and Green), but only the first, because each special band name can only be used once in a given input configuration.

Experienced users may wish to experiment with changing the order of the channels (for example, interleaving the channels as R01/G01/B01/R02/G02/B02..., instead of listing all of the red channels, then all of the green channels, et cetera). This will not affect statistical image output, but will significantly alter the permutations that are generated by processing configurations that generate all possible three-channel false colour (R/G/B combinations) images.

<SpectralBands>



<SpectralBand>

<BandName>

R01

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-01.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<SpecialBandName>

Red

</SpecialBandName>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

R02

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-02.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

R03

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-03.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

R04

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-04.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

R05

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-05.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>



<SpectralBand>

<BandName>

G01

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-01.TIF

</FileName>

<Plane>

2

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<SpecialBandName>

Green

</SpecialBandName>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

G02

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-02.TIF

</FileName>

<Plane>

2

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

G03

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-03.TIF

</FileName>

<Plane>

2

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

G04

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-04.TIF

</FileName>

<Plane>

2

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

G05

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-05.TIF

</FileName>

<Plane>

2

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>



<SpectralBand>

<BandName>

B01

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-01.TIF

</FileName>

<Plane>

3

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<SpecialBandName>

Blue

</SpecialBandName>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

B02

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-02.TIF

</FileName>

<Plane>

3

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

B03

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-03.TIF

</FileName>

<Plane>

3

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

B04

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-04.TIF

</FileName>

<Plane>

3

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

<SpectralBand>

<BandName>

B05

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-05.TIF

</FileName>

<Plane>

3

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

</SpectralBand>

</SpectralBands>
Save the file and exit.
If you closed out of TMSB previously, launch the GUI again. Otherwise, from the Tools menu, select Reload Configuration Files. In the Input Configuration tab, the drop-down should now contain a new entry (named Custom: Steve Smeed Multitemporal RGB (TIFF - Up To 5 Files), if you followed all of the instructions correctly).
If you like, give the new configuration a try, using the example input image set available at the bottom of this page. I recommend starting out with either the No Funny Business or Focused: Statistical Images 01 (Fast) (Photography) processing configuration.

Mutually-Exclusive Channels Variation 1

This additional customization will create a variation of the previous configuration which uses the <MutuallyExclusiveBand> tag to prevent multiple channels of the same type (red, for example) from being used in the same image. For example, a three-channel permutation that combined the red channel from the first image, the green channel from the second image, and the blue channel from the fifth image would be allowed, but one which mapped the red channel of the first image to red, the red channel of the second image to green, and the red channel of the third image to blue would not be created.

These restrictions also apply to calculated variations (but not statistical variations!), so some of the simpler calculated image configurations can be used with this modified configuration (as well as the second variation, further along in this article).

In Windows Explorer, make a copy of the Input-Multitemporal_RGB-5F-TIFF.xml file that you created in the previous section. Name the copy Input-Multitemporal_RGB-5F-ME1-TIFF.xml
Launch your text editor and open the new file.
In the <Metadata> section:
1. In the <Name> section, change the text to read Custom: Steve Smeed Multitemporal RGB - Mutually-Exclusive Variation 1 (TIFF - Up To 5 Files).
2. In the <Description> section, change the text to This is a variation of the regular "Multitemporal RGB" input configuration which only allows one each of red, green, and blue bands from the source images to be used in composites. That is, it prevents images such as one in which three different red channels are mapped to red, green, and blue in a single output image.
3. If you like, enter the date/time, in XML format in the <Created> and/or <LastModified> sections.
Inside each <SpectralBand> block, just before the </SpectralBand> tag, add four <MutuallyExclusiveBand> blocks, which contain the names of the other channels of the same colour.

For example, the modified R01 <SpectralBand> block would look like this, because the <MutuallyExclusiveBand> blocks would be intended to exclude R02, R03, R04, and R05 (the other red channels).

<SpectralBand>

<BandName>

R01

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-01.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<SpecialBandName>

Red

</SpecialBandName>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

<MutuallyExclusiveBand>

R02

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

R03

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

R04

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

R05

</MutuallyExclusiveBand>

</SpectralBand>

...whereas the R04 definition would also contain four <MutuallyExclusiveBand> blocks, but they would be R01, R02, R03, and R05.

Just to be clear, you need to create four <MutuallyExclusiveBand> blocks in each <SpectralBand> block, and since there are fifteen <SpectralBand> blocks, that means a total of 60 <MutuallyExclusiveBand> blocks in the file.
Save the file and exit.
If you closed out of TMSB previously, launch the GUI again. Otherwise, from the Tools menu, select Reload Configuration Files. In the Input Configuration tab, the drop-down should now contain a new entry (named Custom: Steve Smeed Multitemporal RGB - Mutually-Exclusive Variation 1 (TIFF - Up To 5 Files), if you followed all of the instructions correctly).
If you like, give the new configuration a try, using the example input image set available at the bottom of this page.

Mutually-Exclusive Channels Variation 2

This customization acts in a manner opposite to the previous example: it only allows combinations of channels that represent the same colour (for instance, all blue channels).

This variation can be safely used with any of the calculated variations, but it is actually so restrictive that unless you use five source images, only greyscale calculated images will be output when using any of the calculated processing configurations included with TMSB^[1].

In Windows Explorer, make a copy of the Input-Multitemporal_RGB-5F-TIFF.xml file that you created in the first section of this article. Name the copy Input-Multitemporal_RGB-5F-ME2-TIFF.xml
Launch your text editor and open the new file.
In the <Metadata> section:
1. In the <Name> section, change the text to read Custom: Steve Smeed Multitemporal RGB - Mutually-Exclusive Variation 2 (TIFF - Up To 5 Files).
2. In the <Description> section, change the text to This is a variation of the regular "Multitemporal RGB" input configuration which only allows combinations in which all bands used in an output image are from the same source channel in the input images (e.g. all of them are the red channel in their respective source images).
3. If you like, enter the date/time, in XML format in the <Created> and/or <LastModified> sections.
Inside each <SpectralBand> block, just before the </SpectralBand> tag, add ten <MutuallyExclusiveBand> blocks, which contain the names of every channel which is not the same colour.

For example, the modified R01 <SpectralBand> block would look like this, because the <MutuallyExclusiveBand> blocks would be intended to exclude all of the green and blue bands. The R02, R03, R04, and R05 blocks would contain the same ten <MutuallyExclusiveBand> blocks.

<SpectralBand>

<BandName>

R01

</BandName>

<DataSources>

<DataSource>

<FileName>

RGB-01.TIF

</FileName>

<Plane>

1

</Plane>

</DataSource>

</DataSources>

<BandCategory>

HumanVisible

</BandCategory>

<SpecialBandName>

Red

</SpecialBandName>

<UseAsLuminanceChannel>

false

</UseAsLuminanceChannel>

<UseAsThreeChannelChannel>

true

</UseAsThreeChannelChannel>

<MutuallyExclusiveBand>

G01

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

G02

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

G03

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

G04

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

G05

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

B01

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

<MutuallyExclusiveBand>

B02

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

B03

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

B04

</MutuallyExclusiveBand>

<MutuallyExclusiveBand>

B05

</MutuallyExclusiveBand>

</SpectralBand>

...whereas the green channel definitions (G01, and so on) would replace the first five of these with <MutuallyExclusiveBand> blocks for R01, R02, and so on, but retain the second five (which exclude the blue bands).

Just to be clear, you need to create ten <MutuallyExclusiveBand> blocks in each <SpectralBand> block, and since there are fifteen <SpectralBand> blocks, that means a total of 150 <MutuallyExclusiveBand> blocks in the file.
Save the file and exit.
If you closed out of TMSB previously, launch the GUI again. Otherwise, from the Tools menu, select Reload Configuration Files. In the Input Configuration tab, the drop-down should now contain a new entry (named Custom: Steve Smeed Multitemporal RGB - Mutually-Exclusive Variation 2 (TIFF - Up To 5 Files), if you followed all of the instructions correctly).
If you like, give the new configuration a try, using the example input image set available at the bottom of this page.

Download
File	Size	Version	Release Date	Author
A River in Oregon (Multitemporal RGB)	9 MiB	1.0	2011-04-17	Ben Lincoln
A sample image set for use with the TMSB tutorial "Steve Smeed Multitemporal Processing". This set will generate output images that occupy about 2.5 MiB on disk for each variation in the selected processing configuration.

Footnotes

Because unless the <GenerateThreeChannelOtherPermutations> value is set to true, a minimum of five distinct source bands are required in order for there to be enough variety for colour images to result, and by only allowing channels of the same colour to be combined, the maximum number of them in a single image will be five (since there are five source images in the configuration).