Sunday, August 26, 2007

Suppose you want to sample regularly along a line...

Line profiles are useful tools. They allow you to get a sense of the intensities in the image, specifically, under the line itself. When attempting to determine the resolution of a system, line phantoms are often used; these phantoms can be seen in almost any camera review at dpreview. Using a line profile tool, you can get a sense of the amount of contrast at the resolution extinction point of the detector; ie, the peak to peak distance determines the pixel-by-pixel resolution, and the peak to trough distance determines the contrast at that resolution.

I recently wrote one of these, and became very frustrated with the lack of straightforward code to do so. So, here's my code, hopefully to save someone else the headache.

First things first-- you need to define where pixels are located. One convention has the pixels at the intersections of lines on a grid. This convention is most commonly used in math applications, solutions of PDEs on grids, and so forth. Another convention holds that the pixels are in the space inside the intersections of the pixels; that is, the grid designates the borders of the pixels, and the interior space of the grid is the location of the intensity value of a given pixel. This convention more commonly reflects the actual nature of a CCD or CMOS imager (those used in digital cameras). The flux of light across a certain element is measured in a process that converts photons into electrons, and those electrons are then converted to a digital signal via an analog/digital converter. Thus, this second convention more accurately reflects the physical reality from which the pixel grid was derived.

The implications of the selection of convention have to do with how many pixels you will sample to determine the value of points along the line. If you choose the first convention, many common formulae for the interpolation of data on a grid (nearest neighbor, linear, bicubi, Lanczos) are all available to you in their conventional forms. If you choose the second convention, while perhaps more accurately reflecting reality, the formulae all have to be converted into this new convention. Thus, I opt for the first, if only so that I can keep with mathematical convention and not rewrite everyone's formulae.

So, once that decision is out of the way, the line itself must be drawn. Presumably, the user provides start and end points, and it is up to you to sample along the line. You may be tempted to use a Bresenham approach, especially if you come from a graphics background. However, remember that we're trying to get numbers that reflect reality, we are not drawing and creating our own reality. The Bresenham approach is useful for drawing, but not for sampling, because you lose the actual floating-space locations of pixels along the line. Not only that, the division of values by the delta-Y term requires the use of a special case when the line is drawn entirely vertically, and avoiding special cases makes programming easier.

The code to sample along the line, then, will depend on using the length of the hypotenuse to determine the change in x and y that should be performed for each pixel. The code looks like:

float dxPrime = (mCurrEndPoint.X - mCurrStartPoint.X);

float
dyPrime = (mCurrEndPoint.Y - mCurrStartPoint.Y);
float
hyp = (float)Math.Sqrt(dxPrime * dxPrime + dyPrime * dyPrime);

if
(hyp > 0) {
float
dx = dxPrime / hyp;

float
dy = dyPrime / hyp;
float
[] xcoord = new float[(int)Math.Ceiling(hyp)];

float
[] ycoord = new float[(int)Math.Ceiling(hyp)];

xcoord[0] = (int)mCurrStartPoint.X;
ycoord[0] = (int)mCurrStartPoint.Y;

int
i = 1;
for
(i = 1; i < hyp; i++) {

xcoord[i] = xcoord[i - 1] + dx;

ycoord[i] = ycoord[i - 1] + dy;
}
}



Now that you've sampled, you need to get intensity values along those lines. There are three approaches.

Nearest neighbor (just choose the closest pixel as the value to use, cheapest in time and writing, worst accuracy):
mLineVals = new float[(int)Math.Ceiling(hyp)];

int
xval, yval;
float
a, b;
float
c, d, e, f, g, h; //for bicubic

for (i = 0; i < hyp; i++) {

xval = (int)Math.Floor(xcoord[i]+0.5f);

yval = (int)Math.Floor(ycoord[i]+0.5f);

mLineVals[i] = inImage.GetPixelAt(xval, yval);
}



Linear Interpolation (chooses the four closest pixels as the values to use, second cheapest in time and a bit more accurate):
mLineVals = new float[(int)Math.Ceiling(hyp)];

int
xval, yval;
float
a, b;
float
c, d, e, f, g, h; //for bicubic

for (i = 0; i < hyp; i++) {

xval = (int)Math.Floor(xcoord[i]);
yval = (int)Math.Floor(ycoord[i]);

a = xcoord[i] - (float)xval;
b = ycoord[i] - (float)yval;

if
(xval >= 0 && xval+1 < inImage.XSize &&

yval >= 0 && yval+1 < inImage.YSize) {

mLineVals[i] =
(
1 - a) * (1 - b) * inImage.GetPixelAt(xval, yval) +

a * (1 - b) * inImage.GetPixelAt(xval + 1, yval) +
(
1 - a) * b * inImage.GetPixelAt(xval, yval + 1) +

a * b * inImage.GetPixelAt(xval + 1, yval + 1);

}
}





Bicubic interpolation (chooses the 16 closest pixels as the values to use, most expensive in time but still fast enough on modern hardware, and the smoothest of all, and what I use):

mLineVals = new float[(int)Math.Ceiling(hyp)];

int
xval, yval;
float
a, b;
float
c, d, e, f, g, h; //for bicubic

for (i = 0; i < hyp; i++) {

xval = (int)Math.Floor(xcoord[i]);
yval = (int)Math.Floor(ycoord[i]);

a = xcoord[i] - (float)xval;
b = ycoord[i] - (float)yval;

c = 1.0f - a;
d = 1.0f - b;

e = a + 1.0f;//backwards distance is a + 1 pixel
f = b + 1.0f;

g = 2.0f - a;//forwards distance is 2 pixels - a;
h = 2.0f - b;

if
(xval-1 >= 0 && xval + 2 < inImage.XSize &&

yval-1 >= 0 && yval + 2 < inImage.YSize) {

mLineVals[i] = //inImage.GetPixelAt(xval, yval);
(1 - 2 * a * a + a * a * a) * (1 - 2 * b * b + b * b * b) * inImage.GetPixelAt(xval, yval) +
(
1 - 2 * c * c + c * c * c) * (1 - 2 * b * b + b * b * b) * inImage.GetPixelAt(xval + 1, yval) +
(
1 - 2 * a * a + a * a * a) * (1 - 2 * d * d + d * d * d) * inImage.GetPixelAt(xval, yval + 1) +
(
1 - 2 * c * c + c * c * c) * (1 - 2 * d * d + d * d * d) * inImage.GetPixelAt(xval + 1, yval + 1) +

//top line
(4 - 8 * e + 5 * e * e - e * e * e) * (4 - 8 * f + 5 * f * f - f * f * f) * inImage.GetPixelAt(xval - 1, yval - 1) +
(
1 - 2 * a * a + a * a * a) * (4 - 8 * f + 5 * f * f - f * f * f) * inImage.GetPixelAt(xval, yval - 1) +
(
1 - 2 * c * c + c * c * c) * (4 - 8 * f + 5 * f * f - f * f * f) * inImage.GetPixelAt(xval + 1, yval - 1) +
(
4 - 8 * g + 5 * g * g - g * g * g) * (4 - 8 * f + 5 * f * f - f * f * f) * inImage.GetPixelAt(xval + 2, yval - 1) +

//bottom line
(4 - 8 * e + 5 * e * e - e * e * e) * (4 - 8 * h + 5 * h * h - h * h * h) * inImage.GetPixelAt(xval - 1, yval + 2) +
(
1 - 2 * a * a + a * a * a) * (4 - 8 * h + 5 * h * h - h * h * h) * inImage.GetPixelAt(xval, yval + 2) +
(
1 - 2 * c * c + c * c * c) * (4 - 8 * h + 5 * h * h - h * h * h) * inImage.GetPixelAt(xval + 1, yval + 2) +
(
4 - 8 * g + 5 * g * g - g * g * g) * (4 - 8 * h + 5 * h * h - h * h * h) * inImage.GetPixelAt(xval + 2, yval + 2) +

//left side
(4 - 8 * e + 5 * e * e - e * e * e) * (1 - 2 * b * b + b * b * b) * inImage.GetPixelAt(xval - 1, yval) +
(
4 - 8 * e + 5 * e * e - e * e * e) * (1 - 2 * d * d + d * d * d) * inImage.GetPixelAt(xval - 1, yval + 1) +

//right side
(4 - 8 * g + 5 * g * g - g * g * g) * (1 - 2 * b * b + b * b * b) * inImage.GetPixelAt(xval + 2, yval) +
(
4 - 8 * g + 5 * g * g - g * g * g) * (1 - 2 * d * d + d * d * d) * inImage.GetPixelAt(xval + 2, yval + 1);

}
}




(there's a fourth, Lanczos, but I'm ignoring it, because bicubic is good enough, I think).

And there you have it. From there, you need to be able to draw the line profile. However, that drawing code is specific to each platform. To do it in C#, I would have a class that inherits from PictureBox and overwrites the draw routine, like so:


private
void JustPaint() {

//start from 1% of the edge and go to 99% of the edge
//round down for the start, up for the end, but want to make sure leave some space

if
(mValues == null) return; //no drawing yet

if (mValues.Length == 0) return; //no drawing yet


if
(Image == null || mOldWidth != Width || mOldHeight != Height)

Image = new Bitmap(Width, Height);
Graphics g = Graphics.FromImage(Image);

mOldWidth = Width;
mOldHeight = Height;

int
theStart = (int)((float)Width * 0.01f);

int
theEnd = (int)((float)Width * 0.99f + 0.5f);

int
theMaxHeight = (int)((float)Height * 0.01f);

int
theMinHeight = (int)((float)Height * 0.99f);

float
theHeightRange = (int)(theMinHeight - theMaxHeight);

g.Clear(Color.White);

//rescale the values to be between 0 and 1
//note that mRescale is set with mLineVals, so that when this code
//is called during redraws that are not relevant to line moving, less work is done

//the x step is from theStart to theEnd, scaled by the length of the line.
//clearly, we drop some points for long lines.
//so, we have to get a sampling rate from mRescaled.length vs theEnd-theStart
float theRange = theEnd - theStart;
float
theSample = 1;

float
theDrawStep = 1;
if
(theRange > mValues.Length) {

theSample = 1.0f;
theDrawStep = (float)theRange / (float)mValues.Length;
}
else {

theSample = (float)mValues.Length / (float)theRange;
theDrawStep = 1.0f; //every pixel gets hit

}
float
i = theSample;
float
theDrawStart = theStart;

float
theDrawSpot = (theStart) + (theDrawStep);
int
intDrawStart = (int)(theDrawStart + 0.5f);

int
intDrawSpot = (int)(theDrawSpot + 0.5f);
while
((int)(i+0.5f) < mValues.Length){

g.DrawLine(mBluePen,
new
Point((int)(theDrawStart + 0.5f),

theMinHeight - (int)(mValues[(int)((i - theSample)+0.5f)] * theHeightRange + 0.5f)),

new
Point((int)(theDrawSpot + 0.5f),
theMinHeight - (int)(mValues[(int)(i+0.5f)] * theHeightRange + 0.5f)));

theDrawStart = theDrawSpot;
theDrawSpot += theDrawStep;

i += theSample;
}

}


There you have it. Assuming you can draw a line, you can now display a graph of that line.

Monday, August 20, 2007

Some Flower Pictures

I see a lot of flower pictures turning up and around on the web. Some are pretty simple things, with the flower straight on. Then you do something where you put the flower to the side, and maybe you get some nice light on it, and everything's awesome. Trick is, to make the flower image stand out as a unique image, as one that will make people sit up and take notice, rather than say, "Ah, good, you've learned something in your photography class."

There are, of course, things you can do to get the flower to be interesting. One good ploy is to have a bug on it, like so:

That's pretty neat (or at least, I like to think so), but it's not about the flower, it's about the bug and the flower. It might also have depth of field issues, if you want to be a total perfectionist, but that's another subject for another time.

But, eventually, there is something in the beauty of a flower that can be pulled out. For those of you shooting with Nikons, I highly recommend the 60mm 2.8 macro lens, as it can give ridiculously sharp results, even after having been dropped on concrete (my bad).

As an example:


Unfortunately, these small images don't do the detail in the image justice. The rightmost petal, up close, looks like:


But how do you do it? Here's the simple steps:

1) Get a tripod. A nice sturdy one. There are loads of tripods out in the world, and plenty of reviews. The Manfrotto 3021 seems to be pretty popular; that's what I use, but it's by no means perfect.
2) Get a flower. Preferably one you like.
3) Arrange the flower and the light in such a way that some aspect of the flower is emphasized, some part of it you want to expose to the viewer, so that it's more than just a flower, but also has your expression on it. This step is really where the 'art' part comes in, and you can futz with this for a long time. Suffice it to say, the above shots all have several decisions that went into making them. I chose black and white for the rose, for instance, because I really like the detail in the shot, and the way the flower petals overlap with one another, coupled with the softness of the light, makes for an image I like to look at. Others might want to emphasize the color, or spray some mist on it, or something.
4) Set your camera to a timer. This step ensures that you won't affect the camera or the tripod when the shot is taken; if you have your hands on the thing, twitching might move the camera, or the force of pressing the button might change your composition.
5) If you're shooting digital, go to the lowest ISO you can. You're on a tripod; there's no reason not to take your time.
6) Check your depth of field. One of the reasons I like true macro lenses is the ability to go to f/22 or something ridiculous; except, it's not ridiculous when close up. The depth of field shrinks the closer you focus on something, and if you're only an inch or two away from your subject, an f/4 aperture is paper thin. The selection of the aperture is really part of composition from step 3, and should be given due consideration (and considerable experimentation when learning).

I prefer natural light, frankly, because I'm lazy. I also happen to think flowers look just fine in natural light, but I'm not the final word.

(all of these shots, btw, can be found on http://www.markmroden.com)

Sunday, August 12, 2007

One year, w00t!

One year anniversary! Yay!

Check out the photos from Granrose's 80th, in Pottsville, where we celebrated:

http://www.markmroden.com/gallery/3292544#182808129

Friday, August 3, 2007

Since when does the government get to fire me?

So I read this article here in this morning's LA Times, and I am pretty angry about it.

Since when does the Federal Government have the ability to fire me? Am I now employed by them, or employed by my employer?

How do they know that workers with inconsistencies in their SSN's and their W-2's are, in fact, illegal? Have they done any studies to show the correlation? It appears, from the article, that two thirds of people who have these discrepancies are, in fact, native born, so how does that square up?

What's the process for correcting the problem? Will it end up being something as messed up as Medicare, part D, which requires a deep and intimate knowledge of Parseval's Theorem and it's solution in some kind of n-dimensional Hilbert space? I'm not even sure I know if those mathematical concepts go together, but I'm pretty sure they do, like I'm pretty sure that since I've been working 3 jobs to support myself in graduate school, I'm gonna be one of the flagged workers.

This administration started out with claims that all come from Reagan, that we're going to make government smaller, and that more government is the problem and not the solution. It seems to have caught up to them that all of the social problems they want to fix, be it showing nipples at football games, underage sex, illegal aliens, underage sex with illegal aliens while watching naked football, or any other hobgoblins that keep such God-fearing busybodies awake at night, all require some kind of government intervention.

So what's it gonna be, Republicans? More government to fix your perceived ills of society, or less government and keeping your promises?

Wednesday, August 1, 2007

"My Recent Documents" rant

So, I like to get the images I'm working on at the moment via the Recent Documentations option in the directory tree.

Unfortunately, for reasons that are beyond me, this dialog is, by default, sorted in ascending date order, instead of descending date order, so that you get the oldest documents you've worked with, instead of the newest.

Like anyone has _ever_ wanted that feature in a Recent Documents list.  

While I'm on the subject of this dialog box, how hard is it to save the state of the damn thing from opening to opening?  If I'm using the 'details' view, the next time I open the dialog, I want the 'details' view up.  Not difficult.  Why revert to some standard template that I don't want?  Would it be possible to get around this by keeping a copy of the dialog box around, so that I can open and close it at will, but flush the previous selection, and maybe copy settings to another dialog, so I can have different options for save and load?

And why don't other programs do that, if it's so easy?

Walking across America

I would walk five hundred miles
and I would walk five hundred more
Just to be the man who walked a thousand miles
To end this stupid war.

http://www.marchforpeace.info/

With apologies to The Proclaimers.

Return to C++!

Finally, I get to return to C++.

Not that C# is all bad, mind you. It's pretty decent when it comes to UI and the day-to-day meanderings and wanderings of what needs to happen to keep a program running. Perhaps its nicest feature is the ease with which I can attach an outside dll and use that, because now that I need to make sure I have some serious horsepower, C# is stepping out of the way.

One thing that irritates me: how hard it is to debug unmanaged C++ from inside a managed C# environment. But mmr, I hear you cry, the whole freakin' point of C# is that everything can be managed for you, and the IDE will hold your hand. But sometimes, I don't want my hand held, I want to run free. Free as the wind blows, free as the grass grows.

To attach a dll, just do this:
using System.Runtime.InteropServices;

[
DllImport("mydll.dll")]

public static extern
void MyFunc(int inImagePtr, int outImagePtr, int inYSize, int inXSize);



private
unsafe ushort[] UseMyFunc(ushort[] inChannelData, int inRows, int inColumns) {

ushort[] theOutData = new ushort[inChannelData.Length];

fixed (ushort* inBufferPtr = &inChannelData[0]) {
fixed (ushort* outBufferPtr = &theOutData[0]) {

MyFunc((int)inBufferPtr, (int)outBufferPtr, inRows, inColumns);
}
}


return
theOutData;
}
}


Essentially, you need to have a function exposed by the dll, and then you can use that function. So you get the speed of C++ inside the function, but the handiness of C# outside the function. Not a bad combination.

Friday, July 27, 2007

Mark's Law of Dancing Music

Here is Mark's Law of Dancing Music, or Music To Dance To:

No song which leads with a guitar, or multiple guitars, be they analog or electric, is a dance song. Period. Do not try to argue with this law, your argument flops dying on the floor like a fish gasping for water, much like how you must dance.

Corollary: Spanish music is exempted.
Corollary to the Corollary: If it is Spanish music, there is a dance, and one specific dance, which may be danced to that song, be it a Mambo, Salsa, etc. If you do not know that dance, sit down. You will simply continue to embarrass yourself, as you did while squawking to the chicken dance.

Wedding DJ's, take heed: stop playing songs with guitars in them as primary instruments. Especially if the song has a solo. No one knows how to dance to these songs, because they are undancable to all but either the most skilled (who aren't at your wedding), the most drunk (who don't care what's being played), or the most painfully inept (which no one really wants to watch, except to be thankful that it is not them). No more Def Leoppard, Metallica, Rolling Stones, Offspring, or anything like that, unless the bride is specifically paying you for that song. In which case, it is she who needs to consult the rule, not you. Grooms don't count.

That is all.

Thursday, July 26, 2007

Nikon: d40x, or d80? And which lens?

I shoot Nikon when I take pictures. My progression through the various cameras and so forth basically went like this:

1) point and shoot, 2 mpix. This was the Canon sd100 (or something like that), a little guy the size of a deck of cards. Nice camera, not particularly cheap but cheap for the time, but slow as a dog. I could not take a picture within a few seconds, and when I did, the exposure time had to be very long to capture what I thought was a shot in decent light.
2) Canon S1 IS. I liked this camera, it had all kinds of bells and whistles, and with a macro lens attached, could let me take shots like:



Very cool, and very close in, since that little guy could not have been more than a centimeter high.

Problem was, the S1 IS (and presumably it's more recent cousin, the Canon S5 IS) suffered from the same shutter lag problem, as well as requiring a tripod for shots made in all but bright sunlight. The noise was bad enough that I couldn't go past ISO 100, and so the exposure times would just be too high to be fixed by the image stablization.

So I then decided to buy an SLR. It came down to the Canon Rebel (first generation), or the Nikon D70. Based on specs and noise profiles from dpreview.com, as well as the couple hundred dollar difference in price, it was pretty clear to me that the Canon was the way to go. I went to my local Ritz (Wolf?) Camera, just to try them out, and the Nikon was clearly the better way to go. It felt like an actual tool, a black and meaningful piece of equipment used to do serious things. The Canon felt cheap and chintzy, and was too small for my gargantuan paws. So, to the Nikon!

I've since moved on to a D200, with a variety of prime and zoom lenses, and really couldn't be happier. I have realized that I needed to change equipment as the needs I had weren't being met by my previous equipment; but, since changing to an SLR, I now have equipment that meets or exceeds my needs, so I have more to learn. Which is good, because it keeps me interested in the hobby.

Recently, a friend of mine asked me for links about the D40x, and this is what I told her. I replicate it because I actually get asked this question quite a bit:

=========================

Here's the d40x:
http://www.bhphotovideo.com/c/product/494398-REG/Nikon__D40x_Digital_Camera_with.html
(that's with a 2gb card)

And the lens I recommend:
http://www.bhphotovideo.com/c/product/324190-USA/Nikon_2149_18_70mm_f_3_5_4_5_G_AFS_ED_IF.html

If you're thinking of getting some of the older lenses, or the prime lenses, this might not be the right camera.

Almost all of the pictures I take that make people drool, like this one:
http://www.markmroden.com/gallery/3165979#174004648
are done with a prime lens.

Why is that image worth noting? From a technical perspective (whether or not you think the subject matter is emotionally grabbing or not, just the technical), note a few things:
1) action is frozen
2) he is in sharp focus, the grass behind him is blurred nicely, and blurs more the further things are to the lens

Both of those feats are accomplished by what's known as a large aperture. In this case, the lens was an 85mm 1.8 lens, which can be had for ~$350 or so, and was set to f/2.0. By comparison, most non-professional zooms are between 3.5 and 5.6 (the one I've recommended is 3.5-4.5), which means that there is much less subject isolation from background (ie, more stuff is in focus), and the camera needs more time to gather light for a decent exposure, so you may not be able to freeze action as well.

If you think that you may want to explore this direction, not just with zooms, but also primes, then you may want to consider a d80. It costs more:
http://www.bhphotovideo.com/c/product/449061-REG/Nikon_25412_D80_SLR_Digital_Camera.html
but you can pair it with this lens:
http://www.bhphotovideo.com/c/product/247091-USA/Nikon_2137_Normal_AF_Nikkor_50mm.html
which is perhaps the best lens in the nikon lineup, for the price.

If you can stretch it, I really recommend the d80 over the d40, if only because you can then expand into lenses made before the last 7 years. If you really see yourself needing a zoom and not too concerned with subject isolation, then the d40 is probably fine, but as of this moment, lacks much in the way of expansion to those older lenses.

Having said all of that, though, here's some reviews.
http://www.bythom.com/d40review.htm
http://www.bythom.com/1870lens.htm
http://www.bythom.com/d80review.htm
http://www.naturfotograf.com/lens_norm.html (and look for the 50mm 1.8 review)
http://www.naturfotograf.com/lens_zoom_01.html#AFS18-70G

I hope that helps.

Here's some museum shots that were also taken with the 85 mm:
http://www.markmroden.com/gallery/3166046

Note that, by the second reviewer's scale, the 85mm 1.8 that I use is a 4, same score as the 18-70.

Wednesday, July 25, 2007

Random numbers in C#

I've learned, over the last few days, a very disturbing thing.

Say you want to make a random number. Or pseudorandom, since most of us don't have radiation sources and the readers necessary to record such noise.

Now, say you're using C#.

Yay! They've made a Random class. You can just do:

Random r = new Random();
int theRandomNumber = r.Next();
int anotherRandomNumber = r.Next();


Yay! Problem solved!

But wait.

Suppose you've put this code in, say, a constructor, one that gets called a few times. Now you have a problem; it turns out, the constructor will make the exact same random number generator, with the exact same seed value! How do I know this? Because I just lost three days and a huge number of hair follicles to this problem.

Here's the deal:
1) MainObject makes, say, an array of ContainedObjects.
2) ContainedObjects all call the random number generator as above during startup.
3) Each random number generator has the same seed.
4) So, the 'random' sequence is mimicked in each of the ContainedObjects.

Technically, I suppose it's still random, much like a random number generator that always returns 4.

How do you fix this problem?

1) Seed with the tick count. Apparently, this is a bad idea, because it doesn't work consistently. After testing on three different machines with three different clock speeds, using DateTime.Now.Milliseconds may or may not return the exact same milliseconds. Not sufficiently random, not a solution.
2) Uptime of the computer. I have no idea how to get this.
3) Have the MainObject hold the random number generator. However, it now needs to be a member; how many times do you initialize an array of ContainedObjects? If it's more than once, but the random number generator is initialized the same way again, then you're back to the same problem. Heaven forfend that you need multiple random numbers throughout the place; you'll need to make your random number generator global! Global variables make my skin crawl.
4) I don't know. I'm currently doing option number 3, and it hurts my head, but at least it's just a member in MainObject.

What's the lesson in all of this?

Be careful of random number generators you don't write yourself. One of Knuth's exercises is to check the random number generators in programs in the nearest computer lab; apparently, the C# guys didn't check out what he had to say.

So you want to save and load 16 bit images in C#.

Say, for instance, that you're working for a medical device company, and you need to write in C#.

How do you load in the images from the device?

For those of you who don't know, images from medical devices are usually 16 bit grayscale images, maybe sometimes 15. That's ushort, or UInt16, and making it a short or SInt16 can sometimes have dire consequences. Dire, as in, people die, because this is a medical device we're making. That's the first thing-- our images are 16 bit graqyscale, from now until 18 bit or 20 bit imagers are made. For those of you who are curious, I'll end this post with a small discussion about what bit depth really means, but for now, let's just say you want to display the damn thing.

The C# (or .NET) environment has a 16 bit grayscale bitmap class. If it worked, it would be ideal for us, because we could display it right after loading it. Alas, it is not to be; the 16 bit class doesn't work. Microsoft's documentation on the subject of PixelFormats is woefully inadequate; even after that, though, just try using a 16 bit grayscale. Watch it fail with no real error code.

Microsoft, as a side note, really has no idea what goes into this kind of application, as they demonstrate here. I'm not sure they really should, either; it's not their purview to mess with medical devices, but it is annoying that they seem to think they know anything about the subject.

So, we need an outside library to load our images.

If you're in the medical imaging field, then you are using DICOM. DICOM is a standard format for storing and retrieving medical image data, either locally or from a central server, that should be entirely independent of vendor. Really, this is the agreed-upon format for medical images, and if you're making a serious medical imaging app, you need this format, period.

To use it, I use dcmtk. I don't claim to be an expert; luckily, one of my coworkers knows a lot about DICOM, and wrote me a little reader code snippet for our appplication. I could show it to you, but the problem is, each application is different, and uses their own DICOM tags. Plus, there may be IP issues; if I get to post my code in its entirety, then I can show you the DICOM load function.

For me, I really don't care what format the data comes in, just so long as I can get my hands on numbers. I just want the data, the width, and the height. Depth is good for 3D images as well. Everything else can be figured out. So, my internal representation of an image is quite simple, and you can see it here:



namespace
YourNamespaceHere {

public class
ImageContainer {
private
bool mNulled;
public
bool Nulled {

get { return mNulled; }
}

private
ushort[] mData;

public
ushort[] Data {//note that this returns a reference, not a copy
get { return mData; }
}


private
int mXSize, mYSize;
public
int XSize {

get { return mXSize; }
}

public
int YSize {

get { return mYSize; }
}

private
int mAssessmentID;

public
int AssessmentID {
get { return mAssessmentID; }
}


private
String mName;
public
String Name {
get {

if
(mAssessmentID < 0) {
if
(mkVp < 0 && mmAs < 0) {

return
mName;
}
else {
return
mName + " " + mkVp.ToString() + " " + mmAs.ToString();
}
}
else {

return
"Assessment Image " + mAssessmentID;
}
}
}

private
int mkVp;

public
int kVp {
get { return mkVp; }

set { mkVp = value; }
}

private
int mmAs;

public
int mAs{
get { return mmAs; }

set { mmAs = value; }
}


//private DicomHeader
//private Annotations
//private string openedas, so that save can use the default type

public
ImageContainer(String inNullString) {

System.Console.WriteLine("Null image constructor used.");
mData = null;

mXSize = -1;
mYSize = -1;
mName = "Null Image";

mkVp = -1;
mmAs = -1;
mAssessmentID = -1;

mNulled = true;
}


//generic constructor
//should get an 'opened as' tag as well, plus perhaps dicom header info if present.
public ImageContainer(ushort[] inData, int inXSize, int inYSize, String inName, int inAssessmentID) {

mData = inData;//note that this line means that this structure is responsible
//for the memory that's passed to it!
mXSize = inXSize;
mYSize = inYSize;

mName = inName;
mkVp = -1;
mmAs = -1;

mAssessmentID = inAssessmentID;
mNulled = false;
}


public
int GetPixelAt(int inX, int inY) {

return
mData[inY * mXSize + inX];
}


public
Rectangle ImageRect {
get { return new Rectangle(0, 0, XSize, YSize); }
}

}
}




I'm sorry the formatting blows, but I guess that's just Blogger for you. If anyone knows other tools, please let me know.

Note a few things:
1) I'm not allowing for _any_ data processing methods. Those go elsewhere.
2) I'm not allowing for _any_ display methods. Those also go elsewhere.
3) Essentially, this is just a name, width, height, and data. That's it, that's all I need.
4) I'm using get/set methods here for data access. That's the way it's done in C#. I'm not entirely sure why, but I'm told that it makes the compiler happier.
5) I will only use the getPixel(x, y) method very infrequently, and NEVER to march through the image. It's just way too slow; the overhead for calling a function in C# is ridiculous.

So, we have to fill that with data. For that, I will use the Free Image API, with their handily provided C# wrapper. You may have some issues compiling the C# wrapper; if so, you may need to change some classes to structs to make the compiler work. Don't worry, we're not even going to touch the methods that it complains about.

To Load, we will need to make sure:
1) We call the methods from the FreeImageAPI properly, ie:

UInt32 theBitmap = FreeImage.Load(FreeImage.GetFileType(theCompleteName, 0), theCompleteName, 0);


2) We go into unsafe code in order to read from the pointers. Email me if you want this code; blogger's formatting for code makes it entirely unreadable. There's sample code in the FreeImageAPI about transferring data from their bitmap into our image. It should be fairly straightforward, I'd think, but the casting may cause you some headaches.
3) Make SURE you unload whatever you load, ie:

FreeImage.Unload(theBitmap);

Otherwise, you will leak a block of memory the size of the bitmap (which can be quite substantial).
4) Call GC.Collect() and GC.WaitForPendingNotifiers() in order to clean up any blocks lying around. If you're loading significantly large images and then making copies or running FFT's on them or anything else that could be memory intensive, waiting for the GC to collect memory can sometimes be a bad idea, because it will cause you to thrash. So, clean up after yourself, and call the GC. (It's a pet peeve of mine that C# seems to encourage laziness with the way the GC works, but then doesn't always clean up allocated memory).

How about saving?

Pretty much the same thing, create a bitmap, move your data into it, and then make sure you unload the bitmap. The incantation for saving is:
FreeImageAPI.FreeImage.Save(FreeImageAPI.FREE_IMAGE_FORMAT.FIF_TIFF, theBitmap, inDirectory + "\\" + inName, 0);

This code is if you're saving to TIFF format, and theBitmap is the FreeImage image you've made and put your data into.

Next time: Display!


Bit depth, you say? What about it? Here's the deal: Medical devices, whether they have truly 16 bits or not, often claim that they do, and use ushorts as their type. Sometimes, you'll get cross mixing with 15bit data, and have to make sure that you're displaying the image properly. As to whether or not your device is using all 16 bits of dynamic range, I think you'll have to answer that yourself (and it's not an easy thing to answer).

First things first: why?

So, with all of that having been said, why am I writing an image processing application in C#? Because my advisor/boss told me to, in order to match with existing code. The story goes like this:

Once upon a time, there was a lone coder who worked at a company that made x-ray devices. This coder was constantly having the patch the system to account for all kinds of strange bugs, and he would find the patches in various Microsoft libraries. So, in order to keep himself current on all the new technologies and to fix these bugs, whenever Microsoft released a new library, he would add it into the application. He then left the company, his code a Jenga-tower filled with holes and teetering to collapse, a fine structure that only he could understand.

The code, when synced from source control, weighed in a hefty 2 gb. It may be one of the few times I've seen source code larger than the executable (150mb).

Into this I came, bright-eyed and pink-cheeked, to write some image processing code for my PhD thesis in Biomedical Engineering. By working directly for this company, I get direct access to the bits. I can plug an oscilloscope up to the x-ray machine if I wanted to (or if it would help, which it wouldn't) to be able to get real numbers. Just try doing that with a Fuji or Kodak CR machine-- just try. They lock that stuff down so hard, and you have no idea where those numbers came from or what processing has been done on them before. This way, I can know exactly what's been going on to the numbers before they form an image on my screen.

The downside, of course, is that I have to mesh with this Jenga code.

So, I've been learning C# in order to write an image processing application. And currently, once I've learned a bit about it, it's not that bad a language to write for. It really isn't. It's got some real annoyances, and until I read a book on it, my code was as weak as a newborn kitten (but not as cute). It does have some serious drawbacks, though, and these are the things I've learned to get around in order to make this application work.

Who knows-- maybe someday, I'll get to post that application. We'll see if the boss lets me; I know that more eyes will make it suck way less than it does now.

mmr

C# image processing? Ha-ha!

So you want to write an image processing program in C#, do you? Why on earth would you want to do that? I can only think of a few reasons why that would be a good idea:

1) The existing codebase is in C#, and the idea of mixing languages with something more appropriate makes your head ache.
2) You're starting from scratch, and C# is new and shiny and everyone must love it.
3) Your boss told you to.
4) The voices in your head told you to.

Some of these reasons are valid, and some are not. Let's assume, for the sake of argument, that they are valid, and for whatever reason, you're forced to use C#. Some things you should know:

1) C# is slow. Yes, yes, there are claims (such as these) that C# is as fast, or faster. I must tell you, that for the purposes of running image processing applications, C# is slower than C# by a good long margin. In a later post, I'll talk about how much slower, but were talking 200ms for C++ vs 350 ms for C# for a particular algorithm on a 100x100 image. Scale that to a 9 megapixel 16 bit image, and you're looking at a very significant speed difference.

2) C# can be sped up, but usually through unsafe code. Unsafe code means that you lose a lot of what is good about the environment, like run-time checking of almost every error, and really nice debugging tools. C++ used to be unsafe, and we could edit that code on the fly while debugging, but you can no longer edit unsafe code on the fly in Visual Studio 2005. A shame, really, as it will reduce you to stopping and recompiling each time you make a change.

3) C# has a very different object model than C/C++/Java. C and C++ have C# struct type objects, Java has C# reference type objects, and if you don't know the difference, you could get burned really badly.

4) C# does have automatic garbage collection. However, that automatic garbage collection can be really slow when you're making multiple copies of a 9 megapixel 16 bit image, or if you're converting the image to floats or (god forbid) doubles for some interesting math (FFTW, anyone?). Get used to calling GC.Collect() and GC.WaitForPendingNotifiers() a lot.

There's plenty of other things that I've learned while making an image processing application in C#, and these are just the beginning. As I go through the things I've learned, I'll post them here, so that other people won't have to suffer (as much) as I did, or at least, that's the hope.

mmr