Screencast Tutorials

I have been talking about OpenGL tutorials for some time, but I haven't gotten around to them. Recently I've been spending a lot of time with Core Audio, and time permitting, I'm going to do screencasts about it too. University is keeping me busy, but I hope to have the first screencast posted around Christmas. I plan to use GLUT to start, and I might move on to Cocoa from there, but I don't want to complicate things from the outset.

Siggraph is over

Siggraph has come to a close, and I've walked away a little more informed than I was last week.

If there was an OpenGL BOF, I missed it. I was busy with the talks, and missed a number of things I'd liked to have seen. 

I enjoyed the Photon Mapping talk today; While I've been aware of it, and the basic ideas, I'd never looked at it in depth.  I like what I saw, and I'll have to pick up Henrik Jensen's book.  Matt Pharr's book also looks quite good, and I'll probably order both from Amazon when I get home. 

The talk on crowd simulation and perception was also interesting, and I'll have to have a look through the paper.

I picked up "Practical Multi-Projector Display Design", "Tensor Voting" and "Real-Time Collision Detection", each of which should keep me busy for a few weeks. Practical Multi-Projector Display Design should be helpful over the next few weeks as I setup our cave-like environment.

I'm looking forward to getting home, and I hope everyone had a good Siggraph. 

OpenCL, DirectX Compute Shader, CUDA, CAL, Larrabee, OpenGL 3.0, and a neat demo by Jon Olick

A few of the presentations this week have covered Larrabee, both in terms of architecture and conceptually.  A talk today mentioned that Larrabee could submit draw calls to itself, which I hadn't considered before.  Even if it fails to live up as a rasterizer, there will still be plenty of market for it as a compute-enabling device.

A high level overview and sample code of both OpenCL and DirectX Compute Shaders were presented, and I'm fairly excited.  They're quite different approaches, with different goals, and I whole heartedly hope they both work out.

I've used CUDA a bit in the past, but I didn't know until today that AMD had updated their offering from CTM to a new higher-level api called CAL.  I'll have to check it out next time I have an ATi board to play with.

I've already said I was disappointed with what was missing in OpenGL 3.0. What I didn't say is that I'm excited about working with the new extensions and seeing what I can do.

OpenGL 3.0, PhysX

OpenGL 3.0 is a bit of a disappointment. The API promised never materialized. It would have been better recieved by the community if the ARB/Khronos had been up front about issues as they arose. Instead, they decided to keep things quiet, and many developers are quite upset. I'm going to spend a couple of months evaluating it, but I am disappointed and I've started http://osxna.blogspot.com to cover my XNA and D3D projects. (I'm going to keep this blog alive too.)

I went to the PhysX presentation at Siggraph, and asked the presenter about OS X support. His reply was something along the lines of "This question has come up before. I can't say anything official, but we're looking at it." From the way he said it, I took it as a "Yes, but I can't say yes because we haven't officially announced it." CUDA is available on OS X, and PhysX is built on CUDA, so the framework is there. We'll see what happens.

Texture Arrays

We need them in OS X. Please?

Sony Vector Math Library

I've decided to dump my own vector math library and use Sony's vector math library provided on the Bullet svn. 

I had planned to use the C++ library, but it doesn't compile with GCC. I was considering making it portable, but I learned something new about POD types and decided to go with the C interface.

Thus far my experince has been extremely positive, and I'd recommend it to anyone looking for a math library for OpenGL.

Physics

I enjoyed working with PhysX on Windows a few years ago, so when I recently started to look into physics systems for the Mac it was my first choice; Unfortunately, PhysX doesn't support OS X.

I haven't used Bullet before, but I'm going to give it a shot.

Andrew.

GLOSX Quick Trick

If you're using GLUT in OS X and you need to enable VSync you can do it by adding the following two lines of code to your project and including

int vsync = 1;
CGLSetParameter(CGLGetCurrentContext(), kCGLCPSwapInterval, &vsync);

I usually toss it in right after my call to glutCreateWindow.

OpenGL Programming On Mac OS X

I've had this book on my shelf for a while now, referring to it as I need it, but I've finally decided to give it a thorough reading.  The first five chapters provide background material for the graphics layers in OS X, and the API bindings are discussed in chapter 6 and onwards. 

The book is well written and I'd recommend it to any new Mac OpenGL developer.

Andrew

Course Review

My independent study has come to an end, but I'm going to continue exploring a number of the topics I learned about. I've gathered a foothold into current techniques, but there is a lot left to cover, and "current" doesn't last very long.

I found dealing with content available online for free was a bit of a limitation. Beyond that, getting set up took more time than I had expected. I had to put together code for handling the basic math; Luckily I'd already written some of it for some OpenGL work I had done in the past.

Learning The OpenGL Shading Language went fairly quickly, but while I'm able to use it, I still need to look things up from time to time. The most thing I found most difficult had to do with interactions between OpenGL and the shading language.

Do you need to enable fixed function texturing to access texture data in GLSL?
Do you need to enable client states when using vertex buffer objects?
Do you need to enable fixed function lighting when using the fixed function lighting parameters in GLSL?

For a while I wasn't sure if I was encountering driver bugs in the Leopard drivers for my ATi 9700 Mobility, or if I had a bug in my code, or if I simply didn't understand the interactions.

It turns out that some of my confusion was due to driver bugs, and things became much more clear after switching to new hardware.

I've discovered that one of the most fundamental operations in modern real-time rendering is rendering to a texture. That said, it's what you do with the texture that makes things interesting. A number of effects are the result of post-processing rendered information. To that end I wrote a deferred shader backend that stores information required for shading in a number of buffers and composites them in the end. I looked at ( but didn't have time to implement ) writing a water shader, which follows this formula fairly closely. I read the articles in GPU Gems about wave simulation, and felt I got a handle on them, but I didn't understand how to actually do the shading. After writing the deferred shader backend and the reflection/refraction shader, the basic operations were obvious.

Assume your water surface is a plane (this assumption will be broken later, it is used for generating reflection and refraction maps). Render pools of water setting the stencil buffer where ever water fragments pass the stencil-depth test. Scale the scene by Y=-1 and re-render to a texture. Now you've got an inverted version of the scene stored in a texture. Next, render the scene below the water plane and store it in a texture, and you've got your refraction texture. Finally, render the scene normally, with some simulation of the waters surface. When shading the water, simply look up into your reflection and refraction maps to calculate those terms, and potentially use the depth of things below the water plane as a fogging value.
They wont be perfectly accurate because they assumed a planar water surface, but they should be close enough to do a convincing job.

I studied Preetham's Sky Model, and implemented it, but was disappointed with the results of the simplified atmospheric scattering equations. Apparently, Nishita's Sky Model was used for Crysis, and I had a look at it, but didn't get as far as an implementation.

Ambient Occlusion was a rewarding topic to spend time on. I feel like I've managed to learn a great deal about it, and I'm looking forward to trying out some ideas I have to improve the quality of my renders. I'll need to acquire better model and texture data to make it worth my while, but I've come across a handful of models this evening that might be useful in that regard.

Shadow maps were simply infuriating to work with. The technique is quite simple, but there were driver problems I wasn't aware of. I implemented the the technique twice, and spent a good amount of time debugging before trying it on a new machine only to find that it worked.

I'm excited by Variance Shadow Maps, and Exponential Shadow Maps, and I'll probably do an implementation of both techniques. As I understand VSM (and ESM to a lesser extent) suffers from light bleeding, but there are adaptations to reduce the artifacts.

Spending time learning about recent extensions to OpenGL proved rewarding, and has definitely changed the way I write OpenGL code. The extensions should have that effect, since OpenGL is moving towards an object oriented structure and away from its state machine heritage.

I'm going to start exploring DirectX 10, and I'd love to write some OpenGL ES 2.0 code, but I'm really just waiting for GL 3.0.

Andrew.

Reflection and Refraction

I haven't tone mapped the image, but here's a shot of a reflection & refraction demo. The cube map is generated using one of Paul Debevecs light probe images in cross-cube format. Look for the ImageIO post for code to read images using Apple's Image IO toolkit.

Preetham Sky Model

Here's a shot from my implementation of Preethams Sky Model

A few screen shots

Two shots comparing a texture mapped quad and parallax normal mapped quad:

Deferred Shading

I wrote a deferred shading demo to get familiar with the technique. The lack of support for translucent objects and MSAA are non-issues for me. I can write a separate path for translucent objects, and I can do some hackery or switch to DX10 to get MSAA. Prior to writing the demo I had expected that it was a tool I would use most of my rendering needs; Now I look at it as a tool to use in certain situations. I need to play with it further to see if it's generally useful to me.

Screen Space Ambient Occlusion

I've had my fun, I think I'll adopt Crysis style SSAO.

To resolve artifacts and improve image quality in method I'm using, I've got to start taking samples from the depth buffer, finding world space positions and re-projecting anyway.

GLSL Array Index Problems & Software Mode

I was revisiting my per-pixel lighting code, just making sure it was simple and clear. I pulled out the attenuation (and spot attenuation) code and tossed it into its own function.

Bam. Software Rendering.

So what happened? Originally I had specified literal light-array indices. Once I added in the functions for attenuation, I passed the indices as const ints. Apparently the GLSL driver on my mac can't deal with this, and decides that the best way to handle the situation is to drop to software rendering mode. It would be nice if GL told you it was dropping to a software renderer; I would prefer crashing to failing silently.

I'm getting to a point where I'm seriously considering DX10 again. I've already installed Vista via Boot Camp....

Screen Space Ambient Occlusion

I decided to take a couple of days and try developing my own SSAO algorithm. The images are the computed AO coefficients. I'm not finished yet, but the images give an idea of the evolution of the algorithm.

Andrew.

Parallax Mapping

I've implemented parallax mapping and thought I'd throw up a picture. I'd like to extend it to parallax occlusion mapping, but I'll have to see how much time I have.

Crossing Every T, Dotting Every I.

I've had a bug in some GL code for about an hour, and I've been beating my head against the wall trying to figure out what I'd done wrong.  glGetUniformLocation was returning crazy values, and glUniform1f was generating an invaild operation.  The problem? glUniform1f where I had intended glUniform1i. A simple typo, but somehow it also screwed up the values coming back from glGetUniformLocation, which is where I noticed it.

On that note, remember that glUniform* act on the currently bound program, so make sure there is one before you call it.

NSArray indexOfObjectIdentialTo:

NSArray's indexOfObjectIdenticalTo: does not behave as documented, and the the sample code given by Apple demonstrating it does not do what it says it does. For a more complete description see my two posts at:
http://www.idevapps.com/forum/showthread.php?t=6463

Another Demo Finished

Up to fixing up the user controls, I've finished my reflection/refraction demo. I've used hdr textures, but I haven't tone mapped them, so they're a bit bright. I'm going to do that in a future demo. At some point I'm going to turn these into a set of tutorials, but I might wait until OpenGL 3.0 finally arrives.

ImageIO OpenGL and Radiance .hdr Files

I promised to post my ImageIO code once I had things working, and now that it's working, I'm going to do just that. I just worked through the last bug I had in my texturing code. 

If you're using ImageIO to read Radiance .hdr files, and you're getting garbage, check the type parameter you're passing to glTexImage2D. The Apple documentation concerning ImageIO and OpenGL uses the format GL_UNSIGNED_INT_8_8_8_8_REV. While .hdr files are floating point, its the REV you should take note of.  The byte ordering of each R, G, B component is individually reversed between what ImageIO provides and OpenGL expects. Simply call glPixelStorei(GL_UNPACK_SWAP_BYTES,1) and things should look a bit more normal.

I hope this saves someone else a few minutes. 

Boolean osx_CGImageSupportsFileFormat(std::string format) {
CFArrayRef imageFormats = CGImageSourceCopyTypeIdentifiers();
CFStringRef targetImageFormat = CFStringCreateWithCString( kCFAllocatorDefault, format.c_str(), kCFStringEncodingUTF8 );
CFRange imageFormatsRange = CFRangeMake( 0, CFArrayGetCount( imageFormats ) );
Boolean didContainFormat = CFArrayContainsValue(imageFormats, CFRangeMake(0, CFArrayGetCount(imageFormats)), targetImageFormat);
CFRelease(targetImageFormat);
CFRelease(imageFormats);
return didContainFormat;
}

CGImageRef osx_CGImageCreateFromFile(std::string filename) {
CGImageRef image = 0;
CFStringRef fileString = 0;
CFURLRef fileURL = 0;
CGImageSourceRef imageSource = 0;
CFDictionaryRef imageSourceOptions = 0;
CFStringRef imageSourceKeys[] = { kCGImageSourceShouldAllowFloat, kCGImageSourceShouldCache };
CFBooleanRef imageSourceValues[] = { kCFBooleanTrue, kCFBooleanTrue };

try {
fileString = CFStringCreateWithCString( kCFAllocatorDefault, filename.c_str(), kCFStringEncodingUTF8 );
if (fileString == 0) throw std::runtime_error("CFStringCreateWithCString");

fileURL = CFURLCreateWithFileSystemPath( kCFAllocatorDefault, fileString, kCFURLPOSIXPathStyle, 0);
if (fileURL == 0) throw std::runtime_error("CFURLCreateWithFileSystemPath");

imageSource = CGImageSourceCreateWithURL(fileURL, 0);
if (imageSource == 0) throw std::runtime_error("CGImageSourceCreateWithURL");

imageSourceOptions = CFDictionaryCreate( kCFAllocatorDefault, (const void**)&imageSourceKeys, (const void**)&imageSourceValues, 1, 0, 0 );
if (imageSourceOptions == 0) throw std::runtime_error("CFDictionaryCreate");

image = CGImageSourceCreateImageAtIndex(imageSource, 0, imageSourceOptions);
if (image == 0) throw std::runtime_error("CGImageSourceCreateImageAtIndex");
}
catch(std::runtime_error &problem) {
std::cerr << problem.what() << std::endl;
}

CFRelease(imageSourceOptions);
CFRelease(imageSource);
CFRelease(fileURL);
CFRelease(fileString);

return image;
}

void osx_CGImageCrossCubeToImageCube(CGImageRef source, ImageCube &imageCube) {
size_t x = CGImageGetWidth(source) / 3;
size_t y = CGImageGetHeight(source) / 4;

assert( x == y ); // Images should be square. Could scale to ensure this instead of aborting.

CGRect imageBounds[6] = {0};

imageBounds[0] = CGRectMake(2*x, y, x, y); // +x 1
imageBounds[1] = CGRectMake(0, y, x, y); // -x 0
imageBounds[2] = CGRectMake(x, 2*y, x, y); // -y 2
imageBounds[3] = CGRectMake(x, 0, x, y); // +y 3
imageBounds[4] = CGRectMake(x, y, x, y); // -z 4
imageBounds[5] = CGRectMake(x, 3*y, x, y); // +z 5

imageCube.size = x;
imageCube.bitsPerComponent = CGImageGetBitsPerComponent(source);
imageCube.numComponents = 4;

const size_t bytesPerRow = imageCube.size * imageCube.numComponents * imageCube.bitsPerComponent / 8;

CGImageRef partitions[6] = {0};
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();

for (int i = 0; i < 6; ++i) {
imageCube.image[i].resize( imageCube.size * bytesPerRow );
partitions[i] = CGImageCreateWithImageInRect(source, imageBounds[i]);
CGContextRef imageContext = CGBitmapContextCreate( &imageCube.image[i][0],
imageCube.size, imageCube.size,
imageCube.bitsPerComponent,
bytesPerRow,
colorSpace,
kCGImageAlphaNoneSkipLast | kCGBitmapFloatComponents );

CGContextSaveGState(imageContext);

// Up Down and Backwards are flipped in the cube cross format
if ( i == 2 || i == 3 || i == 5 ) {
CGContextTranslateCTM(imageContext, imageCube.size, imageCube.size);
CGContextScaleCTM(imageContext, -1.0f, -1.0f);
}

CGContextDrawImage(imageContext, CGRectMake(0, 0, imageCube.size, imageCube.size), partitions[i]);
CGContextRestoreGState(imageContext);
CGContextRelease(imageContext);
CGImageRelease(partitions[i]);
}

CGColorSpaceRelease(colorSpace);
}

Apple and OpenGL 3.0

OS X has never had good OpenGL drivers, by which I mean few bugs, and wide support of current standards. Things have improved over time, but there are clearly still problems.

OpenGL 3.0 promises to clean up and slim down the specification, making it clearer and easier to develop and maintain a standards compliant implementation. I'm holding out hope that the change will help Apple out and the quality of their GL drivers will improve.

Speaking of OpenGL 3.0, I'm hoping that GLSL is being moved out of core, and/or that it's being replaced with something better, namely Cg; I'm tired of dealing with vendor-dependent GLSL bugs.

Cube Mapping

The GLSL mat3 constructor that is supposed to take a mat4 and slice out the upper 3x3 doesn't work on my MacBook Pro. Apparently I'm not the only one. http://emergencyexit.untergrund.net/2007/12/glsl/

As described in the above url, you can use the following instead:

mat3(matrix[0].xyz, matrix[1].xyz, matrix[2].xyz)

CGImage Problems

I'm trying to use ImageIO to load Radiance HDR files, then draw the contents to a user-backed floating point context, but things are crashing. Once I figure out what's wrong, I'll post code.

Andrew.

OpenGL Demos

I've written an implementation of Preetham's "A Practical Analytic Model for Daylight", an ambient occlusion demo, a vertex buffer object demo, and a GLSL per-pixel lighting demo, I'll post them soon. Tomorrow I'm going to start on a reflection/refraction demo. I'll likely follow them up with a set of tutorials explaining them in some detail; I'd like to start doing more tutorials.

New Macbook Pro

I'd had enough of OpenGL driver problems on my old mac, so today I went out and picked up a new one. I grabbed a new 2.5Ghz, which has an 8600GM 512M card, which should outpace my G4 17" by quite a bit. I'm eager to start using the new gesture controls in my programs.

gl_NormalMatrix

I've run into a few driver problems under OS X

For comparison, here are my system specs:
Mac OS X Version 10.5.2
Processor: 1.5 GHz PowerPC G4
Memory 512 MB DDR SDRAM

gl_NormalMatrix is computed incorrectly on my mac.
The problem, and a temporary solution are listed below.

frag_Normal = gl_NormalMatrix * gl_Normal

frag_Normal = (gl_ModelViewMatrix * vec4(gl_Normal, 0.0)).xyz

Framebuffer objects must be powers of two.

More to come.

Revisiting Shadow Mapping

I'm revisiting my shadow mapping code, and I've hit a problem. A picture will do a better job of describing it than I can in words:

Marching Through a Volume

I'm trying to decide between implementing marching cubes and marching tetrahedra. I'm aware of (some of the) limitations of each, and I'm trying to figure out which best serves my purposes.

I'm starting with an easy to follow instructional document written by Paul Bourke, and I hope to manage an original implementation in the next week or two.

shadow2DProj Resolved

I'd like to offer a quick thanks to oc2k1 from the OpenGL message boards. He realized immediately that I was missing the following:

glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE);

Alternatively, I could compare the value in the depth texture to the texCoord's z value myself, but I expect that this comparison is hardware accelerated.

If you're doing GLSL development, you might want to have a peek at Lumina. I'm going to be playing with it in the near future.

Thanks again to oc2k1, and the rest of the incredibly helpful people on the OpenGL message boards.

Andrew.

shadow2DProj

I'm having some difficulty implementing shadow maps in OpenGL. The Orange Book (section 13.2.3) says that shadow2DProj should return a value that is either 0 or 1. Below is an image of the diffuse term multiplied by the result of the lookup, which clearly is not giving values between 0 and 1. Beyond that, the shadows are much sharper than I had expected. I can imagine some sort of PCF going on, but that still doesn't account for the pale shadows.

I can simply force 0,1 values via the shader, but I'd like to understand what's going on rather than force my expectations onto things.





For completeness, the fragment shader follows.

#version 110

varying vec3 frag_Position;
varying vec3 frag_Normal;
varying vec3 frag_LightDirection;
varying vec4 frag_ShadowCoordinate;

uniform sampler2DShadow frag_ShadowMap;

void main()
{
vec3 E =-normalize(frag_Position);
vec3 L = normalize(frag_LightDirection);
vec3 H = normalize(E + L);

// Get shadow value
float shadow = shadow2DProj( frag_ShadowMap, frag_ShadowCoordinate ).x;
//if (shadow < 1.0) shadow = 0.0;

// Light properties
const vec3 light_ads = vec3( 0.0, 1.0, 0.0 );

// Material properties
const vec4 material_adss = vec4( 0.0, 0.5, 0.0, 0.0 );

// Light/Material interaction
const vec3 lightCoefficient = light_ads * material_adss.rgb;

vec3 ambientDiffuseSpecular = vec3( 0.0,
shadow * max(0.0, dot(frag_Normal, L)),
shadow == 0.0 ? 0.0 : pow( max(0.0, dot(frag_Normal, H)), material_adss.a ));

// Ambient, Diffuse, Specular contributions
float shade = dot(lightCoefficient, ambientDiffuseSpecular);

gl_FragColor = vec4( shade * gl_Color.rgb, 1.0 );
}