S3D is a direct numerical solver for Navier-Stokes with complex chemistries. It’s used for the simulation of turbulent flows during combustion.
Without a doubt, S3D has some of the best visualizations around:



As one of my first projects at ORNL, I accelerated some of the S3D’s kernels in CUDA. Aside from the usual data parallelization and tuning, this was a great introduction to a large scientific codebase spanning Fortran and Matlab.
As CUDA was much less common in 2009 than it is today, I wrote a summary of the speedups in my first workshop paper: Accelerating S3D: A GPGPU Case Study.