I did some work on data visualization for the astrophysics department when I was in college. I started to work with the simulation code, but found that the math was sprinkled everywhere, which made it really difficult for me to make structural changes without risking the integrity of the program.
One of the most elusive skills for self-taught programmers is how to structure code properly. A good architecture would allow domain experts and non-expert programmers to coexist, but that would require throwing away a lot of existing spaghetti code written by domain experts, which is not going to be a popular decision.
I'm a programmer who is studying physics in college, and a couple years back I had a similar experience with simulation code as you did. I didn't have any issues with the math—the program I was working on didn't have anything more conceptually advanced than multivariable calculus—but I did struggle significantly to understand the physics behind the simulation.
It didn't help that most programs use, for example, the variable 'rho' for density instead of just writing out 'density'.
On the other hand, reading game physics libraries (written by programmers, not physicists) can be just as bad. There are physics hacks all over ("it's not stable, so let's throw in an arbitrary constant") and there's code repetition where the programmer doesn't understand that two concepts are closely related.
This is why I think, rather than every data analysis/visualization program being written from scratch with it's own custom UI and I/O, formatting etc, these programs should be written as modules to a data workflow program like RapidMiner that handles all that for you.
One of the most elusive skills for self-taught programmers is how to structure code properly. A good architecture would allow domain experts and non-expert programmers to coexist, but that would require throwing away a lot of existing spaghetti code written by domain experts, which is not going to be a popular decision.