Disclosure: I'm a recent astronomy grad who specialized in computational astrophysics. Definitely biased.
The issue is that at least for many scientists and mathematicians, mathematical abstraction and code abstraction are topics that oftentimes run orthogonal to each other.
Mathematical abstractions (integration, mathematical vernacular, etc) are abstractions hundreds of years old, with an extremely precise, austere, and well defined domain, meant to manage complexity in a mathematical manner. Code abstractions are recent, flexible, and much more prone to wiggly definitions, meant to manage complexity in an architectural manner.
Scientists often times have already solved a problem using mathematical abstractions, e.g. each step of the Runge-Kutta [1] method. The integrations and function values for each step is well defined, and results in scientists wanting to map these steps one-to-one with their code, oftentimes resulting in blobs of code with if/else statements strewn about. This is awful by software engineering standards, but in the view of the scientist, the code simply follows the abstraction laid out by the mathematics themselves. This is also why it's often times correct to trust results derived from spaghetti code, since the methods that the code implements themselves are often times verified.
Software engineers see this complexity as something that's malleable, something that should be able to handle future changes. This is why it code abstractions play bumper cars with mathematical abstractions, simply because mathematical abstractions are meant to be unchanging by default, which makes tools like inheritance, templates, and even naming standards poorly suited for scientific applications. It's extremely unlikely I'll ever rewrite a step of symplectic integrators [2], meaning that I won't need to worry about whether this code is future proof against architectural changes or not. Functions, by and large in mathematics, are meant to be immutable.
Tl; dr: Scientists want to play with Hot Wheels tracks while software engineers want to play with Lego blocks.
>mathematical abstractions are meant to be unchanging by default
Let's say today I am doing RK2, and tomorrow I want RK4, how do I easily make my change? In my codes, it's a change of a single line and I get higher order convergence, etc. It is not a week or month project, as for many codes, it would be because of some of those abstractions you derride.
Also, computational math is an active area of research, the method you mentioned is not hundreds of years old, although yes, it was developed in the early 1900's. To this day, people are developing new methods that give higher order accuracy (orders above O(err^10) to abuse notation)...but as you can guess, no one uses them because changing the current codes are so difficult they just don't.[0] Of course, I agree O(err^4) is often enough, so the motivation to change codes now isn't that over-powering, but it again is something we lose by learning things a little but outside our field which could be helpful.
[0]Instead we, choose smaller and smaller mesh-sizes and timesteps to deal with small order error, and request millions of cpu hours, use electricity, kill trees and contribute to global warming.
Higher order integration methods are not always more accurate or more power efficient. They are typically worse in terms of stability if the step is too big, they require more computations per step and they may have higher error constant, so they actually often require smaller steps than low order methods. That's why in circuit simulators only methods of order up to 4 are really useful, and most of the time simple schemes of order 2 are used.
It sounds like you want a language like Haskell. Abstractions are based on mathematical (algebraic and category theoretic) abstractions with well-defined laws. The language has immutable semantics and admits equations reasoning. Using libraries like Dimensional has made me better at physics; many fields of physics play fast and loose with units and dimensions and aren't even aware of it.
I'm glad someone picked up on it! I've definitely picked "FP is the truth for astrophysics programming" as my hill to die on, but the issue with Haskell is it's combination of vernacular and tools that make it hard to approach for the novice scientific programmer. Almost all dedicated astrophysics programmers wind up using Fortran90, which is sorta the de facto due to it's imperative nature.
Modern Fortran is pretty awesome though; it has pure functions, it's fast, scales well, is easy to read and understand. Probably much more suited to most scientific fields than Haskell.
meeeeh, come on. You can't say the sloppy code can be trusted because the clean math it is based on is verified. The sloppiness of the code prevents validation that it properly implements that precious math of yours.
The problem is that you want to treat the code as not your "real" job. Your real job is getting correct answers into published papers, and providing a proof of that correctness. If your code, on which your results rely, is too sloppy for anyone else to understand (and note that "anyone else" can include "you, in 6 months"), then you've not proven correctness at all.
I'm not treating anything, it's because coding isn't my job. The job of a scientist is to do research, and coding is nothing more than a tool towards that goal.
>your code, on which your results rely, is too sloppy for anyone else to understand...then you've not proven correctness at all
No, my results rely on my experimental methods, my mathematical models, and my code. Correctness can be proven in spite of sloppy code. Would you dispute a claim on the basis that calculations done on a calculator can't be seen by others?
Furthermore, the burden of proof after peer review in academia is on the person disproving in it. If my code is wrong at a basic level, what good does it do for anyone? If someone is to disprove my paper, they should reimplement the code in order to account for errors.
Does this excuse spaghetti level code that often accompanies papers? Of course not. Scientists have a lot to learn from software engineering about proper programming skills, but programming is simply another tool in the repertoire, not something that should be put on a pedestal.
> coding is nothing more than a tool towards that goal.
That's an important idiom that most devs need to understand at some points in their career, but don't. It's not even exclusive to business goals, but sanity and complexity ones as well..
Chances are, coding isn't your "real" job in a lot of cases (including most software engineers and programmers). Your real job is solving a problem for someone, using code. Good software architecture, coding style, etc. are there to help you achieve this goal, but the end user of the software doesn't care about them.
No more than a home owner cares whether or not their house's blueprints were printed on a napkin. If such a thing were to happen, it would be indicative of an incompetent contractor.
Unless you're the first owner of the house, you probably don't even have meaningful engineering drawings for it. And even if you did, they are not what you care about. (Well, it'd be nice to have some documentation about the wiring, plumbing, which walls are load bearing, etc. but that's another rant.)
What you care about as a homeowner is that your house is solid, watertight, safe to live in, acceptable to look at, and meets your needs as a tenant. Whether your builder designed it down to the last tack and cable-tie in SolidWorks, or sketched it on a napkin, or made it up as they went along, makes no difference to you as the homeowner.
Quality of process is only ever a proxy for quality of results.
There are things you just don't see together. You don't see quality blueprints printed on napkins, and you don't see quality code that is well-suited to its task written in a sloppy way. Technically speaking, you can lose weight eating all your meals from McDonald's. But the person who eats all their meals from McDonald's isn't going to exercise the portion control necessary to do it. It's just not a thing that happens often enough to bother considering it.
Sloppy code is an excellent indicator that the program is a buggy piece of junk. I don't care if it's technically "possible" to write a "good" program in a sloppy way. If your code is sloppy, you aren't that person who makes that program. The person who is capable of making good programs doesn't write sloppy code, even if they are technically capable of doing it.
> The issue is that at least for many scientists and mathematicians, mathematical abstraction and code abstraction are topics that oftentimes run orthogonal to each other.
Excellent observation. I'm an ex-physicist and on the few occasions that I had to use computers the only thing I cared about was how computer functions mapped into the mathematical abstractions that I cared about. Everything else was just noise.
The issue is that at least for many scientists and mathematicians, mathematical abstraction and code abstraction are topics that oftentimes run orthogonal to each other.
Mathematical abstractions (integration, mathematical vernacular, etc) are abstractions hundreds of years old, with an extremely precise, austere, and well defined domain, meant to manage complexity in a mathematical manner. Code abstractions are recent, flexible, and much more prone to wiggly definitions, meant to manage complexity in an architectural manner.
Scientists often times have already solved a problem using mathematical abstractions, e.g. each step of the Runge-Kutta [1] method. The integrations and function values for each step is well defined, and results in scientists wanting to map these steps one-to-one with their code, oftentimes resulting in blobs of code with if/else statements strewn about. This is awful by software engineering standards, but in the view of the scientist, the code simply follows the abstraction laid out by the mathematics themselves. This is also why it's often times correct to trust results derived from spaghetti code, since the methods that the code implements themselves are often times verified.
Software engineers see this complexity as something that's malleable, something that should be able to handle future changes. This is why it code abstractions play bumper cars with mathematical abstractions, simply because mathematical abstractions are meant to be unchanging by default, which makes tools like inheritance, templates, and even naming standards poorly suited for scientific applications. It's extremely unlikely I'll ever rewrite a step of symplectic integrators [2], meaning that I won't need to worry about whether this code is future proof against architectural changes or not. Functions, by and large in mathematics, are meant to be immutable.
Tl; dr: Scientists want to play with Hot Wheels tracks while software engineers want to play with Lego blocks.
[1]: https://en.wikipedia.org/wiki/Runge–Kutta_methods
[2]: https://en.wikipedia.org/wiki/Symplectic_integrator