> CVSS scores are literally a Ouija Board that can come out to whatever the user wants them to.
Not really. They are very good at describing the technical impact. Sometimes pre-condition is very rare and that reduces overall likelihood but for those few it applies, the impact still could be catastrophic. Who wants to risk it if whole business could go down?
Published CVSS is a base score that gives you a hint of how important the analysis of a vulnerability is to prioritise the patching or mitigation. What you see on websites is only ever the base spiciness so to speak. If you have for example wordpress only running in an isolated environment behind internal firewall rules you‘d downgrade it accordingly. It’s a imperfect metric but so far the best we have to signal priorities. It’s all described in its spec that no one seems to read and websites also communicate it badly.
> These metrics enable the analyst to customize the CVSS score depending on the importance of the affected IT asset to a user’s organization, measured in terms of complementary/alternative security controls in place, Confidentiality, Integrity, and Availability. The metrics are the modified equivalent of base metrics and are assigned metric values based on the component placement in organization infrastructure.
CVSS is impossible to communicate effectively. We don't need a metric; I'm already going to have to read and assess the vulnerability to decide how I actually want to assess the risk given my infrastructure, so the number's not doing me any good.
This isn't just a CVSS issue: there have been a variety of attempts to reduce a risk score down to a single general number and they all end up as somewhere between marketing material, scare tactic, and junk science.
> We don't need a metric; I'm already going to have to read and assess the vulnerability to decide how I actually want to assess the risk given my infrastructure, so the number's not doing me any good.
Would you say that vulnerability with CVSS score that points to low is equally important to verify and take care of than CVSS which points to critical?
Yes. I believe that using CVSS scores as a first pass to decide which vulnerabilities to review is risky.
The most boring reason, even if you take CVSS scores at face value, is that in many cases it is possible to leverage multiple "low" severity vulnerabilities into a massive impact.
But the bigger reason is that CVSS scores are all over the place, and the people operating roulette wheel that generates them do not have any insight into any specific person's systems.
You need to reason about both probability and impact, not one or the other. In your example, it just means that the probability is very low, perhaps even down to 0 in your specific case. But even at a low probability, the impact of the vulnerability remains very high so it helps you reason about it.
For example, you might react differently to these scores:
- <8/10: check that your systems are indeed secure
- 8.6/10: check that your systems are indeed secure and tell your junior analyst to train on creating a custom monitoring rule for that attack and follow-up with you
- 9.8/10: double-check that your systems are indeed secure, ensure that if you had a hole another security layer would have caught it (if not, that's a problem!), set up a honeypot to get some info on the assholes that have repeatedly attacked you lately and will undoubtedly try to 0-day you in the next few hours, etc.
just checked, yes they don't seem to provide that information, most other providers are advertising fp8 or fp4 which is okay, but "together" doesn't, so they are likely using fp4
> It's just AI slop and it should be taken with a mountain of salt.
Can't you see the irony. You are defeating the argument that LLMs are incorrect or weak with low effort with the term "AI slop" that itself is a narrative that AIs produce weak outputs with low effort.
> Every problem you solve, ten new ones open up. Like a fractal, the more you zoom in, the more detail emerges. No matter how much better AI is at solving problems, it's not going to generate the "final, complete compendium of mathematics" that that seems to hover over this post.
I guess Terence's main point has been all the time that if we let AI solve all these existing problems, we don't notice the new ones and then there is stagnation.
I think that if they don’t block by default, is quite significant. Chrome + Edge has superior marketshare and then add the % people who have no idea what these mean and don’t change defaults.
I would say that it is like gardening. If you let them go havoc from the start, the weed will take over. If you keep focusing on removing the weed and enforce specific standards and practices over the code base and it keeps growing, over time LLMs start to suddenly follow that and they don't make so much slop anymore. At least that is my experience. But I force specific audit agent after every added feature which says them to force compliance with AGENTS.md and check the consistency with the code base.
reply