How does one decide whether to trust any software? Not just the small things we do have the source code for, but big things that we don't have the source code for? I don't think this is a new problem.
After a significant number of iterations of testing, and improving it. When we wrote code by hand, something like changing an api endpoint or a simple front end change would still involve dozens of "write code" then "retest manually". This would not only get the work item done, but you would often encounter edge cases or other issues you hadn't thought of when writing/reading the ticket.
When AI can one shot the implementation and testing, it likely only gets 1 round of human testing if you are lucky. Then scale this to an entire AI generated project, the ratio of features to manually run tests is astronomical. In the old days this ratio was inverted, and the tool has been battle tested before reaching any users.
I don't think human vs agent inherently means that the app is built any more rigorously. Sure, it might be. But it's not guaranteed that a human app is more rigorously built than one guided by human, or vice versa. It depends, on the developer and how much they care, as it always has even before agents. I remember some shocking software from back in the day, Windows Vista being the top of that steaming pile. There's nothing new here.
Theres a distinction between built more rigorously and tested more rigorously. By many definitions, agent created apps are built more rigorously. They write more unit tests, attempt to handle more edge cases, write more lines of code, create features faster. For certain things, I do think agent created tools tend to be better.
But - I have found that every single model available is lacking in creating a great UX, even when you are aggressively using playwright in your agent loop. IMO this comes down to a fundamental limitation of how LLMs work. If an app has a button that is too small, or white text on white background, or inconsistent layouts, or non deterministic ux state due to network calls (looking at you spotify).... the agent will not notice or care. A playwright driving agent can click on a button wether its 1px or 1000px wide.
But ultimately, a good ux is about designing for the constraints that we as humans have to live with as mere mortals with all our imperfections. This is not easy to do, and requires a lot of craftmanship by the developers as well as buy in from the suites and middle managers.
In your windows vista example, the issue is almost surely the latter of my previous sentance - middle manager buy in. No one at microsoft leadership gives a shit about the user experience its all kpi seeking nonsense. The user having a good experience is not measured in their spreadsheet.
When you think about it, pretty much every single product we use helps address a problem related to our fundamental human flaws. Im typing this on an expensive ergonomic keyboard because my wrists hurt, youtube premium exists because I dont want to waste my precious time on stuff I dont want to see, we use operating systems with GUIs because our mortal minds cant easily comprehend thousands of lines of text in seconds.
When I use a horrible website I have a very intuitive, almost instinctive reaction similar to how I react to physical pain. An LLM will never experience this, so it will never create a ux as good as a human can on its own.
Nothing is foolproof, but signals can go a long way. If an app's UI is poorly considered or comes across as careless for example there's a good chance that its other aspects are like that too.
Not OP but I have this too. It’s like a voice in your head constantly. It can make writing very easy as you just transcribe the internal monologue.
It’s weird, I’d be hesitant to say “it’s a voice” but it kind of is and it is not my own which I find curious (who on earth is speaking in my head). In some ways it sad, if I close my eyes, I can’t picture a sunset and I can’t really dream. I love reading books but I can’t visualise the settings properly but it resonates with how my mind describes the world to itself.
I know people who have trained themselves to skip the "sound out the word in your head" part of reading and are incredibly fast readers. I personally can't do it, but there are neurotypical people who can.
He says he just knows concepts. His brain works in non-verbal, non-visual concepts. Like he knows that his wife's eyes are blue and can draw her, but he can't visualize it.
reply