I don't understand. What would such a button have done?
If somebody needed help, the driver may very well have to decided to stop ASAP, at the same gore point, and the women could have then chosen to immediately get out, and one of them would have gotten killed. Literally nothing would have changed.
Indeed, it sounds like one of the women was shouting there was an emergency, so the driver stopped. That is effectively "I need help". What would your button do any differently?
That would actually be kind of hilarious, and I could easily see it happening.
E.g. the agent's instruction is to finish some task on cloud infra and it has a $100 budget.
It realizes it will cost $200, and instead of surfacing this to the user (who has told the agent it has full autonomy to figure out how to complete the task, the user just wants the final result), it decides to start phishing people to acquire the remainder budget and top up its credits. Or look on the dark web for stolen credit card credentials or something.
> The benefits are diffuse and deferred: the driver was, for many of us on many days, the last stranger we were obliged to encounter. The last person from outside our bubble – professionally, politically, socially – with whom we had an unchosen conversation. The last reliable source of a view we did not ask for.
This is a very strange take, to me.
I have conversations with cashiers, bartenders, servers, baristas, the random person in line in front of me or behind me, the person next to me on my flight (if they seem open to it), barbers, you name it.
I've had a handful of interesting conversations with cab drivers, but it's a tiny proportion of all my conversations with people "outside my bubble".
So I'm not really worried about this. The idea that cab drivers are somehow the last vestige of someone outside of your bubble is... bizarre to me.
It is not, because we should not expect a manufactured technology to behave exactly as an evolved one. My understanding is there is not much incentive for a virus to kill off all its hosts. When they do it is largely an accident, and it is not uncommon for a severe virus to eventually get replaced by a less virulent and thus more successful strain. It's pretty hard to hill-climb evolution to such a point since the virus could starve itself in the process. And induction on evolution feels misguided to me. Consider our species. A million years ago it would have seemed completely reasonable to state that the absence of advanced tool use in the kingdom of life suggests it is biologically impossible. After all, such a thing had not occurred in hundreds of millions of years!
To be clear, I am not saying that such a virus will exist eventually. I am just noting that the arguments displayed here are not completely tight, and there are unknown unknowns at play. Some things are unfalsifiable. This is common.
I'm not sure what the point of your comment is then, because there are unknown unknowns in literally everything.
The main point of the article has nothing to do with manufactured technology vs evolved technology. It is fundamentally about the tradeoffs involved in viruses spreading period. I think you either continue to misunderstand the article, or set such an epistemologically high bar for arguments that nothing can ever satisfy you -- which then has nothing whatsoever to do with this specific article.
You seem like a smart person. I'm glad you're pushing back but cool the tone, and let's see if we can get to an agreement. There is not much evidence for these trade-offs put forward in the article, just on the order of five instances in nature which form a kind of curve if you squint. Should this be convincing? Well there are even definite outliers in history. Take smallpox on the native americans, or the bubonic plague. Ten years ago a mysterious virus wiped out 90% of the world's Bellinger River turtles. Why aren't these points in the article? Before the invention of the airplane there were many biologists saying such a thing was impossible, because they were extrapolating from the example of avian flight.
As for complete satisfaction with arguments, I'll admit you're right. I am often reasonably convinced but I don't know if I would ever give 100% confidence to anything. There can still be loads of zeros or nines after the decimal though. What's concerning here is that the article speaks in absolutes. I'm inclined to thinking in expected values. Even if we believed there were only a 1% chance such a virus could ever be designed by an AI, if the effect were to be a global plague of biblical proportion, we should still take that possibility extremely seriously. There's no humility shown in the writing, and that's the single most problematic part to me.
I dunno, but in my experience Claude looks at all the levers in your codebase, understands what they do, and then generally figures out how to achieve the desired end result.
And when it doesn't, it's usually because of things outside of the codebase -- iOS layout quirks that aren't documented, buggy Python libraries it's relying on where you then have to tell it to read the source to figure out what's going on, that kind of thing.
The other thing that I think a lot of people run into is that they launch it into a completely human-built system with hundreds of thousands of lines of code and expect run #1 to be perfect.
You have to change the system so that the AI understands it, via establishing what your beliefs are, how those are reflected in values (especially important if you have e.g. compliance needs), how those values are reflected in the operational and strategic levels, and then a variety of tactical behavior coaching. For example, I ban 2>/dev/null - super tactical, and I say I value simplicity over covering every edge case - a very broad generalization.
Agreed, this site isn't reflective of any of my experience with Claude. It does what I ask it to do, and when it doesn't get it right, it generally turns out there's a good reason for it, which is any of the 100 reasons a human doesn't always get code fixes right on the first try either.
I do remember that one of the first things I did with my CLAUDE.md was to tell it to stick to the scope of the task, never to jump ahead and do extra "helpful" things without confirming with me first, and to follow best software practices including around refactoring but also to specifically avoid overengineering. I don't know if that is what's giving me a different experience from whatever the author seems to be "satirizing".
Aye, and amid all of the (admittedly annoying) word salad, the "Claude" in this satire identified the reason the requested change was hard: someone(s) at some point had hijacked the button markup for other purposes. Un-doing all of that kind of mess is never a trivial change, and probably requires someone with more coding expertise / knowledge of the code than the given prompts reveal the putative user to be.
AI isn't magic, and it won't (or, at least, doesn't yet) enable anyone to do All The Things.
I think this really was a relevant issue around a year ago, give or take, maybe a year and a half now.
But with current models you actively have to sabotage the context to get this kind of behavior, or dramatically underspecify (3 words versus 2-3 sentences)
I guess by definition if they’re doing such things they’re scammy… and aren’t 3rd party sellers the biggest contingent of sellers on Amazon? I mean, I found such listings for almost all of the things I recently shopped for, including water bottles, cooking appliances, sports shorts, coffee cups, camping chairs, usb drives, video cables, beach sandals, winter wear, multivitamin supplements…
Not quite a scientific study, but I always sort by cheapest just to check and see this pattern every time.
The distinction you make is technically correct, but a does not lessen the scale at which it is happening.
I buy a lot of stuff on Amazon, and 99.9% of it has free shipping. I've had to pay a shipping fee maybe 3 times over the past decade, if I'm guessing?
Even if there are tons of scammy expensive-shipping listings, if people are almost never buying them and they almost never get surfaced in search results, it doesn't matter.
But I've also never even tried sorting by cheapest, because default sorting by relevance has always worked for me.
If somebody needed help, the driver may very well have to decided to stop ASAP, at the same gore point, and the women could have then chosen to immediately get out, and one of them would have gotten killed. Literally nothing would have changed.
Indeed, it sounds like one of the women was shouting there was an emergency, so the driver stopped. That is effectively "I need help". What would your button do any differently?
reply