That's not the conclusion. I started using AI after the Jacobian conjecture counterexample and have used a particular problem to learn how to use AI and to explore it's capabilities. I'm not a great mathematician but I'm full faculty with 25+ years of research experience and lots of articles and I just proved in a few weeks something that had resisted my efforts for some years.
The exploration process is much easier now. Ideas are quickly testable and multiple tests can help identify a technical obstruction. The tool requires good guidance and input but as it trains on people like me it will need those less.
At the very least our way of doing things must change. More pessimistic views seem to me defensible.
GP never said it was the conclusion. He was praising you, after you on your own volition decided to comment here to let know others about your experience with AI.
It seems to me you're a very privileged individual, I'd suggest you practice gratitude regularly in your life.
You sound a bit troubled. I was responding to the leading the way part. I'm hardly doing that. I just jumped on the bandwagon. What I do see is a tool that forces me to change behaviors learned over decades and potentially renders useless many of them while facilitating others. That's not all a rosy picture.
As for privilege, don't assume everyone lives and works in the US or gets paid lots of money to be a professor.
I think upweighting the live components of academia is inevitable in the age of automatically produced writing, but I also find it depressing that people think so little of writing that they imagine it obsolete because of AI.
AI writing is aggressively, aggressively mediocre. It is only good for how cheap it is. If you're indifferent to how much better good human writing is than AI writing, you should not be the one to evaluate human writing.
> the problem may be less about knowing a particular technique and more about the agent’s ability to keep its actual goal as the primary constraint.
I think the limiting factor is often the human's ability to articulate the goal and hold the agent accountable.
The working memory of technical people and the way they communicate often seems to prioritize how they're doing a task and not why they're doing it. So when they talk to you about their problem, they give you the human equivalent of modem noises and stack traces. This is such a common problem on tech support and Q&A forums like StackOverflow that it spawned its own terminology and website: https://xyproblem.info/
The agents don't respond to XY problems any better than a colleague does, and usually much worse.
An effective colleague asks why, and we need a similar relationship between the agent and its human supervisor.
> Makes sense. The kids can see that education isn't really going to be a sellable skill when AI ends up coming into full force.
If true, this is a disastrously foolish thing to believe. Education is only going to become more important as supervising increasingly advanced AI agents becomes more difficult. We need people who understand all different domains of human endeavor at the deepest level possible.
I mean, what's the alternative? AI does everything for me and I play Fortnite all day? Might as well plug yourself into the Matrix.
The situation will probably be like with airline pilots. The plane is mostly an automated system, but we simply don't let automation to work through exceptional situations, without human review. Therefore, lots of pilot training (and their presence) is still required.
But it's also hard to guess what exactly skill should the pilots have. It's clear that lots of things from manual aircraft do not translate well to automated.
Children of the Magenta Line seems very topical, especially wrt AI
>In 1997, American Airlines captain Warren VanderBurgh gave a lecture warning about the dangers of autopilot. He titled it "children of the magenta line," after the magenta coloured course line the flight computer draws across a cockpit display. He noticed that pilots were so used to following that line, to managing the automation rather than actually flying the plane, that they lost their own skills to manually fly the plane. This wasn’t a problem until it was a problem.
>VanderBurgh’s concern was not about autopilot itself but rather the complete dependency on autopilot at the expense of the internal knowledge of how it works and the loss of the judgment to know when to use it. In 2009, his warning came true in the very worst way possible over the Atlantic. When Air France 447’s sensors iced over, the autopilot handed a perfectly good airliner back to its crew, the pilots, confused and out of practice, stalled it into the sea.
Just like kids eventually take over most of the daily tasks for their parents (for people in countries where you don't have any kind of retirement fund and assisted living, etc.). Same will happen here - on one hand because the human population keeps growing older without any replacement and on the other because all the new tech will be too complicated for humans to understand.
It seems unlikely that we'll be able to keep up with AI in any capacity. Your alternative seems like the inevitable outcome of AI succeeding, and it doesn't sound so bad.
Strong resonance with the famous essay "The Rise of Worse is Better" [1], which contrasted the (better) "MIT/Stanford style of design" with the (worse) "New Jersey approach".
MIT/Stanford:
> Simplicity -- the design must be simple, both in implementation and interface. It is more important for the interface to be simple than the implementation.
New Jersey:
> Simplicity -- the design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.
TFA maps "simplicity" to "MIT/Stanford simplicity" (simplicity for the user) and "smallness" to "New Jersey simplicity" (simplicity for the developer).
I wonder if the root of the tension between the two schools comes down to the ambiguity of the user/developer distinction. Developers are also users. Simplicity of implementation is helpful to developers when they are working directly on implementation, while simplicity of interface is helpful to developers when they are using other developers' work.
I appreciate the New Jersey simplicity as a user (with development skills) too, though. It's usually just a matter of time before I have to dive into the program/library/whatever internals to fix a bug.
Pretty much anything. I mean, even consider a text editor, monospace font, no syntax highlighting. It's already pretty daunting. Doable, certainly, but a big task. We're lucky people already made some, and we can copy their designs even when we choose not to copy their exact code.
From the authors, Charles Simonyi went to Microsoft as one of the original Word implementers, while Excel was born on Mac OS and only later ported to Windows.
I think you're right on that point of contention, it's too far to assert a universal and clear good/bad dynamic here because that line between developer/user is contextual and fuzzy.
Another interesting irony I'll note, Lisp is the "New Jersey approach" towards symbolic AI. Americans clinging to their Lisp systems were deeply entrenched in a "worse-is-better" mindset. Your interface, the computational model, didn't need to be designed for logic programming, that was wholly secondary. Do everything as much as possible in Lisp, and then offload the relational description to a small (not simple) library. American knowledge engineers were looked at as overpaid procedural hackers with zero mathematical elegance and very little credibility. More or less the same perception these self-same Lisp-machine users had for Unix and C programmers.
It's all about perspective, at the end of the day. Where we draw the line in the sand on these categories is free-to-choose, yet it also determines everything. We're always someone else's villain under different semantics.
The prompt was actually huge, containing all the central ideas of the articles in details. I actually found my first prompt in case you're curious:
---
Human contexts are way more limited compared to computers. We can't reason about complex software when they go through many branches with so many implications. It's just too hard for humans to keep track of all interconnected pieces. So humans have historically split the system parts into manageable modules that can be understood in isolation and then we spend some time connecting those parts. That's how we can keep the context reasonable for human understanding.
So, when senior engineers found themselves lost while trying to debug an issue in a complex system they would naturally decide to pause and rewrite or refactoring the confusing piece of the system to make it manageable so developers can easily understand what's going on and review future changes.
Usually a system doesn't start that confusing. But as requirements change developers add additional branches and code until the code is no longer manageable. Sometimes the requirements changed significantly since the code was first written and all we have in the code are exceptions rather than the rule. That's usually when historically senior developers would take the time to rewrite that part of the system so they can reason about it.
But AI agents are not as limited as humans context-wise and they can reason about those confusing (to humans) systems and make sense of it. So they simply keep adding additional branches to the existing mess without ever suggesting a major refactoring like a senior developer would do in those cases, unless there's specific harness to tell agents to act like that.
This article is about bringing this into attention so that developers can policy themselves and keep asking themselves whether it's time for a major refactoring instead of relying on the AI agents and trust them because they no longer understand the code because it's too complex for humans to follow. Can you draft an article focused on this concern?
Thank you, I appreciate you sharing this very much! I wish I could read the entire chat, but I imagine there might be some NDA stuff in there if you talk abt real world experiences.
Altman and Zitron are like pro wrestlers. Their speech acts aren't for truth, they're for some spectacular effect on your feelings and your imagination that keeps you coming back for more.
I encourage anyone interested in this to read On Bullshit by Harry Frankfurt, the best popular philosophy work in a long time.
Ed Zitrons wild claims might be for entertainment value, but Altman’s are for his own valuation. He’s bringing in hundreds of billions of dollars off those claims.
Without reference to either person, I think this is essentially a misunderstanding of Frankfurt’s analysis: On Bullshit is about expressing sentiments without caring about their truth value, i.e. the concept of a “bull session.”
Both Altman and Zitron are the opposite of Frankfurt’s bullshitter, because they both appear to evince genuine care for the truth value of their position.
(Frankfurt is very subtle about this, in that he distinguishes the kind of performative lying you’re suggesting as distinct in moral and rhetorical content from bullshitting. The liar wants you to believe a specific thing; the bullshitter wants to regale you.)
Bullshitting is where you don't care about whether what you say is true. Your goal is to influence a certain way of thinking or feeling, and you just say stuff that you think will cause that in other people. Appearing to care about truth is not opposed to that, it's part of it.
I don’t think these mental states are in evidence for either of them. I think they’re both credibly earnest in their views.
(Frankfurt’s characterization of the bullshitter rests on not just the truth value being absent, but also on the bullshitter’s misrepresentation of what he’s “up to.” I don’t feel that either Zitron or Altman is misrepresenting what they’re getting up to.)
I mean, the point of Frankfurt’s bullshitter is that we know they’re bullshitting (or more expansively, Frankfurt gives us a set of criteria to test them against). We know they don’t care about the truth value of their statement, only that they are misrepresenting themselves because of a hidden “enterprise.” But there’s no such unknown enterprise in either’s case, and neither appears to be misrepresenting themselves (to my point about appearing earnest). Maybe that latter part is itself deception, but without the former they would fall into Frankfurt’s classification of a “liar” instead.
(I have to admit a bias when it comes to Frankfurt: I don’t think On Bullshit is that convincing. In particular, I think Frankfurt doesn’t do a good job of motivating the connection between bull sessions and bullshit in his strong sense, and many of his examples - like the Pascal/Wittgenstein one - don’t demonstrate misrepresentation either.)
> Both Altman and Zitron are the opposite of Frankfurt’s bullshitter, because they both appear to evince genuine care for the truth value of their position.
I can't really judge Zitron, but when it comes to Altman, I have absolutely no idea how you arrived at that assessment. In my view, he's clearly dishonest. I’ll remind you of his attempt to use government intervention to erect barriers to market entry for his competitors. Or his claims about AGI being just around the corner—claims that haven’t come true so far and were clearly aimed at investors. I’ve never heard a single statement from this man that struck me as honest.
I think it’s appropriate in this day that people have to really shout and lament on and on about their thing just to get a few people to listen and/or consider parts of their argument. Nobody listens if you’re just casually proselytizing your grift.
reply