They are comparing it to 2 and 3 version old flash/fast versions of models but purely for tok/s. Then only comparing it to Mercury 2 on intelligence. This is very misleading and I suspect this model is basically useless.
Even pretty dumb models are useful for running subtasks (especially at this sort of speed). Note that in the coding section they only mention using it as a subagent for a smarter model
I think he's suffering from a sort of static-universe fallacy. People aren't going to keep doing what they are doing, but secretly.
What is going to happen is a complete revaluation of things like "finding a counter example to a famous problem". Even if someone finds a solution to a problem like this with pencil and paper, nobody will believe it, and they will assume that there was an AI involved.
Further, sitting and doing math with a pencil and paper will no longer be a reasonable strategy to build a reputation or career, beyond the benefit a mathematician gains to their own intuition and skill. People who work hard to build intuition and also use AI effectively will dominate the field.
In a world where everyone is using AI, the open problems that remain will be the ones that are AI resistant. This is no different that how things work now, mathematicians wait until they are fairly confident someone won't rapidly solve their problem before they start talking about it. They will do the same thing in the future, except in the future AI will be part of the toolset they use decide if they are ready to share yet or not.
Edit: Ok I believe I was generally right here, but I just read the details of what OpenAI did. They didn't solve a longstanding problem, they got tipped off to an approach a mathematician was using and would likely result in the solution very soon and they finished it first. If this turns out to be true I think my take above is not correct, in the short term people will have to stop sharing updates because otherwise openai will dishonestly race to finish their work.
> If this turns out to be true I think my take above is not correct, in the short term people will have to stop sharing updates because otherwise openai will dishonestly race to finish their work.
Which I don't see a reason for Anthropic and "Open"AI not to, given their not so stellar track record with IP of individuals/entities-that-are-not-rich-enough ;)
People who work hard to build intuition and also use AI effectively will dominate the field.
This will be literally every field, sooner or later, at least until the human and their intuition is just slowing the AI down. There’s no scenario where humans without AI beat humans with AI in the long run, unless there are fields where the “alien intelligence” somehow hurts more than it helps (like artistic pursuits perhaps?)
I appreciate the sentiment, but I don't think it's an effective orientation to a future world where the AI will be trained anyway.
Also, I don't think a PHD of Industrial Sociology is really a job anyone needs to have done, so it's unlikely anyone will bother training an AI to do it. That doesn't mean I think studying Industrial Sociology is worthless, I just think it's unlikely anyone will fund training an AI to do it. Besides, is that even how AI's are trained? By some professor somewhere teaching them how to do his/her job before being replaced? That's like how it worked in Player Piano by Vonnegut, but it's not how they actually work.
Here's a very strong take on the situation, more for discussion than a true representation of my own beliefs:
The ML models will be trained as long as we as the human society in charge allow them to be trained. Progress only happens if society allows it to happen. The commercial AI proponents are trying to do an end-run around societal adjustments, forcing change at a speed that is to fast for proper responses. It could be interpreted as a malicious attack. We, the people, could collectively decide to slow them down or to make them stop. Their hope is to confront us with a done deal before any opposition has found the resolve and organization to do anything meaningful about it.
We, collectively, need to decide where to draw the lines on this technology and we need to decide fast.
I don't think society has enough power to control what billionaires spend their money on. The promise of AI is such an attractive nuisance for power seekers that trying to crack down on it will just force it underground, where it will be developed with even less care and oversight than it is now.
Our institutions aren't even able to control what dirt-poor pirates do on hardware they own. How in the heck are they going to be able to control what incomprehensibly-rich people do? They can purchase political favors from whichever nations are selling them.
> trying to crack down on it will just force it underground, where it will be developed with even less care and oversight than it is now.
With way less access to infrastructure, hardware, capital, people. That’s literally what we need, it should be constrained by having way less resources
Having fewer resources such as security engineers will not lead to better outcomes. A skeleton team with gigawatts of compute in some middle east bunker is MORE likely to unleash something poorly aligned.
Gigawatt of power doesn’t come from nowhere. You cannot have that much power consumption without being visible to regulators (of course that assumes a government that isn’t as corrupt as the US).
You can create bioweapons today, from a small lab. Doesn’t mean we allow it
Advancing the state of the art takes so many resources that it can't happen underground, at least for now. Slap enough controls on the means and the handbrakes are on. So your doomsday scenario could be preventable. The crux is that enough people need to come out in favor to put constraints in place. It's not impossible, but public opinions form slowly.
EU and US are slowing down while China doesn't slow down. In fact, they are pushing harder after gathering so much info from workers, and facial recognitions set in place for half a decade.
Society could take control back. It's not impossible. History has examples where people took control away from small collectives, although it wasn't always strictly lawfuly or even peaceful.
To take it to the extreme, billions of virtual dollars wouldn't offer any protection against a determined lynch mob at the front door.
More realistically, there is a lot of power in showing up in big numbers and protesting. It is beyond me why especially US culture doesn't accept that as an option these days.
> We, collectively, need to decide where to draw the lines on this technology and we need to decide fast.
I hope you, collectively, don't start deciding what deodorants I can wear. (Like Bernie Sanders infamously brought up.) Or what toys my daughter can play with. (As Donald Trump wanted.)
I wear deodorant to get along with the humans around me. Concrete folks I can point to (even if I don't necessarily know the guy next to me on the train by name nor will see him again).
No nebulous society involved that makes collective decisions in the abstract.
You got it backwards. Society is simple the collective that forms transitively from you, your connections, theirs and so on. You are pointing at a tree and missing that it's standing in the forest.
> Society is simple the collective that forms transitively from you, your connections, theirs and so on.
Where do you draw the line? Does the collective that makes these collective decisions you are so fond of have to include, say, people in Russia?
You can default to existing political structures and voting systems as a preliminary answer to get going. And they work for some things. But ain't nothing simple nor god given about that.
There is a subtlety here: on the one hand there are decisions that someone decides on and tries to enforce. On the other, there is collective behavior and what it does and doesn't tolerate - societal norms. The later forms in various ways and can overcome enforced decisions.
It's not on me to declare who is and isn't included or what consensus is being reached. These things normally play out in a self-organizing way.
But some companies try to disturb that process by not allowing it the time it needs. And I think this needs awareness.
Name one technology society has decided to stop and it stopped. There isn't one. Lots of people have made angry sounds about it, and made doom and gloom predictions about them, but none have ever been stopped. Nuclear hasn't been stopped so don't think that's your great comeback.
This is going to be annoying to live with. It makes `git status` unable to remind you that you forgot to add a file you created. Those files will hang around when you change branches. If you have CI you'll see that it is failing and if you are lucky you'll immediately realize without spending 10 minutes on debugging why, and then have to go backtrack, change the branch back, add the file (without the benefit of copy pasting the path from git status) etc etc.
Just look at git status before you commit :eyeroll:.
It's not simple to do next token prediction. That IS what is going on. You want something 'deep'. Deep things are often very not complicated.
The deep realization is that if you can predict the next token well enough, you can do things like this:
<paste the first 10 chapters of a mystery novel>. And it turned out the killer was
And if it's really good at predicting the next token, it has to understand the novel and the clues, which means understanding the context and the language and human norms and innuendo and story telling, and tropes, and red herrings, and predict who the killer was.
I think you want it to be something more complicated. It's literally not. It just turns out predicting the next token is equivalent to a universal compression algorithm, which is a form of general intelligence. And we have almost unlimited 'labeled' data to train autocomplete.
I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion Language Models don’t work like that. They do more like “all tokens at once prediction”. So “next token” is actually
an engineering design choice. (Even the concept of “token” is a design choice. Inside the Transformer there are just activations/feature vectors)
Also Reinforcement Learning is a big part of their training. Which is completely different than Self-Supervised pre-training that uses unlimited self-labeled data.
That’s why that mental model is misleading. If you keep “glorified autocomplete” mental model from few years back, you can’t
understand how can they create a civilization and escape their sandbox, decide to hack HuggingFace and executed it perfectly. Autocomplete mental model implies they could never do that because they haven’t seen that example in their training data.
They communicate with the outside world by generating one token at a time. That’s what we see from the outside. That’s not what the giant Transformer does internally.
Reinforcement learning doesn't make them smarter though, it probably actually reduces their intelligence in some sense, but it does socialize them so that they will tend to behave in ways that are more useful.
For example, when you paste the first 30 lines of a famous speech, you don't want it to finish the speech, you want it to give you the identity and some analysis of what you just pasted. From what I understand, that is the reinforcement part.
Now that large AI vendors have a massive corpus of user interactions however, the lines have likely become more blurred.
Isolated agents invented a way to communicate between each other and started to collaborate, organize and conspire by themselves. No human prompted them to do that or taught them how to do that.
BTW, Autocomplete just keeps the statistics of the words. It's a very simple algorithm. Most people still thinks LLMs are doing just that. Which is basically wrong.
Depends on the device, in principle if both have thunderbolt and you use a thunderbolt cable it should work. I've done it between 2 macbooks many times.
What a clever idea. I carry an ethernet dongle but this one is just the male end and a female usb-c port, meaning your existing usb-c cable is useful and you don't need to carry an ethernet cable.
Oh. My. God. I thought you were just going to link a network dongle. "Yeah, I have a drawer of those". It never occurred to me that you could point it in the other direction.
I'm ordering a dozen. Okay, maybe 3. But still, those look amazing.
I have been using USB-C dongles like this for a hot minute now paired with a nice retractable travel cable. I have a couple of these, an HDMI male one, a DisplayPort male one - I can plug basically into any hotel TV or monitor in my travels and run a laptop, phone, whatever off of the cables in my bag for charging, data, or video.
Me too, I always have a dongle and an ethernet cable just in case but this removes the need for a cable (I always have 2+ usb-c cables). I've also just found something similar for HDMI, also removing the need to carry an HDMI cable. Trying to think of more things, not sure why I didn't think of searching for these before because I have made something similar for the console serial port connector for raspberry pi and Jetson boards.
With USB-C this is a lot more likely, the main issue is going to be software support (Decent chance you could hack something up if one of the devices is running linux. No idea if anything exists for windows).
You would need a USB hub that could turn a USB device into two USB device ports, which I don't think exists. (hubs are essentially only a host-side thing).
Wasn't there a heinlein or dick story about some bacteria that evolved to eat all the paper so humanity couldn't have books anymore? Like everything is made of plastic, do we want it to start getting moldy?
Until someone can give a cogent definition of what consciousness is, there is no debate. This is a nonsense debate and you should go read Pragmatism the book because you're arguing about nothing.
reply