Does regulators really care about a predicted age? I feel like they require hard proof of being above age to show explicit content. The only ones that care about predicted age is advertisers.
It's not much for the regulators as much as its for the advertisers.
At this point, just use gemini (yes its google and has its issues if you need SOTA) or I have recently been trying out more and more chat.z.ai for simple text issues (like hey can you fix this docker issue etc.) and I feel like chat.z.ai is pretty good plus open source models (honestly chat.z.ai feels pretty SOTA to me)
Kagi's Assistant is the most useful tool I've found as far as searching goes, and occasionally simple codegen. Let's you use a wide variety of models and isn't tracking me.
Edit from my previous comment: Actually tried Kagi assistant through orion/signed up and its really good (GLM 4.7) but still there is some amount of tracking/logs still kept
I also didn't find in kagi what provider its using for glm 4.7 (I am assuming the same as glm 4.6) which is Cerebras,DeepInfra,Fireworks.ai and some of these use large companies like aws etc. so you are still putting trust into them
I somehow made kagi assistant mention proton lumo and it sort of agreed with me that its a good option too
I think Proton lumo (for simple queries, although i still don't know which model they use & it's pretty restrictive plus I wish they might have used glm 4.7) but its good.
Cerebras terms and policy needs to be seen again by me but I had talked to cerebras on discord once and they mention that they don't log too but I might ask them again but cerebras might make more sense but for basic codegen proton's lumo is good too.
My eyes are on proton lumo & confer.to but kagi's pretty good too (as much as they can be fwiw but they still rely on api, I feel like kagi's make sense if you are already using kagi search or want AI to use kagi search feature imo)
If we are talking about complete privacy. I am trying out https://confer.to (created by signal team) too and I am unable to run it on my mac (passkey support)(I tried it both in zen & orion) but I tried it on an android chrome (tablet) and I am kinda more optimistic about it too
I have heard good things about Kagi in fact, that's the reason why I tried orion and still have it in the first place but I haven't bought Kagi, I just used the free searches orion gives & I don't know if it has Kagi's assistant.
I think proton's Lumo is another good bet.
If you want something to not track you, I once asked cerebras team on their discord if they track the queries and responses from their website try now feature and they said that they don't. I don't really see a reason why they might lie about it given that its only meant for very basic purposes & they don't train models or anything.
You also get one of the fastest inferences for models including GLM 4.7 (which z.ai uses)
You might not get search results though but for search related queries duck.ai's pretty good and you can always mix and match.
But Cerebras recently got a 10 billion $ investment from OpenAI and I Have been critical of them from now on so do be wary now.
Kagi Assistant does seem to be good if someone already uses Kagi or has a subscription of it from what I feel like.
In the UK, age verification just has to be "robust" (not further specified). Face scanning, for example, is explicitly designated as an allowed heuristic.
Any proof that you are above a certain age will also expose you identity.
That is the only reason regulators care about children safety online, because they care about ID.
LLMs are very good at profiling users in hacker news and finding alt accounts, for example. profiling is the best use case for llms.
So there you go, maybe it wont give exactly what regulators say they want, but it will give exactly what they truly want.
I think this is kinda sad. I always thought coding was so beautiful for creating a (good paying) job for socially awkward people on the spectrum who pre-coding was considered dorks and went into idk what but probably some unsatisfying job. It seems like we're heading back to that which makes me sad and worried about the highly functioning autistic population.
I am sorry but is being employed a job or a daycare?
Not only this is extremely patronizing towards all people on spectrum, but at the same time extremely hurtful statement for people who are treating employment as a job(ie - most of population).
And what are you going to say to people who are stuck in low-end jobs?
"Conflict is essential to human life, whether between different aspects of oneself, between oneself and the environment, between different individuals or between different groups."
The obvious default interpretation of your comment is that the other person is using something as an excuse. If you say you did not intend it as a personal putdown, I believe you, but the rest of us don't have direct access to your intent, so you'd need to include enough information in your comments to disambiguate it.
As for Marion Milner's classic paragraph, I'm delighted that you found it worthwhile enough to quote! But you have to read the entire paragraph to understand it, and you can't leave out the word "toleration". That's the most important word there, both in the text and in the title.
Sorry but did we read the same comment? It's not patronising. The people who are stuck in low end jobs were not in the scope of this comment (there are also people in war zones or very sick, also out of scope). And how did you manage to find this extremely hurtful to any group...?
I heard that there's so many variants of string theory. All theories that have interesting low-energy predictions have lost to standard theory (see: SUSY experiments at LHC). Now we're left with high-energy ones which may actually be wrong too but we can't test them yet.
String theory isn't a theory it's a family of related theories sharing some common mathematical tools.
People talk about this as though it's an attempt at deception, whereas two people notionally working in string theory could in fact be proposing highly incompatible models which would be conclusively ruled out (and a lot of them have been in so far as that can be done - i.e. experimentation has put tight bounds on their possible parameters).
They are getting close to making a testable prediction, any day now. Have been for the last 30 years. History is not always an indication of the future, but it is often a good sign.
It being activated by microphones makes me think you could add speakers to this tiny format and make a tiny digital instrument that's influenced by blow intensity etc.
There are a lot of very strong opinions on relatively minor variances. I really like this font, but apparently it isn't nearly as complete as some alternatives.
I've tended towards fonts that I just find readable at the relatively small sizes most sites tend to use. I like Roboto a lot, I like this slightly more... I'm not as big on the Libre Franklin it's also being compared to. It's really personal and some people care more or less than others depending on their needs, and even visibility concerns.
The best counter argument to that is that he did manage to predict/make into reality electric vehicles (when going into that industry was crazy) and reusable rockets. If someone makes a thousand moonshot attempts but still succeeds with two that's impressive.
Nobody with any knowledge at all is claiming that Elon Musk invented electric cars.
The simple truth is that he made electric cars viable competitors to gas-powered cars. His genius is not that he invented them, it's that he profitably manufactured decently reliable cars for a price that lots of people found attractive.
You can try and dismiss it as "marketing," but things like the Gigapress and FSD/Autopilot are impressive technical achievements in their own right. Even more impressive is that he built up a new car company that didn't fold and has had the best selling car in the US for significant chunks of time.
I don't like the guy, I think that FSD is dangerous, and I will never buy a Tesla for as long as he's in charge, but it's crazy that so many people feel the need to discredit his achievements. Sure, he benefited from selling carbon credits and EV subsidies, but if it were such an easy thing to do why did it take so long for anyone else to sell a good EV?
Gigapress has almost nothing to do with Tesla. It is just the name given by Tesla to a process they purchase from a third party vendor(Idra Group). Tesla was the first to use this product for large scale automotive production though.
You say that like they bought something off the shelf which just worked the first time they used it. They did not - it was a collaboration and Tesla spent a lot of money and time to get it to work.
I'm not gonna link the articles, but there are photos of the mountains of defective parts and plenty of people complaining about how terrible the first cars produced that way were. Tesla persevered and now other car manufacturers are trying to duplicate their results.
That's very silly. Weed doesn't turn people into habitual liars. Secondly, he was abusing drugs before that interview. Thirdly, he was telling absurd lies before that interview too. The hand wringing about him smoking a blunt is absurd, he doesn't have "reefer madness".
It's not the weed that fried his brain, it's the ketamine. That moment where he smoked up on camera seems to coincide pretty well with him losing his mind, though.
People get ketamine treatments for depression all the time. It's not drugs, he's just a nasty person who's been good at manipulating people in the past. People have just finally caught on to the con, at least in part because he's terminally online.
Before he smoked that reefer, his space company was catching the largest booster ever made with metal chopsticks, all paid for by global satellite internet revenue.
His electric crossover/SUV was the best selling car in America.
Now that he’s gotten distracted by politics I dislike, he’s not doing any of that. Definitely no longer the world’s greatest builder.
> all paid for by global satellite internet revenue.
Huh? You think starlink is funding space-x? If they lost all government and private launch business tomorrow and had to rely on Stalink revenues to stay in business they wouldn’t last through next month.
> His electric crossover/SUV was the best selling car in America.
It was, and then he fried his brain and decided to support fascists across the globe and can’t understand why people no longer want to support him or his businesses.
He apparently watched handmaid’s tale and thought “man those Gilead guys are really onto something”.
I dont think Starlink can actually make money without government subsidies and a whole lot of inactive users. It simply cannot scale, the width spot beams are limited by physics - they cannot get small enough to get the density needed.
I think that's the point? I'd always assumed Starlink was a way to fill in coverage gaps in low-density areas where cable would cost more than it was worth, not cities?
In the last year alone, around 2/3 of space X's revenue was directly tied to starlink launches.
> If they lost all government and private launch business tomorrow and had to rely on Stalink revenues to stay in business they wouldn’t last through next month.
> the truth is that he's accomplished some crazy things.
I would argue: yes, to the extent that a leader gets to be described as having "accomplished" the work of the team.
It's not nothing, to be a visionary and charismatic leader!
But at the same time… when the reality distortion field seems to be in the process of transforming into a cult of personality, I think it's fair to ask if he'll ever again do something like a new SpaceX or a new Tesla, either as a maker or an investor.
I'm not sure when the cut-off between the two states, RDF and cult, would be. Not unreasonable to say it was when he libelled the cave diver, but there are other times it could've been.
> Nearly all of his profit was government subsidies designed to push EV adoption
The government subsidies were available to his competitors at the time. Its not like that gave him a competitive advantage. Everyone else was on the same playing field.
> Electric vehicles were the first types of cars invented.
Where is this claim coming from? I don't see that in the history of the automobile wiki [1], and given that the first early motorized carriages were a century before Ben Franklin flew a kite I have to assume they were electric vehicles.
Something is missing here. Once you get two moonshots done, you have free pass to claim anything any number of times with zero results? I cannot agree.
That’s underselling the Leaf quite a lot. The original 2011 model had 107 HP and 207 ft-lb of torque (later bumped to 147 and 236, respectively), which puts it handily above several gas models of gas cars that don’t get labeled as golf carts. It was a perfectly fine car, it just had a poor battery.
> The issue is it had the range of a golf cart. So it basically ruled out 98% of the population that needs a car that can go on road trips.
You're trying to use weasel words to try to hide the fact that the Nissan Leaf, which was released in 2010 and elected world car of the year, was the world's most successful electric car and top-selling electric car until 2020.
That does not happen if 98% of anything doesn't like it.
Any claim involving "road trips" is a red herring because the Nissan Leaf was designed as a city car used in daily commutes, which means a daily driver for your 1h trips. This is by far the most popular use of a car in the world.
Why do you think it's design range was slightly over 300km? That roughly represents a ceiling of a round trip that takes 2 hours each direction.
For over a decade, the whole world has been buying Nissan Leafs more than any other electric car. How do you explain it?
> That does not happen if 98% of anything doesn't like it.
Actually it does. Electric car sales were so anemic during that time claiming the title made it trivial to be supported by 2% of the population.
> Any claim involving "road trips" is a red herring because the Nissan Leaf was designed as a city car used in daily commutes, which means a daily driver for your 1h trips. This is by far the most popular use of a car in the world.
No it’s not. “Range anxiety” was a constant refrain for anything mentioning electric cars during the first 20 years of the century.
A “city car” isn’t a concept in the US. Only when you get into upper middle class where people can afford multiple cars per household is when you could sacrifice one car like this.
I dunno, as someone who was raised in a pretty rural area and has since lived in both cities and suburbs, I think the need for long distance driving is dramatically overstated.
From my rural hometown, the drive to varying degrees of civilization (just big enough to have a small shopping center up to the state capital) is about 25 and 75 miles, respectively. Cities sized in between are around 40-50mi out. The drive to the nearest tiny town for groceries and such is about 2 miles.
I currently live in a suburb and everything one might need, including an international airport, is within a 30mi radius, with the majority of that being within a 5mi radius.
With that in mind and remembering that the bulk of the population lives in cities or their surrounding metro areas, "city cars" are viable for more people than they aren't. Sometimes they'd be better suited as secondary vehicles dedicated to errands, which at first glance might seem more expensive, but the dramatically better fuel economy of e.g. a tiny hybrid or even plain gas car quickly adds up, and in states with cheap electricity combined with scheduled charging at off-peak times, the scales are tilted even further if you have a plug-in hybrid or full EV. The up-front cost is higher, but you quickly make that back from not having to haul the big gas hungry SUV or truck around all over the place.
More than half of US households have multiple cars. The market that can handle a limited-range car is enormous; most of those households and many single car households too. And the existence of range anxiety doesn't change that.
> Actually it does. Electric car sales were so anemic during that time claiming the title made it trivial to be supported by 2% of the population.
What are you talking about? The Nissan Leaf was the world's best-selling electric car until 2020, outselling all Tesla's until Tesla Model 3 surpassed it. Are you trying to claim with a straight face that electric cars weren't being used en masse until 2020?
> No it’s not. “Range anxiety” was a constant refrain for anything mentioning electric cars during the first 20 years of the century.
I don't think you are being serious. "Rage anxiety" was literally GM propaganda to throw FUD at electric cars.
As a second car in a two-car family, we love our Leaf. It’s obviously unusable for road trips, but in a country with more registered cars than drivers, there are plenty of multi-car households where one could be a Leaf-class (cheap but still reliable) electric.
Sure, but the original Tesla car received exactly 0 Musk input. That was pretty much a done design when he bought the company. And ofc he ousted the original designers and tried to erase them from history. And the model 3 is pretty much building upon that.
AC propulsion was founded in 1992 and began developing an AC electric powertrain then, using lead acid batteries. By 2003 they had three prototypes built, and in 2003 they converted to lithium ion. At this point they were encouraged to commercialize.
Tesla was founded in 2003, and licensed the power train developed above. Musk bought into the company in 2004. Tesla teamed up with Lotus in 2004. The first Tesla Roadster prototype was shown in 2006 and delivery of production cars began in 2008. By 2009 they had made 500 of them.
I don't like the man very much either, but exaggerating the state of Tesla before Musk was involved is silly. Before the Model S, Tesla was very small and it wouldn't have surprised anybody if it dried up and blew away in the wind.
Yes, early Tesla cabins just oozed luxury, for twice or more what the Leaf cost. :eyeroll: Regardless, Nissan put out production EVs before Tesla did, accouterments aside.
So Elon invented selling a slightly more expensive EV in a state with generous government support for this?
A business plan that the real Tesla founders actually came up with because they'd seen Silicon Valley homes with Porsches and Prius parked next to each other and thought they could combine those two things?
Likewise, but those were
famously slow. Might have been expandable into other delivery vehicles, but neither the
batteries nor the motors were up to being commuter vehicles… well, possibly electric bicycles back then, the European Blue Banana* was better positioned than much of the world to commute by bike, but not much more than that in performance or geography until much more recently.
The Lancet[1] forecasted Musk's 'bit of a jerk' elimination of USAID[1] will cause a death toll that puts him around 10x that of Pol Pot.
> Projections suggest that ongoing deep funding cuts—combined with the potential dismantling of the agency—could result in more than 14 million additional deaths by 2030, including 4·5 million deaths among children younger than 5 years.
Whether or not they're credible to you, they're still the #2 ranked general medicine journal in the world, second only to the NEJM.
> USAID isn’t an aid organization, it’s a front for CIA efforts internationally
This is a conspiracy theory that can be trivially refuted by simply following the money. You can do this because their budget is public, unlike the budget of the CIA. The stuff you're citing from a half century ago isn't relevant to the work they've been doing when Musk said "Time for it to die."
What USAID actually was was a vital tool of US soft power and influence globally, and if you believe that it's important to wipe out the last remaining vestiges of the United States' perception as 'the good guys' then by all means it was very important to stop their work immediately.
> Whether or not they're credible to you, they're still the #2 ranked general medicine journal in the world, second only to the NEJM.
Their ranking is based on how often papers are referenced in total, not the validity of any one paper. As I said, they been criticized for serious lapses in publishing fraudulent papers.
> The stuff you're citing from a half century ago isn't relevant to the work they've been doing when Musk said "Time for it to die."
I'm sorry what? It's the same organization? The fact that USAID was funding paramilitary organizations during war tells you USAID has nothing to do with aid.
> USAID actually was was a vital tool of US soft power and influence globally....if you believe that it's important to wipe out the last remaining vestiges of the United States' perception as 'the good guys' then by all means it was very important to stop their work immediately.
So you're saying unless the US can project soft power and influence (which is ALWAYS to the US' benefit, it absolutely is not altruistic) it won't be viewed as the "good guys"?
Let’s not pretend that Trump knew or cared what USAID was. Musk was extremely hands-on with the dismantling of that agency specifically.
I didn’t share my thoughts, I shared a Lancet article calculating the death toll. I leave the math, the comparisons, and the moral judgments as an exercise for the reader.
I don't speak for mullingitover, but… "other" reasons? Surely all the stuff he's done are the reasons?
And Musk seems to have tarred himself:
Tesla sales are down a lot even in places where the market is growing, in part because it was lefty liberals who were the original primary market for EVs.
Musk's support for Trump (who openly hates eco-friendly anything and appears to be tanking the US economy with inflation and tariffs and the only growth sector being AI DCs) also appears to be the reason the entire EV market in the US is going down.
He's also having spats with various national leaders. But… look, in UK, Keir Starmer has catastrophically poor opinion poll ratings, Musk's managing to bob around the same level, slightly worse, in part due to tweeting things seen as calling for a civil war in the UK.
Similar in Germany. Where the Gigafactory is… ah, still a building site, not having needed to expand to the full potential of the water licence it had. (A factoid I only know about due to comparisons with the combined AI data centre use across the state of Arizona).
Reusable rockets are a rehash of old tech that was considered - at the time - not economically feasible; Given how subject to interpretation spacex commercial numbers are, there is nothing indicating a clear cost or efficiency advantage compared with traditional launch systems so far. What we clearly know is that using software development methodologies to building critical hardware is as a bad idea as it sounds.
I’ve got as much of a distaste for Musk as anybody else these days, but SpaceX’s methodology has if nothing else netted them velocity and turnaround times that no other company or governmental space agency has been able to hold a candle to thus far, and do it with a very low failure rate. They’re clearly doing something right.
There is no “subject to interpretation”. The costs they charge for launches are lower than any other provider by a significant margin. And fundraising docs have shown many times that the Falcon launches make money and Starlink was just starting to make money about 1.5 years ago.
> What we clearly know is that using software development methodologies to building critical hardware is as a bad idea as it sounds.
This methodology is what provides high speed, low latency internet to the South Pole and every other spot on earth allowed by regulatory.
Yeah, Falcon rockets are a regular workhorse kinda rockets. Nothing special about them. NASA could have made their own but someone decided it needs to be outsourced.
I mean they did a fine job there, but nothing to write home about IMHO.
And on the topic of reusability I can't really find much info besides that it is just partially reusable. Not sure what the point of it actually is. I guess what matters is the launch price?
The question I still have it, wasn't SpaceX supposed to get USA back on the moon? And I heard they got billions in subsidies but have nothing to show for it.
> The question I still have it, wasn't SpaceX supposed to get USA back on the moon? And I heard they got billions in subsidies but have nothing to show for it.
AFAICT, SpaceX are not the bottleneck holding this back. Or at least, not the only one.
And they do have something to show for it, just not a complete final version. Starship is not yet fully reusable, and I will not make any bet on if they even can make it so as this is not my domain, but if you skip the re-use it is already capable of yeeting up a massive payload to LEO, enough to do a lunar mission.
It’s a commercial launch company. Of course the price matters and it being so much cheaper than the trash from ULA, Russia, etc is why there has been an explosion in new space endeavors (see the bandwagon launches).
> Nothing special about them. NASA could have made their own but someone decided it needs to be outsourced.
“Anyone could have done it bro,” is such an ignorant response. Nobody did it and there was the entire launch industry to collect if they did.
Even if NASA could have, they were derelict of duty in enabling space utilization because they never did it.
> And I heard they got billions in subsidies but have nothing to show for it.
Should probably check stuff before you repeat it. SpaceX has not received billions in subsidies for going to the moon. It did win a contract to do it, which as the name implies has required deliverables.
Its a private startup. It may operate on a loss, leveraged by private equity and government contracts.
Everything else you mention becomes irrelevant. Until we know the costs and operational margins, there is no certainty if they are delivering what they promised.
Spacex is a private company; this means "we" know nothing about actual costs. Fundraising documents dont show this either, as they are a washed-down version for, well, fundraising purposes. As an investor, it is common practice to sign an NDA just to get access to actual somewhat relevant numbers, so any actual relevant info isnt public.
Also it seems you conflate "making money" with being profitable - its not the same thing. A private company can easily "massage" the PNL sheet to present itself as at a break-even point, and some back-of-the-napkin calculation seems to point to it. Granted, I may be wrong, but the fact is we don't know for sure.
You also seem to not be aware that there are multiple internet satellite providers with south pole coverage, as well as other regions in the globe.
> Spacex is a private company; this means "we" know nothing about actual costs. Fundraising documents dont show this either, as they are a washed-down version for, well, fundraising purposes. As an investor, it is common practice to sign an NDA just to get access to actual somewhat relevant numbers, so any actual relevant info isnt public.
None of this is correct. You don’t get fidelity as an investor repeatedly publishing fraudulent documents.
Also, it’s not like spacex can hide costs. There is no other supply of money to cover operations.
> You also seem to not be aware that there are multiple internet satellite providers with south pole coverage, as well as other regions in the globe.
They are a joke. Completely different leagues of access. Coverage of the South Pole (not McMurdo) got effective continuous bandwidth around the throughput of dialup and periodic passes from a polar sat to upload scientific data.
GEO is absolutely terrible in terms of latency and cost. Starlink is currently the only good option for the entire ocean and any remote place on earth not reachable by fiber infra.
The only up and coming potential competitor is Amazon’s Kuiper/Leo. China is also experimenting here but it’s not clear that will be available to the world.
Claiming there are alternatives to Starlink is extremely ignorant. It only takes a brief glimpse of what it’s doing to both maritime and aviation to understand that it’s unique.
> None of this is correct. You don’t get fidelity as an investor repeatedly publishing fraudulent documents.
Did I say they were fraudulent? I'm merely stating that tag price means nothing, as they probably are "selling" it at a loss (btw the initial projected falcon price was 10 mil per launch, and the current tag price is ~60 mil, with no strong stats nor costs on reusability). The only way to know for sure is to have access to privileged info behind an NDA. Do you even know what you're talking about? Have you ever reviewed this kind of documents?
> They are a joke. Completely different leagues of access. Coverage of the South Pole (not McMurdo) got effective continuous bandwidth around the throughput of dialup and periodic passes from a polar sat to upload scientific data.
South pole coverage is relevant for like, 3 people. None of the data collected from/to there requires urgency; there is zero scientific advantage other than quality-of-life. Consider this, we receive scientific data from mars.
> GEO is absolutely terrible in terms of latency and cost. Starlink is currently the only good option for the entire ocean and any remote place on earth not reachable by fiber infra.
Remote places tend to have no coverage, because they have no subscribers. Not sure what you think a profitable business is, but you come off as really asinine. There is nothing inherently unique to starlink - except the fact that they're polluting LEO with their garbage. If its sustainable or not, time will tell.
tbh, it still isn't economically feasible. spacex 'cheated' to achieving reuse by just making the the entire plumbing and engine assembly bolt-on to the lower stage on F9 and they just replace that every time one is 'reused'. to my knowledge, they still haven't reused an engine without either replacing the nozzle, turbopumps or both, which are so expensive that reuse might actually cost them more money in the end for the benefit of faster turnaround times in years where launches are booked heavily.
Ultimately capitalism doesn't care if a job is fun or not. The vast majority aren't I've realized. It's an odd bit of coincidence that coding with flow is hugely enjoyable but it seems like that amazing job at this rate will be a momentary bit of history where profit making and fun had a non zero intersection for a strange reason.
Don't really see why you'd need to understand how the transformer works to do LLMs at work. LLMs is just a synthetic human performing reasoning with some failure modes that in-depth knowledge of the transformer interals won't help you predict what they are (just have to use experience with the output to get a sense, or other peoples experiments).
In my experience this is a substantial difference in the ability to really get performance in LLM related engineering work from people who really understand how LLMs work vs people who think it's a magic box.
If your mental model of an LLM is:
> a synthetic human performing reasoning
You are severely overestimating the capabilities of these models and not realizing potential areas of failure (even if your prompt works for now in the happy case). Understanding how transformers work absolutely can help debug problems (or avoid them in the first place). People without a deep understanding of LLMs also tend to get fooled by them more frequently. When you have internalized the fact that LLMs are literally optimistized to trick you, you tend to be much more skeptical of the initial results (which results in better eval suites etc).
Then there's people who actually do AI engineering. If you're working with local/open weights models or on the inference end of things you can't just play around with an API, you have a lot more control and observability into the model and should be making use of it.
I still hold that the best test of an AI Engineer, at any level of the "AI" stack, is how well they understand speculative decoding. It involves understanding quite a bit about how LLMs work and can still be implemented on a cheap laptop.
But that AI engineer who is implementing speculative decoding is still just doing basic plumbing that has little to do with the actual reasoning. Yes, he/she might make the process faster, but they will know just as little about why/how the reasoning works as when they implemented a naive, slow version of the inference.
What "actual reasoning" are you referring to? I believe you're making my point for me.
Speculative decoding requires the implementer to understand:
- How the initial prompt is processed by the LLM
- How to retrieve all the probabilities of previously observed tokens in the prompt (this also help people understand things like the probability of the entire prompt itself, the entropy of the prompt etc).
- Details of how the logits generate the distribution of next tokens
- Precise details of the sampling process + the rejection sampling logic for comparing the two models
- How each step of the LLM is run under-the-hood as the response is processed.
Hardly just plumbing, especially since, to my knowledge, there are not a lot of hand-holding tutorials on this topic. You need to really internalize what's going on and how this is going to lead to a 2-5x speed up in inference.
Building all of this yourself gives you a lot of visibility into how the model behaves and how "reasoning" emerges from the sampling process.
edit: Anyone who can perform speculative decoding work also has the ability to inspect the reasoning steps of an LLM and do experiments such as rewinding the thought process of the LLM and substituting a reasoning step to see how it impacts the results. If you're just prompt hacking you're not going to be able to perform these types of experiments to understand exactly how the model is reasoning and what's important to it.
But I can make a similar argument about a simple multiplication:
- You have to know how the inputs are processed.
- You have to left-shift one of the operands by 0, 1, ... N-1 times.
- Add those together, depending on the bits in the other operand.
- Use an addition tree to make the whole process faster.
Does not mean that knowing the above process gives you a good insight in the concept of A*B and all the related math and certainly will not make you better at calculus.
I'm still confused by what you meant by "actual reasoning", which you didn't answer.
I also fail to understand how building what you described would not help your understanding of multiplication, I think it would mean you understand multiplication much better than most people. I would also say that if you want to be a "multiplication engineer" then, yes you should absolutely know how to do what you've described there.
I also suspect you might have lost the main point. The original comment I was replying to stated:
> Don't really see why you'd need to understand how the transformer works to do LLMs at work.
I'm not saying implementing speculative decoding is enough to "fully understand LLMs". I'm saying if you can't at least implement that, you don't understand enough about LLMs to really get the most out of them. No amount of twiddling around with prompts is going to give you adequate insight into how an LLMs works to be able to build good AI tools/solutions.
1) ‘human’ encompasses behaviours that include revenge cannibalism and recurrent sexual violence —- wish carefully.
2) not even a little bit, and if you want to pretend then pretend they’re a deranged delusional psych patient who will look you in the eye and say genuinely “oops, I guess I was lying, it won’t ever happen again” and then lie to you again, while making sure happens again.
3) don’t anthropomorphize LLMs, they don’t like it.
I read through 80% of them last night by myself. I mean, I didn't go to bed until 3am but spread across a handful of agents? yeah you could do it in an hour.
Would be interesting if the US decides to ban or heavily tariff these chips and if the consequence will be significanly cheaper data center access through chinese-owned sites/platforms
reply