Calling PETG "utterly problem free" is quite a stretch lol. PLA is pretty objectively much easier to print than PETG, and perhaps than all the popular filament types out there, especially if you are trying to print anything where precision/detail matters. .
PETG is just oozier and stickier by default, so stringiness is almost guaranteed to happen, bridging at a greater risk of failure, etc. It is tougher, so unless you have a printer that can use multiple filaments on the same print, removing supports is more difficult.
Can you reduce these factors by tuning your 3D printer - yes, a bit. But that's not "utterly problem free".
PLA is the plug and play of the 3D printing world right now.
When you print objects with 10's of printers 'tuning your 3D printer' is no longer an option other than to tune it to be 'in spec' You can only tune your designs and the profile for your filament and for a particular model of printer but then all of those have to be close to identical. As soon as you start tweaking your design or filament profile to offset possible issues with the printer you've lost reproducibility.
Incidentally, a lot of the stuff on thingiverse and other similar sites suffers from those kind of issues. They are tuned for PLA on a particular printer without realizing it.
In many ways HN is Reddit in denial at this point :) Comments and upvotes that are based mostly on vibes, with depth and discussion usually happening somewhere towards the middle of the comment tree.
I'd say the author's thoughts are valid for basic data processing. Outside of that, most of claims in this article, such as:
"We're moving towards a simpler world where most tabular data can be processed on a single large machine1 and the era of clusters is coming to an end for all but the largest datasets."
become very debatable. Depending on how you want to pivot/ scale/augment your data, even datasets that seemingly "fit" on large boxes will quickly OOM you.
The author also has another article where they claim that:
"SQL should be the first option considered for new data engineering work. It’s robust, fast, future-proof and testable. With a bit of care, it’s clear and readable." (over polars/pandas etc)
This does not map to my experience at all, outside of the realm of nicely parsed datasets that don't require too much complicated analysis or augmentation.
Author here. Re: 'SQL should be the first option considered', there are certainly advantages to other dataframe APIs like pandas or polars, and arguably any one is better in the moment than SQL. At the moment Polars is ascendent and it's a high quality API.
But the problem is the ecosystem hasn't standardised on any of them, and it's annoying to have to rewrite pipelines from one dataframe API.
I also agree you're gonna hit OOM if your data is massive, but my guess is the vast majority of tabular data people process is <10GB, and that'll generally process fine on a single large machine. Certainly in my experience it's common to see Spark being used on datasets that are no where big enough to need it. DuckDB is gaining traction, but a lot of people still seem unaware how quickly you can process multiple GB of data on a laptop nowadays.
I guess my overall position is it's a good idea to think about using DuckDB first, because often it'll do the job quickly and easily. There are a whole host of scenarios where it's inappropriate, but it's a good place to start.
I think both of us are ultimately wary of using the wrong tool for the job.
I see your point, even though my experience has been somewhat the opposite. E.g. a pipeline that used to work fast enough/at all up until some point in time because the scale of the data or requirements allowed it. Then some subset of these conditions changes, the pipeline cannot meet them, and one has to reverse engineer obscure SQL views/stored procedures/plugins, and migrate the whole thing to python or some compiled language.
I work with high density signal data now, and my SQL knowledge occupies the "temporary solution" part of my brain for the most part.
Yeah, my experiences match yours and I very, very much work with messy data (FOIA data), though I use postgres instead of duckdb.
Most of the datasets I work with are indeed <10GB but the ones that are much larger follow the same ETL and analysis flows. It helps that I've built a lot of tooling to help with types and memory-efficient inserts. Having to rewrite pipelines because of "that one dataframe API" is exactly what solidified my thoughts around SQL over everything else. So much of my life time has been lost trying to get dataframe and non-dataframe libraries to work together.
Thing about SQL is that it can be taken just about anywhere, so the time spent improving your SQL skills is almost always well worth it. R and pandas much less so.
I advocated for a SQL solution at work this week and it seems to have worked. My boss is wary of the old school style SQL databases with their constraints and just being a pain to work with. As a developer, these pains aren't too hard to get used to or automate or document away for me and never understood the undue dislike of sql.
The fact that I can use sqlite / local sql db for all kinds of development and reliably use the same code (with minor updates) in the cloud hosted solution is such a huge benefit that it undermines anything else that any other solution has to offer. I'm excited about the sql stuff I learned over 10 years ago being of of great use to me in the coming months.
The last product I worked heavily on used a nosql database and it worked fine till you start tweak it just a little bit - split entities, convert data types or update ids. Most of the data access layer logic dealt with conversion between data coming in from the database and the guardrails to keep the data integrity in check while interacting with the application models. To me this is something so obviously solved years ago with a few lines of constraints. Moving over to sql was totally impossible. Learned my lesson, advocated hard for sql. Hoping for better outcomes.
I totally understand the apprehension towards SQL. It's esoteric as all hell and it tends to be managed by DBAs who you really only go to whenever there's a problem. Organizationally it's simpler to just do stuff in-memory without having to fiddle around with databases.
My experience is that if you want to do ML, viz, or advanced analytics, dataframes give a better experience.
If you are shuffling data around in pipelines, sure, go for SQL.
Readability is in the eye of the beholder. I much prefer dataframes for that, though a good chunk of the internet claims to throw up in their mouths upon seeing it...
SQL is popular because everyone can learn and start using it after a while. I agree that Python sometimes is a better tool but I don't see SQL going away anytime.
From my experience, the data modelling side is still overwhelmingly in SQL. The ingestion side is definitely mostly Python/Scala though.
I'm running duckdb over 500gb of parquet on a largish desktop (50gb ram) and it's been smooth & fast. I guess OOM issues will matter at some point, but I think it's going to be in the top 1% of real world use cases.
With a decent SSD (or eight), spilling to disk is really not bad these days! Yes!
And if that's still not enough, if you just need to crunch data a couple times a week, it's not unreasonable to get a massive massive cloud box with ridiculous amounts of ram or ram+SSD. I7i or i8g boxes. Alas, we have cheap older gen epycs & some amazing cheap motherboards but RAM prices to DIY are off the charts unbelievable, but so be it.
Polars also has all of these benefits (to some degree), but also allows for larger-than-memory datasets. Also has GPU backend, distributed backend, etc. Polars is heavily underrated, even with the recent hype.
I downvoted all your recs for polars, 1 because this is a DuckDB thread and it's low-key rude, and 2 because there are 4 of them. I wouldn't have minded if there were a single post that were like "DuckDB is cool, polars could be an alternative if..."
That sounds damned near useless for typical data analysis purposes and I would very much prefer a distributed system to a system that would take an hour to fill main memory over its tiny network port. Also, those cost $400/hr and are specifically designed for businesses where they have backed themselves into a corner of needing to run a huge SAP HANA instance. I doubt they would even sell you one before you prove you have an SAP license.
For a tiny fraction of the cost you can get numerous nodes with 600gbps ethernet ports that can fill their memory in seconds.
Which is a lot for a single user, but when you have a dozens or hundreds of analysts who all want to run their own jobs on your hundred terabyte data warehouse then even the largest single machine wont cut it.
> "SQL should be the first option considered for new data engineering work. It’s robust, fast, future-proof and testable. With a bit of care, it’s clear and readable." (over polars/pandas etc)
SQL has nothing to do with fast. Not sure what makes it any more testable than polars? Future-proof in what way? I guess they mean your SQL dialect won't have breaking changes?
I’m also a duckdb convert. All my notebooks have moved from Pandas and polars to Duckdb. It is faster to write and faster to read (after you return to a notebook after time away) and often faster to run. Certainly not slower to run.
My current habit is to suck down big datasets to parquet shards and then just query them with a wildcard in duckdb. I move to bigquery when doing true “big data” but a few GB of extract from BQ to a notebook VM disk and duckdb is super ergonomic and performant most of the time.
It’s the sql that I like. Being a veteran of when the world went mad for nosql it is just so nice to experience the revenge of sql.
I personally find polars easier to read/write than sql. Especially when you start doing UDFs with numpy/et. al. I think for me, duckdb's clear edge is the cli experience.
> It is faster to write and faster to read
At least on clickbench, polars and duckdb are roughly comparable (with polars edging out duckdb).
I use them both depending on which feels more natural for the task, often within the same project. The interop is easy and very high performance thanks to Apache Arrow: `df = duckdb.sql(sql).pl()` and `result = duckdb.sql("SELECT * FROM df")`.
For the bigger tasks, Exasol might also be a very neat option for you. We have a free personal edition that can scale regarding data volumes, #servers (MPP architecture) and complex workloads.
Recently, we have also compared ourselves against DuckDB and were 4 times faster even on a single node. We are in-memory optimized, but data doesn't need to fit in the RAM.
Author here. I wouldn't argue SQL or duckdb is _more_ testable than polars. But I think historically people have criticised SQL as being hard to test. Duckdb changes that.
I disagree that SQL has nothing to do with fast. One of the most amazing things to me about SQL is that, since it's declarative, the same code has got faster and faster to execute as we've gone through better and better SQL engines. I've seen this through the past five years of writing and maintaining a record linkage library. It generates SQL that can be executed against multiple backends. My library gets faster and faster year after year without me having to do anything, due to improvements in the SQL backends that handle things like vectorisation and parallelization for me. I imagine if I were to try and program the routines by hand, it would be significantly slower since so much work has gone into optimising SQL engines.
In terms of future proof - yes in the sense that the code will still be easy to run in 20 years time.
> I disagree that SQL has nothing to do with fast. One of the most amazing things to me about SQL is that, since it's declarative, the same code has got faster and faster to execute as we've gone through better and better SQL engines.
Yeah, but SQL isn't really portable between query all query engines. You always have to be speaking the same dialect. Also, SQL isn't the only "declarative" dsl, polars's lazyframe api is similarly declarative. Technically Ibis's dataframe dsl also works as a multi-frontend declarative query language. Or even substrait.
Anways my point is that SQL is not inherently a faster paradigm than "dataframes", but that you're conflating declarative query planning with SQL.
I think some people overblow the lawsuit risk in the US. It really does suck here, however one of the benefits to certain types of innovation is that the US has a lot of IP protection infrastructure. Which stiffles innovation in a lot of ways, but also makes investment easier in some cases.
This is true in a way. We are all very free to research and innovate, it is just when you get it in your mind to actually make any money that the lawsuits show up.
Legally risky research, but if it has high enough rewards will eventually end up in the hands of extremely large companies that have the legal backing to do anything they want.
Momentum is a large part. I also do think there's somewhat of a motivation that once you've gotten to the top you can sue people who try to displace you into oblivion - ye olde classic "temporarily embarrassed millionaires" syndrome.
It is, at this point, an easily verifiable fact that software engineering jobs in Europe are en masse poorly paid relative to the equivalent positions in the US, and the majority of the EU companies that actually pay well are actually US based. This causes a lot of outflow of talent. That VC funding in Europe is atrocious is obvious to anyone who has had to go through it, and have fun letting a poor performer go from your 5 person startup.
Your polite wishful responses are frustrating to read to anyone who's had to go through this hell.
Yes that is what OP is saying and OP is correct. If you are young, talented and you have a crazy idea that might just work and if you are given the opportunity then you go to the US every-time.
Why? Because it makes sense. Why stay in Berlin or Paris where you can make 50K to 70K euros a year at most and pay close to 40% or 50% in taxes when you can make double or triple that in the US or better yet, start your own company there and then expand in Europe after building it knowing that if you eventually sell it, you get to keep a lot of the sale price.
Talented people don't work for nothing. Motivated and ambitious people don't work for nothing.
If Europe wants to see it's own tech giants emerge, then it's needs to compensate founders and employees well. That's as simple as it is.
Unfortunately it's just not the case at the moment and until that changes, the most ambitious Europeans will continue leaving and building companies on the other side of the Atlantic.
> Why stay in Berlin or Paris where you can make 50K to 70K euros a year at most and pay close to 40% or 50% in taxes when you can make double or triple that in the US
It always amazes me that there is a subset of EU people here on HN who are so detached from reality that it makes me wonder if these same people are sitting in Brussels board making decisions. According to the sh*thole in which EU ended up, it is not completely implausible. Can't agree more with what you're saying. My experience is the same.
It's frustrating really to have these conversations with people who will deny the reality this much.
Instead of using this opportunity to ask the questions that everyone should be asking, which are amongst others "how can we fix this mess?" , you end up with people who think you are simply attacking the EU for spite when it couldn't be further from the truth.
It's sad but not surprising and until that changes then nothing will change.
Europeans need to start being humble again and need to get to work otherwise then we shouldn't be surprised if the US squeezes us from one side while China out-innovates us from the other side. In the meantime the EU is debating if/when it should amend it's cookie law, I am not sure if I should laugh or cry at this point.
> you end up with people who think you are simply attacking the EU
That's not what I think. But you obviously lack the level of empathy you would need to accept that I may think differently. I accept that you think the way you think: you benefit from it as a business owner.
> Instead of using this opportunity to ask the questions
Yeah, because you are so superior to everybody else that people should ask you question, and their opinions are not worth your time.
"I am the best, I am so frustrated that so many people don't recognise it".
When one has no response it's easier to blame the messenger. A nice cop out if I may say so.
> Yeah, because you are so superior to everybody else that people should ask you question, and their opinions are not worth your time
You are putting words in my mouth and arguing in bad faith but that's easier than making a compelling case with facts and figures.
> "I am the best, I am so frustrated that so many people don't recognise it".
If I was the only one highlighting these problems then maybe you would have point but as I have demonstrated in my other comments, the EU commission disagrees with you, the Draghi report disagrees with you and a few people here also disagree with you.
Unfortunately instead of engaging in a healthy debate and addressing the points that have been raised, you have brushed aside all the arguments and refuse to respond with anything that supports your point of view.
You decided that you were right so you are right but you don't seem to have a problem calling out other people by telling then that their opinions are not facts, while yourself ignoring all the facts presented to you. That is a clear case of double standards if I have ever seen one.
> Unfortunately instead of engaging in a healthy debate
Take it from my point of view: I said that I don't believe that Europe has no software industry at all. And I said that I don't believe that regulations are the reason Europe's software industry stuggles against the US monopolies.
What the US monopolies have been doing to maintain their position is very well documented, and has nothing to do with regulations.
You claim that it's possible to compete with a monopoly if you are not regulated, and you call it a fact. I disagree. I believe that we as a society should prevent monopolies, and the way to do that is regulations. And of course, if you prevent monopolies in Europe but play against monopolies who are allowed in other countries, it's hard.
Not yet and I'm not sure I'm gonna receive it ever but one of the most important things I learned for myself is to understand my limits. I advise you to do the same.
Exactly. This sentence on its own is enough to explain your opinion, there is no need to explain anything else. Those who are against regulations are the ones who benefit from the lack of regulations.
And those who are in a dominant position generally think that they are there because they deserve it, because they are superior.
> And those who are in a dominant position generally think that they are there because they deserve it, because they are superior.
And this sentence explains your opinion. Business owner = superior = bad
You haven't responded to anything anyone has said despite the fact that a few of us seem to agree on these issues.
You basically brush aside anything that does not conform to your narrative and you also dismiss the entire report of Mario Draghi on European competitiveness: https://commission.europa.eu/topics/competitiveness/draghi-r... who agrees that there are many issues currently with the entire tech industry in Europe.
The facts are simple:
- the EU tech scene is minuscule compared to the US tech scene
- the average salary of a tech worker is higher in the US than in Europe
- there are many incredible challenges in Europe that stop Europe from competing properly with the US companies
- there is a clear lack of innovation and a very real risk aversion and less cash spent on R&D in Europe compared to the US
- European competitiveness is lacking and decreasing as time goes on compared to the US
> Those who are against regulations are the ones who benefit from the lack of regulations.
But you are right let's keep adding more rules and regulations, I am sure the next one will do the trick and help us expand our tech sector.
That would be a great narrative except that even the European commission disagrees with you since they are now exploring the option of scaling back some part of GDPR and even delaying the new AI laws because they suddenly realized that it was a major impediment to a healthy tech sector in Europe: https://iapp.org/news/a/european-commission-proposes-signifi...
In conclusion, you are arguing in bad faith, you believe what you believe because it's easier than to face the facts that Europe dropped the ball 20 years ago and missed the boat on many new technologies and that it is now trying to play catch up and unfortunately catching up will take many decades if that ever happens.
Nope. I said that business owners are generally in a dominant position, and those who are in a dominant position generally think that they deserve it because they are superior. I didn't say "bad", I explained the bias. Everyone is biased.
> you also dismiss the entire report of Mario Draghi on European competitiveness
Maybe we can do that: I will look into Mario Draghi's report, and on your end you could have a look at what Cory Doctorow has to say about the US monopolies and antitrust policies?
So that at least we would get some amount of open-mindness out of this not-so-constructive discussion :-).
> In conclusion, you are arguing in bad faith
Yeah, I too think that you can go ** yourself :-). But I will read about Mario Draghi's opinion, maybe you can get some perspective from Doctorow. Bye!
Did I use the word "all" anywhere in my comment? There are good developers in Europe. What I am saying is that there would have been even more of them had the incentives not been so lackluster. More talent seems to generally result in a greater competitive ability.
Fwiw, I have doubts that currently Europe can compete with the US at the startup level, let alone at the bigco one.
I am not trying to drag Europe down - it worries me that sophisticated complacency, overconfidence based on the achievements of previous generations, and addiction to comfort, will start eroding the very aspects that make it a great place to live at.
I have found that HN is, ironically, a horrible place to post experimental work on, with a few exceptions - e.g. things "written in Rust" etc. I think it's because the majority of the commentators here haven't really made anything from scratch.
"You should remain in charge, and best way to do that is to either not use agentic workflows at all (just talk to Gemini 2.5/3 Pro in AI Studio) or use OpenCode, which is like Claude Code, but it shows you all the code changes in git diff format, and I honestly can't understand how anyone would settle for anything else."
I 100% agree with the author here. Most of the "LLMs are slowing me down/are trash/etc" discussions I've had at work usually come from people who are not great developers to begin with - they end up tangled into a net of barely vetted code that was generated for them.
> Most of the "LLMs are slowing me down/are trash/etc" discussions I've had at work usually come from people who are not great developers to begin with
This seems to be something both sides of the debate agree on: Their opponents are wrong because they are subpar developers.
It seems uncharitable to me in both cases, and of course it is a textbook example of an ad hominem fallacy.
Well both sides could be right, no? I don't think it is necessarily uncharitable to note that lack of experience could cause developers to have strongly held, yet potentially invalid opinions about the practical applications of a technology.
I think it’s actually a combination of people who have seen bad results from ai code generation (and have not looked deeper or figured out how to wield it properly yet) and another segment of the developer population who are now feeling threatened because it’s doing stuff they can’t do. Different groups
> Most of the "LLMs are slowing me down/are trash/etc" discussions I've had at work usually come from people who are not great developers to begin with - they end up tangled into a net of barely vetted code that was generated for them.
This might be your anecdotal experience but in mine, reviewing large diffs of (unvetted agent-written) code is usually not much faster than writing it yourself (especially when you have some mileage in the codebase), nor does it offset the mental burden of thinking how things interconnect and what the side effects might be.
What IMO moves the needle towards slower is that you have to steer the robot (often back and forth to keep it from undoing its own previous changes). You can say it's bad prompting but there's no guarantee that a certain prompt will yield the desired results.
That's my feeling as well, I work on a fairly large and old code base I know pretty well, and generally Claude doesn't build things really faster than I would and then I spend more time reviewing. I end up using it for the most boring tasks (like dumb refactoring/code reorganization) where review is dumb as well and try to have it work when I'm not myself coding (like during meetings etc), this way I never lose time.
I think that is the skill that separates agentic power users from others.
You have to be really good at skimming code quickly and looking at abstractions/logic.
Through my career, I almost never ask another team a question about their services, etc. I always built the habit of just looking at their code first. 9 times out of 10, I could answer my question or find a workaround for the bug in their code.
I think this built the skill of holding a lot of structure in my head and quickly accumulating it.
This is the exact thing you need when letting an agent run wild and then making sure everything looks ok.
Absolutely - one of my favorite uses of Aider is telling it to edit config files/small utility scripts for me. It has prompted me to write more comments and more descriptive variable names to make the process smoother, which is a win by itself. I just wish it could analyze project structure as well as Claude Code... if I end up with less work at work I might try to poke at that part of the code.
I too have observed that aider seems to use significantly less context than claude code though I have found myself drifting from its use more and more in favor of claude code as skills and such have been added. I may have to revisit it soon. What are you using now instead (as you had said sometimes, depending)?
Note that Aider is not much maintained over last 3 months or so, there is a fork Aider CE, though I'm just watching their changes through rss and not used myself.
I'm more in Opencode world now and its in general more efficient for me (I'm sorta sysadmin by day, not a programmer, so agentic mode with Opencode saves a lot of time cuz you can just tell - write adhoc Python script and check which objects/methods present at that library- savings me from a boring part of you know programming/diving deep in unknown languages).
On Aider part, I especially liked ability to nitpick the function name, which is great for more focused changes/investigations.
why do i have to choose? in opencode, i can have both, let it run autonomously, but also look at the diff and whenever it goes off rails, i can stop it
reply