In the UK at A-level (age 16-18) you may still be taught linear and dynamic programming before ever touching a line of code! (Indeed, that was the same for me!)
Did your school used the specific terms "linear programming" and "dynamic programming" to refer to those topics? My original comment didn't phrase this clearly, but I was thinking less about techniques themselves, and more about encountering them under those specific labels.
As an analogy, it's not unusual to learn a good chunk of calculus before learning that "calculus" is a thing they're part of - for example, by having an over-eager physics teacher who teaches you some of it so we can understand physics material deeper, but without ever mentioning the branch of math we're now using.
I agree that the copycats are wearing thin, but the original paper's title seems fine to me. It's an accurate description of the breakthrough they made. The first few sentences of the Attention is All You Need abstract explain it pretty well:
> The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
The first sentence seems to imply that the main thing they want to do away with is the encoder–decoder arch, which they actually kept. (But BERT and GPT later did manage to simplify it.)
Given that redundancy is considered harmful, you probably want to create a ConsideredHarmful class and then a ConsideredHarmfulFactory to make for a more enterprise-ready structure.
A bit click baity at least. And without opening it you have no chance to understand what this is about. I know HN has a policy against editorializing but in this case, a brief summary would have been helpful.
The paper introduces a new method for text generation, named Copy-Over-Generate (COG), which differs from traditional approaches that generate words from a fixed vocabulary. Instead, COG progressively copies phrases from a massive text collection, aiming to generate coherent text continuations through multiple rounds of phrase retrieval.
COG stands on the line of retrieval-augmented text generation research but takes a radical step forward. Unlike previous work that combines retrieval and generation, in COG, retrieval is generation.
COG shares some ideas with previous work such as replacing the fixed vocabulary with a nonparametric phrase table.
The paper presents experimental results showing the advantages of COG over strong baselines in three experimental settings: standard language modeling (using the WikiText-103 dataset), domain adaptation (using the Law-MT dataset), and an enlarged phrase index (using the En-Wiki dataset).
Despite the promising results, the authors acknowledge that there are some flaws in the COG method. For example, COG may copy a phrase that is incoherent with the previously copied phrase, or it may only copy a part of a complete phrase, leading to inaccurate generation results.
Before long, if these NN refinements continue at their current pace, it's going to become impossible to tell synthetic HN posts from organic ones. Going to get weird.
Most of this seems sensible. Governments should be spending to prepare for a changed climate, in case global warming is irreversible (which I think is the likely trajectory). Investing totally in net zero by 2050 is not responsible governing.
In the U.K., that means building flood defences for the winter, water storage for the dry summers (avoiding the inevitable cries of NIMBYs like Layla Moran) and possibly investing in AC for hotter summers.
They were probably strong-armed into this. Perhaps the CCP and the FBI talked, and Apple was told they'd be cut off from their suppliers if they didn't introduce this.
It doesn't matter. This is the wrong choice, and everyone should rebuke and abandon Apple for this.
