[Azath Vitr (D'ivers]

the broken, on 08 March 2025 - 07:31 PM, said:

QuickTidal, on 03 March 2025 - 01:27 PM, said:

the broken, on 02 March 2025 - 11:56 PM, said:

Unsurprisingly, Deepseek turned out to be mcuh less impressive than first thought.

Presented without information about how that's true. Cool. Cool.

DeepSeek Debates: Chinese Leadership On Cost, True Training Cost, Closed Model Margin Impacts – SemiAnalysis

Quote

DeepSeek's Cost and Performance
DeepSeek's price and efficiencies caused the frenzy this week, with the main headline being the "$6M" dollar figure training cost of DeepSeek V3. This is wrong. This akin to pointing to a specific part of a bill of materials for a product and attributing it as the entire cost. The pre-training cost is a very narrow portion of the total cost.


Training Cost
We believe the pre-training number is nowhere the actual amount spent on the model. We are confident their hardware spend is well higher than $500M over the company history. To develop new architecture innovations, during the model development, there is a considerable spend on testing new ideas, new architecture ideas, and ablations. Multi-Head Latent Attention, a key innovation of DeepSeek, took several months to develop and cost a whole team of manhours and GPU hours.


The $6M cost in the paper is attributed to just the GPU cost of the pre-training run, which is only a portion of the total cost of the model. Excluded are important pieces of the puzzle like R&D and TCO of the hardware itself. For reference, Claude 3.5 Sonnet cost $10s of millions to train, and if that was the total cost Anthropic needed, then they would not raise billions from Google and tens of billions from Amazon. It's because they have to experiment, come up with new architectures, gather and clean data, pay employees, and much more.

People ran with the 6m headline, even though it was just the final training run cost. Without that price tag, it's just another AI like the others.

First of all, you claimed they lied, but they did in fact specify from the beginning that they were referring to training cost.

Second, the cost of training the model actually was shockingly low, and a major breakthrough. (And it's not called a "training run" in English---it's training the model, the process by which the artificial neural network "learns" from the training data.)

On that note, in case anyone missed it:

Quote

Chinese tech giant Alibaba Thursday released a new model, called Qwen QWQ-32B, that matches the performance of DeepSeek's R1 but requires a fraction of the computing power to run.

Like DeepSeek, the new Qwen model is open source, and both Chinese projects are reasoning models that excel at technical work.

The Qwen release is one more sign that plenty of further progress is possible in making this kind of AI more efficient.

Reasoning models are making the transition from "amazing new breakthrough" to "cheap and widely available commodity" in record time.

[... Meanwhile, in the West:]

[...] After a week of hands-on experience with OpenAI's latest and biggest model, GPT-4.5, AI experts remain a little puzzled by it, given that it costs a fortune to use yet doesn't break benchmark records.

But one consensus has emerged among fans of GPT-4.5: The new model has "taste."

https://www.axios.co...-45-qwen-sesame

lol, oh yes... a taste of poppycockshitstuffedgarbageinferno (that's a German word, isn't it?...).

And Trump is doing a lot to sabotage the US advantage in AI hardware---and not just by saying he wants to revoke the CHIPS act (which would be difficult, though I'd suppose he could withhold further funds or come up with some other totalitarian bullshit to sabotage it):

Quote

The US is burning bridges as China builds relationships

[...] rebuilding the industrial ecosystem after decades of offshoring is a time-consuming and complex process. There is a shortage of skilled labour in the US[...] In discouraging imports, the US would not be able to produce many products due to a lack of capability to manufacture all necessary components and parts.

[...] Making 2-nanometre semiconductors needs Zeiss lenses from Germany, extreme ultraviolet light from ASML in the Netherlands and speciality gases from Japan, as well as materials from Applied Materials and testing equipment from KLA in the US. Tariffs [...] would drive up costs and slow technological progress.

While the US is burning bridges with its protectionist trade policies, China is building bridges

Spoiler

with other nations. China has been enhancing its agricultural productivity, advancing AI and technology, strengthening domestic innovation, expanding global trade and investing in Latin America. [...] Despite the US and EU imposing tariffs on Chinese goods, China is leveraging its port investments around the world and its trade relationships with other Brics countries to expand its trade with Southeast Asia, the Middle East, Africa and Latin America. [...]

China is focusing on agricultural productivity to achieve self-sufficiency in feeding its population. With declining arable land and water resources, China is developing new agricultural plans that emphasise AI, biotechnology and advanced seeds to boost productivity.

[...] Facing a declining population, China is leveraging advancements in AI, robotics and autonomous driving to boost productivity in various sectors, including education, healthcare, culture and transport.

https://www.scmp.com...s-relationships

This post has been edited by Azath Vitr (D'ivers: 08 March 2025 - 09:05 PM

[Azath Vitr (D'ivers]

Bloomberg's newsletter this morning:

Quote

The Forecast: Robot waiters plug labor gaps

A cat-themed robot with big blue eyes glances from side to side as it purrs across a Tokyo restaurant [...]

"Your order's here," [...] "Meow!"

[...] it's an increasingly common sight at more than 2,000 restaurants operated across the country by Skylark Holdings Co., the nation's largest table service restaurant chain.

Faced with a severe labor shortage in one of the world's most rapidly aging populations, service-sector businesses in Japan are increasingly investing in robots that don't need expert supervision and can work alongside people, instead of simply replacing them. So-called service robots are also making it easier for firms to employ older or foreign workers — who are crucial to plugging the shortfall — by helping them overcome language barriers or cope with the physical demands of a role.

Japan's service robot market is expected to grow exponentially

Service robots are also increasingly being used in Japan's aged care sector

https://www.bloomber...common-in-japan

I'd guess that foreign workers should also be able to control the robots in close enough to real-time over the internet, so they don't even have to live in Japan---though there might be legal reasons to prefer that they do (IDK if the robots, if mispiloted, could go on destructive rampages, ram into people or tables at dangerous momentums, run people over etc.), unless the Japanese government has an agreement with the government of the other countries that is actually enforced.

I read about a restaurant like that in Philadelphia (cat robot servers! though I'd prefer ones that look like actual gigantic cats who can be petted, that's probably impractical---though I'd imagine a robo-tiger or robo-lion could have tentacles or retractable bars on their back holding food trays, and cats do have excellent balance, so perhaps robo-cats would as well... still, probably not the most efficient design, at least for the short-term).

As I predicted years ago, Japan's aging population combined with their cultural fondness for robots is proving to be an advantage. It's a little like how the Black Plague contributed to innovations in productivity. OTOH in other ways modern Japanese firms have had a history of clinging to inefficient tradition and outdated technologies like fax machines... from 2024:

Quote

In Japan, the fax machine isn't just surviving; it's thriving. The country's commitment to fax technology can be perplexing to outsiders, especially given its reputation for high-tech innovation.

https://faxination.c...h%20innovation.

This post has been edited by Azath Vitr (D'ivers: 09 March 2025 - 09:47 AM

[Lady Bliss]

Blade Runner is becoming more like our future. The replicants will be built to harvest the food that our ejected migrants were doing first. I’m betting Tesla is building them right now.

This post has been edited by Lady Bliss: 09 March 2025 - 11:09 AM

[Macros]

at least they'll explode on the launch pads

[Azath Vitr (D'ivers]

Macros, on 09 March 2025 - 05:09 PM, said:

at least they'll explode on the launch pads

It's more efficient when they explode far above the Earth and the "falling space debris" causes US flights to be grounded (as happened in Florida after his last launch)... because that way they don't need as many air traffic controllers.

The most replicant-like humanoids (with realistic looking faces and skin and so on) will be in social roles---Immortal Robo-Putin (or army of Robo-Putins), etc. Though IDK whether elderly people or young children would strongly prefer robots who look like real humans (or close enough for people with severely impaired eyesight). For those with dementia they could even have replicas of their favorite relatives (or celebrities who agree to the use of their likenesses---probably including Robo-Trump and Robo-Putin) taking care of them.

Wonder if the restrictions the Chinese government has placed on verbal content (for political reasons) have prompted Chinese AI development to focus on "technical" aspects and functional abilities like programming, mathematics, etc. So the language restrictions which were predicted to handicap Chinese commercial AI may have given it an advantage in certain important fields. Of course it plays into the modern American stereotype of Asians as being "good at math" and science at the expense of verbal fluency (incidentally I doubt that stereotype is accurate, since at least anecdotally Asian Americans have a high participation in the literary and cultural arts, but more like at the expense of sports, which for most Americans is much of "culture"), though that's also a bit ironic, since historically Asia fell behind the West in technological development because of a focus on culture rather than technological (and associated scientific and mathematical) development.

Meanwhile the Trump administration has apparently still not complied with the court orders to unfreeze NIH funding, as part of his disastrous plan to defund science in the United States:

Quote

Trump reportedly aims to slash the budget of the National Science Foundation by up to two-thirds. [...] In defiance of court orders, Trump has largely maintained a freeze on NIH funding.

As a result, many of America's great research universities have stopped hiring and are cutting Ph.D. programs — in some cases rescinding offers to accepted students.

[...] Trump speaks of putting America First, but his attack on the nation's great research universities is ensuring that the U.S. comes in second — to China.

Although America has long been the global leader in scientific output, China is now surging ahead. Even before Trump's cuts in research funding, China was projected to match U.S. research spending within five years.

China has already surpassed the U.S. as the top producer of highly cited papers and international patent applications. It now awards more science and engineering Ph.D.s than the U.S.

Trump is trying to destroy our nation's most vital asset—our innovative minds and ability to think for ourselves - Robert Reich