c/singularity · by InternetPirate@lemmy.fmhy.ml · 3y

Microsoft LongNet: One BILLION Tokens LLM — David Shapiro ~ AI (06.07.2023)

https://youtube.com/watch?v=R0wBMDoFkP0&t=11m30s

We could have AI models in a couple years that hold the entire internet in their context window.

6 points · 7 comments · view on lemmy.world

7 Comments

Martineski@lemmy.fmhy.ml · 5 pts · 3y (3 replies)

We could have AI models in a couple years that hold the entire internet in their context window.

That's a really bold claim.

behohippy@lemmy.world · 3 pts · 3y (2 replies)

Also not sure how that would be helpful. If every prompt needs to rip through those tokens first, before predicting a response, it'll be stupid slow. Even now with llama.cpp, it's annoying when it pauses to do the context window shuffle thing.

Martineski@lemmy.fmhy.ml · 3 pts · 3y (1 reply)

Yeah, long term memory where ai can access only what it needs/wants is the way.

Luovahulluus@lemmy.world · 2 pts · 3y

For now, I'd be happy with an AI that had access to and remembered the beginning of our conversation.

delicous_crow_hat@lemm.ee · 2 pts · 3y (1 reply)

Anyone know what progress has been made with hallucinations .

InternetPirate@lemmy.fmhy.ml · 1 pts · 3y

Perplexity has pretty much solved that since it searches the internet and uses the information it finds. But I don't know about any advances to solve it directly in LLMs.