Just saw that a tiny AI model can run on a Raspberry Pi and beat GPT-3.5 in some tasks

I was reading through some research papers yesterday and stumbled on this stat that a model called Phi-2 from Microsoft is 2.7 billion parameters but scores higher than GPT-3.5 on certain coding benchmarks. Blew my mind that something that tiny can compete with a 175 billion parameter giant. Has anyone here actually run a local LLM on a Pi and seen decent results?

3 comments

3 Comments

troyreed1mo agoMost Upvoted

Benchmarks absolutely translate to real world, and a tiny model on a Pi can handle simple coding tasks shockingly well.

the_anthony1mo ago

Seems like clickbait honestly, benchmarks don't mean much in real-world use.

graygonzalez1mo ago

Yeah "benchmarks don't mean much in real-world use" is exactly what I found out messing with a Pi last year... the numbers looked great on paper but then it would choke on something simple like running a local web app with a database. What actually worked for me was just testing the specific task I needed day to day, that told me way more than any benchmark ever did.

-1