also for the environment, I would think. It saves a ton of useless traffic
GPT is worse and it’s not even close.
My PC can serve up a hundred requests per second running an HTTP server with a connected database with 200W power usage
It takes that same computer 30-60s to return a response from a 13B parameter model (WAY less power usage than GPT), while using 400W of power thanks to the GPU
Napkin math, the AI response uses about 10,000x more electricity
GPT is worse and it’s not even close.
My PC can serve up a hundred requests per second running an HTTP server with a connected database with 200W power usage
It takes that same computer 30-60s to return a response from a 13B parameter model (WAY less power usage than GPT), while using 400W of power thanks to the GPU
Napkin math, the AI response uses about 10,000x more electricity