You don’t need to run the LLM on the device. It supports 13 different protocols for both text completion and chat completion.

There’s also RisuAI which has some nicer features like better integration with function calling. Sillytavern is working on Function calling more but it’s not quite there yet.

The nice thing about Sillytavern is that it also has plugins for Alltalk for TTS and ComfyUi/A1111 for image generation directly from the roleplay interface.

It also has support for RAG through upload of documents and web scraping and a shitload of other features it would take awhile to list here.

What do you mean by personal package manager?

What do people use for command line utilities? The selection on flatpak is a bit sparse

It depends on whether those dependencies are shared with other programs.

Bluesky is not great, but it’s at least (for now) a better platform than X and the AT protocol is actually very well written. (For instance having a moderation service separated from the service that provides the posts I think is a hands-down better way to handle it than most ActivityPub servers having their admins handle all incoming and outgoing moderation)

Bluesky federation is just now getting started so it’ll be interesting to see if it goes anywhere/where it goes.

Pretty sure a significant portion of nazis are also moving to bluesky

The key is how healthy atproto is outside of bluesky by then

From one post-communist Mixed Socialist-Market economy to another.

I’ve heard this called “social chaff”

AIs generating false data for you to hide behind may be one of the few good things to come out of the LLM craze.

I would not say Johnny Harris is a reliable source

It’s your queries + your IP combined with the rest of the data the net collects from you that identifies you.

Anyone from the local llama communities experienced with Gemma models?

I’ve heard good things but I only use Mistral because it’s proved the most versatile.

I’ve been curious about google coral, but their memory is so tiny I’m not sure what kinds of models you can run on them

If nothing else the atproto is pretty great, we’re starting to see a proper federated net start opening up around it.

Waiting for a 8x1B MoE

Anyone else remember when people were making expert systems with scheme and saying that was the end of doctors etc?

Ironically thanks in no small part to Facebook releasing Llama and kind of salting the earth for similar companies trying to create proprietary equivalents.

Nowadays you either have gigantic LLMs with hundreds of billions of parameters like Claude and ChatGPT or you have open Models that are sub-200B.

I could be mistaken too, this has all only recently become interoperable so there’s some growing pains

Isn’t that what Whitewind is doing?

Yes! Actually.

The full atproto up and running with bluesky is only in the last month or so, so people are finally starting to trickle out and set up their own services and hosts.

It’s actually very promising and hopeful.