I tried Mistral Nemo 12B instruct this morning. It’s actually quite good. I’d say it’s close to dolphin mistral 8x7B which is a monster in size and very smart, about 45 or 50GB. So I’d say Arli is a good deal Mistral Nemo 12B for 4 or $5 per month and privacy so they claim.

If you don’t mind logging for some questions, you can get access to very good or if not the best models at lmsys.org without monetary cost. Just go to the “Arena”. This is where you contribute with your blind evaluation by voting which of two is better. I often get models like 4o and sonnet 3.5 by Anthropic, google’s best, etc., and at other times many good 70B models. You see two answers at once and vote your favorite between the two. In return, you get “free” access.

Be careful with AMD GPUs as they are not as well supported for local AI. However, support is gaining ground. Some people are doing it but it takes effort and hassle, from what I’ve read.

I know that people are using P40 and P100 GPUs. These are outdated but still work with some software stacks / applications. The P40 GPU, once very cheap for the amount of VRAM, is no longer as cheap as it was probably because folks have been picking them up for inference.

I’m getting a lot done with an NVidia GTX 1080 which only has 8GB VRAM. I can run a quant of dolphin Mixtral 7x8B and it works well enough. It takes minutes to load, almost too long for me, but after that I get 3-5 TPS with some acceptable delay between questions.

I can even run Miqu quants at 2 or 3 bits. It’s super smart even at these low quant levels.

llama 3.1 8B runs great with this 1080 8BG GPU at 4_K_M and also 5 or 6_K_M. I believe I can run gemma 9B f16 at 8 bpw.

I installed it in Linux and it’s headed for a live environment.

Starling looks good so far.

One improvement I’d recommend is to make links visible. They are currently the same color as general text in the chat, black by default. I’d recommend blue.

The community can only read the source code, as of yet. All of the source code has been provided by a set of internal developers.

The fact that it is open source means that, if somehow two malware elements have made it into the source code, then someone will eventually report it. But this doesn’t mean that two malware elements cannot be there right now.

These two malware hits on total virus scan should be communicated to the developers.

Locally, an attacker still needs to know your password. A strong password can make it too expensive or impractical to brute force.

Filen.io

Works well so far, is end to end encrypted, open source, and the apps are nice and solid.

Works great for me. I’m running mx23 after running mx19 for a few years.

I hope mx23 is better with updates, or making easier to update, as updates broke in mx19 not long after I first installed it. My only complaint. Otherwise great.

Checking out [email protected] I saw very few posts by bots. Mainly saw posts by you. I saw one post coming from alien.top .

What’s interesting is that only posts by bots have any comments. So maybe this could be a good way to get communities started.

Therefore, if it’s okay with the admins at the following community, I’d nominate [email protected]

There’s almost nothing happening there.

Yes

Did you check with the admins on lemmy first or are you bot posting without permission?

Let’s say I have a favorite sport and there exists a sub_ named: r/.

Let’s also say there already exits a Lemmy community and that community is struggling to get off the ground: [email protected]

I can see a value add if your project directly helps [email protected] get started; but I don’t see how it does. If anything wouldn’t your project compete with [email protected] and therefore hinder it?

It might be different if your project directly tied r/ to [email protected] but it doesn’t.

If downvotes are the issue, beehaw.org doesn’t allow downvotes. Those folks are automatically eliminated from that. You can then just ignore the comments you don’t like and it’s all good. 👍

Another example

Trending Communities for Friday 22nd September 2023

Testing feedback:

The links in this post don’t load, https://feddit.nl/post/3654890

Testing feedback: There appears to be a missing feature. When a post includes a gallery of images, no thumbs or images are shown as a preview. Only an empty gray box is displayed.

Something has to be done to bring back the onside kick. It’s not dangerous from the standpoint of concussions.

4th and 25 or 15 is not as interesting and favors passing teams.

I went over to their Discord server and here’s what I was able to glean.

I gather they run a web-facing server which accepts text I/O from your Textual apps running on your personal machine or server, probably as a daemon. The connection between these two is via normal TCP/IP connections which your firewall already allows. Your Textual apps receive keyboard and mouse events and text.

They claim it should be “essentially free” for hobby use.

The text stream between your apps and their servers will eventually be (or are) encrypted.

N.B. This flying squid article is behind a pay wall.

“What hump?”

This piped link did not lead me to a video.