so i got some vms on a 1U server with cool hot swap sas HDs and it's all good. but my prob is i have way more cpu and ram than i can use because 1U doesn't hold enough disk. so i'm thinking about moving the vm storage out to dedicated 2U storage servers. is that a bad idea to run vms off of disk space that's over the network on a separate server?

demido: If you have a 10Gb or faster network, you'll likely be fine.

no only 1gig ethernet to 1gig switch then to 1gig storage server

but most of what the vms transfer (in or out) is through the network and the network link is only 1gig

(fwiw)

Depending on the amount of data, it may be okay. But, I would guess it's likely not going to be a good time.

where do you see it breaking down?

I've tried it before myself, and even two VM's running over a 1Gb network got pretty sluggish.

if the lan only adds like 2ms, that really adds up that bad?

1-2ms

It breaks down because of constant send/recv for the OS.

You can always test it out.

so it just adds up to being enough to suck.. damn

Pretty much, yeah. I was kinda bummed.

Switched to iscsi over 10Gb and it's basically been flawless.

but isn't the latency still the same? or is latency not the actual problem, but the throughput

Latency didn't seem too bad, really. It was just a saturated line so everything was really laggy.

ya i can see that. OS+app is pretty chatty with disk. especially when you start getting swap right?

Sure seemed that way. I didn't test it for too long to really look into it. It was so sluggish, I just gave up immediately. Lol

hehe

so you have 10gbit all the way through? or 1gig from server into switch fabric and 10gbit between switches?

10Gb on everything. Looking to upgrade to 100Gb soon (if I can convince myself to spend that much money.) ISP's around here are offering ~10Gb a, so I figure might as well stay ahead.

s/a,/already,/

damn nice. what cat level ethernet cables you use?

6a?

I'm using fiber. SR. Or, when I can find/afford it, MPO.

damn that's nice

what brand switches you use for that?

Cisco. Was able to get a good deal when $JOB upgraded.

damn my man eating good over there

shoutout from the poor house

can your same fiber cables carry 100gig?

Haha. Naw. I just wait it out for good deals. And, sure. Just gotta upgrade the optics for it. Again, generally not cheap. :(

ek so i can't do iscsi/nfs until i get 10gbit. trying to think of a temporary bandaid, what do you think about plugging an external nvme into the usb port on the server and using that for bhyve vm storage?

demido: As long as the USB port and stick/adapter/whatever are 3.0/3.1, it'll be way snappier than NFS over a 1Gb ethernet.

ah dang it's only usb 2

wait there's 1 usb 3!

usb 3.0, so 450MB/s. is that fast enough?

demido: Yep. USB 3.0 or higher does 5Gbps. Much faster than 1Gbps. :)

the server is using ufs on raid 10. would you use ufs on the external nvme drive or zfs?

demido: Personally, I use ZFS anywhere I possibly can.

Hi

naoki: Hello

ek: same

rtprio: Yeppers. Why not, right?

on vm's less than 2gb i generally skip it

rtprio 2gb disk or ram?

.14

err

14 is feeling good, looking forward to 15 in a year or whenever 15.0 comes

really appreciate freebsd devs making a solid OS

have you seen OSDB (i think it's called) or kerneldb something like that. really sexy tech coming our way

demido: si. The folks in the video. Are you?

are i what?

luke_jobless_sb ?

it was a video i saw the tech in but i wasn't 1 of the ppl in it

but it was basically structured kernel state + sql query language. really powerful

I saw it in the video as well. it was interesting

ya

if there's a port that need libcap (the linux capabilities thingie), how do I compile it on FreeBSD?

privatebin.at/?50c7cd488cbc6afb#4Ud…LcRseTBFDC8kHJX44LTmA2Jw1QVcRsYnnuv , is there a reason freebsd doesn't have include directories /usr/local/include ?

quickrr: reviews.freebsd.org/V6

babz: ah, thanks

:S blame cmake :)

Hello friends. I am currently running 13.2-RELEASE (yes i just realized its EOL). Can I upgrade directly to 13.4, or do I need to go to 13.3 first?

RoyalYork: You can upgrade directly to 13.4.

Thank you.

hey - i have been building ports with poudriere and i really do not want to have to do it for EVERTYHING. At this point in time, i have made freebsd jail clients ONLY point to the poudriere repository.. is there a way to have it check this library first.. and if it can't find the package default to the "main" freebsd repo.. for example postgresql@17

voy4g3r2: yes, you can define your repository in /usr/local/etc/pkg/repos/mylocal.repo.conf and give it higher priority

with the contents like here pastebin.com/XFiZ4tNJ

but if the package was installed from one repo, the upgrade will usually stick to the same repo regardless of the priority

so the priority is for installing new packages

ohhh, that is what that does

the priority tag i did not have an djust disabled the other one

thank you!

sweet, it worked