any idea if FreeBSD support RDMA on the target-side of iSCSI ?

hello, we have to modernize and merge some of our storage infrastructure. One of the merge side ("my" side) uses FreeBSD and no SAN-like storage (only DAS) and the other an HPE MSA 2060, connected (with MPIO) to two windows server through SAS

we received an offer to replace the MSA side part (MSA 2070, VMWARE, 2 servers, setup, etc) which is ~90 000€

the load is heavy on this side (~500+ clients)

do you think it is feasible to replace all of this with two supermicro-like servers backed by SSD (or flash), connected to a 10GB switch (4x10GBITS with LACP), create ZVOLs on each server and export them through iSCSI to the Windows and FreeBSD hosts ?

I've already played with iSCSI and setuped something like this in my $homelab and it worked well (for the little tests I made). But I haven't any idea if it would handle the load like the MSA does ..?

mage, A while back someone had a problem with Samsung EVO QVO SSDs in a high rate storage array. Something you might want to be aware of. lists.freebsd.org/archives/freebsd-hackers/2022-April/000954.html and lists.freebsd.org/archives/freebsd-hackers/2022-April/000964.html

Hi

Guys remember me from yesterday.. I got access to an another pc

After I rebooted for freebsd install, the system is no more working; due to removed amdgpu pkgs

I forget to remove them from loader.conf and rc.conf

I tried to enter the loader promat "OK" and "unload" all modules, then boot with the kernel only in single user mode

I used "boot /boot/kernel/kernel -s"

-s for single as far as I know

But no luck still the same

with my access to an another pc, I will try internet..

*searching

I think -s is a linux thing. The kernel should pause at a menu, allowing you to chose single user and some other options.

As I know at that menu..

the kernel still not loaded

I used that command; to continuethe boot with the kernel module only..

CrtxReavr: I entered the loader promat

(number 3) from that menu

Retrofan: this on x86?

yeah

interrupt boot and set boot_single

then boot

see if that works

It'll mount / read only. You'll need to run 'mount -u -w /' (I think.) to write to it.

Tenkawa: My problem is stopping the system from loading gpu modules, not booting in single user..

then why were you adding -s?

thats single user at best

To prevent kernel from loading modules

the modules is loaded from

loader.con and rc.conf at kernel level

set module_blacklist='some previously present modules nvidia nvidia-modeset'

there's an example

already done... then the rc.conf load them

*will

loader.conf at the boot level

rc.conf is at kernel level as I know

are you using the "boot" command .. you can't yse the menu

er use

and the twice have amd modules added

boot? yeah I am at OK promat..

then I tried to load only the kernel  "boot /boot/kernel/kernel"

with -s for single mode...

But no luck :(

Unix learned me pain XD

Is the GPU locking up your display?

yeah.. its an amd readeon bug

ouch

I remember years ago first time installed freebsd, it took me 2weeks to make gpu working for x11

due to that bug

I think the problem is in rc.conf

wait..

Could I use rescue commands in OK promat?

like /rescue/vi ?

Retrofan: Might be fastest to just boot into mfsBSD, mount your boot and root and edit the .conf files.

on USB?

Retrofan: Sure. However you'd like to boot it.

I think I can't do that

No media right now :(

*I own

PXE?

No PXE or like that stuff

I am using freebsd as my Workstation desktop stuff

*for

Do you know the exact module names you're loading?

Does: set module_blacklist="amdgpu drm whatever"

ANd then: boot

... not work?

Yeah

Wait..

I blacklisted them before that

I readed now that makes loader pass to rc.conf

maybe if I set blacklist to nothing will work?

+ unload the modules like I did before

not working :(

What did you try?

set blacklist to nothing

I'd think blacklisting would keep both loader.conf and rc.conf from loading those modules.

+ unload the modules like I did before

So there is some module that I don't know is caseing that?

Just try an 'unload' and then 'set module_blacklist="vesa"' and then a 'boot' at the OK prompt.

Retrofan: Could be another module. Hard to say without seeing it.

yeah

no

what?

module_blacklist only works for loader

Err... 'set module_blacklist="amdgpu whatever else"'. Not vesa.

kevans: Ah. Bummer.

thre's a separate thing for devmatch, if you're actually getting it picked up there. haven't read scrollback

devmatch_blocklist in rc.conf

kevans: yeah for loader.conf

devmatch_blocklist for rc.conf?

boot -s is fine, but the -s needs to be before a filename

I was thinking if you could get past loader.conf, a 'boot -s' would get your through rc.conf.

yeah that what I am saying

I need to stop rc.conf

if you're loading the gpu module, you only need to worry about devmatch. we've long-since blocked them from being loaded accidentally in loader.conf

(boot_single is another way to spell `boot -s`)

if you boot -s then you're interrupting the boot sequence prior to where /etc/rc would've been run at all, yes

Ah, I remembered I done that long ago; to make modules only load from rc.conf

due to bug when been loaded from loader.conf

yeah, we stopped that common footgun around 12.0

12.0?

recently added nvidia modules to the list because they will similarly blow up if loaded that early

freebsd 12.0

I am 13.2

yes

and bug is still there

not the same bug, no

So now add them to the devmatch block list?

Retrofan: I believe kevans is strictly talking about module loading conditions. Not the module itself. Do what kevans said and use devmatch_blocklist.

yeah I understood

Retrofan: Are you saying this module is crashing the system or are you just not able to get to a login prompt after boot?

look like I can't add more than one module in devmatch_blocklist

Sorry. Kind of scrolling through the issue as mentioned above.

note that ... ok

i'd put $$ that they're just reassigning to module_blocklist multiple times rather than space-separated

sorry my internet

Retrofan: you're using space-separated value and not reassigned to the same variable multiple times, right?

it crashes my screen, can't seee anything

kevans: mm, yeah

Retrofan: Have you tried CTRL+F1 (or F2, or F3, etc...) to get to a new tty?

tried it before

it's an old bug

I know it

I can write.. but I don't know what I do..

*am doing

Retrofan: Is it ZFS?

no

If you can write blindly, have you tried su'ing to root and then "mv /etc/rc.conf /etc/rc.conf.orig" and rebooting?

I'm still surprised that a single-user mode boot won't allow you in.

ah

i don't think i've seen any evidence that they've actually tried it

look like that will fix it :)

Unix you need to think outter the box XD

kevans: There was mention earlier they tried single-user but it didn't work (I think?)

multi user work

I can write blindly

they tried `boot <kernel> -s`, but didn't seem to confirm that they'd tried boot_single, the menu option, or the more correct `boot -s <kernel>`

but never thinked of moving rc.conf file to .old

Retrofan: Did you not try the boot menu option for single-user mode?

I figured that was the very first thing tried.

tried

tried boot_single

no luck

can you define 'no luck'?

if you booted into single user you would not be having the same problem unless you've misdiagnosed it and presented an X-Y problem

we drop into single-user in init before it had a chance to invoke /etc/rc, devmatch isn't a thing until we've fired up devd

or maybe I misunderstood: do you just never have a console on this machine unless you have amdgpu loaded?

tried to rename the /etc/rc.conf no luck :(

No consle until amdgpu load

ok, so I misunderstood all the way back here: 15:33 < Retrofan> Tenkawa: My problem is stopping the system from loading gpu modules, not booting in single

we don't have a console driver for whatever reason (maybe it's just not associated) and you're trying to recover amdgpu, not prevent it from loading

wait

Yup. Now I'm confused.

kevans, the lack of luck.

I remember It was loading the console back in the day without any modules.. but resolution is low and no x11

I can't recover amdgpu, it deleted while my upgrade process!

are you setting kern.vty in loader.conf?

if you're relying on vesa, I don't remember if vesafb landed in time for 13.2. try dropping to the loader prompt and switching to syscons. set kern.vty=sc

the system is loading a non-exist module

ok I will try

(er, vbefb)

it seems to have landed in time for 13.0, but still worth trying

kevans: amm, it's just freeze

the system

even with unload and single user mode

define freeze. blank screen? tail end of loader output? other?

no blank, just tail end of loader output

then freeze

kevans: Need I to try vbefb?

haha

this the last thing I had done before I moved form WIN to BSD

To late

Going for sleep

see you tomorrow

what version of gnome is in the 14 repo?