i think i'm closer to my zfs issues

ketas: what issues?

others finally pointed out that maybe the write cache will fill up ram

because storage is slow

didn't you limit the ARC?

but arc is for reads

?

true

basically git pull on ports tree and poof

ok never had any issues on write

took few seconds until ram ran out

but must be specific to your zpool layout and/or devices

when it ran out, all userland was killed off

just 2 disk mirror

but should it happen?

spinning rust?

yes

nah I dont know why it would happen, IO will block if it cant write out to disk

1 could be failing now

i guess noone tested it well?

tested what?

apparently io will not block :/

ZFS is well proven and tested

on good large systems mainly?

powerful

what's this then?

a lot of people use it, but I wouldn't bother with it on certain memory restrained systems

i found it elsewhere too

zfs is well proven and tested, but also using native encryption will cause random in-memory snapshot corruption...

ketas: yeah 2GB RAM is stupid

what about io restrained

in my case it's 4

works except when it doesn't

ketas: ZfS was simply not designed for memory constraint devices

nor io, right?

so dont work against its design, use the tool which fits

fwiw, i use zfs on 2GB systems and it works fine, if you have a lot of memory pressure you may need to limit arc

i haven't tested performance but those systems are generally not performance-limited and the benefits are worth it

now it self tunes better?

previously zfs was much worse

unsure if it's now better or worse if it can handle low ends better

RAM is so cheap these days... no one bothers

any typical VPS has like 8GB nowadayx

-x+s

zfs actually has several good features for embedded systems too

nah

checksums, multiple copies, compresses

why would embedded systems want that?

that's not even remotely true. i have a bunch of VMs that act as routers (so they need to be separate VMs) and upgrading them all to 8GB would be a notable expense

but should zfs keep accepting writes nonblockingly until ram runs out?

which is what seems to happen

i bet you can run out of 2tb ram too

unsure if storage is that slow, but who knows

storage might also get slow due something bottlenecking

that would cause everything to crap out?

its not ZFS fault, nor FreeBSDs. If you are OOM you are OOM... nothing will help about that

its not only ZFS which wants RAM, git wants RAM too... the OS too... you are just hitting the limit and then you are out of luck

just give it 4GB more an you'll be fine, for sure

maybe

we have the same old story since ZFS became a thing on FreeBSD in the first place, people wanting to use it on 32bit and it failed.. then some optimizations came.. but its not really what its designed for

i bet packedGitWindowSize = 32m packedGitLimit = 128m limits the amount of writes?

no?

that eliminates the issue seeminly

maybe try to check with iostat then

without limits the machine goes down so fast you barely switch tmux window

fun

if it would just start swapping you have more time

but here it can't

well if its swapping you are already in trouble too

depends what swapped out

heh

it's just memory that is very slow :p

there's something around txg you can poke i see

and, why would embedded systems want checksums, multiple copies of data or compression?

because

what data?

indeed zfs is not for that

embedded systems are like industrial machines

microcontrollers etc.

raspi's are not embedded

like copies=3 :p that could give some redundancy on flash?

actual rpis are actually used in some places

ZFS is just a memory hog, nobody want that

where does it's memory go anyway?

nobody wants ZFS. nobody

not even the developers of ZFS want ZFS

hernan604: nobody working in industrial applications with embedded systems

its simply a completly different field

what do they want then

ZFS is about big storage

ketas: certified proprietary devices which have a lifespan like 20y probably :)

megaTherion: dont use ZFS

its definately not for you

use windows and whatever they have there

I use ZFS where it suits :)

ok

yes sure, windows is also a thing in many companies :)

meh my bbb is 10y old and has old fbsd in it

ufs tho

:p

what is bbb?

beaglebone black

ya but those are SBCs

big difference

it's soc is actually moderatively industrial

my 1gb vps runs with zfs without much issue even.

foxiepaws: so help ketas please :)

my 512m machine once did too

:p

what features are you using? deduplication doesn't work in that size obviously.

megaTherion: well sbc is embedded, no?

no, dedup is off

ketas: to me its more like dedicated devices, development boards, arduinos or smaller scale stuff like 16bit MCUs

unsure what dedup is for, 10k people sharing same cat photo around?

vpses with same os?

i mean, thats a good way to describe the usecase lol.

megaTherion: that term eroded i guess

"embedded"

just call them small computers now

but yeah, if its not working by default, and its x86 <1gb ram, i'm not exactly sure what the problem is. on one machine i did have to mess with tunables for arc and the like but thats not been the case for quite some time

sorry to say

yes that was fixed i think

from one machine i recalled 40mb/s write

was also wtf

it was not disk

this was ages ago too

yeah i had some issues with really bad ZFS performance for a while but it did eventually clear up

like 2013

i think it's large write burst in my case

i hate to be the "works for me" person,but when you don't have the issues or they just go away for you anyway without even looking... it's pretty impossible to figure out whats going on

that doesn't block

I'd still just get more RAM :)

i did eventually trace back issues for myself recently for that to the virtio driver for my host

didn't matter what OS was loaded, the hypervisor was getting slammed

i also had thing like buildworld failed on fs issues since i was just gitting around and / or cleaning obj

i was wtf, waited and ran it again, it was ok

so who knows

pretty sure it was because of other people on the same server that got banned off for abusing [lot of steal time and the like]

more RAM wouldn't have fixed that, and changing OSes or FS wouldn't have either. like to verify it i loaded a livecd of linux and it still was unusable

would you have preferred the writes to lock up or whole machine to go down?

virtual or real

uh, for me, i don't really think that it would have made a difference either way.

writes could open back up but if things get killed, data is gone

thats true, my workloads are primarily pretty chill and non-life-critical. a bit of dataloss sucks but isn't a big deal

for cases it does matter. i'd probably not be using a VPS with 1gb ram tbh

it would be quite annoying too

yep it is incredibly annoying

i saw bug of someone's zfs backed mysql server going down

was much bigger beast

ram, drives

still went down

idk and?

sucks, did it get reported? has anyone else duplicated?

shit breaks sometimes. just how it is.

seems similar

and bug is 2018 iirc

open

:)

it just needs specific conditions

has anyone else duplicated is the main point, just because a bug is open doesn't mean that its like, actually a bug in the software. I've had disk controllers take down my metal a few times and had to replace them, but it looked like a software bug

i think those are same issue still

so answer is yes?

but it's not very common issue?

because it's not very common cause?

gnop delays can maybe emulate this

i think when i asked people to test this, it didn't work because they used some super fast vm host

just limited the ram

no idea if found bug or limitation

i

reminds me the serial data corruption issue :p

got random input going to output mixup that i found no way to figure out

finally suspected a burst of data

tested it deliberately and it worked

unsure if it was usb uart hw, or driver or kernel

was even suggested to run memory checks, like overflows etc

so when i try to do `pkg install kde` i get all sorts of errors, "No SRV record found for the repo FreeBSD" and a bunch of others, whats that about?

lol

i think i figured it out

the ethernet cable wasn't pushed in all the way

lol in my defense, i think the cables little lock is worn smooth

sooo I installed KDE Plasma via the instructions on freebsd.org but... how do I start it up?

it feels like theres a step missing

GoSox: Do you want a gui login screen?

yeah might as well

Then you should install and enable sddm

i did

Did you start it too?

what do you mean, did i start it too?

service sddm start

Installed software doesn't just automatically start running

good morning!

"sysrc sddm_enable="YES"" that should start it at system boot, no?

Yes

then yes, i started it

But no login screen presented itself?

nope

just the normal text one

What happens when you do service sddm start?

standby, doing a second reboot just to mkae sure

That starts it at system boot. Did you re-boot after setting it?

Welp, guess that answers that.

"service sddm start" gives me sddm already running

and a pid

Press alt-f9 in the console

nothing happens

Anything in the system logs? Issue with graphics driver?

i don't know how to answer those questions. I can tell you this is a fresh install I just set up the other day and haven't done anything with except install sudo and nano

Take a look at /var/log/messages, there might be some error messages there

/var/log is where your log files usually go. Look for Xorg.0.log too.

You might need something from this section: docs.freebsd.org/en/books/handbook/x11/#x-graphic-card-drivers

the messages log has a lot in it but nothing that looks obvious as a problem to me

this machine has intel integrated graphics, so i need that driver package for kde?

there aren't "generic" drivers that get defaulted to?

Install the i915kms pkg

and if that doesn't work you can fall back to using the unaccelerated frame buffer scfb iirc

you should also be starting dbus

dbus start is in my startup file

still no GUI even with the intel drivers installed

You loaded the kernel module?

i dunno what that means?

ahh shit

ok i'm just now seeing on the 'install a desktop' page that it says i should already have the x window system installed. that doesn't install as a dependency when you install your UI?

Correct, the kde metaport does not install xorg

well, probably mystery solved then

It doesn't depends on x because there are other ways of using it

does i have to install x window first, or does it not matter?

like can i just install it now and will it all work

Order doesn't matter, you can do it now

ok

i'm actually not going to :P

this will be my project for next time

the good news is that i am keeping very detailed notes so i should be able to blast through this setup next time

can I in someway passthrough this pastebin.com/z9sfkJbd to a virtual machine in bhyve so I can acess my usb-dongle for zigbee ?

what do rackmount servers typically have for GPUs on built in video ports? just basic intel integrated?

depends on vendor agreements;)

well then i guess i'm in for a surprise when my server arrives ... after i order it ... after i pick it ou

t

i want to put one or two low end GPUs in there but just for BOINC crunching, not for any video use

nwe: you can only pass through an entire PCI device, so if this hub is an integrated one, probably not directly. It would need its own PCI card, these are not expensive and almost any one would do.

does anyone know of a pam_time module for openPAM? Can't seem to find one in the ports tree or via google

dch: maybe I can use socat for it instead..

oh I see, login.conf has all the pam_time functionality in FreeBSD

how can I see which usb-device is connected to cuaU0 ?

I trying to get SONOFF dongle to work over USB->TCP via socat.. main host running freebsd, and all the guests too.. pastebin.com/pmk1ffK0 has someone get it to work?

does anybody recall how to get tcpdump & pflog interface to list what pf rule blocked traffic?

or am I mis-remembering things again

all my block rules have log on them

`tcpdump -venttti igc0 port 2200` seems sensible enough

dch: do you need to request link level headers?

I added a `-l` already and don't see anything different

don't you need to capture on the pflog interface?

I've not done this for a very long time

#facepalm

write a blog post :D

Oh yes I will, because #FutureMe will forget this again

the irony is I just checked, and I have the precise command in my diy-ials talk

so double fail for me

s/ials/jails/

how can I see which port ugen0.2 will have in /dev/cuaUXX ? or will it be ttyU0 ?

nwe: depends, you want cua here

as for which device made it, it's in sysctl somewhere

and logs if recent plugin

ketas: demsg says umodem0: <ITEAD SONOFF Zigbee 3.0 USB Dongle Plus V2, class 2/0, rev 1.10/4.42, addr 3> on usbus0

so I guessing /dev/cuaU0

sysctl dev | less -p ' U0'

there it is

ketas: thanks :)

earlier log

i don't know

i never tried to redirect usb

wait

or i mean serial

I using in my guest socat -d -d -d pty,raw,echo=0 tcp:10.50.20.2:8888 and creating a pty device

should it even work like this?

and using that pty device in zigbee2mqtt configuration file but get stuck on this pastebin.com/hfKi9BCd

there's serial over ip redirector as well

tcpdump network maybe?

i'm unsure how those things work

do they do ascii?

that would be helpful

one solution is that I running this zigbee sonoff-usbd-dongel from rpi

wait it also runs on fbsd

there's port too

yes everything is running freebsd

so what's the issue?

the issue is that the dongle is on the doomU and I cant usb-passthrougth so I can use the usb-dongle inside my virtual machine

well, something runs, it's not same i think

but you wanted it to be mqtt :p

ehm what

what's the vm host and guest again

[usb-dongle]->[server] -->[virtual-firewall(freebsd) bhyve with dhcpd ] --> [zigbee (freebsd) bhyve] [mosquitto (freebsd bhyve)]

the problem I have is that I dont have access to the usb-dongle inside my virtual machine named zigbee

I then I think maybeI can use socat to share my USB(dongle) over TCPÅ to my virtual machine

oh complex eh

setup

when I suing scrren /dev/pts/2 Im connected and can see traffic with tcpdump

using*

so why not putting zigbee2mqtt to host?

easier?

it's just ip after that

yeah maybe thats better idea :D

no need to proxy usb or serial

both could still work too

but it looks like it´s working in some way ^^ atleast I can connect too it with screen :P

dedicated serial over ip emulators exist too

but maybe as you said it´s easoer

and are in ports as well

but if simpler solution works

I haven't try that

maybe even with socat

never tried any of that

ketas: for ports are you using portmaster or?

still portupgrade :p

or maybe just pkg install it :)

i could do poudriere eventually

ketas: how's ZFS?

yean, you can do that

megaTherion: same

:(

i still wonder if it's zfs

or mmap

now mmap is at fault

how come?

I went to bed yesterday, didnt track what happened - was really late here

i'm not really getting what mmap actually does but i have vague idea

I bet its mapping memory

there are issues around that as well i see

well no shit but?

nah, complex stuff

rescue here: reddit.com/r/LocalLLaMA/comments/1gghsy0/comment/luzjsuz

:p

well i knew most of it

anyway, whatever it is

ram is nomnommed off

ketas: there is also a manpage

but mmap is most likely not the cause of your problem

b1

ketas stop teasing the patrons lol

..is there any serious freebsd irc channel on this network?

like #bsdcode used to be on efnet

this one seems to be #freebsd-social when it's not supposed to be XD

have all the kernel devs (and people who understand it) fled irc?

Koston, there's #freebsd-dev

CrtxReavr, appreciate it

I think a lot of them are still on EFnet, honestly.

There's also #freebsd-ports

Koston: efnet, yeah

EFnet is a mere shadow of its former self though.

Sadly.

that's useful, I do have a few ports to submit

well, efnet was always just a fork of ircnet

like linux pretending to be a unix

efnet still exists?

to be serious though, an unfortunate split - people demand drama, but users just wanted function

i think i was more of an undernet fan myself

we had fistfights at our primary school over irc wars in mid 90s

EFnet prior to channel timestamping was probably the most fun I've ever had on the internet.

i'm imaginingn efnet like this old cave and if you spelunk in far enough you find a wizened freebsd dev floating over a rock.

timestamping is an irc protocol feature?

I spent a lot of time over at efnet, CrtxReavr was one of the old faces there

12:48 -!- Channel #freebsd created Wed May 19 09:50:15 2021

There was a time when that wasn't tracked. . . which allowed people to ride a split in and take ops.

And oh, what fun that was!

IRCNet #freebsd and especially #unix killed themselves over irc wars..

CrtxReavr, that's how most of the irc wars worked yeah

when having a bot written in C mattered, because it was just faster

I had a front-row seat to some crazy shit in #unix on EFnet.

Like when irc.eskimo.com got de-linked.

I was just a patron of #unix on IRCNet.. not that it was any less crazy I assume, but it sure killed itself

poor eskimo lol

"Nanook" bragging about how secure his shit was. . . when he was already owned.

LOL

I guess it was ultimately for the common good that US and EU irc networks split back in the early days

why did they do that

I won't reveal who, but someone already owned him, then after he bragged, started encrypting his backups. .

but IRCNet #unix is truly a miserable place nowadays, they still have a +k and there's like ten people left

johnjaye, well it was US vs EU split, do the math

Then had to fight to old on to it for weeks while the relevant backups got older and older. . .

Then nuked everything.

EFnet had a rule you got delinked if you were exploited. . . and irc.esmkimo.com was no more.

it's so silly too, it wasn't more than a few years ago I last checked efnet website and it still somehow claimed to be The Original IRC Network. for one it doesn't matter one bit, but it's also untrue and everyone knows it

let old people have their fights I guess

As far as anything that still remains, it's probably as close to true as anyone can claim.

for all intents and purposes I guess that's true

When I dipped my toe into IRC in college, (in '91) it was EFnet.

I believe the protocol was written and first implemented in '89.

I was just sad it had already split by the time I started gaining clue (around '96/97 I guess?)

If you want to be an IRCop and no one will make you one, then you start your own network.

but IRCNet for any *nix has been a pathetic excuse for idiots say "hello" for a few decades now

#unix in particular performed a magnificent suicide by setting +k permanently and having an intense civil war over it

let us older people be an example for the younger generations. or... not do that?

embarrassing

I have UNIX tattooed on my fingers, which is unlikely to come off next time I shower. it was supposed to stand for live free or die (fighting), not live to have moronic irc wars

man, it's a shame bash.org shut down

I had it done over two decades ago now and I still love UNIX, what it stands for in my mind which is POSIX mainly, and the idea of working together to create a common standard

but now everyone wants to have their own idea of doing the same thing, over and over again. I couldn't believe it, there are actual $PS1 _managers_ entire software suites designed to replace your shell prompt and much of your shell functionality. what the hell are you doing

I want to move to the countryside and grow pumpkins and use copious amounts of LSD pretending none of this ever happened

Koston: Tatoo? Are you from the Yakusa?

I'm just an old punk

covered in tattoos, can't really visit japan for that reason sadly

Koston: You might like this comedy scatch: youtu.be/DsdSTY6Y-rs

one of the happiest moments of my life tbh was kirk mckusick and george neville-neil joining me at bsdcon lunch table, latter excitedly showing off his train tattoos

lol

remiliascarlet: another classic, youtu.be/1oZQCqPRa8E?si=_M0ck_OBHTluOcDN

but seriously, I was told long ago that I should go to japan office to fix up their network but they can't send me because half of the office staff would resign

to this day I have never had a japanese co-worker who agreed to shake my hand

but at least one agreed to speak with me. I was happy :)

Koston: hashnix.club/~kona/cybersecurity.jpg

It's that time of the year surveymonkey.com/r/freebsdsurvey25

i used ohmyzsh for a while but then i encountered some git repos that were Too Big To Fail and my shell session lagged all the way to alpha centauri and back so now i just use fish.

Koston: Too bad it's the *network* that's the problem. Remote admin was invented to handle tattooed individuals working on things :)

hmm doesnt pkg install haproxy create user for haproxy and example configuration?

nwe: A quick look at the port's Makefile and pkg-plist suggests it does not

Dummy question: assuming I have 4 identical disk drives. ZFS writing to a single VDEV (raidz1 x4disks) versus ZFS writing to single VDEV (raidz1-3xdisks). Should I expect better stats? io_time, iops, latency? Or ZFS can only gain benefits when writing to more VDEVs? (totally agnostic about nunber of disks per vdev?)

Not a dummy question at all — it actually touches on some of the core ZFS architecture principles that people often miss.

thanks, I know that converting these 4xdisks into a single vdev mirror would give me 4 times performance (at least in iops I guess)... at the cost of risking full data loss when any drive fails. :D

striping into several vdevs (each vdev 3xdisks mirror) would be my utopic dream (not enougth budget!) but certainly that would be great

vkarlsen: okey then I understand why I couldn't find the configuration file, also any specific reason why not shipping haproxy.conf.sample and creating user/group for haproxy?

nwe: I have no idea. I've never used it. I just looked at the port.

as a minimum, for operations approaching normal, which directories under / need to be read-write? etc, home (or equiv), usr/local, usr/ports, var, root, tmp, run (if your local usage uses this; mine does, most's doesn't)?

could be none

what's normal?

a root user would not notice that those directories are read only if she did not check mount or whatever

as in, the directories not listed

e.g. /usr/lib, /lib, /bin, /sbin, /usr/lib32