what cmd lets me hash the contents of a file? i wanna manually compare local file contents match the file on a remote server

sha256, sha512, and their friends.

ty!

Good lord.

?

Writing a raw disk image to a drive. Need to do it with a live CD. Was looking for a generic Linux or BSD based live CD that's modern which I can re-use for future purposes.

Found Ventoy, got my USB drive prepared with it (and supposedly "supports" FreeBSD, but the build instructions were way more time than I was willing to invest, so I rebooted into Windows and installed it from there).

Copied the image (Home Assistant, which comes as a raw disk image apparently) to the drive, along with mfsBSD, mfsLinux, UBCD, TuxPE, and Lubuntu for good measure just in case.

Tried to boot TuxPE first, but it failed. Then said screw it and booted mfsBSD, which worked...but then I remembered that the partition on the USB drive is exFAT, and I didn't want to spend the extra time getting that installed in mfsBSD after it was live booted.

So I rebooted again, this time into Lubuntu since I figured there was a greater chance of Linux-land having a bunch of extra tools coming with the distribution (so wouldn't have to install), but turns out...that's not the case. But installing it was pretty easy so I'm thankful for that. I could have probably installed it using mfsBSD, but a lot of

customized BSD-based systems I've used tend to mess with pkg repos and I was getting annoyed as to how long this is taking.

So I'm now at the point where I'm looking at a boot live system of Lubuntu with exFAT mounting available (I think). Now I need to remember / figure out how to list USB drives, mount the exFAT partition, and then dd (or whatever tool) it to the system's hard drive.

dd the raw disk image on the USB drive to the local system hard drive I mean.

...though I wonder how well just using curl + dd to stream the image and write it to disk would work out.

Thoughts?

worth a try?

I know this isn't the case, but some strange (and illogical) reason my mind feels like that's not a good idea because I'm reminded of back in the 90s when CD writers needed a continuous stream from the buffer due to the fact that the optical write heads couldn't really stop and resume (the first generation hardware anyway).

Don't know why I'm associating it with that.

Hmm, blocksize on dd only matters for the target, right? Is there any kind of input buffer setting for it or would it have to be paired with mbuffer for that?

dd has three blocksize parameters

if the input is slow, you do have to be a little careful

what you probably want is dd obs=128k or similar

Ah that's right, I forgot about the other two.

note that using bs= is usually wrong when the input reads might come up short, since it'll try and pass through each input block

Hmm, interesting.

I was just looking into optimal bs values a couple of days ago. Had to write a 6gb image to a USB flash drive and wasn't sure whether I should omit bs or set it to 4m or 16m or something.

But that was from a USB3 source, so I wasn't really thinking about input throughput. Was more wondering about writes that would be ok while also not taking forever.

Is fuse included in GENERIC?

Hey all, had a quick question about virtualizing FreeBSD on Hyper-V. The VM has two disks, one is the boot disk that is unencrypted and is where FreeBSD is installed and boots off of. This works great. However, the other disk is a 60GB disk that is mounted as a ZFS zpool within the OS and will contain email data since this is going to be a mail server. This disk is encrypted and any time I restart the server, and type my password to unencrypt

it, I notice that the VHDX file size increases in Windows. Additionally, I wgetted a 40GB DD file from the Internet, and deleted it upon download. However, in Windows the VHDX is still showing up as a 42 GB file on my hard drive. Keep in mind that the upper max limit for the VM is a 60GB disk and so while FreeBSD shows it as 0% used, the actual disk on file (on physical drive) is still that 42GB. What should I do?

sorry for the long wall of text :)

Check if the VHDX is sparse? (or whatever it's called in Hyper-V-land, can't remember anymore)

it's thin allocated

meaning not all 60GB is allocated all at once

so when it hits 60GB, it won't grow any more

I'd imagine that it would cause all kinds of issues when it happens

because then I'd have a ZFS drive that is 40% full with emails, and new mail will bounce because the underlying virtual disk can't take it

IDK if that makes sense?

hyper-v on what architecture?

Windows 11 Pro x64

you want the model of the CPU too?

no

yep didn't think so

so what you're seeing is that the backing file size reflects the amount of non-erased data on the volume

I don't see why you think there is any issue here

RhodiumToad: that's the amount of non-wiped data... ok that makes so much more sense

thank you for your help :)

so I'd be using the 'wipe' command to I guess write 0's over that data

but that's not necessary I don't think as even though the backing file size becomes 60GB it won't prevent new writes I wouldn't imagine

trying to use py39-ansible pkg. so i type ansible-galaxy collection install ommunity.general and it says error importlib_resources is not installed and is required. any clue how to fix that?

on 13.2

Hmm

I booted mfsBSD and it shows tmpfs is mounted at /rw, but /rw doesn't exist.

`df -h` does I mean.

I was going to create a mem disk, but saw tmpfs was already created and it looks like it's sized to use remaining ram (which is fine). Going to copy an image into the disk.

RhodiumToad: sorry to be annoying but if I download a 30GB file and then rm -rf it, it's normal for that gain in 30GB to not be reflected on the underlying virtual drive, yes?

If you're looking to shrink the VHDX but it's encrypted, I don't think writing zeros would allow the hypervisor to shrink it.

If it's encrypted, I don't believe the hypervisor would be able to identify any of the data is redundant, as it'll be using a block cipher.

I'm guess VHDX just looks at what blocks get touched. If it can dynamically shrink from the hypervisor, I assume it'd look for contiguous regions of zeroes, but anything touched/written from the guest will appear to be just random data, whether replaced with 0s from the guest or not.

There is likely no benefit to making a sparse file that backs an encrypted filesystem, since the hypervisor can't shrink it. I think it could only possibly worsen performance in such a case.

Any idea why the guest does not get net on vm-bhyve if vm switch created and activated in ifconfig? No pf of ipfw or ipfilter applied.

about the weechat version... the ports version was at 3.8 in january, but 13.2 release was made in april and the included ports is 3.7

is that just part of how ports is maintained?

ports has a main branch and every quarter a quarterly branch gets cut off; weechat 3.8 was committed to main in january, so q1 still had 3.7, no idea why q2 wasn't in there with freebsd 13.2 when it was in april (q2 should have weechat 3.8)

what jail helper framework do the cool kids use these days?

I play more and more with bastillebsd

ok, thanks

there are new alternatives too, github.com/illuria/jailer and github.com/DtxdF/AppJail

Title: GitHub - illuria/jailer: Minimal, flexible, and easy-to-expand FreeBSD jail manager.

but I didn't really tried them yet

*try

"Jailer is heavily under development and not yet ready for production use."

i think best approach is no jail manager at all. pure manual setup. numerous jailers is just a mess. hiding the very guts of what jailing is. well fast and handy, maybe automated but meh.

signalblue: deleting files does not, on most filesystems by default, do anything that would cause the underlying storage to be released (it'll just be reused as applicable).

signalblue: on backing stores that support TRIM, which may or may not be the case in your scenario, there's usually a way to cause freed space to be trimmed. for zfs this is via 'zpool trim' or the autotrim property.

signalblue: from what i've seen before, you would need to zero the deleted space then use a VM tool that trims the image

depends if the hypervisor is supporting TRIM

yeah, if it has the tool

no, I mean if it supports TRIM requests on the virtual block device

it's not a tool, it's a type of command to the disk

well, i'm not qualified to talk about it, but you need to zero the data because otherwise it doesn't know what's in use or not

no you don't, that's the point of TRIM

it's a request from the filesystem layer that effectively means "I don't need this data anymore"

the hypervisor would have to be specific to the file system

no

TRIM is a block device command

how would it know what it could trim?

changing the actual file size of the image

suppose you're using zfs with autotrim enabled,

then any time zfs knows that a given block is no longer needed, it issues a TRIM command to the underlying device

the hypervisor can, if it wants, detect that and do whatever it wants with it

it's through a VM image though, the VM doesn't know anything specific about zfs

so then it would have to know about the file system

it doesn't need to know about zfs or any file system, it only has to know about TRIM commands, which are filesystem-independent

well how about this then

if i have a .img file that's 16G, and i fill it full, then delete everything, and make a clone of it, the clone is 16G

what allows the TRIM command without knowing about the file system

it's the file system which is responsible for issuing the TRIM command (if configured to do so)

TRIM is just a request to the underlying block device, just like READ or WRITE

so either way then

enable TRIM, or manually zero data

and the hypervisor would have to support both or either

there's no reason why the hypervisor would check for zeroed data.

only to do a 'manual trim'

to resize the image

i don't know, but i don't think the TRIM command is issued whenever anything gets deleted, for example

that depends on the filesystem and its configuration.

zfs has an autotrim property for zpools, which causes TRIM to be issued for blocks no longer needed

zfs also has a zpool trim command, which issues TRIM for all blocks not currently in use

ufs has a flag that sets whether TRIM is issued when deleting files

all i'm saying is there is a reason the hypervisor would check for zerod data

no there isn't - it would be silly overhead given that TRIM exists

not all filesystems use TRIM

given the increased use of SSDs and assorted flash media, pretty much any filesystem that anyone cares about has the option to use it

well, hey, the ":" key centers the view

RhodiumToad: Thank you once again for that

basically I just didn't think this one through it looks like :)

signalblue: I also don't see any obvious indication whether the freebsd hyper-v disk driver supports trim.

There's no need for TRIM in my environment as everything is running off of spinning disks

diskinfo -v says Yes for TRIM/UNMAP

my ":" comment is in reference to a chat from yesterday in #cataclysmdda, so i'm leaving the discussion

greetings! I created a raidz2 pool with 8 disks displaying "512 bytes logical, 4096 bytes physical" sector size in smartctl, but I didn't set the ashift manually and it is now set to 0 for the pool. How can I make sure the 0 (autodetect) translates to 'ashift=12' in practice?

(I already have data on the pool so I would love the ability to not have to recreate it)

veg: Did you zpool get yet?

ted-ious: yes, I did, but it shows 0

I just figured out to get the result of the autodetect through zdb though!

and I'm relieved, for I see 'ashift=12' when doing `zdb -C`

That's a relief. :)

yesss

from the docs, I've got ipnat.rules with map dc0 192.168.1.0/24 -> 0/32 portmap tcp/udp auto

I can't do outbound ping from a windows machine on the LAN

I have no firewall ruls in place yet... but how would I enable outbound ping? is icmp a valid target in the map statement?

.vim

re jails: is unionfs for /usr and stuff still the way to go or each with their own filesystem

afaik unionfs is still tagged "beware of dog"

Is "Here be dragons" not in vogue no more?

bah, someone removed it from the text

it used to say this:

"THIS FILE SYSTEM TYPE IS NOT YET FULLY SUPPORTED (READ: IT DOESN'T WORK) AND USING IT MAY, IN FACT, DESTROY DATA ON YOUR SYSTEM. USE AT YOUR OWN RISK. BEWARE OF DOG. SLIPPERY WHEN WET. BATTERIES NOT INCLUDED."

now it just ends after "OWN RISK."

Ok, warning sign is still there

fbsd 13.2, latest pkgs, when trying to use py39-ansible pkg, i type `ansible-galaxy collection install community.general` and it says error importlib_resources is not installed and is required. there's a py39-importlib-resources pkg that i assume i need to install, but i never needed to in the past. is that expected?

I'm vaguely interested in where the issues are in unionfs, I could have a bash at fixing them

it's very unprofessional to be removing valuable contributions like that from the documentation :P

But then, everyone knows dogs aren't slippery when wet, they're smelly. :-)

i can't wait to get so high later

doing a half dozen reinstalls to 13.2 then celebration time my G's

k doing a fresh 13.2 install. trying to finish up but when i do pkg install pkg-provides it lists 2 pkgs, pcre and pkg-provides. i say `y` to continue and it fails. for both it says cached package missing or size mismatch

why it do that on fresh install?

does pkg update -f fix it?

ya it did

first rule of pkg seems to be "if anything weird happens, try pkg update -f"

:/

RhodiumToad: nullfs maybe?