hi, I'm repeatedly having trouble compiling mysql80-server, it's getting stuck on storage/innobase/pars/pars0pars.cc for 3+ hours

here's the info I have so far - bsd.to/s8uE . it used to build on 13.0, unforunately I had an issue updating the ports tree for a long time, so there's a 2 year gap

Title: dpaste/s8uE (Plain Text)

ultramage: all the cool kids use mariadb now

are you saying you're trying to build a port that's two years old?

rtprio: no I mean my last successful build was 2 years ago, because after that I was stuck not being to update for a long time. And when I finally did, I found out that the thing wouldn't build

I was suggested to attach a sampling profiler to the stuck process to see what it's doing, but I don't know how (and not sure if buildworld saves debug symbols)

going to check if it really hangs

one thing I could try is building the older mysql to see if it might be related to a src change on their end. Another would be to install latest llvm from ports, since freebsd's is 2.5 years out of date.

I took the full commandline and tried building just that one file with -O0 into /tmp, but it got stuck the same way.

finally, this is a ~2010 intel atom cpu running a single-core process. It's been running for 4 hours now whereas all other files compile within 30 seconds. I could try letting it run for 24 hours to see if maybe it'll eventually grind through whatever loop it's stuck on. Although maybe it'll never finish

I just had a deja-vu moment. I know I investigated it 3 months ago when it first started happening. I believe I did thoroughly debug it with gdb and found that it's stuck in a register allocation loop, flipflopping between two states

ultramage: so far it is building fine for me but

yea I assume it's not a general issue, otehrwise a lot of people would complain

[ 62%] Building CXX object storage/innobase/CMakeFiles/innobase.dir/pars/pars0pars.cc.o / [ 62%] Building CXX object storage/innobase/CMakeFiles/innobase.dir/pars/pars0sym.cc.o was a long while ago

it is at 94% now

workshop.palace.tearmoon.com/data/1…m36s/logs/mysql80-server-8.0.32.log build progress log

if it processed that file and produced its .o then it's fine

do note I have innobase linked statically

I have taken that stuck job's commandline, added -emit-llvm, and then tried a simple clang -c -O0 file.bc and it worked. However when I use -O1 it gets stuck.

maybe it is something about the instruction set

2 months ago when I first investigated with gdb, I found that it's looping in runOnMachineFunction -> allocatePhysRegs -> selectOrSplit -> selectOrSplitImpl -> tryLastChanceRecoloring -> ..., there is an enqueue() - dequeue() loop and in my case it's just repeating the same two steps

why do you use mysql by the way? proprietary software that demands mysql/percona specifically?

so the way their code is constructed there, there is an input on which this loop never terminates

it's just the only thing I'm familiar with, and this php thing is built with mysql in mind (it does use pdo, but idk how independent it is)

[01:22:34] [01] [01:10:36] Finished databases/mysql80-server | mysql80-server-8.0.32: Success

thx

this was a single threaded run inside Hyper-V on an AMD 3700X

do you happen to know how to force cc into targeting a specific architecture?

I would like to try a i386 build

cross-compiling ports is possible but I really suggest you do not use the ports tree itself for that

you are using ports, right?

not anything more complex

oh right the thing in base is only built for my arch

use poudriere

it might even be that it will build fine in poudriere for your arch

-march gives me a wide selection of 64bit cpus. I tried 'sapphirerapids' which is the latest intel cpu, but it got stuck there too

well freebsd 13.1's base llvm is 13.0, from 2021 ... it's 2 major versions behind. I imagine that if I installed devel/llvm latest then it would build. that's my next go-to

my alternative is to build the sql parser with -O0 lmao

poudriere uses base llvm

my build above is 13.1, with llvm 13

> if I installed devel/llvm latest

I do not think ports will use that

you would have to override

in any case, consider poudriere, it is extremely convenient and gives you a binary package repo

well yea

I build from source because I have stupid obsessive requirements, like minimizing dependencies

poudriere builds from source

ultramage: workshop.palace.tearmoon.com/build.…-default&build=2023-02-25_08h52m36s

Title: Poudriere bulk results

hmhm but then what would be the point? you mean I should set up a standalone build server, and get packages from there?

you can set it up on the same system

it runs build jobs in clean jails

and you get .pkg files you can instantly upgrade to

I don't think there would be any benefit over just portmaster local src build

there is also synth as an alternative

it's not a build framework issue, it's an internal defect in llvm

that apparently only manifests on my potato atom router box

there is a huge benefit: the jails are *clean* images of a given FreeBSD version with no extra libs etc.

I'm pretty sure it has something to do with the architecture, since the problematic place is physical register allocation using a graph coloring heuristic

my router box is a Jaguar and I would never build ports there :D

which is why I have poudriere on my desktop PC building packages for the router and the server

ah yea but I don't want clean default freebsd-pkg builds. I flip all sorts of toggles in the ports tree

anyway

uhh

who prevents you from doing that?

isn't there `poudriere options`_

?

ah okay so I would customize the package repo to my needs. but then I ask again, what is the point, over just building them locally in /

I use poudriere precisely because it lets me build multiple package repos with different options etc.

the point is pkg upgrade is near instantaneous

the host still has to spend 8 hours building the packages, that's just an extra step. unless I set up a separate build host that is faster.

imagine running a portmaster-based upgrade for something that depends on mysql

that something might be a daemon

that forks at times, like php-fpm

during the local build you can happen upon long periods of missing libraries

it is why I switched to poudriere in 2009

it gets worse if the build fails

ah I see. well, in my case that's not a requirement, temporary malfunction is fine. That was a good example though.

plus you can build on a fast system?

in my case it's mt worker mode so nothing ever forks, and I just restart the whole thing after the build is done

my router is "CPU: AMD GX-412TC SOC (998.16-MHz K8-class CPU)"

with 4 GB RAM and a 32 GB mSATA SSD

can you imagine building llvm for postgres on that

or Rust, for py-cryptography

building on a separate host would be nice, but I don't have any, plus the extra complexity of a second freebsd instance would not be worth it for just one router box

i have a vm for that

^

my desktop PC is Windows and poudriere is a Hyper-V VM

this thing has 2 atom HT cores at 1.8ghz max I think, with 4 gb ram. it isn't that bad. buildworld takes around 16 hours I think.

it gets 8 virtual cores and 8-16 GB RAM on this 3700X

I buildworld for the router in the poudriere VM too haha

well, I no longer do

but back when I still did buildworld and not just buildkernel, it took like 30-40 minutes

a dedicated build system would defo make sense if you're dealing with more than 1 freebsd host

it makes sense when your router is slow or when your server is a production system where extra build load would affect users

if I tried building ports on this router I think I would be unable to use youtube

ah that is interesting. never ran into issues with this thing, even when all ht cores are busy building something, it still moves packets around just as fine

anyway, one of the biggest advantages of poudriere is that you build in a clean system without any noise

(kernel networking proably runs at a higher priority than userland processes)

which you cannot guarantee in your built-world case

I would be worried if the VM got the target architecture right. even in this OS the compiler sometimes has trouble deciding. So I leave it at 'native', and let it figure it out.

huh.

that is really weird

I never ever had problems like that

the only case where I had issues was when I was over-eager with port options and built something like ffmpeg? I think? with 'optimise for this platform'

and AMD Ryzen is very different from AMD Jaguar

if your Atom system is x86 rather than x86_64, you just make an x86 poudriere jail

hello guys, can I remove /usr/src ? I don't use ports, only packages and I need of space..

Question. When a port's configure script seffaults and coredumps, it is logged into /var/log/messages, creating log noise.

mystic: yes, you can remove /usr/src and /usr/ports

otis: thanks :)

I have found libtool, gdb, screen and bind generating these events. I have debugged libtool's configure script, it happens when testing "checking whether a statically linked program can dlopen itself". It fails during puts() inside jemalloc's tsd.h tsd_state_get(), with "Address not mapped to object.". Is this an okay thing to be happening? If configure scripts are expected to coredump, shouldn't the

messages be silenced?

huh, bifrost27:/# dd if=img of=/dev/nvme0n1 conv=sync bs=1M

I just dd'd FreeBSD over an alpine / boot device and rebooted ... into alpine again? wtf

Easy questions first: 1- did you write to the right device/partition? 2- are you sure you're booting that same partition? 3- is that partition still recognized as bootable after you wrote into it? (Thinking of secure booting or similar, which may require you to jump through additional hoops.)

you did not specify count. iirc in the absence of count, count = 1

(I don't remember if it's possible to say 'until `if` is consumed')

actually I might be wrong, since dd manual says nothing about count. I might be mixing it up with some third party dd tool

debdrup, makes sense (newsyslog.conf(5)). For some reasons I was thinking that particular bit of info was stored elsewhere, given its very short retention.

In absense of a count, count should be all the counts.

dch, I've nto worked with nvme drives on FreeBSD. . . what's the n1 portion of the device name?

Also: file -s /dev/nvme0

And: file -s /dev/nvme0n1

fixed, I just need to dd *all* the drives, turns out they install a non-mirrored alpine to both drives

"a bigger hammer required"

Ball-peen?

nvme has namespaces, allowing subdivision of devices, so nvme0n1 is the first "namespace" and nvme0n1p1 a partition within that

o_O

Hi you all.. having some problem starting xorg. everything worked just fine before I replaced my motherboard and cpu... I guess the problem might be caused by the built-in graphics card some interfering with my nvidia card.

pciconf -lv

but dont know where to start on how to resolve this.

(pastebin it)

You're on the console? Not ssh'd into it?

hold on, need to ssh to my machine then

No - do this:

alias tb='ncat termbin.com 9999'

Then: pciconf -lv | tb

Then share the URL it prints.

neat trick

Also. . . ncat is in ports, so you may not have it, you can substitute nc from base.

termbin.com/dzorj

went with netcat, seemed to work

So you have onboard ATI. . . and your'e trying to use the nVidia 1080 card?

Using the binary blob driver?

yea, thats right

IT's loaded?

yupp

kldstat | tb

all I have done since it worked, is swapping out the motherboard and cpu

Well, those are not small things to change.

termbin.com/sq59v

And I assume you're looking at this console through a monitor connected to the nVidia?

ultramage: you cannot really silence a segfault message

CrtxReavr: yea.. Almost got a disaster on my hands.... computer didnt boot, and I started making things worse and worse last night.

involving thermal paste and bent pins on the motherboard socket

CrtxReavr: you assume correctly

also not remembering in what order my zfs-pool was connect is clearly a thing as well =D

brb

startx 2>&1 | tb

Remilia: I'm just wondering why messy configure-releated noise is showing up in the main log like that

is there a guideline that configure scripts must not segfault / use some sort of trap to silence them?

I build everything from source and this is the first time I saw stuff like that (that is a lie, logs say that it first appeared in my libtool-2.4.6_1 build on 2021-02-27)

how can i specify primary device in xorg?

aaah, =) now it works.

CrtxReavr: thank you for the help

ultramage: no, there are no such guidelines and you cannot 'silence' kernel messages arbitrarily

segfaults during configure stage are normal and expected

Remilia: I know, it's just that nobody except libtool is having them. so either they were smart enough not to do that, or port maintainers patched around them, or there's a way to write those test cases in a silent way

on windows I can install an exception handler that just returns, and all segfaults will go off the record

... setjmp is the equivalent, iirc?

'nobody' is quite a strong statement

a segmentation fault is not an exception

anyways, libtool is testing if it can compile a -static -fPIC shared library as an executable, and then dlopen() on itself. it fails, but on top of that, puts() causes segfault, either due to the process information block getting destabilized, or because the so executable was never ready to invoke libc anyway

I can say that I get numerous segfaults in /var/log/messages when I build packages in poudriere

it never really bothered me because it is expecteds

-s

oh. in my case, it's just libtool, gdb (probs uses libtool), screen (same?) and bind (same?)

you could patch the kernel to not emit that message?

locally

since you are building world anyway

no that was just portmaster updating some ports

I dunno how to explain that what you see in the logs/console is the standard kernel message emitted when a process is killed

Feb 21 15:06:16 poudriere kernel: pid 25379 (conftest), jid 137, uid 0: exited on signal 11 (core dumped)

I know, I'm just saying that nutty test cases during compilation of userland software should shut up about these

userland software is not kernel

it cannot block kernel messages

you may want to switch to Windows Server

I mean, should not be causing those. If apache and php and sshd other good stuff were casually producing 10 of these during build, then I guess I would consider it a common thing

the fact that nobody but libtool produces untrapped segfaults during every build is really suspicious, and makes me assume that that is not the norm, or, is even actively discouraged

you are probably the only person bothered by segfaults in dmesg so I doubt this will be 'solved'

I'm not a port dev, but my observation from the current state is that having your test case segfault (instead of failing to compile, or returning a negative test result) is an anomaly

2022-03-21_10h18m50s/logs/libmemcached-1.0.18_7.log:checking whether the compiler supports GCC C++ ABI name demangling... Segmentation fault (core dumped)

=D

I guess this is libtool?

we are talking about software written in C

and autoconf

that seems like their own configure script log message. was that segfault also logged in /var/log/messages?

of course.

hmhm

do you know what C/C++ have?

UB. like, loads of UB.

do you know what autotools are for?

they let you find out what kinds of UB and other stuff you can expect on the platform you are building for.

certain kinds of UB and certain approaches result in hard errors like segmentation faults

that means the test binary crashes with a non-zero return value and the script knows it did not work

I wonder why perl doesn't generate any. I know its configure script is long as hell

because it is an interpreter

it does not need to do black magic

well, you say that, but php is casually engaging in UB by building all of its dynamic modules with static thread local storage mode (meant for .a files)

it does not crash so it must be doing it in an acceptable way

you could likely open an issue with the upstream libtool developers/maintainers?

'hey guys I do not like that you test something via inducing a segfault, can you stop'

the freebsd dynamic linker is providing it with a kludge 128-byte buffer to help it run (except 128 is not much room for global variables). linux gives around 2kb.

ultramage: php is a waver thin layer on top of C

no, php is a disaster

on top of C

(which is a disaster)

that, too

every year or so, i reread eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design and it's fascinating how little has gotten fixed each time

Title: PHP: a fractal of bad design / fuzzy notepad

Perl is at least an exponential function of bad design

found logs from 2019 where libtool, libtdl, libmcrypt were doing it. They all have 'conftest', so I assume it's the same piece of code

ultramage: is this a production system where you're trying to make sense of logs, and this is generating noise you don't know how to filter, or what is the issue here?

kinda. I would prefer if these messages were limited to services that are failing at runtime. though this noise is just that, noise - it's easy to spot and skip over, since it only appears in the middle of package operations

Title: Runtime - Autoconf

re: conftest

yuripv: pr 666 on github is going to be a duplicate of openzfs/zfs #14527 when that gets merged

666 – The ldconfig program in SNAP wasn't good bugs.freebsd.org/bugzilla/show_bug.cgi?id=666

Title: Add vdevprops.7 to the Makefile by debdrup · Pull Request #14527 · openzfs/zfs · GitHub

ultramage: if you do not want port build-related messages on your production system, build your ports elsewhere and use built packages

sort of like microsoft's dcom permission warnings that litter the system log, where ms says they're expected (and they can't be arsed to silence them within the offending software even though iirc the flag is there)

it is weird to expect the kernel to filter out messages based on arbitrary rules

=)

nah I wanted the configure script itself to contain the segfaults. It is possible to do by adding a bunch of sigaction code, but that goes against the idea of minimalistic configure test cases. I can't think of anything good. Maybe if `sh` had an operator that trapped all signals coming from child processes?

that said, in the case of libtool, puts(dlerror()) is completely unnecessary, since the test case just wants the return value from main, and it's the puts() that's causing the fault. So if they ditched those text outputs that nobody reads anyway, that'd do the job

not sure if it's libtool writing the test case, or if it's some predefined autoconf macro

right now, configure's function is built on segfaults of code that doesn't work

do what Remilia said and build ports elsewhere

I'm sure I could obsessively file upstream issue reports, but I'm too burnt out from doing nothing but freebsd maintenance activities for the last 10 hours, in total silence. I'll let this one go for now.

what upstream issues?

autotools just works that way

there's a reason people think it's a blight

well, there's actually upwards of 36 reasons

as I said, I could ask libtool to make their dlopen testing macro not invoke libc string functions and try to print the reason why dlopen failed, instead just return 0 or 1 from main. I assume that would clean up the segfault without any loss in functionality.

I could ask freebsd kernel devs about why that PoS UB test executable is crashing inside jemalloc, and what it would take (different compiler flags?) to make it execute safely

I could ask libtool devs why the f they're testing if you can compile a small program as a static library with an elf header, then try to execute it, and then try to dynamic load a copy of itself into the same process space.

IIRC, this isn't the first time I see this config segfault question. I wonder if it should be a FAQ.

so, if i have a vm-bhyve vm and it is using a flat file for its disk image, can i move it to a zfs volume by dding it to the volume?

if I pt 3 usb 3 external drives into zfs 1 pool will the speed be combined speed of the disks?

Combined lack of speed would be my guess :D

concrete_houses: raid 1 will improve read speeds in theory, as long as you don't max out the usb bus

with mirrors, it uses round robin to read from the disks in the mirror

what about write?

as slow as normal, as it has to write the data to all disks in the mirror

say there are 3 disks in the zfs raid 1, will it break the file into small parts put put 1 on 1 2 on 2 3 on 33 and repreat so all sped is used?

no mirror just stripe

when you said zfs 1 i thought you meant raid 1, sorry

0

i've never used stripes

i'd reccommend reading michael lukas' book

whats that

a book on zfs on freebsd

Title: Distributed Control Systems: Their Evaluation and Design (Mechanical Engineering (Marcel Dekker Hardcover)): Lukas, Michael P.: 9780442260200: Amazon.com: Books

not that one

type "zfs" into amazon

in the past I had some problem with disks rearraging on reboot

does taht happen in zfs

as far as i know, not

zfs uses a duid to identify the disks

if unxz is not a POSIX utility and my goal is to only use POSIX utilities, what POSIX command can I use to uncompress a file with the .xz extension?

if I were you I'd worry about compatibility with real systems and not with standards. I'm sure someone has ported xz to whatever system you want your program to run on

POSIX has a "compress" command

it doesn't have anything to uncompress xz

hi all quick q... the iwm(4) driver.. does it support 5ghz in FreeBSD 13? And if so, how do I tell it to prefer 5 Ghz over 2.4 GHz?

or should I be usin iwlwifi?

iwm0: <Intel(R) Dual Band Wireless AC 8265>

im learning about jails and it says there's 1 main program the jail runs. but what if i want a jail to have a few daemons not just 1 running, like postgresql nginx and my web app binary?

polyex: first of, you get just start rc

but, secondly, and more importantly, what's the point of putting them into one jail if you're not separating them?

so i can deploy a self-contained web app that's made of 3 pieces of tech

if i break it into 3 jails then it's just OS level virtualization but not a nicer deployment vehicle like first class containers

am i wrong?

right now i just run these things on a vps, no jails, but i wanna try container style deployment with jails

having three different things with vastly different resource demands in one single container is nice for easy deployment, yes, but, uh

The trade off is in ease of sysadmining (1 jail or 3) versus security (less unwanted interactions between them).

the tradeoff is: easier deployment vs everything else

oh, and migration

but, upgrades, resource management, security issues, etc

means you have three components down instead of one

i can maybe see wisdom in putting db server in its own jail then web server and app in its own jail. then does web app talk to db server over network like db.mysite.com? (even tho it's on the same host)

yes, or you do crimes against security and mount the socket into the other jail

ya i dont wanna do that i think i rather have jails be ignorant of eachother and not assume they're local to eachother

so let's say the web app takes file uploads. i guess i wouldn't store them in the web app jail because then when i deploy a new version it would get overwritten. so how are stateful jails done best?

polyex: by giving it storage for state

same with DB

that's the path the jail is given in the host FS to live in right?

meena: why is mounting a socket in another jail crimes against security?

satanist: reviews.freebsd.org/D27411#882100

Title: ⚙ D27411 add altlog_jaillist to syslogd's rc script

yes there are problems, but is the general idee to let two jails talk about a socket in a shared mountpoint a crime against security?

satanist: maybe? i don't know… yet

satanist, it may allow colluding jails to escape partially by passing directory file descriptors over the socket

ya i don't want to open any leaks from jail out unless it's through the network or something. so a db jail is given some state to save the db files in, and i guess a web app jail that takes file uploads but should be stateless needs to have another file upload service jail that the web app middles to?

jilles: but to achieve that one jail need a directory fd, if this fd is from outside the jail the problem is the access to the dirfd outside the jail not pasing it over a socket

Anyone run into issues with FreeBSD (this is 12.4) install media and older Tyan IP-KVM "virtual media" acting like this?

If I saw that I would be thinking the disk drive is failing.

I've verified checksums, reset the BIOS to "optimal defaults", tried the mini ISO (which gets closer to working).

Yeah, the puzzler is there's no actual disk involved. :)

It's a VM image? Oh.

It's an ISO.

The IPMI/BMC has full remote access, including the ability to "mount" an ISO as a drive. It appears as a USB CDROM drive (also an option to use a floppy).

I'm guessing this Tyan board is so old (2011 or so?), and so uncommon that there's probably some USB quirks FreeBSD needs that nobody ever ran into.

Hmm... I am running VMs in a Linux hosted KVM system, running a 12.4-RELEASE system, and I haven't hit this myself. That's all I know.

it might need some quirks added to function properly

I'm going to try some flavor of linux in a bit and see if the problem persists.

It's not a VM, it's real hardware. The only "virtual" thing is the CDROM.

cd1 is presumably a virtual CD-ROM drive supplied by the OOB BMC? Having it return read errors would, to me, indicate there's something wrong with the OOB BMC chip (which, being one of many completely separate computers in a modern system, is completely non-debuggable).

I doubt that anything that implements a block device is going to be able to tell you errors like CAM is doing there, because since it's a block device it can't per definition correlate errors with their operations.

That's... pretty much the biggest reason block devices were removed from FreeBSD. ;)

Considering that the snapshot also contains a vm_fault, I suspect there are bigger problems afoot.

snapshot/screenshot

I don't find it outside the realm of the possibility that the a +10 year old 400MHz ARM9 (or possibly the even older 200MHz FreeScale, which is based on a M68000), which forms the basis of that era of OOB BMCs, would start experiencing severe issues.

Even if Tyan splurged on Aspeed chip with ECC (which is an option according to the specs), that's not really a guarentee.

satanist, the issue I'm hinting at occurs when nullfs mounting some directory into another jail

jilles: can you be a bit more specific about the issue?

satanist, given jail A and jail B, if one is not an (indirect) child of the other and sends a directory fd to the other, the normal jail restriction doesn't work properly

because it doesn't apply the sender's root to the receiver's use of the fd

ok yes thats clear, but I would say this maybe (depending on the setup) acceptable

making a little progress with the Tyan BMC... Wouldn't boot linux either, then the virtual CD just stopped appearing.

just rebooted the BMC itself and it boots the same linux image that failed.

I wonder if my first attempt of using the normal install ISO was just too much for it - not clear if the image resides in RAM or what.

Or a memory leak, or bad RAM in the BMC that only gets tickled after it's been up for awhile...

spork_css_: generally the virtual CD-ROM drive does a streaming read of a file from the filesystem that it's hosted on

The OOB BMC is a completely separate system.

spork_css_: i suspect it's not the motherboard that's the problem

rtprio: it's not exactly simple to desolder the OOB BMC chip :3

out of band bmc.. board management controller?

Baseboard Management Controller(?)

Back to "linux boots, freebsd doesn't" - but the "linux boots" only works until you try the freebsd ISO, then the "linux boots" stops working as well.

Of note, the FreeBSD bootonly ISO is around 375MB, the Linux ISO is 170MB. I kind of feel like this is all a little half-assed.

This sucker's going into quasi-production, so I guess I'll just keep a little USB stick in it for any future reinstalls - the rest of the BMC features work fine.

All academic at this point to some extent, the colo is only about 15 minutes away and I have errands to run anyhow.

reflashing the BMC for giggles