Hello all - I am trying to make a Intel Arc B70 card to work in FreeBSD 15.1 - I want to use it with llama.cpp to serve llms --- I have a problem, I installed drm-kmod , gpu-firmware-intel-kmod-battlemage and enable in /etc/rc.conf kld_list="i915kms" ( though I understand that battle image is not 1915kms) ---- I wonder if anyone can hint what to do, how to test it - nvtop tells me no GPU to load...

it seem that it is an intel xe driver...

It seem that people are working on it since 2024 freebsd/drm-kmod #316

mary751: I think it's kind of impossible to use Nvidia's API (CUDA) on Intel's GPU cards.

checkpoint, obviously - maybe I misstype something - I wonder how should I configure Vulkan

mary751: have you tried installing latest drm-kmod driver (pkg install drm-kmod-20260508) ?

checkpoint, yes - I install it - i wonder if in /etc/rc.conf I need to add kld_list="i915kms" ? or xe --- or what ?

mary751: try "kldload i915kms" as root and see if that detects your GPU card.

mary751: frankly, I don't know much about Vulkan on FreeBSD, try readinig these forum discussions: forums.freebsd.org/tags/vulkan, seems some ppl were ably to run wisper models.

you can use vulkan with nvidia driver to run a local LLM on a nvidia GPU. But you're probably better offer buying a box dedicated to it like a DGX Spark.

anything worthwhile requires at least 200GB

bsdroberts: pls define "worthwhile" ? :)

if you want to be able to run a open weight flash model and essentially replace the role a cloud provider has.

Actually I am running FreeBSD 15.1 llama.cpp with qwen 3.6 and 3.7 very well or RTX NVIDIA cards --- Intel Arc B70 with 32G is only $1200 --- that is why I am trying to use it...

Qwen 3.6 and Qwen 3.7 are absolutely phenomenal

checkpoint, can you share the steps -- I was planing to do this

cmake -B build -DGGML_VULKAN=1

cmake --build build --config Release

that's all I did

OK, but about whisper, how do you install and configure it ?

I have two GPUs on my laptop: nvidia and amdgpu, both detected and used.

Can you use your microphone and get it to the text

mary751: yes works fine and very quickly

once build it complete, just run it: ./build/bin/whisper-stream -l ru -m models/ggml-small.bin

or -l <whatever_lang>

I think you should make a video - step by step --- would be of great use

stangely whisper.cpp builds flowlessly on my laptop (FreeBSD 13.5), but fails with error "Could NOT find Vulkan (missing: glslc)" on another machine with same AMDGPU but with FreeBSD 15.1.

looks like some package (library) is missing, but which one ?