Can anyone tell me what the recommended method is to protect the GZ network from other zones?

I am guessing per zone firewall_enabled=true + /zones/<uuid>/config/ipf.conf I could use an example though.

Eric_F: I'd say it depends on what is your network design looks like (do you use admin and non-admin tag NICs, is it a single server, do you have router zone, etc)

We were trying to decide if we needed a router zone (we being codex and I). I have had basic smartos running since maybe 2014. It is on a household lan (192) with internal routing on 10.x. I've built a router/firewall zone before and codex has one now but it seems to be in the wrong place. The goal is just to install codex in an LX zone where it can be root and spin up any zones it wants (using wrappers and json

filters) but cannot get to the GZ

So, it and any zone it spins up needs to be blocked from gz on both interfaces, I assume

The machine it is in has at least 3 and maybe 5 (not sure if the motherboard NIC's work in smartos) NICs

if the "agent" could create a zone, it's in control of what nic configuration it would have, isn't it?

vmadm is done through wrappers. Those wrappers scrape the json and reject various parameters. NIC network is one

well, ok. then a router zone and an internal network should be sufficient to contain this stuff (if any additional zone even needs an access to gz or internet)

zones include web-server, jupyter server, backup. So definite internet access. Is the router a better solution? It looked like triton used firewall_enabled=true + /zones/<uuid>/config/ipf.conf

I guess, of those, only backup needs internet. The rest are houshold LAN

keeping things simple and having ipfilter configured in router where it is out of control of the entity that has root access in a zone seems sufficient

and if you may, separate those zones that don't need internet access into own network

There is no distinction between LAN access and internet access at the smartos boundary. Internet access is NAT'ed

you may control that in the router zone via its ipfilter (ie which client is allowed what)

Ah, true...interesting possibility. Thanks

jperkin: percona-toolkit has mariadb-client as a dependency, tried to use it with percona-server fails cause mariadb-client and percona-client cannot be installed both. Is there a way to fix it?

yeh I need to fix a few things around that, the problem is that p5-DBD-mysql stopped supporting mariadb so we now have multiple splits

yes saw that too :-) will try now to use mariadb cause ways are moving in this direction.

Hi, I have RTL8168H (pciex10ec,8161)not working with the driver on illumos/SmartOS and RTL8111C (pciex10ec,8168) network card working. Are they really that different and 8161/RTL8168H have no driver under illumos, or is it just an PCI-e ID to be mapped to driver.

I used to add PCI id to some file , back on OI, for a wireless card to start working. Maybe i could do the same on SmartOS , with global zone service I have.

RTL8168B/RTL8168E/RTL8168G/RTL8168H are in the same group. realtek.com/Download/List?cate_id=584&menu_id=297

Re-asking: Is there anywhere that documents the firewall topology and rules used to separate customer zones from GZ in triton datacenter code? And is that the recommended methodology (firewall_enabled=true + /zones/<uuid>/config/ipf.conf) ?

Eric_F, I am on SmartOS single machine. Isn't global zone separated on the hardware level, with 'admin' interface is totally not accessible from 'external' interface that is used for internet-faced VMs and zones?

And there are etherstubs through which are connected vnic interfaces of zones and VMs, that is isolated, unless connected to outside world through gateway zone. Etherstub creation is also defined in /usbkey/config file .

nikolam, The threat model is a compromised/malicious non-global zone trying to reach GZ management services. I know same-host isolation is not absolute, but I want the supported SmartOS defense-in-depth pattern: physical/admin separation if possible, plus whether fwadm with firewall_enabled=true is the right per-zone enforcement layer

Yes, that matches my understanding. I created an etherstub-backed NIC tag for an internal service network, with a gateway/firewall zone connecting it to the LAN. My open question is about enforcement when a zone has a route through that gateway zone, or when a management-capable zone has an external NIC. I want to prevent non-management zones from reaching GZ management addresses/ports.

nikolam, so the model I’m considering is: use etherstubs/router zone for topology isolation, and additionally enable firewall_enabled=true/fwadm on zones for per-zone GZ-controlled policy, especially outbound blocks to GZ/control-plane IPs. Does that sound like the intended SmartOS standalone pattern

If they are on separate IP range on external and can access global internet and can pass firewall rules in the gateway zone (firewall rules can additionally forbid all sorts of things ), I usually let them access global internet through gateway zone if they initiate connections , communications is allowed, where only exact ports are available par zone for their service(s) they perform on internet. But once outside on the external, packets behaves

accordingly to network rules. The fact that admin interface(s) are not connected to internet at all plus all admin network physically separated also helps :)

when you are on etherstubs , they act like switches and you can create whatever topology you deem fit. Every zone can have firewall in it and do routing as you please.

If I could choose and have hardware, power and networking, I would always go Triton way of private cloud, plus MNX/Edgecast support is really close to the metal support with actually ability to make changes in both kernel , services and operations as customer needs

nicolam: yes that sounds like Triton topology. For my home setup, there is only Lan and admin. The question is at least partially whether to create another routing configuration (zone or external ) to only allow specific IPs to access GZ, or is the firewall_enabled=true/fwadm approopriate for blocking zone egress to GZ/control-plane IPs?

my approach, with single NIC server, is to put all NGZs on VLANs and then firewall as appropriate on external physical router

dbrooke, That makes sense. So with one NIC, use VLANs on the trunk, put GZ/admin and NGZ networks on separate VLANs, and let the external router/firewall enforce NGZ-to-admin restrictions. That avoids making GZ recovery depend on a SmartOS gateway zone. I may still use fwadm as a per-zone defense-in-depth layer, but VLAN + external router seems like the cleaner topology boundary.

It is supported configuration to have a separate PCI-E NIC for admin (usually one that also have KVM over IP on it for branded machines) and separate NIC for external. Even for small single-machine setup, I won't go lower then 2 nics, USB flash drive boot (without booting from HDDs) .I actuallly last yera had a physicall break-in and they couldn't read anything form ZFS encrypted drives, couldn't connect to admin network and couldn't put rootkit on HDDs

for it boots from USB. (and of course admin not accessible form outside) Few simple things made it as much bulletproof as possible.

Also don't use USB NIC, USB does not have error correction and it is yet to see what USB NIC works on illumos. I have seen enough even PCI-E NICs in small deployment that create errors , errors are catched by illumos FMA (Fault management) and replaced.

ZFS in illumos also uses FMA , unlike Linux and FreeBSD implementations.

Eric_F: I actually leave the GZ/admin access untagged but otherwise pretty much that

nikolam, That makes sense. The principle seems clear: keep the admin plane physically or at least VLAN-separated from tenant/service networks. My current home topology is weaker because GZ has a normal LAN IP, so I’m trying to decide how far to improve it: second NIC or VLAN for GZ/admin, external router/firewall rules, and then fwadm as defense-in-depth for zones

dbrooke, That sounds close to what I may want: leave GZ/admin untagged for simple recovery, put NGZ/service networks on tagged VLANs, and enforce NGZ-to-admin restrictions on the external router/firewall. Then use fwadm inside SmartOS as defense-in-depth rather than as the only boundary

Just make sure, that , beside encryption, you also use 2FA for login into your workstation, where you actually keep private keys for SSH authentication , for those bastards also used _hardware_ keylogger mounted on the keyboard and came back to grab passwords from the keyboard directly. So you need 2FA for both SSH and local login on your desktop you use, for protections on your cloud or VMs have effect.

This really is in my house. The only thing leaving is backup and curl requests/pkgin. The exposure is that codex/chatGPT will run root in a NGZ and we want, if it is compromised for it to be extra difficult for it to gain GZ control

Also, As I have seen on SmartOS , external interface actually does not even get an IP address that is used on GZ. It is only used for VMs and zones's VNICss.

My admin network is RFC1918 with very restricted access on the router, other VLANs are either public or private addresses as needed but all firewall protected on external router

Yeah they broke into my flat, unlocked keys without braking anything, put keylogger and went back in flat. Employ Encryption and 2FA.

my ssh key is on a smartcard or yubikey

And you can have multiple networks if multiple NICs, not only 2. etc. nictagadm list , shows what is available

We have multiple NICs but are unfortunately on a wifi-wifi bridge at the moment. So 1 real network that can be separated by VLANs if need be.

You guys have been super helpful, thanks

I'd use separate NICs for GZ and NGZs if practical but some of my devices are small form factor with no interal expansion

Eric_F, there is also #illumos for kernel side and general things, there are youtube videos called "Office hours" you can maybe check

And booting from USB is really doable for eliminating rootkits. Just swap USB flash and reboot, for upgrade of GZ :P

Eric_F, docs.tritondatacenter.com/private-cloud/networks/sdn/architecture

Eric_F, Also if using PCI-Passthrough of some graphics or other LLM acceleration hardware, to HVM / bhyve VM, consider that whole machine dedicated to that use and networked accordingly. Don't count that PCI-E passthrough will separate you enough from HVM that has PCI-E bus access.

tsoome_: thanks! i’ll look at it a bit later