15:03:26 <Smithx10> I have a CN with no aggregations and ran lldpneighbors  and then i ran it again and go t " lldpneighbors couldn't connect to the OpenLLDP transport socket.  Is lldpd running?'
15:05:59 <Smithx10> https://gist.github.com/Smithx10/544e6a72bacf5e2ae97947b102e62c83
15:35:49 <danmcd> COnfig file oddness?
15:35:50 <danmcd> `[ERROR] in config file: invalid location_data_format '0'`
15:36:53 <danmcd> And /var/run/lldpd.sock can't be bound?
15:37:28 <Smithx10> i never configured lldp* 
15:37:46 <danmcd> Oh...
15:37:54 <Smithx10> smartos just had it running
15:38:05 <Smithx10> i figured it was supposed to just work.
15:38:59 <danmcd> I wonder if lldpneighbors attempts to run the daemon?  `pgrep lldp` say anything?
15:39:36 <Smithx10> 253625
15:39:47 <danmcd> And `svcs -xv lldp/server` ?
15:39:48 <Smithx10> Is there another command I'm suppoed to use to get ToR switch port number
15:40:03 <danmcd> (I'll note it's running on my no-aggr systems as well.)
15:40:11 <Smithx10>  State: online since December 15, 2022 at 03:39:27 PM UTC  
15:40:15 <Smithx10> no impact
15:40:31 <Smithx10> I know it doesn't work over aggregations 
15:40:45 <danmcd> SOrry.
15:40:45 <Smithx10> but we are just debugging this 1 where we believe we had another cross connection
15:41:00 <danmcd> I just got up and was going to start testing this week's Triton.
15:41:20 <Smithx10> No problem not a big deal
15:41:24 <Smithx10> its all jbk  fault :P
15:41:47 <danmcd> Anyway... `svcs -xv lldp/server` on both your local machine and any appropriate remote ones.
15:48:44 <neuroserve> Smithx10 same here - lldpneighbors answers once - maybe there's some kind of rate limiting - or it's waiting for the next lldp packet
15:48:59 <Smithx10> neuroserve:  yea,  looks like thats the behavior 
15:49:51 <neuroserve> Smithx10 but on some CNs/HNs it answers every time...
15:50:14 <bahamat> I think that a peer is sending it data that makes it crash.
15:50:31 <bahamat> I've seen it happen, but not consistently enough to track it down.
15:50:59 <Smithx10> Probably Cisco 
15:51:04 <Smithx10> Ive got Cisco toR 
15:51:11 <Smithx10> ACI Nexus switches
15:51:18 <neuroserve> restarting lldpd changes the behaviour...
15:51:37 <neuroserve> Juniper here...
15:52:02 <Smithx10> neuroserve:  your using ixgbe?
15:52:34 <neuroserve> nope - igb - oldschool
15:53:17 <bahamat> I've been dying to replace our lldpd, but the alternatives don't compile on illumos. I started writing one in rust, but I got hung up with it and didn't get very far.
15:56:59 <bahamat> Though, there is now rust_lldp, so maybe I should look into that again.
15:59:40 <Smithx10> bahamat:  can you send over the url please?  I googled it and couldn't find it
16:12:09 <Smithx10> https://crates.io/crates/rust_lldp
16:12:24 <Smithx10> ahh strange when I typed it exactly into google nothing popped up,  but just lldp crate poped right up
16:26:22 <Smithx10> https://github.com/NE4Y/rust_lldp
17:21:44 <bahamat> Yeah, but docs.rs says it doesn't build, and there's literally no documentation for it.
17:28:04 <Smithx10> yea,  very minimal
17:28:30 <jbk> maybe i should finish up my version :)
17:29:34 <jbk> though it still doesn't solve the aggr issue, but it'd at least integrate with smf and other bits as needed
21:33:58 <Smithx10> do we have a /dev/kmsg ?
21:37:26 <danmcd> no.
21:38:10 <danmcd> we DO have /dev/msglog though.
21:38:32 <danmcd> ooops, that's just for the console (but maybe that's what you need?)
21:38:46 <danmcd> What problem are you trying to solve Smithx10?
21:39:17 <Smithx10> We got hit by the flapping interface again
21:39:34 <Smithx10> So I was looking at go libs for parsing the last 15 seconds from it 
21:39:39 <Smithx10> found a lib https://github.com/euank/go-kmsg-parser
21:39:57 <Smithx10> think I will just probably read /var/adm/messages 
21:40:07 <Smithx10> and look at the last timestamp *
21:40:44 <rmustacc> Smithx10: sysevents?
21:40:44 <danmcd> Yeah, that's the supported interface.  NOTE that we collapse repeated messages there, so you may wonder "why am I only seeing one" and then seeing something like:
21:40:46 <danmcd> Dec 13 23:41:05 neuromancer last message repeated 1 time 
21:41:47 <Smithx10> rmustacc:  I'm using prometheus that come around to do a scrape every 15s 
21:42:12 <Smithx10> Is there a counter for a flapping interface,  or would I have to subscribe to the events and keep that count myself
21:48:09 <danmcd> If it's flapping out long enough to make the NIC go "down" from IP's POV, it's possible `route monitor` may suffice.
21:49:33 <rmustacc> Right now you'd have to keep the counter yourself.
22:18:06 <jbk> possibly crazy/bad idea... there's a change from robj that was never done for adding an interface to generate ireports in the kernel.. i wonder if link notifications might be an appropriate use case for such a thing
22:18:28 <jbk> (IIRC, the example there was for something like noting SAS link speed change)
22:22:02 <rmustacc> jbk: I mean, we already have that part of it there.
22:22:13 <rmustacc> We are already generating a sysevent.
22:22:19 <rmustacc> So changing that to an ireport doesn't buy much.
22:31:34 <jbk> can ireports be used with the fmd modules? it might be nice to be able to mark the interface as degraded or such if it's flapping too frequently (vs. having to scrape logfiles to find out)
22:38:55 <rmustacc> It can subscribe to the sysevents?
23:00:46 <jbk> hmm.. probably.. looks like there are modules that already do for other stuff
23:00:58 <jbk> and fmd doesn't fork() after the initial startup
23:01:33 <jbk> so don't have to worry about it deadlocking
23:03:15 <jbk> next time I hit it, I should really file a bug since apparently no one has
23:04:16 <jbk> to get all the details, but IIRC, if a thread tries to fork() while another is in a door call, it can cause a deadlock and unkillable process
23:11:51 <rmustacc> OK. If you can write up some details, give me a shout and we can look at it.
23:28:30 <Smithx10> so /var/log/messages isn't syslog  lol
23:29:29 <Smithx10> at least not 3164 or 5424