Skip to content

Model data pipeline

Modeling Process Explained

  • Zenmodeler performs most modeling tasks
  • On startup, and every configuration flush interval (6 hours) zenmodeler gets a ModelerService config from zenhub
  • zenhub pulls device data (ip address, credentials, etc) and modeler plugin bindings from ZODB (and leveraging solr) and generates a ModelerService config
  • zenhub serves that config to zenmodeler
  • Zenmodeler uses the ModelerService config as a list of tasks it needs to complete
  • Zenmodeler executes modeler plugins
  • Modeler plugins retrieve data from devices
  • Modeler plugins format that data and construct an ObjectMap
  • an ObjectMap is a key:value object that contains component data
  • Modeler plugins return the ObjectMap to zenmodeler
  • Zenmodeler builds a DataMap using ObjectMaps from one or more modeler plugins and returns it to zenhub
  • zenhub executes an applyDataMap task and commits model data to ZODB and the global catalog via solr

Troubleshooting modeling

Failure to collect data & Failure to process data

  • Run zenmodeler in debug mode to see what data is being returned

  • First, connect to the appropriate zenmodeler container

    serviced service attach zenmodeler

  • Switch to the 'zenoss' user

    su - zenoss

  • Run zenmodeler in debug mode against the device

    zenmodeler run -v10 -d $DEVICENAME --monitor=$COLLECTORNAME

  • Optionally, you can specify the 'collect' switch to run a specific plugin

    zenmodeler run -v10 -d $DEVICENAME --monitor=$COLLECTORNAME --collect=zenoss.snmp.IpInterface

Failure to connect to zenhub?

  • Is zenhub running? Is it overloaded?
  • Check health checks
  • Look in the zenhub container; put it in debug mode and see if there are any messages like “All workers are busy” as this could indicate that you need more zenhub workers or instances

Failure to load a modeler plugin?

  • Is this a new ZenPack?
  • Have you installed a ZenPack without restarting zenoss.resmgr services?

Failure to collect data - Auth/access issues?

  • Check credentials and ZenPack documentation for end-device configuration

Failure to collect data - Incorrect modeler plugin?

  • Does this modeler plugin support the device you’re trying to model?

Failure to parse data

  • Look for tracebacks - normally things like KeyError exceptions in modeler plugins are indicative of a failure to parse data
  • These types of failures are often indicative of either something wrong with the data the device is returning to us (potentially an end-device compatibility problem), or a potentially incorrect assumption we’ve made in code

Failure to apply data to the model

  • Look at zenhub; it may be overloaded
  • Bad relationships
  • Try deleting and re-adding the device
  • Make sure the device was created with the correct zPythonClass
  • If you’re running a WinRM modeler plugin on a device that was created in a class outside the /Server/Microsoft device class, for example, the device object’s class may not possess the required attributes to accept that model data