Model data pipeline
Modeling Process Explained
- Zenmodeler performs most modeling tasks
- On startup, and every configuration flush interval (6 hours) zenmodeler gets a ModelerService config from zenhub
- zenhub pulls device data (ip address, credentials, etc) and modeler plugin bindings from ZODB (and leveraging solr) and generates a ModelerService config
- zenhub serves that config to zenmodeler
- Zenmodeler uses the ModelerService config as a list of tasks it needs to complete
- Zenmodeler executes modeler plugins
- Modeler plugins retrieve data from devices
- Modeler plugins format that data and construct an ObjectMap
- an ObjectMap is a key:value object that contains component data
- Modeler plugins return the ObjectMap to zenmodeler
- Zenmodeler builds a DataMap using ObjectMaps from one or more modeler plugins and returns it to zenhub
- zenhub executes an applyDataMap task and commits model data to ZODB and the global catalog via solr
Failure to collect data & Failure to process data
- Run zenmodeler in debug mode to see what data is being returned
First, connect to the appropriate zenmodeler container
serviced service attach zenmodeler
Switch to the 'zenoss' user
su - zenoss
Run zenmodeler in debug mode against the device
zenmodeler run -v10 -d $DEVICENAME --monitor=$COLLECTORNAME
Optionally, you can specify the 'collect' switch to run a specific plugin
zenmodeler run -v10 -d $DEVICENAME --monitor=$COLLECTORNAME --collect=zenoss.snmp.IpInterface
Failure to connect to zenhub?
- Is zenhub running? Is it overloaded?
- Check health checks
- Look in the zenhub container; put it in debug mode and see if there are any messages like “All workers are busy” as this could indicate that you need more zenhub workers or instances
Failure to load a modeler plugin?
- Is this a new ZenPack?
- Have you installed a ZenPack without restarting zenoss.resmgr services?
Failure to collect data - Auth/access issues?
- Check credentials and ZenPack documentation for end-device configuration
Failure to collect data - Incorrect modeler plugin?
- Does this modeler plugin support the device you’re trying to model?
Failure to parse data
- Look for tracebacks - normally things like KeyError exceptions in modeler plugins are indicative of a failure to parse data
- These types of failures are often indicative of either something wrong with the data the device is returning to us (potentially an end-device compatibility problem), or a potentially incorrect assumption we’ve made in code
Failure to apply data to the model
- Look at zenhub; it may be overloaded
- Bad relationships
- Try deleting and re-adding the device
- Make sure the device was created with the correct zPythonClass
- If you’re running a WinRM modeler plugin on a device that was created in a class outside the /Server/Microsoft device class, for example, the device object’s class may not possess the required attributes to accept that model data