Follow up PR to #34 with complex fields serialized as string representation.

Requires some more testing with a real prometheus instance to make sure this works. I really want to make sure not to break peoples service discovery with custom fields in labels.

Add new query with iregex search and add custom_fields data to the return.

Example route : https://netbox.com/api/plugins/prometheus-sd/devices/?deviceModel=d.*nk -> this iregex can return all "dlink" models

I tried to install this plugin under netbox v3.2 that was released a few days ago. Unfortunately the plugin fails with:

netbox-worker_1        | Traceback (most recent call last):
netbox-worker_1        |   File "/opt/netbox/netbox/manage.py", line 10, in <module>
netbox-worker_1        |     execute_from_command_line(sys.argv)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
netbox-worker_1        |     utility.execute()
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 440, in execute
netbox-worker_1        |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/management/base.py", line 414, in run_from_argv
netbox-worker_1        |     self.execute(*args, **cmd_options)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/management/base.py", line 455, in execute
netbox-worker_1        |     self.check()
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/management/base.py", line 487, in check
netbox-worker_1        |     all_issues = checks.run_checks(
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/checks/registry.py", line 88, in run_checks
netbox-worker_1        |     new_errors = check(app_configs=app_configs, databases=databases)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/checks/urls.py", line 14, in check_url_config
netbox-worker_1        |     return check_resolver(resolver)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/checks/urls.py", line 24, in check_resolver
netbox-worker_1        |     return check_method()
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/urls/resolvers.py", line 480, in check
netbox-worker_1        |     for pattern in self.url_patterns:
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/utils/functional.py", line 49, in __get__
netbox-worker_1        |     res = instance.__dict__[self.name] = self.func(instance)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/urls/resolvers.py", line 696, in url_patterns
netbox-worker_1        |     patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/utils/functional.py", line 49, in __get__
netbox-worker_1        |     res = instance.__dict__[self.name] = self.func(instance)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/django/urls/resolvers.py", line 689, in urlconf_module
netbox-worker_1        |     return import_module(self.urlconf_name)
netbox-worker_1        |   File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
netbox-worker_1        |     return _bootstrap._gcd_import(name[level:], package, level)
netbox-worker_1        |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
netbox-worker_1        |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
netbox-worker_1        |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
netbox-worker_1        |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
netbox-worker_1        |   File "<frozen importlib._bootstrap_external>", line 855, in exec_module
netbox-worker_1        |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
netbox-worker_1        |   File "/opt/netbox/netbox/netbox/urls.py", line 9, in <module>
netbox-worker_1        |     from extras.plugins.urls import plugin_admin_patterns, plugin_patterns, plugin_api_patterns
netbox-worker_1        |   File "/opt/netbox/netbox/extras/plugins/urls.py", line 35, in <module>
netbox-worker_1        |     urlpatterns = import_object(f"{plugin_path}.api.urls.urlpatterns")
netbox-worker_1        |   File "/opt/netbox/netbox/extras/plugins/utils.py", line 31, in import_object
netbox-worker_1        |     spec.loader.exec_module(module)
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/netbox_prometheus_sd/api/urls.py", line 2, in <module>
netbox-worker_1        |     from .views import VirtualMachineViewSet, DeviceViewSet, IPAddressViewSet
netbox-worker_1        |   File "/opt/netbox/venv/lib/python3.9/site-packages/netbox_prometheus_sd/api/views.py", line 4, in <module>
netbox-worker_1        |     from extras.api.views import CustomFieldModelViewSet
netbox-worker_1        | ImportError: cannot import name 'CustomFieldModelViewSet' from 'extras.api.views' (/opt/netbox/netbox/extras/api/views.py)

It seems some changes are needed to make it compatible with newer netbox.

There seems to be a ticket with changes that were made to the plugin framework.

When a contactAssignment is created the plugin will now extract the following fields when they are populated. If no value is present for email or role then the label will not be created.

        "__meta_netbox_contact_primary_name": "test contact",
        "__meta_netbox_contact_primary_email": "[email protected]",
        "__meta_netbox_contact_primary_comments": "comment",
        "__meta_netbox_contact_primary_role": "contact role",
        "__meta_netbox_contact_secondary_name": "Test Name 2",
        "__meta_netbox_contact_secondary_role": "contact role"

Have also bumped the plugin version as shown in Netbox to 0.4 as it was still 0.1. (on assumption it gets a bump)

I have not updated test cases as I cannot run them locally here to verify. Testing has been conducted against a running netbox instance and using to curl to verify plugin output.

How hard would it be to make a single endpoint which exposes both devices and virtual-machines?

Reason: AFAICS, currently every Prometheus scrape job needs to be duplicated, since prometheus http-sd can only scrape a single URL. If you've got a lot of rewriting rules, that can be painful.

A union endpoint would solve this, and __meta_netbox_model can be used to distinguish between the objects if necessary.

However, I can see there is much Django ORM queryset and serializer magic going on, so it may not be easy. Maybe this is relevant.

is it possible to scrape multiple exporters on one target? i.e. node_exporter on port 2000, mariadb_exporter on port 2001

how can i define exporter port?

Bumps pylint from 2.9.6 to 2.10.2.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps pylint from 2.15.5 to 2.15.8.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps pylint from 2.15.5 to 2.15.7.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps pylint from 2.15.5 to 2.15.6.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Based on on #69

Add custom fields as labels. This is an early and simple approach, may be we need some more filters for more complex custom fields. For now only primitive types (bool, str, int) are exported. For more complex types like multiple choice or relations we need to investigate how to export them to labels.

Hi everybody, I'm trying Netbox 3.4.0 with netbox-plugin-prometheus-sd and I can't see any device using the /api/plugins/prometheus-sd/devices API even if I have at least 1 device.

This is the request:

GET http://netbox.local:8000/api/plugins/prometheus-sd/devices/

and this is the answer:

` HTTP 200 OK Allow: GET, POST, HEAD, OPTIONS Content-Type: application/json Vary: Accept

[] `

But I have at least 1 device (Dell R630 in state Active)

Bumps setuptools from 65.3.0 to 65.5.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps pylint from 2.15.5 to 2.15.9.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hi, below is the Netbox API return from /api/dcim/devices/?name=router-01.example.com and I would like to know if you could implement in your API a filter on the model or model slug (using the model__iregex) that will return all devices matching this model as well as an option to return all custom_fields as labels, please.

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 8,
            "url": "https://netbox.example.com/api/dcim/devices/8/",
            "display": "router-01.example.com",
            "name": "router-01.example.com",
            "device_type": {
                "id": 61,
                "url": "https://netbox.example.com/api/dcim/device-types/61/",
                "display": "Dlink xSeries",
                "manufacturer": {
                    "id": 12,
                    "url": "https://netbox.example.com/api/dcim/manufacturers/12/",
                    "display": "DLINK",
                    "name": "DLINK",
                    "slug": "dlink"
                },
                "model": "Dlink xSeries",
                "slug": "dlink-xseries"
            },
            "device_role": {
                "id": 5,
                "url": "https://netbox.example.com/api/dcim/device-roles/5/",
                "display": "ROUTER",
                "name": "ROUTER",
                "slug": "ROUTER"
            },
            "tenant": {
                "id": 1,
                "url": "https://netbox.example.com/api/tenancy/tenants/1/",
                "display": "HOME",
                "name": "HOME",
                "slug": "home"
            },
            "platform": null,
            "serial": "01234567890",
            "asset_tag": null,
            "site": {
                "id": 1,
                "url": "https://netbox.example.com/api/dcim/sites/1/",
                "display": "Malakoff",
                "name": "Paris",
                "slug": "paris"
            },
            "location": null,
            "rack": {
                "id": 8,
                "url": "https://netbox.example.com/api/dcim/racks/8/",
                "display": "RACK 11",
                "name": "RACK 11"
            },
            "position": 6,
            "face": {
                "value": "front",
                "label": "Front"
            },
            "parent_device": null,
            "status": {
                "value": "active",
                "label": "Active"
            },
            "airflow": {
                "value": "side-to-rear",
                "label": "Side to rear"
            },
            "primary_ip": null,
            "primary_ip4": null,
            "primary_ip6": null,
            "cluster": null,
            "virtual_chassis": null,
            "vc_position": null,
            "vc_priority": null,
            "comments": "",
            "local_context_data": null,
            "tags": [],
            "custom_fields": {
                "IP": "10.1.1.10",
                "snmp_communaute": "read",
                "snmp_communaute_alternate": null,
                "snmp_version": "2c",
                "snmp_port": "161",
                "ping_frequence": "60",
                "snmp_frequence": "60",
                "client": null,
                "exploitant": "Operator",
                "partenaire": null
            },
            "config_context": {},
            "created": "2022-07-12T12:21:22.461170Z",
            "last_updated": "2022-07-12T12:23:18.688284Z"
        }
    ]
}

Thank you in advance.

Hey @FlxPeters.

I got this up and running and solved a few problems in testing. Ace work!

I was just testing this out and noticed that the custom fields are not coming out where the netbox-prometheus-sd does pull them out and turn the fields into labels. (I'd patched a bug there with label txt). I think it would be nice to export them as I'd based some functionality around it based on the other tool and it allows people to pass some params outbound for things like escalation ownership etc.

from the other closed issue about this - I'm just testing using tags etc, services looks like a right pain and not a fast way to turn it on.

I'll have a look at the code and see if I can figure out something and use your logic from the other application. might look at a url option for customlablels=true to turn them on and off etc.

happy to bounce a few things about.