I love your project and was hoping we can feature it more prominently in the main torchserve repo - I was wondering if you'd be OK and interested in this. And if so I was wondering if you could give me some feedback on the below

Installation instructions

I tried to torchserve-dashboard --server.port 8105 -- --config_path ./torchserve.properties --model_store ./model_store but the page never seems to load regardless of whether I use the network or external url that I have

I setup a config

(torchservedashboard) [email protected]:~/torchserve-dashboard$ cat torchserve.properties 
inference_address=http://127.0.0.1:8443
management_address=http://127.0.0.1:8444
metrics_address=http://127.0.0.1:8445
grpc_inference_port=7070
grpc_management_port=7071
number_of_gpu=0
batch_size=1
model_store=model_store

But perhaps makes the most sense to just add a default one to the repo so things just work. I'm happy to make the PR just let me know what you suggest. Ideally things just work with zero config and people can come back and change stuff once they feel more comfortable.

Also on Ubuntu I had to type export PATH="$HOME/.local/bin:$PATH" so I could call torchserve-dashboard

Features

Also there's some new features we're excited like the below which would be very interesting to see like

1. Model interpretability with Captum https://github.com/pytorch/serve/blob/master/captum/Captum_visualization_for_bert.ipynb
2. Workflow support coming in 0.4 which will allow much more configurable pipelines https://github.com/pytorch/serve/pull/1024/files

In all cases please let me know if you think we're on the right track and how we can make the torchserve more useful to you. I liked your suggestion on automatic doc generation and it's something I'm looking into so please keep them coming!

This PR is related to #1 . It improves the structure of the package setup: All package related info is moved to torchserve_dashboard.init.py.

Requirement files are added which are split up depending on the usage of the repo/package.

All functions linked to setup are moved to torchserve_dashboard.setup_tools.py. The function parsing the requirements can handle commented requirements as well as references to github etc (#egg included in requirement)

Couldn't run the dashboard initially

Traceback (most recent call last):
  File "/Users/me/Desktop/pytorch-mnist/venv/bin/torchserve-dashboard", line 8, in <module>
    sys.exit(main())
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/torchserve_dashboard/cli.py", line 16, in main
    ctx.forward(streamlit.cli.main_run, target=filename, args=args, *kwargs)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/core.py", line 784, in forward
    return __self.invoke(__cmd, *args, **kwargs)
  File "/Users/me/Desktop/pytorch-mnist/venv/lib/python3.8/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
TypeError: main_run() got multiple values for argument 'target'

After a bit of googling, I found this: https://github.com/rytilahti/python-eq3bt/issues/30

The default install brought in click==8.0.1. I had to downgrade to 7.1.2 to get past the error.

• Better caching using @st.experimental_singleton

  • argument parsing and API class initialization should only happen once (across sessions) on initial load.
  • Should be way better compared to before which ran those functions after each page refresh 😱 Might be optimized further later...need to refactor cli-param parsing/init logic.

• Added --init option to initialize torchserve on start. as per this issue: https://github.com/cceyda/torchserve-dashboard/issues/16 Use like: torchserve-dashboard --init Although you still have to load the dashboard screen once for it to actually start!

• Update to match changes in torchserve v0.6

  • there seems to be only one update to ManagementAPI in v0.6 https://github.com/pytorch/serve/pull/1421 which adds ?customized=true option to return custom_metadata in model details. Although the feature seems to be buggy for old .mar files not implementing it. (tested on: https://github.com/pytorch/serve/blob/master/frontend/archive/src/test/resources/models/noop-customized.mar)

  Anyway I added a checkbox (defaulted to False) to return custom_metadata if needed.

update streamlit version to include security update v1.11.1 Although torchserve-dashboard isn't using any custom components and therefore not effected by it.

update torchserve:

• v0.5
• add aws encrypted model feature
• add log-config to options

update steamlit:

• v1.2.0 -> drop beta_ prefix
• drop python 3.6

https://github.com/cceyda/torchserve-dashboard/issues/13

• Add workflow management endpoints (untested)
• Add version check
• Refactor api.py (remove streamlit)

Closes: https://github.com/cceyda/torchserve-dashboard/issues/2

Self-todo ETA: june 3rd https://github.com/pytorch/serve/tree/release_0.4.0/examples/Workflows https://github.com/pytorch/serve/blob/release_0.4.0/docs/workflow_management_api.md

I'm getting this error while on the start

Traceback (most recent call last):
  File "/home/kavan/.local/bin/torchserve-dashboard", line 5, in <module>
    from torchserve_dashboard.cli import main
  File "/home/kavan/.local/lib/python3.8/site-packages/torchserve_dashboard/cli.py", line 2, in <module>
    import streamlit.cli
  File "/home/kavan/.local/lib/python3.8/site-packages/streamlit/__init__.py", line 49, in <module>
    from streamlit.proto.RootContainer_pb2 import RootContainer
  File "/home/kavan/.local/lib/python3.8/site-packages/streamlit/proto/RootContainer_pb2.py", line 22, in <module>
    create_key=_descriptor._internal_create_key,
AttributeError: module 'google.protobuf.descriptor' has no attribute '_internal_create_key'

Btw thanks for this awesome lib.

Mentioned in https://github.com/cceyda/torchserve-dashboard/issues/1 Model interpretability with Captum https://github.com/pytorch/serve/blob/master/captum/Captum_visualization_for_bert.ipynb

This would be good to add if we end up adding an InferenceAPI

Moving the discussion from https://github.com/cceyda/torchserve-dashboard/issues/1#issuecomment-863911194 to here

Current challenges blocking this:

• there is no way to know the format of the expected request/response. Especially for custom handlers.

(I prefer not do model_name->type matching manually)

If a request/response schema is added to the returned OpenAPI definitions, I can probably auto generate something like SwaggerUI.

I think it would be great for users and for developers to be able to easily share their dashboard or run it in production without deploying via Streamlit. I could add a simple Dockerfile wrapping everything up into a container.

Torchserve-Dashbord would be to Torchserve what MongoExpress is to MongoDB. Thoughts?