feat(medcat): CU-869bydfx8 Allow downloading missing plugins

[mart-r]

The issue with plugins so far has been that it would be difficult for users to install them unless they know what they are or where they are how how to install them.

This PR attempts to rectify that. Its main contribution is as follows:

• Add CLI to allow downloading known plugins
  • python -m medcat install-plugins <plugin list>
• Add curated list of plugins and how to install them
  • They're defined at medcat/plugins/data/plugin_catalog.json
  • If/when appropriate this can be updated from the GH
• Add instructions to install if/when a known plugin is missing
• Add some documentation regarding plugins and curated plugins

A sample stack trace when trying to load a medcat-gliner model:

% python -c "from medcat.cat import CAT;cat = CAT.load_model_pack('../../cogstack-ops/medcat-gliner/temp/2023_w_gliner_234dda1597f635e3.zip')"
Missing required plugins for this model pack. Attempting to load anyway, but it may fail. Missing: ['medcat_gliner']
Traceback (most recent call last):
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/cat.py", line 857, in load_model_pack
   cat = deserialise(model_pack_path, model_load_path=model_pack_path,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/storage/serialisers.py", line 369, in deserialise
   return ser.deserialise_all(
          ^^^^^^^^^^^^^^^^^^^^
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/storage/serialisers.py", line 210, in deserialise_all
   part = self.deserialise_all(
          ^^^^^^^^^^^^^^^^^^^^^
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/storage/serialisers.py", line 210, in deserialise_all
   part = self.deserialise_all(
          ^^^^^^^^^^^^^^^^^^^^^
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/storage/serialisers.py", line 210, in deserialise_all
   part = self.deserialise_all(
          ^^^^^^^^^^^^^^^^^^^^^
 [Previous line repeated 2 more times]
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/storage/serialisers.py", line 189, in deserialise_all
   module = import_module(module_path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Users/martratas/.pyenv/versions/3.12.10/lib/python3.12/importlib/__init__.py", line 90, in import_module
   return _bootstrap._gcd_import(name[level:], package, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
 File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
 File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
 File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
 File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
 File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'medcat_gliner'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "<string>", line 1, in <module>
 File "/Users/martratas/Documents/CogStack/.MedCAT.nosync/monorepo-nlp/medcat-v2/medcat/cat.py", line 874, in load_model_pack
   raise MissingPluginError(missing_plugins) from e
medcat.utils.exceptions.MissingPluginError: The following required plugins are missing:
 - Plugin: medcat_gliner
   Provides components: ner:gliner_ner
   Author: Mart Ratas <mart.ratas@kcl.ac.uk>
   URL: Homepage, https://github.com/CogStack/medcat-ops/tree/main/medcat-gliner

   NB: You should be able to install this plugin using:
       python -m medcat install-plugins medcat_gliner

Please install the missing plugins to load this model pack.

Yet after doing the install as per the above hint, the model load works as expected.

EDIT:
Also verified that the wheel does in fact contain the *.json data using:

import importlib.resources as r

path = r.files("medcat.plugins.data") / "plugin_catalog.json"
print("Exists:", path.is_file())
print("Contents snippet:", path.read_text()[:200])

[tomolopolis]

Task linked: CU-869bydfx8 Add option to download missing plugins

[tomolopolis]

nice - I think we need a guide internally to do some 🐶 fooding

[tomolopolis]

this repo does require_auth no?

[mart-r]

Yep, absolutely! I remember fixing it at some point, but I think I changed schema a little afterwards and forgot to circle back. Should be fixed now.