[feature] API compatibility with openhab-helper-libraries

[mbronk]

Is it planned for these libs to eventually become interface-compatible with the legacy OpenHAB Helper Libraries❓
Alternatively, provide a migration guide?

Why consider?

I don't have any current stats to back this up, but I believe the helper libs are/were pretty popular back in the day (esp. as they were available in multiple languages 🥇 and really well documented 📖 ).
I believe the repo might have been abandoned and the original libraries available at: https://github.com/openhab-scripters/openhab-helper-libraries may no longer work as-is with OH >= 3.x, but there are forks, such as these:

• https://github.com/CrazyIvan359/openhab-helper-libraries/
• https://github.com/jsjames/openhab-helper-libraries

... that just work ™ - to this day (OH 5.0 snapshot, legacy Jython-based binding).

I see these libs adopt some of similar syntax, like @rule annotations, so it would be awesome 🚀 to make the migration as painless as possible.

• Note: I'm not at all proposing to overencumber the libraries with needless legacy.
  Just making a practical observation that since there's something out there (with non-zero install base), it may make some sense to consider leveling with those APIs where it makes sense, to make it easier for people to jump ship to the new coolness 😉.
  Same goes with directory organization etc.

Alternative ideas / constraints

1. IMHO it only makes sense to support python 3.x going forward, which means even if these libs were 100% API-compatible with the openhab-helper-libraries (legacy), one would likely need to at least run their scripts through 2to3 for syntax compatibility.
2. The new libraries get to use much more modern stuff with some legacy APIs likely not being needed at all. A "breaking changes" kind of note for these would be helpful though (and while the implementation will surely differ, I'm not sure if the APIs need to vary too much).
3. Alternatively, the org.openhab.automation.pythonscripting binding could allow choice of which helper libs to use (this cleanroom implementation vs. a legacy-compatible).

---

PS. @HolgerHees hats off to you for the great work you've done w/ both these libs and the OH binding. 👏 👏 👏
I haven't been able to run it on my production setup yet (too many legacy rules to rewrite), but it is looking very promising already when I was playing with it on a sandbox. Don't want to overpromise anything as I don't have much time for personal projects lately, but would happily help out in the limited capacity I can (at least testing & feedback, possibly some coding)

[HolgerHees]

The goal was to develop a lightweight API that is as close as possible to the openHAB API and eliminates all unnecessary features. This facilitates future maintainability and makes new OpenHAB functions available without having to implement them in the API first. In the old legacy API, they tried to encapsulate pretty much everything. This was exactly the path I wanted to avoid.

So the idea was to create something completely new, essentially from scratch, a clean new implementation based on best practices and experiences. I also think porting the legacy API is the wrong approach, as it's far too bloated and overloaded with unnecessary features.

My recommended approach is therefore:

Legacy scripts should continue to run with Jython. I will try to support this binding as long as it can be run with new Java versions. You can use the deprecated legacy helper lib there. I think this approach will work for > 2-3 years until a technical reason prevents us from supporting jython.

Anything new can be ported to the new API gradually.

I've already ported a lot of things, like @rule, @when, etc. From my own experience, porting ~150 rules and ~10,000 pieces of code took about two days. Most of it was search and replace.

to make it short, the new API will never be a drop in replacement. Otherwise, we'll be carrying around such legacy issues forever. And since jython will be running for a few more years, there's a smooth migration path, because you can still run both in parallel.

but this does not mean I will not implement anything additionally. If you feel there is really something missing or it is difficult to solve with the current API, lets talk about how to make it possible.

[rkoshak]

In the old legacy API, they tried to encapsulate pretty much everything.

The reason why the JS Scripting helper library uses this approach (i.e. by default everything is a JS Object) is because of the whole "is this a JS Object or is it a Java Object" problem. Otherwise you have to constantly be aware of what type of Object you are working with because a JS.Array is not the same and does not work the same as a java.util.List.

If the help library is too thin, that means the actual parts of the code that interact with OH itself end up not being compatible with third party libraries and core features of the language. Or worse you end up needing to use different syntax for something that is conceptually the same becuase one is native and one is Java.

I'm not arguing for or against any given approach. Just giving a data point and a decision that was made elsewhere when faced with the same problem. It probably would be useful to make a little bit of effort to convert or wrap Java stuff with Python native stuff in places where it makes a difference.

[mbronk]

Actually, I am not at all suggesting to port the legacy libraries as-is, or to achieve 100% compatibility.

Rather was thinking what is smart to retain to allow folks to transition between the implementations easier (both ways!).
From a subjective POV, it may bring you bigger launch-day install base if folks were able to experiment w/o much upfront investment (incl. falling back to old ways, if they don't like the new approach).

In that spirit, if the APIs must stay disjoined, some kind of a migration guide would be very much helpful.

It varies on a case by case basis, but in the new lib, there would likely also be areas where backward-compat* is not preserved simply due to naming mismatch, not even an interface change. Surely, this implementation is more lean, but I don't think adding a compat wrapper on top would be that costly either (behavior-wise it's the same engine underneath, just additional indirections added by the legacy lib, to be compatible w/ JS libs as well).

* - if we can even use this term, as these are separate impls.!

Naive examples:

openhab-helper-libraries (legacy) syntax	openhab-python (new, this) syntax
method_name.log	method_name.logger
itemRegistry.getItem("Item1")	Registry.getItem("Item1")

---

@HolgerHees you're the 👑 here, so if the choice to break compatibility was deliberate - it's of course a valid direction😉. I am merely probing how "close" you'd want to make this new implementation resemble the (arguably) de-facto standard for the Jython version and if the answer is not at all I think it is as valid as any other.

BTW - Yes, probably could run legacy stuff with Jython emulation enabled, but IMHO it defeats the point of migrating to new binding in the first place.

[HolgerHees]

Regarding the migration guide, i totally agree. Helping here is always welcome :-)

and regarding the other reasons, I would like to wait for now, collect some more feedback from people using it and decide after, which further steps are needed. I know, it will not be the perfect version from the beginning, but I also do not want to implement things which are not needed or used at all.

The biggest pain point, the conversion between Java ZonedDateTime and python datetime is already solved and the following additional things are already wrapped transparently.

Python class	Java class
datetime with timezone	ZonedDateTime
datetime without timezone	Instant
timedelta	Duration
list	Collection
Set(set)	Set

For everything else, it does no feel so painful, from my perspective. I mean, I run ~10.000 lines of python code without any difficulties. But I'm open for other perspectives. Lets wait, to see where other people exactly have their specific pain points.

I personally see it as a big advantage, that I can just look into the openhab API documentation, and I know, that this is reachable and usable from python without any limitations or restrictions.

a good starting point is the initial readme. It gives you already an overview what exactly is wrapped and what is just forwarded to the java API.

btw. I already got feedback from 8 people migrating their whole production setup to python3 and after the migration no one was missing a specific feature. So my feeling is, start trying it out, check if you can do what you want, and tell me if you have some ideas how to improve or simplify things. Some of these people was showing me already very specific improvements which are already included in the current version.

here you can also find a lot of examples, using the new API

https://github.com/HolgerHees/openhab-config/tree/master/conf/automation/python

[HolgerHees]

far from being complete, but at least a tiny starting point

https://github.com/HolgerHees/openhab-python/blob/main/MIGRATION.md

[mbronk]

far from being complete, but at least a tiny starting point
https://github.com/HolgerHees/openhab-python/blob/main/MIGRATION.md

Wow, this is a great starting point! Thanks!
Will do my best to document my adventures when migrating my own code, and PR/suggest additions to this guide (if any).
Sadly, can't commit to any concrete timeframe, so please don't mind me, though.

I know, it will not be the perfect version from the beginning, but I also do not want to implement things which are not needed or used at all.

Agreed, but incorporating into a stable 5.0 release 🤞 , will create an implicit assumption of all the APIs (incl. helper libs) to be somewhat stable and extend-only (at least till 6.x)...

• In an ideal world, I'd consider this library THE new API (most lean/efficient), but allow existence of a compat layer, providing adapters for the legacy interface (not written from scratch, rather on top of this lib).
• At the present, the API is disjoint from the legacy libs (and likley for the right reasons❗ - not trying to bring back this argument).
  However, if you ever decide to make it a subset of the old APIs later, I figure it may be difficult to do renames or removals after 1st 'official' release (unless of course, you explicitly mark the library as experimental).

I personally see it as a big advantage, that I can just look into the openhab API documentation, and I know, that this is reachable and usable from python without any limitations or restrictions.

Well, you could argue the same was possible before as well (you could always go 'native API' - only difference is what Jython vs. GraalPy are doing for you wrt. built-in conversions).
In fact, I think one of the biggest selling points of the legacy lib (and Ivan's fork) was a ready-to-go integration w/ VSCode, including full typings and docstrings generated off Javadocs. Ref:

• https://github.com/CrazyIvan359/openhab-stubs
• https://community.openhab.org/t/ivan-s-helper-libraries-oh3-python-javascript/116458

These are, of course, orthogonal to this lib... but at the same time... perhaps it is a good idea to consider bundling the typings w/ the lib (ex. generated via stubgenj) to get code completion and linting ❔
Possibly... even bundle a 'default' workspace settings for popular editors such as VSCode/PyCharm (this likely needs to be opt-in by the user though).

[HolgerHees]

Agreed, but incorporating into a stable 5.0 release 🤞 , will create an implicit assumption of all the APIs (incl. helper libs) to be somewhat stable and extend-only (at least till 6.x)...

I totally agree and this is why I need feedback, to see if people are happy with it or if further work or refactoring is needed. From my current perspective, I would say, the API is done. I don't miss anything and I can do everything I want. But again, this is only my perspective and I'm open for improvements coming from other people. @jsjames provides a lot of feedback and pull requests here too, to bring it in a state where it is now. He also contributed some changes from his helper lib fork. Additionally he helped a lot with code reviews and suggestions.

The only thing I'm not 100% happy with (Just 99% :-)), are some namings. The suggestion came from @jsjames too and as I had no real preferences before and no better idea, I was going this direction.

from openhab import rule, logging, Registry
from openhab.triggers import when, ChannelEventTrigger

import scope

1. the naming of the helper lib module "openhab", before it was "core".
2. uppercase and loewercase differences

but I think both cases make sense and I don't have a better idea, so can live with it. If other people are happy with it too, I would say it is finalized.

but allow existence of a compat layer,

I'm also 100% open to this, but I can't afford it alone, because I never really used the old helper libs. So I depend on other peoples help here too. If someone is willing to write a combat layer to make the life easier for other people, I'm open to review and merge. Also if people are just telling me, where the pain point are, I can try to implement something to solve this pain points.

was a ready-to-go integration w/ VSCode, including full typings and docstrings generated off Javadocs

I was not aware of this and I would be happy if we can provide something similar for the new lib. I will check stubgenj and see what I can do.

Also I'm still looking for a better way to generate the documentation somehow, instead of just maintaining the README.md or MIGRATION.md, but I have to do some research here too.