Feature Req: LVGL bindings

[jdtsmith]

LVGL (a mini graphics library) has really usable bindings for micropython, and compiles pretty easily. They seem to be focussing more and more on MP. I'm using it with MP on a tiny TTGO-TDISPLAY and it works amazingly well for drawing and building interfaces. Really simple and clean and powerful. But man do you spend time flipping back and forth to the docs.

The auto-generated LVGL bindings in MicroPython have an incredibly broad set of attributes (e.g. lv.textarea.* has ~400 entries!). Their gen_mpy script does all the heavy lifting to create the module. I wonder how easy it would be to perform your stub-making on the LVGL bindings generated by gen_mpy, to add them to your collection? That would make building LVGL apps in MP almost trivial! Thanks for micropython-stubs!

[Josverl]

Hi, there is a similar request for the LVGL in the stubber repo , so lets try to do this.

Im currently working on generating much richer MicroPython stubs from the documentation, and that seems to be nearly done.

if you can point me to firmware or instructions to build a MPY + LVGL for a M5Stack Basic or M5 Fire then I'll have a look on what can be done .

[jdtsmith]

Great. I'm not familiar with that hardware, but looks like it's ESP32-based and has a (likely?) supported display. If you want to build your own, the build directions are pretty straightforward., especially if you've built MP yourself before. It's just slow. I built my own to add more fonts in (by default only 2 are included).

If you just want to give it a quick try, the 16MB firmware from this link should get your started.

I do wonder if gen_mpy (linked above) could be modified to easily produce only stubs instead of the full modules (which call into the LVGL C code functions).

Happy to try out your richer stubs if it helps.

[robertoetcheverryr]

I have uploaded my basic lv_micropython build to this repo:
https://github.com/robertoetcheverryr/lv_micropython-fw
The GENERIC_SPIRAM should work for the M5 Fire (And I strongly recommend using SPIRAM boards with lvgl. In a non-SPIRAM board there is less than 90k of RAM available after boot).
I flash it with:
esptool.py -p PORT -b 460800 --before default_reset --after hard_reset --chip esp32 write_flash --flash_mode dio --flash_size detect --flash_freq 40m 0x1000 bootloader.bin 0x8000 partition-table.bin 0x10000 micropython.bin

[jdtsmith]

I echo the benefits of SPIRAM, but one of the cool things about LVGL is it lets you allocate a single buffer of say 1/10th the screen area, and it uses that buffer behind the scenes to batch graphics updates to the display. E.g. an entire image never needs to be in RAM. So even with 90k free you're able to do interesting things. Still, extra RAM is always nice, especially for canvas drawing.

[Josverl]

I managed to get the firmware flashed , but am not able to get anything on the display yet ( probably not having the correct initialization in place just resets the MCU)

I was able to :

• flash the firmware
• get the module list
• merge / add them with the existing module list
• run createstubs sucesfully
• failed downloading the lvlg.py module ( too large)
• mounted an SD card and generated the stubs to SD
• got an inital view at the lvlg file ( its huge , 52.000 lines 😮)
• isolate the lvlg files
• fix / extend createstubs.py
• create a stub_lvgl.py to simplify creating the FW stubs and add
• need to run the process to create the *.pyi files
• add this to the stubber repo as lvlg_8_1_0_dev-esp32
• Add the lv_micropython frozen modules
• create a procedure to automate this

[robertoetcheverryr]

@Josverl I have been able to run the stubber against my firmware and, just like you say, it's extremely big. One thing I noticed is that not all of the information is stubbed, for example, this is part of the btn class:

class btn:
    ''
    CLASS_EDITABLE = None
    CLASS_GROUP_DEF = None
    DRAW_PART = None
    FLAG = None
    TREE_WALK = None
    def add_event_cb():
        pass

    def add_flag():
        pass

    def add_state():
        pass

    def add_style():
        pass

    def align():
        pass

    def align_to():
        pass

    def allocate_spec_attr():
        pass

    def area_is_visible():
        pass

As you can see, the class' properties and functions are there, but not the functions' parameters:

In the end, the list of modules that lvgl adds to the micropython project is:
['display_driver_utils', 'espidf', 'fs_driver', 'ili9341', 'ili9XXX', 'imagetools', 'lodepng', 'lv_colors', 'lv_utils', 'lvgl', 'rtch', 'uftpd', 'utelnetserver', 'xpt2046']
Of those, I know that espidf should come from the general micropython stub (which wasn't available yet for 1.17 when I ran the stubber a couple of days ago IIRC).

Now, so far I have a hodgepodge of paths for Intellisense to read since it seems that having the unfrozen module is better than a stub and therefore the following modules are available in either the lv_micropython or the bindings repos:
['display_driver_utils', 'display_driver', 'fs_driver', 'ili9341', 'ili9XXX', 'imagetools', 'lv_colors', 'lv_utils', 'uftpd', 'utelnetserver', 'xpt2046']

That leaves:

['lodepng','lvgl', 'rtch']

As the only modules really stubbed from the firmware. I haven't had the time to check if it's possible to compile the bindings and obtain those modules in an unfrozen/more complete state or if the lack of additional information is a function of being cross-compiled.

Currently, I have this order of priorities for the modules:
from Micropython-stubs:
cpython_core
micropython-1.17-frozen
from lv_binding_micropython:
/driver/esp32
/lib
from lv_micropython:
/ports/esp32/modules
And finally, for those that don't exist, the directory with the stubs generated by the stubber.

[jdtsmith]

I managed to get the firmware flashed , but am not able to get anything on the display yet ( probably not having the correct initialization in place just resets the MCU)

It should come up to the MP prompt without any resetting. I had resetting on boot when my custom build overflowed the default "app" partition size, and I had to edit partitions.csv.

Once stable, you can import your driver (ili9XXX?) and import LVGL and try out some of the examples. Here's my display startup:

from ili9XXX import st7789
import lvgl as lv
import fs_driver

disp = None

def start():
    global disp
    if not disp:
        disp = st7789(width=135, height=240, double_buffer = False,
                      factor=8, rot=st7789.LANDSCAPE)
        fs_drv = lv.fs_drv_t()
        fs_driver.fs_register(fs_drv, 'S')
        lv.init()

(I'm using a pre-release st7789 build). Note the "factor" which is the factor by which to reduce the size of the display buffer. The fs_driver stuff is so you can specify images or fonts by file path (with a leading S:).

[robertoetcheverryr]

According to this:
https://docs.makerfactory.io/m5stack/core/fire/
The M5 Fire uses the ILI9341 which should work with this initialization parameters:

display = ili9341(mosi=23, miso=19, clk=18, dc=27, cs=14, rst=33, power=-1, backlight=32, rot=LANDSCAPE, width=320, height=240)

I've tried lvgl's advanced demo application (https://github.com/lvgl/lv_binding_micropython/blob/master/examples/advanced_demo.py) on my non-M5 ESP32 and it works (still working on fully calibrating the TS though)

[Josverl]

Roberto
some answers:

1. The missing function/method/class init parameters
   this is indeed the case as MicroPython does not include that part of the information as far as i have been able to determine, for space considerations. Therefore I have not found a way to interrogate that from the MCU runtime.
   I have been able to get a bit more information ( CONSTANT types and values as well as method decorators ) but that is about as far as that can go.
   PS in comparing my recent export of the `lvlg.btn' class with yours above I have found a 🪲/omission in my code that does not emit all the constants in the updated version. ( more work)

2. thanks for the research on the modules:
   generic frozen modules:

   • 'uftpd',
   • 'utelnetserver',

   lvlg frozen modules (for ESP32):

   • 'display_driver_utils',
   • 'display_driver',
   • 'fs_driver',
   • 'ili9341',
   • 'ili9XXX',
   • 'imagetools',
   • 'lv_colors',
   • 'lv_utils',
   • 'xpt2046'
     C based modules
   • 'lodepng',
   • 'lvgl',
   • 'rtch'
   • possibly other modules on other ports ( but that is a lower prio IMO )

   Note that for other builds there are other frozen modules , but I do have a workflow to gather the frozen modules from the Micropython repo, that is something I could extend/copy to run against the lv_micropython repro as well.

[Josverl]

WRT to getting the display to work ,
Progress is that it does not reset anymore with the below code:
display101.py

but that is does not display anything either :-(
tried this on both the M5stack Core 1 (quite old) and the M5Stack fire
Possible related to this lvgl/lvgl_esp32_drivers#57

[Josverl]

Gents,

I needed to fix more than a few things to deal with the lvgl native modules

• nested classes
• deal with non python classes
• better version of createstubs ( with better, but still limited type inferring)

Note that this (unfortunately) does not include the parameter info for the functions / methods , as that information is not included in the firmware.

• Firmware stubs :
  https://github.com/Josverl/micropython-stubs/tree/master/stubs/lvgl-8_1_0_dev-esp32

• stub_lvgl.py
  https://github.com/Josverl/micropython-stubber/blob/LVGL/board/stub_lvgl.py
  https://github.com/Josverl/micropython-stubber/blob/LVGL/board/createstubs.py

[jdtsmith]

Finally had a chance to test these... brilliant! Seem to be working well. I'll let you know if I run into any issues.

[Josverl]

thanks for letting me know.
Ill close this for now

[jdtsmith]

OK I discovered something quite interesting: lv_micropython includes a json file with all the calling information, structure members, etc. In my build I found it in mpy-cross/build/lvgl/lv_mpy.json (see comment here).

Seems like an easy target to translate and give type information to parameters, with some type translation (e.g. enum_type -> int, etc.). This could be done either by fully parsing this file, or by adding in parameter information from this file to your generated .pyi's. Here's an example from the json:

                "set_style_text_opa": {
                    "type": "function",
                    "args": [
                        {
                            "type": "lv_obj_t*",
                            "name": "obj"
                        },
                        {
                            "type": "int",
                            "name": "value"
                        },
                        {
                            "type": "int",
                            "name": "selector"
                        }
                    ],
                    "return_type": "NoneType"

vs:

    def set_style_text_opa(self, *args) -> Any: ...

Merging these would end up with something like:

    def set_style_text_opa(self, value: int, selector: int) -> None: ...

Here are the types mentioned (and their frequency), leaving out those ending _t, which are already exposed as classes:

mp_arr_to_lv_calendar_date_t_____: 1
mp_arr_to_lv_img_dsc_t_ptr____: 1
mp_arr_to_lv_btnmatrix_ctrl_t_____: 2
function pointer: 2
mp_arr_to_char_ptr____: 2
mp_arr_to_lv_point_t_____: 5
char*: 69
callback: 110
mp_arr_to_lv_coord_t_____: 138
enum_type: 187
bool: 259
void*: 296
enum_member: 1800
lv_obj_t*: 12191
function: 12476
int: 13429

[jdtsmith]

Comparing the json with your stubs also reveals a small problem: everything listed under "functions" in the json incorrectly gets a "self" parameter in your stubs. E.g. color_black. The nice thing about the json is it tells you exactly which things are methods which take an lv_obj_t* as self.

[Josverl]

Interesting , something to look into

1. see if it is possible to recognize/fix the "functions" , perhaps they are class- or static-methods ?
2. read the JSON file to generate richer stubs , and or merge the parameter information into the stubs.

wrt to 2, im close to releasing similar functionality for micropython by generating stubs directly from the documentation.
providing richer information

I also have PoC code to 'merge/enrich' function / method parameters between different files using libCST
perhaps that can be used to take its input from the json you describe.

[jdtsmith]

1. see if it is possible to recognize/fix the "functions" , perhaps they are class- or static-methods ?

It's quite easy to fix the non-methods using the json file, since they are nested in the structure under a top level category "functions" (vs. "objects" for all the class/method info, and "enums" for the static constants like ALIGN.TOP_LEFT, etc.). Also they have no "type": "lv_obj_t*" argument, so no self parameter.

2. read the JSON file to generate richer stubs , and or merge the parameter information into the stubs.

wrt to 2, im close to releasing similar functionality for micropython by generating stubs directly from the documentation. providing richer information

That sounds great. Are you basing on the PyBoardTypeshedGenerator? I believe it also scans the docs. In my brief testing the stubs it produces are truly excellent (but with some typing hard-coding to make various corrections per module). But PyBoard-specific thus far.

I also have PoC code to 'merge/enrich' function / method parameters between different files using libCST perhaps that can be used to take its input from the json you describe.

Yes this sounds good. I doubt you could pull enough info from lvgl docs... this file is structured perfectly to look up a method/class/etc. and enrich parameters. Take a look!

Thanks for your work on this.

[Josverl]

Is def black lvgl.py line 14798 a good example issue 1) ?

If so , that appears to be a regression in the updates I made so that needs to get fixed.
Should not need to use the json to see that self is not needed there

[jdtsmith]

The top-level functions in lvgl.py are now fixed, thanks.

[jdtsmith]

The other advantage of the json is it has all the typing info as well.

[kdschlosser]

that json file cannot be used in it's current form. to create a correct stub file with type hints and documentation. I have tried and it is not correct/missing a lot of things like what classes are a subclass of lv_obj_t. How the naming is done remains a complete mystery to me because you have functions like lv_arc_set_style_outline_width that are in the lv.obj class. I know the first parameter is one of the things used and in come cases the function name gets truncated while in others it does not. Take lv.color.color_to32 the function name in C code is lv_color_to32 but only the lv_ gets removed. That function also gets assigned to all lv_color*_t structures except for lv_color32_t.

There is no standard to the behavior when the micropython code is generated and I have looked at the gen_mpy script for hours on end and have not been able to sort out what in the heck it is doing. I can tell the script was written by a C developer that is unfamiliar with Python and object orientated languages. I have thought about tearing the script down and making it more "Pythonic". Lots of time needs to be invested to do that.