My experience with Embeetle so far

1

Hi all!

I recently started using Embeetle after having difficulties with using a custom library with the Arduino IDE. Perhaps I’ll first situate myself somewhat, so that you have at least some idea from which perspective I’m writing. I’ve currently just finished my engineering studies at KU Leuven and I am doing a personal embedded project. While I learned about many engineering subjects, I did not have much classes about implementation of embedded projects. In short, I’m basically a novice when it comes to embedded development.

My project is not very large (a few thousand lines of own code). Most of my own code concerns control algorithms and is therefore platform independent. My interest lies mostly in the control theory parts and less in implementing it in software and hardware. Nevertheless, in a real-life product the latter step is always necessary, so I’m trying to get some experience with it.

I’ll provide a few comments of my experience with Embeetle so far. I hope these might be useful in the development, and perhaps serve as a motivation to continue developing this nice piece of software.

I started using it last Saturday after having issues with trying to use functionality from my library. I used the Arduino IDE which appears rather restrictive when it comes to library structures. I developed my library separately (I compile it for testing on my PC, with corresponding CMakeLists etc.) and using it with the Arduino IDE appeared close to impossible unless I restructured it completely. Last week, Bert told me “try the beetle”. So I thought, why not?

Well, 30 minutes after clicking “download” I was running code from the library on my Uno. Fantastic!

General thoughts

  • Fast
  • Excellent tiling system
  • Good file usage (I use a second editor with files open in both and have no conflicts)
  • Clean interface
  • Good documentation, both on the website as well as spread out over the program. Please continue doing this. It helps novices like me a lot!
  • Integrating my library went very smoothly. Being able to deselect and force a few files allowed it to compile.
  • Transition from Arduino IDE was excellent. Project conversion went almost perfectly (see comment later).

Scaling

  • “Embeetle IDE” logo in “Embeetle Home” window does not scale when adjusting “Global scale” slider?
  • “Global scale” is difficult to adjust using the provided slider. Suggestions:
    • a wider slider in a wider window
    • logarithmic instead of linear such that a distance delta (on the slider) represents a scaling multiplication factor
    • box to enter a number
    • a few clickable presets, for example 0.50, 1.00, 2.00, 4.00
  • On Ubuntu 20.04.2, GNOME desktop (version 3.36.8), adjustments to the “Scale” setting in the Ubuntu displays settings do not appear to be applied to all visual elements. Screenshot in the next post. (I tested this on a 4K screen).

VCS

  • I currently use Git. It is configured with a remote (even though I work alone). I do not desire to track files that are easily regenerated. I currently have in my .gitignore: “build”, “.beetle”, “config/filetree.mk” and “.cache”. I haven’t yet tested whether I track sufficient files. Could Embeetle perhaps put a .gitignore in the project directory automatically such that I don’t have to add it manually?
  • I read that one of the arguments behind self-contained projects relies on making good use of a VCS. Does that perhaps involve (git) submodules? I do have a bit of experience with branching and merging, yet not with submodules. A concise write-up about using Git together with Embeetle might be helpful for novice users?

Other

  • I imported an Arduino .ino file. All required files were copied over except for a few header and source files that were in the same directory as the .ino file. The .ino had includes referring to this file. I expected that those files would also be copied over.

  • I’ve the impression that the IDE (currently) only supports self-contained projects. There certainly is something to be said for self contained projects. On the other hand, forcing the user to do so feels somewhat over the top. Therefore I suggest:

    • Allow using directories from outside the project directory by allowing
      • adding search paths
      • usage of symlinks that point from inside the project directory to outside it
    • Ease using both local and nonlocal files using the following functionality
      • Marking the files with colors (for example, a colored tab, or perhaps a shaded background) depending on their type such that it is immediately clear to the user which files they can safely change and which not. Something like
        • inside the project, no dependencies on files outside: green
        • inside the project, yet dependencies on files outside: yellow
        • outside the project: red
      • Easy copying of the library to the project directory. For example, when editing a file outside the project, show a pop-up that offers to copy the library in the project, such that the modified version of the library becomes part of the project. And also add the required stuff so that the modified version of the library gets used.
    • Output human-readable summary of all external files and their paths, and of the library version numbers, such that when a dependency issue occurs, it can be resolved reasonably easily. Perhaps even a hash of each file such that the ‘library once used’ can be compared with the current library. This file can be tracked using the VCS such that one can go back in time.

  • Symlink support, for linking files within the project source directory. I don’t use symlinks often, but when I do, I find them very valuable. (edit Tue 13h15) I tried again and I see that the code compiles, yet that the labels and settings in the Filetree do not appear to work as I expected. I expected that if I force excluded the symlink that the file would not be used (and hence yield a compilation error). However, the compiler still uses the file if the original file (not the simlink) is force included. Perhaps this behaviour is desired, but I did not expect it. Suggestions:

    • Some documentation on how Embeetle handles symlinks
    • Show in the Filetree which files are actually symlinks. Currently the user has to right click, click “Show full path” and then scrutinize the path text. Perhaps put symlink names in another color or font (italic?), or show an icon to the right of the name?
    • Regarding desired behaviour I see three options (not sure how realistic these are):
      1. Bind the exclude/include settings to the real file/directory. Hence, changing the force exclude/include settings of the symlink changes those of the original file and vice versa.
      2. Do not synchronize them and treat them as completely separate files/directories.
      3. A mix of 1 and 2 with some complicated, yet well-documented synchronization rules. Here, the option to travel the symlinks in the other direction, something like right clicking the original file to get a view of all the symlink that point to it (with their exclude/include status etc.), appears useful to me.
        (end edit Tue 13h15)

  • In the Filetree I can see which files are used in the compilation. In my project, there are currently many files marked used (green) while they do not appear to be contribute relevant functionality (such as IPAddress.cpp). Concerning the used files, is there an easy way to see why they are used? I could force exclude them and try to compile, but that feels more complicated than necessary.

  • Editor “search” and “replace” functionality

    • The highlighting could be a bit more striking, for example orange or red instead of grey and dark blue.
    • It offers “replace next occurrence”. However, it appears that this “next occurrence” is not highlighted, so it remains up to the user to know where this “next occurrence” exactly lies and what text will be replaced. I would prefer that the next occurrence is highlighted as well.
    • when doing “search”, would it make sense to always highlight all occurrences by default in a first highlight style and the current one in a more obtrusive style?

  • I use a second display. It appears that the current tiling system does not allow dragging a frame to another display? It would be nice if doing so created a second window.

  • For the Arduino Uno (with the ATMEGA328P-PU dinosaur), the dashboard shows “128 Kb” for “text” and “63 Kb” for “data”. I’m used to some slightly smaller numbers, or was Embeetle able to download a bit of extra RAM?

  • I had an issue after I messed with the file names of the source files. The compiler kept nagging that a particular file was unfindable. Deleting the cache (as described when clicking the “Re-analyze all” button of the Source Analyzer solved the issue. It took some time to find this button. The “Re-analyse all” button, could it perhaps be put just left of the Source Analyzer progress bar such that it is easier to find?

  • A dark theme? Perhaps support for the Geany color scheme format (Themes | Geany)?

  • Would it be possible to have a lexer for Markdown? Something like gedit does.

  • A small install script that ‘registers’ Embeetle, such that is shows up in the program list (Ubuntu), would be nice?

  • (edit Tue 13h15) Documentation: is there a search function somewhere? For example, I wanted to search whether you have any information about Git. (end edit Tue 13h15)

I hope my thoughts about my experience so far are somewhat helpful. I’m well aware that it is written from the standpoint of a novice in embedded development, and it that regard I’m open to feedback.

Best regards,
Ramses

2 Likes
2

embeetle_scaling_300pct
embeetle_scaling_300pct2423×1809 525 KB

2 Likes
3

Hi Ramses,
How nice you wrote down your experience in such detail. Feedback like yours is invaluable to us. Thank you! :+1:

I’ll go over some of your topics. My colleagues will certainly reply on the other topics (like the scaling issues, etc.) that don’t involve my part in the Embeetle codebase.

That is a great idea! Sure, we can do that.

Yes, that would be helpful indeed. Another good idea!

That’s unfortunate. Could you send me that directory with the .ino file, along with the other source files? That could help me to solve the problem quicker. From what I understand, the project you’re working on is not (yet) for a company, so I suppose there is no non disclosure agreement involved. In any case, I’m not going to spread your code against your will.

Agreed. I plead guilty to our current ‘self-contained projects only’ approach. However, recent internal discussions about this have shifted my viewpoint towards a more balanced approach. In the future, you can add extra folders to the Filetree that are outside the project.

The suggestions you make about this topic (allowing external files) are great! Only symlinks make me a bit uncomfortable. A project with symlinks won’t work on Windows. Up until now, all our projects run equally well on Windows and Linux. To be discussed :slight_smile:

You must have been reading our minds. This is a super-nice feature that we’ve been talking about internally from the very start of Embeetle. Unfortunately, we’ve been overwhelmed with so much other things, but it is definitely on our TODO list.

You have a sharp eye! I admit there is a mistake here. The problem lies in the linkerscript. Embeetle parses the linkerscript to extract the sizes of all memory segments (typically FLASH and RAM, but also others like DTCMRAM, ITCMRAM, … if they’re available and declared in the linkerscript). This way, Embeetle doesn’t need to wait for a compilation. It can show you all memory segments (and sections withing those segments) right from the start.

After compilation, Embeetle has an extra valuable file at its disposal: the .elf file. From this file, Embeetle then extracts the actual memory usage per section/segment. This way, the progressbars get filled with the memory usage:

image
image1161×440 44.4 KB

This system works nice. Unfortunately, the Arduino linkerscript is a damn hard nut to crack. This is how it declares the text memory segment:

MEMORY
{
    text (rx): ORIGIN = 0, LENGTH = __TEXT_REGION_LENGTH__
    ...
}

Instead of giving the size of the text memory segment right away, this linkerscript puts a variable there. Higher up in the linkerscript, the variable is defined as follows:

__TEXT_REGION_LENGTH__ = DEFINED(__TEXT_REGION_LENGTH__) ? __TEXT_REGION_LENGTH__ : 128K;

That definition basically says: “If by now the variable __TEXT_REGION_LENGTH__ is not defined yet, define it as 128K”

In other words, the 128K serves as a fallback value, in case it has not been defined anywhere else. But where could it be defined?

Not in the project, unfortunately. None of the source files define this variable. Most probably, it’s defined in the AVR toolchain itself. I did not yet find where exactly, but if you’re interested to dig into this topic deeper, the following StackOverflow question can be a good start:

Long story short: Embeetle IDE right now doesn’t know where to find that damn __TEXT_REGION_LENGTH__ variable, so it turns to the fallback value. The same is true for other variables like __DATA_REGION_LENGTH__, __EEPROM_REGION_LENGTH__, …

I could hardcode this kind of information into Embeetle - but that feels wrong. The better approach is to figure out where these variables are defined. I just can’t find it.

If you know - please tell me :slight_smile:

Yikes - we were going to remove that button from the next version and put it inside a menu. Apparently we’re making a mistake here.

But let’s first consider the case we have here. You renamed a few files, after which the Source Analyzer went bananas :banana:. I can think of the following reasons:

  1. Did you rename the files outside Embeetle (eg. in a file explorer), while Embeetle was running? If yes, then you are to blame. The poor beetle cannot know what you’re doing behind its back.

  2. Did you rename the files outside Embeetle - while it being closed - and then launched the beetle? In that case, you uncovered a bug. The beetle should - at startup - properly process all files in the project without being misled by its own cache.

  3. Did you rename the files inside Embeetle (in the Filetree tab)? In that case, you also uncovered a bug. When renaming a file in Embeetle itself, the Filetree should hand over the namechange to the SA (Source Analyzer).

I’ll look into the matter with my colleague Johan - he is in charge of the Source Analyzer. If you happen to have the project at hand, and you are able to reproduce the issue, that would be super handy.

But back to your original question: the 'Re-analyze all' button is hard to find. I admit we did not make it super-obvious to find, because we considered it quite unnecessary. In an ideal world, the SA doesn’t make caching mistakes and clearing the cache is therefore unecessary. Maybe we’ve been a bit overly confident here. I noticed that even the JetBrains IDEs have a cache clearance button (but somewhat hard to find in the menus).

Do you have a suggestion where to put such a button? Should it be in the top-toolbar, or perhaps a menu entry?

Food for thought - to be continued :slight_smile:

I strongly disagree. They’re not somewhat helpful. They’re very helpful :+1:

@ramses ,
As I final thought, I’d like to mention that we’re currently working hard on PlatformIO integration into Embeetle IDE. This is a huge undertaking, probably several months of work. Because of this, it might look like we’re not doing anything with your suggestions (at least not on the short term). Please don’t let this be a discouragement :wink:

1 Like
4

I’ve sent an MWE to kristof@embeetle.com.

1 Like
5

I opened Embeetle, closed Embeetle (“Embeetle Home” was still open, but that shouldn’t be a problem, is it?), changed “Arduino.h” to “Arduinoo.h” (in “<project_dir>/source/core/arduino/”), restarted Embeetle, tried to compile and it failed to compile (of course). Closed Embeetle, gave the file its original name and reopenend Embeetle. Compilation still fails, and with a seemingly unrelated error (no mention of the file in question):

user_code/sound/sound.cpp.o (symbol from plugin): In function `init_myTone()':
(.text+0x0): multiple definition of `__vector_7'
core/arduino/Tone.cpp.o (symbol from plugin):(.text+0x0): first defined here
/tmp/ccSkQtRW.ltrans0.ltrans.o: In function `setBacklight':
/home/ramses/main_emb/build/../source/libraries/SparkFun_SerLCD_Arduino_Library/src/SerLCD.cpp:585: undefined reference to `map(long, long, long, long, long)'
/home/ramses/main_emb/build/../source/libraries/SparkFun_SerLCD_Arduino_Library/src/SerLCD.cpp:586: undefined reference to `map(long, long, long, long, long)'
/home/ramses/main_emb/build/../source/libraries/SparkFun_SerLCD_Arduino_Library/src/SerLCD.cpp:587: undefined reference to `map(long, long, long, long, long)'
collect2: error: ld returned 1 exit status
make: *** [../config/makefile:517: application.elf] Error 1
make: Leaving directory '/home/ramses/main_emb/build'
/usr/bin/make: exit code '2'

I’ll see whether I can reproduce it with a simpler project.

I would expect it to be left of the progress bar:
refresh_cache

Not sure whether that makes sense :).

1 Like
6

Hi @ramses ,

I received the Blink.zip project per mail. Thank you!

You are an amazing tester! We’ll try this and let you know the results. However, Johan is going on a two-week-holiday today. He’s the one in charge of the Source Analyzer, so it will take some time before the issue can be solved.

7

That’s a pity!

The tool avr-objdump can reveal the fractions… yet not the available sizes directly. I guess you experienced the same issue. Perhaps it is possible to calculate the available sizes from this information though (with a bit of rounding)? While doing so doesn’t feel right, it might be preferable to hardcoding sizes…

user@computer:~/myproject/main_emb/build$ avr-objdump -Pmem-usage application.elf

application.elf:     file format elf32-avr
AVR Memory Usage
----------------
Device: atmega328p

Program:   29426 bytes (89.8% Full)
(.text + .data + .bootloader)

Data:       1586 bytes (77.4% Full)
(.data + .bss + .noinit)

Something else: I’m not sure how to interpret the difference between avr-objdump’s “Data” and Embeetle’s “data”.

embeetle_dashboard

1 Like
8

Hi @ramses ,
Thank you for these interesting suggestions.

I’m currently in the midst of a refactoring effort (related to the upcoming PlatformIO support). But I will get back to this post later, for sure :slight_smile:

Kind regards,
Kristof

9

Looking forward to it!

Regarding the apparent issue with the Source Analyzer: last month I didn’t have time to create an MWE. Are you still interested in an MWE?

10

Hi @ramses ,
I browsed through the previous messages, and we’ve been talking about many subjects. The MWE you had in mind, what precisely would it be about? I just need a memory refreshment :slight_smile:

About the Source Analyzer clear cache functionality: we’ve taken your suggestions to heart and implemented the following:

image
image1920×1030 142 KB

In the coming release, you’ll notice that the Source Analyzer engine now has it’s own tab where you can view and handle everything related to this engine.

If you close the tab, it can always be reopened with the button in the bottom-right corner.

11

Looks great!

Concerning the MWE, it relates to the following unexpected behaviour:

Perhaps you already made an MWE yourself, or found the bug? If not, please let me know and I’ll send you an MWE.

12

Hi @ramses ,
Unfortunately we didn’t have time yet to make our own MWE for this bug. If you could do that, it would be super helpful!

Kind regards,
Kristof