Bootlin at Embedded Open Source Summit North America, Apr 16-18: 4 talks, 11 engineers

Embedded Open Source SummitThe Embedded Open Source Summit will take place on April 16-18 in Seattle (Washington, US), with a wide range of events, including the Embedded Linux Conference which is of particular interest to Bootlin considering our core expertise.

As usual, a significant fraction of the Bootlin team will attend the event, with no less than 11 engineers from our team going to Seattle: Alexandre Belloni, Bastien Curutchet, Grégory Clement, Jérémie Dautheribes, João Marcos Costa, Köry Maincent, Louis Chauvet, Luca Ceresoli, Maxime Chevallier, Thomas Petazzoni and an additional engineer who will join us at the beginning of April. At Bootlin, all engineers, regardless of their seniority level, are offered the possibility of attending up to 2 conferences per year, to expand their knowledge on embedded Linux topics, to create connections with the community and therefore provide better services to our customers.

In addition to participating, we will also be delivering no less than 4 talks at the conference:

  • In the Kernel Trenches: Mastering Ethernet Drivers on Linux – Maxime Chevallier, Bootlin
    An Ethernet Controller driver in linux is a complex piece of software, with many entrypoints and callbacks, even for a basic Ethernet Controller we can find in Embedded Systems. In this talk, we’ll take a tour of what’s inside a typical Ethernet driver. We’ll start with the base of what every network driver provides, even non-Ethernet ones, a net_device. We will discuss the various required steps to get a basic driver working, ranging from simple initialization, queue setup, DMA mappings, before diving into more advances topics such as the NAPI loop implementation, memory-management through the page-pool API and finally XDP, which is now getting implemented in more and more drivers. We’ll continue the tour through the ethernet-specific bits, such as the ethtool operations, and phylink support to interact with a potential Ethernet PHY, and discuss a little bit about the specifics of embedded Ethernet Controllers, where low-level configuration for a PCS or some Serdes lanes might be required. Finally, we’ll discuss about testing and debugging tools that can be useful to help you in your driver development, but also in debugging and optimizing existing drivers.
  • Introduction to DAPM, Linux Power Management for Embedded Audio Devices – Luca Ceresoli, Bootlin
    DAPM (Digital Audio Power Management) is the part of ASoC (Alsa for System-on-Chips) that dynamically turns on and off power to various parts of the audio hardware in embedded Linux systems. DAPM is very useful, but not well known and documented. After introducing the motivations for DAPM and how it models the hardware, Luca will present some user space tools to understand how DAPM works at runtime. He will then show what DAPM practically means for developers, with code examples of how to add DAPM support to device drivers. Finally he will mention some improvements to the DAPM tooling and documentation.
  • Unlocking the Potential of Suspend to RAM Using Linux in a Multi-Core, Multi-Firmware Automotive SoC – Grégory Clement, Bootlin
    Fast booting in modern, complex System-on-Chips (SoCs) demands innovative solutions. This talk delves into the intricacies of implementing Suspend to RAM (S2R) on a sophisticated SoC featuring diverse CPU cores—high-performance A72, real-time R5, and low-power M3—as well as multiple firmware components, including open-source ones like TF-A, U-Boot, and OP-TEE. We begin with a hardware overview of the TI DRA821 SoC from the Jacinto family, emphasizing the open-source firmware utilized in the boot process. The focus then shifts to presenting a practical approach for integrating S2R into this multifaceted platform. Notably, our implementation poses a unique challenge: the SoC is entirely powered off during S2R, adding an extra layer of complexity. Join us as we share insights into the hurdles encountered and resolved during the implementation, along with a discussion on the outstanding challenges for which we’ve devised viable solutions. This session offers a comprehensive exploration into the software, including bootloaders and the Linux kernel, as well as the hardware aspects, showcasing a real-world approach to bringing Suspend to RAM to life on a cutting-edge automotive SoC.
  • Adding Support for Power Over Ethernet (PoE or IEEE Clause 33) to Linux Network Stack – Köry Maincent, Bootlin
    Power over Ethernet (PoE) is a technology that combines electrical power and data transmission over a single Ethernet cable. It eliminates the need for separate power sources, simplifying installations for devices like IP cameras, and VoIP phones. In this presentation, we’ll initially delve into Power over Ethernet (PoE), It debuted in IEEE Clause 33 without explicit reference to the PoE nomenclature. We will look at what currently exists in the Linux kernel and user space to support PoE. We’ll continue our discussion with some details of the Linux implementation currently in development and the PSE core changes brought by this new support. The PSE framework core and bindings happen to need modification as it was not prepared for the PoE specificities. In parallel to these extensions of the PSE framework, we developed Linux kernel drivers for two distinct PoE controllers: the Microchip PD692x0 and the Texas Instruments TPS23881. Finally, we will look into the mainline status, the things that still need to be merged and the future features that need development. This PoE Linux support is sponsored and funded by DENT Project.

See the full schedule of talks.

The Embedded Open Source Summit isn’t just about the presentations—it’s about forging connections, sharing ideas, and exploring opportunities. Join us for insightful conversations with the Bootlin team, where you can discuss your projects, ambitions, and maybe even discover new career pathways.

And mark your calendars because following the summit, our CEO Thomas Petazzoni will be in the Bay Area until April 24, ready to delve deeper into your embedded Linux needs.

So, whether you’re a seasoned veteran or a curious newcomer, don’t miss out on the Embedded Open Source Summit 2024. We’ll see you there!

Back from the Embedded Linux Conference Europe 2023

From June 28 to June 30, Bootlin participated to the Embedded Linux Conference Europe, which was organized as part of the new and larger Embedded Open Source Summit.

In addition, the day before the conference, on June 27, our team had a great team building event, spending the day visiting Prague, having lunch in a traditional restaurant, enjoying a boat tour on the Vltava river, and an evening with a traditional dinner and folklore music. As our team is distributed, conferences are a great opportunity to meet each other and Prague was for several members of our team their first in-person meeting.

With 14 Bootlin engineers at the conference, almost our entire engineering team participated. Indeed, we have a policy at Bootlin to offer to all our engineers, regardless of their seniority level, the chance of attending 2 technical conferences each year.

Continue reading “Back from the Embedded Linux Conference Europe 2023”

Embedded Linux Conference Europe 2022: a selection of talks by Bootlin engineers

Almost the entire engineering team of Bootlin attended the Embedded Linux Conference Europe 2022 in Dublin mid-september, an important event for Bootlin as it helps everyone in the team stay up to date with the latest developments in the Embedded Linux ecosystem, and connect with members of the community.

All the slides and videos are available at https://elinux.org/ELC_Europe_2022_Presentations, which is one of the great things about the Embedded Linux Conference.

After such conferences, we have a tradition at Bootlin: share with our readers a selection of talks that we found interesting. Several members of our engineering team were asked to select one of their favorite talks, and highlight it with a short summary.

Continue reading “Embedded Linux Conference Europe 2022: a selection of talks by Bootlin engineers”

Bootlin at the Embedded Linux Conference North America 2022

Bootlin CEO Thomas Petazzoni and COO Alexandre Belloni will both be attending the next Embedded Linux Conference North America, on June 21-24 in Austin, Texas.

In addition, both Thomas and Alexandre will be speaking at the event:

  • Thomas Petazzoni will give a talk Buildroot: what’s new?, providing an update on the improvements and new features in the Buildroot build system that have been integrated over the past two years
  • Alexandre Belloni will give a talk Yocto Project Autobuilders and the SWAT Team, during which he will explain what’s happening behind in the scenes in the Yocto Project to review and validate contributions before they are integrated.

Thomas and Alexandre will also naturally be available during the event to discuss business or career opportunities, so do not hesitate to get in touch if you’re interested.

Finally, prior to the event, Thomas Petazzoni will be in the Bay Area on June 13-15, also available for meetings or discussions.

Embedded Linux Conference 2021 schedule published, 4 talks from Bootlin

The schedule for the Embedded Linux Conference 2021 has been published and features 4 talks proposed by Bootlin !

This year, the ELC will take place in Seatle but will be organised as a hybrid virtual/physical event  due to the pandemic.  As usual the ELC will have a really interesting schedule with 46 talks covering a wide range of topics: build system, kernel graphics, boot process, security, etc.

See below the details of Bootlin talks that will be presented as virtual talks.

Advanced Camera Support on Allwinner SoCs with Mainline Linux – Paul Kocialkowski, Bootlin

Capturing pixels with a camera involves a number of steps, from the ADC reading the photosites in the image sensor to the final pixel values that are ready for encode/display, with various processing and transmission taking place along the way. While simple cases put most of the heavy lifting on the image sensor’s side (through its embedded processor) and use a simple parallel bus for transmission, advanced cases require more work to be done outside of the sensor. In addition, modern high-speed transmission buses also bring-in more complexity. This talk will present how support for such an advanced use case was integrated into the mainline Linux kernel, using the Media and V4L2 APIs. It involves supporting a sensor using the raw Bayer RGB format, transmission over the MIPI CSI-2 bus as well as support for the Image Signal Processor (ISP) found on Allwinner platforms. A specific focus will be set on this ISP, with details about the features it implements as well as the internal and userspace APIs that are used to support it. The integration between all of the involved components will also be highlighted.

Talk given by Paul Kocialkowski, at 4:50 PM PDT on September 27, 2021. See this talk in the schedule.

Embedded Linux Nuggets found in Buildroot Package Eldorado

To this date, Buildroot supports more than 2,500 packages, selected for the ability to run them on embedded Linux systems. We’ve gone exploring this Eldorado, and came back with multiple nuggets of all shapes and colors. Join this playful presentation and as if you were still a new comer to the embedded Linux community, discover lesser known tools and resources that can add to the functionality of your systems or make your life as a developer easier and more fun. Whenever possible, each resource will be shown through a quick demonstration or video capture. During this talk, I’ll also open an Etherpad for all participants to share their favorite solutions with the rest of the audience, especially the ones that deserve to be better known, and could be worth supporting in Buildroot too. We will close the session by an open review and discussion based on the nuggets shared by the audience.

Michael Opdenacker

Talk given by Michael Opdenacker, at 12:00 PM PDT on September 28, 2021. See this talk in the schedule.

I3C in Tomorrow’s Design

I3C is the new bus specification by the MIPI Alliance. While being compatible with I2C devices, this bus brings a colorful set of new features such as dynamic address assignment, in-band interrupts, hot-join, master handover and many others. It was improved once again recently with the 1.1 version of the specification which brought timer based sampling synchronization and targeted reset. All this make the I3C bus a good candidate for a number of new situations compared to its I2C cousin. It is then more and more being included in new hardware designs. With this talk we would like to propose a reminder of the various components and concepts of this relatively new bus. We will then detail how it is implemented in the Linux kernel with a short guided tour in the I3C core. Since the previous talk on I3C in 2018 by Boris Brezillon, I3C has now become a reality and starts to become available in real hardware designs. This talk will recap the basics of I3C as well as add details of the 1.1 specification and improvements in the Linux support.

Miquèl Raynal

Talk given by Miquèl Raynal, at 4:00 PM PDT on September 28, 2021. See this talk in the schedule.

OP-TEE: When Linux Loses Control

OP-TEE is an open-source Trusted Execution Environment designed to be executed in a secure context as a companion to a non secure Linux system. But what happens to the peripherals control since OP-TEE can forbid the non-secure OS to access them ? When running with a TEE, Linux isn’t in charge anymore of some critical peripherals and relies on the TEE to access and configure them. There are multiple protocols and methods to access these peripherals that are supported by Linux (SCMI, PSCI, SMC). Supporting them for a SoC requires understanding the various interactions between the systems and how to modify them to fit that new control scheme. Additionally, the configuration must be passed from OP-TEE to Linux to allow a seamless integration. This talk will cover the boot process to start a secure system and the modifications needed to run Linux when OP-TEE is in charge of some peripherals. The work that has been done for a specific SoC will be described to have a tangible real-world use-case.

Clément Léger

Talk given by Clément Léger, at 12:00 PM PDT on September 29, 2021. See this talk in the schedule.

Bootlin at the Embedded Linux Conference 2020

Bootlin has been a participant at the Embedded Linux Conference for many years, and despite the special conditions this year, we will again be participating to this online event, from June 29 to July 1.

More specifically:

  • Bootlin engineer Alexandre Belloni and Bootlin’s audio expert will give a talk ASoC: supporting Audio on an Embedded Board, which presents how audio complex in embedded devices are typically supported by the Linux kernel ALSA System-on-Chip framework. This talk takes place on June 29 at 2:05 PM UTC-5.
  • Bootlin engineer and CTO Thomas Petazzoni will give his usual Buildroot: what’s new ? talk, giving an update on the latest developments and improvements of the Buildroot project. This talk takes place on July 1 at 11:15 AM UTC-5.
  • The vast majority of the Bootlin engineering team will be attending many of talks proposed during the event. Bootlin has been offering to all its engineers the participation to two conferences a year: with the Embedded Linux Conference going virtual, we’ve simply allowed all our engineers to participate, with no restriction. This is part of Bootlin’s policy to ensure our engineers stay as up to date as possible with embedded Linux technologies.
  • Bootlin CTO Thomas Petazzoni was once again part of the program committee for this edition of the Embedded Linux Conference, as part of this committee he reviewed and selected the different talks that were submitted.

We are interested in seeing how this virtual version of the Embedded Linux Conference will compare to the traditional physical event. For many old timers to these conferences, the most useful part of a conference is the hallway track and all the side discussions, meetings and dinners with members of the embedded Linux community, and a virtual version makes such interactions more challenging.

In any case, we hope you’ll enjoy the conference! Don’t hesitate to join us in the Q&A session after our talks, or on the 2-track-embedded-linux room of the Slack workspace set up for the event by the Linux Foundation.

Back from ELC North-America: selection of talks from the Bootlin team

As discussed in our previous blog post, Bootlin had again a strong presence at the Embedded Linux Conference North-America, with 8 attendees, 5 talks, one BoF and two E-ALE tutorial sessions.

In this blog post, we would like to highlight a number of talks from the conference that we found interesting. Each Bootlin engineer who attended the conference has selected one talk, and gives his/her feedback about this talk.

Device Tree BoF – Frank Rowand

Talk selected by Michael Opdenacker

The Device Tree BoF (Birds of a Feather session, which means an informal session about a technical topic, allowing participants to openly share questions and information) has been part of Embedded Linux Conferences for at least 2 or 3 years. For me, it has always been a good source of updates about the topic.

Frank started by sharing details about the Device Tree Workshop held in October in Prague, a one day meet-up and workshop for Device Tree contributors (like Maxime Ripard and Thomas Petazzoni from Bootlin who were invited), to address issues and plan for the next months. Slides and notes can be found on elinux.org.

Frank then went on by mentioning utilities, such as:

  • scripts/dtc/dt_to_config. It is not very new in the mainline kernel, but useful to generate a kernel configuration suitable for the devices present on your platform, in case you didn’t know this tool exists.
  • There’s an upcoming patch adding options to dtc (--annotate --full) to keep track of the line numbers in the device tree sources. This helps to locate in which DT source file a given property value comes from. The patch was idle for some time but Julia Lawall volunteered to take care of it. Thanks to her updates, this feature should be accepted in mainline soon.
  • The device tree compiler in mainline has also been augmented with further build checks. You can now use them by adding W=1 to make dtb. Here is an example for the Beagle Bone Black dtb:
make W=1 am335x-boneblack.dtb
  CHK     scripts/mod/devicetable-offsets.h
  DTC     arch/arm/boot/dts/am335x-boneblack.dtb
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/i2c@44e0b000/tda19988 has a reg or ranges property, but no unit name
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/i2c@44e0b000/tda19988/ports/port@0 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/i2c@44e0b000/tda19988/ports/port@0/endpoint@0 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/ethernet@4a100000/slave@4a100200 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/ethernet@4a100000/slave@4a100300 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/lcdc@4830e000/port/endpoint@0 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/prcm@200000/clocks missing or empty reg/ranges property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/prcm@200000/clockdomains missing or empty reg/ranges property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/scm@210000/scm_conf@0/clocks missing or empty reg/ranges property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/scm@210000/clockdomains missing or empty reg/ranges property

As far as I am concerned, the most interesting news remained the one that since Linux 4.15, device tree overlays are now easier to code. You no longer have to define weird “fragment” elements. You can now directly write normal nodes and use labels. The syntax is now exactly the same as for regular device tree sources!

For more details, grab the slides and if you event want to follow the discussions that happened that day, watch the video.

Tutorial: Introduction to Reverse Engineering – Mike Anderson

Talk selected by Quentin Schulz

Mike presented in an unusual 2-hour-long slot the different techniques to reverse-engineer things and the different reasons why you’d do so. After a mandatory disclaimer that reverse engineering may be illegal in some regions of the world, he introduced the different tools that anyone aspiring to reverse engineer should possess: from the obvious logic analyzer, multimeter, screwdrivers to the surprising heat gun. He then gave the first and very important step of the reverse engineering process: gathering information about the product by looking for patents, the FCC registration, manufacturer as well as carefully opening its case to examine the different components (maybe with the help of a microscope).

Later, Mike gave the multiple ways to retrieve the firmwares from the product, from the soldering of a JTAG interface to the downloading from the official website. He then offered some tools that can be used to dive into binaries and start the guessing game, and he finished his talk with an example of a reverse engineering of a protocol which required a lot of guessing and social reverse engineering.

Mike’s talk was pleasant to attend because of the high-level presentation of how to do reverse engineering while giving a quick real-life example.

For more details, watch the video and grab the slides.

Graphics Performance Analysis with FrameRetrace: A Responsive UI for ApiTrace – Mark Janes, Intel

Talk selected by Boris Brezillon

I first heard of FrameRetrace when Eric Anholt asked us to add support for the VC4 GPU to this tool, and my experience with it had been rather frustrating in that I was mainly struggling to make it work on a not yet supported architecture instead of being a simple user. So, when I saw that Mark was giving a talk on FrameRetrace usefulness and how to use it, I figured I couldn’t miss it. Well, those who looked carefuly at the schedule know I couldn’t attend it because I was giving my talk at the same time, but I did see it at FOSDEM a month before, and I’m pretty sure not much has changed since then.

Mark first described the GPU debugging/perf-anlysis tools ecosystem, saying that each GPU vendor has its own proprietary tools which most of the time are only supported on Windows. FrameRetrace is an attempt at providing a tool that exposes similar features while being open-source, cross-platform, and easily extensible to new hardware. This project is actually based on an exisiting project called ApiTrace, which it uses to capture OpenGL traces. The new feature that is interesting in FrameRetrace is that you can select the frame you want to replay, get all the hardware perf counters exposed by the GPU for this specific rendering job in order to figure out what is going wrong and then play with the OpenGL code to see how you can make things better and replay the rendering job with your local modification to see if it actually solves the problem.

I must admit I was really impressed by the demo, and now I understand why Eric (and others in the community) would like to have their GPU properly supported in FrameRetrace. It really looks like the kind of tool you don’t know you need until you’ve tested it, but once you do, you can’t do without it.

For more details, watch the video and grab the slides.

The Salmon Diet: Up-Streaming Drivers as a Form of Optimization – Gilad Ben-Yossef

Talk selected by Miquèl Raynal

Gilad was hired about a year ago to become the maintainer of the ARM® TrustZone® CryptoCell® device driver. Until now this driver was out of tree until it has been decided to upstream it. Here starts Gilad’s story.

It appeared that the right way to make it upstream was to go through the staging tree and the whole process around it. It was the first time for him to do it that way, that is why he felt it was interesting to share his experience.

At the beginning of his talk, he recalls that the driver was actually working, people already relied on it. Plus, it was released under the GPL. While all of this could make you think it was clean enough, Gilad realized that people who wrote it actually did not think about upstreaming and almost every patch to clean that driver removed more lines than it added, shrinking step by step the driver until 30% of the lines were removed!

Of course, removing the existing hardware abstraction macros was something to do, as well as running and correcting the whole checkpatch.pl output, but there are plenty of other good habits that one can adapt to his own situation, explained and well illustrated all along this talk.

For more details, watch the video and grab the slides.

Measuring and Summarizing Latencies using the Trace Event Subsystem – Tom Zanussi

Talk selected by Maxime Chevallier

Having some experience dealing with RT topics on Linux, I was looking forward to seeing Tom’s talk about these tracing features.

He gave really good examples on how to use the existing ‘latency histogram’ traces to get a cyclictest-like metric of wakeup latencies by measuring the time between sched_waking and sched_switch, and explaining how this could be re-used for other measurements such as network latencies.

What he presented was more than just having a trace in a buffer when a function is called. The tracing subsystem allows the use of handlers to perform actions when an event occurs. As an example, he demonstrated how to use the onmax handler to accumulate the maximum wakeup latency observed, each time saving crucial pieces of information on the execution context.

He then went on to describe the next-level features that are being merged, namely function events by Steven Rostedt, and Inter events by Tom himself. They allow the user to use any of the kernel functions as tracepoints, and build complex events and traces to pinpoint really specific sequences.

I recommend to read this LWN article on inter-event tracing, and of course have a look at Tom’s talk and slides.

Steering Xenomai into the Real-Time Linux Future – Jan Kiszka

Talk selected by Thomas Petazzoni

In this talk, long-term Xenomai developer and Siemens engineer Jan Kiszka gave a very interesting status of the Xenomai project and its roadmap. He started by refreshing the audience about what Xenomai is: an RTOS-to-Linux portability framework. It comes in two flavors: a co-kernel extension for a patched Linux kernel, and as libraries running for native Linux (including PREEMPT_RT). He then went on to compare the respective advantages and drawbacks of the two flavors, citing accurate modeling of legacy RTOS behavior and strong separation of real-time vs. non real-time code as the key advantages of the co-kernel approach.

Jan then summarized the history of Xenomai, from the early days as a sub-project of RTAI to the current status of Xenomai 3.0, released in 2015 after more than 5 years of development. However, even though Xenomai is widely used in the industry, its development relies on just a few individuals. Siemens is a heavy user of Xenomai, and in 2017, they started a discussion: should they migrate away from Xenomai or invest into it. Siemens made the decision to invest in the project. The same year, Xenomai main developer Philippe Gerum published an e-mail RTnet, Analogy and the elephant in the room also calling for help to maintain some parts of Xenomai.

Following those discussions, some changes were decided in the Xenomai project: Philippe Gerum will step back from the project lead, and Jan Kiszka will take over his role.

Regarding the I-Pipe kernel patch (which allows to support the co-kernel approach), the Xenomai project will discontinue a number of architectures (nios2, SH, Blackfin, PPC64, ARM < v7) and will only maintain patches for the latest Linux kernel LTS, in order to reduce the maintenance effort. Jan announced that Xenomai 3.0.7 is soon to be released, that Xenomai 3.1 will introduce ARM64 support, and that Xenomai 2 is unmaintained and therefore users should migrate to Xenomai 3. He also gave a status on the driver stacks, citing that RTnet needs more love, and that Analogy is orphaned and needs a new maintainer. Towards the end of his talk, Jan then started discussing the more distant future of Xenomai. The future version of Xenomai has the goal of improving the integration of the co-kernel approach, to simplify maintenance and possibly provide a chance to be upstreamed in Linux. This new approach will be split in two elements, called Dovetail (interrupt routing, co-kernel hooks) and Steely (co-kernel implementation). He made it clear that this is currently not production-ready at all. He noted that this new implementation allows a significant reduction of the code base, about 50% smaller than the current implementation. The code is already available in two Git repositories: Dovetail and Steely.

All in all, Jan’s talk was a very interesting one, providing a good coverage of Xenomai’s status and future. The video of his talk is definitely worth watching, and the slides are also available.

An Introduction to Asymmetric Multiprocessing: When this Architecture can be a Game Changer and How to Survive It – Nicola La Gloria & Laura Nao

Talk selected by Mylène Josserand

In this talk, Nicola La Gloria and Laura Nao from Kynetics presented how to handle communication between a micro-controller running bare metal code and a CPU with a full OS (such as GNU/Linux or Android).

They showed the different approaches for communication (supervised or not supervised: i.e. CPU and MCU can communicate using an hypervisor or directly) and presented the Inter-Processing Communication.

After this introduction, Laura told us about their use-case, running on an NXP i.MX7, which comprises a Cortex-M4 micro-controller and a Cortex-A7 processor: the MCU retrieves data from a sensor, which will be displayed by the CPU.
She gave feedback on how they implemented this communication and the different mechanisms used (Message Unit, RPMsg, RDC, kexec/kdump, etc). The explanation of the different mechanisms was really interesting, and particularly relevant for those who had never heard about them.

They did a short tutorial and gave some tips that would definitely be appreciated by people who start this kind of project. And finally, they did a demonstration of all the work they have done.

So if you are interested in the subject or even only for your general knowledge, have a look at their talk (video and slides).

System-in-Package Technology: Making it Easier to Build Your Own Linux Computer – Erik Welsh & Jason Kridner

Talk selected by Alexandre Belloni

Eric Welsh started to talk about how software influences hardware design and why open source hardware matters. He then presented the System-in-Package technology and in particular the Octavo OSD3358. It includes a TI AAM335x SoC, DDR3 SDRAM, a PMIC and all the related power circuitry, components which are always required. This allows the hardware engineers to concentrate on the added value of the final product.

Great pictures and videos of the SiP internals and its manufacturing process were shown.

Jason then came on stage to present the OSD3358 integration on the PocketBeagle.

Eric finally explained how easy it is to assemble the OSD3358 on a PCB, even by hand with a video to prove it. He finally concluded by summing up the benefits of using a SiP: easy bring-up, lower cost of PCB, easy manufacturing and migration from SBC prototyping to custom PCB.

It was quite enlightening for software engineers as it showed the hardware internals with some great details.
Have a look at the video and slides.

In conclusion, this was again a really nice opportunity to share and acquire knowledge from other engineers deeply involved in the Open Source community, as well as meeting people that we sometimes know only by their name on a mailing list. Next year this event will happen in Monterey Bay, California (March 19 – 21, 2019). See you there!

Bootlin at the Embedded Linux Conference 2018

Like every year for more than 10 years, Bootlin engineers will participate to the next Embedded Linux Conference, which takes place in Portland on March 12-14. Of course, it will be our first ELC with our new company name! In total, eight engineers from Bootlin will participate to the event. Maxime Chevallier, who joined Bootlin last Monday, will be attending the conference, his first one with a Bootlin hat (but Maxime has already been a speaker at the last Embedded Linux Conference Europe).

Embedded Linux Conference 2018

We will also be giving a number of talks, tutorials or moderating Bird of a Feather sessions:

We’re really happy to again meet the embedded Linux open-source community at this event! It is worth mentioning that following this event, Bootlin CTO Thomas Petazzoni will be in the Silicon Valley on March 15-16, available for business meetings: do not hesitate to contact us if you’re interested.

Bootlin at the Embedded Linux Conference 2017

Last month, five engineers from Bootlin participated to the Embedded Linux Conference in Portlan, Oregon. It was once again a great conference to learn new things about embedded Linux and the Linux kernel, and to meet developers from the open-source community.

Bootlin team at work at ELC 2017, with Maxime Ripard, Antoine Ténart, Mylène Josserand and Quentin Schulz

Bootlin talks

Bootlin CEO Michael Opdenacker gave a talk on Embedded Linux Size Reduction techniques, for which the slides and video are available:

Bootlin engineer Quentin Schulz gave a talk on Power Management Integrated Circuits: Keep the Power in Your Hands, the slides and video are also available:

Bootlin selection of talks

Of course, the slides from many other talks are progressively being uploaded, and the Linux Foundation published the video recordings in a record time: they are all already available on Youtube!

Below, each Bootlin engineer who attended the conference has selected one talk he/she has liked, and gives a quick summary of the talk, hopefully to encourage you watch the corresponding video recording.

Using SWupdate to Upgrade your system, Gabriel Huau

Talk selected by Mylène Josserand.

Gabriel Huau from Witekio did a great talk at ELC about SWUpdate, a tool created by Denx to update your system. The talk gives an overview of this tool, how it is working and how to use it. Updating your system is very important for embedded devices to fix some bugs/security fixes or add new features, but in an industrial context, it is sometimes difficult to perform an update: devices not easily accessible, large number of devices and variants, etc. A tool that can update the system automatically or even Over The Air (OTA) can be very useful. SWUpdate is one of them.

SWUpdate allows to update different parts of an embedded system such as the bootloader, the kernel, the device tree, the root file system and also the application data.
It handles different image types: UBI, MTD, Raw, Custom LUA, u-boot environment and even your custom one. It includes a notifier to be able to receive feedback about the updating process which can be useful in some cases. SWUPdate uses different local and OTA/remote interfaces such as USB, SD card, HTTP, etc. It is based on a simple update image format to indicate which images must be updated.

Many customizations can be done with this tool as it is provided with the classic menuconfig configuration tool. One great thing is that this tool is supported by Yocto Project and Buildroot so it can be easily tested.

Do not hesitate to have a look to his slides, the video of his talk or directly test SWUpdate!

GCC/Clang Optimizations for embedded Linux, Khem Raj

Talk selected by Michael Opdenacker.

Khem Raj from Comcast is a frequent speaker at the Embedded Linux Conference, and one of his strong fields of expertise is C compilers, especially LLVM/Clang and Gcc. His talk at this conference can interest anyone developing code in the C language, to know about optimizations that the compilers can use to improve the performance or size of generated binaries. See the video and slides.

Khem Raj slide about compiler optimization optionsOne noteworthy optimization is Clang’s -Oz (Gcc doesn’t have it), which goes even beyond -Os, by disabling loop vectorization. Note that Clang already performs better than Gcc in terms of code size (according to our own measurements). On the topic of bundle optimizations such as -O2 or -Os, Khem added that specific optimizations can be disabled in both compilers through the -fno- command line option preceding the name of a given optimization. The name of each optimization in a given bundle can be found through the -fverbose-asm command line option.

Another new optimization option is -Og, which is different from the traditional -g option. It still allows to produce code that can be debugged, but in a way that provides a reasonable level of runtime performance.

On the performance side, he also recalled the Feedback-Directed Optimizations (FDO), already covered in earlier Embedded Linux Conferences, which can be used to feed the compiler with profiler statistics about code branches. The compiler can use such information to optimize branches which are the more frequent at run-time.

Khem’s last advise was not to optimize too early, and first make sure you do your debugging and profiling work first, as heavily optimized code can be very difficult to debug. Therefore, optimizations are for well-proven code only.

Note that Khem also gave a similar talk in the IoT track for the conference, which was more focused on bare-metal code optimization code and portability: “Optimizing C for microcontrollers” (slides, video).

A Journey through Upstream Atomic KMS to Achieve DP Compliance, Manasi Navare

Talk selected by Quentin Schulz.

This talk was about the journey of a new comer in the mainline kernel community to fix the DisplayPort support in Intel i915 DRM driver. It first presented what happens from the moment we plug a cable in a monitor until we actually see an image, then where the driver is in the kernel: in the DRM subsystem, between the hardware (an Intel Integrated Graphics device) and the libdrm userspace library on which userspace applications such as the X server rely.

The bug to fix was that case when the driver would fail after updating to the requested resolution for a DP link. The other existing drivers usually fail before updating the resolution, so Manasi had to add a way to tell the userspace the DP link failed after updating the resolution. Such addition would be useless without applications using this new information, therefore she had to work with their developers to make the applications behave correctly when reading this important information.

With a working set of patches, she thought she had done most of the work with only the upstreaming left and didn’t know it would take her many versions to make it upstream. She wished to have sent a first version of a driver for review earlier to save time over the whole development plus upstreaming process. She also had to make sure the changes in the userspace applications will be ready when the driver will be upstreamed.

The talk was a good introduction on how DisplayPort works and an excellent example on why involving the community even in early stages of the development process may be a good idea to quicken the overall driver development process by avoiding complete rewriting of some code parts when upstreaming is under way.

See also the video and slides of the talk.

Timekeeping in the Linux Kernel, Stephen Boyd

Talk selected by Maxime Ripard.

Stephen did a great talk about one thing that is often overlooked, and really shouldn’t: Timekeeping. He started by explaining the various timekeeping mechanisms, both in hardware and how Linux use them. That meant covering the counters, timers, the tick, the jiffies, and the various POSIX clocks, and detailing the various frameworks using them. He also explained the various bugs that might be encountered when having a too naive counter implementation for example, or using the wrong POSIX clock from an application.

See also the video and slides of the talk.

Android Things, Karim Yaghmour

Talk selected by Antoine Ténart

Karim did a very good introduction to Android Things. His talk was a great overview of what this new OS from Google targeting embedded devices is, and where it comes from. He started by showing the history of Android, and he explained what this system brought to the embedded market. He then switched to the birth of Android Things; a reboot of Google’s strategy for connected devices. He finally gave an in depth explanation of the internals of this new OS, by comparing Android Things and Android, with lots of examples and demos.

Android Things replaces Brillo / Weave, and unlike its predecessor is built reusing available tools and services. It’s in fact a lightweight version of Android, with many services removed and a few additions like the PIO API to drive GPIO, I2C, PWM or UART controllers. A few services were replaced as well, most notably the launcher. The result is a not so big, but not so small, system that can run on headless devices to control various sensors; with an Android API for application developers.

See also the video and slides of the talk.

Slides from the Embedded Linux Conference

Two weeks ago, the entire Bootlin engineering team (9 persons) attended the Embedded Linux Conference in San Diego. We had some really good time there, with lots of interesting talks and useful meetings and discussions.

Tim Bird opening the conferenceDiscussion between Linus Torvalds and Dirk Hohndel

In addition to attending the event, we also participated by giving 5 different talks on various topics, for which we are publishing the slides:

Boris Brezillon, the new NAND Linux subsystem maintainer, presented on Modernizing the NAND framework: The big picture.

Boris Brezillon's talk on the NAND subsystem

Antoine Ténart presented on Using DT overlays to support the C.H.I.P’s capes.

Antoine Tenart's talk on using DT overlays for the CHIP

Maxime Ripard, maintainer of the Allwinner platform support in Linux, presented on Bringing display and 3D to the C.H.I.P computer.

Maxime Ripard's talk on display and 3D for the CHIP

Alexandre Belloni and Thomas Petazzoni presented Buildroot vs. OpenEmbedded/Yocto Project: a four hands discussion.

Belloni and Petazzoni's talk on OpenEmbedded vs. Buildroot

Thomas Petazzoni presented GNU Autotools: a tutorial.

Petazzoni's tutorial on the autotools

All the other slides from the conference are available from the event page as well as from eLinux.org Wiki. All conferences have been recorded, and the videos will hopefully be posted soon by the Linux Foundation.