Free Electrons at the X.org Developer Conference 2016

The X.org Foundation hosts every year around september the X.org Developer Conference, which, unlike its name states, is not limited to X.org developers, but gathers all the Linux graphics stack developers, including X.org, Mesa, wayland, and other graphics stacks like ChromeOS, Android or Tizen.

This year’s edition was held last week in the University of Haaga-Helia, in Helsinki. At Free Electrons, we’ve had more and more developments on the graphic stack recently through the work we do on Atmel and NextThing Co’s C.H.I.P., so it made sense to attend.

XDC 2016 conference

There’s been a lot of very interesting talks during those three days, as can be seen in the conference schedule, but we especially liked a few of those:

DRM HWComposer – SlidesVideo

The opening talk was made by two Google engineers from the ChromeOS team, Sean Paul and Zach Reizner. They talked about the work they did on the drm_hwcomposer they wrote for the Pixel C, on Android.

The hwcomposer is one of the HAL in Android that interfaces between Surface Flinger, the display manager, and the underlying display driver. It aims at providing hardware composition features, so that Android can leverage the capacities of the display engine to perform compositions (through planes and sprites), without having to use the CPU or the GPU to do this work.

The drm_hwcomposer started out as yet another hwcomposer library implementation for the tegra-drm driver in Linux. While they implemented it, it turned into some generic enough implementation that should be useful for all the DRM drivers out there, and they even introduced some particularly nice features, to split the final screen content into several planes based on the actual displayed content rather than on windows like it’s usually done.

Their work also helped to point out a few flaws in the hwcomposer API, that will eventually be fixed in a new revision of that API.

ARC++ SlidesVideo

The next talk was once again from a ChromeOS engineer, David Reveman, who came to show his work on ARC++, the component in ChromeOS that allows to run Android applications. He was obviously mostly talking about the display side.

In order to achieve that, he had to implement an hwcomposer that would just act as a proxy between SurfaceFlinger and Wayland that is used on the ChromeOS side. The GL rendering is still direct though, and each Android application will talk directly to the GPU, as usual. Only the composition will be forwarded to the ChromeOS side.

In order to minimize that composition process, whenever possible, ARC++ tries to back each application with an overlay so that the composition would happen directly in hardware.

This also led to some interesting challenges, especially since some of the assumptions of both systems are in contradiction. For example, any application can be resized in ChromeOS, while it’s not really a thing in Android where all the applications run full screen.

HDR Displays in Linux – SlidesVideo

The next talk we found interesting was Andy Ritger from nVidia explaining how the HDR displays were supposed to be handled in Linux.

He first started by explaining what HDR is exactly. While the HDR is just about having a wider range of luminance than on a regular display, you often also get a wider gamut with HDR capable displays. This means that on those screens you can display a wider range of colors, and with a better range and precision in their intensity. And
while the applications have been able to generate HDR content for more than 10 years, the rest of the display stack wasn’t really ready, meaning that you had convert the HDR colors to colors that your monitor was able to display, using a technique called tone mapping.

He then explained than the standard, non-HDR colorspace, sRGB, is not a linear colorspace. This means than by doubling the encoded luminance of a color, you will not get a color twice brighter on your display. This was meant this way because the human eye is much more sensitive to the various shades of colors when they are dark than when they are bright. Which essentially means that the darker the color is, the more precision you want to get.

However, the luminance “resolution” on the HDR display is so good that you actually don’t need that anymore, and you can have a linear colorspace, which is in our case SCRGB.

But drawing blindly in all your applications in SCRGB is obviously not a good solution either. You have to make sure that your screen supports it (which is exposed through its EDIDs), but also that you actually tell your screeen to switch to it (through the infoframes). And that requires some support in the kernel drivers.

The Anatomy of a Vulkan Driver – SlidesVideo

This talk by Jason Ekstrand was some kind of a war story of the bring up Intel did of a Vulkan implementation on their GPU.

He first started by saying that it was actually a not so long project, especially when you consider that they wrote it from scratch, since it took roughly 3 full-time engineers 8 months to come up with a fully compliant and open source stack.

He then explained why Vulkan was needed. While OpenGL did amazingly well to cope with the hardware evolutions, it was still designed over 20 years ago, This proved to have some core characteristics that are not really relevant any more, and are holding the application developers back. For example, he mentioned that at its core, OpenGL is based on a singleton-based state machine, that obviously doesn’t scale well anymore on our SMP systems. He also mentioned that it was too abstracted, and people just wanted a lower level API, or that you might want to render things off screen without X or any context.

This was fixed in Vulkan by effectively removing the state machine, which allows it to scale, push things like the error checking or the synchronization directly to the applications, making the implementation much simpler and less layered which also simplifies the development and debugging.

He then went on to discuss how we could share the code that was still shared between the two implementations, like implementing OpenGL on top of Vulkan (which was discarded), having some kind of lighter intermediate language in Mesa to replace Gallium or just sharing through a library the common bits and making both the OpenGL and Vulkan libraries use that.

Motivating preemptive GPU scheduling for real-time systems – SlidesVideo

The last talk that we want to mention is the talk on preemptive scheduling by Roy Spliet, from the University of Cambridge.

More and more industries, and especially the automotive industry, offload some computations to the GPU for example to implement computer vision. This is then used in a car to implement the autonomous driving to make the car recognize signs or stay in its lane. And obviously, this kind of computations are supposed to be handled in a real time
system, since you probably don’t want your shiny user interface for the heating to make your car crash in the car before it because its rendering was taking too long.

He first started to explain what real time means, and what the usual metrics are, which should to no surprise to people used to “CPU based” real time systems: latency, deadline, execution time, and so on.

He then showed a bunch of benchmarks he used to test his preemptive scheduler, in a workload that was basically running OpenArena while running some computations, on various nouveau based platforms (both desktop-grade GPUs, and embedded SoCs).

This led to some expected conclusions, like the fact that a preemptive scheduler is indeed adding some overhead, but is on average worth it, while some have been quite interesting. He was for example observing some worst case latencies that were quite rare (0.3%), but were actually interferences from the display engine filling up its empty FIFOs, and creating some contention on the memory bus.

Conclusion

Overall, this has been a great experience. The organisation was flawless, and the one-track-only format allows you to meet easily both the speakers and attendees. The content was also highly technical, as you might expect, which made us learn a lot and led us to think about some interesting developments we could do on our various projects in the future, such as NextThing Co’s CHIP.

Posted in Conference, Technical | Tagged , , , | Leave a comment

Yocto project and OpenEmbedded training updated to Krogoth

yocto

Continuing our efforts to keep our training materials up-to-date we just refreshed our Yocto project and OpenEmbedded training course to the latest Yocto project release, Krogoth (2.1.1). In addition to adapting our training labs to the Krogoth release, we improved our training materials to cover more aspects and new features.

The most important changes are:

  • New chapter about devtool, the new utility from the Yocto project to improve the developers’ workflow to integrate a package into the build system or to make patches to existing packages.
  • Improve the distro layers slides to add configuration samples and give advice on how to use these layers.
  • Add a part about quilt to easily patch already supported packages.
  • Explain in depth how file inclusions are handled by BitBake.
  • Improve the description about tasks by adding slides on how to write them in Python.

The updated training materials are available on our training page: agenda (PDF), slides (PDF) and labs (PDF).

Join our Yocto specialist Alexandre Belloni for the first public session of this improved training in Lyon (France) on October 19-21. We are also available to deliver this training worldwide at your site, contact us!

Posted in News, training | Tagged , , , | Leave a comment

Free Electrons at the Kernel Recipes conference

Kernel RecipesThe 2016 edition of the Kernel Recipes conference will take place from September 28th to 30th in Paris. With talks from kernel developers Jonathan Corbet, Greg Kroah-Hartmann, Daniel Vetter, Laurent Pinchart, Tejun Heo, Steven Rosdedt, Kevin Hilman, Hans Verkuil and many others, the schedule looks definitely very appealing, and indeed the event is now full.

Thomas Petazzoni, Free Electrons CTO, will be attending this event. If you’re interested in discussing business or career opportunities with Free Electrons, this event will be a great place to meet together.

Posted in Conference | Tagged | Leave a comment

Free Electrons at the X Developer Conference

The next X.org Developer Conference will take place on September 21 to September 23 in Helsinki, Finland. This is a major event for Linux developers working in the graphics/display areas, not only at the X.org level, but also at the kernel level, in Mesa, and other related projects.

Free Electrons engineer Maxime Ripard will be attending this conference, with 80+ other engineers from Intel, Google, NVidia, Texas Instruments, AMD, RedHat, etc.

Maxime is the author of the DRM/KMS driver in the upstream Linux kernel for the Allwinner SoCs, which provides display support for numerous Allwinner platforms, especially Nextthing’s CHIP (with parallel LCD support, HDMI support, VGA support and composite video support). Maxime has also worked on making the 3D acceleration work on this platform with a mainline kernel, by adapting the Mali kernel driver. Most recently, Maxime has been involved in Video4Linux development, writing a driver for the camera interface of Allwinner SoCs, and supervising Florent Revest work on the Allwinner VPU that we published a few days ago.

Posted in Conference, Technical | Tagged , | Leave a comment

Free Electrons mentioned in Linux Foundation’s report

Linux Kernel Development Report 2016Lask week, the Linux Foundation announced the publication of the 2016 edition of its usual report “Linux Kernel Development – How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It”.

This report gives a nice overview of the evolution of the Linux kernel since 3.18, especially from a contribution point of view: the rate of changes, who is contributing, are there new developers joining, etc.

Free Electrons is mentioned in several places in this report. First of all, even though Free Electrons is a consulting company, it is shown individually rather than part of the general “consultants” category. As the report explains:

The category “consultants” represents developers who contribute to the kernel as a work-for-hire effort from different companies. Some consultant companies, such as Free Electrons and Pengutronix, are shown individually as their contributions are a significant number.

Thanks to being mentioned separately from the “consultants” category, the report also shows that:

  • Free Electrons is the #15 contributing company over the 3.19 to 4.7 development period, in number of commits. Free Electrons contributed a total of 1453 commits, corresponding to 1.3% of the total commits
  • Free Electrons is ranked #13 in the list of companies by number of Signed-off-by from developers who are not the author of patches. This happens because 6 of our engineers are maintainers or co-maintainers from various areas in the kernel: they merge patches from contributors, sign-off on them, and send them to another maintainer (either arm-soc maintainers or directly Linus Torvalds, depending on the subsystem).

We’re glad to see Free Electrons mentioned in this report, which shows that we are a strong contributor to the official Linux kernel. Thanks to this contribution effort, we have tremendous experience with adding support for new hardware in the kernel, so contact us if you want your hardware supported in the official Linux kernel!

Posted in Technical | Leave a comment

Support for the Allwinner VPU in the mainline Linux kernel

Over the last few years, and most recently with the support for the C.H.I.P platform, Free Electrons has been heavily involved in initiating and improving the support in the mainline Linux kernel for the Allwinner ARM processors. As of today, a large number of hardware features of the Allwinner processors, especially the older ones such as the A10 or the A13 used in the CHIP, are usable with the mainline Linux kernel, including complex functionality such as display support and 3D acceleration. However, one feature that was still lacking is proper support for the Video Processing Unit (VPU) that allows to accelerate in hardware the decoding and encoding of popular video formats.

During the past two months, Florent Revest, a 19 year old intern at Free Electrons worked on a mainline solution for this Video Processing Unit. His work followed the reverse engineering effort of the Cedrus project, and this topic was also listed as a High Priority Reverse Engineering Project by the FSF.

The internship resulted in a new sunxi-cedrus driver, a Video4Linux memory-to-memory decoder kernel driver and a corresponding VA-API backend, which allows numerous userspace applications to use the decoding capabilities. Both projects have both been published on Github:

Currently, the combination of the kernel driver and VA-API backend supports MPEG2 and MPEG4 decoding only. There is for the moment no support for encoding, and no support for H264, though we believe support for both aspects can be added within the architecture of the existing driver and VA-API backend.

A first RFC patchset of the kernel driver has been sent to the linux-media mailing list, and a complete documentation providing installation information and architecture details has been written on the linux-sunxi’s wiki.

Here is a video of VLC playing a MPEG2 demo video on top of this stack on the Next Thing’s C.H.I.P:

Posted in Technical | Tagged , , | 3 Comments

Free Electrons at the Embedded Linux Conference Europe

The next Embedded Linux Conference Europe will take place from October 11 to October 13 in Berlin, Germany. As usual, the entire Free Electrons engineering team will participate, which means this time 10 participants from Free Electrons!

Embedded Linux Conference Europe 2016

The schedule for the conference has been published recently, and a number of our talk proposals have been accepted, so we will present on the following topics:

Like every year, we’re looking forward to attending this conference, and meeting all the nice folks of the Embedded Linux community!

Posted in Conference, Technical | Tagged , | 1 Comment

Linux 4.7 statistics: Free Electrons engineer #2 contributor

LWN.net has published yesterday an article containing statistics for the 4.7 development cycle. This article is available for LWN.net subscribers only during the coming week, and will then be available for everyone, free of charge.

It turns out that Boris Brezillon, Free Electrons engineer, is the second most active contributor to the 4.7 kernel in number of commits! The top three contributors in number of commits are: H Hartley Sweeten (208 commits), Boris Brezillon (132 commits) and Al Viro (127 commits).

LWN.net 4.7 kernel statistics

In addition to being present in the most active developers by number of commits, Boris Brezillon is also in the #11 most active contributor in terms of changed lines. As we discussed in our previous blog post, most contributions from Boris were targeted at the PWM subsystem on one side (atomic update support) and the NAND subsystem on the other side.

Another Free Electrons engineer shows up in the per-developer statistics: Maxime Ripard is the #17 most active contributor by lines changed. Indeed, Maxime contributed a brand new DRM/KMS driver for the Allwinner display controller.

As a company, Free Electrons is ranked for the 4.7 kernel as the #12 most active company by number of commits, and #10 by number of changed lines. We are glad to continue being such a contributor to the Linux kernel development, as we have been for the last four years. If you want your hardware to be supported in the official Linux kernel, contact us!

Posted in Technical | Tagged , | Leave a comment

“Understanding D-Bus” talk at the Toulouse Embedded Linux Meetup

A few months ago, in May, Free Electrons engineer Mylène Josserand presented a talk titled Understanding D-Bus at the Toulouse Embedded Linux and Android meetup.

In this talk, Mylène shared her experience working with D-Bus, especially in conjunction with the OFono and Connman projects, to support modem and 3G connections on embedded Linux systems.

Understanding D-Bus

We are now publishing the slides of Mylène’s talk, they are available in PDF format.

Posted in Conference, Technical | Tagged , , | Leave a comment

Linux 4.7 released, Free Electrons contributions

Adelie PenguinLinux 4.7 has been released on Sunday by Linus Torvalds, with numerous new features and improvements that have been described in details on LWN: part 1, part 2 and part 3. KernelNewbies also has an updated page on the 4.7 release. We contributed a total of 222 patches to this release.

Our most significant contributions:

  • Boris Brezillon has contributed a core improvement to the PWM subsystem: a mechanism that allows to update the properties of a PWM in an atomic fashion. This is needed when a PWM has been initialized by the bootloader, and the kernel needs to take over without changing the properties of the PWM. See the main patch for more details. What prompted the creation of this patch series is a problem on Rockchip based Chromebook platforms where a PWM is used for a regulator, and the PWM properties need to be preserved across the bootloader to kernel transition. In addition to the changes of the core infrastructure, Boris contributed numerous patches to fix existing PWM users.
  • In the MTD subsystem, Boris Brezillon continued his cleanup efforts
    • Use the common Device Tree parsing code provided by nand_scan_ident() in more drivers, rather than driver-specific code.
    • Move drivers to expose their ECC/OOB layout information using the mtd_ooblayout_ops structure, and use the corresponding helper functions where appropriate. This change will allow a more flexible description of the ECC and OOB layout.
    • Document the Device Tree binding that should now be used for all NAND controllers / NAND chip, with a clear separation between the NAND controller and the NAND chip. See this commit for more details.
  • In the RTC subsystem, Mylène Josserand contributed numerous improvements to the rv3029 and m41t80 drivers, including the addition of the support for the RV3049 (the SPI variant of RV3029). See also our previous blog post on the support of Microcrystal’s RTCs/.
  • On the support of Atmel platforms
    • Boris Brezillon contributed a number of fixes and improvements to the atmel-hlcdc driver, the DRM/KMS driver for Atmel platforms
  • On the support of Allwinner platforms
    • Maxime Ripard contributed a brand new DRM/KMS driver to support the display controller found on several Allwinner platforms, with a specific focus on Allwinner A10. This new driver allows to have proper graphics support in the Nextthing Co. C.H.I.P platform, including composite output and RGB output for LCD panels. To this effect, in addition to the driver itself, numerous clock patches and Device Tree patches were made.
    • Boris Brezillon contributed a large number of improvements to the NAND controller driver used on Allwinner platforms, including performance improvements.
    • Quentin Schulz made his first kernel contribution by sending a patch fixing the error handling in a PHY USB driver used by Allwinner platforms.
  • On the support of Marvell platforms
    • Grégory Clement made some contributions to the mv_xor driver to make it 64-bits ready, as the same XOR engine is used on Armada 3700, a Cortex-A53 based SoC. Grégory then enabled the use of the XOR engines on this platform by updating the corresponding Device Tree.
    • Romain Perier did some minor updates related to the Marvell cryptographic engine support. Many more updates will be present in the upcoming 4.8, including significant performance improvements.
    • Thomas Petazzoni contributed some various fixes (cryptographic engine usage on some Armada 38x boards, HW I/O coherency related fixes).
    • Thomas also improved the support for Armada 7K and 8K, with the description of more hardware blocks, and updates to drivers.

Here are in details, the different contributions we made to this release:

Posted in Technical | Tagged , , | 1 Comment