copyleft hardware planet

September 29, 2018

Bunnie Studios

Name that Ware, September 2018

The Ware for September 2018 is shown below.

Been a busy month banging my head against the wall of getting FCC/CE certification for NeTV2, and spending thousands of dollars on dozens of tests — more time, effort, and treasure than developing the product itself. This is my least favorite aspect of product development — the regulatory burdens are just so immense if you actually try to comply with all the rules, especially with such a global marketplace (every region you legally serve multiplies your paperwork load, not to mention different SKUs for power supplies & manual/packaging translations).

Rather ironic to have finally figured out all the technical tricks to make production in small batches efficient, only to find there’s no efficient way to deal with regulatory hurdles. It’s a discouraging message for small-time makers and innovators, and tilts things in the favor of large corporations with the funding and scale to build internal certification teams and facilities to make the regulatory process efficient and predictable.

by bunnie at September 29, 2018 10:14 AM

Winner, Name that Ware August 2018

The Ware for August 2018 is Micom router, with an EasyRouter HCF feature pack installed. It’s interesting how the ROM cartridge drew many people to the conclusion this was some sort of an old laser printer motherboard thanks to the ancient practice of purchasing fonts as physical ROM cartridges — I would have thought the same initially, except most laser printers also have a pretty substantial general purpose CPU built into them, and there isn’t enough RAM on board either for that type of application. SAM nailed it — I only slightly blurred some of the numbers to try and give something to search for, as Micom is a pretty obscure company and almost everything about this piece of hardware pre-dates modern search engines. Congrats, email me for your prize!

by bunnie at September 29, 2018 10:08 AM

September 28, 2018

Harald Welte

Fernvale Kits - Lack of Interest - Discount

Back in December 2014 at 31C3, bunnie and xobs presented about their exciting Fernvale project, how they reverse engineered parts of the MT6260 ARM SoC, which also happens to contain a Mediatek GSM baseband.

Thousands (at least hundreds) of people have seen that talk live. To date, 2506 people (or AIs?) have watched the recordings on youtube, 4859 more people on media.ccc.de.

Given that Fernvale was the closest you could get to having a hackable baseband processor / phone chip, I expected at least as much interest into this project as we received four years earlier with OsmocomBB.

As a result, in early 2015, sysmocom decided to order 50 units of Fernvale DVT2 evaluation kits from bunnie, and to offer them in the sysmocom webshop to ensure the wider community would be able to get the boards they need for research into widely available, inexpensive 2G baseband chips.

This decision was made purely for the perceived benefit of the community: Make an exciting project available for anyone. With that kind of complexity and component density, it's unlikely anyone would ever solder a board themselves. So somebody has to build some and make it available. The mark-up sysmocom put on top of bunnie's manufacturing cost was super minimal, only covering customs/import/shipping fees to Germany, as well as minimal overhead for packing/picking and accounting.

Now it's almost four years after bunnie + xobs' presentation, and of those 50 Fernvale boards, we still have 34 (!) units in stock. That means, only 16 people on this planet ever had an interest in playing with what at the time I thought was one of the most exciting pieces of equipment to play with.

So we lost somewhere on the order of close to 3600 EUR in dead inventory, for something that never was supposed to be a business anyway. That sucks, but I still think it was worth it.

In order to minimize the losses, sysmocom has now discounted the boards and reduced the price from EUR 110 to to EUR 58.82 (excluding VAT). I have very limited hope that this will increase the amount of interest in this project, but well, you got to try :)

In case you're thinking "oh, let's wait some more time, until they hand them out for free", let me tell you: If money is the issue that prevents you from playing with a Fernvale, then please contact me with the details about what you'd want to do with it, and we can see about providing them for free or at substantially reduced cost.

In the worst case, it was ~ 3600 EUR we could have invested in implementing more Osmocom software, which is sad. But would I do it again if I saw a very exciting project? Definitely!

The lesson learned here is probably that even a technically very exciting project backed by world-renowned hackers like bunnie doesn't mean that anyone will actually ever do anything with it, unless they get everything handed on a silver plate, i.e. all the software/reversing work is already done for them by others. And that actually makes me much more sad than the loss of those ~ 3600 EUR in sysmocom's balance sheet.

I also feel even more sorry for bunnie + xobs. They've invested time, money and passion into a project that nobody really seemed to want to get involved and/or take further. ("nobody" is meant figuratively. I know there were/are some enthusiasts who did pick up. I'm talking about the big picture). My condolences to bunnie + xobs!

by Harald Welte at September 28, 2018 10:00 PM

September 24, 2018

Michele's GNSS blog

Learning a bit about Beidou3 signals

I have pretty busy these last couple of years contributing to features and improvements to the Piksi Multi measurement engine, but during the Summer I managed to contribute to a blog post on the Swift Navigation page together with two colleagues of mine who I thank sincerely: Michael Wurm (lead FPGA engineer) and Keerthan Jaic (python and many other languages expert).
We studied a bit the signals of Beidou3, which is growing at incredible pace. For the interested readers, here is the link.

A screenshot of the Swift Console says more than 1000 words:



There should be data and code you all can play with.And questions welcome, hope you enjoy the read.

P.S.: I won't be at the ION this year (too busy with other things) but I will likely be at the Stanford PNT meeting in November.

by noreply@blogger.com (Michele Bavaro) at September 24, 2018 12:39 AM

September 18, 2018

Harald Welte

Wireshark dissector for 3GPP CBSP - traces wanted!

I recently was reading 3GPP TS 48.049, the specification for the CBSP (Cell Broadcast Service Protocol), which is the protocol between the BSC (Base Station Controller) and the CBC (Cell Broadcast Centre). It is how the CBC according to spec is instructing the BSCs to broadcast the various cell broadcast messages to their respective geographic scope.

While OsmoBTS and OsmoBSC do have support for SMSCB on the CBCH, there is no real interface in OsmoBSC yet on how any external application would instruct it tot send cell broadcasts. The only existing interface is a VTY command, which is nice for testing and development, but hardly a scalable solution.

So I was reading up on the specs, discovered CBSP and thought one good way to get familiar with it is to write a wireshark dissector for it. You can find the result at https://code.wireshark.org/review/#/c/29745/

Now my main problem is that as usual there appear to be no open source implementations of this protocol, so I cannot generate any traces myself. More surprising is that it's not even possible to find any real-world CBSP traces out there. So I'm facing a chicken-and-egg problem. I can only test / verify my wireshark dissector if I find some traces.

So if you happen to have done any work on cell broadcast in 2G network and have a CBSP trace around (or can generate one): Please send it to me, thanks!

Alternatively, you can of course also use the patch linked above, build your own wireshark from scratch, test it and provide feedback. Thanks in either case!

by Harald Welte at September 18, 2018 10:00 PM

September 09, 2018

Dieter Spaar

A Pico Base Station from Ericsson

The RBS6401 is a small indoor base station from Ericsson for WCDMA or LTE. The device is about two times the size of an ip.access nanoBTS. It is based on a KeyStone I SoC from TI and runs Linux (in fact there are two KeyStone I SoCs inside, but it seems that only one of them is used, at least for WCDMA).

Compared to the other commercial base stations I have seen so far the RBS6401 makes it quite hard to get access to the operating system. It tries to setup a VPN to a Security Gateway for Autointegration into the operator's network and there is only a simple Web Interface to set the network parameters of the device.

Unfortunately I have only found the WCDMA software on the three devices I have access to. It would really be nice to use the RBS6401 with LTE, WCDMA is not that interesting (I am not aware of any Open Source RNC). If anyone has the LTE software for the RBS6401, please let me know.

September 09, 2018 02:00 AM

August 31, 2018

Video Circuits

VECTOR HACK 2018


I haven't updated in a while (there is still a write up of my amazing signal culture trip last year coming)

But this is too good not to leave a short post about. I am helping Ivan and Derek with their amazing Vector Hack Festival project. Vector Hack is a festival centred around experimental vector graphics using oscilloscopes and lasers. It's happening in over two sites starting in Zagreb on 01/10/18 and ending in Ljubljana on 07/10/18. I will be attending so if you decide to come see you there!

Ivan and Derek have worked tirelessly to make this happen and it's going to be amazing.

vectorhackfestival.com
instagram.com/vectorhackfestival
facebook.com/vectorhackfestival


Vector Hack festival 2018. trailer I from i.m. klif on Vimeo.

by Chris (noreply@blogger.com) at August 31, 2018 03:49 AM

August 26, 2018

Harald Welte

Still alive, just not blogging

It's been months without any update to this blog, and I feel sad about that. Nothing particular has happened to me, everything is proceeding as usual.

At the Osmocom project we've been making great progress on a variety of fronts, including

  • 3GPP LCLS (Local Call, Local Switch)
  • Inter-BSC hand-over in osmo-bsc
  • load Based hand-over in osmo-bsc
  • reintroducing SCCPlite compatibility to the new BSC code in osmo-bsc / libosmo-sigtran
  • finishing the first release of the SIMtrace2 firmware
  • extending test coverage on all fronts, particularly in our TTCN-3 test suites
  • tons of fixes to the osmo-bts measurement processing / reporting
  • higher precision time of arrival reporting in osmo-bts
  • migrating osmocom.org services to new, faster servers

At sysmocom, next to the Osmocom topics above, we've

  • made the sysmoQMOD remote SIM firmware much more robust and reliable
  • after months of delays, finally SIMtrace2 hardware kits are available again
  • created autoamtic testing of pySim-prog and sysmo-usim-util
  • extended our osmo-gsm-tester based automatic testing setup to include multi-TRX nanoBTS setups

In terms of other topic,

  • my wife and I have been to a three week motorbike tour all over the Alps in July
  • I've done tons of servicing (brake piston fittings, brake tubes, fuel line, fixing rust/paint, replacing clutch cable, choke cable, transmission chain, replacing several rusted/worn-out needle bearings, and much more) on my 22year old BMW F650ST to prepare it for many more yers to come. As some type-specific spare parts (mostly plastic parts) are becoming rarer, it was best to take care of replacements sooner than later
  • some servicing/repairs to my 19 year old Audi A4 car (which passed German mandatory inspection without any deficiency at the first attempt!)
  • some servicing of my Yamaha FZ6
  • repaired my Fairphone 2 by swapping the microphone module (mike was mute)
  • I've re-vamped a lot of the physical/hardware infrastructure for gnumonks.org and other sites I run, which was triggered by having to move racks

by Harald Welte at August 26, 2018 10:00 PM

August 14, 2018

Bunnie Studios

Name that Ware, August 2018

The Ware for August 2018 is shown below.

Thanks to Patricio Worthalter for contributing this ware!

by bunnie at August 14, 2018 10:02 AM

Winner, Name that Ware July 2018

The Ware for July 2018 is an I/O board for the x86-based Sega Nu arcade platform. Congrats to megabytephreak for nailing the ware, email me for your prize! Also props to Vegard for pointing out the JVS connector (the thing that looks like USB but isn’t).

I was curious how certain readers were able to identify this as an arcade-related ware, and the answer I received is that the I/O breakout board is the key — the DIP switches and push-buttons are typical of how arcade machines are configured and serviced, and the JVS connector, if you can recognize it, is a dead give-away. The coin cell for retaining configs & high scores even if the power is pulled is also a potential tell. You learn something new every day!

by bunnie at August 14, 2018 10:02 AM

July 18, 2018

Bunnie Studios

Tariffs in a Nutshell

I was asked to distill a previous post about tariffs into something more accessible to the general public. The resulting article ended up being run on CNN Digital as an opinion piece:

In retaliation for unfair trade practices and the theft of American innovations and ideas, the US Trade Representative’s office is imposing a 25% tariff on a broad range of goods imported from China.

But these tariffs won’t help American workers. Instead, they will encourage American companies to push ideas and production overseas by raising the cost of raw materials without penalizing the import of finished goods.
[…]
Imagine a bakery located in the US. It uses imported flour, sugar and cacao to make delectable cakes based on a closely-guarded secret family recipe handed down for generations, and it employs dozens of bakers to do this. Now suppose a bakery in China has tried to copy the recipe…

The article uses a bakery as an analogy to explain the trade war situation, as well as thinking through why trade deficits are OK through the notion that buying a T-shirt at a store creates a “trade deficit” between you and the store, but in the end that trade deficit is actually quite helpful to you. You can read the full article on CNN Digital.

I had also prepared a short infographic to accompany the article, which wasn’t picked up by CNN, but you can enjoy it here.

by bunnie at July 18, 2018 11:44 AM

Name that Ware July 2018

The Ware for July 2018 is shown below.

Thanks to the little birdie that dropped this ware in my inbox. :) Really nice photography work.

by bunnie at July 18, 2018 06:45 AM

Winner, Name that Ware June 2018

The ware for June 2018 is a TI TALP1000B. Marcan nailed it, congrats! email me for your prize.

Here’s some more photos of the ware, in case you want more context:

It’s like a science project that became a product.

by bunnie at July 18, 2018 06:44 AM

June 19, 2018

Bunnie Studios

New US Tariffs are Anti-Maker and Will Encourage Offshoring

The new 25% tariffs announced by the USTR, set to go into effect on July 6th, are decidedly anti-Maker and ironically pro-offshoring. I’ve examined the tariff lists (List 1 and List 2), and it taxes the import of basic components, tools and sub-assemblies, while giving fully assembled goods a free pass. The USTR’s press release is careful to mention that the tariffs “do not include goods commonly purchased by American consumers such as cellular telephones or televisions.”

Think about it – big companies with the resources to organize thousands of overseas workers making TVs and cell phones will have their outsourced supply chains protected, but small companies that still assemble valuable goods from basic parts inside the US are about to see significant cost increases. Worse yet educators, already forced to work with a shoe-string budget, are going to return from their summer recess to find that basic parts, tools and components for use in the classroom are now significantly more expensive.


Above: The Adafruit MetroX Classic Kit is representative of a typical electronics education kit. Items marked with an “X” in the above image are potentially impacted by the new USTR tariffs.

New Tariffs Reward Offshoring, Encourage IP Flight

Some of the most compelling jobs to bring back to the US are the so-called “last screw” system integration operations. These often involve the complex and precise process of integrating simple sub-assemblies into high-value goods such as 3D printers or cell phones. Quality control and IP protection are paramount. I often advise startups to consider putting their system integration operations in the US because difficult-to-protect intellectual property, such as firmware, never has to be exported if the firmware upload operation happens in the US. The ability to leverage China for low-value subassemblies opens more headroom to create high-value jobs in the US, improving the overall competitiveness of American companies.

Unfortunately, the structure of the new tariffs are exactly the opposite of what you would expect to bring those jobs back to the US. Stiff new taxes on simple components, sub-assemblies, and tools like soldering irons contrasted against a lack of taxation on finished goods pushes business owners to send these “last screw” operation overseas. Basically, with these new tariffs the more value-add sent outside the borders of the US, the more profitable a business will be. Not even concerns over IP security could overcome a 25% increase in base costs and keep operations in the US.

It seems the intention of the new tariff structure was to minimize the immediate pain that voters would feel in the upcoming mid-terms by waiving taxes on finished goods. Unfortunately, the reality is it gives small businesses that were once considering setting up shop in the US a solid reason to look off-shore, while rewarding large corporations for heavy investments in overseas operations.

New Tariffs Hurt Educators and Makers

Learning how to blink a light is the de-facto introduction to electronics. This project is often done with the help of a circuit board, such as a Microbit or Chibi Chip, and a type of light known as an LED. Unfortunately, both of those items – simple circuit boards and LEDs – are about to get 25% more expensive with the new tariffs, along with other Maker and educator staples such as capacitors, resistors, soldering irons, and oscilloscopes. The impact of this cost hike will be felt throughout the industry, but most sharply by educators, especially those serving under-funded school districts.


Above: Learning to blink a light is the de-facto introduction to electronics, and it typically involves a circuit board and an LED, like those pictured above.

Somewhere on the Pacific Ocean right now floats a container of goods for ed-tech startup Chibitronics. The goods are slated primarily for educators and Makers that are stocking up for the fall semester. It will arrive in the US the second week of July, and will likely be greeted by a heavy import tax. I know this because I’m directly involved in the startup’s operations. Chibitronics’ core mission is to serve the educator market, and as part of that we routinely offered deep discounts on bulk products for educators and school systems. Now, thanks to the new tariffs on the basic components that educators rely upon to teach electronics, we are less able to fulfill our mission.

A 25% jump in base costs forces us to choose between immediate price increases or cutting the salaries of our American employees who support the educators. These new tariffs are a tax on America’s future – it deprives some of the most vulnerable groups of access to technology education, making future American workers less competitive on the global stage.


Above: Educator-oriented learning kits like the Chibitronics “Love to Code” are slated for price increases this fall due to the new tariffs.

Protectionism is Bad for Technological Leadership

Recently, I was sent photos by Hernandi Krammes of a network card that was manufactured in Brazil around 1992. One of the most striking features of the card was how retro it looked – straight out of the 80’s, a full decade behind its time. This is a result of Brazil’s policy of protectionist tariffs on the import of high-tech components. While stiff tariffs on the import of microchips drove investment in local chip companies, trade barriers meant the local companies didn’t have to be as competitive. With less incentive to re-invest or upgrade, local technology fell behind the curve, leading ultimately to anachronisms like the Brazilian Ethernet card pictured below.


Above: this Brazilian network card from 1992 features design techniques from the early 80’s. It is large and clunky compared to contemporaneous cards.

Significantly, it’s not that the Brazilian engineers were any less clever than their Western counterparts: they displayed considerable ingenuity getting a network card to work at all using primarily domestically-produced components. The tragedy is instead of using their brainpower to create industry-leading technology, most of their effort went into playing catch-up with the rest of the world. By the time protectionist policies were repealed in Brazil, the local industry was too far behind to effectively compete on a global scale.

Should the US follow Brazil’s protectionist stance on trade, it’s conceivable that some day I might be remarking on the quaintness of American network cards compared to their more advanced Chinese or European counterparts. Trade barriers don’t make a country more competitive – in fact, quite the opposite. In a competition of ideas, you want to start with the best tech available anywhere; otherwise, you’re still jogging to the starting line while the competition has already finished their first lap.

Stand Up and Be Heard

There is a sliver of good news in all of this for American Makers. The list of commodities targeted in the trade war is not yet complete. The “List 2” items – which include all manner of microchips, motors, and plastics (such as 3D printer PLA filament and acrylic sheets for laser cutting) that are building blocks for small businesses and Makers – have yet to be ratified. The USTR website has indicated in the coming weeks they will disclose a process for public review and comment. Once this process is made transparent – whether you are a small business owner or the parent of a child with technical aspirations – I encourage you to please share your stories and concerns on how you will be negatively impacted by these additional tariffs.

Some of the List 2 items still under review include:

9030.31.00 Multimeters for measuring or checking electrical voltage, current, resistance or power, without a recording device
8541.10.00 Diodes, other than photosensitive or light-emitting diodes
8541.40.60 Diodes for semiconductor devices, other than light-emitting diodes, nesoi
8542.31.00 Electronic integrated circuits: processors and controllers
8542.32.00 Electronic integrated circuits: memories
8542.33.00 Electronic integrated circuits: amplifiers
8542.39.00 Electronic integrated circuits: other
8542.90.00 Parts of electronic integrated circuits and microassemblies
8501.10.20 Electric motors of an output of under 18.65 W, synchronous, valued not over $4 each
8501.10.60 Electric motors of an output of 18.65 W or more but not exceeding 37.5 W
8501.31.40 DC motors, nesoi, of an output exceeding 74.6 W but not exceeding 735 W
8544.49.10 Insulated electric conductors of a kind used for telecommunications, for a voltage not exceeding 80 V, not fitted with connectors
8544.49.20 Insulated electric conductors nesoi, for a voltage not exceeding 80 V, not fitted with connectors
3920.59.80 Plates, sheets, film, etc, noncellular, not reinforced, laminated, combined, of other acrylic polymers, nesoi
3916.90.30 Monafilament nesoi, of plastics, excluding ethylene, vinyl chloride and acrylic polymers

Here’s some of the “List 1” items that are set to become 25% more expensive to import from China, come July 6th:

Staples used by every Maker or electronics educator:

8515.11.00 Electric soldering irons and guns
8506.50.00 Lithium primary cells and primary batteries
8506.60.00 Air-zinc primary cells and primary batteries
9030.20.05 Oscilloscopes and oscillographs, specially designed for telecommunications
9030.33.34 Resistance measuring instruments
9030.33.38 Other instruments and apparatus, nesoi, for measuring or checking electrical voltage, current, resistance or power, without a recording device
9030.39.01 Instruments and apparatus, nesoi, for measuring or checking

Circuit assemblies (like Microbit, Chibi Chip, Arduino):

8543.90.68 Printed circuit assemblies of electrical machines and apparatus, having individual functions, nesoi
9030.90.68 Printed circuit assemblies, NESOI

Basic electronic components:

8532.21.00 Tantalum fixed capacitors
8532.22.00 Aluminum electrolytic fixed capacitors
8532.23.00 Ceramic dielectric fixed capacitors, single layer
8532.24.00 Ceramic dielectric fixed capacitors, multilayer
8532.25.00 Dielectric fixed capacitors of paper or plastics
8532.29.00 Fixed electrical capacitors, nesoi
8532.30.00 Variable or adjustable (pre-set) electrical capacitors
8532.90.00 Parts of electrical capacitors, fixed, variable or adjustable (pre-set)
8533.10.00 Electrical fixed carbon resistors, composition or film types
8533.21.00 Electrical fixed resistors, other than composition or film type carbon resistors, for a power handling capacity not exceeding 20 W
8533.29.00 Electrical fixed resistors, other than composition or film type carbon resistors, for a power handling capacity exceeding 20 W
8533.31.00 Electrical wirewound variable resistors, including rheostats and potentiometers, for a power handling capacity not exceeding 20 W
8533.40.40 Metal oxide resistors
8533.40.80 Electrical variable resistors, other than wirewound, including rheostats and potentiometers
8533.90.80 Other parts of electrical resistors, including rheostats and potentiometers, nesoi
8541.21.00 Transistors, other than photosensitive transistors, with a dissipation rating of less than 1 W
8541.29.00 Transistors, other than photosensitive transistors, with a dissipation rating of 1 W or more
8541.30.00 Thyristors, diacs and triacs, other than photosensitive devices
8541.40.20 Light-emitting diodes (LED’s)
8541.40.70 Photosensitive transistors
8541.40.80 Photosensitive semiconductor devices nesoi, optical coupled isolators
8541.40.95 Photosensitive semiconductor devices nesoi, other
8541.50.00 Semiconductor devices other than photosensitive semiconductor devices, nesoi
8541.60.00 Mounted piezoelectric crystals
8541.90.00 Parts of diodes, transistors, similar semiconductor devices, photosensitive semiconductor devices, LED’s and mounted piezoelectric crystals
8504.90.75 Printed circuit assemblies of electrical transformers, static converters and inductors, nesoi
8504.90.96 Parts (other than printed circuit assemblies) of electrical transformers, static converters and inductors
8536.50.90 Switches nesoi, for switching or making connections to or in electrical circuits, for a voltage not exceeding 1,000 V
8536.69.40 Connectors: coaxial, cylindrical multicontact, rack and panel, printed circuit, ribbon or flat cable, for a voltage not exceeding 1,000 V
8544.49.30 Insulated electric conductors nesoi, of copper, for a voltage not exceeding 1,000 V, not fitted with connectors
8544.49.90 Insulated electric conductors nesoi, not of copper, for a voltage not exceeding 1,000 V, not fitted with connectors
8544.60.20 Insulated electric conductors nesoi, for a voltage exceeding 1,000 V, fitted with connectors
8544.60.40 Insulated electric conductors nesoi, of copper, for a voltage exceeding 1,000 V, not fitted with connectors

Parts to fix your phone if it breaks:

8537.10.80 Touch screens without display capabilities for incorporation in apparatus having a display
9033.00.30 Touch screens without display capabilities for incorporation in apparatus having a display
9013.80.70 Liquid crystal and other optical flat panel displays other than for articles of heading 8528, nesoi
9033.00.20 LEDs for backlighting of LCDs
8504.90.65 Printed circuit assemblies of the goods of subheading 8504.40 or 8504.50 for telecommunication apparatus

Power supplies:

9032.89.60 Automatic regulating or controlling instruments and apparatus, nesoi
9032.90.21 Parts and accessories of automatic voltage and voltage-current regulators designed for use in a 6, 12, or 24 V system, nesoi
9032.90.41 Parts and accessories of automatic voltage and voltage-current regulators, not designed for use in a 6, 12, or 24 V system, nesoi
9032.90.61 Parts and accessories for automatic regulating or controlling instruments and apparatus, nesoi
8504.90.41 Parts of power supplies (other than printed circuit assemblies) for automatic data processing machines or units thereof of heading 8471

by bunnie at June 19, 2018 04:13 AM

June 08, 2018

Harald Welte

Re-launching openmoko USB Product ID and Ethernet OUI registry

Some time after Openmoko went out of business, they made available their USB Vendor IDs and IEEE OUI (Ethernet MAC address prefix) available to Open Source Hardware / FOSS projects.

After maintaining that for some years myself, I was unable to find time to continue the work and I had handed it over some time ago to two volunteers. However, as things go, those volunteers also stopped to respond to PID / OUI requests, and we're now launching the third attempt of continuing this service.

As the openmoko.org wiki will soon be moved into an archive of static web pages only, we're also moving the list of allocated PID and OUIs into a git repository.

Since git.openmoko.org is also about to be decommissioned, the repository is now at https://github.com/openmoko/openmoko-usb-oui, next to all the archived openmoko.org repository mirrors.

This also means that in addition to sending an e-mail application for getting an allocation in those ranges, you can now send a pull-request via github.

Thanks to cuvoodoo for volunteering to maintain the Openmoko USB PID and IEEE OUI allocations from now on!

by Harald Welte at June 08, 2018 10:00 PM

June 05, 2018

Bunnie Studios

Developing Apps for Your TV the Easy and Open Way

The biggest screen in your house would seem a logical place to integrate cloud apps, but TVs are walled gardens. While it’s easy enough to hook up a laptop or PC and pop open a browser, there’s no simple, open framework for integrating all that wonderful data over
the TV’s other inputs.

Until now. Out of the box, NeTV2’s “NeTV Classic Mode” makes short work of overlaying graphics on top of any video feed. And thanks to the Raspberry Pi bundled in the Quickstart version, NeTV2 app developers get to choose from a diverse and well-supported ecosystem of app frameworks to install over the base Raspbian image shipped with every device.

For example, Alasdair Allan’s article on using the Raspberry Pi with Magic Mirror and Google AIY contains everything you need to get started on turning your TV into a voice-activated personal assistant. I gave it a whirl, and in just one evening I was able to concoct the demo featured in the video below.



Magic Mirror is a great match for NeTV2, because all the widgets are formatted to run on a black background. Once loaded, I just had to set the NeTV2’s chroma key color to black and the compositing works perfectly. Also, Google AIY’s Voicekit framework “just worked” out of the box. The only fussy bit was configuring it to work with my USB microphone, but thankfully there’s a nice Hackaday article detailing exactly how to do that.

Personally, I find listening to long-form replies from digital assistants like Alexa or Google Home a bit time consuming. As you can see from this demo, NeTV2 lets you build a digital assistant that pops up data and notifications over the biggest screen in your house using rich visual formats. And the best part is, when you want privacy, you can just unplug the microphone.

If you can develop an app that runs on a Raspberry Pi, you already know everything you need to integrate apps into any TV. Thanks to NeTV2, there’s never been an easier or more open way to make the biggest screen in your house the smartest screen.

The NeTV2 is crowdfunding now at CrowdSupply.com, and we’re just shy of our first stretch goal: a free Tomu bundled with every board. Normally priced at $30, Tomu is a tiny open-source computer that fits in a USB Type-A port, and it’s the easiest way to add an extra pair of status LEDs to an NeTV2. Help unlock this deal by backing now and spreading the word!

by bunnie at June 05, 2018 04:01 PM

June 03, 2018

Bunnie Studios

Name that Ware, June 2018

The Ware for June 2018 is shown below.

This month we’ll start with the very zoomed in and slightly torn down view of the ware, and if nobody’s nailed it I’ll release some more contextual images throughout the month.

Special thanks to Nava for giving me this ware!

Update June 19, 2018: Looks like people are stumped by this one, so providing a little more context to see if anyone gets it…

by bunnie at June 03, 2018 10:14 AM

Winner, Name that Ware May 2018

The NeTV2-MVP DVT1E rev shown in the May 2018 ware has 933 vias and 63 holes for a total of 996 drill hits. The closest guess for total number of drill hits was Jonathan, at 69 holes and 888 vias = 957 drill hits (39 shy of the total). Mangel was in a close second place with 84 hols and 962 vias for a total of 1046 drill hits (50 over the total). So, Jonathan’s the winner! Congrats, please email me to claim your board!

by bunnie at June 03, 2018 10:14 AM

May 28, 2018

Dieter Spaar

A personal remark about security research

In the last few years I have done a lot of automotive security research, one of my findings about cars from BMW, which was the result of working on contract for the ADAC, was published in February 2015: the original German version and the translated English version.

A few days ago Tencent Keen Security Lab published their findings about BMW cars in this paper.

While in general I find it great that there is more public research in the area of automotive security, I really don't like it if previous work isn't respected. The report of Tencent Keen Security Lab does not contain any reference to previous work besides their own. I would at least have expected references to previous QNX research (QNX is the OS used in the BMW infotainment ECU) and my work when it comes to the TCB (Telematic Communication Box, the name of the BMW telematic ECU). For example regarding the TCB, the description of the communication protocol with the backend and how encrypted SMS is used, with identical encryption keys in all cars, was already contained in my report. To find out why there are no references to others work I contacted them, but I didn't receive any reply.

Could it be that they really aren't aware of previous work? If you start researching the BMW telematic ECU, you would very soon become aware of NGTP (Next Generation Telematics Protocol), the protocol used to communicate with the BMW backend. Searching on Google for "TCB" and "NGTP" will give you the link to the German article on the first place. OK, it's just the German article and you would have to open it to find the link to the English translation, which requires an additional step.

What if they don't use Google but Baidu? Searching for "TCB" and "NGTP" on Baidu gave this link on the fourth place. I can't read the page, but this seems to be the Chinese translation of the article from Heise, even the text in the diagrams is translated. Interestingly when I tried to reproduce my search from last week on Baidu today, this page no longer appears in the search results, although the page itself still exists.

So for me its looks like this: Tencent Keen Security Lab either simply doesn't give reference to previous work and follows the old Chinese tradition of "Clone and Copy" or they don't have any clue about Information Gathering, which is one the first steps when starting to research a target. In my opinion neither of the two possibilities is what you really want.

May 28, 2018 02:00 AM

May 24, 2018

Harald Welte

openmoko.org archive down due to datacenter issues

Unfortunately, since about 11:30 am CEST on MAy 24, openmoko.org is down due to some power outage related issues at Hetzner, the hosting company at which openmoko.org has been hosting for more than a decade now.

The problem seems to have caused quite a lot of fall-out tom many servers (Hetzner is hosting some 200k machines, not sure how many affected, though), and Hetzner is anything but verbose when it comes to actually explaining what the issue is.

All they have published is https://www.hetzner-status.de/en.html#8842 - which is rather tight lipped about some power grid issues. But then, what do you have UPSs for if not for "a strong voltage reduction in the local power grid"?

The openmoko.org archive machine is running in Hetzner DC10, by the way. This is where they've had the largest number of tickets.

In any case, we'll have to wait for them to resolve their tickets. They appear to be working day and night on that.

I have a number of machines hosted at Hetzner, and I'm actually rather happy that none of the more important systems were affected that long. Some machines simply lost their uplink connectivity for some minutes, while some others were rebooted (power outage). The openmoko.org archive is the only machine that didn't automatically boot after the outage, maybe the power supply needs replacement.

In any case, I hope the service will be back up again soon.

btw: Guess who's been paying for hosting costs ever since Openmoko, Inc. has shut down? Yes, yours truly. It was OK for something like 9 years, but I want to recursively pull the dynamic content through some cache, which can then be made permanent. The resulting static archive can then be moved to some VM somewhere, without requiring a dedicated root server. That should reduce the costs down to almost nothing.

by Harald Welte at May 24, 2018 10:00 PM

OsmoCon 2018 CfP closes on 2018-05-30

One of the difficulties with OsmoCon2017 last year was that almost nobody submitted talks / discussions within the deadline, early enough to allow for proper planning.

This lad to the situation where the sysmocom team had to come up with a schedule/agenda on their own. Later on much after the CfP deadline,people then squeezed in talks, making the overall schedule too full.

It is up to you to avoid this situation again in 2018 at OsmoCon2018 by submitting your talk RIGHT NOW. We will be very strict regarding late submissions. So if you would like to shape the Agenda of OsmoCon 2018, this is your chance. Please use it.

We will have to create a schedule soon, as [almost] nobody will register to a conference unless the schedule is known. If there's not sufficient contribution in terms of CfP response from the wider community, don't complain later that 90% of the talks are from sysmocom team members and only about the Cellular Network Infrastructure topics.

You have been warned. Please make your CfP submission in time at https://pretalx.sysmocom.de/osmocon2018/cfp before the CfP deadline on 2018-05-30 23:59 (Europe/Berlin)

by Harald Welte at May 24, 2018 10:00 PM

May 23, 2018

Harald Welte

Mailing List hosting for FOSS Projects

Recently I've encountered several occasions in which a FOSS project would have been interested in some reliable, independent mailing list hosting for their project communication.

I was surprised how difficult it was to find anyone running such a service.

From the user / FOSS project point of view, the criteria that I would have are:

  • operated by some respected entity that is unlikely to turn hostile, discontinue the service or go out of business altogether
  • free of any type of advertisements (we all know how annoying those are)
  • cares about privacy, i.e. doesn't sell the subscriber lists or non-public archives
  • use FOSS to run the service itself, such as GNU mailman, listserv, ezmlm, ...
  • an easy path to migrate away to another service (or self-hosting) as they grow or their requirements change. A simple mail forward to that new address for the related addresses is typically sufficient for that

If you think mailing lists serve no purpose these days anyways, and everyone is on github: Please have a look at the many thousands of FOSS project mailing lists out there still in use. Not everyone wants to introduce a dependency to the whim of a proprietary software-as-a-service provider.

I never had this problem as I always hosted my own mailman instance on lists.gnumonks.org anyway, and all the entities that I've been involved in (whether non-profit or businesses) had their own mailing list hosts. From franken.de in the 1990ies to netfilter.org, openmoko.org and now osmocom.org, we all pride oursevles in self-hosting.

But then there are plenty of smaller projects that neither have the skills nor the funding available. So they go to yahoo groups or some other service that will then hold them hostage without a way to switch their list archives from private to public, without downloadable archives or forwarding in the case they want to move away :(

Of course the larger FOSS projects also have their own list servers, starting from vger.kernel.org to Linux distributions like Debian GNU/Linux. But what if your FOSS project is not specifically Linux related?

The sort-of obvious candidates that I found all don't really fit:

Now don't get me wrong, I'm of course not expecting that there are commercial entities operating free-of charge list hosting services where you neither pay with money, nor your data, nor by becoming a spam receiver.

But still, in the wider context of the Free Software community, I'm seriously surprised that none of the various non-for-profit / non-commercial foundations or associations are offering a public mailing list hosting service for FOSS projects.

One can of course always ask any from the above list and ask for a mailing list even though it's strictly speaking off-topic to them. But who will do that, if he has to ask uninvited for a favor?

I think there's something missing. I don't have the time to set up a related service, but I would certainly want to contribute in terms of funding in case any existing FOSS related legal entity wanted to expand. If you already have a legal entity, abuse contacts, a team of sysadmins, then it's only half the required effort.

by Harald Welte at May 23, 2018 10:00 PM

May 12, 2018

Bunnie Studios

Innovation Should Be Legal. That’s Why I’m Launching NeTV2.

I’d like to share a project I’m working on that could have an impact on your future freedoms in the digital age. It’s an open video development board I call NeTV2.

The Motivation

It’s related to a lawsuit I’ve filed with the help of the EFF against the US government to reform Section 1201 of the DMCA. Currently, Section 1201 imbues media cartels with nearly unchecked power to prevent us from innovating and expressing ourselves, thus restricting our right to free speech.

Have you ever noticed how smart TVs seem pretty dumb compared to our phones? It’s because Section 1201 enables a small cartel of stakeholders to pick and choose who gets to process video. So, for example, anyone is allowed to write a translation app for their smartphone that does real-time video translation of text. However, it’s potentially unlawful to build a box, even in the privacy of my own home, that implements the same thing over the HDCP-encrypted video feeds that go from my set top box to my TV screen.

This is due to a quirk of the DMCA that makes it unlawful for most citizens to bypass encryption – even for lawful free-speech activities, such as self-expression and innovation. Significantly, since the founding of the United States, it’s been unlawful to make copies of copyrighted work, and I believe the already stiff penalties for violating copyright law offer sufficient protection from piracy and theft.

However, in 1998 a group of lobbyists managed to convince Congress that the digital millennium presented an existential threat to copyright holders, and thus stiffer penalties were needed for the mere act of bypassing encryption, no matter the reason. These penalties are in addition to the existing penalties written into copyright law. By passing this law, Congress effectively turned bypassing encryption into a form of pre-crime, empowering copyright holders to be the sole judge, jury and executioner of what your intentions might have been. Thus, even if you were to bypass encryption solely for lawful purposes, such as processing video to translate text, the copyright holder nonetheless has the power to prosecute you for the “pre-crimes” that could follow from bypassing their encryption scheme. In this way, Section 1201 of the DMCA effectively gives corporations the power to license when and how you express yourself where encryption is involved.

I believe unchecked power to license freedom of expression should not be trusted to corporate interests. Encryption is important for privacy and security, and is winding its way into every corner of our life. It’s fundamentally a good thing, but we need to make sure that corporations can’t abuse Section 1201 to also control every corner of our life. In our digital age, the very canvas upon which we paint our thoughts can be access-controlled with cryptography, and we need the absolute right to paint our thoughts freely and share them broadly if we are to continue to live in a free and just society. Significantly, this does not diminish the power of copyrights one bit – this lawsuit simply aims to limit the expansive “pre-crime” powers granted to license holders, that is all.

Of course, even though the lawsuit is in progress, corporations still have the right to go after developers like you and me for the notional pre-crimes associated with bypassing encryption. However, one significant objection lodged by opponents of our lawsuit is that “no other users have specified how they are adversely affected by HDCP in their ability to make specific noninfringing use of protected content … [bunnie] has failed to demonstrate … how “users ‘are, or are likely to be,’ adversely affected by the prohibition on circumventing HDCP.” This is, of course, a Catch-22, because how can you build a user base to demonstrate the need for freedoms when the mere act of trying to build that user base could be a crime in itself? No investor would touch a product that could be potentially unlawful.

Thankfully, it’s 2018 and we have crowd funding, so I’m launching a crowd funding campaign for the NeTV2, in the hopes of rallying like-minded developers, dreamers, users, and enthusiasts to help build the case that a small but important group of people can and would do more, if only we had the right to do so. As limited by the prevailing law, the NeTV2 can only process unencrypted video and perform encryption-only operations like video overlays through a trick I call “NeTV mode”. However, it’s my hope this is a sufficient platform to stir the imagination of developers and users, so that together we can paint a vibrant picture of what a future looks like should we have the right to express our ideas using otherwise controlled paints on otherwise denied canvases.


Some of the things you might be able to do with the NeTV2, if you only had the right to do it…

The Hardware

The heart of the NeTV2 is an FPGA-based video development board in a PCIe 2.0 x4 card form factor. The board supports up to two video inputs and two video outputs at 1080p60, coupled to a Xilinx XC7A35T FPGA, along with 512 MiB of DDR3 memory humming along at a peak bandwidth of 25.6 Gbps. It also features some nice touches for debugging including a JTAG/UART header made to plug directly into a Raspberry Pi, and a 10/100 Ethernet port wired directly to the FPGA for Etherbone support. For intrepid hackers, the reserved/JTAG pins on the PCI-express header are all wired to the FPGA, and microSD and USB headers are provisioned but not specifically supported in any mode. And of course, the entire PCB design is open source under the CERN OHL license.


The NeTV2 board as mounted on a Raspberry Pi

The design targets two major use scenarios which I refer to as “NeTV classic” mode (video overlays with encryption) and “Libre” mode (deep video processing, but limited to unencrypted feeds due to Section 1201).

In NeTV classic mode, the board is paired with a Raspberry Pi, which serves as the source for chroma key overlay video, typically rendered by a browser running in full-screen mode. The Raspberry Pi’s unencrypted HDMI video output is fed into the NeTV2 and sampled into a frame buffer, which is “genlocked” (e.g. timing synchronized) to a video feed that’s just passing through the FPGA via another pair of HDMI input/outputs. The NeTV2 has special circuits to help observe and synchronize with cryptographic state, should one exist on the pass-through video link. This allows the NeTV2 to encrypt the Raspberry Pi’s overlay feed so that the Pi’s pixels can be used for a simple “hard overlay” effect. NeTV classic mode thus enables applications such as subtitles and pop-up notifications by throwing away regions of source video and replacing it entirely with overlay pixels. However, a lack of access to unencrypted pixels disallows even basic video effects such as alpha blending or frame scaling.

In Libre mode, the board is meant to be plugged into a desktop PC via PCI-express. Libre mode only works with unencrypted video feeds, as the concept here is full video frames are sampled and buffered up inside NeTV2 so that it can be forwarded on to the host PC for further processing. Here, the full power of a GPU or x86 CPU can be applied to extract features and enhance the video, or perhaps portions of the video could even be sent to to the cloud for processing. Once the video has been processed, it is pushed back into the NeTV2 and sent on to the TV for viewing. Libre mode is perhaps the most interesting mode to developers, yet is very limited in every day applications thanks to Section 1201 of the DMCA. Still, it may be possible to craft demos using properly licensed, unencrypted video feeds.

The reference “gateware” (FPGA design) for the NeTV2 is written in Python using migen/LiteX. I previously compared the performance of LiteX to Vivado: for an NeTV2-like reference design, the migen/LiteX version consumes about a quarter the area and compiles in less than a quarter the time – a compelling advantage. migen/LiteX is a true open source framework for describing hardware, which relies on Xilinx’s free-to-download Vivado toolchain for synthesis, place/route, and bitstream generation. There is a significant effort on-going today to port the full open source FPGA backend tools developed by Clifford Wolf from the Lattice ICE40 FPGAs to the same Xilinx 7-series FPGAs used in NeTV2. Of course, designers that prefer to use the Vivado tools to describe and compile their hardware are still free to do so, but I am not officially supporting that design methodology.

I wanted to narrow the gap between development board and field deployable solution, so I’ve also designed a hackable case for the NeTV2. The case can hold the NeTV2 and a mated Raspberry Pi, and consists of three major parts, a top shell, bottom shell/back bezel, and a stand-alone front bezel. It also has light pipes to route key status LEDs to the plane of the back bezel. It’s designed to be easily disassembled using common screw drivers, and features holes for easy wall-mounting.

Most importantly, the case features extra space with a Peek Array on the inside for mounting your own PCBs or parts, and the front bezel is designed for easier fabrication using either subtractive or additive methodologies. So, if you have a laser cutter, you can custom cut a bezel using a simple, thin sheet of acrylic and slot it into the grooves circumscribing the end of the case. Or, if you have a low-res 3D printer, you can use the screw bosses to attach the bezel instead, and skip the grooves. When you’re ready to step up in volume, you can download the source file for the bezel and make a relatively simple injection mold tool for just the bezel itself (or the whole case, if you really want to!).

The flexibility of the PCI-express edge connector and the simplified bezel allows developers to extend the NeTV2 into a system well beyond the original design intention. Remember, for an FPGA, PCI-express is just a low-cost physical form factor for generic high speed I/O. So, a relatively simple to design and cheap to fabricate adapter card can turn the PCI-express card-edge connector into a variety of high-speed physical standards, including SATA, DisplayPort, USB3.0 and more. There’s also extra low-speed I/O in the header, so you can attach a variety of SPI or I2C peripherals through the same connector. This electrical flexibility, combined with PCBs mounted on the Peek Array and a custom bezel enables developers to build a customer-ready solutions with minimal effort and tooling investment.

The NeTV2 is funding now at Crowd Supply. I’m offering a version with a higher-capacity FPGA only for the duration of the campaign, so if you’re developer be sure to check that out before the campaign ends. If you think that reforming the DMCA is important but the NeTV2 isn’t your cup of tea, please consider supporting the EFF directly with a donation. Together we can reform Section 1201 of the DMCA, and win back fundamental freedoms to express and innovate in the digital age.

by bunnie at May 12, 2018 02:22 AM

Name that Ware, May 2018

The Ware for May 2018 is shown below.

This month’s contest isn’t about naming a ware. Instead, the challenge is to guess the total amount of vias + holes on the NeTV2 PCB. The closest (or first correct) guess will get a DVT-revision of the board (same as the one shown above) as a prize! Please note: depending on your country of residence you may have to cover import duties or VAT.

At the end of this month, once the contest is over, I’ll upload the PCB source files to a public repo. The contest would be just too easy otherwise :)

by bunnie at May 12, 2018 02:20 AM

Winner, Name that Ware April 2018

The ware for April 2018 is an SAE 316L stainless steel cerclage currently embedded in my right kneecap. There’s also a pair of 2.5mm holes drilled lengthwise through the patella, and a set of sutures woven into my quadriceps, neither of which show up in an X-ray. The hardware is to repair a double-break of my knee; I simultaneously suffered a patellar avulsion fracture and a tear of the quadriceps tendon. It’s rare to get both injuries at once, but hey why do anything half-assed. As for how it happened – well, you’ll have to believe me that it’s just from falling while walking on flat ground. There was a puddle of greasy/soapy water outside a hawker stall and boom, I’m unable to walk for months. I wish I had some cooler story, so that’s why I had this month’s competition – to try and find a better bar story. To that end, gratz to perlfriend for the most imaginative story; email me for your prize!

This was my first time going through major surgery. I’ve previously had some minor patches and cuts, where I only used local anesthesia and watched the surgeon perform. As a hardware guy, I think your body is the most fascinating piece of hardware you’ll ever own so I don’t miss an opportunity to take a peek inside when maintenance is required (to clarify, I find it totally gross to watch other people’s surgeries, but somehow when it’s my own body its different – my sense of wonder overtakes my aversion to blood).

So I’ve never been put full under before. When the anesthesiologist came to interview me, I asked what my options were for local anesthesia. She then tried to convince me that general anesthesia is very safe because they have this computer running some complicated algorithm that considers my weight and height to dose me correctly. Besides the fact that they had to estimate my weight because I couldn’t stand on a scale, my inner monologue was screaming “As a hardware engineer, putting my life in the hands of a computer sounds terrifying” but instead I manage to ask politely, “and what happens if the computer has a problem?” At which point she smiles and re-assures me that she’s constantly monitoring my vital signs and trimming the dosage.

Side note, a friend of mine later on pointed me to an article where apparently some infusion pumps have Wifi that’s on during surgery and the protocol is pathetically hackable. OK, medical equipment makers: there’s some equipment that simply should not be IoT, and an infusion pump is absolutely one of them. There is simply no valid reason for the computer attending to the dosage level of a potentially fatal chemical to be spending any cycles answering TCP packets. Death by DDoS is not an acceptable scenario. If you need to push an update, do it by a USB disk or a plug-in Ethernet jack, so they can at least air gap the damn thing during an operation.

Had I known about the wifi vulnerability of anesthesia infusion pumps, I probably would not have consented to general anesthesia, or at least asked to check that the model they were using had no IoT capability. But the anesthesiologist finally convinced me under the argument that the local procedure would require sticking a needle into my spinal cord and withdrawing cerebrospinal fluid, and I’m like, I don’t know what that is but it sounds important for my brain to work and I kind of like my spinal cord intact, so maybe it’s not worth the risk.

So, I finally consent to being put under. Propofol, the drug she used to initiate anesthesia, is pretty incredible. It acts within seconds, and despite taking over an hour to metabolise in the liver, it clears from the brain in minutes so you can come out of anesthesia quickly. I have to wonder how they discovered it, or developed it. Anyways, the anesthesiologist inserted an IV into my right hand – I appreciated her attentiveness to the detail that I’m left-handed – and proceeded to administer the propofol. I could definitely feel the propofol as it entered my system. I had the same feeling of pins and needles that one gets when a limb falls asleep, a sort of searing, prickling pain. The pain quickly shot up my arm, then a warm wave of prickling across my face, and then…

I’m in the recovery room, and my surgeon is there giving me the post-op interview. About 2.5 hours of my life went by and I had no memory of it. Sure enough, no nausea or dizziness – the anesthesiologist did a good job. I was lucid and talking to the surgeon about the operation. Then, maybe about five or maybe fifteen minutes later, my body goes into shock – I’m shivering uncontrollably, and I’m starting to become acutely aware of the pain in my knee.

Apparently, during the surgery, I was given Fentanyl and Ultiva, both potent opioid painkillers. They give it to you because even though you’re out, your body has autonomous responses to pain that can complicate surgery with excess bleeding and so forth. From what I read, both clear pretty quickly, so I’m guessing the onset of shock corresponded with the painkilling effect of the injected opioids wearing off.

I know exactly what drugs and equipment they used because at the conclusion of the surgery I was given a detailed itemized bill of everything involved; above is a tiny sample of it. It was fascinating to reverse engineer the surgery from the BOM. That’s how I knew they used a 2.5mm drill bit, how I knew they put a breathing tube in me, and that I also got injected with Aloxi to help suppress nausea. I actually sort of wish I knew about the drill bit ahead of time, because I would have asked to keep it. I figured they’re throwing it away so getting the surgical-grade 2.5mm bit that was used to drill holes in my body to use later on to drill mounting bosses would be pretty neat.

Just for the record, the overall hospital bill including surgeon’s fees, surgical materials, OT rental, recovery ward, drugs, X-rays and my first round of physiotherapy was about S$20.7k, or US$15.5k at the prevailing exchange rate. This is considered expensive by Singapore standards, as I went to one of the nicer private hospitals for the operation, but as I understand it is quite cheap by US standards. Fortunately my insurance covered all but about $350 of it.

This was also my first time using opioid painkillers in a serious way. I’m deeply concerned about the addictive nature of opioids, so I don’t take them lightly. I had taken Vicodin once before when I had a wisdom tooth removed, and I remember it having the effect of “moving the pain sideways” – it felt like the pain was just outside arms length, so I could ignore it if I wanted to, but I definitely knew it was still there. Knee surgery was about a hundred times more traumatic than getting a wisdom tooth removed, and for the first day there’s a terrific amount of pain. So they gave me Targin, which is a mix of oxycodone and naloxone.

Targin is a clever way to manage opioid addictiveness. Oxycodone is the opioid; naloxone is the stuff they use to treat opioid overdoses, and it blocks the painkilling effects. Nalaxone is mixed in with the oxycodone to prevent mis-use (crushing & snorting or injecting), and naloxone is poorly absorbed through the digestive system, so when taken orally the oxycodone can still work. For a short time. When the surgical painkillers wore out, basically all the pain receptors in my knee were screaming at me and informing me rather viscerally about what I already knew intellectually – that my knee was cut up, had holes drilled in it, wired up with stainless steel, fibers woven into the muscles, and stitched back together. So I got one dose of Targin, which kicked in in about 20-30 minutes. The oxycodone effect was pretty strong – the acute pain just went away, everything felt okay and I drifted into sleep…only to be woken up about 2-3 hours later by the pain in my knee, presumably due to the nalaxone component finally kicking in and suppressing the opioid effect. However, I was only allowed one Targin pill every 12 hours, and thus had only acetaminophen to manage the pain until the next scheduled dose. It was definitely manageable with some distraction, but there was no way I was going back to sleep again. I could see how oxycodone could be addictive; everything just seemed so okay despite everything being incredibly wrong, so I’m grateful they mixed it with nalaxone and administered it such that my final memory of the trip was being clobbered with my knee pain rather than a lingering desire for more.

Of course, the pain doen’t end in 24 hours, but they “graduated” me to a weaker painkiller, Ultracet. The Ultracet is a mix of Tramadol and acetaminophen. Tramadol itself really kicks in only after it’s processed by the liver into desmetramadol by the enzyme CYP2D6, and I’m heterozygous for a copy with reduced activity. I guess that might explain why when I took the prescribed dosage of two pills it was reasonably effective, but when I halved it to one pill I felt almost nothing at all. Overall, Tramadol was less effective at painkilling and more effective in just making me feel a little dizzy and sleepy. In addition to binding to opioid receptors, it’s also a seratonin releasing agent, a bit like MDMA. So during the daytime I would just deal with the pain by distracting myself with work, and at night time when trying to fall asleep, I’d take the Ultracet and sleep like a baby – I was sleeping almost 12 hours a night for the couple weeks after surgery, which I think was extremely helpful in my recovery. Significantly, Tramadol doesn’t quite suppress pain; like my one experience with Vicoden, it turns it into a fact I’m aware of but can deal with. When I would sleep, my dreams would be deep and lucid, but my brain would often reference the pain in my knee and ascribe it to some random weird dream explanation, like trying to go in for a big penalty kick in a soccer game over and over again. Fortunately, Tramadol didn’t feel like it had too much of an addiction risk for me – the awake-state “high” was a bit unpleasant, as I don’t particularly enjoy the dizziness or drowsiness. It was more of a sleeping aid, as it allowed my mind to let go of the pain in my knee while falling asleep. I was finally able to wean myself off painkillers entirely and fall asleep drug-free about two weeks after the operation, but in a weird way I ever so slightly missed that really sound sleep I’d have while on them. Opioid painkillers are no joke. They’re absolutely essential for dealing with pain, but take them with discipline and caution.

While the story behind the accident itself wasn’t very interesting, I did find going through the surgery, healthcare system, and drugs to be an interesting learning experience.

by bunnie at May 12, 2018 12:00 AM

April 29, 2018

Harald Welte

OsmoDevCon 2018 retrospective

One week ago, the annual Osmocom developer meeting (OsmoDevCon 2018) concluded after four long and intense days with old and new friends (schedule can be seen here).

It was already the 7th incarnation of OsmoDevCon, and I have to say that it's really great to see the core Osmocom community come together every year, to share their work and experience with their fellow hackers.

Ever since the beginning we've had the tradition that we look beyond our own projects. In 2012, David Burgess was presenting on OpenBTS. In 2016, Ismael Gomez presented about srsUE + srsLTE, and this year we've had the pleasure of having Sukchan Kim coming all the way from Korea to talk to us about his nextepc project (a FOSS implementation of the Evolved Packet Core, the 4G core network).

What has also been a regular "entertainment" part in recent years are the field trip reports to various [former] satellite/SIGINT/... sites by Dimitri Stolnikov.

All in all, the event has become at least as much about the people than about technology. It's a community of like-minded people that to some part are still working on joint projects, but often work independently and scratch their own itch - whether open source mobile comms related or not.

After some criticism last year, the so-called "unstructured" part of OsmoDevCon has received more time again this year, allowing for exchange among the participants irrespective of any formal / scheduled talk or discussion topic.

In 2018, with the help of c3voc, for the first time ever, we've recorded most of the presentations on video. The results are still in the process of being cut, but are starting to appear at https://media.ccc.de/c/osmodevcon2018.

If you want to join a future OsmoDevCon in person: Make sure you start contributing to any of the many Osmocom member projects now to become eligible. We need you!

Now the sad part is that it will take one entire year until we'll reconvene. May the Osmocom Developer community live long and prosper. I want to meet you guys for many more years at OsmoDevCon!

There is of course the user-oriented OsmoCon 2018 in October, but that's a much larger event with a different audience.

Nevertheless, I'm very much looking forward to that, too.

The OsmoCon 2018 Call for Participation is still running. Please consider submitting talks if you have anything open source mobile communications related to share!

by Harald Welte at April 29, 2018 10:00 PM

April 24, 2018

Bunnie Studios

Paper As a Substrate for Circuits

I’ve spent a considerable portion of my time in the past couple of years helping to develop products for Chibitronics, a startup that blends two unlikely bedfellows together, papercraft and electronics, to create paper circuits. The primary emphasis of Chibitronics is creating a more friendly way to learn, design and create electronics. Because of this, much of the material relating to paper circuitry on the Internet looks more like art than electronics.

This belies the capabilities of paper as an engineering material. Google’s Cardboard and Nintendo’s Labo are mainstream examples of paper’s extraordinary capability as an engineering material. Prof. Nadya Peek at the University of Washington has written several academic papers on building multi-axis CNC machines using paper products.

A couple points to clarify up top: for the sake of brevity, I will use the term “paper” instead of “paper and/or cardboard”, analogous to how one would refer to a PCB made of Kapton or FR-4 both as printed circuits. Furthermore, while Chibitronics focuses on providing solderless solutions for younger learners, the techniques shared in this post targets engineers who have the skill to routinely assemble modern SMT designs. I assume you’ve got a good soldering iron and a microscope, and you know how to use both (or perhaps are up to the challenge of learning how to use them better).

The Argument for Paper
For prototyping and learning the principles of electronics, paper has several distinct advantages over breadboards.

The primary advantage of a breadboard is that it’s solderless, and as a result you can re-use the components. This made a lot more sense back when a 6502 used to cost $25 in 1975 ($115 in 2018 money), but today the wire jumper kit for a solderless breadboard can cost more than the microcontroller. Considering also the relatively high cost of a solderless breadboard and the relatively low value of the parts, you’re probably better off buying extra parts and soldering them to disposable paper substrates than purchasing a re-usable solderless breadboard for all but the simplest of circuits.


Electronic components used to be really expensive, so you wanted to re-use them as much as possible. The 8-bit 6502 at $115 (adjusted for inflation) was considerably cheaper than its competition in 1975 (from Wikipedia).

On the other hand, paper has a number of important advantages. The first is that it’s compatible with surface mount ICs. This is increasingly important as chip vendors have largely abandoned DIP packages in favor of SMT packages: mobile computing represents the highest demand for chips, and SMT packages beat DIP packages in both thermal and parasitic electrical characteristics. So if you want a part that wasn’t designed by someone wearing a jean jacket and highwaters, you’re probably going to find it only available in SMT.


The evolution of packaging (from left to right): DIP, SOIC, TSSOP, and WLCSP. The WLCSP is shown upside-down so you can see how solder balls are applied directly to a naked silicon chip. It’s the asymptotic size limit of packaging, and is quite popular in mobile phones today.

The second and perhaps more important advantage is that it’s electrically similar to a printed circuit substrate. Breadboards feature long, loose wires with no sense of impedance control at all. Printed circuits are 2.5-D (e.g. planar multi-layer) constructions that feature short, flat wires and often times ground planes that enable impedance control. Paper circuit construction is much closer to that of a printed circuit, in that flat copper tape forms traces that can be layered on top of each other (using non-conductive tape to isolate the layers). Furthermore, when laid on top of a controlled-thickness substrate such as cardboard, the reverse side can be covered with a plane of copper tape, thus allowing for impedance control. The exact same equations govern impedance control in a paper circuit constructed with a ground plane as a printed circuit constructed with a ground plane – just the constants are different.


This equation works for both FR-4 and cardboard. Just plug in the corresponding ε, w, t, and h. From rfcafe.com.

This means you can construct RF circuits using paper electronic techniques — from directional couplers to antennae to amplifiers. The low parasitics of copper tape also means you can construct demanding circuits that would be virtually impossible to breadboard, such as high-power switching regulators, where ripple performance is heavily impacted by parasitic resistance and inductance in the ground connections.


A 10W, 5V buck regulator laid out with paper electronics. The final layout closely resembles the datasheet layout example and performs smoothly at 2A load; this circuit probably wouldn’t regulate properly at high loads if built with a SMT-to-DIP breakout and a breadboard.

In addition to impedance control and lower parasitics, the use of copper tape to form planes means paper electronics can push the power envelope by leveraging copper plans as heatsinks. This is an important technique in FR-4 based PCBs; in fact, for many chips, the dominant path for heat to escape a chip is not through the package surface, but instead through the pins and package traces.


Copper conducts heat about 1000x better than plastic, so even the tiny metal pins on a chip can conduct heat more efficiently from an IC than the surface of the plastic package. Flip-chip on lead frame graphic adapted from Electronic Design.

The copper which forms the pins and lead frames of a chip package is a vastly superior (about 1000x better) heat conductor compared to air or plastic, so a cheap and effective method of heatsinking is to lay out a large plane of copper connected to the chip. Below is an example of a 60-watt power driver that I built using paper electronics, leveraging a copper tape plane plus extra foil as heat sinks.


That’s a 12A power transistor, and this heater control circuit can use much of that ampacity. Additional copper foil was soldered on for extra heat sinking. The equivalent in DIP/TO packages might melt a breadboard during normal operation.

Paper electronics has one additional advantage that is unique to itself: the ability to fold and bend into 3-dimensional shapes. This is something that neither breadboards nor FR-4 circuit boards can readily do. Normally, circuit boards that can bend require more exotic processes like rigi-flex or flex PCB manufacture; but paper supports this natively. Artists take advantage of this property to create stunning electronic origami, but engineers can also use this property to great effect.


Trox Circuit Study 05 by Jonathan Bobrow, from the Paper Curiosities gallery

The ability to fold a sheet of cardboard or paper means that paper circuits can be slotted around tight corners and conformed to irregular or flexible surfaces, eliminating connectors and creating a thinner, sleeker packages. Need a test point? Cut a hanging tab out of your substrate, and you’ve got a fold-up point where you can attach an alligator clip!

Using Paper to Facilitate Prototyping with SMT
Here’s a detailed example of the construction techniques I use when working with paper electronics. I built a breakout board to solve a common problem: matching voltages between chips. Older chips are powered by 5V, newer ones by 1.8V or lower, and none of these are a match for your typical 3.3V-tolerante microcontroller. There are small circuits called “level shifters” that can safely take digital signals of one signal swing to another range. The problem is that most of the “good” ICs today come only in SMT packages, so if you’re prototyping on a breadboard or using alligator clips to cobble something together, you’ve got very limited options. In fact, one of my “go-to” ICs for this purpose is the 74LVC1T45; it’s capable of 420Mbps data rates, and can convert anywhere from 1.65V to 5.5V in a direction that can be selected using an input pin. The packaging options for this chip range from a DSBGA to a SOT-23 – clearly a chip targeted at the mobile phone generation, and not meant for breadboarding.

However, I’m often confronted with the problem of driving a WS281B LED strip from the I/O of a modern microprocessor. WS2812B LEDs operate off of 5V, and expect 5V CMOS levels; and no modern microprocessor can produce that. You can usually get away with driving a single WS2812B with a 3.3V-compatible I/O, but if you’re driving a long chain of them you’ll start to see glitches down the chain because of degraded timing margins due to improper voltage levels at the head of the chain. So, I’d love to have a little breakout board that adapts a SOT-23-sized 74LVC1T45 to an alligator-clip friendly format.

Instead of laying out a PCB, fabbing a one-off, and soldering it together, I took a piece of cardboard and built a breakout board in under an hour. Furthermore, because I can bend cardboard, I can make my breakout board dual-purpose: I can add pins to it that make it breadboard-compatible, while having fold-up “wings” for alligator clips. Without the ability to fold up, the alligator clip extensions would block access to the breadboard connections. Below are some shots of the finished project.


Native comments plus on-board decoupling caps makes this simple to use, even with long alligator clips


DIP pins coming out the bottom side allow this to be used in a breadboard, too


SMT, DIP, and alligator clips all coexisting in a single breakout — easy to do with paper!

The first step in making a paper circuit is to grab a suitable piece of cardboard. I’ve come to really enjoy the cardboard used to make high-quality mats for picture framing. It’s about 1.3-1.4mm thick, which is fairly similar to FR-4 thickness, and its laminate structure means you can score one side and make accurate folds into the third dimension. The material is also robust to soldering temperatures, and its dense fiber construction and surface coating keeps the paper surface intact when pulling off mild adhesives, like the ones found on copper tape.

I’ll then cut out a square about the size I think it needs to be. I’ll usually cut a little larger, because it’s trivial to trim it back later on, but janky to tape on an extension if it’s too small.

Then, I lay the components on top and sketch a layout – this one’s pretty simple, I just note where I want the SOT-23 to go, and where the breakouts should run to.

Once I’m happy with the sketch, I’ll lay down copper tape, solder on the components, and then fold/bend the breakout into the final shape.

The trickiest and most important technique to master is how to mate the copper tape to the tiny pins of the SOT-23 (or other SMT) package. I use a trick that Dr. Jie Qi taught me, which is to cut a set of triangular notches into the tip of a wider piece of copper tape of roughly the right pitch. The triangular shape lets you adjust the size of the landing pad by simply changing the gap between the two ends of the tape, alleviating the need for precise alignment. Then, once the component is soldered to the wide piece of copper tape, you take a knife and cut the tape into individual traces – voilà, an SMT breakout is born!

A lot of this is better shown not told, so I’ve created a little video, below, that walks you through the entire process of building the breakout.

Try Something Different, and You Might be Rewarded!
Paper as an electrical engineering material is something I would never have thought of on my own – I grew up prototyping with breadboards and wire-wrap, and I was prejudiced against paper as a cheap, throwaway material that I incorrectly thought was too flammable to solder. Instead, I spent hundreds of dollars on breadboards and wire wrap sockets, when I could have made do with much cheaper materials. Indeed, there is an irrational psychology that regards expensive things as inherently better than cheap things, which means cheap options are often overlooked in the search for solutions to hard problems.

But this is why it’s important to collaborate with experts outside your normal field of expertise – the further outside, the better. In addition to being a great engineer, Jie Qi is a prodigious artist. Through our Chibitronics collaboration, she’s added so much more depth and dimension to my world on so many fronts. She’s imparted upon me invaluable gifts of skills and perspectives that I would never have developed otherwise.

It’s my hope that by sharing a little more about paper electronics, I can bring a fresh perspective on old problems while broadening awareness and getting more users to improve upon the basics. After all, this is a new area, and we’re just starting to explore the possibilities.

Interested in hacking paper electronics? Check out the Chibitronics Creative Coding Kit, and the Love to Code product line. It’s a gentle introduction to paper electronics targeted at newcomers, but it’s also open source, so you can take it as far as your imagination can go — hook up a JTAG box, build the OS, and get hacking! Get 30% off the Creative Coding Kit with the TRY-LTC-18 coupon code until June 30, 2018!


Quick edit: some basic techniques on using copper tape are documented at Chibitronics’ Copper Tape Chronicles. It’s a small compilation of videos like the one below:

Also, here’s a handy tip on how to keep copper tape from falling off the roll:

by bunnie at April 24, 2018 12:20 PM

April 23, 2018

Bunnie Studios

Name that Ware, April 2018

The Ware for April 2018 is shown below:

It’s a simple ware, but you could say I’m rather personally attached to it. Prize to the funniest/most creative story about this ware :)

by bunnie at April 23, 2018 01:40 PM

Winner, Name that Ware March 2018

The Ware for March 2018 is a Fuso 150 Fish Finder. Mangel hit the nail on the head with this guess — excellent work, as always! email me for your prize :)

by bunnie at April 23, 2018 01:39 PM

April 22, 2018

Harald Welte

osmo-fl2k - Using USB-VGA dongles as SDR transmitter

Yesterday, during OsmoDevCon 2018, Steve Markgraf released osmo-fl2k, a new Osmocom member project which enables the use of FL2000 USB-VGA adapters as ultra-low-cost SDR transmitters.

How does it work?

A major part of any VGA card has always been a rather fast DAC which generates the 8-bit analog values for (each) red, green and blue at the pixel clock. Given that fast DACs were very rare/expensive (and still are to some extent), the idea of (ab)using the VGA DAC to transmit radio has been followed by many earlier, mostly proof-of-concept projects, such as Tempest for Eliza in 2001.

However, with osmo-fl2k, for the first time it was possible to completely disable the horizontal and vertical blanking, resulting in a continuous stream of pixels (samples). Furthermore, as the supported devices have no frame buffer memory, the samples are streamed directly from host RAM.

As most USB-VGA adapters appear to have no low-pass filters on their DAC outputs, it is possible to use any of the harmonics to transmit signals at much higher frequencies than normally possible within the baseband of the (max) 157 Mega-Samples per seconds that can be achieved.

osmo-fl2k and rtl-sdr

Steve is the creator of the earlier, complementary rtl-sdr software, which since 2012 transforms USB DVB adapters into general-purpose SDR receivers.

Today, six years later, it is hard to think of where SDR would be without rtl-sdr. Reducing the entry cost of SDR receivers nearly down to zero has done a lot for democratization of SDR technology.

There is hence a big chance that his osmo-fl2k project will attain a similar popularity. Having a SDR transmitter for as little as USD 5 is an amazing proposition.

free riders

Please keep in mind that Steve has done rtl-sdr just for fun, to scratch his own itch and for the "hack value". He chose to share his work with the wider public, in source code, under a free software license. He's a very humble person, he doesn't need to stand in the limelight.

Many other people since have built a business around rtl-sdr. They have grabbed domains with his project name, etc. They are now earning money based on what he has done and shared selflessly, without ever contributing back to the pioneering developers who brought this to all of us in the first place.

So, do we want to bet if history repeats itself? How long will it take for vendors showing up online advertising the USB VGA dongles as "SDR transmitter", possibly even with a surcharge? How long will it take for them to include Steve's software without giving proper attribution? How long until they will violate the GNU GPL by not providing the complete corresponding source code to derivative versions they create?

If you want to thank Steve for his amazing work

  • reach out to him personally
  • contribute to his work, e.g.
  • help to maintain it
  • package it for distributions
  • send patches (via osmocom-sdr mailing list)
  • register an osmocom.org account and update the wiki with more information

And last, but not least, carry on the spirit of "hack value" and democratization of software defined radio.

Thank you, Steve! After rtl-sdr and osmo-fl2k, it's hard to guess what will come next :)

by Harald Welte at April 22, 2018 10:00 PM

March 31, 2018

Bunnie Studios

Name that Ware, March 2018

The Ware for March 2018 is shown below.

Thanks to Charl for contributing this ware!

by bunnie at March 31, 2018 03:15 PM

Winner, Name that Ware February 2018

The ware for February 2018 is an Ethernet card by Digitel, a Brazilian manufacturer, circa 1992. Brazil is an interesting market because protectionist trade measures made import electronics very expensive. The nominal theory, as it was explained to me, was to protect and encourage local industries, thus creating and maintaining high-paying local jobs. I had never seen a piece of electronics from Brazil, but indeed, many of the circuit board’s components bear company logos I had never seen before and a Brazilian country of origin. While at least facially it seems the trade policies created local jobs, a comparison of this card to its contemporaries outside Brasil — such as this 1992-vintage SMC “Elite 16” Ethernet card featured at vintagecomputer.net — gives a hint at how these policies might have also impeded the progress of technology.

While 0x3d named the ware almost immediately, I really appreciated the cultural insight that Paulo Peres shared about the ware. For example, the fact that the ROM labeled MAQUEST is probably “MAQuina de ESTado” (state machine) and could have been a hack at the time to use locally-produced components to substitute for imported components. Even though in a free market a ceramic EEPROM + 74-series registers would be much more expensive than a PAL, the fact that the EEPROM and registers were produced in Brazil would have made the combo cheaper than an imported PAL once the trade tariffs were factored in. So congrats Paulo, email me for your prize! Although, my understanding is the trade barriers are still in place to this day, so maybe sending you something from overseas would cost you much more duty than it’s worth if you’re located in Brazil… :-O

by bunnie at March 31, 2018 03:15 PM

March 29, 2018

Harald Welte

udtrace - Unix domain socket tracing

When developing applications that exchange data over sockets, every so often you'd like to analyze exactly what kind of data is exchanged over the socket.

For TCP/UDP/SCTP/DCCP or other IP-based sockets, this is rather easy by means of libpcap and tools like tcpdump, tshark or wireshark. However, for unix domain socket, unfortunately no such general capture/tracing infrastructure exists in the Linux kernel.

Interestingly, even after searching for quite a bit I couldn't find any existing tools for this. This is surprising, as unix domain sockets are used by a variety of programs, from sql servers to bind8 ndc all the way to the systemctl tool to manage systemd.

In absence of any kernel support, the two technologies I can think of to implement this is either systemtap or a LD_PRELOAD wrapper.

However, I couldn't find an example for using either of those two to get traces of unix domain soocket communications.

Ok, so I get to write my own. My first idea hence was to implement something based on top of systemtap, the Linux kernel tracing framework. Unfortunately, systemtap was broken in Debian unstable (which I use for decades) at the time, so I went back to the good old LD_PRELOAD shim library / wrapper approach.

The result is called udtrace and can be found at

git clone git://git.gnumonks.org/udtrace

or alternatively via its github mirror.

Below is a copy+paste of its README file. Let's hope this tool is useful to other developers, too:

udtrace - Unix Domain socket tracing

This is a LD_PRELOAD wrapper library which can be used to trace the data sent and/or received via unix domain sockets.

Unlike IP based communication that can be captured/traced with pcap programs like tcpdump or wireshark, there is no similar mechanism available for unix domain sockets.

This LD_PRELOAD library intercepts the C library function calls of dynamically linked programs. It will detect all file descriptors representing unix domain sockets and will then print traces of all data sent/received via the socket.

Usage

Simply build libudtrace.so using the make command, and then start your to-be-traced program with

LD_PRELOAD=libudtrace.os

e.g.

LD_PRELOAD=libudtrace.so systemctl status

which will produce output like this:

>>> UDTRACE: Unix Domain Socket Trace initialized (TITAN support DISABLED)
>>> UDTRACE: Adding FD 4
>>> UDTRACE: connect(4, "/run/dbus/system_bus_socket")
4 sendmsg W 00415554482045585445524e414c20
4 sendmsg W 3331333033303330
4 sendmsg W 0d0a4e45474f54494154455f554e49585f46440d0a424547494e0d0a
[...]

Output Format

Currently, udtrace will produc the following output:

At time a FD for a unix domain socket is created:

>>> UDTRACE: Adding FD 8

At time a FD for a unix domain socket is closed:

>>> UDTRACE: Removing FD 8

At time a FD for a unix domain socket is bound or connected:

>>> UDTRACE: connect(9, "/tmp/mncc")

When data is read from the socket:

9 read R 00040000050000004403000008000000680000001c0300002c03000000000000

When data is written to the socket:

9 write W 00040000050000004403000008000000680000001c0300002c03000000000000
Where
  • 9 is the file dsecriptor on which the event happened
  • read/write is the name of the syscall, could e.g. also be sendmsg / readv / etc.
  • R|W is Read / Write (from the process point of view)
  • followed by a hex-dump of the raw data. Only data successfully written (or read) will be printed, not the entire buffer passed to the syscall. The rationale is to only print data that was actually sent to or received from the socket.

TITAN decoder support

Getting hex-dumps is nice and fine, but normally one wants to have a more detailed decode of the data that is being passed on the socket.

For TCP based protocols, there is wireshark. But most protocols on unix domain sockets don't follow inter-operable / public standards, so even if one was to pass the traces into wireshark somehow, there would be no decoder.

In the Osmocom project, we already had some type definitions and decoders for our protocols written in the TTCN-3 programming language, using Eclipse TITAN. In order to build those decoders fro MNCC and PCUIF, please use

make ENABLE_TITAN=1

when building the code.

Please note that this introduces a run-time dependency to libttcn3-dynamic.so, which is (at least on Debian GNU/Linux) not installed in a default library search path, so you will have to use something like:

LD_LIBRARY_PATH=/usr/lib/titan LD_PRELOAD=libudtrace.so systemctl status

by Harald Welte at March 29, 2018 10:00 PM

March 19, 2018

Bunnie Studios

An Intuitive Motor: IQ Control’s Serial-to-Position Module

Back when I was a graduate student, my advisor Tom Knight bestowed upon me many excellent aphorisms. One of them was, “just wrap a computer around it!” – meaning, rather than expending effort to build more perfect systems, wrap imperfection-correcting computers around imperfect systems.

An everyday example of this is the noise-cancelling headphone. Headphones offer imperfect noise cancellation, but by “wrapping a computer around it” – adding one or more microphones and a computer in the from of a digital signal processor (DSP) – the headphones are able to measure the ambient noise and drive the headphones with the exact inverse of the noise, thus cancelling out the surrounding noise and creating a more perfect listening experience.

Although the principle has found its way rapidly into consumer goods, it’s been very slow to find its way onto the engineer’s workbench. It’s the case of the cobbler’s children having no shoes.

In particular, it’s long bothered me that motors are so dumb. Motors are typically large, heavy, costly, power-hungry, and riddled with small mechanical imperfections. In comparison, microcontrollers are tiny, cheap, power-efficient, and could run software that trims imperfections while improving efficiency to the point where the motor + microcontroller combo is a win over a dumb motor on almost every metric. So why aren’t we wrapping a computer around every motor and just calling it a day?

Then one day a startup called IQ Motion Control showed me a demo of their smart motor, the IQ Position Module, at HAX in Shenzhen. My eyes instantly lit up – these guys have done it, and done it in a tasteful manner. This is the motor I’ve been waiting years for!

Meet the IQ Position Module
Simply put, the IQ Position Module is a brushless DC motor that talks serial and “thinks” at a higher level. I don’t have to design any complicated drive circuitry or buy a proprietary controller that talks some arcane or closed standard. I just plug an FTDI cable into my laptop, hook up power, clone a small git repo and I’m good to go.

Because of the microcontroller on the inside, the IQ Position Module can emulate a range of behaviors, from a simple stepper to a range of BLDC drive standards, but the real magic happens when you tell it where you want it to go and how fast, and it figures out the best way to get there.

“But wait”, you say, “my servos and brushed DC motors can do that just fine, I just control the pulse width!” This is true for crude and slow motion control applications, but if you really want to run at high speeds – like the ones achievable by a BLDC – you have to consider things like acceleration and deceleration profiles.

The video below shows what I mean. Here is a pre-production IQ Position Module that’s being commanded to turn once in two seconds; then twice, three times, and finally ten times in two seconds. The motor can go even faster, but the figurine I attached on top isn’t balanced well enough to do that safely. Notice how the speed “ramps up” and back down again, so that the motor stops with the figurine in precisely the same position at the end of every cycle, regardless of how fast I commanded the motor to turn.

That is magic.

And here’s a snippet of the core code used in the above demo, to give you an idea of how simple the API can be:

Just tell it where you want it, and by when — and the motor figures out an acceleration profile. Of course other parameters can be tweaked but the default behavior is reasonable enough!

A Motor That’s Also an Input Device
But wait! There’s more. Because this is a “direct drive” system, there’s no gears to shear. Anyone who has busted a geared servo motor by stalling or back-driving it knows what I mean. IQ Position Modules don’t have this problem. When you stop driving the IQ module – put it in a “coast” mode – it turns freely and without resistance.

This means the IQ motor doesn’t just “write” motion – it can “read” motion as well. Below is a video of a simple motion copy demo I cooked up in about an hour (including time spent refactoring the original API), where I implement bidirectional read/write of motion between two IQ Position Modules.

The ability to tolerate back-drive and also “go limp” is advantageous in robotics applications. Impact-oriented tasks — such as hammering a nail or kicking a ball — would rapidly degrade the teeth in a geared drive train. Furthermore, natural human motion incorporates the ability to go limp, such as the forward swing of a leg during walking. Finally, biological muscles are capable of applying a static force without changing position, such as when holding a cup on its sides without crushing it. Roboticists have developed a wide range of specialty actuators and techniques, including series elastic and variable stiffness actuators to address these scenarios. However, these mechanisms are often complicated and pricey.

The IQ Position Module’s lack of gearing means it’s back-drive tolerant, and it can apply an open-loop force without any risk of damage. This means you could, for example, use it to build a robot arm that can hammer a nail or pick up a cup. Robotic elements built using these would have far greater resilience to motion interference and impact forces than ones built using geared servos.

Having Fun with the IQ Position Module
While attending 34C3 back in December 2017, I managed to sit down for about an hour with my good friends Prof. Nadya Peek and Ole Steinhauer, and we built a 2-axis robot arm that could do kinesthetic learning through keyframing, using nothing more than two IQ Position Modules, a Dunkin’ Donuts box, a bunch of schwag stickers we stole from the FOSSASIA assembly, and the base plate of an old PS4 … because fail0verflow.

This was improvisational making at its best; we didn’t really plan the encounter so much as it emerged out of the chaos that is the Computer Congress. While Nadya was busy cutting, folding, and binding the cardboard into a 2-axis robot arm, Ole “joined” (#lötwat?!) together the power & serial connectors, and I furiously wrote the code that would do the learning and playback — while also doing my best to polish off a couple beers. Nadya methodically built one motion axis first, and we tested it; satisfied with the result, she built and stacked a second axis on top. With just a bit of tweaking and prodding we managed to pull off the demo below:

It’s a little janky, but given the limited materials and time frame for execution, it hints at the incredible promise that IQ Position Modules hold.

So, if you’ve ever wanted to dabble in robotics or motion control, but have been daunted by control theory and arcane driver protocols (like I’ve been), check out the IQ Position Module. They are crowdfunding now at CrowdSupply. I backed their campaign to reserve a few more Position Modules for my lab – by wrapping a smart computer around a dumb motor, they’ve created a widget that lets me go from code to physical position and back with a minimal amount of wiring and an accessible API.

Their current funding campaign heavily emphasizes the capabilities of their motor as a “better BLDC” for the lucrative drone market, and I respect their wisdom in focusing their campaign message around a single, economically significant vertical. A cardinal sin of marketing revolutionary tech is to sell it as a floor wax and a dessert topping — as painful as it may be, you have to pick just one message and push hard around it. However, I’m happy they are offering the IQ Position Module as part of the campaign, and enabling me to express my enthusiasm to the maker and robotics communities. I’ve waited too long to have a motor with this capability in my toolbox — finally, the cobbler’s children has shoes!

by bunnie at March 19, 2018 12:49 PM

March 12, 2018

Dieter Spaar

Communication on a FlexRay bus

FlexRay is an automotive bus system which is not as common as the CAN bus, but is used in several car brands, e.g. BMW, Mercedes and Audi. I won't go into the details of FlexRay, there are several good introductions elsewhere.

What is needed to read the data on a FlexRay bus, or even better, actively participate in the communication (send your own data)? Compared to the CAN bus a FlexRay bus is complicated:

  • Sending and receiving on a CAN bus is simple, similar to serial (UART) communication: You basically only need to know the bus speed and you are done.
  • For a FlexRay bus you have to know about 50 more or less important parameters which define things like the length and number of the frames in the static segment or the frames in the dynamic segment.

If you are only interested in seeing the data on the FlexRay bus, than it is actually pretty simple, knowing the communication speed (typically 10 Mbit/s) is enough. There are several oscilloscopes and logic analyzers which can decode the FlexRay protocol. Such a decoder is simple, I use a few lines of AWK script to process the exported CSV file from a cheap logic analyzer to do so (AWK just to show how simple it is).

However if you also want to send data, you need to know nearly all the parameters, otherwise you would most certainly just disturb the bus communication. There are several (expensive) FlexRay analyzers available, which can help to solve this problem.

I wanted to find out if it is also possible to get this done with a relatively cheap (around 100 EUR) development board with a FlexRay interface. While I won't go into the details (maybe this is topic for an upcoming talk), I will just present my experimental setup:

I started with two ECUs (gateway and engine ECU) from a BMW with FlexRay. Those two ECUs are enough for a properly running FlexRay bus (each of those ECUs is a so called coldstart node, you need at least two of them to get the FlexRay bus up and running). To get the development board running with this setup you could start with coarse communication parameters (maybe from measurements with an oscilloscope) and fine-tune them until you can communicate (receive and send frames) without errors. This actually worked quite well.

The next setup were several ECUs from an Audi with FlexRay: I had the gateway, ACC radar and front camera. However only the gateway is a coldstart node, so the FlexRay bus would not start with only those ECUs. I could have bought a second coldstart node ECU (either ABS or airbag), however those ECUs for this specific car are rather expensive. Additionally I wanted to see if it is possible to program the development board as a coldstart node, so I decided to go this way. The problem now is that you don't have a running FlexRay bus to get your first estimation of the communication parameters: the single coldstart node trying to start the bus will only give you a few of them (basically you only have one frame from the static segment). The communication parameters from the BMW won't help also, Audi uses something else (only the 10 Mbit/s bus speed and the 5 ms cycle time are the same). Again I skip the details, but all problems could be resolved and the development board acts as a coldstart node to get the bus running and of course can also properly communicate on the bus.

Lessons learned: You don't necessarily need expensive tools to solve a problem which seems complicated on the first look. If you are willing to spend some time, you can succeed with rather cheap equipment. The additional benefit is that you learn a lot from such an approach.

March 12, 2018 01:00 AM

March 06, 2018

Harald Welte

Report from the Geniatech vs. McHardy GPL violation court hearing

Today, I took some time off to attend the court hearing in the appeal hearing related to a GPL infringement dispute between former netfilter colleague Partrick McHardy and Geniatech Europe

I am not in any way legally involved in the lawsuit on either the plaintiff or the defendant side. However, as a fellow (former) Linux kernel developer myself, and a long-term Free Software community member who strongly believes in the copyleft model, I of course am very interested in this case.

History of the Case

This case is about GPL infringements in consumer electronics devices based on a GNU/Linux operating system, including the Linux kernel and at least some devices netfilter/iptables. The specific devices in question are a series of satellite TV receivers built by a Shenzhen (China) based company Geniatech, which is represented in Europe by Germany-based Geniatech Europe GmbH.

The Geniatech Europe CEO has openly admitted (out of court) that they had some GPL incompliance in the past, and that there was failure on their part that needed to be fixed. However, he was not willing to accept an overly wide claim in the preliminary injunction against his company.

The history of the case is that at some point in July 2017, Patrick McHardy has made a test purchase of a Geniatech Europe product, and found it infringing the GNU General Public License v2. Apparently no source code (and/or written offer) had been provide alongside the binary - a straight-forward violation of the license terms and hence a violation of copyright. The plaintiff then asked the regional court of Cologne to issue a preliminary injunction against the defendant, which was granted on September 8th,2017.

In terms of legal procedure, in Germany, when a plaintiff applies for a preliminary injunction, it is immediately granted by the court after brief review of the filing, without previously hearing the defendant in an oral hearing. If the defendant (like in this case) wishes to appeal the preliminary injunction, it files an appeal which then results in an oral hearing. This is what happened, after which the district court of cologne (Landgericht Koeln) on October 20, 2017 issued ruling 14 O 188/17 partially upholding the injunction.

All in all, nothing particularly unusual about this. There is no dispute about a copyright infringement having existed, and this generally grants any of the copyright holders the right to have the infringing party to cease and desist from any further infringement.

However, this injunction has a very wide scope, stating that the defendant was to cease and desist not only from ever publishing, selling, offering for download any version of Linux (unless being compliant to the license). It furthermore asked the defendant to cease and desist

  • from putting hyperlinks on their website to any version of Linux
  • from asking users to download any version of Linux

unless the conditions of the GPL are met, particularly the clauses related to providing the complete and corresponding source code.

The appeals case at OLG Cologne

The defendant now escalated this to the next higher court, the higher regional court of Cologne (OLG Koeln), asking to withdraw the earlier ruling of the lower court, i.e. removing the injunction with its current scope.

The first very positive surprise at the hearing was the depth in which the OLG court has studied the subject matter of the dispute prior to the hearing. In the many GPL related court cases that I witnessed so far, it was by far the most precise analysis of how Linux kernel development works, and this despite the more than 1000 pages of filings that parties had made to the court to this point.

Just to give you some examples:

  • the court understood that Linux was created by Linus Torvalds in 1991 and released under GPL to facilitate the open and collaborative development
  • the court recognized that there is no co-authorship / joint authorship (German: Miturheber) in the Linux kernel as a whole, as it was not a group of people planning+developing a given program together, but it is a program that has been released by Linus Torvalds and has since been edited by more than 15.000 developers without any "grand joint plan" but rather in successive iterations. This situation constitutes "editing authorship" (German: Bearbeiterurheber)
  • the court further recognized that being listed as "head of the netfilter core team" or a "subsystem maintainer" doesn't necessarily mean that one is contributing copyrightable works. Reviewing thousands of patches doesn't mean you own copyright on them, drawing an analogy to an editorial office at a publisher.
  • the court understood there are plenty of Linux versions that may not even contain any of Patric McHardy's code (such as older versions)

After about 35 minutes of the presiding judge explaining the court's understanding of the case (and how kernel development works), it went on to summarize the summary of their internal elaboration at the court prior to the meeting.

In this summary, the presiding judge stated very clearly that they believe there is some merit to the arguments of the defendant, and that they would be inclined in a ruling favorable to the defendant based on their current understanding of the case.

He cited the following main reasons:

  • The Linux kernel development model does not support the claim of Patrick McHardy having co-authored Linux. In so far, he is only an editing author (Bearbeiterurheber), and not a co-author. Nevertheless, even an editing author has the right to ask for cease and desist, but only on those portions that he authored/edited, and not on the entire Linux kernel.
  • The plaintiff did not sufficiently show what exactly his contributions were and how they were forming themselves copyrightable works
  • The plaintiff did not substantiate what copyrightable contributions he has made outside of netfilter/iptables. His mere listing as general networking subsystem maintainer does not clarify what his copyrightable contributions were
  • The plaintiff being a member of the netfilter core team or even the head of the core team still doesn't support the claim of being a co-author, as netfilter substantially existed since 1999, three years before Patrick's first contribution to netfilter, and five years before joining the core team in 2004.

So all in all, it was clear that the court also thought the ruling on all of Linux was too far-fetching.

The court suggested that it might be better to have regular main proceedings, in which expert witnesses can be called and real evidence has to be provided, as opposed to the constraints of the preliminary procedure that was applied currently.

Some other details that were mentioned somewhere during the hearing:

  • Patrick McHardy apparently unilaterally terminated the license to his works in an e-mail dated 26th of July 2017 towards the defendant. According to the defendant (and general legal opinion, including my own position), this is in turn a violation of the GPLv2, as it only allowed plaintiff to create and publish modified versions of Linux under the obligation that he licenses his works under GPLv2 to any third party, including the defendant. The defendant believes this is abuse of his rights (German: Rechtsmissbraeuchlich).
  • sworn affidavits of senior kernel developer Greg Kroah-Hartman and current netfilter maintainer Pablo Neira were presented in support of some of the defendants claims. The contents of those are unfortunately not public, neither is the contents of the sworn affidavists presented by the plaintiff.
  • The defendant has made substantiated claims in his filings that Patrick McHardy would perform his enforcement activities not with the primary motivation of achieving license compliance, but as a method to generate monetary gain. Such claims include that McHardy has acted in more than 38 cases, in at least one of which he has requested a contractual penalty of 1.8 million EUR. The total amount of monies received as contractual penalties was quoted as over 2 million EUR to this point. Please note that those are claims made by the defendant, which were just reproduced by the court. The court has not assessed their validity. However, the presiding judge explicitly stated that he received a phone calls about this case from a lawyer known to him personally, who supported that large contractual penalties are being paid in other related cases.
  • One argument by the plaintiff seems to center around being listed as a general kernel networking maintainer until 2017 (despite his latest patches being from 2015, and those were netfilter only)

Withdrawal by Patrick McHardy

At some point, the court hearing was temporarily suspended to provide the legal representation of the plaintiff with the opportunity to have a Phone call with the plaintiff to decide if they would want to continue with their request to uphold the preliminary injunction. After a few minutes, the hearing was resumed, with the plaintiff withdrawing their request to uphold the injunction.

As a result, the injunction is now withdrawn, and the plaintiff has to bear all legal costs (court fees, lawyers costs on both sides).

Personal Opinion

For me, this is all of course a difficult topic. With my history of being the first to enforce the GNU GPLv2 in (equally German) court, it is unsurprising that I am in favor of license enforcement being performed by copyright holders.

I believe individual developers who have contributed to the Linux kernel should have the right to enforce the license, if needed. It is important to have distributed copyright, and to avoid a situation where only one (possibly industry friendly) entity would be able to take [legal] action.

I'm not arguing for a "too soft" approach. It's almost 15 years since the first court cases on license violations on (embedded) Linux, and the fact that the problem still exists today clearly shows the industry is very far from having solved a seemingly rather simple problem.

On the other hand, such activities must always be oriented to compliance, and compliance only. Collecting huge amounts of contractual penalties is questionable. And if it was necessary to collect such huge amounts to motivate large corporations to be compliant, then this must be done in the open, with the community knowing about it, and the proceeds of such contractual penalties must be donated to free software related entities to prove that personal financial gain is not a motivation.

The rumors of Patrick performing GPL enforcement for personal financial gain have been around for years. It was initially very hard for me to believe. But as more and more about this became known, and Patrick would refuse to any contact requests by his former netfilter team-mates as well as the wider kernel community make it hard to avoid drawing related conclusions.

We do need enforcement, both out of court and in court. But we need it to happen out of the closet, with the community in the picture, and without financial gain to individuals. The "principles of community oriented enforcement" of the Software Freedom Conservancy as well as the more recent (but much less substantial) kernel enforcement statement represent the most sane and fair approach for how we as a community should deal with license violations.

So am I happy with the outcome? Not entirely. It's good that an over-reaching injunction was removed. But then, a lot of money and effort was wasted on this, without any verdict/ruling. It would have been IMHO better to have a court ruling published, in which the injunction is substantially reduced in scope (e.g. only about netfilter, or specific versions of the kernel, or specific products, not about placing hyperlinks, etc.). It would also have been useful to have some of the other arguments end up in a written ruling of a court, rather than more or less "evaporating" in the spoken word of the hearing today, without advancing legal precedent.

Lessons learned for the developer community

  • In the absence of detailed knowledge on computer programming, legal folks tend to look at "metadata" more, as this is what they can understand.
  • It matters who has which title and when. Should somebody not be an active maintainer, make sure he's not listed as such.
  • If somebody ceases to be a maintainer or developer of a project, remove him or her from the respective lists immediately, not just several years later.
  • Copyright statements do matter. Make sure you don't merge any patches adding copyright statements without being sure they are actually valid.

Lessons learned for the IT industry

  • There may be people doing GPL enforcement for not-so-noble motives
  • Defending yourself against claims in court can very well be worth it, as opposed to simply settling out of court (presumably for some money). The Telefonica case in 2016 <>_ has shown this, as has this current Geniatech case. The legal system can work, if you give it a chance.
  • Nevertheless, if you have violated the license, and one of the copyright holders makes a properly substantiated claim, you still will get injunctions granted against you (and rightfully so). This was just not done in this case (not properly substantiated, scope of injunction too wide/coarse).

Dear Patrick

For years, your former netfilter colleagues and friends wanted to have a conversation with you. You have not returned our invitation so far. Please do reach out to us. We won't bite, but we want to share our views with you, and show you what implications your actions have not only on Linux, but also particularly on the personal and professional lives of the very developers that you worked hand-in-hand with for a decade. It's your decision what you do with that information afterwards, but please do give us a chance to talk. We would greatly appreciate if you'd take up that invitation for such a conversation. Thanks.

by Harald Welte at March 06, 2018 11:00 PM

February 27, 2018

Bunnie Studios

Name that Ware, February 2018

The Ware for February 2018 is shown below.

Ware courtesy of Hernandi Krammes!

Every board designer leaves a thumbprint on their ware — and this one is from a region I had never previously seen a ware from before. So while probably easy to guess the function, I still appreciated it for the small, unique details.

by bunnie at February 27, 2018 08:20 AM

Winner, Name that Ware January 2018

The Ware for January 2018 is a front panel VFD/switch controller board for an HP Laserjet 4+. Archels nailed it — I checked u19pb1996 in Google for hits and nothing came up, but maybe I was too hasty and typo’d the number when cross-checking the image. Anyways, this post is now the top hit for that part number :) Congrats, email me for your prize.

+1 to zebonaut’s comment about the firmware code from the early 90’s never needing an update, ever…they just don’t write software like they used to anymore! It’s funny to see the panic in the eyes of a modern software developer when you tell them a subsystem has no firmware update path, ever, and their code just has to work reliably from day one. And then you tell a hardware developer the same thing and they go “yah, so?”…there’s no such thing as a downloadable hardware update, of course the product ships complete, working & tested. And not only does the hardware have to work, it carries a warranty, unlike most software…

by bunnie at February 27, 2018 08:18 AM

February 11, 2018

Bunnie Studios

When More is Less: China’s Perception of the iPhone X “Notch”

I recently saw a Forbes article citing rumors that the iPhone X is being cancelled this summer. Assuming the article is correct, it claims that a “lack of interest in China” is the main reason for the relatively early cancellation of production. They’re hoping that 6.1″ and 6.5″ versions of their phone with a less pronounced Face ID notch would excite Chinese customers.

The notion of a “less pronounced” Face ID notch is what got me — Apple embracing the notch as iconic, and worth carrying forward to future models, rather than simply making the top bezel a bit larger and eliminating the notch altogether. Historically, Apple has taken a “less is more” strategy, meticulously eliminating even the tiniest design facets: replacing radii with splines, polishing off injection mold parting lines, even eliminating the headphone jack. Putting a notch on the iPhone feels a bit like watching a woman painstakingly apply face whitening cream day after day to remove tiny blemishes, and then don a red clown nose.

Like the red clown nose, the problem with pushing the notch is that anyone can put one on, should they decide it’s a feature they want to copy. Xobs recently showed me an app on his Xiaomi Mix 2 that does exactly that. Below is what his Xiaomi Mix 2 looks like normally.

It’s got a screen that goes right up to the top bezel, without a notch.

Interestingly, there’s an app you can run called “X out of 10” that simply draws in the notch (including subtle details like a simulated camera lens). Here’s the app in the off state:

And now in the on state:

Once activated and given permission to draw over other apps, Xiaomi Mix 2 users can don the red clown nose and experience the full glory of the iconic Apple notch all the time:

This glass-half-empty situation is a parable for design leadership versus market perception: if a market previously lacked a smartphone with a minimal top bezel, the notch is perceived as “How innovative! I’ve got extra pixels to the left and right of my earpiece/camera assembly!”. But once a market has seen a smartphone with minimal top bezel, the notch turns into “Hey where did my pixels go? What’s this notch doing here?”. It’s a case where the additional design feature is seen as a loss of function, not a gain.

Thus it will be interesting to see if Apple’s bet to introduce a phone with a larger screen that can compete head to head in China against the likes of the Xiaomi Mix 2’s 6″ screen will pay out, especially if Apple retains the notch.

Of course, as the design space for phones becomes more and more crowded, Apple’s room to maneuver becomes increasingly limited. The minimalist design space is winner-takes-all: the first company to elegantly remove a design facet wins the minimalism race, and now that Xiaomi has planted a flag in the bezel-less top space, it may be that Apple has no option but to sport the top-notch, or run the risk of being seen as copying a Chinese company’s design language.

Edit (added Feb 12, 17:43 SGT):

Several comments have been made about the iPhone X still having a greater amount of screen real estate than the Xiaomi Mix 2.

To clarify, the key point of the article isn’t about comparing active area. It’s about running out of options to place a sensor cluster, because the smartphone design space has gotten a lot more competitive. To spell it out explicitly, there are three main ways this can play out:

    1) Apple can’t hide a camera underneath the display, and so there always has to be a “dark area” that’s an affordance for the camera (and more significantly, the multitude of sensors that comprise FaceID).
    2) Apple (or perhaps someone else!) figures out how to hide a camera under the display and creates a true bezel-to-bezel phone.
    3) Apple convinces us all that the notch is truly iconic and it’s hailed as one of the greatest design innovations of this decade (hey, they did it for the headphone jack…).

In the case of 1 (Apple can’t hide the sensor cluster), these are their options:

    (a) Continue to push the top notch as iconic – status quo
    (b) Lose the notch by increasing top bezel area for sensor cluster — that’s “taking a step backward” – so not really an option
    (c) Move sensor cluster to the bottom, with no notch. This is copying the Xiaomi Mix 2 almost exactly – so not an option
    (d) Continue to push the notch as iconic, but put it on the bottom. Risks the top-half of the phone looking too much like a Xiaomi Mix 2 – so probably not an option

So in the race for minimalism, because Xiaomi has “claimed” the minimal bezel top-half design space, Apple has far fewer options for backing out of the notch, should it be perceived by the market as a loss of real estate, rather than a gain. But this is the world Apple has created for themselves, by patenting and litigating over the rounded rectangle as a phone design.

In the case of 2 (Apple figures out how to hide all the sensors), Apple can really win the minimalist design space if they can do it without reducing functionality. However, if they could have done this, I think they would have done it for the X. They certainly have the cash to throw the equivalent budget of SpaceX’s Falcon rocket program into eliminating that notch. Indeed, perhaps in a year or two Apple will come out with some crazy fiber optic wave guide assembly with holographic lenses to wrap light around the bezel into a sensor assembly stashed in the body of the phone. I wouldn’t put it beyond them.

But until then, it seems Apple is looking at option (1) for the next generation at least, and the point of this article is that the competition has robbed Apple of at least two options elegantly to back out of the notch and create a phone with greater appeal to markets like China.

by bunnie at February 11, 2018 12:30 PM

ICE40 for Novena

For any and all Novena users, a quick note: Philipp Gühring is organizing a production run of ICE40 FPGA add-in cards for people who want a 100% open software stack for making FPGA bitfiles. Register your interest in the production run by visiting his website!

by bunnie at February 11, 2018 09:08 AM

February 01, 2018

Bunnie Studios

Spectre/Meltdown Pits Transparency Against Liability: Which is More Important to You?

There is a lot of righteous anger directed toward Intel over CPU bugs that were revealed by Spectre/Meltdown. I agree that things could have been handled better, particularly with regards to transparency and the sharing of information among the relevant user communities that could have worked together to deploy effective patches in a timely fashion. People also aren’t wrong that consumer protection laws obligate manufacturers to honor warranties, particularly when a product is not fit for use as represented, if it contains defective material or workmanship, or fails to meet regulatory compliance.

However, as an open source hardware optimist, and someone who someday aspires to see more open source silicon on the market, I want to highlight that demanding Intel return, exchange, or offer rebates on CPUs purchased within a reasonable warranty period is entirely at odds with demands that Intel act with greater transparency in sharing bugs and source code.

Transparency is Easy When There’s No Penalty for Bugs

It’s taken as motherhood and apple pie in the open source software community that transparency leads to better products. The more eyes staring at a code base, the more bugs that can be found and patched. However, a crucial difference between open source software and hardware is that open source software carries absolutely no warranty. Even the most minimal, stripped down OSS licenses stipulate that contributors carry no liability. For example, the BSD 2-clause license has 189 words, of which 116 (60%) are dedicated to a “no warranty” clause – and all in caps, in case you weren’t paying attention. The no-warranty clause is so core to any open source license it doesn’t even count as a clause in the 2-clause license.

Of course contributors have no liability: this lack of liability is fundamental to open source. If people could sue you for some crappy code you you pushed to github years ago, why would you share anything? Github would be a ticking time bomb of financial ruin for every developer.

It’s also not about code being easier to patch than hardware. The point is that you don’t have to patch your code, even if you could. Someone can file a bug against you, and you have the legal right to ignore it. And if your code library happens to contain an overflow bug that results in a house catching fire, you walk away scot-free because your code came with no warranty of fitness for any purpose whatsoever.

Oohh, Shiny and New!

Presented a bin of apples, most will pick a blemish-free fruit from the bushel before heading to the check-out counter. Despite the knowing the reality of nature – that every fruit must grow from a blossom under varying conditions and hardships – we believe our hard-earned money should only go toward the most perfect of the lot. This feeling is so common sense that it’s codified in the form of consumer protection laws and compulsory warranties.

This psychology extends beyond obvious blemishes, to defects that have no impact on function. Suppose you’re on the market to buy a one-slot toaster. You’re offered two options: a one-slot toaster, and a two-slot toaster but with the left slot permanently and safely disabled. Both are exactly the same price. Which one do you buy?

Most people would buy the toaster with one slot, even though the net function might be identical to the two-slot version where one slot is disabled. In fact, you’d probably be infuriated and demand your money back if you bought the one-slot toaster, but opened the box to find a two-slot toaster with one slot disabled. We don’t like the idea of being sold goods that have anything wrong with them, even if the broken piece is irrelevant to performance of the device. It’s perceived as evidence of shoddy workmanship and quality control issues.

News Flash: Complex Systems are Buggy!

Hold your breath – I’d wager that every computer you’ve bought in the past decade has broken parts inside of them, almost exactly like the two-slot toaster with one slot permanently disabled. There’s the set of features that were intended to be in your chips – and there’s the subset of series of features that finally shipped. What happened to the features that weren’t shipped? Surely, they did a final pass on the chip to remove all that “dead silicon”.

Nope – most of the time those partially or non-functional units are simply disabled. This ranges from blocks of cache RAM, to whole CPU cores, to various hardware peripherals. Patching a complex chip design can cost millions of dollars and takes weeks or even months, so no company can afford to do a final “clean-up” pass to create a “perfect” design. To wit, manufacturers never misrepresent the product to consumers – if half the cache was available, the spec sheet would simply report the cache size as 128kB instead of 256kB. But surely some customers would have complained bitterly if they knew of the defect sold to them.

Despite being chock full of bugs, vendors of desktop CPUs or mobile phone System on Chips (SoCs) rarely disclose these bugs to users – and those that do disclose almost always disclose a limited list of public bugs, backed by an NDA-only list of all the bugs. The top two reasons cited for keeping chip specs secret are competitive advantage and liability, and I suspect in reality, it’s the latter that drives the secrecy, because the crappier the chipset, the more likely the specs are under NDA. Chip vendors are deathly afraid users will find inconsistencies between the chip’s actual performance and the published specs, thus triggering a recall event. This fear may seem more rational if you consider the magnitude of Intel’s FDIV bug recall ($475 million in 1994).


This is a pretty typical list of SoC bugs, known as “errata”. If your SoC’s errata is much shorter than this, it’s more likely due to bugs not being disclosed than there actually being less bugs.

If you Want Messages, Stop Shooting the Messengers

Highly esteemed and enlightened colleagues of mine are strongly of the opinion that Intel should reimburse end users for bugs found in their silicon; yet in the same breath, they complain that Intel has not been transparent enough. The point that has become clear to me is that consumers, even open-source activists, are very sensitive to imperfections, however minor. They demand a “perfect” machine; if they spend $500 on a computer, every part inside better damn well be perfect. And so starts the vicious cycle of hardware manufacturers hiding all sorts of blemishes and shortcomings behind various NDAs, enabling them to bill their goods as perfect for use.

You can’t have it both ways: the whole point of transparency is to enable peer review, so you can find and fix bugs more quickly. But if every time a bug is found, a manufacturer had to hand $50 to every user of their product as a concession for the bug, they would quickly go out of business. This partially answers the question why we don’t see open hardware much beyond simple breakout boards and embedded controllers: it’s far too risky from a liability standpoint to openly share the documentation for complex systems under these circumstances.

To simply say, “but hardware manufacturers should ship perfect products because they are taking my money, and my code can be buggy because it’s free of charge” – is naïve. A modern OS has tens of millions of lines of code, yet it benefits from the fact that every line of code can be replicated perfectly. Contrast to a modern CPU with billions of transistors, each with slightly different electrical characteristics. We should all be more surprised that it took so long for a major hardware bug to be found, than the fact that one was ever found.

Complex systems have bugs. Any system with primitives measured in the millions or billions – be it lines of code, rivets, or transistors – is going to have subtle, if not blatant, flaws. Systems simple enough to formally verify are typically too simple to handle real-world tasks, so engineers must rely on heuristics like design rules and lots and lots of hand-written tests.

There will be bugs.

Realities of the Open Hardware Business

About a year ago, I had a heated debate with a SiFive founder about how open they can get about their documentation. SiFive markets the RISC-V CPU, billed as an “open source CPU”, and many open source enthusiasts got excited about the prospect of a fully-open SoC that could finally eliminate proprietary blobs from the boot chain and ultimately through the same process of peer review found in the open source software world, yield a more secure, trustable hardware environment.

However, even one of their most ardent open-source advocates pushed back quite hard when I suggested they should share their pre-boot code. By pre-boot code, I’m not talking about the little ROM blob that gets run after reset to set up your peripherals so you can pull your bootloader from SD card or SSD. That part was a no-brainer to share. I’m talking about the code that gets run before the architecturally guaranteed “reset vector”. A number of software developers (and alarmingly, some security experts) believe that the life of a CPU begins at the reset vector. In fact, there’s often a significant body of code that gets executed on a CPU to set things up to meet the architectural guarantees of a hard reset – bringing all the registers to their reset state, tuning clock generators, gating peripherals, and so forth. Critically, chip makers heavily rely upon this pre-boot code to also patch all kinds of embarrassing silicon bugs, and to enforce binning rules.

The gentleman with whom I was debating the disclosure of pre-boot code adamantly held that it was not commercially viable to share the pre-boot code. I didn’t understand his point until I witnessed open-source activists en masse demanding their pound of flesh for Intel’s mistakes.

As engineers, we should know better: no complex system is perfect. We’ve all shipped bugs, yet when it comes to buying our own hardware, we individually convince ourselves that perfection is a reasonable standard.

The Choice: Truthful Mistakes or Fake Perfection?

The open source community could use the Spectre/Meltdown crisis as an opportunity to reform the status quo. Instead of suing Intel for money, what if we sue Intel for documentation? If documentation and transparency have real value, then this is a chance to finally put that value in economic terms that Intel shareholders can understand. I propose a bargain somewhere along these lines: if Intel releases comprehensive microarchitectural hardware design specifications, microcode, firmware, and all software source code (e.g. for AMT/ME) so that the community can band together to hammer out any other security bugs hiding in their hardware, then Intel is absolved of any payouts related to the Spectre/Meltdown exploits.

This also sets a healthy precedent for open hardware. In broader terms, my proposed open hardware bargain is thus: Here’s the design source for my hardware product. By purchasing my product, you’ve warranted that you’ve reviewed the available design source and decided the open source elements, as-is, are fit for your application. So long as I deliver a product consistent with the design source, I’ve met my hardware warranty obligation on the open source elements.

In other words, the open-source bargain for hardware needs to be a two-way street. The bargain I set forth above:

  • Rewards transparency with indemnity against yet-to-be-discovered bugs in the design source
  • Burdens any residual proprietary elements with the full liability of fitness for purpose
  • Simultaneously conserves a guarantee that a product is free from defects in materials and workmanship in either case

The beauty of this bargain is it gives a real economic benefit to transparency, which is exactly the kind of wedge needed to drive closed-source silicon vendors to finally share their full design documentation, with little reduction of consumer protection.

So, if we really desire a more transparent, open world in hardware: give hardware makers big and small the option to settle warranty disputes for documentation instead of cash.

Author’s Addendum (added Feb 2 14:47 SGT)
This post has 2 aspects to it:

The first is whether hardware makers will accept the offer to provide documentation in lieu of liability.

The second, and perhaps more significant, is whether you would make the offer for design documentation in lieu of design liability in the first place. It’s important that companies who choose transparency be given a measurable economic advantage over those who choose obscurity. In order for the vicious cycle of proprietary hardware to be broken, both consumer and producer have to express a willingness to value openness.

by bunnie at February 01, 2018 05:32 PM

January 29, 2018

Bunnie Studios

Name that Ware, January 2018

The Ware for January 2018 is shown below.

This side of the board might be a little too non-descript to make a solid guess, so if nobody gets it within a couple of weeks, I’ll push a picture of the other side.

Thanks to spida for handing me pictures of this well-photographed ware at 34C3!

by bunnie at January 29, 2018 11:00 AM

Winner, Name that Ware December 2017

The winner of Name that Ware Dec 2017 is Piotr! Congrats, email me for your prize. He nailed it very quickly as a 2000-series multimeter by Keithley. I figured since this is a classic meter found on many engineer’s benches, it would be named fairly quickly, even with a tightly cropped shot of the main board.

My decades-old Keithley 2000 finally gave up the ghost; something seems wrong in the input stage that’s preventing it from both doing autoranging and measuring negative voltages. A quick thermal scan showed a few transistors getting hot that shouldn’t be, and the self-test codes indicate that something may have gone very wrong with the core ADC pathway. I’ve set it aside to look at later and ended up picking up a Keithley 2110 as a stand-in replacement until I can afford a DMM7510. There’s some pretty nifty debugging tricks you can do with a 7.5-digit multimeter (it’s sensitive enough to differentiate between code execution paths on an MCU, or observe a battery’s self-discharge rate in real time), but at the moment I simply can’t afford one. Ah well — it’s always good to have goals!

by bunnie at January 29, 2018 11:00 AM

January 22, 2018

Dieter Spaar

Modify the VoIP provider of a Speedport ISDN Adapter

The Speedport ISDN Adapter is a relatively cheap VoIP-to-ISDN adapter from Deutsche Telekom. The drawback is that the adapter is "locked" to Deutsche Telekom and the user interface (web interface) is disabled.

Here is how you can access the web interface. While the adapter is already powered on, you have to press both the "Register" and "Reset" button at once for more than 20 seconds. This will temporarily enable a still limited web interface, just point your browser to the IP address of your ISDN adapter. The login password is the "device password" written on the bottom side of the case.

To fully enable the web interface you have to do a bit more. I use "curl" which is available for nearly all operating systems. You have to replace "12345678" with the device password and "192.168.1.2" with the IP address of your adapter.

curl -d "login_pwd=1&pws=12345678" "http://192.168.1.2/cgi-bin/login.cgi"
curl -d "password=12345678&debug_enable=1&uart_tx_eb=1" "http://192.168.1.2/cgi-bin/debug.cgi"

Now the web interface is fully functional and you can modify the settings, e.g. disable the TR-069 interface and enter the parameters for the VoIP account of your provider. "uart_tx_eb" enables the serial console, which offers a few debug commands. However you have to open the case to get access to the serial console.

January 22, 2018 01:00 AM

January 21, 2018

Dieter Spaar

More LTE Base Stations

Since running my first own LTE eNodeB in 2013, I acquired a few more. I now have one from NSN, Ericsson and Huawei. All of them are fully functional including the required Remote Radio Heads to actually transmit.

Operating an LTE eNodeB is not very complicated, these days it is even easier with software like NextEPC. The tricky part is setting up and configuring the LTE eNodeB because there is no standard to do so and every manufacturer has its own way. If you are lucky, you might get an already configured system and there is not much you have to adjust. If your system is new or the configuration has been erased, than it can get complicated, at least for the Ericsson eNodeB I have.

January 21, 2018 01:00 AM

December 31, 2017

Harald Welte

Osmocom Review 2017

As 2017 has just concluded, let's have a look at the major events and improvements in the Osmocom Cellular Infrastructure projects (i.e. those projects dealing with building protocol stacks and network elements for mobile network infrastructure.

I've prepared a detailed year 2017 summary at the osmocom.org website, but let me write a bit about the most note-worthy topics here.

NITB Split

Once upon a time, we implemented everything needed to operate a GSM network inside a single process called OsmoNITB. Those days are now gone, and we have separate OsmoBSC, OsmoMSC, OsmoHLR, OsmoSTP processes, which use interfaces that are interoperable with non-Osmocom implementations (which is what some of our users require).

This change is certainly the most significant change in the close-to-10-year history of the project. However, we have tried to make it as non-intrusive as possible, by using default point codes and IP addresses which will make the individual processes magically talk to each other if installed on a single machine.

We've also released a OsmoNITB Migration Guide, as well as our usual set of user manuals in order to help our users.

We'll continue to improve the user experience, to re-introduce some of the features lost in the split, such as the ability to attach names to the subscribers.

Testing

We have osmo-gsm-tester together with the two physical setups at the sysmocom office, which continuously run the latest Osmocom components and test an entire matrix of different BTSs, software configurations and modems. However, this tests at super low load, and it tests only signalling so far, not user plane yet. Hence, coverage is limited.

We also have unit tests as part of the 'make check' process, Jenkins based build verification before merging any patches, as well as integration tests for some of the network elements in TTCN-3. This is much more than we had until 2016, but still by far not enough, as we had just seen at the fall-out from the sub-optimal 34C3 event network.

OsmoCon

2017 also marks the year where we've for the first time organized a user-oriented event. It was a huge success, and we will for sure have another OsmoCon incarnation in 2018 (most likely in May or June). It will not be back-to-back with the developer conference OsmoDevCon this time.

SIGTRAN stack

We have a new SIGTRAN stakc with SUA, M3UA and SCCP as well as OsmoSTP. This has been lacking a long time.

OsmoGGSN

We have converted OpenGGSN into a true member of the Osmocom family, thereby deprecating OpenGGSN which we had earlier adopted and maintained.

by Harald Welte at December 31, 2017 11:00 PM

34C3 and its Osmocom GSM/UMTS network

At the 34th annual Chaos Communication Congress, a team of Osmocom folks continued the many years old tradition of operating an experimental Osmocom based GSM network at the event. Though I've originally started that tradition, I'm not involved in installation and/or operation of that network, all the credits go to Lynxis, neels, tsaitgaist and the larger team of volunteers surrounding them. My involvement was only to answer the occasional technical question and to look at bugs that show up in the software during operation, and if possible fix them on-site.

34C3 marks two significant changes in terms of its cellular network:

  • the new post-nitb Osmocom stack was used, with OsmoBSC, OsmoMSC and OsmoHLR
  • both an GSM/GPRS network (on 1800 MHz) was operated ,as well as (for the first time) an UMTS network (in the 850 MHz band)

The good news is: The team did great work building this network from scratch, in a new venue, and without relying on people that have significant experience in network operation. Definitely, the team was considerably larger and more distributed than at the time when I was still running that network.

The bad news is: There was a seemingly endless number of bugs that were discovered while operating this network. Some shortcomings were known before, but the extent and number of bugs uncovered all across the stack was quite devastating to me. Sure, at some point from day 2 onwards we had a network that provided [some level of] service, and as far as I've heard, some ~ 23k calls were switched over it. But that was after more than two days of debugging + bug fixing, and we still saw unexplained behavior and crashes later on.

This is such a big surprise as we have put a lot of effort into testing over the last years. This starts from the osmo-gsm-tester software and continuously running test setup, and continues with the osmo-ttcn3-hacks integration tests that mainly I wrote during the last few months. Both us and some of our users have also (successfully!) performed interoperability testing with other vendors' implementations such as MSCs. And last, but not least, the individual Osmocom developers had been using the new post-NITB stack on their personal machines.

So what does this mean?

  • I'm sorry about the sub-standard state of the software and the resulting problems we've experienced in the 34C3 network. The extent of problems surprised me (and I presume everyone else involved)
  • I'm grateful that we've had the opportunity to discover all those bugs, thanks to the GSM team at 34C3, as well as Deutsche Telekom for donating 3 ARFCNs from their spectrum, as well as the German regulatory authority Bundesnetzagentur for providing the experimental license in the 850 MHz spectrum.
  • We need to have even more focus on automatic testing than we had so far. None of the components should be without exhaustive test coverage on at least the most common transactions, including all their failure modes (such as timeouts, rejects, ...)

My preferred method of integration testing has been by using TTCN-3 and Eclipse TITAN to emulate all the interfaces surrounding a single of the Osmocom programs (like OsmoBSC) and then test both valid and invalid transactions. For the BSC, this means emulating MS+BTS on Abis; emulating MSC on A; emulating the MGW, as well as the CTRL and VTY interfaces.

I currently see the following areas in biggest need of integration testing:

  • OsmoHLR (which needs a GSUP implementation in TTCN-3, which I've created on the spot at 34C3) where we e.g. discovered that updates to the subscriber via VTY/CTRL would surprisingly not result in an InsertSubscriberData to VLR+SGSN
  • OsmoMSC, particularly when used with external MNCC handlers, which was so far blocked by the lack of a MNCC implementation in TTCN-3, which I've been working on both on-site and after returning back home.
  • user plane testing for OsmoMGW and other components. We currently only test the control plane (MGCP), but not the actual user plane e.g. on the RTP side between the elements
  • UMTS related testing on OsmoHNBGW, OsmoMSC and OsmoSGSN. We currently have no automatic testing at all in these areas.

Even before 34C3 and the above-mentioned experiences, I concluded that for 2018 we will pursue a test-driven development approach for all new features added by the sysmocom team to the Osmocom code base. The experience with the many issues at 34C3 has just confirmed that approach. In parallel, we will have to improve test coverage on the existing code base, as outlined above. The biggest challenge will of course be to convince our paying customers of this approach, but I see very little alternative if we want to ensure production quality of our cellular stack.

So here we come: 2018, The year of testing.

by Harald Welte at December 31, 2017 11:00 PM

Michele's GNSS blog

One year in review


As 2017 heads to the end, it calls for a retrospective. Which events characterized this year most?
Both most prominent magazines in the domain (gpsworld.com and insidegnss.com) dedicate a piece of their December issue to this very topic,  and one can find there their point of view which I hope to complement here. Let's begin from GNSS itself, in no order what-so-ever.

GPS.
Probably one of the most difficult years for the Navstar constellation, with no satellite launched and OCX still not yet. Yes the US maintains the gold standard of satellite radio-navigation but it's hard to predict if and how long that will hold true. In the congress itself there is strong controversy on how adequate GPS is for 21th century warfare. As vulnerabilities of SIS against jamming and spoofing became evident to everyone, and as Navstar satellites continue to exceed their life expectancy, most of the R&D in the US was dedicated to PTA (Protect, Toughen and Augment), with little room for modernization. Yet, despite reading about the first GPSIII satellites readiness and even a new GPSIIIF generation in the works, it all sounded to me quite like a collection of forward-looking statements and promotional material.

Galileo.
Europe managed to declare initial services and launch 4 satellites this year. On the web it is not difficult to gather proof that Worldwide acceptance of the EU system is growing... many GNSS Companies have now claimed compatibility of their services with Galileo, and even penetration in smartphone chipsets became a reality this year (I was already involved with BQ and its Qualcomm IZAT 8C last year). We will not see another quadruple launch for a few more months now, but in the meantime there will be ~20 quadruple-open-frequency birds to work with in 2018. As resilience is developed on the GPS side, modernization and new signal multi-carrier processing techniques are worked on in the old continent.

Beidou.
The Chinese government promised 6 BDS3 satellites this year, later de-scoped to 4 and finally managed to launch 2 last November. Not discouraging though, as the 4 gone missing this year are queued up for flying in early 2018 ...together with another 12. Yes, 16 in total. If even half of that promise is kept, it would put Beidou on the top of the list of GNSS to watch out for. It feels like Beidou is at a cross-roads as new generation birds don't appear to broadcast B2 (the 2 MHz signal on E5b), but just B1. So multi-frequency users will have two heterogeneous groups to deal with: BDS2 with B1+B2 (BPSK2 on 1526f0 and 1180f0) and BDS3 with the legacy B1 plus B1C (TMBOC at 1540f0) and B2a (BPSK10 at 1150f0). Understandably there isn't much clarity towards foreign users on which signal/services combinations China will continue to support or slowly discontinue.

Glonass.
I cannot help but not being very excited about Glonass. Russia launched one satellite in September this year. The system has greatly improved, but FDMA signals at frequencies far away from the GPS ones enough to need a separate RF down-conversion chain do not sound fun to me. However, many more launches (18 according to this source?) are scheduled for 2018, with as many as 5 K2 generation (full CDMA) birds. That yes would change my mind. Right now, with codes already just half as accurate as GPS L1C/A, L1OF/L2OF to me looks only useful for high-sensitivity urban navigation. Not to be mistaken, it's all very good but just a great headache of biases for precision navigation. Glonass modernization is about switching to CDMA signals and new frequencies (L1+E5b on K2, E5a on KM), but it could be a long time before there will be enough satellites to make that constellation a concrete option.

QZSS.
Japan quietly launched 3 satellites this year. QZSS is often overlooked as a purely regional system, versus its global counterparts. But each satellite carries an advanced navigation payload, capable of many frequencies and signals, not to mention services on top of them. So Japan has now a regional constellation of GPSIII-like satellites which are almost always in view. That comes with great benefit for users in the Asia-pacific region as each QZSS satellite is worth approximately 4+ GPSIIF MEO birds. So effectively multi-frequency users have the equivalent of a FOC of GPSIIF - which is pretty cool. I encourage everyone to follow Tomoji's and Takuji's blogs for more hacker friendly info on QZSS.

Others.
I followed on the side India's Navic and Australia's efforts towards a SBAS, but I am not much up to date with those or other systems, so feel free to share with me updates and links in the comments section and I will try to integrate them.

System of systems 2017 wrap-up.
Many ask questions about GNSS adoption and especially the future roles of Galileo, Beidou and Glonass. There are always unknowns of course, but to me the answer is clear:  Beidou3 is (involuntarily?) Galileo's best mate and Glonass will eventually align. The new Beidou English ICD is out and all bets are off: BDS3 promotes a dual frequency L1C+L5 open service. Navstar's L2C seems already old, and yet the user segment adoption of it is largely immature. Most services and receivers still rely on L2P(Y) as 19 satellites aren't enough to justify switching across to the civil signal, despite its many benefits over semi-codeless tracking of the encrypted military one. The GPSIIF L5 signal is affected by line biases which make PPP difficult, but Galileo is free of them if my results are correct. Incidentally I have not yet tried with BDS3 satellites... it's slightly more tricky without a third open frequency. When the legacy P(Y) signals are turned off we might witness a shift between GPS L1C/A + L2P to GPS/Galileo/Beidou L1 + L5 before L2C has a chance of picking up universally. Each system will offer a third open frequency: GPS L2C, Galileo/BDS3 E6/E5b and Glonass L3.


Industry news.
Trying to be as unbiased as possible, the GNSS industry also had some interesting developments this year. Pure GNSS looks slowly disappearing as the low-end of the GNSS offering is nowadays largely owned by the smartphone industry. Multi-aggregated-band LTE and WiFI are more complex than GNSS from a DSP perspective so understandably the latter is considered a commodity. As general purpose modems have multiple wireless capabilities, they can easily do single frequency all-constellation tracking. Navigation is provided through a fusion of technologies and sensors, where GNSS plays the poorest availability highest accuracy one. While Mediatek has been quiet for a while now, maybe reaping the results of its flagship MTK3333, others have moved forward. HED Navigation is continuing to sell through locosys.com its high-sensitivity all-constellation single frequency engine. u-blox released this year its 8-th generation silicon for the IoT/wearables market as well as a single-frequency RTK-capable chipset (the M8P line). Geostar-navigation also came out with a single frequency RTK capable engine associated to its last GeoS-5MR baseband. NVS released its sub-1K$ multi-constellation dual-frequency receiver and we, at Swift Navigation, continue to release FW updates enabling new features of the Piksi Multi. Last but not least, Broadcom stepped on the high-precision field releasing a multi-frequency multi-constellation chip the BCM4775x. I am glad to see I had anticipated this some time ago, and my vision has materialized.


A little give-back
As prof. Borre disappeared this year and I had offered him my SdrNav40 sources as academic material for his new book, I think it's appropriate to open-source that design as promised.
Here is an archive with the 4-channels front-end HW design:
https://drive.google.com/open?id=1QCFZQDv7fTunZxiyiULaOhVfQj2DNdbp
Here the SW for Linux and Windows:
https://drive.google.com/open?id=1PjwSCmO3ZRsHWPlxo8tptu41s7bza_1Q
https://drive.google.com/open?id=1Vqp5zocYVbbohhXGh-bgXhh1cB4N4ncn
It's an old design, not nearly as polished as Peter Monta's firehose.. but it's done an egregious job for me in lots of personal research projects. Feel free to ask questions if needed.


つづく

by noreply@blogger.com (Michele Bavaro) at December 31, 2017 04:13 PM

December 26, 2017

Bunnie Studios

Name that Ware December 2017

The Ware for December 2017 is shown below.

I’ve partially cropped the photo to make it a bit more challenging, but I have a feeling this will be guessed … rather quickly … despite the impediment.

It’s a real beauty on the inside, isn’t it?

by bunnie at December 26, 2017 03:13 PM

Winner, Name that Ware November 2017

The Ware for November 2017 is the “Front Panel Display Board” from an Intel Paragon supercomputer. Many moons ago one of these was being decommissioned at MIT and I raided the cabinet for interesting-looking parts. I snagged a ginormous 5V power supply (the monster could pump out 400A @ 5V) and one of these front panel LED boards. As alluded to by some of the comments, this is the sort of LED board that gets designed when money is not an issue.

Nobody got quite close enough on this one to call a winner, so again this month we have a winner-less name that ware. I have a feeling, however, that next month’s should be a cinch :)

by bunnie at December 26, 2017 03:13 PM

December 12, 2017

Altus Metrum

Altos1.8.3

AltOS 1.8.3 — TeleMega version 3.0 support and bug fixes

Bdale and I are pleased to announce the release of AltOS version 1.8.3.

AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, STMF042, LPC11U14 and ATtiny85 based electronics and Java-based ground station software.

This is a minor release of AltOS, including support for our new TeleMega v3.0 board and a selection of bug fixes

Announcing TeleMega v3.0

TeleMega is our top of the line flight computer with 9-axis IMU, 6 pyro channels, uBlox Max 7Q GPS and 40mW telemetry system. Version 3.0 is feature compatible with version 2.0, incorporating a new higher-perfomance 9-axis IMU in place of the former 6-axis IMU and separate 3-axis magnetometer.

AltOS 1.8.3

In addition to support for TeleMega v3.0 boards, AltOS 1.8.3 contains some important bug fixes for all flight computers. Users are advised to upgrade their devices.

  • Ground testing EasyMega and TeleMega additional pyro channels could result in a sticky 'fired' status which would prevent these channels from firing on future flights.

  • Corrupted flight log records could prevent future flights from capturing log data.

  • Fixed saving of pyro configuration that ended with 'Descending'. This would cause the configuration to revert to the previous state during setup.

The latest AltosUI and TeleGPS applications have improved functionality for analyzing flight data. The built-in graphing capabilities are improved with:

  • Graph lines have improved appearance to make them easier to distinguish. Markers may be placed at data points to show captured recorded data values.

  • Graphing offers the ability to adjust the smoothing of computed speed and acceleration data.

Exporting data for other applications has some improvements as well:

  • KML export now reports both barometric and GPS altitude data to make it more useful for Tripoli record reporting.

  • CSV export now includes TeleMega/EasyMega pyro voltages and tilt angle.

by keithp's rocket blog at December 12, 2017 05:44 PM

December 04, 2017

Free Electrons

Feedback from the Netdev 2.2 conference

The Netdev 2.2 conference took place in Seoul, South Korea. As we work on a diversity of networking topics at Free Electrons as part of our Linux kernel contributions, Free Electrons engineers Alexandre Belloni and Antoine Ténart went to Seoul to attend lots of interesting sessions and to meet with the Linux networking community. Below, they report on what they learned from this conference, by highlighting two talks they particularly liked.

Linux Networking Dietary Restrictions — slides

David S. Miller gave a keynote about reducing the size of core structures in the Linux kernel networking core. The idea behind his work is to use smaller structures which has many benefits in terms of performance as less cache misses will occur and less memory resources are needed. This is especially true in the networking core as small changes may have enormous impacts and improve performance a lot. Another argument from his maintainer hat perspective is the maintainability, where smaller structures usually means less complexity.

He presented five techniques he used to shrink the networking core data structures. The first one was to identify members of common base structures that are only used in sub-classes, as these members can easily be moved out and not impact all the data paths.

The second one makes use of what David calls “state compression”, aka. understanding the real width of the information stored in data structures and to pack flags together to save space. In his mind a boolean should take a single bit whereas in the kernel it requires way more space than that. While this is fine for many uses it makes sense to compress all these data in critical structures.

Then David S. Miller spoke about unused bits in pointers where in the kernel all pointers have 3 bits never used. He argued these bits are 3 boolean values that should be used to reduce core data structure sizes. This technique and the state compression one can be used by introducing helpers to safely access the data.

Another technique he used was to unionize members that aren’t used at the same time. This helps reducing even more the structure size by not having areas of memory never used during identified steps in the networking stack.

Finally he showed us the last technique he used, which was using lookup keys instead of pointers when the objects can be found cheaply based on their index. While this cannot be used for every object, it helped reducing some data structures.

While going through all these techniques he gave many examples to help understanding what can be saved and how it was effective. This was overall a great talk showing a critical aspect we do not always think of when writing drivers, which can lead to big performance improvements.

David S. Miller at Nedev 2.2

WireGuard: Next-generation Secure Kernel Network Tunnel — slides

Jason A. Donenfeld presented his new and shiny L3 network tunneling mechanism, in Linux. After two years of development this in-kernel formally proven cryptographic protocol is ready to be submitted upstream to get the first rounds of review.

The idea behind Wireguard is to provide, with a small code base, a simple interface to establish and maintain encrypted tunnels. Jason made a demo which was impressive by its simplicity when securely connecting two machines, while it can be a real pain when working with OpenVPN or IPsec. Under the hood this mechanism uses UDP packets on top of either IPv4 and IPv6 to transport encrypted packets using modern cryptographic principles. The authentication is similar to what SSH is using: static private/public key pairs. One particularly nice design choice is the fact that Wireguard is exposed as a stateless interface to the administrator whereas the protocol is stateful and timer based, which allow to put devices into sleep mode and not to care about it.

One of the difficulty to get Wireguard accepted upstream is its cryptographic needs, which do not match what can provide the kernel cryptographic framework. Jason knows this and plan to first send patches to rework the cryptographic framework so that his module nicely integrates with in-kernel APIs. First RFC patches for Wireguard should be sent at the end of 2017, or at the beginning of 2018.

We look forward to seeing Wireguard hit the mainline kernel, to allow everybody to establish secure tunnels in an easy way!

Jason A. Donenfeld at Netdev 2.2

Conclusion

Netdev 2.2 was again an excellent experience for us. It was an (almost) single track format, running alongside the workshops, allowing to not miss any session. The technical content let us dive deeply in the inner working of the network stack and stay up-to-date with the current developments.

Thanks for organizing this and for the impressive job, we had an amazing time!

by Antoine Ténart at December 04, 2017 10:25 AM

November 26, 2017

Bunnie Studios

Name that Ware November 2017

The Ware for November 2017 is shown below.

Happy holidays to everyone!

by bunnie at November 26, 2017 09:25 PM

Solution, Name that Ware October 2017

It’s unusual to find a ware without a clear winner, and reading through the comment thread I found a lot of near-misses but none of them close enough for me to declare a winner.

The Ware for October 2017 is a Minibar Systems automated hotel minibar (looks like a “SmartCube 40i”, “The Minibar of the Future”). During an overnight layover, I decided to check the minibar for snack options, but upon pulling what I thought was the handle for the fridge, lo and behold a tray of electronics presented itself. My friend, extremely amused by my enthusiastic reaction, snapped this picture of me adding the ware to my catalog:

by bunnie at November 26, 2017 09:25 PM

November 21, 2017

Free Electrons

Back from ELCE: award to Free Electrons CEO Michael Opdenacker

The Embedded Linux Conference Europe 2017 took place at the end of October in Prague. We already posted about this event by sharing the slides and videos of Free Electrons talks and later by sharing our selection of talks given by other speakers.

During the closing session of this conference, Free Electrons CEO Michael Opdenacker has received from the hands of Tim Bird, on behalf of the ELCE committee, an award for its continuous participation to the Embedded Linux Conference Europe. Indeed, Michael has participated to all 11 editions of ELCE, with no interruption. He has been very active in promoting the event, especially through the video recording effort that Free Electrons did in the early years of the conference, as well as through the numerous talks given by Free Electrons.

Michael Opdenacker receives an award at the Embedded Linux Conference Europe

Free Electrons is proud to see its continuous commitment to knowledge sharing and community participation be recognized by this award!

by Thomas Petazzoni at November 21, 2017 12:49 PM

November 20, 2017

Free Electrons

Linux 4.14 released, Free Electrons contributions

Penguin from Mylène Josserand

Drawing from Mylène Josserand,
based on a picture from Samuel Blanc under CC-BY-SA

Linux 4.14, which is going to become the next Long Term Supported version, has been released a week ago by Linus Torvalds. As usual, LWN.net did an interesting coverage of this release cycle merge window, highlighting the most important changes: The first half of the 4.14 merge window and The rest of the 4.14 merge window.

According to Linux Kernel Patch statistics, Free Electrons contributed 111 patches to this release, making it the 24th contributing company by number of commits: a somewhat lower than usual contribution level from our side. At least, Free Electrons cannot be blamed for trying to push more code into 4.14 because of its Long Term Support nature! 🙂

The main highlights of our contributions are:

  • On the RTC subsystem, Alexandre Belloni made as usual a number of fixes and improvements to various drivers, especially the ds1307 driver.
  • On the NAND subsystem, Boris Brezillon did a number of small improvements in various areas.
  • On the support for Marvell platforms
    • Antoine Ténart improved the ppv2 network driver used by the Marvell Armada 7K/8K SoCs: support for 10G speed and TSO support are the main highlights. In order to support 10G speed, Antoine added a driver in drivers/phy/ to configure the common PHYs in the Armada 7K/8K SoCs.
    • Thomas Petazzoni also improved the ppv2 network driver by adding support for TX interrupts and per-CPU RX interrupts.
    • Grégory Clement contributed some patches to enable NAND support on Armada 7K/8K, as well as a number of fixes in different areas (GPIO fix, clock handling fixes, etc.)
    • Miquèl Raynal contributed a fix for the Armada 3700 SPI controller driver.
  • On the support for Allwinner platforms
    • Maxime Ripard contributed the support for a new board, the BananaPI M2-Magic. Maxime also contributed a few fixes to the Allwinner DRM driver, and a few other misc fixes (clock, MMC, RTC, etc.).
    • Quentin Schulz contributed the support for the power button functionality of the AXP221 (PMIC used in several Allwinner platforms)
  • On the support for Atmel platforms, Quentin Schulz improved the clock drivers for this platform to properly support the Audio PLL, which allowed to fix the Atmel audio drivers. He also fixed suspend/resume support in the Atmel MMC driver to support the deep sleep mode of the SAMA5D2 processor.

In addition to making direct contributions, Free Electrons is also involved in the Linux kernel development by having a number of its engineers act as Linux kernel maintainers. As part of this effort, Free Electrons engineers have reviewed, merged and sent pull requests for a large number of contributions from other developers:

  • Boris Brezillon, as the NAND subsystem maintainer and MTD subsystem co-maintainer, merged 68 patches from other developers.
  • Alexandre Belloni, as the RTC subsystem maintainer and Atmel ARM platform co-maintainer, merged 32 patches from other developers.
  • Grégory Clement, as the Marvell ARM platform co-maintainer, merged 29 patches from other developers.
  • Maxime Ripard, as the Allwinner ARM platform co-maintainer, merged 18 patches from other developers.

This flow of patches from kernel maintainers to other kernel maintainers is also nicely described for the 4.14 release by the Patch flow into the mainline for 4.14 LWN.net article.

The detailed list of our contributions:

by Thomas Petazzoni at November 20, 2017 12:59 PM

November 16, 2017

Free Electrons

Mender: How to integrate an OTA updater

Recently, our customer Senic asked us to integrate an Over-The-Air (OTA) mechanism in their embedded Linux system, and after some discussion, they ended up chosing Mender. This article will detail an example of Mender’s integration and how to use it.

What is Mender?

Mender is an open source remote updater for embedded devices. It is composed of a client installed on the embedded device, and a management server installed on a remote server. However, the server is not mandatory as Mender can be used standalone, with updates triggered directly on the embedded device.

Image taken from Mender’s website

In order to offer a fallback in case of failure, Mender uses the double partition layout: the device will have at least 2 rootfs partitions, one active and one inactive. Mender will deploy an update on the inactive partition, so that in case of an error during the update process, it will still have the active partition intact. If the update succeeds, it will switch to the updated partition: the active partition becomes inactive and the inactive one becomes the new active. As the kernel and the device tree are stored in the /boot folder of the root filesystem, it is possible to easily update an entire system. Note that Mender needs at least 4 partitions:

  • bootloader partition
  • data persistent partition
  • rootfs + kernel active partition
  • rootfs + kernel inactive partition

It is, of course, customizable if you need more partitions.

Two reference devices are supported: the BeagleBone Black and a virtual device. In our case, the board was a Nanopi-Neo, which is based on an Allwinner H3.

Mender provides a Yocto Project layer containing all the necessary classes and recipes to make it work. The most important thing to know is that it will produce an image ready to be written to an SD card to flash empty boards. It will also produce “artifacts” (files with .mender extension) that will be used to update an existing system.

Installation and setup

In this section, we will see how to setup the Mender client and server for your project. Most of the instructions are taken from the Mender documentation that we found well detailed and really pleasant to read. We’ll simply summarize the most important steps.

Server side

The Mender server will allow you to remotely update devices. The server can be installed in two modes:

  • demo mode: Used to test a demo server. It can be nice to test it if you just want to quickly deploy a Mender solution, for testing purpose only. It includes a demo layer that simplify and configure for you a default Mender server on localhost of your workstation.
  • production mode: Used for production. We will focus on this mode as we wanted to use Mender in a production context. This mode allows to customize the server configuration: IP address, certificates, etc. Because of that, some configuration will be necessary (which is not the case in the demo mode).

In order to install the Mender server, you should first install Docker CE and Docker Compose. Have a look at the corresponding Docker instructions.

Setup

  • Download the integration repository from Mender:
  • $ git clone https://github.com/mendersoftware/integration mender-server
    
  • Checkout 1.1.0 tag (latest version at the moment of the test)
  • $ cd mender-server
    $ git checkout 1.1.0 -b my-production-setup
    
  • Copy the template folder and update all the references to “template”
  • $ cp -a template production
    $ cd production
    $ sed -i -e 's#/template/#/production/#g' prod.yml
    
  • Download Docker images
  • $ ./run pull
    
  • Use the keygen script to create certificates for domain names (e.g. mender.foobar.com and s3.foobar.com)
  • $ CERT_API_CN=mender.foobar.com CERT_STORAGE_CN=s3.foobar.com ../keygen
    
  • Some persistent storage will be needed by Mender so create a few Docker volumes:
  • $ docker volume create --name=mender-artifacts
    $ docker volume create --name=mender-deployments-db
    $ docker volume create --name=mender-useradm-db
    $ docker volume create --name=mender-inventory-db
    $ docker volume create --name=mender-deviceadm-db
    $ docker volume create --name=mender-deviceauth-db
    

Final configuration

This final configuration will link the generated keys with the Mender server. All the modifications will be in the prod.yml file.

  • Locate the storage-proxy service in prod.yml and set it to your domain name. In our case s3.foobar.com under the networks.mender.aliases
  • Locate the minio service. Set MINIO_ACCESS_KEY to “mender-deployments” and the MINIO_SECRET_KEY to a generated password (with e.g.: $ apg -n1 -a0 -m32)
  • Locate the mender-deployments service. Set DEPLOYMENTS_AWS_AUTH_KEY and DEPLOYMENTS_AWS_AUTH_SECRET to respectively the value of MINIO_ACCESS_KEY and MINIO_SECRET_KEY. Set DEPLOYMENTS_AWS_URI to point to your domain such as https://s3.foobar.com:9000

Start the server

Make sure that the domain names you have defined (mender.foobar.com and s3.foobar.com) are accessible, potentially by adding them to /etc/hosts if you’re just testing.

  • Start the server
  • $ ./run up -d
    
  • If it is a new installation, request initial user login:
  • $ curl -X POST  -D - --cacert keys-generated/certs/api-gateway/cert.crt https://mender.foobar.com:443/api/management/v1/useradm/auth/login
    
  • Check that you can create a user and login to mender UI:
  •  $ firefox http://mender.foobar.com:443 

Client side – Yocto Project

Mender has a Yocto Project layer to easily interface with your own layer.
We will see how to customize your layer and image components (U-Boot, Linux kernel) to correctly configure it for Mender use.

In this section, we will assume that you have your own U-Boot and your own kernel repositories (and thus, recipes) and that you retrieved the correct branch of this layer.

Machine and distro configurations

  • Make sure that the kernel image and Device Tree files are installed in the root filesystem image
  • RDEPENDS_kernel-base += "kernel-image kernel-devicetree"
    
  • Update the distro to inherit the mender-full class and add systemd as the init manager (we only tested Mender’s integration with systemd)
  • # Enable systemd for Mender
    DISTRO_FEATURES_append = " systemd"
    VIRTUAL-RUNTIME_init_manager = "systemd"
    DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
    VIRTUAL-RUNTIME_initscripts = ""
    
    INHERIT += "mender-full"
    
  • By default, Mender assumes that your storage device is /dev/mmcblk0, that mmcblk0p1 is your boot partition (containing the bootloader), that mmcblk0p2 and mmcblk0p3 are your two root filesystem partitions, and that mmcblk0p5 is your data partition. If that’s the case for you, then everything is fine! However, if you need a different layout, you need to update your machine configuration. Mender’s client will retrieve which storage device to use by using the MENDER_STORAGE_DEVICE variable (which defaults to mmcblk0). The partitions themselves should be specified using MENDER_BOOT_PART, MENDER_ROOTFS_PART_A, MENDER_ROOTFS_PART_B and ROOTFS_DATA_PART. If you need to change the default storage or the partitions’ layout, edit in your machine configuration the different variables according to your need. Here is an example for /dev/sda:
  • MENDER_STORAGE_DEVICE = "/dev/sda"
    MENDER_STORAGE_DEVICE_BASE = "${MENDER_STORAGE_DEVICE}"
    MENDER_BOOT_PART = "${MENDER_STORAGE_DEVICE_BASE}1"
    MENDER_ROOTFS_PART_A = "${MENDER_STORAGE_DEVICE_BASE}2"
    MENDER_ROOTFS_PART_B = "${MENDER_STORAGE_DEVICE_BASE}3"
    MENDER_DATA_PART = "${MENDER_STORAGE_DEVICE_BASE}5"
    
  • Do not forget to update the artifact name in your local.conf, for example:
  • MENDER_ARTIFACT_NAME = "release-1"
    

    As described in Mender’s documentation, Mender will store the artifact name in its artifact image. It must be unique which is what we expect because an artifact will represent a release tag or a delivery. Note that if you forgot to update it and upload an artifact with the same name as an existing in the web UI, it will not be taken into account.

    U-Boot configuration tuning

    Some modifications in U-Boot are necessary to be able to perform the rollback (use a different partition after an unsuccessful update)

    • Mender needs BOOTCOUNT support in U-Boot. It creates a bootcount variable that will be incremented each time a reboot appears (or reset to 1 after a power-on reset). Mender will use this variable in its rollback mechanism.
      Make sure to enable it in your U-Boot configuration. This will most likely require a patch to your board .h configuration file, enabling:
    • #define CONFIG_BOOTCOUNT_LIMIT
      #define CONFIG_BOOTCOUNT_ENV
      
    • Remove environment variables that will be redefined by Mender. They are defined in Mender’s documentation.
    • Update your U-Boot recipe to inherit Mender’s one and make sure to provide U-Boot virtual package (using PROVIDES)
    • # Mender integration
      require recipes-bsp/u-boot/u-boot-mender.inc
      PROVIDES += "u-boot"
      RPROVIDES_${PN} += "u-boot"
      BOOTENV_SIZE = "0x20000"
      

      The BOOTENV_SIZE must be set the same content as the U-Boot CONFIG_ENV_SIZE variable. It will be used by the u-boot-fw-utils tool to retrieve the U-Boot environment variables.

      Mender is using u-boot-fw-utils so make sure that you have a recipe for it and that Mender include’s file is included. To do that, you can create a bbappend file on the default recipe or create your own recipe if you need a specific version. Have a look at Mender’s documentation example.

    • Tune your U-Boot environment to use Mender’s variables. Here are some examples of the modifications to be done. Set the root= kernel argument to use ${mender_kernel_root}, set the bootcmd to load the kernel image and Device Tree from ${mender_uboot_root} and to run mender_setup. Make sure that you are loading the Linux kernel image and Device Tree file from the root filesystem /boot directory.
      setenv bootargs 'console=${console} root=${mender_kernel_root} rootwait'
      setenv mmcboot 'load ${mender_uboot_root} ${fdt_addr_r} boot/my-device-tree.dtb; load ${mender_uboot_root} ${kernel_addr_r} boot/zImage; bootz ${kernel_addr_r} - ${fdt_addr_r}'
      setenv bootcmd 'run mender_setup; run mmcboot'
      

    Mender’s client recipe

    As stated in the introduction, Mender has a client, in the form of a userspace application, that will be used on the target. Mender’s layer has a Yocto recipe for it but it does not have our server certificates. To establish a connection between the client and the server, the certificates have to be installed in the image. For that, a bbappend recipe will be created. It will also allow to perform additional Mender configuration, such as defining the server URL.

    • Create a bbappend for the Mender recipe
    • FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
      SRC_URI_append = " file://server.crt"
      MENDER_SERVER_URL = "https://mender.senic.com"
      
    • Copy your server certificates in the bbappend recipe folder

    Recompile an image and now, we should have everything we need to be able to update an image. Do not hesitate to run the integration checklist, it is really a convenient way to know if everything is correctly configured (or not).

    If you want to be more robust and secure, you can sign your artifacts to be sure that they come from a trusted source. If you want this feature, have a look at this documentation.

    Usage

    Standalone mode

    To update an artifact using the standalone mode (i.e. without server), here are the commands to use. You will need to update them according to your needs.

    • On your work station, create a simple HTTP server in your Yocto deploy folder:
    • $ python -m SimpleHTTPServer
    • On the target, start mender in standalone mode
    • $ mender -log-level info -rootfs http://192.168.42.251:8000/foobar.mender

      You can also use the mender command to start an update from a local .mender file, provided by a USB key or SD card.

    • Once finished, you will have to reboot the target manually
    • $ reboot

      After the first reboot, you will be on the the new active partition (if the previous one was /dev/mmcblk0p2, you should be on /dev/mmcblk0p3). Check the kernel version, artifact name or command line:

      $ uname -a
      $ cat /etc/mender/artifact_info
      $ cat /proc/cmdline
      

      If you are okay with this update, you will have to commit the modification otherwise the update will not be persistent and once you will reboot the board, Mender will rollback to the previous partition:

      $ mender -commit

    Using Mender’s server UI

    The Mender server UI provides a management interface to deploy updates on all your devices. It knows about all your devices, their current software version, and you can plan deployments on all or a subset of your devices. Here are the basic steps to trigger a deployment:

    • Login (or create an account) into the mender server UI: https://mender.foobar.com:443
    • Power-up your device
    • The first time, you will have to authorize the device. You will find it in your “dashboard” or in the “devices” section.
    • After authorizing it, it will retrieve device information such as current software version, MAC address, network interface, and so on
    • To update a partition, you will have to create a deployment using an artifact.
    • Upload the new artifact in the server UI using the “Artifacts” section
    • Deploy the new artifact using the “deployment” or the “devices” section. You will retrieve the status of the deployment in the “status” field. It will be in “installing”, “rebooting”, etc. The board will reboot and the partition should be updated.

    Troubleshooting

    Here are some issues we faced when we integrated Mender for our device. The Mender documentation also has a troubleshooting section so have a look at it if you are facing issues. Otherwise, the community seems to be active even if we did not need to interact with it as it worked like a charm when we tried it.

    Update systemd’s service starting

    By default, the Mender systemd service will start after the service “resolved” the domain name. On our target device, the network was available only via WiFi. We had to wait for the wlan0 interface to be up and configured to automatically connect a network before starting Mender’s service. Otherwise, it leads to an error due to the network being unreachable. To solve this issue which is specific to our platform, we set the systemd dependencies to “network-online.target” to be sure that a network is available:

    -After=systemd-resolved.service
    +After=network-online.target
    +Wants=network-online.target
    

    It now matches our use case because the Mender service will start only if the wlan0 connection is available and working.

    Certificate expired

    The certificates generated and used by Mender have a validity period. In case your board does not have a RTC set, Mender can fail with the error:

    systemctl status mender
    [...]
    ... level=error msg="authorize failed: transient error: authorization request failed: failed to execute authorization request:
    Post https:///api/devices/v1/authentication/auth_requests: x509: certificate has expired or is not yet valid" module=state
    

    To solve this issue, update the date on your board and make sure your RTC is correctly set.

    Device deletion

    While testing Mender’s server (version 1.0), we always used the same board and got into the issue that the board was already registered in the Server UI but had a different Device ID (which is used by Mender to identify devices). Because of that, the server was always rejecting the authentication. The next release of the Mender server offers the possibility to remove a device so we updated the Mender’s server to the last version.

    Deployments not taken into account

    Note that the Mender’s client is checking by default every 30 minutes if a deployment is available for this device. During testing, you may want to reduce this period, which you can in the Mender’s configuration file using its UpdatePollIntervalSeconds variable.

    Conclusion

    Mender is an OTA updater for Embedded devices. It has a great documentation in the form of tutorials which makes the integration easy. While testing it, the only issues we got were related to our custom platform or were already indicated in the documentation. Deploying it on a board was not difficult, only some U-Boot/kernel and Yocto Project modifications were necessary. All in all, Mender worked perfectly fine for our project!

    by Mylène Josserand at November 16, 2017 09:06 AM

    November 15, 2017

    Free Electrons

    Back from ELCE: selection of talks from the Free Electrons team

    As discussed in our previous blog post, Free Electrons had a strong presence at the Embedded Linux Conference Europe, with 7 attendees, 4 talks, one BoF and one poster during the technical show case.

    In this blog post, we would like to highlight a number of talks from the conference that we found interesting. Each Free Electrons engineer who attended the conference has selected one talk, and gives his/her feedback about this talk.

    uClibc Today: Still Makes Sense – Alexey Brodkin

    Talk selected by Michael Opdenacker

    uClibc Today: Still Makes Sense – Alexey BrodkinAlexey Brodkin, an active contributor to the uClibc library, shared recent updates about this C library, trying to show people that the project was still active and making progress, after a few years during which it appeared to be stalled. Alexey works for Synopsys, the makers of the ARC architecture, which uClibc supports.

    If you look at the repository for uClibc releases, you will see that since version 1.0.0 released in 2015, the project has made 26 releases according to a predictable schedule. The project also runs runtime regression tests on all its releases, which weren’t done before. The developers have also added support for 4 new architectures (arm64 in particular), and uClibc remains the default C library that Buildroot proposes.

    Alexey highlighted that in spite of the competition from the musl library, which causes several projects to switch from uClibc to musl, uClibc still makes a lot of sense today. As a matter of fact, it supports more hardware architectures than glibc and musl do, as it’s the only one to support platforms without an MMU (such as noMMU ARM, Blackfin, m68k, Xtensa) and as the library size is still smaller that what you get with musl (though a static hello_world program is much smaller with musl if you have a close look at the library comparison tests he mentioned).

    Alexey noted that the uClibc++ project is still alive too, and used in OpenWRT/Lede by default.

    Read the slides and watch the video of his talk.

    Identifying and Supporting “X-Compatible” hardware blocks – Chen-Yu Tsai

    Talk selected by Quentin Schulz

    Identifying and Supporting “X-Compatible” hardware blocks – Chen-Yu TsaiAn SoC is made of multiple IP blocks from different vendors. In some cases the source or model of the hardware blocks are neither documented nor marketed by the SoC vendor. However, since there are only very few vendors of a given IP block, stakes are high that your SoC vendor’s undocumented IP block is compatible with a known one.

    With his experience in developing drivers for multiple IP blocks present in Allwinner SoCs and as a maintainer of those same SoCs, Chen-Yu first explained that SoC vendors often either embed some vendors’ licensed IP blocks in their SoCs and add the glue around it for platform- or SoC-specific hardware (clocks, resets and control signals), or they clone IP blocks with the same logic but some twists (missing, obfuscated or rearranged registers).

    To identify the IP block, we can dig into the datasheet or the vendor BSP and compare those with well documented datasheets such as the one for NXP i.MX6, TI KeyStone II or the Zynq UltraScale+ MPSoC, or with mainline drivers. Asking the community is also a good idea as someone might have encountered an IP with the same behaviour before and can help us identify it quicker.

    Some good identifiers for IPs could be register layouts or names along with DMA logic and descriptor format. For the unlucky ones that have been provided only a blob, they can look for the symbols in it and that may slightly help in the process.

    He also mentioned that identifying an IP block is often the result of the developer’s experience in identifying IPs and other time just pure luck. Unfortunately, there are times when someone couldn’t identify the IP and wrote a complete driver just to be told by someone else in the community that this IP reminds him of that IP in which case the work done can be just thrown away. That’s where the community plays a strong role, to help us in our quest to identify an IP.

    Chen-Yu then went on with the presentation of the different ways to handle the multiple variants of an IP blocks in drivers. He said that the core logic of all IP drivers is usually presented as a library and that the different variants have a driver with their own resources, extra setup and use this library. Also, a good practice is to use booleans to select features of IP blocks instead of using the ID of each variant.
    For IPs whose registers have been altered, the way to go is usually to write a table for the register offsets, or use regmaps when bitfields are also modified. When the IP block differs a bit too much, custom callbacks should be used.

    He ended his talk with his return from experience on multiple IP blocks (UART, USB OTG, GMAC, EMAC and HDMI) present in Allwinner SoCs and the differences developers had to handle when adding support for them.

    Read the slides and watch the video of his talk.

    printk(): The Most Useful Tool is Now Showing its Age – Steven Rostedt & Sergey Senozhatsky

    Talks selected by Boris Brezillon. Boris also covered the related talk “printk: It’s Old, What Can We Do to Make It Young Again?” from the same speakers.

    printk(): The Most Useful Tool is Now Showing its Age – Steven Rostedt & Sergey SenozhatskyMaybe I should be ashamed of saying that but printk() is one of the basic tool I’m using to debug kernel code, and I must say it did the job so far, so when I saw these presentations talking about how to improve printk() I was a bit curious. What could be wrong in printk() implementation?
    Before attending the talks, I never digged into printk()’s code, because it just worked for me, but what I thought was a simple piece of code appeared to be a complex infrastructure with locking scheme that makes you realize how hard it is when several CPUs are involved.

    At its core, printk() is supposed to store logs into a circular buffer and push new entries to one or several consoles. In his first talk Steven Rostedt walked through the history of printk() and explained why it became more complex when support for multi CPU appeared. He also detailed why printk() is not re-entrant and the problem it causes when called from an NMI handler. He finally went through some fixes that made the situation a bit better and advertised the 2nd half of the talk driven by Sergey Senozhatsky.

    Note that between these two presentations, the printk() rework has been discussed at Kernel Summit, so Sergey already had some feedback on his proposals. While Steven presentation focused mainly on the main printk() function, Sergey gave a bit more details on why printk() can deadlock, and one of the reasons why preventing deadlocks is so complicated is that printk() delegates the ‘print to console’ aspect to console drivers which have their own locking scheme. To address that, it is proposed to move away from the callback approach and let console drivers poll for new entries in the console buffer instead, which would remove part of the locking issues. The problem with this approach is that it brings even more uncertainty on when the logs are printed on the consoles, and one of the nice things about printk() in its current form is that you are likely to have the log printed on the output before printk() returns (which helps a lot when you debug things).

    He also mentioned other solutions to address other possible deadlocks, but I must admit I got lost at some point, so if you’re interested in this topic I recommend that you watch the video (printk(): The Most Useful Tool is Now Showing its Age, sadly no video is available for the second talk) and read the slides (printk(): The Most Useful Tool is Now Showing its Age and printk: It’s Old, What Can We Do to Make It Young Again?).

    More robust I2C designs with a new fault-injection driver – Wolfram Sang

    Talk selected by Miquèl Raynal

    More robust I2C designs with a new fault-injection driver – Wolfram SangAlthough Wolfram had a lot of troubles starting its presentation lacking a proper HDMI adaptater, he gave an illuminating talk about how, as an I2C subsystem maintainer, he would like to strengthen the robustness of I2C drivers.

    He first explained some basics of the I2C bus like START and STOP conditions and introduced us to a few errors he regularly spots in drivers. For instance, some badly written drivers used a START and STOP sequence while a “repeated START” was needed. This is very bad because another master on the bus could, in this little spare idle delay, decide to grab the medium and send its message. Then the message right after the repeated START would not have the expected effect. Of course plenty other errors can happen: stalled bus (SDA or SCL stuck low), lost arbitration, faulty bits… All these situations are usually due to incorrect sequences sent by the driver.

    To avoid so much pain debugging obscure situations where this happens, he decided to use an extended I2C-gpio interface to access SDA and SCL from two external GPIOs and this way forces faulty situations by simply pinning high or low one line (or both) and see how the driver reacts. The I2C specification and framework provide everything to get out of a faulty situation, it is just a matter of using it (sending a STOP condition, clocking 9 times, operate a reset, etc).

    Wolfram is aware of his quite conservative approach but he is really scared about breaking users by using random quirks so he tried with this talk to explain his point of view and the solutions he wants to promote.

    Two questions that you might have a hard time hearing were also interesting. The first person asked if he ever considered using a “default faulty chip” designed to do by itself this kind of fault injection and see how the host reacts and behaves. Wolfram said buying hardware is too much effort for debugging, so he was more motivated to get something very easy and straightforward to use. Someone else asked if he thought about multiple clients situation, but from Wolfram’s point of view, all clients are in the same state whether the bus is busy or free and should not badly behave if we clock 9 times.

    Watch the video and grab the slides.

    HDMI 4k Video: Lessons Learned – Hans Verkuil

    Talk selected by Maxime Ripard

    HDMI 4k Video: Lessons Learned – Hans VerkuilHaving worked recently on a number of display related drivers, it was quite natural to go see what I was probably going to work on in a quite close future.

    Hans started by talking about HDMI in general, and the various signals that could go through it. He then went on with what was basically a war story about all the mistakes, gotchas and misconceptions that he encountered while working on a video-conference box for Cisco. He covered the hardware itself, but also more low-level oriented aspects, such as the clocks frequencies needed to operate properly, the various signals you could look at for debugging or the issues that might come with the associated encoding and / or color spaces, especially when you want to support as many displays as you can. He also pointed out the flaws in the specifications that might lead to implementation inconsistencies. He concluded with the flaws of various HDMI adapters, the issues that might arise using them on various OSes, and how to work around them when doable.

    Watch the video and the slides.

    The Serial Device Bus – Johan Hovold

    Talk selected by Thomas Petazzoni

    The Serial Device Bus – Johan HovoldJohan started his talk at ELCE by exposing the problem with how serial ports (UARTs) are currently handled in the Linux kernel. Serial ports are handled by the TTY layer, which allows user-space applications to send and receive data with what is connected on the other side of the UART. However, the kernel doesn’t provide a good mechanism to model the device that is connected at the other side of the UART, such as a Bluetooth chip. Due to this, people have resorted to either writing user-space drivers for such devices (which falls short when those devices need additional resources such as regulators, GPIOs, etc.) or to developing specific TTY line-discipline in the kernel. The latter also doesn’t work very well because a line discipline needs to be explicitly attached to a UART to operate, which requires a user-space program such as hciattach used in Bluetooth applications.

    In order to address this problem, Johan picked up the work initially started by Rob Herring (Linaro), which consisted in writing a serial device bus (serdev in short), which consists in turning UART into a proper bus, with bus controllers (UART controllers) and devices connected to this bus, very much like other busses in the Linux kernel (I2C, SPI, etc.). serdev was initially merged in Linux 4.11, with some improvements being merged in follow-up versions. Johan then described in details how serdev works. First, there is a TTY port controller, which instead of registering the traditional TTY character device will register the serdev controller and its slave devices. Thanks to this, one can described in its Device Tree child nodes of the UART controller node, which will be probed as serdev slaves. There is then a serdev API to allow the implementation of serdev slave drivers, that can send and receive data of the UART. Already a few drivers are using this API: hci_serdev, hci_bcm, hci_ll, hci_nokia (Bluetooth) and qca_uart (Ethernet).

    We found this talk very interesting, as it clearly explained what is the use case for serdev and how it works, and it should become a very useful subsystem for many embedded applications that use UART-connected devices.

    Watch the video and the slides.

    GStreamer for Tiny Devices – Olivier Crête

    Talk selected by Grégory Clement

    GStreamer for Tiny Devices – Olivier CrêteThe purpose of this talk was to show how to shrink Gstreamer to make it fit in an embedded Linux device. First, Olivier Crête introduced what GStreamer is, it was very high level but well done. Then after presenting the issue, he showed step by step how he managed to reduce the footprint of a GStreamer application to fit in his device.

    In a first part it was a focus on features specific to GStreamer such as how to generate only the needed plugins. Then most of the tricks showed could be used for any C or C++ application. The talk was pretty short so there was no useless or boring part Moreover, the speaker himself was good and dynamic.

    To conclude it was a very pleasant talk teaching step by step how to reduce the footprint of an application being GStreamer or not.

    Watch the video and the slides.

    by Thomas Petazzoni at November 15, 2017 08:33 AM

    November 14, 2017

    Video Circuits

    Video Circuits Workshop 01/07/17

    Alex and I are running another workshop, this time at Brighton Modular meet book here:
    https://www.attenboroughcentre.com/events/902/video-synthesis-workshop/
    We will also be running a video synthesis room all Sunday.
    http://brightonmodularmeet.co.uk/brighton-modular-meet---about.html

    Come hang out and enjoy the rest of the meet.

    Some pics of the panels we had made by Matt for Jona's CH/AV project (which we will be making on the day). There is also a shot of some audio oscillators driving the CH/AV to a VGA monitor.








    by Chris (noreply@blogger.com) at November 14, 2017 05:47 AM

    November 08, 2017

    Bunnie Studios

    A Clash of Cultures

    There’s an Internet controversy going on between Dale Dougherty, the CEO of Maker Media and Naomi Wu (@realsexycyborg), a Chinese Maker and Internet personality. Briefly, Dale Doughtery tweeted a single line questioning Naomi Wu’s authenticity, which is destroying Naomi’s reputation and livelihood in China.

    In short, I am in support of Naomi Wu. Rather than let the Internet speculate on why, I am sharing my perspectives on the situation preemptively.

    As with most Internet controversies, it’s messy and emotional. I will try my best to outline the biases and issues I have observed. Of course, everyone has their perspective; you don’t have to agree with mine. And I suspect many of my core audience will dislike and disagree with this post. However, the beginning of healing starts with sharing and listening. I will share, and I respectfully request that readers read the entire content of this post before attacking any individual point out of context.

    The key forces I see at play are:

    1. Prototype Bias – how assumptions based on stereotypes influence the way we think and feel
    2. Idol Effect – the tendency to assign exaggerated capabilities and inflated expectations upon celebrities
    3. Power Asymmetry – those with more power have more influence, and should be held to a higher standard of accountability
    4. Guanxi Bias – the tendency to give foreign faces more credibility than local faces in China

    All these forces came together in a perfect storm this past week.

    1. Prototype Bias

    If someone asked you to draw a picture of an engineer, who would you draw? As you draw the figure, the gender assigned is a reflection of your mental prototype of an engineer – your own prototype bias. Most will draw a male figure. Society is biased to assign high-level intellectual ability to males, and this bias starts at a young age. Situations that don’t fit into your prototypes can feel threatening; studies have shown that men defend their standing by undermining the success of women in STEM initiatives.

    The bias is real and pervasive. For example, my co-founder in Chibitronics, Jie Qi, is female. The company is founded on technology that is a direct result of her MIT Media Lab PhD dissertation. She is the inventor of paper electronics. I am a supporting actor in her show. Despite laying this fact out repeatedly, she still receives comments and innuendo implying that I am the inventor or more influential than I really am in the development process.

    Any engineer who observes a bias in a system and chooses not to pro-actively correct for it is either a bad engineer or they stand to benefit from the bias. So much of engineering is about compensating, trimming, and equalizing imperfections out of real systems: wrap a feedback loop around it, and force the error function to zero.

    So when Jie and I stand on stage together, prototype bias causes people to assume I’m the one who invented the technology. Given that I’m aware of the bias, does it make sense to give us equal time on the stage? No – that would be like knowing there is uneven loss in a channel and then being surprised when certain frequency bands are suppressed by the time it hits the receivers. So, I make a conscious and deliberate effort to showcase her contributions and to ensure her voice is the first and last voice you hear.

    Naomi Wu (pictured below) likely challenges your prototypical ideal of an engineer. I imagine many people feel a cognitive dissonance juxtaposing the label “engineer” or “Maker” with her appearance. The strength of that dissonant feeling is proportional to the amount of prototype bias you have.

    I’ve been fortunate to experience breaking my own prototypical notions that associate certain dress norms with intelligence. I’m a regular at Burning Man, and my theme camp is dominated by scientists and engineers. I’ve discussed injection molding with men in pink tutus and learned about plasmonics from half-naked women. It’s not a big leap for me to accept Naomi as a Maker. I’m glad she’s challenging these biases. I do my best engineering when sitting half-naked at my desk. I find shirts and pants to be uncomfortable. I don’t have the strength to challenge these social norms, and secretly, I’m glad someone is.

    Unfortunately, prototype bias is only the first challenge confronted in this situation.

    2. Idol Effect

    The Idol Effect is the tendency to assign exaggerated capabilities to public figures and celebrities. The adage “never meet your childhood hero” is a corollary of the Idol Effect – people have inflated expectations about what celebrities can do, so it’s often disappointing when you find out they are humans just like us.

    One result of the Idol Effect is that people feel justified taking pot shots at public figures for their shortcomings. For example, I have had the great privilege of working with Edward Snowden. One of my favorite things about working with him is that he is humble and quick to correct misconceptions about his personal abilities. Because of his self-awareness of his limitations, it’s easier for me to trust his assertions, and he’s also a fast learner because he’s not afraid to ask questions. Notably, he’s never claimed to be a genius, so I’m always taken aback when intelligent people pull me aside and whisper in my ear, “You know, I hear Ed’s a n00b. He’s just using you.” Somehow, because of Ed’s worldwide level of fame that’s strongly associated with security technology, people assume he should be a genius level crypto-hacker and are quick to point out that he’s not. Really? Ed is risking his life because he believes in something. I admire his dedication to the cause, and I enjoy working with him because he’s got good ideas, a good heart, and he’s fun to be with.

    Because I also have a public profile, the Idol Effect impacts me too. I’m bad at math, can’t tie knots, a mediocre programmer…the list goes on. If there’s firmware in a product I’ve touched, it’s likely to have been written by Sean ‘xobs’ Cross, not me. If there’s analytics or informatics involved, it’s likely my partner wrote the analysis scripts. She also edits all my blog posts (including this one) and has helped me craft my most viral tweets – because she’s a genius at informatics, she can run analyses on how to target key words and pick times of day to get maximum impact. The fact that I have a team of people helping me polish my work makes me look better than I really am, and people tend to assign capabilities to me that I don’t really have. Does this mean I am a front, fraud or a persona?

    I imagine Naomi is a victim of Idol Effect too. Similar to Snowden, one of the reasons I’ve enjoyed interacting with Naomi is that she’s been quick to correct misconceptions about her abilities, she’s not afraid to ask for help, and she’s a quick learner. Though many may disapprove of her rhetoric on Twitter, please keep in mind English is her second language — her sole cultural context in which she learned English was via the Internet by reading social media and chat rooms.

    Based on the rumors I’ve read, it seems fans and observers have inflated expectations for her abilities, and because of uncorrected prototype bias, she faces extra scrutiny to prove her abilities. Somehow the fact that she almost cuts her finger using a scraper to remove a 3D print is “evidence” that she’s not a Maker. If that’s true, I’m not a Maker either. I always have trouble releasing 3D prints from print stages. They’ve routinely popped off and flown across the room, and I’ve almost cut my fingers plenty of times with the scraper. But I still keep on trying and learning – that’s the point. And then there’s the suggestion that because a man holds the camera, he’s feeding her lines.

    When a man harnesses the efforts of a team, they call him a CEO and give him a bonus. But when a woman harnesses the efforts of a team, she gets accused of being a persona and a front. This is uncorrected Prototype Bias meeting unrealistic expectations due to the Idol Effect.

    The story might end there, but things recently got a whole lot worse…

    3. Power Asymmetry

    “With great power comes great responsibilities.”
    -from Spider Man

    Power is not distributed evenly in the world. That’s a fact of life. Not acknowledging the role power plays leads to systemic abuse, like those documented in the Caldbeck or Weinstein scandals.

    Editors and journalists – those with direct control over what gets circulated in the media – have a lot of power. Their thoughts and opinions can reach and influence a massive population very quickly. Rumors are just rumors until media outlets breathe life into them, at which point they become an incurable cancer on someone’s career. Editors and journalists must be mindful of the power they wield and held accountable for when it is mis-used.

    As CEO of Maker Media and head of an influential media outlet, especially among the DIY community, Dale Dougherty wields substantial power. So a tweet promulgating the idea that Naomi might be a persona or a fake does not land lightly. In the post-truth era, it’s especially incumbent upon traditional media to double-check rumors before citing them in any context.

    What is personally disappointing is that Dale reached out to me on November 2nd with an email asking what I thought about an anonymous post that accused Naomi of being a fake. I vouched for Naomi as a real person and as a budding Maker; I wrote back to Dale that “I take the approach of interacting with her like any other enthusiastic, curious Maker and the resulting interactions have been positive. She’s a fast learner.”

    Yet Dale decided to take an anonymous poster’s opinion over mine (despite a long working relationship with Make), and a few days later on November 5th he tweeted a link to the post suggesting Naomi could be a fake or a fraud, despite having evidence of the contrary.

    So now Naomi, already facing prototype bias and idol-effect expectations, gets a big media personality with substantial power propagating rumors that she is a fake and a fraud.

    But wait, it gets worse because Naomi is in China!

    4. Guanxi Bias

    In China, guanxi (关系) is everything. Public reputation is extremely hard to build, and quick to lose. Faking and cloning is a real problem, but it’s important to not lose sight of the fact that there are good, hard-working people in China as well. So how do the Chinese locals figure out who to trust? Guanxi is a major mechanism used inside China to sort the good from the bad – it’s a social network of credible people vouching for each other.

    For better or for worse, the Chinese feel that Western faces and brands are more credible. The endorsement of a famous Western brand carries a lot of weight; for example Leonardo DiCaprio is the brand ambassador for BYD (a large Chinese car maker).

    Maker Media has a massive reputation in China. From glitzy Maker Faires to the Communist party’s endorsement of Maker-ed and Maker spaces as a national objective, an association or the lack thereof with Maker Media can make or break a reputation. This is no exception for Naomi. Her uniqueness as a Maker combined with her talent at marketing has enabled her to do product reviews and endorsements as source of income.

    However, for several years she’s been excluded from the Shenzhen Maker Faire lineup, even in events that she should have been a shoo-in for her: wearables, Maker fashion shows, 3D printing. Despite this lack of endorsement, she’s built her own social media follower base both inside and outside of China, and built a brand around herself.

    Unfortunately, when the CEO of Maker Media, a white male leader of an established American brand, suggested Naomi was a potential fake, the Internet inside China exploded on her. Sponsors cancelled engagements with her. Followers turned into trolls. She can’t be seen publicly with men (because others will say the males are the real Maker, see “prototype bias”), and as a result faces a greater threat of physical violence.

    A single innuendo, amplified by Power Asymmetry and Guanxi Bias, on top of Idol Effect meshed against Prototype Bias, has destroyed everything a Maker has worked so hard to build over the past few years.

    If someone spread lies about you and destroyed your livelihood – what would you do? Everyone would react a little differently, but make no mistake: at this point she’s got nothing left to lose, and she’s very angry.

    Reflection

    Although Dale had issued a public apology about the rumors, the apology fixes her reputation as much as saying “sorry” repairs a vase smashed on the floor.

    Image: Mindy Georges CC BY-NC

    At this point you might ask — why would Dale want to slander Naomi?

    I don’t know the background, but prior to Dale’s tweet, Naomi had aggressively dogged Dale and Make about Make’s lack of representation of women. Others have noted that Maker Media has a prototype bias toward white males. Watch this analysis by Leah Buechley, a former MIT Media Lab Professor:

    Dale could have recognized and addressed this core issue of a lack of diversity. Instead, Dale elected to endorse unsubstantiated claims and destroy a young female Maker’s reputation and career.

    Naomi has a long, uphill road ahead of her. On the other hand, I’m sure Dale will do fine – he’s charismatic, affable, and powerful.

    When I sit and think, how would I feel if this happened to the women closest to me? I get goosebumps – the effect would be chilling; the combination of pervasive social biases would overwhelm logic and fact. So even though I may not agree with everything Naomi says or does, I have decided that in the bigger picture, hiding in complicit silence on the sidelines is not acceptable.

    We need to acknowledge that prototype bias is real; if equality is the goal, we need to be proactive in correcting it. Just because someone is famous doesn’t mean they are perfect. People with power need to be held accountable in how they wield it. And finally, cross-cultural issues are complicated and delicate. All sides need to open their eyes, ears, and hearts and realize we’re all human. Tweets may seem like harmless pricks to the skin, but we all bleed when pricked. For humanity to survive, we need to stop pricking each other lest we all bleed to death.

    /me dons asbestos suit

    Update: November 20, 2017
    Make has issued an apology, and Naomi has accepted the apology. My sincere thanks to the effort and dedication of everyone who helped make this right.

    by bunnie at November 08, 2017 03:19 PM

    November 06, 2017

    Harald Welte

    On the Linux Kernel Enforcement Statement

    I'm late with covering this here, but work overload is having its toll on my ability to blog.

    On October 16th, key Linux Kernel developers have released and anounced the Linux Kernel Community Enforcement Statemnt.

    In its actual text, those key kernel developers cover

    • compliance with the reciprocal sharing obligations of GPLv2 is critical and mandatory
    • acknowledgement to the right to enforce
    • expression of interest to ensure that enforcement actions are conducted in a manner beneficial to the larger community
    • a method to provide reinstatement of rights after ceasing a license violation (see below)
    • that legal action is a last resort
    • that after resolving any non-compliance, the formerly incompliant user is welcome to the community

    I wholeheartedly agree with those. This should be no surprise as I've been one of the initiators and signatories of the earlier statement of the netfilter project on GPL enforcement.

    On the reinstatement of rights

    The enforcement statement then specifically expresses the view of the signatories on the specific aspect of the license termination. Particularly in the US, among legal scholars there is a strong opinion that if the rights under the GPLv2 are terminated due to non-compliance, the infringing entity needs an explicit reinstatement of rights from the copyright holder. The enforcement statement now basically states that the signatories believe the rights should automatically be re-instated if the license violation ceases within 30 days of being notified of the license violation

    To people like me living in the European (and particularly German) legal framework, this has very little to no implications. It has been the major legal position that any user, even an infringing user can automatically obtain a new license as soon as he no longer violates. He just (really or imaginary) obtains a new copy of the source code, at which time he again gets a new license from the copyright holders, as long as he fulfills the license conditions.

    So my personal opinion as a non-legal person active in GPL compliance on the reinstatement statement is that it changes little to nothing regarding the jurisdiction that I operate in. It merely expresses that other developers express their intent and interest to a similar approach in other jurisdictions.

    by Harald Welte at November 06, 2017 11:00 PM

    SFLC sues SFC over trademark infringement

    As the Software Freedom Conservancy (SFC) has publicly disclosed on their website, it appears that Software Freedom Law Center (SFLC) has filed for a trademark infringement lawsuit against SFC.

    SFLC has launched SFC in 2006, and SFLC has helped and endorsed SFC in the past.

    This lawsuit is hard to believe. What has this community come to, if its various members - who used all to be respected equally - start filing law suits against each other?

    It's of course not known what kind of negotiations might have happened out-of-court before an actual lawsuit has been filed. Nevertheless, one would have hoped that people are able to talk to each other, and that the mutual respect for working at different aspects and with possibly slightly different strategies would have resulted in a less confrontational approach to resolving any dispute.

    To me, this story just looks like there can only be losers on all sides, by far not just limited to the two entities in question.

    On lwn.net some people, including high-ranking members of the FOSS community have started to spread conspiracy theories as to whether there's any secret scheming behind the scenes, particularly from the Linux Foundation towards SFLC to cause trouble towards the SFC and their possibly-not-overly-enjoyed-by-everyone enforcement activities.

    I think this is complete rubbish. Neither have I ever had the impression that the LF is completely opposed to license enforcement to begin with, nor do I have remotely enough phantasy to see them engage in such malicious scheming.

    What motivates SFLC and/or Eben to attack their former offspring is however unexplainable to the bystander. One hopes there is no connection to his departure from FSF about one year ago, where he served as general counsel for more than two decades.

    by Harald Welte at November 06, 2017 11:00 PM

    November 03, 2017

    Free Electrons

    Back from Kernel Recipes: slides and videos

    As we announced previously, we participated to the Embedded Recipes and Kernel Recipes conferences in September in Paris. Three people from Free Electrons attended the event: Free Electrons CEO Michael Opdenacker, and Free Electrons engineers Mylène Josserand and Maxime Ripard.

    Introduction to Yocto Project / OpenEmbedded, by Mylène Josserand

    Mylène Josserand gave an Introduction to the Yocto Project / OpenEmbedded-core. The slides are available in PDF or as LaTeX code.

    An introduction to the Linux DRM subsystem, by Maxime Ripard

    Maxime Ripard gave an Introduction to the Linux DRM subsystem. The slides are available in PDF or as LaTeX code.

    Other videos and slides

    The slides of all talks are available on the Embedded Recipes and Kernel Recipes conference websites. Youtube playlists are available for Embedded Recipes 2017 and Kernel Recipes 2017 as well.

    Conclusion

    With its special one track format, an attendance limited to 100 people, excellent choice of talks and nice social events, Kernel Recipes remains a very good conference that we really enjoyed. Embedded Recipes, which was in its first edition this year, followed the same principle, with the same success. We’re looking forward to attending next year editions, and hopefully contributing a few talks as well. See you all at Embedded and Kernel Recipes in 2018!

    by Thomas Petazzoni at November 03, 2017 08:44 AM

    November 02, 2017

    Free Electrons

    Back from ELCE 2017: slides and videos

    Free Electrons participated to the Embedded Linux Conference Europe last week in Prague. With 7 engineers attending, 4 talks, one BoF and a poster at the technical showcase, we had a strong presence to this major conference of the embedded Linux ecosystem. All of us had a great time at this event, attending interesting talks and meeting numerous open-source developers.

    Free Electrons team at the Embedded Linux Conference Europe 2017

    Free Electrons team at the Embedded Linux Conference Europe 2017. Top, from left to right: Maxime Ripard, Grégory Clement, Boris Brezillon, Quentin Schulz. Bottom, from left to right: Miquèl Raynal, Thomas Petazzoni, Michael Opdenacker.

    In this first blog post about ELCE, we want to share the slides and videos of the talks we have given during the conference.

    SD/eMMC: New Speed Modes and Their Support in Linux – Gregory Clement

    Grégory ClementSince the introduction of the original “default”(DS) and “high speed”(HS) modes, the SD card standard has evolved by introducing new speed modes, such as SDR12, SDR25, SDR50, SDR104, etc. The same happened to the eMMC standard, with the introduction of new high speed modes named DDR52, HS200, HS400, etc. The Linux kernel has obviously evolved to support these new speed modes, both in the MMC core and through the addition of new drivers.

    This talk will start by introducing the SD and eMMC standards and how they work at the hardware level, with a specific focus on the new speed modes. With this hardware background in place, we will then detail how these standards are supported by Linux, see what is still missing, and what we can expect to see in the future.

    Slides [PDF], Slides [LaTeX source]

    An Overview of the Linux Kernel Crypto Subsystem – Boris Brezillon

    Boris BrezillonThe Linux kernel has long provided cryptographic support for in-kernel users (like the network or storage stacks) and has been pushed to open these cryptographic capabilities to user-space along the way.

    But what is exactly inside this subsystem, and how can it be used by kernel users? What is the official userspace interface exposing these features and what are non-upstream alternatives? When should we use a HW engine compared to a purely software based implementation? What’s inside a crypto engine driver and what precautions should be taken when developing one?

    These are some of the questions we’ll answer throughout this talk, after having given a short introduction to cryptographic algorithms.

    Slides [PDF], Slides [LaTeX source]

    Buildroot: What’s New? – Thomas Petazzoni

    Thomas PetazzoniBuildroot is a popular and easy to use embedded Linux build system. Within minutes, it is capable of generating lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs.

    Since our last “What’s new” talk at ELC 2014, three and half years have passed, and Buildroot has continued to evolve significantly.

    After a short introduction about Buildroot, this talk will go through the numerous new features and improvements that have appeared over the last years, and show how they can be useful for developers, users and contributors.

    Slides [PDF], Slides [LaTeX source]

    Porting U-Boot and Linux on New ARM Boards: A Step-by-Step Guide – Quentin Schulz

    May it be because of a lack of documentation or because we don’t know where to look or where to start, it is not always easy to get started with U-Boot or Linux, and know how to port them to a new ARM platform.

    Based on experience porting modern versions of U-Boot and Linux on a custom Freescale/NXP i.MX6 platform, this talk will offer a step-by-step guide through the porting process. From board files to Device Trees, through Kconfig, device model, defconfigs, and tips and tricks, join this talk to discover how to get U-Boot and Linux up and running on your brand new ARM platform!

    Slides [PDF], Slides [LaTeX source]

    BoF: Embedded Linux Size – Michael Opdenacker

    This “Birds of a Feather” session will start by a quick update on available resources and recent efforts to reduce the size of the Linux kernel and the filesystem it uses.

    An ARM based system running the mainline kernel with about 3 MB of RAM will also be demonstrated.

    If you are interested in the size topic, please join this BoF and share your experience, the resources you have found and your ideas for further size reduction techniques!

    Slides [PDF], Slides [LaTeX source]

    by Thomas Petazzoni at November 02, 2017 10:54 AM

    October 31, 2017

    Free Electrons

    Free Electrons at NetDev 2.2

    NetDev 2.2Back in April 2017, Free Electrons engineer Antoine Ténart participated to NetDev 2.1, the most important conference discussing Linux networking support. After the conference, Antoine published a summary of it, reporting on the most interesting talks and topics that have been discussed.

    Next week, NetDev 2.2 takes place in Seoul, South Korea, and this time around, two Free Electrons engineers will be attending the event: Alexandre Belloni and Antoine Ténart. We are getting more and more projects with networking related topics, and therefore the wide range of talks proposed at NetDev 2.2 will definitely help grow our expertise in this field.

    Do not hesitate to get in touch with Alexandre or Antoine if you are also attending this event!

    by Thomas Petazzoni at October 31, 2017 11:00 AM

    October 30, 2017

    Open Hardware Repository

    WRS Low Jitter Daughterboard - WRS Low Jitter board measurement results

    - Additional hardware can improve jitter by two orders of magnitude -

    Chantal van Tour and Jeroen Koelemeij have made new measurements of the performance of the White Rabbit switch [1] with Mattia Rizzi’s additional Low jitter daughterboard (LJD) [2] integrated and enhanced even more with another clean-up oscillator. They describe their incredibly good results in the article “Sub -Nanosecond Time Accuracy and Frequency Distribution” [3]:

    The LJD improves the stability by about a factor of five, in good agreement with results earlier obtained by Rizzi et al. The LJD implementation improves the root mean-square (rms) phase jitter, integrated over the range 1 Hz – 100 kHz, from 13 ps to 2.9 ps.

    While the LJD implementation already significantly improves the default WR Slave phase noise in the low-frequency range, the PLO (clean-up oscillator) leads to a further suppression of noise by 30 to 40 dB over the frequency range 10 Hz – 100 kHz... As a result, the rms jitter of the 10 MHz output is as low as 0.18 ps over the range 1 Hz – 100 kHz, an improvement by nearly two orders of magnitude over default WR.

    The results presented here suggest that the improved WR system may in certain cases provide a good alternative for hydrogen masers. This may be true in particular for situations in which the observational phase coherence is limited by atmospheric conditions, rather than by local-oscillator stability. Another advantage of a WR-based frequency distribution system is that it also distributes phase. In other words, all nodes in the WR network will have the same phase, which may allow reducing the ‘search window’ in initial fringe searches.

    Note that the results with the additional clean-up oscillator require the White Rabbit Grandmaster (GM) to be locked to a sufficiently stable oscillator, such as a rubidium or cesium clock, or a hydrogen maser. The clean-up oscillator will not be able to track the free-running clock of a WR master, which is too unstable. With only the Low jitter daughterboard it will work fine on a free-running GM.

    We will discuss on how to proceed to make this hardware available via commercial partners, likely as special low-jitter versions of the White Rabbit Switch.

    [1] https://www.ohwr.org/projects/wr-switch-hw/wiki
    [2] https://www.ohwr.org/projects/wrs-low-jitter/wiki
    [3] https://library.nrao.edu/public/memos/ngvla/NGVLA_22.pdf

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at October 30, 2017 09:29 AM

    Bunnie Studios

    LiteX vs. Vivado: First Impressions

    Previously, I had written about developing a reference design for the NeTV2 FPGA using Xilinx’s Vivado toolchain. Last year at 33C3 Tim ‘mithro’ Ansell introduced me to LiteX and at his prompting I decided to give it a chance.

    Vivado was empowering because instead of having to code up a complex SoC in Verilog, I could use their pseudo-GUI/TCL interface to create a block diagram that largely automated the task of building the AXI routing fabric. Furthermore, I could access Xilinx’s extensive IP library, which included a very flexible DDR memory controller and a well-vetted PCI-express controller. Because of this level of design automation and available IP, a task that would have taken perhaps months in Verilog alone could be completed in a few days with the help of Vivado.

    The downsides of Vivado are that it’s not open source (free to download, but not free to modify), and that it’s not terribly efficient or speedy. Aside from the ideological objections to the closed-source nature of Vivado, there are some real, pragmatic impacts from the lack of source access. At a high level, Xilinx makes money selling FPGAs – silicon chips. However, to attract design wins they must provide design tools and an IP ecosystem. The development of this software is directly subsidized by the sale of chips.

    This creates an interesting conflict of interest when it comes to the efficiency of the tools – that is, how good they are at optimizing designs to consume the least amount of silicon possible. Spending money to create area-efficient tools reduces revenue, as it would encourage customers to buy cheaper silicon.

    As a result, the Vivado tool is pretty bad at optimizing designs for area. For example, the PCI express core – while extremely configurable and well-vetted – has no way to turn off the AXI slave bridge, even if you’re not using the interface. Even with the inputs unconnected or tied to ground, the logic optimizer won’t remove the unused gates. Unfortunately, this piece of dead logic consumes around 20% of my target FPGA’s capacity. I could only reclaim that space by hand-editing the machine-generated VHDL to comment out the slave bridge. It’s a simple enough thing to do, and it had no negative effects on the core’s functionality. But Xilinx has no incentive to add a GUI switch to disable the logic, because the extra gates encourage you to “upgrade” by one FPGA size if your design uses a PCI express core. Similarly, the DDR3 memory core devotes 70% of its substantial footprint to a “calibration” block. Calibration typically runs just once at boot, so the logic is idle during normal operation. With an FPGA, the smart thing to do would be to run the calibration, store the values, and then jam the pre-measured values into the application design, thus eliminating the overhead of the calibration block. However, I couldn’t implement this optimization since the DDR3 block is provided as an opaque netlist. Finally, the AXI fabric automation – while magical – scales poorly with the number of ports. In my most recent benchmark design done with Vivado, 50% of the chip is devoted to the routing fabric, 25% to the DDR3 block, and the remainder to my actual application logic.

    Tim mentioned that he thought the same design when using LiteX would fit in a much smaller FPGA. He has been using LiteX to generate the FPGA “gateware” (bitstreams) to support his HDMI2USB video processing pipelines on various platforms, ranging from the Numato-Opsis to the Atlys, and he even started a port for the NeTV2. Intrigued, I decided to port one of my Vivado designs to LiteX so that I could do an apples-to-apples comparison of the two design flows.

    LiteX is a soft-fork of Migen/MiSoC – a python-based framework for managing hardware IP and auto-generating HDL. The IP blocks within LiteX are completely open source, and so can be targeted across multiple FPGA architectures. However, for low-level synthesis, place & route, and bitstream generation, it still relies upon proprietary chip-specific vendor tools, such as Vivado when targeting Artix FPGAs. It’s a little bit like an open source C compiler that spits out assembly, so it still requires vendor-specific assemblers, linkers, and binutils. While it may seem backward to open the compiler before the assembler, remember that for software, an assembler’s scope of work is simple — primarily within well-defined 32-bit or so opcodes. However, for FPGAs, the “assembler” (place and route tool) has the job of figuring out where to place single-bit primitives within an “opcode” that’s effectively several million bits long, with potential cross-dependencies between every bit. The abstraction layers, while parallel, aren’t directly comparable.

    Let me preface my experience with the statement that I have a love-hate relationship with Python. I’ve used Python a few times for “recreational” projects and small tools, and for driving bits of automation frameworks. But I’ve found Python to be terribly frustrating. If you can use their frameworks from the ground-up, it’s intuitive, fun, even empowering. But if your application isn’t naturally “Pythonic”, woe to you. And I have a lot of needs for bit-banging, manipulating binary files, or grappling with low-level hardware registers, activities that are decidedly not Pythonic. I also spend a lot of time fighting with the “cuteness” of the Python type system and syntax: I’m more of a Rust person. I like strictly typed languages. I am not fond of novelties like using “-1” as the last-element array index and overloading the heck out of binary operators using magic methods.



    Comics courtesy of xkcd, CC BY-NC-2.5

    Surprisingly, I was able to get LiteX up and running within a day. This is thanks in large part to Tim’s effort to create a really comprehensive bootstrapping script that checks out the git repo, all of the submodules (thank you!), and manages your build environment. It just worked; the only bump I encountered was a bit of inconsistent documentation on installing the Xilinx toolchain (for Artix builds you need to grab Vivado; and Spartan you grab ISE). The whole thing ate about 19GiB of hard drive space, of which 18GiB is the Vivado toolchain.

    I was rewarded with a surprisingly powerful and mature framework for defining SoCs. Thanks to the extensive work of the MiSoC and LiteX crowd, there’s already IP cores for DRAM, PCI express, ethernet, video, a softcore CPU (your choice of or1k or lm32) and more. To be fair, I haven’t been able to load these on real hardware and validate their spec-compliance or functionality, but they seem to compile down to the right primitives so they’ve got the right shape and size. Instead of AXI, they’re using Wishbone for their fabric. It’s not clear to me yet how bandwidth-efficient the MiSoC fabric generator is, but the fact that it’s already in use to route 4x HDMI connections to DRAM on the Numato-Opsis would indicate that it’s got enough horsepower for my application (which only requires 3x HDMI connections).

    As a high-level framework, it’s pretty magical. Large IP instances and corresponding bus ports are allocated on-demand, based on a very high level description in Python. I feel a bit like a toddler who has been handed a loaded gun with the safety off. I’m praying the underlying layers are making sane inferences. But, at least in the case of LiteX, if I don’t agree with the decisions, it’s open source enough that I could try to fix things, assuming I have the time and gumption to do so.

    For my tool flow comparison, I implemented a simple 2x HDMI-in to DDR3 to 1x HDMI-out design in both Vivado and in LiteX. Creating the designs is about the same effort on both flows – once you have the basic IP blocks, instantiating bus fabric and allocation of addressing is largely automated in each case. Vivado is superior for pin/package layout thanks to its graphical planning tool (I find an illustration of the package layout to be much more intuitive than a textual list of ball-grid coordinates), and LiteX is a bit faster for design creation despite the usual frustrations I have with Python (up to the reader’s bias to decide whether it’s just that I have a different way of seeing things or if my intellect is insufficient to fully appreciate the goodness that is Python).


    Pad layout planning in Vivado is aided by a GUI


    Example of LiteX syntax for pin constraints

    But from there, the experience between the two diverges rapidly. The main thing that’s got me excited about LiteX is the speed and efficiency of its high-level synthesis. LiteX produces a design that uses about 20% of an XC7A50 FPGA with a runtime of about 10 minutes, whereas Vivado produces a design that consumes 85% of the same FPGA with a runtime of about 30-45 minutes.

    Significantly, LiteX tends to “fail fast”, so syntax errors or small problems with configurations become obvious within a few seconds, if not a couple minutes. However, Vivado tends to “fail late” – a small configuration problem may not pop up until about 20 minutes into the run, due to the clumsy way it manages out-of-context block synthesis and build dependencies. This means that despite my frustrations with the Python syntax, the penalty paid for small errors is much less in terms of time – so overall, I’m more productive.

    But the really compelling point is the efficiency. The fact that LiteX generates more efficient HDL means I can potentially shave a significant amount of cost out of a design by going to a smaller FPGA. Remember, both LiteX and Vivado use the same back-end for low-level sythesis and place and route. The difference is entirely in the high-level design automation – and this is a level that I can see being a good match for a Python-based framework. You’re not really designing hardware with Python (eventually it all turns into Verilog) so much as managing and configuring libraries of IP, something that Python is quite well suited for. To wit, I dug around in the MiSoC libraries a bit and there seem to be some serious logic designs using this Python syntax. I’m not sure I want to wrap my head around this coding style, but the good news is I can still write my leaf cells in Verilog and call them from the high-level Python integration framework.

    So, I’m cautiously proceeding to use LiteX as the main design flow going forward for NeTV2. We’ll see how the bitstream proves out in terms of timing and functionality once my next generation hardware is available, but I’m optimistic. I have a few concerns about how debugging will work – I’ve found the Xilinx ILA cores to be extremely powerful tools and the ability to automatically reverse engineer any complex design into a schematic (a feature built into Vivado) helps immensely with finding timing and logic bugs. But with a built-in soft CPU core, the “LiteScope” logic analyzer (with sigrok support coming soon), and fast build times, I have a feeling there is ample opportunity to develop new, perhaps even more powerful methods within LiteX to track down tricky bugs.

    My final thought is that LiteX, in its current state, is probably best suited for people trained to write software who want to design hardware, rather than for people classically trained in circuit design who want a tool upgrade. The design idioms and intuitions built into LiteX pulls strongly from the practices of software designers, which means a lot of “obvious” things are left undocumented that will throw outsiders (e.g. hardware designers like me) for a loop. There’s no question about the power and utility of the design flow – so, as the toolchain matures and documentation improves I’m optimistic that this could become a popular design flow for hardware projects of all magnitudes.


    Interested? Tim has suggested the following links for further reading:

    by bunnie at October 30, 2017 08:45 AM

    Free Electrons

    Buildroot training course updated to Buildroot 2017.08

    BuildrootBack in June 2015, we announced the availability of a training course on Buildroot, a popular and easy to use embedded Linux build system. A year later, we updated it to cover Buildroot 2016.05. We are happy to announce a new update: we now cover Buildroot 2017.08.

    The most significant updates are:

    • Presentation of the Long Term Supported releases of Buildroot, a topic we also presented in a previous blog post
    • Appearance of the new top-level utils/ directory, containing various utilities directly useful for the Buildroot user, such as test-pkg, check-package, get-developers or scanpypi
    • Removal of $(HOST_DIR)/usr/, as everything has been moved up one level to $(HOST_DIR), to make the Buildroot SDK/toolchain more conventional
    • Document the new organization of the skeleton package, now split into several packages, to properly support various init systems. A new diagram has been added to clarify this topic.
    • List all package infrastructures that are available in Buildroot, since their number is growing!
    • Use SPDX license codes for licensing information in packages, which is now mandatory in Buildroot
    • Remove the indication that dependencies of host (i.e native) packages are derived from the dependencies of the corresponding package, since it’s no longer the case
    • Indicate that the check for hashes has been extended to also allow checking the hash of license files. This allows to detect changes in the license text.
    • Update the BR2_EXTERNAL presentation to cover the fact that multiples BR2_EXTERNAL trees are supported now.
    • Use the new relocatable SDK functionality that appeared in Buildroot 2017.08.

    The practical labs have of course been updated to use Buildroot 2017.08, but also Linux 4.13 and U-Boot 2017.07, to remain current with upstream versions. In addition, they have been extended with two additional steps:

    • Booting the Buildroot generated system using TFTP and NFS, as an alternative to the SD card we normally use
    • Using genimage to generate a complete and ready to flash SD card image

    We will be delivering this course to one of our customers in Germany next month, and are of course available to deliver it on-site anywhere in the world if you’re interested! And of course, we continue to publish, for free, all the materials used in this training session: slides and labs.

    by Thomas Petazzoni at October 30, 2017 08:40 AM

    Open Hardware Repository

    Yet Another Micro-controller (YAM) - YAM assembler &#39;yamasm&#39; V1.1

    YAM assembler yamasm (now versionned as V1.1) has got a number of enhancements; main one being the 3-operand support.

    YAM core HDL is unchanged (still V1.4).
    A new component (namely wb_yam) has been added.
    It provides some YAM support through a Whisbone interface
    • Program download
    • Debugging...

    Software related sub-directories have been re-organized.

    The yamebs utility has dependencies to some ESRF packages.
    It is delivered as is and may be useful as a basis for supporting the wb_yam component.

    This new distribution is referred to as YAM V1.4.1.

    by Christian Herve at October 30, 2017 08:37 AM

    October 25, 2017

    Bunnie Studios

    Name that Ware October 2017

    The Ware for October 2017 is shown below.

    Sometimes a ware just presents itself to you among your travels; thankfully cell phone cameras have come a long way.

    by bunnie at October 25, 2017 06:40 PM

    Winner, Name that Ware September 2017

    The Ware for September 2017 is a WP 5007 Electrometer. I’ll give this one to Ingo, for the first mention of an electrometer. Congrats, email me for your prize! And @zebonaut, agreed, polystyrene caps FTW :)

    by bunnie at October 25, 2017 06:39 PM

    October 21, 2017

    Open Hardware Repository

    1:8 Pulse/Frequency Distribution Amplifier - Trimmer-cap modification for output-skew tuning on PDA

    The pulse-distribution amplifier output skew can be tuned close to zero by the addition of a trimmer-cap on the input of the output-buffer. See initial results in Anders' blog http://www.anderswallin.net/2017/09/delay-tuning-with-trimmer-caps/

    by Anders Wallin (anders.e.e.wallin@gmail.com) at October 21, 2017 04:34 PM

    October 19, 2017

    Harald Welte

    Obtaining the local IP address of an unbound UDP socket

    Sometimes one is finding an interesting problem and is surprised that there is not a multitude of blog post, stackoverflow answers or the like about it.

    A (I think) not so uncommon problem when working with datagram sockets is that you may want to know the local IP address that the OS/kernel chooses when sending a packet to a given destination.

    In an unbound UDP socket, you basically send and receive packets with any number of peers from a single socket. When sending a packet to destination Y, you simply pass the destination address/port into the sendto() socket function, and the OS/kernel will figure out which of its local IP addresses will be used for reaching this particular destination.

    If you're a dumb host with a single default router, then the answer to that question is simple. But in any reasonably non-trivial use case, your host will have a variety of physical and/or virtual network devices with any number of addresses on them.

    Why would you want to know that address? Because maybe you need to encode that address as part of a packet payload. In the current use case that we have, it is the OsmoMGW, implementing the IETF MGCP Media Gateway Control Protocol.

    So what can you do? You can actually create a new "trial" socket, not bind it to any specific local address/port, but connect() it to the destination of your IP packets. Then you do a getsockname(), which will give you the local address/port the kernel has selected for this socket. And that's exactly the answer to your question. You can now close the "trial" socket and have learned which local IP address the kernel would use if you were to send a packet to that destination.

    At least on Linux, this works. While getsockname() is standard BSD sockets API, I'm not sure how portable it is to use it on a socket that has not been explicitly bound by a prior call to bind().

    by Harald Welte at October 19, 2017 10:00 PM

    October 13, 2017

    Bunnie Studios

    Why I’m Using Bitmarks on my Products

    One dirty secret of hardware is that a profitable business isn’t just about design innovation, or even product cost reduction: it’s also about how efficiently one can move stuff from point A to B. This explains the insane density of hardware suppliers around Shenzhen; it explains the success of Ikea’s flat-packed furniture model; and it explains the rise of Amazon’s highly centralized, highly automated warehouses.

    Unfortunately, reverse logistics – the system for handling returns & exchanges of hardware products – is not something on the forefront of a hardware startup’s agenda. In order to deal with defective products, one has to ship a product first – an all-consuming goal. However, leaving reverse logistics as a “we’ll fix it after we ship” detail could saddle the venture with significant unanticipated customer support costs, potentially putting the entire business model at risk.

    This is because logistics are much more efficient in the “forward” direction: the cost of a centralized warehouse to deliver packages to an end consumer’s home address is orders of magnitude less than it is for a residential consumer to mail that same parcel back to the warehouse. This explains the miracle of Amazon Prime, when overnighting a pair of hand-knit mittens to your mother somehow costs you $20. Now repeat the hand-knit mittens thought experiment and replace it with a big-screen TV that has to find its way back to a factory in Shenzhen. Because the return shipment can no longer take advantage of bulk shipping discounts, the postage to China is likely more than the cost of the product itself!

    Because of the asymmetry in forward versus reverse logistics cost, it’s generally not cost effective to send defective material directly back to the original factory for refurbishing, recycling, or repair. In many cases the cost of the return label plus the customer support agent’s time will exceed the cost of the product. This friction in repatriating defective product creates opportunities for unscrupulous middlemen to commit warranty fraud.

    The basic scam works like this: a customer calls in with a defective product and gets sent a replacement. The returned product is sent to a local processing center, where it may be declared unsalvageable and slated for disposal. However, instead of a proper disposal, the defective goods “escape” the processing center and are resold as new to a different customer. The duped customer then calls in to exchange the same defective product and gets sent a replacement. Rinse lather repeat, and someone gets rich quick selling scrap at full market value.

    Similarly, high-quality counterfeits can sap profits from companies. Clones of products are typically produced using cut-rate or recycled parts but sold at full price. What happens when customers then find quality issues with the clone? That’s right – they call the authentic brand vendor and ask for an exchange. In this case, the brand makes zero money on the customer but incurs the full cost of supporting a defective product. This kind of warranty fraud is pandemic in smart phones and can cost producers many millions of dollars per year in losses.


    High-quality clones, like the card on the left, can cost businesses millions of dollars in warranty fraud claims.

    Serial numbers help mitigate these problems, but it’s easy to guess a simple serial number. More sophisticated schemes tie serial numbers to silicon IDs, but that necessitates a system which can reliably download the serialization data from the factory. This might seem a trivial task but for a lot of reasons – from failures in storage media to human error to poor Internet connectivity in factories – it’s much harder than it seems to make this happen. And for a startup, losing an entire lot of serialization data due to a botched upload could prove fatal.

    As a result, most hardware startups ship products with little to no plan for product serialization, much less a plan for reverse logistics. When the first email arrives from an unhappy customer, panic ensues, and the situation is quickly resolved, but by the time the product arrives back at the factory, the freight charges alone might be in the hundreds of dollars. Repeat this exercise a few dozen times, and any hope for a profitable run is rapidly wiped out.

    I’ve wrestled with this problem on and off through several startups of my own and finally landed on a solution that looks promising: it’s reasonably robust, fraud-resistant, and dead simple to implement. The key is the bitmark – a small piece of digital data that links physical products to the blockchain.

    Most people are familiar with blockchains through Bitcoin. Bitcoin uses the blockchain as a public ledger to prevent double-spending of the same virtual coin. This same public ledger can be applied to physical hardware products through a bitmark. Products that have been bitmarked can have their provenance tracked back to the factory using the public ledger, thus hampering cloning and warranty fraud – the physical equivalent of double-spending a Bitcoin.

    One of my most recent hardware startups, Chibitronics has teamed up with Bitmark to develop an end-to-end solution for Chibitronics’ newest microcontroller product, the Chibi Chip.

    As an open hardware business, we welcome people to make their own versions of our product, but we can’t afford to give free Chibi Chips to customers that bought cut-rate clones and then report them as defective for a free upgrade to an authentic unit. We’re also an extremely lean startup, so we can’t afford the personnel to build a full serialization and reverse logistics system from scratch. This is where Bitmark comes in.

    Bitmark has developed a turn-key solution for serialization and reverse logistics triage. They issue us bitmarks as lists of unique, six-word phrases. The six-word phrases are less frustrating for users to type in than strings of random characters. We then print the phrases onto labels that are stuck onto the back of each Chibi Chip.


    Bitmark claim code on the back of a Chibi Chip

    We release just enough of these pre-printed labels to the factory to run our authorized production quantities. This allows us to trace a bitmark back to a given production lot. It also prevents “ghost shifting” – that is, authorized factories producing extra bootleg units on a midnight shift that are sold into the market at deep discounts. Bitmark created a website for us where customers can then claim their bitmarks, thus registering their product and making it eligible for warranty service. In the event of an exchange or return, the product’s bitmark is updated to record this event. Then if a product fails to be returned to the factory, it can’t be re-claimed as defective because the blockchain ledger would evidence that bitmark as being mapped to a previously returned product. This allows us to defer the repatriation of the product to the factory. It also enables us to use unverified third parties to handle returned goods, giving us a large range of options to reduce reverse logistics costs.

    Bitmark also plans to roll out a site where users can verify the provenance of their bitmarks, so buyers can check if a product’s bitmark is authentic and if it has been previously returned for problems before they buy it. This increases the buyer’s confidence, thus potentially boosting the resale value of used Chibi Chips.

    For the cost and convenience of a humble printed label, Bitmark enhances control over our factories, enables production lot traceability, deters cloning, prevents warranty fraud, enhances confidence in the secondary market, and gives us ample options to streamline our reverse logistics.

    Of course, the solution isn’t perfect. A printed label can be peeled off one product and stuck on another, so people could potentially just peel labels off good products and resell the labels to users with broken clones looking to upgrade by committing warranty fraud. This scenario could be mitigated by using tamper-resistant labels. And for every label that’s copied by a cloner, there’s one victim who will have trouble getting support on an authentic unit. Also, if users are generally lax about claiming their bitmark codes, it creates an opportunity for labels to be sparsely duplicated in an effort to ghost-shift/clone without being detected; but this can be mitigated with a website update that encouraging customers to immediately register their bitmarks before using the web-based services tied to the product. We also have to exercise care in handling lists of unclaimed phrases because, until a customer registers their bitmark claim phrase in the blockchain, the phrases have value to would-be fraudsters.

    But overall, for the cost and convenience, the solution outperforms all the other alternatives I’ve explored to date. And perhaps most importantly for hardware startups like mine that are short on time and long on tasks, printing bitmarks is simple enough for us to implement that it’s hard to justify doing anything else.

    Disclosure: I am a technical advisor and shareholder of Bitmark.

    by bunnie at October 13, 2017 02:32 PM

    October 12, 2017

    Kristian Paul

    Affordable home automation

    I wanted to monitor my chickens remotely and after some web digging found this open source software homeassistant that had allowed me to use cheap devices like RPI and some dlink/digoo cameras in order to monitor then via web.
    
    
    
    It doesn't end there, i also bought the sonoff device and after some reflash i got it working with hass mqtt, so now i can also turn on light and fans.
    
    
    
    There are also things like automation, so when I'm at home my bluetooth devices presence gets lights on, small details that helps to make your life easier, like turning off lights automatically.
    
    There is probably more to do, next challenge is voice :-)
    
    [1] http://home-assistant.io/
    [2] https://github.com/arendst/Sonoff-MQTT-OTA
    

    October 12, 2017 05:00 AM

    October 09, 2017

    Harald Welte

    Invited keynote + TTCN-3 talk at netdevconf 2.2 in Seoul

    It was a big surprise that I've recently been invited to give a keynote on netfilter history at netdevconf 2.2.

    First of all, I wouldn't have expected netfilter to be that relevant next to all the other [core] networking topics at netdevconf. Secondly, I've not been doing any work on netfilter for about a decade now, so my memory is a bit rusty by now ;)

    Speaking of Rusty: Timing wise there is apparently a nice coincidence that I'll be able to meet up with him in Berlin later this month, i.e. hopefully we can spend some time reminiscing about old times and see what kind of useful input he has for the keynote.

    I'm also asking my former colleagues and successors in the netfilter project to share with me any note-worthy events or anecdotes, particularly also covering the time after my retirement from the core team. So if you have something that you believe shouldn't miss in a keynote on netfilter project history: Please reach out to me by e-mail ASAP and let me know about it.

    To try to fend off the elder[ly] statesmen image that goes along with being invited to give keynotes about the history of projects you were working on a long time ago, I also submitted an actual technical talk: TTCN-3 and Eclipse Titan for testing protocol stacks, in which I'll cover my recent journey into TTCN-3 and TITAN land, and how I think those tools can help us in the Linux [kernel] networking community to productively produce tests for the various protocols.

    As usual for netdevconf, there are plenty of other exciting talks in the schedule

    I'm very much looking forward to both visiting Seoul again, as well as meeting lots of the excellent people involved in the Linux networking subsystems. See ya!

    by Harald Welte at October 09, 2017 10:00 PM

    October 08, 2017

    Harald Welte

    Ten years Openmoko Neo1973 release anniversary dinner

    As I noted earlier this year, 2017 marks the tenth anniversary of shipping the first Openmoko phone, the Neo1973.

    On this occasion, a number of the key people managed to gather for an anniversary dinner in Taipei. Thanks for everyone who could make it, it was very good to see them together again. Sadly, by far not everyone could attend. You have been missed!

    The award for the most crazy attendee of the meeting goes out to my friend Milosch, who has actually flown from his home in the UK to Taiwan, only to meet up with old friends and attend the anniversary dinner.

    You can some pictures in Milosch's related tweet.

    by Harald Welte at October 08, 2017 10:00 PM

    October 05, 2017

    Free Electrons

    Buildroot Long Term Support releases: from 2017.02 to 2017.02.6 and counting

    Buildroot LogoBuildroot is a widely used embedded Linux build systems. A large number of companies and projects use Buildroot to produce customized embedded Linux systems for a wide range of embedded devices. Most of those devices are now connected to the Internet, and therefore subject to attacks if the software they run is not regularly updated to address security vulnerabilities.

    The Buildroot project publishes a new release every three months, with each release providing a mix of new features, new packages, package updates, build infrastructure improvements… and security fixes. However, until earlier this year, as soon as a new version was published, the maintenance of the previous version stopped. This means that in order to stay up to date in terms of security fixes, users essentially had two options:

    1. Update their Buildroot version regularly. The big drawback is that they get not only security updates, but also many other package updates, which may be problematic when a system is in production.
    2. Stick with their original Buildroot version, and carefully monitor CVEs and security vulnerabilities in the packages they use, and update the corresponding packages, which obvisouly is a time-consuming process.

    Starting with 2017.02, the Buildroot community has decided to offer one long term supported release every year: 2017.02 will be supported one year in terms of security updates and bug fixes, until 2018.02 is released. The usual three-month release cycle still applies, with 2017.05 and 2017.08 already being released, but users interested in a stable Buildroot version that is kept updated for security issues can stay on 2017.02.

    Since 2017.02 was released on February 28th, 2017, six minor versions were published on a fairly regularly basis, almost every month, except in August:

    With about 60 to 130 commits between each minor version, it is relatively easy for users to check what has been changed, and evaluate the impact of upgrading to the latest minor version to benefit from the security updates. The commits integrated in those minor versions are carefully chosen with the idea that users should be able to easily update existing systems.

    In total, those six minor versions include 526 commits, of which 183 commits were security updates, representing roughly one third of the total number of commits. The other commits have been:

    • 140 commits to fix build issues
    • 57 commits to bump versions of packages for bug fixes. These almost exclusively include updates to the Linux kernel, using its LTS versions. For other packages, we are more conservative and generally don’t upgrade them.
    • 17 commits to address issues in the licensing description of the packages
    • 186 commits to fix miscellaneous issues, ranging from runtime issues affecting packages to bugs in the build infrastructure

    The Buildroot community has already received a number of bug reports, patches or suggestions specifically targetting the 2017.02 LTS version, which indicates that developers and companies have started to adopt this LTS version.

    Therefore, if you are interested in using Buildroot for a product, you should probably consider using the LTS version! We very much welcome feedback on this version, and help in monitoring the security vulnerabilities affecting software packages in Buildroot.

    by Thomas Petazzoni at October 05, 2017 07:42 PM

    October 04, 2017

    Harald Welte

    On Vacation

    In case you're wondering about the lack of activity not only on this blog but also in git repositories, mailing lists and the like: I've been on vacation since September 13. It's my usual "one month in Taiwan" routine, during which I spend some time in Taipei, but also take several long motorbike tours around mostly rural Taiwan.

    You can find the occasional snapshot in my twitter feed, such as the, pictures, here and there.

    by Harald Welte at October 04, 2017 10:00 PM

    October 01, 2017

    Michele's GNSS blog

    Tribute to Prof. Kai Borre


    Whilst attending the latest ION GNSS+ conference I had the confirmation that Prof. Kai Borre disappeared this Summer. He has been a very important reference to me, especially in the early stages of my career. I am sure many other radio-navigation, geodesy and DSP engineers will convene with me. I knew him pretty well and I could not find anywhere an epitaph, so I today feel compelled to my leave my tribute to him here and hope others will intimately share my feeling.


    Rest in peace Kai. My gratitude, for inspiring me until your very last moment.

    by noreply@blogger.com (Michele Bavaro) at October 01, 2017 07:52 PM

    September 30, 2017

    Bunnie Studios

    Name that Ware, September 2017

    The Ware for September 2017 is shown below.

    And here is the underside of the plug-in module from the left hand side of the PCB:

    Thanks to Chris for sending in this gorgeous ware. I really appreciate both the aesthetic beauty of this ware, as well as the exotic construction techniques employed.

    by bunnie at September 30, 2017 03:34 PM

    Winner, Name that Ware August 2017

    The ware for August 2017 is the controller IC for a self-flashing (two-pin, T1 case) RGB LED. It’s photographed through the lens of the LED, which is why the die appears so distorted. Somehow, Piotr — the first poster — guessed it on the first try without much explanation. Congrats, email me for your prize!

    by bunnie at September 30, 2017 03:32 PM

    September 28, 2017

    Free Electrons

    Free Electrons opens a new office in Lyon, France

    After Toulouse and Orange, Lyon is the third city chosen for opening a Free Electrons office. Since September 1st of this year (2017), Alexandre Belloni and Grégory Clement have been working more precisely in Oullins close to the subway and the train station. It is the first step to make the Lyon team grow, with the opportunity to welcome interns and engineers.


    Their new desks are already crowded by many boards running our favorite system.

    by Gregory Clement at September 28, 2017 07:04 AM

    September 27, 2017

    Free Electrons

    Mali OpenGL support on Allwinner platforms with mainline Linux

    As most people know, getting GPU-based 3D acceleration to work on ARM platforms has always been difficult, due to the closed nature of the support for such GPUs. Most vendors provide closed-source binary-only OpenGL implementations in the form of binary blobs, whose quality depend on the vendor.

    This situation is getting better and better through vendor-funded initiatives like for the Broadcom VC4 and VC5, or through reverse engineering projects like Nouveau on Tegra SoCs, Etnaviv on Vivante GPUs, Freedreno on Qualcomm’s. However there are still GPUs where you do not have the option to use a free software stack: PowerVR from Imagination Technologies and Mali from ARM (even though there is some progress on the reverse engineering effort).

    Allwinner SoCs are using either a Mali GPU from ARM or a PowerVR from Imagination Technologies, and therefore, support for OpenGL on those platforms using a mainline Linux kernel has always been a problem. This is also further complicated by the fact that Allwinner is mostly interested in Android, which uses a different C library that avoids its use in traditional glibc-based systems (or through the use of libhybris).

    However, we are happy to announce that Allwinner gave us clearance to publish the userspace binary blobs that allows to get OpenGL supported on Allwinner platforms that use a Mali GPU from ARM, using a recent mainline Linux kernel. Of course, those are closed source binary blobs and not a nice fully open-source solution, but it nonetheless allows everyone to have OpenGL support working, while taking advantage of all the benefits of a recent mainline Linux kernel. We have successfully used those binary blobs on customer projects involving the Allwinner A33 SoCs, and they should work on all Allwinner SoCs using the Mali GPU.

    In order to get GPU support to work on your Allwinner platform, you will need:

    • The kernel-side driver, available on Maxime Ripard’s Github repository. This is essentially the Mali kernel-side driver from ARM, plus a number of build and bug fixes to make it work with recent mainline Linux kernels.
    • The Device Tree description of the GPU. We introduced Device Tree bindings for Mali GPUs in the mainline kernel a while ago, so that Device Trees can describe such GPUs. Such description has been added for the Allwinner A23 and A33 SoCs as part of this commit.
    • The userspace blob, which is available on Free Electrons GitHub repository. It currently provides the r6p2 version of the driver, with support for both fbdev and X11 systems. Hopefully, we’ll gain access to newer versions in the future, with additional features (such as GBM support).

    If you want to use it in your system, the first step is to have the GPU definition in your device tree if it’s not already there. Then, you need to compile the kernel module:

    git clone https://github.com/mripard/sunxi-mali.git
    cd sunxi-mali
    export CROSS_COMPILE=$TOOLCHAIN_PREFIX
    export KDIR=$KERNEL_BUILD_DIR
    export INSTALL_MOD_PATH=$TARGET_DIR
    ./build.sh -r r6p2 -b
    ./build.sh -r r6p2 -i
    

    It should install the mali.ko Linux kernel module into the target filesystem.

    Now, you can copy the OpenGL userspace blobs that match your setup, most likely the fbdev or X11-dma-buf variant. For example, for fbdev:

    git clone https://github.com/free-electrons/mali-blobs.git
    cd mali-blobs
    cp -a r6p2/fbdev/lib/lib_fb_dev/lib* $TARGET_DIR/usr/lib
    

    You should be all set. Of course, you will have to link your OpenGL applications or libraries against those user-space blobs. You can check that everything works using OpenGL test programs such as es2_gears for example.

    by Maxime Ripard at September 27, 2017 09:34 AM

    September 21, 2017

    Elphel

    Long range multi-view stereo camera with 4 sensors

    Figure 1. Four sensor stereo camera CAD model


    Four-camera stereo rig prototype is capable of measuring distances thousands times exceeding the camera baseline over wide (60 by 45 degrees) field of view. With 150 mm distance between lenses it provides ranging data at 200 meters with 10% accuracy, production units will have higher accuracy. Initial implementation uses software post-processing, but the core part of the software (tile processor) is designed as FPGA simulation and will be moved to the actual FPGA of the camera for the real time applications.

    Scroll down or just hyper-jump to Scene viewer for the links to see example images and reconstructed scenes.

    Background

    Most modern advances in the area of the visual 3d reconstruction are related to structure from motion (SfM) where high quality models are generated from the image sequences, including those from the uncalibrated cameras (such as cellphone ones). Another fast growing applications depend on the active ranging with either LIDAR scanning technology or time of flight (ToF) sensors.

    Each of these methods has its limitations and while widespread smart phone cameras attracted most of the interest in the algorithms and software development, there are some applications where the narrow baseline (distance between the sensors is much smaller, than the distance to the objects) technology has advantages.

    Such applications include autonomous vehicle navigation where other objects in the scene are moving and 3-d data is needed immediately (not when the complete image sequence is captured), and the elements to be ranged are ahead of the vehicle so previous images would not help much. ToF sensors are still very limited in range (few meters) and the scanning LIDAR systems are either slow to update or have very limited field of view. Passive (visual only) ranging may be desired for military applications where the system should stay invisible by not shining lasers around.

    Technology snippets

    Narrow baseline and subpixel resolution

    The main challenge for the narrow baseline systems is that the distance resolution is much worse than the lateral one. The minimal resolved 3d element, voxel is very far from resembling a cube (as 2d pixels are usually squares) – with the dimensions we use: pixel size – 0.0022 mm, lens focal length f = 4.5 mm and the baseline of 150 mm such voxel at 100 m distance is 50 mm high by 50 mm wide and 32 meters deep. The good thing is that while the lateral resolution generally is just one pixel (can be better only with additional knowledge about the object), the depth resolution can be improved with reasonable assumptions by an order of magnitude by using subpixel resolution. It is possible when there are multiple shifted images of the same object (that for such high range to baseline ratio can safely be assumed fronto-parallel) and every object is presented in each image by multiple pixels. With 0.1 pixel resolution in disparity (or shift between the two images) the depth dimension of the voxel at 100 m distance is 3.2 meters. And as we need multiple pixel objects for the subpixel disparity resolution, the voxel lateral dimensions increase (there is a way to restore the lateral resolution to a single pixel in most cases). With fixed-width window for the image matching we use 8×8 pixel grid (16×16 pixel overlapping tiles) similar to what is used by some image/video compression algorithms (such as JPEG) the voxel dimensions at 100 meter range become 0.4 m x 0.4 m x 3.2 m. Still not a cube, but the difference is significantly less dramatic.

    Subpixel accuracy and the lens distortions

    Matching images with subpixel accuracy requires that lens optical distortion of each lens is known and compensated with the same or better precision. Most popular way to present lens distortions is to use radial distortion model where relation of distorted and ideal pin-hole camera image is expressed as polynomial of point radius, so in polar coordinates the angle stays the same while the radius changes. Fisheye lenses are better described with “f-theta” model, where linear radial distance in the focal plane corresponds to the angle between the lens axis and ray to the object.

    Such radial models provide accurate results only with ideal lens elements and when such elements are assembled so that the axis of each individual lens element precisely matches axes of the other elements – both in position and orientation. In the real lenses each optical element has minor misalignment, and that limits the radial model. For the lenses we had dealt with and with 5MPix sensors it was possible to get down to 0.2 – 0.3 pixels, so we supplemented the radial distortion described by the analytical formula with the table-based residual image correction. Such correction reduced the minimal resolved disparity to 0.05 – 0.08 pixels.

    Fixed vs. variable window image matching and FPGA

    Modern multi-view stereo systems that work with wide baselines use elaborate algorithms with variable size windows when matching image pairs, down to single pixels. They aggregate data from the neighbor pixels at later processing stages, that allows them to handle occlusions and perspective distortions that make paired images different. With the narrow baseline system, ranging objects at distances that are hundreds to thousands times larger than the baseline, the difference in perspective distortions of the images is almost always very small. And as the only way to get subpixel resolution requires matching of many pixels at once anyway, use of the fixed size image tiles instead of the individual pixels does not reduce flexibility of the algorithm much.

    Processing of the fixed-size image tiles promises significant advantage – hardware-accelerated pixel-level tile processing combined with the higher level software that operates with the per-tile data rather than with per-pixel one. Tile processing can be implemented within the FPGA-friendly stream processing paradigm leaving decision making to the software. Matching image tiles may be implemented using methods similar to those used for image and especially video compression where motion vector estimation is similar to calculation of the disparity between the stereo images and similar algorithms may be used, such as phase-only correlation (PoC).

    Two dimensional array vs. binocular and inline camera rigs

    Usually stereo cameras or fixed baseline multi-view stereo are binocular systems, with just two sensors. Less common systems have more than two lenses positioned along the same line. Such configurations improve the useful camera range (ability to measure near and far objects) and reduce ambiguity when dealing with periodic object structures. Even less common are the rigs where the individual cameras form a 2d structure.

    In this project we used a camera with 4 sensors located in the corners of a square, so they are not co-linear. Correlation-based matching of the images depends on the detailed texture in the matched areas of the images – perfectly uniform objects produce no data for depth estimation. Additionally some common types of image details may be unsuitable for certain orientations of the camera baselines. Vertical concrete pole can be easily correlated by the two horizontally positioned cameras, but if the baseline is turned vertical, the same binocular camera rig would fail to produce disparity value. Similar is true when trying to capture horizontal features with the horizontal binocular system – such predominantly horizontal features are common when viewing near flat horizontal surfaces at high angles of incidents (almost parallel to the view direction).

    With four cameras we process four image pairs – 2 horizontal (top and bottom) and 2 vertical (right and left), and depending on the application requirements for particular image region it is possible to combine correlation results of all 4 pairs, or just horizontal and vertical separately. When all 4 baselines have equal length it is easier to combine image data before calculating the precise location of the correlation maximums – 2 pairs can be combined directly, and the 2 others after rotating tiles by 90 degrees (swapping X and Y directions, transposing the tiles 2d arrays).

    Image rectification and resampling

    Many implementations of the multi-view stereo processing start with the image rectification that involves correction for the perspective and lens distortions, projection of the individual images to the common plane. Such projection simplifies image tiles matching by correlation, but as it involves resampling of the images, it either reduces resolution or requires upsampling and so increases required memory size and processing complexity.

    This implementation does not require full de-warping of the images and related resampling with fractional pixel shifts. Instead we split geometric distortion of each lens into two parts:

    • common (average) distortion of all four lenses approximated by analytical radial distortion model, and
    • small residual deviation of each lens image transformation from the common distortion model

    Common radial distortion parameters are used to calculate matching tile location in each image, and while integer rounded pixel shifts of the tile centers are used directly when selecting input pixel windows, the fractional pixel remainders are preserved and combined with the other image shifts in the FPGA tile processor. Matching of the images is performed in this common distorted space, the tile grid is also mapped to this presentation, not to the fully rectified rectilinear image.

    Small individual lens deviations from the common distortion model are smooth 2-d functions over the 2-d image plane, they are interpolated from the calibration data stored for the lower resolution grid.

    We use low distortion sorted lenses with matching focal lengths to make sure that the scale mismatch between the image tiles is less than tile size in the target subpixel intervals (0.1 pix). Low distortion requirement extends the distances range to the near objects, because with the higher disparity values matching tiles in the different images land to the differently distorted areas. Focal length matching allows to use modulated complex lapped transform (CLT) that similar to discrete Fourier transform (DFT) is invariant to shifts, but not to scaling (log-polar coordinates are not applicable here, as such transformation would deny shift invariance).

    Enhancing images by correcting optical aberrations with space-variant deconvolution

    Matching of the images acquired with the almost identical lenses is rather insensitive to the lens aberrations that degrade image quality (mostly reduce sharpness), especially in the peripheral image areas. Aberration correction is still needed to get sharp textures in the result 3d models over full field of view, the resolution of the modern sensors is usually better than what lenses can provide. Correction can be implemented with space-variant (different kernels for different areas of the image) deconvolution, we routinely use it for post-processing of Eyesis4π images. The DCT-based implementation is described in the earlier blog post.

    Space-variant deconvolution kernels can absorb (be combined with during calibration processing) the individual lens deviations from the common distortion model, described above. Aberration correction and image rectification to the common image space can be performed simultaneously using the same processing resources.

    Two dimensional vs. single dimensional matching along the epipolar lines

    Common approach for matching image pairs is to replace the two-dimensional correlation with a single-dimensional task by correlating pixels along the epipolar lines that are just horizontal lines for horizontally built binocular systems with the parallel optical axes. Aggregation of the correlation maximums locations between the neighbor parallel lines of pixels is preformed in the image pixels domain after each line is processed separately.

    For tile-based processing it is beneficial to perform a full 2-d correlation as the phase correlation is performed in the frequency domain, and after the pointwise multiplication during aberration correction the image tiles are already available in the 2d frequency domain. Two dimensional correlation implies aggregation of data from multiple scan lines, it can tolerate (and be used to correct) small lens misalignments, with appropriate filtering it can be used to detect (and match) linear features.

    Implementation

    Prototype camera

    Experimental camera looks similar to Elphel regular H-camera – we just incorporated different sensor front ends (3d CAD model) that are used in Eyesis4π and added adjustment screws to align optical axes of the lenses (heading and tilt) and orientations of the image sensors (roll). Sensors are 5 Mpix 1/2″ format On Semiconductor MT9P006, lenses – Evetar N125B04530W.

    We selected lenses with the same focal length within 1%, and calibrated the camera using our standard camera rotation machine and the target pattern. As we do not yet have production adjustment equipment and software, the adjustment took several iterations: calibrating the camera and measuring extrinsic parameters of each sensor front end, then rotating each of the adjustment screws according to spreadsheet-calculated values, and then re-running the whole calibration process again. Finally the calibration results: radial distortion parameters, SFE extrinsic parameters, vignetting and deconvolution kernels were converted to the form suitable for run-time application (now – during post-processing of the captured images).

    Figure 2. Camera block diagram

    This prototype still uses 3d-printed parts and such mounts proved to be not stable enough, so we had to add field calibration and write code for bundle adjustment of the individual imagers orientations from the 2-d correlation data for each of the 4 individual pairs.

    Camera performance depends on the actual mechanical stability, software-compensation can only partially mitigate this misalignment problem and the precision of the distance measurements was reduced when cameras went off by more than 20 pixels after being carried in a backpack. Nevertheless the scene reconstruction remained possible.

    Software

    Multi-view stereo rigs are capable of capturing dynamic scenes so our goal is to make a real-time system with most of the heavy-weight processing be done in the FPGA.

    One of the major challenges here is how to combine parallel and stream processing capabilities of the FPGA with the flexibility of the software needed for implementation of the advanced 3d reconstruction algorithms. This approach is to use the FPGA-based tile processor to perform uniform operations on the lists of “tiles” – fixed square overlapping windows in the images. FPGA processes tile data at the pixel level, while software operates the whole tiles.

    Figure 2 shows the overall block diagram of the camera, Figure 3 illustrates details of the tile processor.

    Figure 3. FPGA tile processor

    Initial implementation does not contain actual FPGA processing, so far we only tested in FPGA some of the core functions – two dimensional 8×8 DCT-IV needed for both 16×16 CLT and ICLT. Current code consists of the two separate parts – one part (tile processor) simulates what will be moved to the FPGA (it handles image tiles at the pixel level), and the other one is what will remain software – it operates on the tile level and does not deal with the individual pixels. These two parts interact using shared system memory, tile processor has exclusive access to the dedicated image buffer and calibration data.

    Each tile is 16×16 pixels square with 8 pixel overlap, software prepares tile list including:

    • tile center X,Y (for the virtual “center” image),
    • center disparity, so the each of the 4 image tiles will be shifted accordingly, and
    • the code of operation(s) to be performed on that tile.

    Figure 4. Correlation processor

    Tile processor performs all or some (depending on the tile operation codes) of the following operations:

    • Reads the tile tasks from the shared system memory.
    • Calculates locations and loads image and calibration data from the external image buffer memory (using on-chip memory to cache data as the overlapping nature of the tiles makes each pixel to participate on average in 4 neighbor tiles).
    • Converts tiles to frequency domain using CLT based on 2d DCT-IV and DST-IV.
    • Performs aberration correction in the frequency domain by pointwise multiplication by the calibration kernels.
    • Calculates correlation-related data (Figure 4) for the tile pairs, resulting in tile disparity and disparity confidence values for all pairs combined, and/or more specific correlation types by pointwise multiplication, inverse CLT to the pixel domain, filtering and local maximums extraction by quadratic interpolation or windowed center of mass calculation.
    • Calculates combined texture for the tile (Figure 5), using alpha channel to mask out pixels that do not match – this is the way how to effectively restore single-pixel lateral resolution after aggregating individual pixels to tiles. Textures can be combined after only programmed shifts according to specified disparity, or use additional shift calculated in the correlation module.
    • Calculates other integral values for the tiles (Figure 5), such as per-channel number of mismatched pixels – such data can be used for quick second-level (using tiles instead of pixels) correlation runs to determine which 3d volumes potentially have objects and so need regular (pixel-level) matching.
    • Finally tile processor saves results: correlation values and/or texture tile to the shared system memory, so software can access this data.

    Figure 5. Texture processor

    Single tile processor operation deals with the scene objects that would be projected to this tile’s 16×16 pixels square on the sensor of the virtual camera located in the center between the four actual physical cameras. The single pass over the tile data is limited not just laterally, but in depth also because for the tiles to correlate they have to have significant overlap. 50% overlap corresponds to the correlation offset range of ±8 pixels, better correlation contrast needs 75% overlap or ±4 pixels. The tile processor “probes” not all the voxels that project to the same 16×16 window of the virtual image, but only those that belong to the certain distance range – the distances that correspond to the disparities ±4 pixels from the value provided for the tile.

    That means that a single processing pass over a tile captures data in a disparity space volume, or a macro-voxel of 8 pixels wide by 8 pixels high by 8 pixels deep (considering the central part of the overlapping volumes). And capturing the whole scene may require multiple passes for the same tile with different disparity. There are ways how to avoid full range disparity sweep (with 8 pixel increments) for all tiles – following surfaces and detecting occlusions and discontinuities, second-level correlation of tiles instead of the individual pixels.

    Another reason for the multi-pass processing of the same tile is to refine the disparity measured by correlation. When dealing with subpixel coordinates of the correlation maximums – either located by quadratic approximation or by some form of center of mass evaluation, the calculated values may have bias and disparity histograms reveal modulation with the pixel period. Second “refine” pass, where individual tiles are shifted by the disparity measured in the previous pass reduces the residual offset of the correlation maximum to a fraction of a pixel and mitigates this type of bias. Tile shift here means a combination of the integer pixel shift of the source images and the fractional (in the ±0.5 pixel range) shift that is performed in the frequency domain by multiplication by the cosine/sine phase rotator.

    Total processing time and/or required FPGA resources linearly depend on the number of required tile processor operations and the software may use several methods to reduce this number. In addition to the two approaches mentioned above (following surfaces and second-level correlation) it may be possible to reduce the field of view to a smaller area of interest, predict current frame scene from the previous frames (as in 2d video compression) – tile processor paradigm preserves flexibility of the various algorithms that may be used in the scene 3d reconstruction software stack.

    Scene viewer

    The viewer for the reconstructed scenes is here: https://community.elphel.com/3d+map (viewer source code).

    Figure 6. 3d+map index page

    Index page shows a map (you may select from several providers) with the markers for the locations of the captured scenes. On the left there is a vertical ribbon of the thumbnails – you may scroll it with a mouse wheel or by dragging.

    Thumbnails are shown only for the markers that fit on screen, so zooming in on the map may reduce number of the visible thumbnails. When you select some thumbnail, the corresponding marker opens on the map, and one or several scenes are shown – one line per each scene (identified by the Unix timestamp code with fractional seconds) captured at the same locations.

    The scene that matches the selected thumbnail is highlighted (as 4-th line in the Figure 6). Some scenes have different versions of reconstruction from the same source images – they are listed in the same line (like first line in the Figure 6). Links lead to the viewers of the selected scene/version.

    Figure 7. Selection of the map / satellite imagery provider

    We do not have ground truth models for the captured scenes build with the active scanners. Instead as the most interesting is ranging of the distant objects (hundreds of meters) it is possible to use publicly available satellite imagery and match it to the captured models. We had ideal view from Elphel office window – each crack on the pavement was visible in the satellite images so we could match them with the 3d model of the scene. Unfortunately they ruined it recently by replacing asphalt :-).

    The scene viewer combines x3dom representation of the 3d scene and the re-sizable overlapping map view. You may switch the map imagery provider by clicking on the map icon as shown in the Figure 7.

    The scene and map views are synchronized to each other, there are several ways of navigation in either 3d or map area:

    • drag the 3d view to rotate virtual camera without moving;
    • move cross-hair icon in the map view to rotate camera around vertical axis;
    • toggle button and adjust camera view elevation;
    • use scroll wheel over the 3d area to change camera zoom (field of view is indicated on the map);
    • drag with middle button pressed in the 3d view to move camera perpendicular to the view direction;
    • drag the the camera icon (green circle) on the map to move camera horizontally;
    • toggle button and move the camera vertically;
    • press a hotkey t over the 3d area to reset to the initial view: set azimuth and elevation same as captured;
    • press a hotkey r over the 3d area to set view azimuth as captured, elevation equal to zero (horizontal view).

    Figure 8. 3D model to map comparison

    Comparison of the 3d scene model and the map uses ball markers. By default these markers are one meter in diameter, the size can be changed on the settings () page.

    Moving pointer over the 3d area with Ctrl key pressed causes the ball to follow the cursor at a distance where the view line intersects the nearest detected surface in the scene. It simultaneously moves the corresponding marker over the map view and indicates the measured distance.

    Ctrl-click places the ball marker on the 3d scene and on the map. It is then possible to drag the marker over the map and read the ground truth distance. Dragging the marker over the 3d scene updates location on the map, but not the other way around, in edit mode mismatch data is used to adjust the captured scene location and orientation.

    Program settings used during reconstruction limit the scene far distance to z = 1000 meters, all more distant objects are considered to be located at infinity. X3d allows to use images at infinity using backdrop element, but it is not flexible enough and is not supported by some other programs. In most models we place infinity textures to a large billboard at z = 10,000 meters, and it is where the ball marker will appear if placed on the sky or other far objects.

    Figure 9. Settings and link to four images

    The settings page () shown in the Figure 9 has a link to the four-image viewer (Figure 10). These four images correspond to the captured views and are almost “raw images” used for scene reconstruction. These images were subject to the optical aberration correction and are partially rectified – they are rendered as if they were captured by the same camera that has only strictly polynomial radial distortion.

    Such images are not actually used in the reconstruction process, they are rendered only for the debug and demonstration purposes. The equivalent data exists in the tile processor only in the frequency domain form as an intermediate result, and was subject to just linear processing (to avoid possible unintended biases) so the images have some residual locally-checkerboard pattern that is due to the Bayer mosaic filter (discussed in the earlier blog). Textures that are generated from the combination of all four images have the contrast of such pattern significantly lower. It is possible to add some non-linear filtering at the very last stage of the texture generation.

    Each scene model has a download link for the archive that contains the model itself as *.x3d file and Wavefront *.obj and *.mtl as well as the corresponding RGBA texture files as PNG images. Initially I missed the fact that x3d and obj formats have opposite direction of surface normals for the same triangular faces, so almost half of the Wavefront files still have incorrect (opposite direction) surface normals.

    Results

    Our initial plan was to test algorithms for the tile processor before implementing them in FPGA. The tile processor provides data for the disparity space image (DSI) – confidence value of having certain disparity for specified 2d position in the image, it also generates texture tiles.

    When the tile processor code was written and tested, we still needed some software to visualize the results. DSI itself seemed promising (much better coverage than what I had with earlier experiments with binocular images), but when I tried to convert these textured tiles into viewable x3d model directly, it was a big disappointment. Result did not look like a 3d scene – there were many narrow triangles that made sense only when viewed almost directly from the camera actual location, a small lateral viewpoint movement – and the image was falling apart into something unrecognizable.

    Figure 10. Four channel images (click for actual viewer with zoom/pan capability)

    I was not able to find ready to use code and the plan to write a quick demo for the tile processor and generated DSI seemed less and less realistic. Eventually it took at least three times longer to get somewhat usable output than to develop DCT-based tile processor code itself.

    Current software is still incomplete, lacks many needed features (it even does not cut off background so wires over the sky steal a lot of surrounding space), it runs slow (several minutes per single scene), but it does provide a starting point to evaluate performance of the long range 4-camera multi-view stereo system. Much of the intended functionality does not work without more parameter tuning, but we decided to postpone improvements to the next stage (when we will have cameras that are more stable mechanically) and instead try to capture more of very different scenes, process them in batch mode (keeping the same parameter values for all new scenes) and see what will be the output.

    As soon as the program was able to produce somewhat coherent 3d model from the very first image set captured through Elphel office window, Oleg Dzhimiev started development of the web application that allows to match the models with the map data. After adding more image sets I noticed that the camera calibration did not hold. Each individual sub-camera performed nicely (they use thermally compensated mechanical design), but their extrinsic parameters did change and we had to add code for field calibration that uses image themselves. The best accuracy in disparity measurement over the field of view still requires camera poses to match ones used at full calibration, so later scenes with more developed misalignment (>20 pixels) are less precise than earlier (captured in Salt Lake City).

    We do not have an established method to measure ranging precision for different distances to object – the disparity values are calculated together with the confidence and in lower confidence areas the accuracy is lower, including places where no ranging is possible due to the complete absence of the visible details in the images. Instead it is possible to compare distances in various scene models to those on the map and see where such camera is useful. With 0.1 pixel disparity resolution and 150 mm baseline we should be able to measure 300 m distances with 10% accuracy, and for many captured scene objects it already is not much worse. We now placed orders to machine the new camera parts that are needed to build a more mechanically stable rig. And parallel to upgrading the hardware, we’ll start migrating the tile processor code from Java to Verilog.

    And what’s next? Elphel goal is to provide our users with the high performance hackable products and freedom to modify them in the ways and for the purposes we could not imagine ourselves. But it is fun to fantasize about at least some possible applications:

    • Obviously, self-driving cars – increased number of cameras located in a 2d pattern (square) results in significantly more robust matching even with low-contrast textures. It does not depend on sequential scanning and provides simultaneous data over wide field of view. Calculated confidence of distance measurements tells when alternative (active) ranging methods are needed – that would help to avoid infamous accident with a self-driving car that went under a truck.
    • Visual odometry for the drones would also benefit from the higher robustness of image matching.
    • Rovers on Mars or other planets using low-power passive (visual based) scene reconstruction.
    • Maybe self-flying passenger multicopters in the heavy 3d traffic? Sure they will all be equipped with some transponders, but what about aerial roadkills? Like a flock of geese that forced water landing.
    • High speed boating or sailing over uneven seas with active hydrofoils that can look ahead and adjust to the future waves.
    • Landing on the asteroids for physical (not just Bitcoin) mining? With 150 mm baseline such camera can comfortably operate within several hundred meters from the object, with 1.5 m that will scale to kilometers.
    • Cinematography: post-production depth of field control that would easily beat even the widest format optics, HDR with a pair of 4-sensor cameras, some new VFX?
    • Multi-spectral imaging where more spatially separate cameras with different bandpass filters can be combined to the same texture in the 3d scene.
    • Capturing underwater scenes and measuring how far the sea creatures are above the bottom.

    by Andrey Filippov at September 21, 2017 05:40 AM

    September 12, 2017

    Open Hardware Repository

    White Rabbit - 12-09-2017: PTP Trackhound smells the White Rabbit

    The software PTP Track Hound which can capture and analyze PTP network traffic now understands White Rabbit TLVs. So the Track Hound can now sniff the tracks that the White Rabbit leaves behind.
    Track Hound is made freely available by Meinberg. One may want to know that the source code is not available under an Open Licence.

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at September 12, 2017 07:35 AM

    September 06, 2017

    Free Electrons

    Free Electrons at the Embedded Linux Conference Europe

    The next Embedded Linux Conference Europe will take place on October 23-25 in Prague, Czech Republic.

    Embedded Linux Conference Europe 2017

    As usual, a significant part of the Free Electrons engineering team will participate to the conference and give talks on various topics:

    In addition to the main ELCE conference, Thomas Petazzoni will participate to the Buildroot Developers Days, a 2-day hackaton organized on Saturday and Sunday prior to ELCE, and will participate to the Device Tree workshop organized on Thursday afternoon.

    Once again, we’re really happy to participate to this conference, and looking forward to meeting again with a large number of Linux kernel and embedded Linux developers!

    by Thomas Petazzoni at September 06, 2017 11:56 AM

    September 05, 2017

    Free Electrons

    Linux 4.13 released, Free Electrons contributions

    Linux 4.13 was released last Sunday by Linus Torvalds, and the major new features of this release were described in details by LWN in a set of articles: part 1 and part 2.

    This release gathers 13006 non-merge commits, amongst which 239 were made by Free Electrons engineers. According to the LWN article on 4.13 statistics, this makes Free Electrons the 13th contributing company by number of commits, the 10th by lines changed.

    The most important contributions from Free Electrons for this release have been:

    • In the RTC subsystem
      • Alexandre Belloni introduced a new method for registering RTC devices, with one step for the allocation, and one step for the registration itself, which allows to solve race conditions in a number of drivers.
      • Alexandre Belloni added support for exposing the non-volatile memory found in some RTC devices through the Linux kernel nvmem framework, making them usable from userspace. A few drivers were changed to use this new mechanism.
    • In the MTD/NAND subsystem
      • Boris Brezillon did a large number of fixes and minor improvements in the NAND subsystem, both in the core and in a few drivers.
      • Thomas Petazzoni contributed the support for on-die ECC, specifically with Micron NANDs. This allows to use the ECC calculation capabilities of the NAND chip itself, as opposed to using software ECC (calculated by the CPU) or ECC done by the NAND controller.
      • Thomas Petazzoni contributed a few improvements to the FSMC NAND driver, used on ST Spear platforms. The main improvement is to support the ->setup_data_interface() callback, which allows to configure optimal timings in the NAND controller.
    • Support for Allwinner ARM platforms
      • Alexandre Belloni improved the sun4i PWM driver to use the so-called atomic API and support hardware read out.
      • Antoine Ténart improved the sun4i-ss cryptographic engine driver to support the Allwinner A13 processor, in addition to the already supported A10.
      • Maxime Ripard contributed HDMI support for the Allwinner A10 processor (in the DRM subsystem) and a number of related changes to the Allwinner clock support.
      • Quentin Schulz improved the support for battery charging through the AXP20x PMIC, used on Allwinner platforms.
    • Support for Atmel ARM platforms
      • Alexandre Belloni added suspend/resume support for the Atmel SAMA5D2 clock driver. This is part of a larger effort to implement the backup mode for the SAMA5D2 processor.
      • Alexandre Belloni added suspend/resume support in the tcb_clksrc driver, used as for clocksource and clockevents on Atmel SAMA5D2.
      • Alexandre Belloni cleaned up a number of drivers, removing support for non-DT probing, which is possible now that the AVR32 architecture has been dropped. Indeed, the AVR32 processors used to share the same drivers as the Atmel ARM processors.
      • Alexandre Belloni added the core support for the backup mode on Atmel SAMA5D2, a suspend/resume state with significant power savings.
      • Boris Brezillon switched Atmel platforms to use the new binding for the EBI and NAND controllers.
      • Boris Brezillon added support for timing configuration in the Atmel NAND driver.
      • Quentin Schulz added suspend/resume support to the Bosch m_can driver, used on Atmel platforms.
    • Support for Marvell ARM platforms
      • Antoine Ténart contributed a completely new driver (3200+ lines of code) for the Inside Secure EIP197 cryptographic engine, used in the Marvell Armada 7K and 8K processors. He also subsequently contributed a number of fixes and improvements for this driver.
      • Antoine Ténart improved the existing mvmdio driver, used to communicate with Ethernet PHYs over MDIO on Marvell platforms to support the XSMI variant found on Marvell Armada 7K/8K, used to communicate with 10G capable PHYs.
      • Antoine Ténart contributed minimal support for 10G Ethernet in the mvpp2 driver, used on Marvell Armada 7K/8K. For now, the driver still relies on low-level initialization done by the bootloader, but additional changes in 4.14 and 4.15 will remove this limitation.
      • Grégory Clement added a new pinctrl driver to configure the pin-muxing on the Marvell Armada 37xx processors.
      • Grégory Clement did a large number of changes to the clock drivers used on the Marvell Armada 7K/8K processors to prepare the addition of pinctrl support.
      • Grégory Clement added support for Marvell Armada 7K/8K to the existing mvebu-gpio driver.
      • Thomas Petazzoni added support for the ICU, a specialized interrupt controller used on the Marvell Armada 7K/8K, for all devices located in the CP110 part of the processor.
      • Thomas Petazzoni removed a work-around to properly resume per-CPU interrupts on the older Marvell Armada 370/XP platforms.
    • Support for RaspberryPi platforms
      • Boris Brezillon added runtime PM support to the HDMI encoder driver used on RaspberryPi platforms, and contributed a few other fixes to the VC4 DRM driver.

    It is worth mentioning that Miquèl Raynal, recently hired by Free Electrons, sees his first kernel patch merged: nand: fix wrong default oob layout for small pages using soft ecc.

    Free Electrons engineers are not only contributors, but also maintainers of various subsystems in the Linux kernel, which means they are involved in the process of reviewing, discussing and merging patches contributed to those subsystems:

    • Maxime Ripard, as the Allwinner platform co-maintainer, merged 113 patches from other contributors
    • Boris Brezillon, as the MTD/NAND maintainer, merged 62 patches from other contributors
    • Alexandre Belloni, as the RTC maintainer and Atmel platform co-maintainer, merged 57 patches from other contributors
    • Grégory Clement, as the Marvell EBU co-maintainer, merged 47 patches from other contributors

    Here is the commit by commit detail of our contributors to 4.13:

    by Thomas Petazzoni at September 05, 2017 07:21 AM

    September 02, 2017

    Harald Welte

    Purism Librem 5 campaign

    There's a new project currently undergoing crowd funding that might be of interest to the former Openmoko community: The Purism Librem 5 campaign.

    Similar to Openmoko a decade ago, they are aiming to build a FOSS based smartphone built on GNU/Linux without any proprietary drivers/blobs on the application processor, from bootloader to userspace.

    Furthermore (just like Openmoko) the baseband processor is fully isolated, with no shared memory and with the Linux-running application processor being in full control.

    They go beyond what we wanted to do at Openmoko in offering hardware kill switches for camera/phone/baseband/bluetooth. During Openmoko days we assumed it is sufficient to simply control all those bits from the trusted Linux domain, but of course once that might be compromised, a physical kill switch provides a completely different level of security.

    I wish them all the best, and hope they can leave a better track record than Openmoko. Sure, we sold some thousands of phones, but the company quickly died, and the state of software was far from end-user-ready. I think the primary obstacles/complexities are verification of the hardware design as well as the software stack all the way up to the UI.

    The budget of ~ 1.5 million seems extremely tight from my point of view, but then I have no information about how much Puri.sm is able to invest from other sources outside of the campaign.

    If you're a FOSS developer with a strong interest in a Free/Open privacy-first smartphone, please note that they have several job openings, from Kernel Developer to OS Developer to UI Developer. I'd love to see some talents at work in that area.

    It's a bit of a pity that almost all of the actual technical details are unspecified at this point (except RAM/flash/main-cpu). No details on the cellular modem/chipset used, no details on the camera, neither on the bluetooth chipset, wifi chipset, etc. This might be an indication of the early stage of their plannings. I would have expected that one has ironed out those questions before looking for funding - but then, it's their campaign and they can run it as they see it fit!

    I for my part have just put in a pledge for one phone. Let's see what will come of it. In case you feel motivated by this post to join in: Please keep in mind that any crowdfunding campaign bears significant financial risks. So please make sure you made up your mind and don't blame my blog post for luring you into spending money :)

    by Harald Welte at September 02, 2017 10:00 PM

    Uwe Hermann

    Website reconstruction

    Website reconstruction Uwe Hermann Sat, 09/02/2017 - 13:36

    by Uwe Hermann at September 02, 2017 11:36 AM

    September 01, 2017

    Harald Welte

    The sad state of voice support in cellular modems

    Cellular modems have existed for decades and come in many shapes and kinds. They contain the cellular baseband processor, RF frontend, protocol stack software and anything else required to communicate with a cellular network. Basically a phone without display or input.

    During the last decade or so, the vast majority of cellular modems come as LGA modules, i.e. a small PCB with all components on the top side (and a shielding can), which has contact pads on the bottom so you can solder it onto your mainboard. You can obtain them from vendors such as Sierra Wireless, u-blox, Quectel, ZTE, Huawei, Telit, Gemalto, and many others.

    In most cases, the vendors now also solder those modules to small adapter boards to offer the same product in mPCIe form-factor. Other modems are directly manufactured in mPCIe or NGFF aka m.2 form-factor.

    As long as those modems were still 2G / 2.5G / 2.75G, the main interconnection with the host (often some embedded system) was a serial UART. The Audio input/output for voice calls was made available as analog signals, ready to connect a microphone and spekaer, as that's what the cellular chipsets were designed for in the smartphones. In the Openmoko phones we also interfaced the audio of the cellular modem in analog, exactly for that reason.

    From 3G onwards, the primary interface towards the host is now USB, with the modem running as a USB device. If your laptop contains a cellular modem, you will see it show up in the lsusb output.

    From that point onwards, it would have made a lot of sense to simply expose the audio also via USB. Simply offer a multi-function USB device that has both whatever virutal serial ports for AT commands and network device for IP, and add a USB Audio device to it. It would simply show up as a "USB sound card" to the host, with all standard drivers working as expected. Sadly, nobody seems to have implemented this, at least not in a supported production version of their product

    Instead, what some modem vendors have implemented as an ugly hack is the transport of 8kHz 16bit PCM samples over one of the UARTs. See for example the Quectel UC-20 or the Simcom SIM7100 which implement such a method.

    All the others ignore any acess to the audio stream from software to a large part. One wonders why that is. From a software and systems architecture perspective it would be super easy. Instead, what most vendors do, is to expose a digital PCM interface. This is suboptimal in many ways:

    • there is no mPCIe standard on which pins PCM should be exposed
    • no standard product (like laptop, router, ...) with mPCIe slot will have anything connected to those PCM pins

    Furthermore, each manufacturer / modem seems to support a different subset of dialect of the PCM interface in terms of

    • voltage (almost all of them are 1.8V, while mPCIe signals normally are 3.3V logic level)
    • master/slave (almost all of them insist on being a clock master)
    • sample format (alaw/ulaw/linear)
    • clock/bit rate (mostly 2.048 MHz, but can be as low as 128kHz)
    • frame sync (mostly short frame sync that ends before the first bit of the sample)
    • endianness (mostly MSB first)
    • clock phase (mostly change signals at rising edge; sample at falling edge)

    It's a real nightmare, when it could be so simple. If they implemented USB-Audio, you could plug a cellular modem into any board with a mPCIe slot and it would simply work. As they don't, you need a specially designed mainboard that implements exactly the specific dialect/version of PCM of the given modem.

    By the way, the most "amazing" vendor seems to be u-blox. Their Modems support PCM audio, but only the solder-type version. They simply didn't route those signals to the mPCIe slot, making audio impossible to use when using a connectorized modem. How inconvenient.

    Summary

    If you want to access the audio signals of a cellular modem from software, then you either

    • have standard hardware and pick one very specific modem model and hope this is available sufficiently long during your application, or
    • build your own hardware implementing a PCM slave interface and then pick + choose your cellular modem

    On the Osmocom mpcie-breakout board and the sysmocom QMOD board we have exposed the PCM related pins on 2.54mm headers to allow for some separate board to pick up that PCM and offer it to the host system. However, such separate board hasn't been developed so far.

    by Harald Welte at September 01, 2017 10:00 PM

    First actual XMOS / XCORE project

    For many years I've been fascinated by the XMOS XCore architecture. It offers a surprisingly refreshing alternative virtually any other classic microcontroller architectures out there. However, despite reading a lot about it years ago, being fascinated by it, and even giving a short informal presentation about it once, I've so far never used it. Too much "real" work imposes a high barrier to spending time learning about new architectures, languages, toolchains and the like.

    Introduction into XCore

    Rather than having lots of fixed-purpose built-in "hard core" peripherals for interfaces such as SPI, I2C, I2S, etc. the XCore controllers have a combination of

    • I/O ports for 1/4/8/16/32 bit wide signals, with SERDES, FIFO, hardware strobe generation, etc
    • Clock blocks for using/dividing internal or external clocks
    • hardware multi-threading that presents 8 logical threads on each core
    • xCONNECT links that can be used to connect multiple processors over 2 or 5 wires per direction
    • channels as a means of communication (similar to sockets) between threads, whether on the same xCORE or a remote core via xCONNECT
    • an extended C (xC) programming language to make use of parallelism, channels and the I/O ports

    In spirit, it is like a 21st century implementation of some of the concepts established first with Transputers.

    My main interest in xMOS has been the flexibility that you get in implementing not-so-standard electronics interfaces. For regular I2C, UART, SPI, etc. there is of course no such need. But every so often one encounters some interface that's very rately found (like the output of an E1/T1 Line Interface Unit).

    Also, quite often I run into use cases where it's simply impossible to find a microcontroller with a sufficient number of the related peripherals built-in. Try finding a microcontroller with 8 UARTs, for example. Or one with four different PCM/I2S interfaces, which all can run in different clock domains.

    The existing options of solving such problems basically boil down to either implementing it in hard-wired logic (unrealistic, complex, expensive) or going to programmable logic with CPLD or FPGAs. While the latter is certainly also quite interesting, the learning curve is steep, the tools anything but easy to use and the synthesising time (and thus development cycles) long. Furthermore, your board design will be more complex as you have that FPGA/CPLD and a microcontroller, need to interface the two, etc (yes, in high-end use cases there's the Zynq, but I'm thinking of several orders of magnitude less complex designs).

    Of course one can also take a "pure software" approach and go for high-speed bit-banging. There are some ARM SoCs that can toggle their pins. People have reported rates like 14 MHz being possible on a Raspberry Pi. However, when running a general-purpose OS in parallel, this kind of speed is hard to do reliably over long term, and the related software implementations are going to be anything but nice to write.

    So the XCore is looking like a nice alternative for a lot of those use cases. Where you want a microcontroller with more programmability in terms of its I/O capabilities, but not go as far as to go full-on with FPGA/CPLD development in Verilog or VHDL.

    My current use case

    My current use case is to implement a board that can accept four independent PCM inputs (all in slave mode, i.e. clock provided by external master) and present them via USB to a host PC. The final goal is to have a board that can be combined with the sysmoQMOD and which can interface the PCM audio of four cellular modems concurrently.

    While XMOS is quite strong in the Audio field and you can find existing examples and app notes for I2S and S/PDIF, I couldn't find any existing code for a PCM slave of the given requirements (short frame sync, 8kHz sample rate, 16bit samples, 2.048 MHz bit clock, MSB first).

    I wanted to get a feeling how well one can implement the related PCM slave. In order to test the slave, I decided to develop the matching PCM master and run the two against each other. Despite having never written any code for XMOS before, nor having used any of the toolchain, I was able to implement the PCM master and PCM slave within something like ~6 hours, including simulation and verification. Sure, one can certainly do that in much less time, but only once you're familiar with the tools, programming environment, language, etc. I think it's not bad.

    The biggest problem was that the clock phase for a clocked output port cannot be configured, i.e. the XCore insists on always clocking out a new bit at the falling edge, while my use case of course required the opposite: Clocking oout new signals at the rising edge. I had to use a second clock block to generate the inverted clock in order to achieve that goal.

    Beyond that 4xPCM use case, I also have other ideas like finally putting the osmo-e1-xcvr to use by combining it with an XMOS device to build a portable E1-to-USB adapter. I have no clue if and when I'll find time for that, but if somebody wants to join in: Let me know!

    The good parts

    Documentation excellent

    I found the various pieces of documentation extremely useful and very well written.

    Fast progress

    I was able to make fast progress in solving the first task using the XMOS / Xcore approach.

    Soft Cores developed in public, with commit log

    You can find plenty of soft cores that XMOS has been developing on github at https://github.com/xcore, including the full commit history.

    This type of development is a big improvement over what most vendors of smaller microcontrollers like Atmel are doing (infrequent tar-ball code-drops without commit history). And in the case of the classic uC vendors, we're talking about drivers only. In the XMOS case it's about the entire logic of the peripheral!

    You can for example see that for their I2C core, the very active commit history goes back to January 2011.

    xSIM simulation extremely helpful

    The xTIMEcomposer IDE (based on Eclipse) contains extensive tracing support and an extensible near cycle accurate simulator (xSIM). I've implemented a PCM mater and PCM slave in xC and was able to simulate the program while looking at the waveforms of the logic signals between those two.

    The bad parts

    Unfortunately, my extremely enthusiastic reception of XMOS has suffered quite a bit over time. Let me explain why:

    Hard to get XCore chips

    While the product portfolio on on the xMOS website looks extremely comprehensive, the vast majority of the parts is not available from stock at distributors. You won't even get samples, and lead times are 12 weeks (!). If you check at digikey, they have listed a total of 302 different XMOS controllers, but only 35 of them are in stock. USB capable are 15. With other distributors like Farnell it's even worse.

    I've seen this with other semiconductor vendors before, but never to such a large extent. Sure, some packages/configurations are not standard products, but having only 11% of the portfolio actually available is pretty bad.

    In such situations, where it's difficult to convince distributors to stock parts, it would be a good idea for XMOS to stock parts themselves and provide samples / low quantities directly. Not everyone is able to order large trays and/or capable to wait 12 weeks, especially during the R&D phase of a board.

    Extremely limited number of single-bit ports

    In the smaller / lower pin-count parts, like the XU[F]-208 series in QFN/LQFP-64, the number of usable, exposed single-bit ports is ridiculously low. Out of the total 33 I/O lines available, only 7 can be used as single-bit I/O ports. All other lines can only be used for 4-, 8-, or 16-bit ports. If you're dealing primarily with serial interfaces like I2C, SPI, I2S, UART/USART and the like, those parallel ports are of no use, and you have to go for a mechanically much larger part (like XU[F]-216 in TQFP-128) in order to have a decent number of single-bit ports exposed. Those parts also come with twice the number of cores, memory, etc- which you don't need for slow-speed serial interfaces...

    Change to a non-FOSS License

    XMOS deserved a lot of praise for releasing all their soft IP cores as Free / Open Source Software on github at https://github.com/xcore. The License has basically been a 3-clause BSD license. This was a good move, as it meant that anyone could create derivative versions, whether proprietary or FOSS, and there would be virtually no license incompatibilities with whatever code people wanted to write.

    However, to my very big disappointment, more recently XMOS seems to have changed their policy on this. New soft cores (released at https://github.com/xmos as opposed to the old https://github.com/xcore) are made available under a non-free license. This license is nothing like BSD 3-clause license or any other Free Software or Open Source license. It restricts the license to use the code together with an XMOS product, requires the user to contribute fixes back to XMOS and contains references to importand export control. This license is incopatible with probably any FOSS license in existance, making it impossible to write FOSS code on XMOS while using any of the new soft cores released by XMOS.

    But even beyond that license change, not even all code is provided in source code format anymore. The new USB library (lib_usb) is provided as binary-only library, for example.

    If you know anyone at XMOS management or XMOS legal with whom I could raise this topic of license change when transitioning from older sc_* software to later lib_* code, I would appreciate this a lot.

    Proprietary Compiler

    While a lot of the toolchain and IDE is based on open source (Eclipse, LLVM, ...), the actual xC compiler is proprietary.

    by Harald Welte at September 01, 2017 10:00 PM

    Open Hardware Repository

    MasterFIP - Order of 130 masterFIP v4

    After having validated the design (through 15 v3) we are now ready to produce 130 v4 masterFIP boards!
    The plan is to install them in machines in LHC operation before the end of this year.

    by Evangelia Gousiou (Evangelia.Gousiou@cern.ch) at September 01, 2017 04:23 PM

    August 30, 2017

    Open Hardware Repository

    White Rabbit - 29-08-2017: Geodetic station connected with WR to UTC(MIKE)

    MIKES, the centre for metrology and accreditation of Finland, has connected the Metsähovi Geodetic Research Station to the official time of Finland, UTC[MIKE]

    Some quotes from the article Metsähovi connected to the official time of Finland:

    The time transfer to Metsähovi, Kirkkonummi, occurs from the UTC-laboratory at VTT MIKES Metrology in Otaniemi via optical fibre using the White Rabbit protocol. VTT MIKES Metrology has been an early adopter of the White Rabbit technology for time transfer across long distances. White Rabbit was developed at CERN, the European Organization for Nuclear Research.

    The measurements show, for example, how the travel time of light each way in a 50-kilometre fibre optic cable varies by approx. 7 nanoseconds within a 24-hour period as temperature changes affect the properties of the fibre optic cable, particularly its length.

    The uncertainty of time transfer is expected to be 100 ps or better. The precision of frequency transfer is currently approx. 15 digits.

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at August 30, 2017 09:41 AM

    August 29, 2017

    Free Electrons

    Free Electrons at the Linux Plumbers 2017 conference

    The Linux Plumbers conference has established itself as a major conference in the Linux ecosystem, discussing numerous aspects of the low-level layers of the Linux software stack. Linux Plumbers is organized around a number of micro-conferences, plus a number of more regular talks.

    Linux Plumbers 2017

    Free Electrons already participated to several previous editions of Linux Plumbers, and will again participate to this year’s edition that takes place in Los Angeles on September 13-15. Free Electrons engineers Boris Brezillon, Alexandre Belloni, Grégory Clement and Thomas Petazzoni will attend the conference.

    If you’re attending this conference, or are located in the Los Angeles area, and want to meet us, do not hesitate to drop us a line at info@free-electrons.com. You can also follow Free Electrons Twitter feed for updates during the conference.

    by Thomas Petazzoni at August 29, 2017 11:43 AM

    August 25, 2017

    Open Hardware Repository

    White Rabbit Switch - Software - WR Switch firmware v5.0.1 released

    Since v5.0 was released we have found a few problems in the WR Switch software package. This new v5.0.1 release does not include new functionality but contains important hotfixes to the v5.0. The FPGA bitstream used in v5.0.1 is exactly the same as in 5.0, therefore those same calibration values apply. As for any other release, you can find all the links to download the firmware binaries and manuals on our v5.0.1 release wiki page

    Main fixes include:
    • USB flashing which was broken in v5.0
    • PPSI pre-master state fix
    • make menuconfig fixes
    • SNMP fixes
    • Webinterface fixes
    For the full list of solved issues please check:

    We advise updating your v5.0 switches to include these latest fixes.

    Greg Daniluk, Adam Wujek

    by Grzegorz Daniluk (grzegorz.daniluk@cern.ch) at August 25, 2017 11:37 AM

    August 19, 2017

    Harald Welte

    Osmocom jenkins test suite execution

    Automatic Testing in Osmocom

    So far, in many Osmocom projects we have unit tests next to the code. Those unit tests are executing test on a per-C-function basis, and typically use the respective function directly from a small test program, executed at make check time. The actual main program (like OsmoBSC or OsmoBTS) is not executed at that time.

    We also have VTY testing, which specifically tests that the VTY has proper documentation for all nodes of all commands.

    Then there's a big gap, and we have osmo-gsm-tester for testing a full cellular network end-to-end. It includes physical GSM modesm, coaxial distribution network, attenuators, splitter/combiners, real BTS hardware and logic to run the full network, from OsmoBTS to the core - both for OsmoNITB and OsmoMSC+OsmoHLR based networks.

    However, I think a lot of testing falls somewhere in between, where you want to run the program-under-test (e.g. OsmoBSC), but you don't want to run the MS, BTS and MSC that normally surroudns it. You want to test it by emulating the BTS on the Abis sid and the MSC on the A side, and just test Abis and A interface transactions.

    For this kind of testing, I have recently started to investigate available options and tools.

    OsmoSTP (M3UA/SUA)

    Several months ago, during the development of OsmoSTP, I disovered that the Network Programming Lab of Münster University of Applied Sciences led by Michael Tuexen had released implementations of the ETSI test suite for the M3UA and SUA members of the SIGTRAN protocol family.

    The somewhat difficult part is that they are implemented in scheme, using the guile interpreter/compiler, as well as a C-language based execution wrapper, which then is again called by another guile wrapper script.

    I've reimplemented the test executor in python and added JUnitXML output to it. This means it can feed the test results directly into Jenkins.

    I've also cleaned up the Dockerfiles and related image generation for the osmo-stp-master, m3ua-test and sua-test images, as well as some scripts to actually execute them on one of the Builders. You can find related Dockerfiles as well as associtaed Makfiles in http://git.osmocom.org/docker-playground

    The end result after integration with Osmocom jenkins can be seen in the following examples on jenkins.osmocom.org for M3UA and for SUA

    Triggering the builds is currently periodic once per night, but we could of course also trigger them automatically at some later point.

    OpenGGSN (GTP)

    For OpenGGSN, during the development of IPv6 PDP context support, I wrote some test infrastructure and test cases in TTCN-3. Those test cases can be found at http://git.osmocom.org/osmo-ttcn3-hacks/tree/ggsn_tests

    I've also packaged the GGSN and the test cases each into separate Docker containers called osmo-ggsn-latest and ggsn-test. Related Dockerfiles and Makefiles can again be found in http://git.osmocom.org/docker-playground - together with a Eclipse TITAN Docker base image using Debian Stretch called debian-stretch-titan

    Using those TTCN-3 test cases with the TITAN JUnitXML logger plugin we can again integrate the results directly into Jenkins, whose results you can see at https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-ggsn-test/14/testReport/(root)/GGSN_Tests/

    Further Work

    I've built some infrastructure for Gb (NS/BSSGP), VirtualUm and other testing, but yet have to build Docker images and related jenkins integration for it. Stay tuned about that. Also, lots more actual tests cases are required. I'm very much looking forward to any contributions.

    by Harald Welte at August 19, 2017 10:00 PM

    August 18, 2017

    Open Hardware Repository

    1:8 Pulse/Frequency Distribution Amplifier - S/N005 phase-noise measurement at 10 MHz

    Phase-noise measurements show a flat spur-free phase-noise and AM-noise floor of -162 dBc/Hz at >100 Hz offset from a 10 MHz carrier. Measurements at 5 MHz to follow.
    See wiki for results.

    by Anders Wallin (anders.e.e.wallin@gmail.com) at August 18, 2017 06:38 AM

    August 16, 2017

    Free Electrons

    Updated bleeding edge toolchains on toolchains.free-electrons.com

    Two months ago, we announced a new service from Free Electrons: free and ready-to-use Linux cross-compilation toolchains, for a large number of architectures and C libraries, available at http://toolchains.free-electrons.com/.

    Bleeding edge toolchain updates

    All our bleeding edge toolchains have been updated, with the latest version of the toolchain components:

    • gcc 7.2.0, which was released 2 days ago
    • glibc 2.26, which was released 2 weeks ago
    • binutils 2.29
    • gdb 8.0

    Those bleeding edge toolchains are now based on Buildroot 2017.08-rc2, which brings a nice improvement: the host tools (gcc, binutils, etc.) are no longer linked statically against gmp, mpfr and other host libraries. They are dynamically linked against them with an appropriate rpath encoded into the gcc and binutils binaries to find those shared libraries regardless of the installation location of the toolchain.

    However, due to gdb 8.0 requiring a C++11 compiler on the host machine (at least gcc 4.8), our bleeding edge toolchains are now built in a Debian Jessie system instead of Debian Squeeze, which means that at least glibc 2.14 is needed on the host system to use them.

    The only toolchains for which the tests are not successful are the MIPS64R6 toolchains, due to the Linux kernel not building properly for this architecture with gcc 7.x. This issue has already been reported upstream.

    Stable toolchain updates

    We haven’t changed the component versions of our stable toolchains, but we made a number of fixes to them:

    • The armv7m and m68k-coldfire toolchains have been rebuilt with a fixed version of elf2flt that makes the toolchain linker directly usable. This fixes building the Linux kernel using those toolchains.
    • The mips32r5 toolchain has been rebuilt with NaN 2008 encoding (instead of NaN legacy), which makes the resulting userspace binaries actually executable by the Linux kernel, which expects NaN 2008 encoding on mips32r5 by default.
    • Most mips toolchains for musl have been rebuilt, with Buildroot fixes for the creation of the dynamic linker symbolic link. This has no effect on the toolchain itself, but also the tests under Qemu to work properly and validate the toolchains.

    Other improvements

    We made a number of small improvements to the toolchains.free-electrons.com site:

    • Each architecture now has a page that lists all toolchain versions available. This allows to easily find a toolchain that matches your requirements (in terms of gcc version, kernel headers version, etc.). See All aarch64 toolchains for an example.
    • We added a FAQ as well as a news page.

    As usual, we welcome feedback about our toolchains, either on our bug tracker or by mail at info@free-electrons.com.

    by Thomas Petazzoni at August 16, 2017 08:20 PM

    Open Hardware Repository

    1:8 Pulse/Frequency Distribution Amplifier - S/N005 assembled and tested

    Amplifier S/N 005 was assembled and tested, housing PDA2017.07 and FDA2017.07 boards.
    Initial phase-noise tests show good performance, similar to the previous generation of the board.
    The new PDA2017.07 design using IDT5PB1108 has very fast rise-time (to be measured), possibly a quite low output-impedance (to be fixed?) and a preliminary channel-to-channel output skew of max 250 ps.

    by Anders Wallin (anders.e.e.wallin@gmail.com) at August 16, 2017 01:29 PM

    Free Electrons

    Free Electrons proposes an I3C subsystem for the Linux kernel

    MIPI I3C fact sheet, from the MIPI I3C white paper

    MIPI I3C fact sheet, from the MIPI I3C white paper

    At the end of 2016, the MIPI consortium has finalized the first version of its I3C specification, a new communication bus that aims at replacing older busses like I2C or SPI. According to the specification, I3C gets closer to SPI data rate while requiring less pins and adding interesting mechanisms like in-band interrupts, hotplug capability or automatic discovery of devices connected on the bus. In addition, I3C provides backward compatibility with I2C: I3C and legacy I2C devices can be connected on a common bus controlled by an I3C master.

    For more details about I3C, we suggest reading the MIPI I3C Whitepaper, as unfortunately MIPI has not publicly released the specifications for this protocol.

    For the last few months, Free Electrons engineer Boris Brezillon has been working with Cadence to develop a Linux kernel subsystem to support this new bus, as well as Cadence’s I3C master controller IP. We have now posted the first version of our patch series to the Linux kernel mailing list for review, and we already received a large number of very useful comments from the kernel community.

    Free Electrons is proud to be pioneering the support for this new bus in the Linux kernel, and hopes to see other developers contribute to this subsystem in the near future!

    by Boris Brezillon at August 16, 2017 01:10 PM

    August 14, 2017

    Bunnie Studios

    Name that Ware, August 2017

    The Ware for August 2017 is below.

    I removed a bit of context to make it more difficult — if it proves unguessable I’ll zoom out slightly (or perhaps just leave one extra, crucial hint to consider).

    by bunnie at August 14, 2017 04:23 PM

    Winner, Name that Ware July 2017

    The ware for July 2017 is a PMT (photomultiplier tube) module. I’d say wrm gets the prize this month, for getting that it’s a PMT driver first, and for linking to a schematic. :) That’s an easy way to win me over. Gratz, email me to claim your prize!

    by bunnie at August 14, 2017 04:22 PM

    August 08, 2017

    Harald Welte

    IPv6 User Plane support in Osmocom

    Preface

    Cellular systems ever since GPRS are using a tunnel based architecture to provide IP connectivity to cellular terminals such as phones, modems, M2M/IoT devices and the like. The MS/UE establishes a PDP context between itself and the GGSN on the other end of the cellular network. The GGSN then is the first IP-level router, and the entire cellular network is abstracted away from the User-IP point of view.

    This architecture didn't change with EGPRS, and not with UMTS, HSxPA and even survived conceptually in LTE/4G.

    While the concept of a PDP context / tunnel exists to de-couple the transport layer from the structure and type of data inside the tunneled data, the primary user plane so far has been IPv4.

    In Osmocom, we made sure that there are no impairments / assumptions about the contents of the tunnel, so OsmoPCU and OsmoSGSN do not care at all what bits and bytes are transmitted in the tunnel.

    The only Osmocom component dealing with the type of tunnel and its payload structure is OpenGGSN. The GGSN must allocate the address/prefix assigned to each individual MS/UE, perform routing between the external IP network and the cellular network and hence is at the heart of this. Sadly, OpenGGSN was an abandoned project for many years until Osmocom adopted it, and it only implemented IPv4.

    This is actually a big surprise to me. Many of the users of the Osmocom stack are from the IT security area. They use the Osmocom stack to test mobile phones for vulnerabilities, analyze mobile malware and the like. As any penetration tester should be interested in analyzing all of the attack surface exposed by a given device-under-test, I would have assumed that testing just on IPv4 would be insufficient and over the past 9 years, somebody should have come around and implemented the missing bits for IPv6 so they can test on IPv6, too.

    In reality, it seems nobody appears to have shared line of thinking and invested a bit of time in growing the tools used. Or if they did, they didn't share the related code.

    In June 2017, Gerrie Roos submitted a patch for OpenGGSN IPv6 support that raised hopes about soon being able to close that gap. However, at closer sight it turns out that the code was written against a more than 7 years old version of OpenGGSN, and it seems to primarily focus on IPv6 on the outer (transport) layer, rather than on the inner (user) layer.

    OpenGGSN IPv6 PDP Context Support

    So in July 2017, I started to work on IPv6 PDP support in OpenGGSN.

    Initially I thought How hard can it be? It's not like IPv6 is new to me (I joined 6bone under 3ffe prefixes back in the 1990ies and worked on IPv6 support in ip6tables ages ago. And aside from allocating/matching longer addresses, what kind of complexity does one expect?

    After my initial attempt of implementation, partially mislead by the patch that was contributed against that 2010-or-older version of OpenGGSN, I'm surprised how wrong I was.

    In IPv4 PDP contexts, the process of establishing a PDP context is simple:

    • Request establishment of a PDP context, set the type to IETF IPv4
    • Receive an allocated IPv4 End User Address
    • Optionally use IPCP (part of PPP) to reques and receive DNS Server IP addresses

    So I implemented the identical approach for IPv6. Maintain a pool of IPv6 addresses, allocate one, and use IPCP for DNS. And nothing worked.

    • IPv6 PDP contexts assign a /64 prefix, not a single address or a smaller prefix
    • The End User Address that's part of the Signalling plane of Layer 3 Session Management and GTP is not the actual address, but just serves to generate the interface identifier portion of a link-local IPv6 address
    • IPv6 stateless autoconfiguration is used with this link-local IPv6 address inside the User Plane, after the control plane signaling to establish the PDP context has completed. This means the GGSN needs to parse ICMPv6 router solicitations and generate ICMPV6 router advertisements.

    To make things worse, the stateless autoconfiguration is modified in some subtle ways to make it different from the normal SLAAC used on Ethernet and other media:

    • the timers / lifetimes are different
    • only one prefix is permitted
    • only a prefix length of 64 is permitted

    A few days later I implemented all of that, but it still didn't work. The problem was with DNS server adresses. In IPv4, the 3GPP protocols simply tunnel IPCP frames for this. This makes a lot of sense, as IPCP is designed for point-to-point interfaces, and this is exactly what a PDP context is.

    In IPv6, the corresponding IP6CP protocol does not have the capability to provision DNS server addresses to a PPP client. WTF? The IETF seriously requires implementations to do DHCPv6 over PPP, after establishing a point-to-point connection, only to get DNS server information?!? Some people suggested an IETF draft to change this butthe draft has expired in 2011 and we're still stuck.

    While 3GPP permits the use of DHCPv6 in some scenarios, support in phones/modems for it is not mandatory. Rather, the 3GPP has come up with their own mechanism on how to communicate DNS server IPv6 addresses during PDP context activation: The use of containers as part of the PCO Information Element used in L3-SM and GTP (see Section 10.5.6.3 of 3GPP TS 24.008. They by the way also specified the same mechanism for IPv4, so there's now two competing methods on how to provision IPv4 DNS server information: IPCP and the new method.

    In any case, after some more hacking, OpenGGSN can now also provide DNS server information to the MS/UE. And once that was implemented, I had actual live uesr IPv6 data over a full Osmocom cellular stack!

    Summary

    We now have working IPv6 User IP in OpenGGSN. Together with the rest of the Osmocom stack you can operate a private GPRS, EGPRS, UMTS or HSPA network that provide end-to-end transparent, routed IPv6 connectivity to mobile devices.

    All in all, it took much longer than nneeded, and the following questions remain in my mind:

    • why did the IETF not specify IP6CP capabilities to configure DNS servers?
    • why the complex two-stage address configuration with PDP EUA allocation for the link-local address first and then stateless autoconfiguration?
    • why don't we simply allocate the entire prefix via the End User Address information element on the signaling plane? For sure next to the 16byte address we could have put one byte for prefix-length?
    • why do I see duplication detection flavour neighbour solicitations from Qualcomm based phones on what is a point-to-point link with exactly two devices: The UE and the GGSN?
    • why do I see link-layer source address options inside the ICMPv6 neighbor and router solicitation from mobile phones, when that option is specifically not to be used on point-to-point links?
    • why is the smallest prefix that can be allocated a /64? That's such a waste for a point-to-point link with a single device on the other end, and in times of billions of connected IoT devices it will just encourage the use of non-public IPv6 space (i.e. SNAT/MASQUERADING) while wasting large parts of the address space

    Some of those choices would have made sense if one would have made it fully compatible with normal IPv6 like e.g. on Ethernet. But implementing ICMPv6 router and neighbor solicitation without getting any benefit such as ability to have multiple prefixes, prefixes of different lengths, I just don't understand why anyone ever thought You can find the code at http://git.osmocom.org/openggsn/log/?h=laforge/ipv6 and the related ticket at https://osmocom.org/issues/2418

    by Harald Welte at August 08, 2017 10:00 PM

    July 31, 2017

    Open Hardware Repository

    White Rabbit - 31-07-2017: WR Switch Production Test Suite published

    For the past few months we were working with INCAA Computers BV on a new WR Switch Production Test Suite. This
    system allows to verify during the production or after delivery that all the components of the WR Switch hardware work properly.
    Please check the WRS PTS wiki page for all the sources and documentation.

    by Grzegorz Daniluk (grzegorz.daniluk@cern.ch) at July 31, 2017 06:11 PM

    July 27, 2017

    Bunnie Studios

    Name that Ware July 2017

    The Ware for July 2017 is shown below.

    Decided to do this one with the potting on to make it a smidgen more challenging.

    by bunnie at July 27, 2017 04:17 AM

    Winner, Name that Ware June 2017

    The Ware for June 2017 is an ultrasonic delay line. Picked this beauty up while wandering the junk shops of Akihabara. There’s something elegant about the Old Ways that’s simply irresistible to me…back when the answer to all hard problems was not simply “transform it into the software domain and then compute the snot out of it”.

    Grats to plum33 for nailing it! email me for your prize.

    by bunnie at July 27, 2017 04:17 AM

    July 19, 2017

    Video Circuits

    Photos From The Video Workshop

    The video workshop Alex and I gave was one of the best I have delivered, all 15 attendees got to take home a working CHA/V module they built in the class, it's a hacked VGA signal generator that basically allows you to build a simple video synth by adding some home brew or off the shelf oscillators. We had a great mix of attendees and they were all from really interesting backgrounds and super engaged. Alex as usual did a nicely paced video synthesis tutorial and I then lead the theory and building part of the class. We rounded up with Alex leading a discussion around historical video synthesis work and then proceeded to enjoy the evening concerts that were also part of the fantastic Brighton modular meet. (Pics 3+9 here are from Fabrizio D'Amico who runs Video Hack Space) Thanks to Andrew for organising the amazing meet which hosted the workshop, Matt for making our panels last minute, George for helping us out on the day and Steve from Thonk for supplying some components for the kits.










    by Chris (noreply@blogger.com) at July 19, 2017 02:39 AM

    July 18, 2017

    Harald Welte

    Virtual Um interface between OsmoBTS and OsmocomBB

    During the last couple of days, I've been working on completing, cleaning up and merging a Virtual Um interface (i.e. virtual radio layer) between OsmoBTS and OsmocomBB. After I started with the implementation and left it in an early stage in January 2016, Sebastian Stumpf has been completing it around early 2017, with now some subsequent fixes and improvements by me. The combined result allows us to run a complete GSM network with 1-N BTSs and 1-M MSs without any actual radio hardware, which is of course excellent for all kinds of testing scenarios.

    The Virtual Um layer is based on sending L2 frames (blocks) encapsulated via GSMTAP UDP multicast packets. There are two separate multicast groups, one for uplink and one for downlink. The multicast nature simulates the shared medium and enables any simulated phone to receive the signal from multiple BTSs via the downlink multicast group.

    /images/osmocom-virtum.png

    In OsmoBTS, this is implemented via the new osmo-bts-virtual BTS model.

    In OsmocomBB, this is realized by adding virtphy virtual L1, which speaks the same L1CTL protocol that is used between the real OsmcoomBB Layer1 and the Layer2/3 programs such as mobile and the like.

    Now many people would argue that GSM without the radio and actual handsets is no fun. I tend to agree, as I'm a hardware person at heart and I am not a big fan of simulation.

    Nevertheless, this forms the basis of all kinds of possibilities for automatized (regression) testing in a way and for layers/interfaces that osmo-gsm-tester cannot cover as it uses a black-box proprietary mobile phone (modem). It is also pretty useful if you're traveling a lot and don't want to carry around a BTS and phones all the time, or get some development done in airplanes or other places where operating a radio transmitter is not really a (viable) option.

    If you're curious and want to give it a shot, I've put together some setup instructions at the Virtual Um page of the Osmocom Wiki.

    by Harald Welte at July 18, 2017 10:00 PM

    July 15, 2017

    Bunnie Studios

    That’s a Big Microscope…

    I’ve often said that there are no secrets in hardware — you just need a bigger, better microscope.

    I think I’ve found the limit to that statement. To give you an idea, here’s the “lightbulb” that powers the microscope:

    It’s the size of a building, and it’s the Swiss Light Source. Actually, not all of that building is dedicated to this microscope, just one beamline of an X-ray synchrotron capable of producing photons at an energy of 6.5keV (X-rays) at a flux of close to a billion coherent photons per second — but still, it’s a big light bulb. It might be a while before you see one of these popping up in a hacker’s garage…err, hangar…somewhere.

    The result? One can image, in 3-D and “non-destructively” (e.g., without having to delayer or etch away dielectrics), chips down to a resolution of 14.6nm.

    That’s a pretty neat trick if you’re trying to reverse engineer modern silicon.

    You can read the full article at Nature (“High Resolution non-destructive three-dimensional imaging of integrated circuits” by Mirko Holler et al). I’m a paying subscriber to Nature so I’m supposed to have access to the article, but at the moment, their paywall is throwing a null pointer exception. Once the paywall is fixed you can buy a copy of the article to read, but in the meantime, SciHub seems more reliable.

    You get what you pay for, right?

    by bunnie at July 15, 2017 01:55 PM

    July 11, 2017

    Elphel

    Current video stream latency and a way to reduce it

    Fig.1 Live stream latency testing

    Recently we had an inquiry whether our cameras are capable of streaming low latency video. The short answer is yes, the camera’s average output latency for 1080p at 30 fps is ~16 ms. It is possible to reduce it to almost 0.5 ms with a few changes to the driver.

    However the total latency of the system, from capturing to displaying, includes delays caused by network, pc, software and display.

    In the results of the experiment (similar to this one) these delays contribute the most (around 40-50 ms) to the stream latency – at least, for the given equipment.


     

    Goal

    Measure the total latency of a live stream over network from 10393 camera.
     

    Setup

    • Camera: NC393-F-CS
      • Resolution@fps: 1080p@30fps,  720p@60fps
      • Compression quality: 90%
      • Exposure time: 1.7 ms
      • Stream formats: mjpeg, rtsp
      • Sensor: MT9P001, 5MPx, 1/2.5″
      • Lens: Computar f=5mm, f/1.4, 1/2″
    • PC: Shuttle box, i7, 16GB RAM, GeForce GTX 560 Ti
    • Display: ASUS VS24A, 60Hz (=16.7ms), 5ms gtg
    • OS: Kubuntu 16.04
    • Network connection: 1Gbps, direct camera-PC via cable
    • Applications:
      • gstreamer
      • chrome, firefox
      • mplayer
      • vlc
    • Stopwatch: basic javascript

     

    Notes

    table{ border-collapse: collapse; } td{ padding:0px 5px; border:1px solid black; } th{ padding:5px; border:1px solid black; background:rgba(220,220,220,0.5); }


    Table 1: Transfer times and data rate

    Resolution/fps Image size1, KB Transfer time2, ms Data rate3, Mbps
    720p/60 250 2 120
    1080p/30 500 4 120

    1 – average compressed (90%) image size
    2 – time it takes to transfer a single image over network. Jitter is unknown. t = Image_size*1Gbps
    3 – required bandwidth: rate = fps*Image_size

    Camera output latency calculation

    All numbers are for the given lens, sensor and camera setup and parameters. Briefly.

    Sensor
    Because of ERS each row’s latency is different. See tables 2 and 3.
     
    Table 2: tROW and tTR

    Resolution tROW1, us tTR2, us
    720p 22.75 13.33
    1080p 29.42 20
    full res (2592×1936) 36.38 27

    1 – row time, see datasheet. tROW = f(Width)
    2 – time it takes to transfer a row over sensor cable, clock = 96MHz. tTR = Width/96MHz
     
    Table 3: Average latency and the whole range.

    Resolution tERS avg1, ms tERS whole range2, ms
    720p 8 0.01-16
    1080p 16 0.02-32

    1 – average latency
    2 – min – last row latency, max – 1st row latency

    Exposure

    tEXP < 1 ms – typical exposure time for outdoors. A display is bright enough to set 1.7 ms with the gains maxed.

    Compressor

    The compressor is implemented in fpga and works 3x times faster but needs a stripe of 20 rows in memory. Thus, the compressor will finish ~20/3*tROW after the whole image is read out.

    tCMP = 20/3*tROW

    Summary

    tCAM = tERS + tEXP + tCMP

    Since the image is read and compressed by fpga logic of the Zynq and this pipeline has been simulated we can be sure in these numbers.
     
    Table 4: Average output latency + exposure

    Resolution tCAM, ms
    720p 9.9
    1080p 17.9

    Stopwatch accuracy

    Not accurate. For simplicity, we will rely on the camera’s internal clock that time stamps every image, and take the javascript timer readings as unique labels, thus not caring what time they are showing.
     

    Results

    Fig.2 1080p 30fps

    Fig.3 720p 60fps

     
    GStreamer has shown the best results among the tested programs.
    Since the camera fps is discrete the result is a multiple of 1/fps (see this article):

    • 30 fps => 33.3 ms
    • 60 fps => 16.7 ms

     

    Resolution/fps Total Latency, ms Network+PC+SW latency, ms
    720p@60fps 33.3-50 23.4-40.1
    1080p@30fps 33.3-66.7 15.4-48.8

     

    Possible improvements

    Camera

    Currently, the driver waits for the interrupt from the compressor that indicates the image is fully compressed and ready for transfer. Meanwhile one does not have to wait for the whole image but start the transfer when the minimum of the compressed is data ready.

    There are 3 more interrupts related to the image pipeline events. One of them is “compression started” – switching to it can reduce the output latency to (10+20/3)*tROW or 0.4 ms for 720p and 0.5 ms for 1080p.

    Other hardware and software

    In addition to the most obvious improvements:

    • For wifi: use 5GHz over 2.4GHz – smaller jitter, non-overlapping channels
    • Lower latency software: for mjpeg use gstreamer or vlc (takes an extra effort to setup) over chrome or firefox because they do extra buffering

     

    Links

     

    Updates

     
    Table 6: Camera ports

    mjpeg rtsp
    port 0 2323 554
    port 1 2324 556
    port 2 2325 558
    port 3 2326 560

    GStreamer pipelines

    • For mjpeg:

    ~$ gst-launch-1.0 souphttpsrc is-live=true location=http://192.168.0.9:2323/mimg ! jpegdec ! xvimagesink

    • For rtsp:

    ~$ gst-launch-1.0 rtspsrc is-live=true location=rtsp://192.168.0.9:554 ! rtpjpegdepay ! jpegdec ! xvimagesink

    VLC

    ~$ vlc rtsp://192.168.0.9:554

    Chrome/Firefox

    Open http://192.168.0.9:2323/mimg

    by Oleg Dzhimiev at July 11, 2017 05:33 PM

    July 10, 2017

    Free Electrons

    Linux 4.12, Free Electrons contributions

    Linus Torvalds has released the 4.12 Linux kernel a week ago, in what is the second biggest kernel release ever by number of commits. As usual, LWN had a very nice coverage of the major new features and improvements: first part, second part and third part.

    LWN has also published statistics about the Linux 4.12 development cycles, showing:

    • Free Electrons as the #14 contributing company by number of commits, with 221 commits, between Broadcom (230 commits) and NXP (212 commits)
    • Free Electrons as the #14 contributing company number of changed lines, with 16636 lines changed, just two lines less than Mellanox
    • Free Electrons engineer and MTD NAND maintainer Boris Brezillon as the #17 most active contributor by number of lines changed.

    Our most important contributions to this kernel release have been:

    • On Atmel AT91 and SAMA5 platforms:
      • Alexandre Belloni has continued to upstream the support for the SAMA5D2 backup mode, which is a very deep suspend to RAM state, offering very nice power savings. Alexandre touched the core code in arch/arm/mach-at91 as well as pinctrl and irqchip drivers
      • Boris Brezillon has converted the Atmel PWM driver to the atomic API of the PWM subsystem, implemented suspend/resume and did a number of fixes in the Atmel display controller driver, and also removed the no longer used AT91 Parallel ATA driver.
      • Quentin Schulz improved the suspend/resume hooks in the atmel-spi driver to support the SAMA5D2 backup mode.
    • On Allwinner platforms:
      • Mylène Josserand has made a number of improvements to the sun8i-codec audio driver that she contributed a few releases ago.
      • Maxime Ripard added devfreq support to dynamically change the frequency of the GPU on the Allwinner A33 SoC.
      • Quentin Schulz added battery charging and ADC support to the X-Powers AXP20x and AXP22x PMICs, found on Allwinner platforms.
      • Quentin Schulz added a new IIO driver to support the ADCs found on numerous Allwinner SoCs.
      • Quentin Schulz added support for the Allwinner A33 built-in thermal sensor, and used it to implement thermal throttling on this platform.
    • On Marvell platforms:
      • Antoine Ténart contributed Device Tree changes to describe the cryptographic engines found in the Marvell Armada 7K and 8K SoCs. For now only the Device Tree description has been merged, the driver itself will arrive in Linux 4.13.
      • Grégory Clement has contributed a pinctrl and GPIO driver for the Marvell Armada 3720 SoC (Cortex-A53 based)
      • Grégory Clement has improved the Device Tree description of the Marvell Armada 3720 and Marvell Armada 7K/8K SoCs and corresponding evaluation boards: SDHCI and RTC are now enabled on Armada 7K/8K, USB2, USB3 and RTC are now enabled on Armada 3720.
      • Thomas Petazzoni made a significant number of changes to the mvpp2 network driver, finally adding support for the PPv2.2 version of this Ethernet controller. This allowed to enable network support on the Marvell Armada 7K/8K SoCs.
      • Thomas Petazzoni contributed a number of fixes to the mv_xor_v2 dmaengine driver, used for the XOR engines on the Marvell Armada 7K/8K SoCs.
      • Thomas Petazzoni cleaned-up the MSI support in the Marvell pci-mvebu and pcie-aardvark PCI host controller drivers, which allowed to remove a no-longer used MSI kernel API.
    • On the ST SPEAr600 platform:
      • Thomas Petazzoni added support for the ADC available on this platform, by adding its Device Tree description and fixing a clock driver bug
      • Thomas did a number of small improvements to the Device Tree description of the SoC and its evaluation board
      • Thomas cleaned up the fsmc_nand driver, which is used for the NAND controller driver on this platform, removing lots of unused code
    • In the MTD NAND subsystem:
      • Boris Brezillon implemented a mechanism to allow vendor-specific initialization and detection steps to be added, on a per-NAND chip basis. As part of this effort, he has split into multiple files the vendor-specific initialization sequences for Macronix, AMD/Spansion, Micron, Toshiba, Hynix and Samsung NANDs. This work will allow in the future to more easily exploit the vendor-specific features of different NAND chips.
    • Other contributions:
      • Maxime Ripard added a display panel driver for the ST7789V LCD controller

    In addition, several Free Electrons engineers are also maintainers of various kernel subsystems. During this release cycle, they reviewed and merged a number of patches from kernel contributors:

    • Maxime Ripard, as the Allwinner co-maintainer, merged 94 patches
    • Boris Brezillon, as the NAND maintainer and MTD co-maintainer, merged 64 patches
    • Alexandre Belloni, as the RTC maintainer and Atmel co-maintainer, merged 38 patches
    • Grégory Clement, as the Marvell EBU co-maintainer, merged 32 patches

    The details of all our contributions for this release:

    by Thomas Petazzoni at July 10, 2017 10:13 AM

    July 09, 2017

    Harald Welte

    Ten years after first shipping Openmoko Neo1973

    Exactly 10 years ago, on July 9th, 2007 we started to sell+ship the first Openmoko Neo1973. To be more precise, the webshop actually opened a few hours early, depending on your time zone. Sean announced the availability in this mailing list post

    I don't really have to add much to my ten years [of starting to work on] Openmoko anniversary blog post a year ago, but still thought it's worth while to point out the tenth anniversary.

    It was exciting times, and there was a lot of pioneering spirit: Building a Linux based smartphone with a 100% FOSS software stack on the application processor, including all drivers, userland, applications - at a time before Android was known or announced. As history shows, we'd been working in parallel with Apple on the iPhone, and Google on Android. Of course there's little chance that a small taiwanese company can compete with the endless resources of the big industry giants, and the many Neo1973 delays meant we had missed the window of opportunity to be the first on the market.

    It's sad that Openmoko (or similar projects) have not survived even as a special-interest project for FOSS enthusiasts. Today, virtually all options of smartphones are encumbered with way more proprietary blobs than we could ever imagine back then.

    In any case, the tenth anniversary of trying to change the amount of Free Softwware in the smartphone world is worth some celebration. I'm reaching out to old friends and colleagues, and I guess we'll have somewhat of a celebration party both in Germany and in Taiwan (where I'll be for my holidays from mid-September to mid-October).

    by Harald Welte at July 09, 2017 02:00 PM

    July 07, 2017

    Open Hardware Repository

    White Rabbit core collection - White Rabbit PTP Core v4.1 released

    We have just released v4.1 of the WR PTP Core. You can find all the links to download the reference designs binaries and documentation on our release wiki page.

    This release contains mainly fixes to the previous v4.0 stable release:
    • fixed PCIe reset for standalone operation
    • fixes to p2p mode in PPSi
    • fixed Rx termination scheme for Spartan6 PHY which made WRPC unable to work with some SFPs
    • fixes and updates to HDL board and platform wrappers
    and also some new features like:
    • new Wishbone registers bank available to read WRPC diagnostics from user application
    • new wrpc-diags host tool to read diagnostics over PCIe or VME
    • built-in default init script that loads SFP calibration parameters and configures WRPC in Slave mode
    • new document WRPC Failures and Diagnostics

    Thank you for all the bug reports and contributions. As always, we encourage you to try this fresh release on your boards.

    Greg Daniluk for the WR PTP Core team

    by Grzegorz Daniluk (grzegorz.daniluk@cern.ch) at July 07, 2017 02:21 PM

    July 03, 2017

    Open Hardware Repository

    OHR Meta Project - 29-06-2017: Open Doors for Universal Embedded Design

    The article Open Doors for Universal Embedded Design in Embedded Systems Engineering, written by Caroline Hayes, Senior Editor, reads:

    Charged with finding cost-effective integration for multicore platforms, the European Union’s (EU) Artemis EMC2 project finished at the end of May this year. A further initiative with CERN could mean the spirit of co-operation and the principles of open hardware herald an era of innovation.

    and

    This collaboration is a new initiative. The PC/104 Consortium will provide design-in examples of new and mature boards, with a reference design, for others to use and create something new. Although the Sundance board is the only [PC/104] product on the CERN Open Hardware Repository, there will be more news in the summer, promises Christensen. “My goal is to get five designs within the first year,” he says, and he is actively working to promote to PC/104 Consortium members that there is a place where they can download—and upload—reference designs which are PC/104-compatible.

    Read the full article.

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at July 03, 2017 02:14 PM

    June 26, 2017

    Bunnie Studios

    Name that Ware June 2017

    The Ware for June 2017 is shown below.

    If nobody can guess this one from just the pointy end of the stick, I’ll post a photo with more context…

    by bunnie at June 26, 2017 08:06 PM

    Winner, Name that Ware May 2017

    The Ware for May 2017 is the “Lorentz and Hertz” carriage board from an HP Officejet Pro 8500. Congrats to MegabytePhreak for nailing both the make and model of the printer it came from! email me for your prize.

    I found the name of the board to be endearing.

    by bunnie at June 26, 2017 08:06 PM

    June 19, 2017

    Free Electrons

    Free and ready-to-use cross-compilation toolchains

    For all embedded Linux developers, cross-compilation toolchains are part of the basic tool set, as they allow to build code for a specific CPU architecture and debug it. Until a few years ago, CodeSourcery was providing a lot of high quality pre-compiled toolchains for a wide range of architectures, but has progressively stopped doing so. Linaro provides some freely available toolchains, but only targetting ARM and AArch64. kernel.org has a set of pre-built toolchains for a wider range of architectures, but they are bare metal toolchains (cannot build Linux userspace programs) and updated infrequently.

    To fill in this gap, Free Electrons is happy to announce its new service to the embedded Linux community: toolchains.free-electrons.com.

    Free Electrons toolchains

    This web site provides a large number of cross-compilation toolchains, available for a wide range of architectures, in multiple variants. The toolchains are based on the classical combination of gcc, binutils and gdb, plus a C library. We currently provide a total of 138 toolchains, covering many combinations of:

    • Architectures: AArch64 (little and big endian), ARC, ARM (little and big endian, ARMv5, ARMv6, ARMv7), Blackfin, m68k (Coldfire and 68k), Microblaze (little and big endian), MIPS32 and MIPS64 (little and big endian, with various instruction set variants), NIOS2, OpenRISC, PowerPC and PowerPC64, SuperH, Sparc and Sparc64, x86 and x86-64, Xtensa
    • C libraries: GNU C library, uClibc-ng and musl
    • Versions: for each combination, we provide a stable version which uses slightly older but more proven versions of gcc, binutils and gdb, and we provide a bleeding edge version with the latest version of gcc, binutils and gdb.

    After being generated, most of the toolchains are tested by building a Linux kernel and a Linux userspace, and booting it under Qemu, which allows to verify that the toolchain is minimally working. We plan on adding more tests to validate the toolchains, and welcome your feedback on this topic. Of course, not all toolchains are tested this way, because some CPU architectures are not emulated by Qemu.

    The toolchains are built with Buildroot, but can be used for any purpose: build a Linux kernel or bootloader, as a pre-built toolchain for your favorite embedded Linux build system, etc. The toolchains are available in tarballs, together with licensing information and instructions on how to rebuild the toolchain if needed.

    We are very much interested in your feedback about those toolchains, so do not hesitate to report bugs or make suggestions in our issue tracker!

    This work was done as part of the internship of Florent Jacquet at Free Electrons.

    by Thomas Petazzoni at June 19, 2017 07:52 AM

    June 15, 2017

    Harald Welte

    How the Osmocom GSM stack is funded

    As the topic has been raised on twitter, I thought I might share a bit of insight into the funding of the Osmocom Cellular Infrastructure Projects.

    Keep in mind: Osmocom is a much larger umbrella project, and beyond the Networks-side cellular stack is home many different community-based projects around open source mobile communications. All of those have started more or less as just for fun projects, nothing serious, just a hobby [1]

    The projects implementing the network-side protocol stacks and network elements of GSM/GPRS/EGPRS/UMTS cellular networks are somewhat the exception to that, as they have evolved to some extent professionalized. We call those projects collectively the Cellular Infrastructure projects inside Osmocom. This post is about that part of Osmocom only

    History

    From late 2008 through 2009, People like Holger and I were working on bs11-abis and later OpenBSC only in our spare time. The name Osmocom didn't even exist back then. There was a strong technical community with contributions from Sylvain Munaut, Andreas Eversberg, Daniel Willmann, Jan Luebbe and a few others. None of this would have been possible if it wasn't for all the help we got from Dieter Spaar with the BS-11 [2]. We all had our dayjob in other places, and OpenBSC work was really just a hobby. People were working on it, because it was where no FOSS hacker has gone before. It was cool. It was a big and pleasant challenge to enter the closed telecom space as pure autodidacts.

    Holger and I were doing freelance contract development work on Open Source projects for many years before. I was mostly doing Linux related contracting, while Holger has been active in all kinds of areas throughout the FOSS software stack.

    In 2010, Holger and I saw some first interest by companies into OpenBSC, including Netzing AG and On-Waves ehf. So we were able to spend at least some of our paid time on OpenBSC/Osmocom related contract work, and were thus able to do less other work. We also continued to spend tons of spare time in bringing Osmocom forward. Also, the amount of contract work we did was only a fraction of the many more hours of spare time.

    In 2011, Holger and I decided to start the company sysmocom in order to generate more funding for the Osmocom GSM projects by means of financing software development by product sales. So rather than doing freelance work for companies who bought their BTS hardware from other places (and spent huge amounts of cash on that), we decided that we wanted to be a full solution supplier, who can offer a complete product based on all hardware and software required to run small GSM networks.

    The only problem is: We still needed an actual BTS for that. Through some reverse engineering of existing products we figured out who one of the ODM suppliers for the hardware + PHY layer was, and decided to develop the OsmoBTS software to do so. We inherited some of the early code from work done by Andreas Eversberg on the jolly/bts branch of OsmocomBB (thanks), but much was missing at the time.

    What follows was Holger and me working several years for free [3], without any salary, in order to complete the OsmoBTS software, build an embedded Linux distribution around it based on OE/poky, write documentation, etc. and complete the first sysmocom product: The sysmoBTS 1002

    We did that not because we want to get rich, or because we want to run a business. We did it simply because we saw an opportunity to generate funding for the Osmocom projects and make them more sustainable and successful. And because we believe there is a big, gaping, huge vacuum in terms of absence of FOSS in the cellular telecom sphere.

    Funding by means of sysmocom product sales

    Once we started to sell the sysmoBTS products, we were able to fund Osmocom related development from the profits made on hardware / full-system product sales. Every single unit sold made a big contribution towards funding both the maintenance as well as the ongoing development on new features.

    This source of funding continues to be an important factor today.

    Funding by means of R&D contracts

    The probably best and most welcome method of funding Osmocom related work is by means of R&D projects in which a customer funds our work to extend the Osmocom GSM stack in one particular area where he has a particular need that the existing code cannot fulfill yet.

    This kind of project is the ideal match, as it shows where the true strength of FOSS is: Each of those customers did not have to fund the development of a GSM stack from scratch. Rather, they only had to fund those bits that were missing for their particular application.

    Our reference for this is and has been On-Waves, who have been funding development of their required features (and bug fixing etc.) since 2010.

    We've of course had many other projects from a variety of customers over over the years. Last, but not least, we had a customer who willingly co-funded (together with funds from NLnet foundation and lots of unpaid effort by sysmocom) the 3G/3.5G support in the Osmocom stack.

    The problem here is:

    • we have not been able to secure anywhere nearly as many of those R&D projects within the cellular industry, despite believing we have a very good foundation upon which we can built. I've been writing many exciting technical project proposals
    • you almost exclusively get funding only for new features. But it's very hard to get funding for the core maintenance work. The bug-fixing, code review, code refactoring, testing, etc.

    So as a result, the profit margin you have on selling R&D projects is basically used to (do a bad job of) fund those bits and pieces that nobody wants to pay for.

    Funding by means of customer support

    There is a way to generate funding for development by providing support services. We've had some success with this, but primarily alongside the actual hardware/system sales - not so much in terms of pure software-only support.

    Also, providing support services from a R&D company means:

    • either you distract your developers by handling support inquiries. This means they will have less time to work on actual code, and likely get side tracked by too many issues that make it hard to focus
    • or you have to hire separate support staff. This of course means that the size of the support business has to be sufficiently large to not only cover the cots of hiring + training support staff, but also still generate funding for the actual software R&D.

    We've tried shortly with the second option, but fallen back to the first for now. There's simply not sufficient user/admin type support business to rectify dedicated staff for that.

    Funding by means of cross-subsizing from other business areas

    sysmocom also started to do some non-Osmocom projects in order to generate revenue that we can feed again into Osmocom projects. I'm not at liberty to discuss them in detail, but basically we've been doing pretty much anything from

    • custom embedded Linux board designs
    • M2M devices with GSM modems
    • consulting gigs
    • public tendered research projects

    Profits from all those areas went again into Osmocom development.

    Last, but not least, we also operate the sysmocom webshop. The profit we make on those products also is again immediately re-invested into Osmocom development.

    Funding by grants

    We've had some success in securing funding from NLnet Foundation for specific features. While this is useful, the size of their projects grants of up to EUR 30k is not a good fit for the scale of the tasks we have at hand inside Osmocom. You may think that's a considerable amount of money? Well, that translates to 2-3 man-months of work at a bare cost-covering rate. At a team size of 6 developers, you would theoretically have churned through that in two weeks. Also, their focus is (understandably) on Internet and IT security, and not so much cellular communications.

    There are of course other options for grants, such as government research grants and the like. However, they require long-term planning, they require you to match (i.e. pay yourself) a significant portion, and basically mandate that you hire one extra person for doing all the required paperwork and reporting. So all in all, not a particularly attractive option for a very small company consisting of die hard engineers.

    Funding by more BTS ports

    At sysmocom, we've been doing some ports of the OsmoBTS + OsmoPCU software to other hardware, and supporting those other BTS vendors with porting, R&D and support services.

    If sysmocom was a classic BTS vendor, we would not help our "competition". However, we are not. sysmocom exists to help Osmocom, and we strongly believe in open systems and architectures, without a single point of failure, a single supplier for any component or any type of vendor lock-in.

    So we happily help third parties to get Osmocom running on their hardware, either with a proprietary PHY or with OsmoTRX.

    However, we expect that those BTS vendors also understand their responsibility to share the development and maintenance effort of the stack. Preferably by dedicating some of their own staff to work in the Osmocom community. Alternatively, sysmocom can perform that work as paid service. But that's a double-edged sword: We don't want to be a single point of failure.

    Osmocom funding outside of sysmocom

    Osmocom is of course more than sysmocom. Even for the cellular infrastructure projects inside Osmocom is true: They are true, community-based, open, collaborative development projects. Anyone can contribute.

    Over the years, there have been code contributions by e.g. Fairwaves. They, too, build GSM base station hardware and use that as a means to not only recover the R&D on the hardware, but also to contribute to Osmocom. At some point a few years ago, there was a lot of work from them in the area of OsmoTRX, OsmoBTS and OsmoPCU. Unfortunately, in more recent years, they have not been able to keep up the level of contributions.

    There are other companies engaged in activities with and around Osmcoom. There's Rhizomatica, an NGO helping indigenous communities to run their own cellular networks. They have been funding some of our efforts, but being an NGO helping rural regions in developing countries, they of course also don't have the deep pockets. Ideally, we'd want to be the ones contributing to them, not the other way around.

    State of funding

    We're making some progress in securing funding from players we cannot name [4] during recent years. We're also making occasional progress in convincing BTS suppliers to chip in their share. Unfortunately there are more who don't live up to their responsibility than those who do. I might start calling them out by name one day. The wider community and the public actually deserves to know who plays by FOSS rules and who doesn't. That's not shaming, it's just stating bare facts.

    Which brings us to:

    • sysmocom is in an office that's actually too small for the team, equipment and stock. But we certainly cannot afford more space.
    • we cannot pay our employees what they could earn working at similar positions in other companies. So working at sysmocom requires dedication to the cause :)
    • Holger and I have invested way more time than we have ever paid us, even more so considering the opportunity cost of what we would have earned if we'd continued our freelance Open Source hacker path
    • we're [just barely] managing to pay for 6 developers dedicated to Osmocom development on our payroll based on the various funding sources indicated above

    Nevertheless, I doubt that any such a small team has ever implemented an end-to-end GSM/GPRS/EGPRS network from RAN to Core at comparative feature set. My deepest respects to everyone involved. The big task now is to make it sustainable.

    Summary

    So as you can see, there's quite a bit of funding around. However, it always falls short of what's needed to implement all parts properly, and even not quite sufficient to keep maintaining the status quo in a proper and tested way. That can often be frustrating (mostly to us but sometimes also to users who run into regressions and oter bugs). There's so much more potential. So many things we wanted to add or clean up for a long time, but too little people interested in joining in, helping out - financially or by writing code.

    On thing that is often a challenge when dealing with traditional customers: We are not developing a product and then selling a ready-made product. In fact, in FOSS this would be more or less suicidal: We'd have to invest man-years upfront, but then once it is finished, everyone can use it without having to partake in that investment.

    So instead, the FOSS model requires the customers/users to chip in early during the R&D phase, in order to then subsequently harvest the fruits of that.

    I think the lack of a FOSS mindset across the cellular / telecom industry is the biggest constraining factor here. I've seen that some 20-15 years ago in the Linux world. Trust me, it takes a lot of dedication to the cause to endure this lack of comprehension so many years later.

    [1]just like Linux has started out.
    [2]while you will not find a lot of commits from Dieter in the code, he has been playing a key role in doing a lot of prototyping, reverse engineering and debugging!
    [3]sysmocom is 100% privately held by Holger and me, we intentionally have no external investors and are proud to never had to take a bank loan. So all we could invest was our own money and, most of all, time.
    [4]contrary to the FOSS world, a lot of aspects are confidential in business, and we're not at liberty to disclose the identities of all our customers

    by Harald Welte at June 15, 2017 10:00 PM

    FOSS misconceptions, still in 2017

    The lack of basic FOSS understanding in Telecom

    Given that the Free and Open Source movement has been around at least since the 1980ies, it puzzles me that people still seem to have such fundamental misconceptions about it.

    Something that really triggered me was an article at LightReading [1] which quotes Ulf Ewaldsson, a leading Ericsson excecutive with

    "I have yet to understand why we would open source something we think is really good software"

    This completely misses the point. FOSS is not about making a charity donation of a finished product to the planet.

    FOSS is about sharing the development costs among multiple players, and avoiding that everyone has to reimplement the wheel. Macro-Economically, it is complete and utter nonsense that each 3GPP specification gets implemented two dozens of times, by at least a dozen of different entities. As a result, products are way more expensive than needed.

    If large Telco players (whether operators or equipment manufacturers) were to collaboratively develop code just as much as they collaboratively develop the protocol specifications, there would be no need for replicating all of this work.

    As a result, everyone could produce cellular network elements at reduced cost, sharing the R&D expenses, and competing in key areas, such as who can come up with the most energy-efficient implementation, or can produce the most reliable hardware, the best receiver sensitivity, the best and most fair scheduling implementation, or whatever else. But some 80% of the code could probably be shared, as e.g. encoding and decoding messages according to a given publicly released 3GPP specification document is not where those equipment suppliers actually compete.

    So my dear cellular operator executives: Next time you're cursing about the prohibitively expensive pricing that your equipment suppliers quote you: You only have to pay that much because everyone is reimplementing the wheel over and over again.

    Equally, my dear cellular infrastructure suppliers: You are all dying one by one, as it's hard to develop everything from scratch. Over the years, many of you have died. One wonders, if we might still have more players left, if some of you had started to cooperate in developing FOSS at least in those areas where you're not competing. You could replicate what Linux is doing in the operating system market. There's no need in having a phalanx of different proprietary flavors of Unix-like OSs. It's way too expansive, and it's not an area in which most companies need to or want to compete anyway.

    Management Summary

    You don't first develop and entire product until it is finished and then release it as open source. This makes little economic sense in a lot of cases, as you've already invested into developing 100% of it. Instead, you actually develop a new product collaboratively as FOSS in order to not have to invest 100% but maybe only 30% or even less. You get a multitude of your R&D investment back, because you're not only getting your own code, but all the other code that other community members implemented. You of course also get other benefits, such as peer review of the code, more ideas (not all bright people work inside one given company), etc.

    [1]that article is actually a heavily opinionated post by somebody who appears to be pushing his own anti-FOSS agenda for some time. The author is misinformed about the fact that the TIP has always included projects under both FRAND and FOSS terms. As a TIP member I can attest to that fact. I'm only referencing it here for the purpose of that that Ericsson quote.

    by Harald Welte at June 15, 2017 10:00 PM

    June 13, 2017

    Free Electrons

    Elixir Cross Referencer: new way to browse kernel sources

    Today, we are pleased to announce the initial release of the Elixir Cross-Referencer, or just “Elixir”, for short.

    What is Elixir?

    Elixir home pageSince 2006, we have provided a Linux source code cross-referencing online tool as a service to the community. The engine behind this website was LXR, a Perl project almost as old as the kernel itself. For the first few years, we used the then-current 0.9.5 version of LXR, but in early 2009 and for various reasons, we reverted to the older 0.3.1 version (from 1999!). In a nutshell, it was simpler and it scaled better.

    Recently, we had the opportunity to spend some time on it, to correct a few bugs and to improve the service. After studying the Perl source code and trying out various cross-referencing engines (among which LXR 2.2 and OpenGrok), we decided to implement our own source code cross-referencing engine in Python.

    Why create a new engine?

    Our goal was to extend our existing service (support for multiple projects, responsive design, etc.) while keeping it simple and fast. When we tried other cross-referencing engines, we were dissatisfied with their relatively low performance on a large codebase such as Linux. Although we probably could have tweaked the underlying database engine for better performance, we decided it would be simpler to stick to the strategy used in LXR 0.3: get away from the relational database engine and keep plain lists in simple key-value stores.

    Another reason that motivated a complete rewrite was that we wanted to provide an up-to-date reference (including the latest revisions) while keeping it immutable, so that external links to the source code wouldn’t get broken in the future. As a direct consequence, we would need to index many different revisions for each project, with potentially a lot of redundant information between them. That’s when we realized we could leverage the data model of Git to deal with this redundancy in an efficient manner, by indexing Git blobs, which are shared between revisions. In order to make sure queries under this strategy would be fast enough, we wrote a proof-of-concept in Python, and thus Elixir was born.

    What service does it provide?

    First, we tried to minimize disruption to our users by keeping the user interface close to that of our old cross-referencing service. The main improvements are:

    • We now support multiple projects. For now, we provide reference for Linux, Busybox and U-Boot.
    • Every tag in each project’s git repository is now automatically indexed.
    • The design has been modernized and now fits comfortably on smaller screens like tablets.
    • The URL scheme has been simplified and extended with support for multiple projects. An HTTP redirector has been set up for backward compatibility.
    Elixir supports multiple projects

    Elixir supports multiple projects

    Among other smaller improvements, it is now possible to copy and paste code directly without line numbers getting in the way.

    How does it work?

    Elixir is made of two Python scripts: “update” and “query”. The first looks for new tags and new blobs inside a Git repository, parses them and appends the new references to identifiers to a record inside the database. The second uses the database and the Git repository to display annotated source code and identifier references.

    The parsing itself is done with Ctags, which provides us with identifier definitions. In order to find the references to these identifiers, Elixir then simply checks each lexical token in the source file against the definition database, and if that word is defined, a new reference is added.

    Like in LXR 0.3, the database structure is kept very simple so that queries don’t have much work to do at runtime, thus speeding them up. In particular, we store references to a particular identifier as a simple list, which can be loaded and parsed very fast. The main difference with LXR is that our list includes references from every blob in the project, so we need to restrict it first to only the blobs that are part of the current version. This is done at runtime, simply by computing the intersection of this list with the list of blobs inside the current version.

    Finally, we kept the user interface code clearly segregated from the engine itself by making these two modules communicate through a Unix command-line interface. This means that you can run queries directly on the command-line without going through the web interface.

    Elixir code example

    Elixir code example

    What’s next?

    Our current focus is on improving multi-project support. In particular, each project has its own quirky way of using Git tags, which needs to be handled individually.

    At the user-interface level, we are evaluating the possibility of having auto-completion and/or fuzzy search of identifier names. Also, we are looking for a way to provide direct line-level access to references even in the case of very common identifiers.

    On the performance front, we would like to cut the indexation time by switching to a new database back-end that provides efficient appending to large records. Also, we could make source code queries faster by precomputing the references, which would also allow us to eliminate identifier “bleeding” between versions (the case where an identifier shows up as “defined in 0 files” because it is only defined in another version).

    If you think of other ways we could improve our service, don’t hesitate to drop us a feature request or a patch!

    Bonus: why call it “Elixir”?

    In the spur of the moment, it seemed like a nice pun on the name “LXR”. But in retrospect, we wish to apologize to the Elixir language team and the community at large for unnecessary namespace pollution.

    by Mikael Bouillot at June 13, 2017 07:39 AM

    June 09, 2017

    Free Electrons

    Beyond boot testing: custom tests with LAVA

    Since April 2016, we have our own automated testing infrastructure to validate the Linux kernel on a large number of hardware platforms. We use this infrastructure to contribute to the KernelCI project, which tests every day the Linux kernel. However, the tests being done by KernelCI are really basic: it’s mostly booting a basic Linux system and checking that it reaches a shell prompt.

    However, LAVA, the software component at the core of this testing infrastructure, can do a lot more than just basic tests.

    The need for custom tests

    With some of our engineers being Linux maintainers and given all the platforms we need to maintain for our customers, being able to automatically test specific features beyond a simple boot test was a very interesting goal.

    In addition, manually testing a kernel change on a large number of hardware platforms can be really tedious. Being able to quickly send test jobs that will use an image you built on your machine can be a great advantage when you have some new code in development that affects more than one board.

    We identified two main use cases for custom tests:

    • Automatic tests to detect regression, as does KernelCI, but with more advanced tests, including platform specific tests.
    • Manual tests executed by engineers to validate that the changes they are developing do not break existing features, on all platforms.

    Overall architecture

    Several tools are needed to run custom tests:

    • The LAVA instance, which controls the hardware platforms to be tested. See our previous blog posts on our testing hardware infrastructrure and software architecture
    • An appropriate root filesystem, that contains the various userspace programs needed to execute the tests (benchmarking tools, validation tools, etc.)
    • A test suite, which contains various scripts executing the tests
    • A custom test tool that glues together the different components

    The custom test tool knows all the hardware platforms available and which tests and kernel configurations apply to which hardware platforms. It identifies the appropriate kernel image, Device Tree, root filesystem image and test suite and submits a job to LAVA for execution. LAVA will download the necessary artifacts and run the job on the appropriate device.

    Building custom rootfs

    When it comes to test specific drivers, dedicated testing, validation or benchmarking tools are sometimes needed. For example, for storage device testing, bonnie++ can be used, while iperf is nice for networking testing. As the default root filesystem used by KernelCI is really minimalist, we need to build our owns, one for each architecture we want to test.

    Buildroot is a simple yet efficient tool to generate root filesystems, it is also used by KernelCI to build their minimalist root filesystems. We chose to use it and made custom configuration files to match our needs.

    We ended up with custom rootfs built for ARMv4, ARMv5, ARMv7, and ARMv8, that embed for now Bonnie++, iperf, ping (not the Busybox implementation) and other tiny tools that aren’t included in the default Buildroot configuration.

    Our Buildroot fork that includes our custom configurations is available as the buildroot-ci Github project (branch ci).

    The custom test tool

    The custom test tool is the tool that binds the different elements of the overall architecture together.

    One of the main features of the tool is to send jobs. Jobs are text files used by LAVA to know what to do with which device. As they are described in LAVA as YAML files (in the version 2 of the API), it is easy to use templates to generate them based on a single model. Some information is quite static such as the device tree name for a given board or the rootfs version to use, but other details change for every job such as the kernel to use or which test to run.

    We made a tool able to get the latest kernel images from KernelCI to quickly send jobs without having a to compile a custom kernel image. If the need is to test a custom image that is built locally, the tool is also able to send files to the LAVA server through SSH, to provide a custom kernel image.

    The entry point of the tool is ctt.py, which allows to create new jobs, providing a lot of options to define the various aspects of the job (kernel, Device Tree, root filesystem, test, etc.).

    This tool is written in Python, and lives in the custom_tests_tool Github project.

    The test suite

    The test suite is a set of shell scripts that perform tests returning 0 or 1 depending on the result. This test suite is included inside the root filesystem by LAVA as part of a preparation step for each job.

    We currently have a small set of tests:

    • boot test, which simply returns 0. Such a test will be successful as soon as the boot succeeds.
    • mmc test, to test MMC storage devices
    • sata test, to test SATA storage devices
    • crypto test, to do some minimal testing of cryptographic engines
    • usb test, to test USB functionality using mass storage devices
    • simple network test, that just validates network connectivity using ping

    All those tests only require the target hardware platform itself. However, for more elaborate network tests, we needed to get two devices to interact with each other: the target hardware platform and a reference PC platform. For this, we use the LAVA MultiNode API. It allows to have a test that spans multiple devices, which we use to perform multiple iperf sessions to benchmark the bandwidth. This test has therefore one part running on the target device (network-board) and one part running on the reference PC platform (network-laptop).

    Our current test suite is available as the test_suite Github project. It is obviously limited to just a few tests for now, we hope to extend the tests in the near future.

    First use case: daily tests

    As previously stated, it’s important for us to know about regressions introduced in the upstream kernel. Therefore, we have set up a simple daily cron job that:

    • Sends custom jobs to all boards to validate the latest mainline Linux kernel and latest linux-nextli>
    • Aggregates results from the past 24 hours and sends emails to subscribed addresses
    • Updates a dashboard that displays results in a very simple page

    A nice dashboard showing the tests of the Beaglebone Black and the Nitrogen6x.

    Second use case: manual tests

    The custom test tool ctt.py has a simple command line interface. It’s easy for someone to set it up and send custom jobs. For example:

    ctt.py -b beaglebone-black -m network
    

    will start the network test on the BeagleBone Black, using the latest mainline Linux kernel built by KernelCI. On the other hand:

    ctt.py -b armada-7040-db armada-8040-db -t mmc --kernel arch/arm64/boot/Image --dtb-folder arch/arm64/boot/dts/
    

    will run the mmc test on the Marvell Armada 7040 and Armada 8040 development boards, using the locally built kernel image and Device Tree.

    The result of the job is sent over e-mail when the test has completed.

    Conclusion

    Thanks to this custom test tool, we now have an infrastructure that leverages our existing lab and LAVA instance to execute more advanced tests. Our goal is now to increase the coverage, by adding more tests, and run them on more devices. Of course, we welcome feedback and contributions!

    by Florent Jacquet at June 09, 2017 02:25 PM

    May 30, 2017

    Bunnie Studios

    Name that Ware May 2017

    The Ware for May 2017 is shown below.

    This is another one where the level difficulty will depend on if I cropped enough detail out of the photo to make it challenging but not impossible. If you do figure this one out quickly, curious to hear which detail tipped you off!

    by bunnie at May 30, 2017 08:04 AM

    Winner, Name that Ware April 2017

    The Ware for April 2017 is an HP 10780A optical receiver. Congrats to Brian for absolutely nailing this one! email me for your prize.

    by bunnie at May 30, 2017 08:04 AM

    May 29, 2017

    Open Hardware Repository

    sfp-plus-i2c - SaFariPark now open for public

    SaFariPark is not a site to book holidays on the African plains - though with additional personal funding I am willing to add that feature. It is a software tool to read and write the digital interface of SFP/SFP+ transceiver modules. Together with a device to plug in multiple (4) SFP/SFP+ modules, creatively called MultiSFP (see Figure 1), it is a versatile tool for all your SFP needs. MultiSFP and SaFariPark have been developed by Nikhef as part of the ASTERICS program, and all is open hardware/open source.


    Figure 1 - MultiSFP front panel

    MultiSFP supports a 10 Gigabit capable connection to the electrical interface of each SFP. Via one USB port each SFP I2C bus can be exercised using SaFariPark. The software main window (Figure 2) exposes most functionality, which are:
    • Editing of individual fields in the SFP module
    • Fixing corrupted SFP EEPROM data, recalculating checksums
    • Showing and saving SFP+ sensor data such as TX/RX power and temperature.
    • Selectively copying content of one SFP module to multiple other modules
    • Laser tuning of optical SFP+ modules


    Figure 2 - Main window of SaFariPark

    Apart from this SaFariPark allows you to dump the entire EEPROM content, and extend the SFP+ EEPROM data dictionary with custom fields using XML. This enables users to add fields for custom or exotic SFP+ modules. As the software is written Java, it has been verified to work on Linux and Windows. Mac has not been tested yet.

    More information can be found here: Also see:

    by Vincent van Beveren (v.van.beveren@nikhef.nl) at May 29, 2017 03:07 PM

    May 28, 2017

    Harald Welte

    Playing back GSM RTP streams, RTP-HR bugs

    Chapter 0: Problem Statement

    In an all-IP GSM network, where we use Abis, A and other interfaces within the cellular network over IP transport, the audio of voice calls is transported inside RTP frames. The codec payload in those RTP frames is the actual codec frame of the respective cellular voice codec. In GSM, there are four relevant codecs: FR, HR, EFR and AMR.

    Every so often during the (meanwhile many years of ) development of Osmocom cellular infrastructure software it would have been useful to be able to quickly play back the audio for analysis of given issues.

    However, until now we didn't have that capability. The reason is relatively simple: In Osmocom, we genally don't do transcoding but simply pass the voice codec frames from left to right. They're only transcoded inside the phones or inside some external media gateway (in case of larger networks).

    Chapter 1: GSM Audio Pocket Knife

    Back in 2010, when we were very actively working on OsmocomBB, the telephone-side GSM protocol stack implementation, Sylvain Munaut wrote the GSM Audio Pocket Knife (gapk) in order to be able to convert between different formats (representations) of codec frames. In cellular communcations, everyoe is coming up with their own representation for the codec frames: The way they look on E1 as a TRAU frame is completely different from how RTP payload looks like, or what the TI Calypso DSP uses internally, or what a GSM Tester like the Racal 61x3 uses. The differences are mostly about data types used, bit-endinanness as well as padding and headers. And of course those different formats exist for each of the four codecs :/

    In 2013 I first added simplistic RTP support for FR-GSM to gapk, which was sufficient for my debugging needs back then. Still, you had to save the decoded PCM output to a file and play that back, or use a pipe into aplay.

    Last week, I picked up this subject again and added a long series of patches to gapk:

    • support for variable-length codec frames (required for AMR support)
    • support for AMR codec encode/decode using libopencore-amrnb
    • support of all known RTP payload formats for all four codecs
    • support for direct live playback to a sound card via ALSA

    All of the above can now be combined to make GAPK bind to a specified UDP port and play back the RTP codec frames that anyone sends to that port using a command like this:

    $ gapk -I 0.0.0.0/30000 -f rtp-amr -A default -g rawpcm-s16le

    I've also merged a chance to OsmoBSC/OsmoNITB which allows the administrator to re-direct the voice of any active voice channel towards a user-specified IP address and port. Using that you can simply disconnect the voice stream from its normal destination and play back the audio via your sound card.

    Chapter 2: Bugs in OsmoBTS GSM-HR

    While going through the exercise of implementing the above extension to gapk, I had lots of trouble to get it to work for GSM-HR.

    After some more digging, it seems there are two conflicting specification on how to format the RTP payload for half-rate GSM:

    In Osmocom, we claim to implement RFC5993, but it turned out that (at least) osmo-bts-sysmo (for sysmoBTS) was actually implementing the ETSI format instead.

    And even worse, osmo-bts-sysmo gets event the ETSI format wrong. Each of the codec parameters (which are unaligned bit-fields) are in the wrong bit-endianness :(

    Both the above were coincidentially also discovered by Sylvain Munaut during operating of the 32C3 GSM network in December 2015 and resulted the two following "work around" patches: * HACK for HR * HACK: Fix the bit order in HR frames

    Those merely worked around those issues in the rtp_proxy of OsmoNITB, rather than addressing the real issue. That's ok, they were "quick" hacks to get something working at all during a four-day conference. I'm now working on "real" fixes in osmo-bts-sysmo. The devil is of course in the details, when people upgrade one BTS but not the other and want to inter-operate, ...

    It yet remains to be investigated how osmo-bts-trx and other osmo-bts ports behave in this regard.

    Chapter 3: Conclusions

    Most definitely it is once again a very clear sign that more testing is required. It's tricky to see even wih osmo-gsm-tester, as GSM-HR works between two phones or even two instances of osmo-bts-sysmo, as both sides of the implementation have the same (wrong) understanding of the spec.

    Given that we can only catch this kind of bug together with the hardware (the DSP runs the PHY code), pure unit tests wouldn't catch it. And the end-to-end test is also not very well suited to it. It seems to call for something in betewen. Something like an A-bis interface level test.

    We need more (automatic) testing. I cannot say that often enough. The big challenge is how to convince contributors and customers that they should invest their time and money there, rather than yet-another (not automatically tested) feature?

    by Harald Welte at May 28, 2017 10:00 PM

    Mirko Vogt, nanl.de

    SonOTA – Flashing Itead Sonoff devices via original OTA mechanism

    Long story short

    There’s now a script with which you can flash your sonoff device via the original internal OTA upgrade mechanism, meaning, no need to open, solder, etc. the device to get your custom firmware onto it.

    This isn’t perfect (yet) — please mind the issues at the end of this post!

    https://github.com/mirko/SonOTA

    Credits

    First things first: Credits!
    The problem with credits is you usually forget somebody and that’s most likely happening here as well.
    I read around quite a lot, gathered information and partially don’t even remember anymore where I read what (first).

    Of course I’m impressed by the entire Tasmota project and what it enables one to do with the Itead Sonoff and similar devices.

    Special thanks go to khcnz who helped me a lot in a discussion documented here.

    I’d also like to mention Richard Burtons, who I didn’t interact with directly but only read his blog. That guy apparently was too bored by all the amazing tech stuff he was doing for a living, so he took a medical degree and is now working as a doctor, has a passion for horology (meaning, he’s building a turrot clock), is sailing regattas with his own rs200, decompiles and reverse-engineers proprietary bootloaders in his spare time and writes a new bootloader called rboot for the ESP8266 as a side project.

    EDIT: Jan Almeroth already reversed some of the protocol in 2016 and also documented the communication between the proprietary EWeLink app and the AWS cloud. Unfortunately I only became aware of that great post after I already finished mine.

    Introduction Sonoff devices

    Quite recently the Itead Sonoff series — a bunch of ESP8266 based IoT homeautomation devices — was brought to my attention.

    The ESP8266 is a low-power consumption SoC especially designed for IoT purposes. It’s sold by Espressif, running a 32-Bit processor featuring the Xtensa instruction set (licensed from Tensilica) and having an ASIC IP core and WiFi onboard.

    Those Sonoff devices using this SoC basically expect high voltage input, therewith having an AC/DC (5V) converter, the ESP8266 SoC and a relais switching the high voltage output.
    They’re sold as wall switches (“Sonoff Touch”), E27 socket adapters (“Slampher”), power sockets (“S20 smart socket”) or as just — that’s most basic cheapest model — all that in a simple case (“Sonoff Basic”).
    They also have a bunch of sensoric devices, measuring temperature, power comsumption, humidty, noise levels, fine dust, etc.

    Though I’m rather sceptical about the whole IoT (development) philosophy, I always was (and still am) interested into low-cost and power-saving home automation which is completely and exclusively under my control.

    That implies I’m obviously not interested in some random IoT devices being necessarily connected to some Google/Amazon/Whatever cloud, even less if sensible data is transmitted without me knowing (but very well suspecting) what it’s used for.

    Guess what the Itead Sonoff devices do? Exactly that! They even feature Amazon Alexa and Google Nest support! And of course you have to use their proprietary app to confgure and control your devices — via the Amazon cloud.

    However, as said earlier, they’re based on the ESP8266 SoC, around which a great deal of OpenSource projects evolved. For some reason especially the Arduino community pounced on that SoC, enabling a much broader range of people to play around with and program for those devices. Whether that’s a good and/or bad thing is surely debatable.

    I’ll spare you the details about all the projects I ran into, there’s plenty of cool stuff out there.

    I decided to go for the Sonoff-Tasmota project which is quite actively developed and supports most of the currently available Sonoff devices.

    It provides an HTTP and MQTT interface and doesn’t need any connection to the internet at all. As MQTT sever (in MQTT speech called broker) I use mosquitto which I’m running on my OpenWrt WiFi router.

    Flashing custom firmware (via serial)

    Flashing your custom firmware onto those devices however always requires opening them, soldering a serial cable, pulling GPIO0 down to get the SoC into programming mode (which, depending on the device type, again involes soldering) and then flash your firmware via serial.

    Side note: Why do all those projects describing the flashing procedure name an “FTDI serial converter” as a requirement? Every serial TTL converter does the job.
    And apart from that FTDI is not a product but a company, it’s a pretty shady one. I’d just like to remind of the “incident” where FTDI released new drivers for their chips which intentionally bricked clones of their converters.

    How to manually flash via serial — even though firmware replacement via OTA (kinda) works now, you still might want unbrick or debug your device — the Tasmota wiki provides instructions for each of the supported devices.

    Anyway, as I didn’t want to open and solder every device I intend to use, I took a closer look at the original firmware and its OTA update mechanism.

    Protocol analysis

    First thing after the device is being configured (meaning, the device got configured by the proprietary app and is therewith now having internet access via your local WiFi network) is to resolve the hostname `eu-disp.coolkit.cc` and attempt to establish a HTTPS connection.

    Though the connection is SSL, it doesn’t do any server certificate verification — so splitting the SSL connection and *man-in-the-middle it is fairly easy.

    As a side effect I ported the mitm project sslsplit to OpenWrt and created a seperate “interception”-network on my WiFi router. Now I only need to join that WiFi network and all SSL connections get split, its payload logged and being provided on an FTP share. Intercepting SSL connections never felt easier.

    Back to the protocol: We’re assuming at this point the Sonoff device was already configured (e.g. by the official WeLink app) which means it has joined our WiFi network, acquired IP settings via DHCP and has access to the internet.

    The Sonoff device sends a dispatch call as HTTPS POST request to eu-disp.coolkit.cc including some JSON encoded data about itself:

    
    POST /dispatch/device HTTP/1.1
    Host: eu-disp.coolkit.cc
    Content-Type: application/json
    Content-Length: 152
    
    {
      "accept":     "ws;2",
      "version":    2,
      "ts":         119,
      "deviceid":   "100006XXXX",
      "apikey":     "6083157d-3471-4f4c-8308-XXXXXXXXXXXX",
      "model":      "ITA-GZ1-GL",
      "romVersion": "1.5.5"
    }
    

    It expects an also JSON encoded host as an answer
    
    HTTP/1.1 200 OK
    Server: openresty
    Date: Mon, 15 May 2017 01:26:00 GMT
    Content-Type: application/json
    Content-Length: 55
    Connection: keep-alive
    
    {
      "error":  0,
      "reason": "ok",
      "IP":     "52.29.48.55",
      "port":   443
    }
    

    which is used to establish a WebSocket connection
    
    GET /api/ws HTTP/1.1
    Host: iotgo.iteadstudio.com
    Connection: upgrade
    Upgrade: websocket
    Sec-WebSocket-Key: ITEADTmobiM0x1DaXXXXXX==
    Sec-WebSocket-Version: 13
    

    
    HTTP/1.1 101 Switching Protocols
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Accept: q1/L5gx6qdQ7y3UWgO/TXXXXXXA=
    

    which consecutively will be used for further interchange.
    Payload via the established WebSocket channel continues to be encoded in JSON.
    The messages coming from the device can be classified into action-requests initiated by the device (which expect ackknowledgements by the server) and acknowledgement messages for requests initiated by the server.

    The first requests are action-requests coming from the device:

    1) action: register

    {
      "userAgent":  "device",
      "apikey":     "6083157d-3471-4f4c-8308-XXXXXXXXXXXX",
      "deviceid":   "100006XXXX",
      "action":     "register",
      "version":    2,
      "romVersion": "1.5.5",
      "model":      "ITA-GZ1-GL",
      "ts":         712
    }

    responded by the server with
    {
      "error":       0,
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "config": {
        "hb":         1,
        "hbInterval": 145
      }
    }

    As can be seen, action-requests initiated from server side also have an apikey field which can be — as long its used consistently in that WebSocket session — any generated UUID but the one used by the device.

    2) action: date

    {
      "userAgent":  "device",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "deviceid":   "100006XXXX",
      "action"      :"date"
    }

    responded with
    {
      "error":      0,
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "date":       "2017-05-15T01:26:01.498Z"
    }

    Pay attention to the date format: it is some kind ISO 8601 but the parser is really picky about it. While python’s datetime.isoformat() function e.g. returns a string taking microseconds into account, the parser on the device will just fail parsing that. It also always expects the actually optional timezone being specified as UTC and only as a trailing Z (though according to the spec “00:00” would be valid as well).

    3) action: update — the device tells the server its switch status, the MAC address of the accesspoint it is connected to, signal quality, etc.
    This message also appears everytime the device status changes, e.g. it got switched on/off via the app or locally by pressing the button.

    {
      "userAgent":      "device",
      "apikey":         "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "deviceid":       "100006XXXX",
      "action":         "update",
      "params": {
        "switch":         "off",
        "fwVersion":      "1.5.5",
        "rssi":           -41,
        "staMac":         "5C:CF:7F:F5:19:F8",
        "startup":        "off"
      }
    }

    simply acknowlegded with
    {
      "error":      0,
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX"
    }

    4) action: query — the device queries potentially configured timers
    {
      "userAgent":  "device",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "deviceid":   "100006XXXX",
      "action":     "query",
      "params": [
        "timers"
      ]
    }

    as there are no timers configured the answer simply contains a "params":0 KV-pair
    {
      "error":      0,
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "params":     0
    }

    That’s it – that’s the basic handshake after the (configured) device powers up.

    Now the server can tell the device to do stuff.

    The sequence number is used by the device to acknowledge particular action-requests so the response can be mapped back to the actual request. It appears to be a UNIX timestamp with millisecond precision which doesn’t seem like the best source for generating a sequence number (duplicates, etc.) but seems to work well enough.

    Let’s switch the relais:

    {
      "action":     "update",
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "userAgent":  "app",
      "sequence":   "1494806715179",
      "ts":         0,
      "params": {
        "switch":     "on"
      },
      "from":       "app"
    }
    
    {
      "action":     "update",
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "userAgent":  "app",
      "sequence":   "1494806715193",
      "ts":         0,
      "params": {
        "switch":     "off"
      },
      "from":       "app"
    }

    As mentioned earlier, each action-request is responded with proper acknowledgements.

    And — finally — what the server now also is capable doing is to tell the device to update itself:

    {
      "action":     "upgrade",
      "deviceid":   "100006XXXX",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "userAgent":  "app",
      "sequence":   "1494802194654",
      "ts":         0,
      "params": {
        "binList":[
          {
            "downloadUrl":  "http://52.28.103.75:8088/ota/rom/xpiAOwgVUJaRMqFkRBsoI4AVtnozgwp1/user1.1024.new.2.bin",
            "digest":       "1aee969af1daf96f3f120323cd2c167ae1aceefc23052bb0cce790afc18fc634",
            "name":         "user1.bin"
          },
          {
            "downloadUrl":  "http://52.28.103.75:8088/ota/rom/xpiAOwgVUJaRMqFkRBsoI4AVtnozgwp1/user2.1024.new.2.bin",
            "digest":       "6c4e02d5d5e4f74d501de9029c8fa9a7850403eb89e3d8f2ba90386358c59d47",
            "name":         "user2.bin"
          }
        ],
        "model":    "ITA-GZ1-GL",
        "version":  "1.5.5",
      }
    }

    After successful download and verification of the image’s checksum the device returns:
    {
      "error":      0,
      "userAgent":  "device",
      "apikey":     "85036160-aa4a-41f7-85cc-XXXXXXXXXXXX",
      "deviceid":   "100006XXXX",
      "sequence":   "1495932900713"
    }

    The downloadUrl field should be self-explanatory (the following HTTP GET request to those URLs contain some more data as CGI parameters which however can be ommitted).

    The digest is a sha256 hash of the file and the name is the partition onto which the file should be written on.

    Implementing server side

    After some early approaches I decided to go for a Python implementation using the tornado webserver stack.
    This decision was mainly based on it providing functionality for HTTP (obviously) as well as websockets and asynchronous handling of requests.

    The final script can be found here: https://github.com/mirko/SonOTA

    ==> Trial & Error

    1st attempt

    As user1.1024.new.2.bin and user2.20124.new.2.bin almost look the same, let’s just use the same image for both, in this case a tasmota build:

    MOEP! Boot fails.

    Reason: The tasmota build also contains the bootloader which the Espressif OTA mechanism doesn’t expect being in the image.

    2nd attempt

    Chopping off the first 0x1000 bytes which contain the bootloader plus padding (filled up with 0xAA bytes).

    MOEP! Boot fails.

    Boot mode 1 and 2 / v1 and v2 image headers

    The (now chopped) image and the original upgrade images appear to have different headers — even the very first byte (the files’ magic byte) differ.

    The original image starts with 0xEA while the Tasmota build starts with 0xE9.

    Apparently there are two image formats (called v1 and v2 or boot mode 1 and boot mode 2).
    The former (older) one — used by Arduino/Tasmota — starts with 0xE9, while the latter (and apparently newer one) — used by the original firmware — starts with 0xEA.

    The technical differences are very well documented by the ESP8266 Reverse Engineering Wiki project, regarding the flash format and the v1/v2 headers in particular the SPI Flash Format wiki oage.

    The original bootloader only accepts images starting with 0xEA while the bootloader provided by Arduino/Tasmota only accepts such starting with 0xE9.

    3rd attempt

    Converting Arduino images to v2 images

    Easier said than done, as the Arduino framework doesn’t seem to be capable of creating v2 images and none of the common tools appear to have conversion functionality.

    Taking a closer look at the esptool.py project however, there seems to be (undocumented) functionality.
    esptool.py has the elf2image argument which — according source — allows switching between conversion to v1 and v2 images.

    When using elf2image and also passing the --version parameter — which normally prints out the version string of the tool — the --version parameter gets redefined and expects an then argument: 1 or 2.

    Besides the sonoff.ino.bin file the Tasmota project also creates an sonoff.ino.elf which can now be used in conjunction with esptool.py and the elf2image-parameter to create v2 images.

    Example: esptool.py elf2image --version 2 tmp/arduino_build_XXXXXX/sonoff.ino.elf

    WORKS! MOEP! WORKS! MOEP!

    Remember the upgrade-action passed a 2-element list of download URLs to the device, having different names (user1.bin and user2.bin)?

    This procedure now only works if the user1.bin image is being fetched and flashed.

    Differences between user1.bin and user2.bin

    The flash on the Sonoff devices is split into 2 parts (simplified!) which basically contain the same data (user1 and user2). As OTA upgrades are proven to fail sometimes for whatever reason, the upgrade will always happen on the currently inactive part, meaning, if the device is currently running the code from the user1 part, the upgrade will happen onto the user2 part.
    That mechanism is not invented by Itead, but actually provided as off-the-shelf OTA solution by Espressif (the SoC manufacturer) itself.

    For 1MB flash chips the user1 image is stored at offset 0x01000 while the user2 image is stored at 0x81000.

    And indeed, the two original upgrade images (user1 and user2) differ significantly.

    If flashing a user2 image onto the user1 part of the flash the device refuses to boot and vice versa.

    While there’s not much information about how user1.bin and user2.bin technically differ from each other, khcnz pointed me to an Espressif document stating:

    user1.bin and user2.bin are [the] same software placed to different regions of [the] flash. The only difference is [the] address mapping on flash.

    4th attempt

    So apparently those 2 images must be created differently indeed.

    Again it was khcnz who pointed me to different linker scripts used for each image within the original SDK.
    Diffing
    https://github.com/espressif/ESP8266_RTOS_SDK/blob/master/ld/eagle.app.v6.new.1024.app1.ld
    and
    https://github.com/espressif/ESP8266_RTOS_SDK/blob/master/ld/eagle.app.v6.new.1024.app2.ld
    reveals that the irom0_0_seg differs (org = 0x40100000 vs. org = 0x40281010).

    As Tasmota doesn’t make use of the user1-/user2-ping-pong mechanism it conly creates images supposed to go to 0x1000 (=user1-partition).

    So for creating an user2.bin image — in our case for a device having a 1MB flash chip and allocating (only) 64K for SPIFFS — we have to modify the following linker script accordingly:

    --- a/~/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/sdk/ld/eagle.flash.1m64.ld
    +++ b/~/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/sdk/ld/eagle.flash.1m64.ld
    @@ -7,7 +7,7 @@ MEMORY
       dport0_0_seg :                        org = 0x3FF00000, len = 0x10
       dram0_0_seg :                         org = 0x3FFE8000, len = 0x14000
       iram1_0_seg :                         org = 0x40100000, len = 0x8000
    -  irom0_0_seg :                         org = 0x40201010, len = 0xf9ff0
    +  irom0_0_seg :                         org = 0x40281010, len = 0xf9ff0
     }
     
     PROVIDE ( _SPIFFS_start = 0x402FB000 );

    So we will now create an user1 (without above applied modification> and an user2 (with above modification> image and converting them to v2 images with esptool.py as described above.

    –> WORKS!

    Depending on whether the original firmware was loaded from the user1 or user2 partition, it will fetch and flash the other image, telling the bootloader afterwards to change the active partition.

    Issues

    Mission accomplished? Not just yet…

    Although our custom firmware is now flashed via the original OTA mechanism and running, the final setup differs in 2 major aspects (compared to if we would have flashed the device via serial):

    • The bootloader is still the original one
    • Our custom image might have ended up in the user2 partition

    Each point alone already results in the Tasmota/Adruino OTA mechniasm not working.
    Additionally — since the bootloader stays the original one — it still only expects v2 images and still messes with us with its ping-pong-mechanism.

    This issue is already being addressed though and discussed on how to be solved best in the issue ticket mentioned at the very beginning.

    Happy hacking!

    by mirko at May 28, 2017 08:05 PM

    May 26, 2017

    Open Hardware Repository

    Hdlmake - HDLMake version 3.0 promoted to Master

    HDLMake 3.0

    After a massive refactoring & upgrade process, we have finally published the brand-new HDLMake 3.0 version. This version not only sports a whole set of new features, but has been carefully crafted so that the source code providing a common interface for the growing set of supported tools can be easily maintained.

    New Features

    These are some of the highlighted features for the new HDLMake v3.0 Release:

    • Updated HDL code parser and solver: the new release includes by default the usage of an embedded HDL code parser and file dependency solver to manage the synthesis and simulation process in an optimal way.
    • Support for Python 3.x: the new release supports both Python2.7 and Python3.x deployments in a single source code branch, enabling an easier integration into newer O.S. distributions.
    • Native support for Linux & Windows shells: The new release not only supports Linux shells as the previous ones, but features native support too for Windows shells such as the classic CMD promt or the new PowerShell.
    • TCL based Makefiles: in order to streamline the process of supporting as many tools as possible in a hierarchical way, in a changing world and rapidly evolving world of FPGA technology and tool providers, we have adopted TCL as the common language layer used by the generated synthesis Makefiles.
    • Proper packaging: from the HDLMake 3.0 onwards, the source code is distributed as a Python package, what allows for a much cleaner installation procedure.

    More info

    You can find more info about the HDLMake 3.0 version in the following links:

    by Javier D. Garcia-Lasheras (jgarcia@gl-research.com) at May 26, 2017 01:35 PM

    May 23, 2017

    Harald Welte

    Power-cycling a USB port should be simple, right?

    Every so often I happen to be involved in designing electronics equipment that's supposed to run reliably remotely in inaccessible locations,without any ability for "remote hands" to perform things like power-cycling or the like. I'm talking about really remote locations, possible with no but limited back-haul, and a very high cost of ever sending somebody there for remote maintenance.

    Given that a lot of computer peripherals (chips, modules, ...) use USB these days, this is often some kind of an embedded ARM (rarely x86) SoM or SBC, which is hooked up to a custom board that contains a USB hub chip as well as a line of peripherals.

    One of the most important lectures I've learned from experience is: Never trust reset signals / lines, always include power-switching capability. There are many chips and electronics modules available on the market that have either no RESET, or even might claim to have a hardware RESET line which you later (painfully) discover just to be a GPIO polled by software which can get stuck, and hence no way to really hard-reset the given component.

    In the case of a USB-attached device (even though the USB might only exist on a circuit board between two ICs), this is typically rather easy: The USB hub is generally capable of switching the power of its downstream ports. Many cheap USB hubs don't implement this at all, or implement only ganged switching, but if you carefully select your USB hub (or in the case of a custom PCB), you can make sure that the given USB hub supports individual port power switching.

    Now the next step is how to actually use this from your (embedded) Linux system. It turns out to be harder than expected. After all, we're talking about a standard feature that's present in the USB specifications since USB 1.x in the late 1990ies. So the expectation is that it should be straight-forward to do with any decent operating system.

    I don't know how it's on other operating systems, but on Linux I couldn't really find a proper way how to do this in a clean way. For more details, please read my post to the linux-usb mailing list.

    Why am I running into this now? Is it such a strange idea? I mean, power-cycling a device should be the most simple and straight-forward thing to do in order to recover from any kind of "stuck state" or other related issue. Logical enabling/disabling of the port, resetting the USB device via USB protocol, etc. are all just "soft" forms of a reset which at best help with USB related issues, but not with any other part of a USB device.

    And in the case of e.g. an USB-attached cellular modem, we're actually talking about a multi-processor system with multiple built-in micro-controllers, at least one DSP, an ARM core that might run another Linux itself (to implement the USB gadget), ... - certainly enough complex software that you would want to be able to power-cycle it...

    I'm curious what the response of the Linux USB gurus is.

    by Harald Welte at May 23, 2017 10:00 PM

    Open Hardware Repository

    Yet Another Micro-controller - YAM first release at OHR

    YAM release V1.4 is now available from CERN OHR.

    The core has been already used in a number of designs at ESRF .

    However there is still some pending work.
    • The co-processors have not been fully tested yet.
    • The yamasm assembler doesn't yet support the 3-operand implementation.

    by Christian Herve at May 23, 2017 08:41 AM

    May 22, 2017

    Free Electrons

    Introducing lavabo, board remote control software

    In two previous blog posts, we presented the hardware and software architecture of the automated testing platform we have created to test the Linux kernel on a large number of embedded platforms.

    The primary use case for this infrastructure was to participate to the KernelCI.org testing effort, which tests the Linux kernel every day on many hardware platforms.

    However, since our embedded boards are now fully controlled by LAVA, we wondered if we could not only use our lab for KernelCI.org, but also provide remote control of our boards to Free Electrons engineers so that they can access development boards from anywhere. lavabo was born from this idea and its goal is to allow full remote control of the boards as it is done in LAVA: interface with the serial port, control the power supply and provide files to the board using TFTP.

    The advantages of being able to access the boards remotely are obvious: allowing engineers working from home to work on their hardware platforms, avoid moving the boards out of the lab and back into the lab each time an engineer wants to do a test, etc.

    User’s perspective

    From a user’s point of view, lavabo is used through the eponymous command lavabo, which allows to:

    • List the boards and their status
      $ lavabo list
    • Reserve a board for lavabo usage, so that it is no longer used for CI jobs
      $ lavabo reserve am335x-boneblack_01
    • Upload a kernel image and Device Tree blob so that it can be accessed by the board through TFTP
      $ lavabo upload zImage am335x-boneblack.dtb
    • Connect to the serial port of the board
      $ lavabo serial am335x-boneblack_01
    • Reset the power of the board
      $ lavabo reset am335x-boneblack_01
    • Power off the board
      $ lavabo power-off am335x-boneblack_01
    • Release the board, so that it can once again be used for CI jobs
      $ lavabo release am335x-boneblack_01

    Overall architecture and implementation

    The following diagram summarizes the overall architecture of lavabo (components in green) and how it connects with existing components of the LAVA architecture.

    lavabo reuses LAVA tools and configuration files

    lavabo reuses LAVA tools and configuration files

    A client-server software

    lavabo follows the classical client-server model: the lavabo client is installed on the machines of users, while the lavabo server is hosted on the same machine as LAVA. The server-side of lavabo is responsible for calling the right tools directly on the server machine and making the right calls to LAVA’s API. It controls the boards and interacts with the LAVA instance to reserve and release a board.

    On the server machine, a specific Unix user is configured, through its .ssh/authorized_keys to automatically spawn the lavabo server program when someone connects. The lavabo client and server interact directly using their stdin/stdout, by exchanging JSON dictionaries. This interaction model has been inspired from the Attic backup program. Therefore, the lavabo server is not a background process that runs permanently like traditional daemons.

    Handling serial connection

    Exchanging JSON over SSH works fine to allow the lavabo client to provide instructions to the lavabo server, but it doesn’t work well to provide access to the serial ports of the boards. However, ser2net is already used by LAVA and provides a local telnet port for each serial port. lavabo simply uses SSH port-forwarding to redirect those telnet ports to local ports on the user’s machine.

    Different ways to connect to the serial

    Different ways to connect to the serial

    Interaction with LAVA

    To use a board outside of LAVA, we have to interact with LAVA to tell him the board cannot be used anymore. We therefore had to work with LAVA developers to add endpoints for putting online (release) and for putting offline (reserve) boards and an endpoint to get the current status of a board (busy, idle or offline) in LAVA’s API.

    These additions to the LAVA API are used by the lavabo server to make reserve and release boards, so that there is no conflict between the CI related jobs (such as the ones submitted by KernelCI.org) and the direct use of boards for remote development.

    Interaction with the boards

    Now that we know how the client and the server interact and also how the server communicates with LAVA, we need a way to know which boards are in the lab, on which port the serial connection of a board is exposed and what are the commands to control the board’s power supply. All this configuration has already been given to LAVA, so lavabo server simply reads the LAVA configuration files.

    The last requirement is to provide files to the board, such as kernel images, Device Tree blobs, etc. Indeed, from a network point of view, the boards are located in a different subnet not routed directly to the users machines. LAVA already has a directory accessible through TFTP from the boards which is one of the mechanisms used to serve files to boards. Therefore, the easiest and most obvious way is to send files from the client to the server and move the files to this directory, which we implemented using SFTP.

    User authentication

    Since the serial port cannot be shared among several sessions, it is essential to guarantee a board can only be used by one engineer at a time. In order to identify users, we have one SSH key per user in the .ssh/authorized_keys file on the server, each associated to a call to the lavabo-server program with a different username.

    This allows us to identify who is reserving/releasing the boards, and make sure that serial port access, or requests to power off or reset the boards are done by the user having reserved the board.

    For TFTP, the lavabo upload command automatically uploads files into a per-user sub-directory of the TFTP server. Therefore, when a file called zImage is uploaded, the board will access it over TFTP by downloading user/zImage.

    Availability and installation

    As you could guess from our love for FOSS, lavabo is released under the GNU GPLv2 license in a GitHub repository. Extensive documentation is available if you’re interested in installing lavabo. Of course, patches are welcome!

    by Quentin Schulz at May 22, 2017 07:25 AM

    May 09, 2017

    Free Electrons

    Eight channels audio on i.MX7 with PCM3168

    Toradex Colibri i.MX7Free Electrons engineer Alexandre Belloni recently worked on a custom carrier board for a Colibri iMX7 system-on-module from Toradex. This system-on-module obviously uses the i.MX7 ARM processor from Freescale/NXP.

    While the module includes an SGTL5000 codec, one of the requirements for that project was to handle up to eight audio channels. The SGTL5000 uses I²S and handles only two channels.

    I2S

    I2S timing diagram from the SGTL5000 datasheet

    Thankfully, the i.MX7 has multiple audio interfaces and one is fully available on the SODIMM connector of the Colibri iMX7. A TI PCM3168 was chosen for the carrier board and is connected to the second Synchronous Audio Interface (SAI2) of the i.MX7. This codec can handle up to 8 output channels and 6 input channels. It can take multiple formats as its input but TDM takes the smaller number of signals (4 signals: bit clock, word clock, data input and data output).


    TDM timing diagram from the PCM3168 datasheet

    The current Linux long term support version is 4.9 and was chosen for this project. It has support for both the i.MX7 SAI (sound/soc/fsl/fsl_sai.c) and the PCM3168 (sound/soc/codecs/pcm3168a.c). That’s two of the three components that are needed, the last one being the driver linking both by describing the topology of the “sound card”. In order to keep the custom code to the minimum, there is an existing generic driver called simple-card (sound/soc/generic/simple-card.c). It is always worth trying to use it unless something really specific prevents that. Using it was as simple as writing the following DT node:

            board_sound {
                    compatible = "simple-audio-card";
                    simple-audio-card,name = "imx7-pcm3168";
                    simple-audio-card,widgets =
                            "Speaker", "Channel1out",
                            "Speaker", "Channel2out",
                            "Speaker", "Channel3out",
                            "Speaker", "Channel4out",
                            "Microphone", "Channel1in",
                            "Microphone", "Channel2in",
                            "Microphone", "Channel3in",
                            "Microphone", "Channel4in";
                    simple-audio-card,routing =
                            "Channel1out", "AOUT1L",
                            "Channel2out", "AOUT1R",
                            "Channel3out", "AOUT2L",
                            "Channel4out", "AOUT2R",
                            "Channel1in", "AIN1L",
                            "Channel2in", "AIN1R",
                            "Channel3in", "AIN2L",
                            "Channel4in", "AIN2R";
    
                    simple-audio-card,dai-link@0 {
                            format = "left_j";
                            bitclock-master = &pcm3168_dac>;
                            frame-master = &pcm3168_dac>;
                            frame-inversion;
    
                            cpu {
                                    sound-dai = &sai2>;
                                    dai-tdm-slot-num = 8>;
                                    dai-tdm-slot-width = 32>;
                            };
    
                            pcm3168_dac: codec {
                                    sound-dai = &pcm3168 0>;
                                    clocks = &codec_osc>;
                            };
                    };
    
                    simple-audio-card,dai-link@2 {
                            format = "left_j";
                            bitclock-master = &pcm3168_adc>;
                            frame-master = &pcm3168_adc>;
    
                            cpu {
                                    sound-dai = &sai2>;
                                    dai-tdm-slot-num = 8>;
                                    dai-tdm-slot-width = 32>;
                            };
    
                            pcm3168_adc: codec {
                                    sound-dai = &pcm3168 1>;
                                    clocks = &codec_osc>;
                            };
                    };
            };

    There are multiple things of interest:

    • Only 4 input channels and 4 output channels are routed because the carrier board only had that wired.
    • There are two DAI links because the pcm3168 driver exposes inputs and outputs separately
    • As per the PCM3168 datasheet:
      • left justified mode is used
      • dai-tdm-slot-num is set to 8 even though only 4 are actually used
      • dai-tdm-slot-width is set to 32 because the codec takes 24-bit samples but requires 32 clocks per sample (this is solved later in userspace)
      • The codec is master which is usually best regarding clock accuracy, especially since the various SoMs on the market almost never expose the audio clock on the carrier board interface. Here, a crystal was used to clock the PCM3168.

    The PCM3168 codec is added under the ecspi3 node as that is where it is connected:

    &ecspi3 {
            pcm3168: codec@0 {
                    compatible = "ti,pcm3168a";
                    reg = 0>;
                    spi-max-frequency = 1000000>;
                    clocks = &codec_osc>;
                    clock-names = "scki";
                    #sound-dai-cells = 1>;
                    VDD1-supply = &reg_module_3v3>;
                    VDD2-supply = &reg_module_3v3>;
                    VCCAD1-supply = &reg_board_5v0>;
                    VCCAD2-supply = &reg_board_5v0>;
                    VCCDA1-supply = &reg_board_5v0>;
                    VCCDA2-supply = &reg_board_5v0>;
            };
    };
    

    #sound-dai-cells is what allows to select between the input and output interfaces.

    On top of that, multiple issues had to be fixed:

    Finally, an ALSA configuration file (/usr/share/alsa/cards/imx7-pcm3168.conf) was written to ensure samples sent to the card are in the proper format, S32_LE. 24-bit samples will simply have zeroes in the least significant byte. For 32-bit samples, the codec will properly ignore the least significant byte.
    Also this describes that the first subdevice is the playback (output) device and the second subdevice is the capture (input) device.

    imx7-pcm3168.pcm.default {
    	@args [ CARD ]
    	@args.CARD {
    		type string
    	}
    	type asym
    	playback.pcm {
    		type plug
    		slave {
    			pcm {
    				type hw
    				card $CARD
    				device 0
    			}
    			format S32_LE
    			rate 48000
    			channels 4
    		}
    	}
    	capture.pcm {
    		type plug
    		slave {
    			pcm {
    				type hw
    				card $CARD
    				device 1
    			}
    			format S32_LE
    			rate 48000
    			channels 4
    		}
    	}
    }

    On top of that, the dmix and dsnoop ALSA plugins can be used to separate channels.

    To conclude, this shows that it is possible to easily leverage existing code to integrate an audio codec in a design by simply writing a device tree snippet and maybe an ALSA configuration file if necessary.

    by Alexandre Belloni at May 09, 2017 08:16 AM

    May 04, 2017

    Free Electrons

    Feedback from the Netdev 2.1 conference

    At Free Electrons, we regularly work on networking topics as part of our Linux kernel contributions and thus we decided to attend our very first Netdev conference this year in Montreal. With the recent evolution of the network subsystem and its drivers capabilities, the conference was a very good opportunity to stay up-to-date, thanks to lots of interesting sessions.

    Eric Dumazet presenting “Busypolling next generation”

    The speakers and the Netdev committee did an impressive job by offering such a great schedule and the recorded talks are already available on the Netdev Youtube channel. We particularly liked a few of those talks.

    Distributed Switch Architecture – slidesvideo

    Andrew Lunn, Viven Didelot and Florian Fainelli presented DSA, the Distributed Switch Architecture, by giving an overview of what DSA is and by then presenting its design. They completed their talk by discussing the future of this subsystem.

    DSA in one slide

    The goal of the DSA subsystem is to support Ethernet switches connected to the CPU through an Ethernet controller. The distributed part comes from the possibility to have multiple switches connected together through dedicated ports. DSA was introduced nearly 10 years ago but was mostly quiet and only recently came back to life thanks to contributions made by the authors of this talk, its maintainers.

    The main idea of DSA is to reuse the available internal representations and tools to describe and configure the switches. Ports are represented as Linux network interfaces to allow the userspace to configure them using common tools, the Linux bridging concept is used for interface bridging and the Linux bonding concept for port trunks. A switch handled by DSA is not seen as a special device with its own control interface but rather as an hardware accelerator for specific networking capabilities.

    DSA has its own data plane where the switch ports are slave interfaces and the Ethernet controller connected to the SoC a master one. Tagging protocols are used to direct the frames to a specific port when coming from the SoC, as well as when received by the switch. For example, the RX path has an extra check after netif_receive_skb() so that if DSA is used, the frame can be tagged and reinjected into the network stack RX flow.

    Finally, they talked about the relationship between DSA and Switchdev, and cross-chip configuration for interconnected switches. They also exposed the upcoming changes in DSA as well as long term goals.

    Memory bottlenecks – slides

    As part of the network performances workshop, Jesper Dangaard Brouer presented memory bottlenecks in the allocators caused by specific network workloads, and how to deal with them. The SLAB/SLUB baseline performances are found to be too slow, particularly when using XDP. A way from a driver to solve this issue is to implement a custom page recycling mechanism and that’s what all high-speed drivers do. He then displayed some data to show why this mechanism is needed when targeting the 10G network budget.

    Jesper is working on a generic solution called page pool and sent a first RFC at the end of 2016. As mentioned in the cover letter, it’s still not ready for inclusion and was only sent for early reviews. He also made a small overview of his implementation.

    DDOS countermeasures with XDP – slides #1slides #2 – video #1video #2

    These two talks were given by Gilberto Bertin from Cloudflare and Martin Lau from Facebook. While they were not talking about device driver implementation or improvements in the network stack directly related to what we do at Free Electrons, it was nice to see how XDP is used in production.

    XDP, the eXpress Data Path, provides a programmable data path at the lowest point of the network stack by processing RX packets directly out of the drivers’ RX ring queues. It’s quite new and is an answer to lots of userspace based solutions such as DPDK. Gilberto andMartin showed excellent results, confirming the usefulness of XDP.

    From a driver point of view, some changes are required to support it. RX hooks must be added as well as some API changes and the driver’s memory model often needs to be updated. So far, in v4.10, only a few drivers are supporting XDP.

    XDP MythBusters – slides – video

    David S. Miller, the maintainer of the Linux networking stack and drivers, did an interesting keynote about XDP and eBPF. The eXpress Data Path clearly was the hot topic of this Netdev 2.1 conference with lots of talks related to the concept and David did a good overview of what XDP is, its purposes, advantages and limitations. He also quickly covered eBPF, the extended Berkeley Packet Filters, which is used in XDP to filter packets.

    This presentation was a comprehensive introduction to the concepts introduced by XDP and its different use cases.

    Conclusion

    Netdev 2.1 was an excellent experience for us. The conference was well organized, the single track format allowed us to see every session on the schedule, and meeting with attendees and speakers was easy. The content was highly technical and an excellent opportunity to stay up-to-date with the latest changes of the networking subsystem in the kernel. The conference hosted both talks about in-kernel topics and their use in userspace, which we think is a very good approach to not focus only on the kernel side but also to be aware of the users needs and their use cases.

    by Antoine Ténart at May 04, 2017 08:13 AM

    May 02, 2017

    Harald Welte

    OsmoDevCon 2017 Review

    After the public user-oriented OsmoCon 2017, we also recently had the 6th incarnation of our annual contributors-only Osmocom Developer Conference: The OsmoDevCon 2017.

    This is a much smaller group, typically about 20 people, and is limited to actual developers who have a past record of contributing to any of the many Osmocom projects.

    We had a large number of presentation and discussions. In fact, so large that the schedule of talks extended from 10am to midnight on some days. While this is great, it also means that there was definitely too little time for more informal conversations, chatting or even actual work on code.

    We also have such a wide range of topics and scope inside Osmocom, that the traditional ad-hoch scheduling approach no longer seems to be working as it used to. Not everyone is interested in (or has time for) all the topics, so we should group them according to their topic/subject on a given day or half-day. This will enable people to attend only those days that are relevant to them, and spend the remaining day in an adjacent room hacking away on code.

    It's sad that we only have OsmoDevCon once per year. Maybe that's actually also something to think about. Rather than having 4 days once per year, maybe have two weekends per year.

    Always in motion the future is.

    by Harald Welte at May 02, 2017 10:00 PM

    Overhyped Docker

    Overhyped Docker missing the most basic features

    I've always been extremely skeptical of suddenly emerging over-hyped technologies, particularly if they advertise to solve problems by adding yet another layer to systems that are already sufficiently complex themselves.

    There are of course many issues with containers, ranging from replicated system libraries and the basic underlying statement that you're giving up on the system packet manager to properly deal with dependencies.

    I'm also highly skeptical of FOSS projects that are primarily driven by one (VC funded?) company. Especially if their offering includes a so-called cloud service which they can stop to operate at any given point in time, or (more realistically) first get everybody to use and then start charging for.

    But well, despite all the bad things I read about it over the years, on one day in May 2017 I finally thought let's give it a try. My problem to solve as a test balloon is fairly simple.

    My basic use case

    The plan is to start OsmoSTP, the m3ua-testtool and the sua-testtool, which both connect to OsmoSTP. By running this setup inside containers and inside an internal network, we could then execute the entire testsuite e.g. during jenkins test without having IP address or port number conflicts. It could even run multiple times in parallel on one buildhost, verifying different patches as part of the continuous integration setup.

    This application is not so complex. All it needs is three containers, an internal network and some connections in between. Should be a piece of cake, right?

    But enter the world of buzzword-fueled web-4000.0 software-defined virtualised and orchestrated container NFW + SDN vodoo: It turns out to be impossible, at least not with the preferred tools they advertise.

    Dockerfiles

    The part that worked relatively easily was writing a few Dockerfiles to build the actual containers. All based on debian:jessie from the library.

    As m3ua-testsuite is written in guile, and needs to build some guile plugin/extension, I had to actually include guile-2.0-dev and other packages in the container, making it a bit bloated.

    I couldn't immediately find a nice example Dockerfile recipe that would allow me to build stuff from source outside of the container, and then install the resulting binaries into the container. This seems to be a somewhat weak spot, where more support/infrastructure would be helpful. I guess the idea is that you simply install applications via package feeds and apt-get. But I digress.

    So after some tinkering, I ended up with three docker containers:

    • one running OsmoSTP
    • one running m3ua-testtool
    • one running sua-testtool

    I also managed to create an internal bridged network between the containers, so the containers could talk to one another.

    However, I have to manually start each of the containers with ugly long command line arguments, such as docker run --network sigtran --ip 172.18.0.200 -it osmo-stp-master. This is of course sub-optimal, and what Docker Services + Stacks should resolve.

    Services + Stacks

    The idea seems good: A service defines how a given container is run, and a stack defines multiple containers and their relation to each other. So it should be simple to define a stack with three services, right?

    Well, it turns out that it is not. Docker documents that you can configure a static ipv4_address [1] for each service/container, but it seems related configuration statements are simply silently ignored/discarded [2], [3], [4].

    This seems to be related that for some strange reason stacks can (at least in later versions of docker) only use overlay type networks, rather than the much simpler bridge networks. And while bridge networks appear to support static IP address allocations, overlay apparently doesn't.

    I still have a hard time grasping that something that considers itself a serious product for production use (by a company with estimated value over a billion USD, not by a few hobbyists) that has no support for running containers on static IP addresses. that. How many applications out there have I seen that require static IP address configuration? How much simpler do setups get, if you don't have to rely on things like dynamic DNS updates (or DNS availability at all)?

    So I'm stuck with having to manually configure the network between my containers, and manually starting them by clumsy shell scripts, rather than having a proper abstraction for all of that. Well done :/

    Exposing Ports

    Unrelated to all of the above: If you run some software inside containers, you will pretty soon want to expose some network services from containers. This should also be the most basic task on the planet.

    However, it seems that the creators of docker live in the early 1980ies, where only TCP and UDP transport protocols existed. They seem to have missed that by the late 1990ies to early 2000s, protocols like SCTP or DCCP were invented.

    But yet, in 2017, Docker chooses to

    Now some of the readers may think 'who uses SCTP anyway'. I will give you a straight answer: Everyone who has a mobile phone uses SCTP. This is due to the fact that pretty much all the connections inside cellular networks (at least for 3G/4G networks, and in reality also for many 2G networks) are using SCTP as underlying transport protocol, from the radio access network into the core network. So every time you switch your phone on, or do anything with it, you are using SCTP. Not on your phone itself, but by all the systems that form the network that you're using. And with the drive to C-RAN, NFV, SDN and all the other buzzwords also appearing in the Cellular Telecom field, people should actually worry about it, if they want to be a part of the software stack that is used in future cellular telecom systems.

    Summary

    After spending the better part of a day to do something that seemed like the most basic use case for running three networked containers using Docker, I'm back to step one: Most likely inventing some custom scripts based on unshare to run my three test programs in a separate network namespace for isolated execution of test suite execution as part of a Jenkins CI setup :/

    It's also clear that Docker apparently don't care much about playing a role in the Cellular Telecom world, which is increasingly moving away from proprietary and hardware-based systems (like STPs) to virtualised, software-based systems.

    [1]https://docs.docker.com/compose/compose-file/#ipv4address-ipv6address
    [2]https://forums.docker.com/t/docker-swarm-1-13-static-ips-for-containers/28060
    [3]https://github.com/moby/moby/issues/31860
    [4]https://github.com/moby/moby/issues/24170

    by Harald Welte at May 02, 2017 10:00 PM

    Free Electrons

    Linux 4.11, Free Electrons contributions

    Linus Torvalds has released this Sunday Linux 4.11. For an overview of the new features provided by this new release, one can read the coverage from LWN: part 1, part 2 and part 3. The KernelNewbies site also has a detailed summary of the new features.

    With 137 patches contributed, Free Electrons is the 18th contributing company according to the Kernel Patch Statistics. Free Electrons engineer Maxime Ripard appears in the list of top contributors by changed lines in the LWN statistics.

    Our most important contributions to this release have been:

    • Support for Atmel platforms
      • Alexandre Belloni improved suspend/resume support for the Atmel watchdog driver, I2C controller driver and UART controller driver. This is part of a larger effort to upstream support for the backup mode of the Atmel SAMA5D2 SoC.
      • Alexandre Belloni also improved the at91-poweroff driver to properly shutdown LPDDR memories.
      • Boris Brezillon contributed a fix for the Atmel HLCDC display controller driver, as well as fixes for the atmel-ebi driver.
    • Support for Allwinner platforms
      • Boris Brezillon contributed a number of improvements to the sunxi-nand driver.
      • Mylène Josserand contributed a new driver for the digital audio codec on the Allwinner sun8i SoC, as well a the corresponding Device Tree changes and related fixes. Thanks to this driver, Mylène enabled audio support on the R16 Parrot and A33 Sinlinx boards.
      • Maxime Ripard contributed numerous improvements to the sunxi-mmc MMC controller driver, to support higher data rates, especially for the Allwinner A64.
      • Maxime Ripard contributed official Device Tree bindings for the ARM Mali GPU, which allows the GPU to be described in the Device Tree of the upstream kernel, even if the ARM kernel driver for the Mali will never be merged upstream.
      • Maxime Ripard contributed a number of fixes for the rtc-sun6i driver.
      • Maxime Ripard enabled display support on the A33 Sinlinx board, by contributing a panel driver and the necessary Device Tree changes.
      • Maxime Ripard continued his clean-up effort, by converting the GR8 and sun5i clock drivers to the sunxi-ng clock infrastructure, and converting the sun5i pinctrl driver to the new model.
      • Quentin Schulz added a power supply driver for the AXP20X and AXP22X PMICs used on numerous Allwinner platforms, as well as numerous Device Tree changes to enable it on the R16 Parrot and A33 Sinlinx boards.
    • Support for Marvell platforms
      • Grégory Clement added support for the RTC found in the Marvell Armada 7K and 8K SoCs.
      • Grégory Clement added support for the Marvell 88E6141 and 88E6341 Ethernet switches, which are used in the Armada 3700 based EspressoBin development board.
      • Romain Perier enabled the I2C controller, SPI controller and Ethernet switch on the EspressoBin, by contributing Device Tree changes.
      • Thomas Petazzoni contributed a number of fixes to the OMAP hwrng driver, which turns out to also be used on the Marvell 7K/8K platforms for their HW random number generator.
      • Thomas Petazzoni contributed a number of patches for the mvpp2 Ethernet controller driver, preparing the future addition of PPv2.2 support to the driver. The mvpp2 driver currently only supports PPv2.1, the Ethernet controller used on the Marvell Armada 375, and we are working on extending it to support PPv2.2, the Ethernet controller used on the Marvell Armada 7K/8K. PPv2.2 support is scheduled to be merged in 4.12.
    • Support for RaspberryPi platforms
      • Boris Brezillon contributed Device Tree changes to enable the VEC (Video Encoder) on all bcm283x platforms. Boris had previously contributed the driver for the VEC.

    In addition to our direct contributions, a number of Free Electrons engineers are also maintainers of various subsystems in the Linux kernel. As part of this maintenance role:

    • Maxime Ripard, co-maintainer of the Allwinner ARM platform, reviewed and merged 85 patches from contributors
    • Alexandre Belloni, maintainer of the RTC subsystem and co-maintainer of the Atmel ARM platform, reviewed and merged 60 patches from contributors
    • Grégory Clement, co-maintainer of the Marvell ARM platform, reviewed and merged 42 patches from contributors
    • Boris Brezillon, maintainer of the MTD NAND subsystem, reviewed and merged 8 patches from contributors

    Here is the detailed list of contributions, commit per commit:

    by Thomas Petazzoni at May 02, 2017 12:23 PM

    May 01, 2017

    Harald Welte

    Book on Practical GPL Compliance

    My former gpl-violations.org colleague Armijn Hemel and Shane Coughlan (former coordinator of the FSFE Legal Network) have written a book on practical GPL compliance issues.

    I've read through it (in the bath tub of course, what better place to read technical literature), and I can agree wholeheartedly with its contents. For those who have been involved in GPL compliance engineering there shouldn't be much new - but for the vast majority of developers out there who have had little exposure to the bread-and-butter work of providing complete an corresponding source code, it makes an excellent introductory text.

    The book focuses on compliance with GPLv2, which is probably not too surprising given that it's published by the Linux foundation, and Linux being GPLv2.

    You can download an electronic copy of the book from https://www.linuxfoundation.org/news-media/research/practical-gpl-compliance

    Given the subject matter is Free Software, and the book is written by long-time community members, I cannot help to notice a bit of a surprise about the fact that the book is released in classic copyright under All rights reserved with no freedom to the user.

    Considering the sensitive legal topics touched, I can understand the possible motivation by the authors to not permit derivative works. But then, there still are licenses such as CC-BY-ND which prevent derivative works but still permit users to make and distribute copies of the work itself. I've made that recommendation / request to Shane, let's see if they can arrange for some more freedom for their readers.

    by Harald Welte at May 01, 2017 10:00 PM

    April 30, 2017

    Harald Welte

    OsmoCon 2017 Review

    It's already one week past the event, so I really have to sit down and write some rewview on the first public Osmocom Conference ever: OsmoCon 2017.

    The event was a huge success, by all accounts.

    • We've not only been sold out, but we also had to turn down some last minute registrations due to the venue being beyond capacity (60 seats). People traveled from Japan, India, the US, Mexico and many other places to attend.
    • We've had an amazing audience ranging from commercial operators to community cellular operators to professional developers doing work relate to osmocom, academia, IT security crowds and last but not least enthusiasts/hobbyists, with whom the project[s] started.
    • I've received exclusively positive feedback from many attendees
    • We've had a great programme. Some part of it was of introductory nature and probably not too interesting if you've been in Osmocom for a few years. However, the work on 3G as well as the current roadmap was probably not as widely known yet. Also, I really loved to see Roch's talk about Running a commercial cellular network with Osmocom software as well as the talk on Facebook's OpenCellular BTS hardware and the Community Cellular Manager.
    • We have very professional live streaming + video recordings courtesy of the C3VOC team. Thanks a lot for your support and for having the video recordings of all talks online already at the next day after the event.

    We also received some requests for improvements, many of which we will hopefully consider before the next Osmocom Conference:

    • have a multiple day event. Particularly if you're traveling long-distance, it is a lot of overhead for a single-day event. We of course fully understand that. On the other hand, it was the first Osmocom Conference, and hence it was a test balloon where it was initially unclear if we'll be able to get a reasonable number of attendees interested at all, or not. And organizing an event with venue and talks for multiple days if in the end only 10 people attend would have been a lot of effort and financial risk. But now that we know there are interested folks, we can definitely think of a multiple day event next time
    • Signs indicating venue details on the last meters. I agree, this cold have been better. The address of the venue was published, but we could have had some signs/posters at the door pointing you to the right meeting room inside the venue. Sorry for that.
    • Better internet connectivity. This is a double-edged sword. Of course we want our audience to be primarily focused on the talks and not distracted :P I would hope that most people are able to survive a one day event without good connectivity, but for sure we will have to improve in case of a multiple-day event in the future

    In terms of my requests to the attendees, I only have one

    • Participate in the discussions on the schedule/programme while it is still possible to influence it. When we started to put together the programme, I posted about it on the openbsc mailing list and invited feedback. Still, most people seem to have missed the time window during which talks could have been submitted and the schedule still influenced before finalizing it
    • Register in time. We have had almost no registrations until about two weeks ahead of the event (and I was considering to cancel it), and then suddenly were sold out in the week ahead of the event. We've had people who first booked their tickets, only to learn that the tickets were sold out. I guess we will introduce early bird pricing and add a very expensive last minute ticket option next year in order to increase motivation to register early and thus give us flexibility regarding venue planning.

    Thanks again to everyone involved in OsmoCon 2017!

    Ok, now, all of you who missed the event: Go to https://media.ccc.de/c/osmocon17 and check out the recordings. Have fun!

    by Harald Welte at April 30, 2017 10:00 PM

    April 28, 2017

    Andrew Zonenberg, Silicon Exposed

    Quest for camp stove fuel

    For those of you who aren't keeping up with my occasional Twitter/Facebook posts on the subject, I volunteer with a local search and rescue unit. This means that a few times a month I have to grab my gear and run out into the woods on zero notice to find an injured hiker, locate an elderly person with Alzheimer's, or whatever the emergency du jour is.

    Since I don't have time to grab fresh food on my way out the door when duty calls, I keep my pack and load-bearing vest stocked with shelf-stable foods like energy bars and surplus military rations. Many missions are short and intense, leaving me no time to eat anything but finger-food items (Clif bars and First Strike Ration sandwiches are my favorites) kept in a vest pocket.

    My SAR vest. Weighs about 17 pounds / 7.7 kg once the Camelbak bladder is added.
    On the other hand, during longer missions there may be opportunities to make hot food while waiting for a medevac helicopter, ground team with stretcher, etc - and of course there's plenty of time to cook a hot dinner during training weekends. Besides being a convenience, hot food and drink helps us (and the subject) avoid hypothermia so it can be a literal life-saver.

    I've been using MRE chemical heaters for this, because they're small, lightweight (20 g / 0.7 oz each), and not too pricey (about $1 each from surplus dealers). Their major flaw is that they don't get all that hot, so during cold weather it's hard to get your food more than lukewarm.

    I've used many kinds of camp stoves (propane and white gas primarily) over the course of my camping, but didn't own one small enough to use for SAR. My full 48-hour gear loadout (including water) weighs around 45 pounds / 20 kg, and I really didn't want to add much more to this. The MSR Whisperlite, for example, weighs in at 430 g / 15.2 oz for the stove, fuel pump, and wind shield. Add to this 150 g / 5.25 oz for the fuel bottle, a pot to cook in, and the fuel itself and you're looking at close to 1 kg / 2 pounds all told.

    I have an aluminum camp frying pan that, including lid, weighs 121 g / 4.3 oz. It seemed hard to get much lighter for something large enough that you could squeeze an MRE entree into, so I kept it.

    After a bit of browsing in the local Wal-Mart, I found a tiny sheet metal folding stove that weighed 112 g / 3.98 oz empty. It's designed to burn pellets of hexamine fuel.

    The stove. Ignore the aluminum foil, it was there from a previous experiment.
    In my testing it worked pretty well. One pellet brought 250 ml of water from 10C to boiling in six minutes, and held it at a boil for a minute before burning out. The fuel burned fairly cleanly and didn't leave that much soot on the pot either, which was nice.

    What's not so nice, however, was the fuel. According to the MSDS, hexamine decomposes upon heating or contact with skin into formaldehyde, which is toxic and carcinogenic. Combustion products include such tasty substances as hydrogen cyanide and ammonia. This really didn't seem like something that I wanted to handle, or burn, in close proximity to food! Thus began my quest for a safer alternative.

    My first thought was to use tea light candles, since I already had a case of a hundred for use as fire starters. In my testing, one tea light was able to heat a pot of water from 10C to 30C in a whopping 21 minutes before starting to reach an equilibrium where the pot lost heat as fast as it gained it. I continued the test out to 34 minutes, at which point it was a toasty 36C.

    The stove was big enough to fit more than one tea light, so the obvious next step was to put six of them in a 3x2 grid. This heated significantly more, at the 36-minute mark my water measured a respectable 78C.

    I figured I was on the right track, but needed to burn more wax per unit time. Some rough calculations suggested that a brick of paraffin wax the size of the stove and about as thick as a tea light contained 1.5 kWh of energy, and would output about 35 W of heat per wick. Assuming 25% energy transfer efficiency, which seemed reasonable based on the temperature data I had measured earlier, I needed to put out around 675 W to bring my pot to a boil in ten minutes. This came out to approximately 20 candle wicks.

    I started out by folding a tray out of heavy duty aluminum foil, and reinforcing it on the outside with aluminum foil duct tape. I then bought a pack of tea light wicks on Amazon and attached them to the tray with double-sided tape.
    Giant 20-wicked candle before adding wax
    I made a water bath on my hot plate and melted a bunch of tea lights in a beaker. I wasn't in the mood to get spattered with hot wax so I wore long-sleeved clothes and a face shield. I was pretty sure that the water bath wouldn't get anywhere near the ignition point of the wax but did the work outside on a concrete patio and had a CO2 fire extinguisher on standby just in case.

    Melting wax. Safety first, everyone!
    The resulting behemoth of a candle actually looked pretty nice!
    20-wick, 700W thermal output candle with tea lights for scale
    After I was done and the wax had solidified I put the candle in my stove and lit it off. It took a while to get started (a light breeze kept blowing out one wick or another and I used quite a few matches to get them all lit), but after a while I had a solid flame going. At the six-minute mark my water had reached 37C.

    A few minutes later, disaster struck! The pool of molten wax reached the flash point and ignited across the whole surface. At this point I had a massive flame - my pot went from 48 to 82C in two minutes! This translates to 2.6 kW assuming 100% energy transfer efficiency, so actual power output was probably upwards of 5 kW.

    I removed the pot (using welding gloves since the flames were licking up the handle) and grabbed a photo of the fireball before thinking about how to extinguish the fire.

    Pretty sure this isn't what a stove is supposed to look like
    Since I was outside on a non-flammable surface the fire wasn't an immediate safety hazard, but I wanted to put it out non-destructively to preserve evidence for failure analysis. I opted to smother it with a giant candle snuffer that I rapidly folded out of heavy-duty aluminum foil.

    The carnage after the fire was extinguished. Note the discolored wax!
    It took me a while to clean up the mess - the giant candle had turned tan from incomplete combustion. It had also sprung a leak at some point, spilling a bit of wax out onto my patio.

    On top of that, my pot was coal-black from all of the soot the super-rich flame was putting out. My wife wouldn't let it anywhere near the sink so I scrubbed it as best I could in the bathtub, then spent probably 20 minutes scrubbing all of the gray stains off the tub itself.

    In order to avoid the time-consuming casting of wax, my next test used a slug of wax from a tea light that I drilled holes in, then inserted four wicks. I covered the top of the candle with aluminum foil tape to reflect heat back up at the pot, in a bid to increase efficiency and keep the melt puddle below the flash point.

    Quad-wick tea light
    This performed pretty well in my test. It got my pot up to 35C at the 12-minute mark, which was right about where I expected based on the x1 and x6 candle tests, and didn't flash over.

    The obvious next step was to make five of them and see if this would work any better. It ignited more easily than the "brick" candle, and reached 83C at the 6-minute mark. Before T+ 7 minutes, however, the glue on the tape had failed from the heat, and the wax flashed. By the time I got the pot out of harm's way the water was boiling and it was covered in soot (again).

    This time, it was a little bit breezier and my snuffer failed to exclude enough air to extinguish the flames. I ended up having to blast it with the CO2 extinguisher I had ready for just this situation. It wasn't hard to put out and I only used about two of the ten pounds of gas. (Ironically, I had planned to take the extinguisher in to get serviced the next morning because it was almost due for annual preventive maintenance. I ended up needing a recharge too...)

    After cleaning off my pot and stove, and scraping some of the spilled wax off my driveway, it was back to the drawing board. I thought about other potential fuels I had lying around, and several obvious options came to mind.

    Testing booze for flammability
    I'm not a big drinker but houseguests have resulted in me having a few bottles of liquor around so I tested it out. Jack didn't burn at all, Captain Morgan white rum burned fitfully and left a sugary residue without putting out much heat. 100-proof vodka left a bit of starchy residue and was tricky to light.

    A tea light cup full of 99% isopropyl alcohol brought my pot to 75C in five minutes before burning out, but was filthy and left soot everywhere. Hand sanitizer (about 60% ethanol) burned cleanly, but slower and cooler due to the water content - peak temperature of 54C and 12 minute burn time.

    Ethanol seemed like a viable fuel if I could get it up to a higher concentration. I wanted to avoid liquid fuels due to difficulty of handling and the risk of spills, but a thick gel that didn't spill easily looked like a good option.

    After a bit of research I discovered that calcium acetate (a salt of acetic acid) was very soluble in water, but not in alcohols. When a saturated solution of it in water is added to an alcohol it forms a stiff gel, commonly referred to as a "California snowball" because it burns and has a consistency like wet snow. I don't have any photos of my test handy, but here's a video from somebody else that shows it off nicely.



    Two tea light cups full of the stuff brought my pot of water to a boil in 8 minutes, and held it there until burning out just before the 13-minute mark. I also tried boiling a FSR sandwich packet in a half-inch or so of water, and it was deliciously warm by the end. This seemed like a pretty good fuel!


    Testing the calcium acetate fuel. I put a lid on the pot after taking this pic.

    I filled two film-canister type containers with the calcium acetate + ethanol gel fuel and left it in my SAR pack. As luck would have it, I spent the next day looking for a missing hiker so it spent quite a while bouncing around driving on dirt roads and hiking.

    When I got home I was disappointed to see clear liquid inside the bag that my stove and fuel were stored in. I opened the canisters only to find a thin whitish liquid instead of a stiff gel.

    It seemed that the calcium acetate gel was not very stable, and over time the calcium acetate particles would precipitate out and the solution would revert to a liquid state. This clearly would not do.

    Hand sanitizer seemed like a pretty good fuel other than being underpowered and perfumed, so I went to the grocery store and started looking at ingredient lists. They all seemed pretty similar - ethanol, water, aloe and other moisturizers, perfumes, maybe colorants, and a thickener. The thickener was typically either hydroxyethyl cellulose or a carbomer.

    A few minutes on Amazon turned up a bag of Carbomer 940, a polyvinyl carboxy polymer cross-linked with esters of pentaerythritol. It's supposed to produce a viscosity of 45,000 to 70,000 CPS when added to water at 0.5% by weight. I also ordered a second bottle of Reagent Alcohol (90% ethanol / 5% methanol / 5% isopropanol with no bittering agents, ketones, or non-volatile ingredients) since my other one was pretty low after the calcium acetate failure.

    Carbomer 940 is fairly acidic (pH 2.7 - 3.3 at 0.5% concentration) in its pure form and gel when neutral or alkaline, so it needs to be neutralized. The recommended base for alcohol-based gels was triethanolamine, so I picked up a bottle of that too.


    Preparing to make carbomer-alcohol fuel gel

    I made a 50% alcohol-water solution and added an 0.5% mass of carbomer. It didn't seem to fully dissolve, leaving a bunch of goopy chunks in the beaker.


    Incompletely dissolved Carbomer 940 in 50/50 water/alcohol
    I left it overnight to dissolve, blended it more, and then filtered off any big clumps with a coffee filter. I then added a few drops of triethanolamine, at which point the solution immediately turned cloudy. Upon blending, a rubbery white substance preciptated out of solution and stuck to my stick blender and the sidewalls of the beaker. This was not supposed to happen!


    Rubbery goop on the blender head
    Precipitate at the bottom of the beaker

    I tried everything I could think of - diluting the triethanolamine and adding it slowly to reduce sudden pH changes, lowering the alcohol concentration, and even letting the carbomer sit in solution for a few days before adding the triethanolamine. Nothing worked.

    I went back to square one and started reading more papers and watching process demonstration videos from the manufacturer. Eventually I noticed one source that suggested increasing the pH of the water to about 8 *before* adding the carbomer. This worked and gave a beautiful clear gel!

    After a bit of tinkering I found a good process: Starting with 100 ml of water, titrate to pH 8 with triethanolamine. Add 1 g of carbomer powder and blend until fully gelled. Add 300 ml of reagent alcohol a bit at a time, mixing thoroughly after each addition. About halfway through adding the alcohol the gel started to get pretty runny so I mixed in a few more drops of triethanolamine and another 500 mg of carbomer powder before mixing in the rest of the alcohol. I had only a little more alcohol left in the bottle (maybe 50 ml) so I stirred that in without bothering to measure.

    The resulting gel was quite stiff and held its shape for a little while after pouring, but could still be transferred between containers without muich difficulty.


    Tea light can full of my final fuel
    I left the beaker of fuel in my garage for several days and shook it around a bit, but saw no evidence of degradation. Since it's basically just turbo-strength hand sanitizer (~78% instead of the usual 30-60%) without all of the perfumes and moisturizers, it should be pretty stable. I had no trouble igniting it down to 10C ambient temperatures, but may find it necessary to mix in some acetone or other low-flash-point fuel to light it reliably in the winter.

    The final batch of fuel filled two polypropylene specimen jars perfectly with just a little bit left over for a cooking test.


    One of my two fuel jars
    One tea light canister held 10.7 g / 0.38 oz of fuel, and I typically use two at a time, so 21.4 / 0.76 oz. One jar thus holds enough fuel for about five cook sessions, which is more than I'd ever need for a SAR mission or weekend camping trip. The final weight of my entire cooking system (stove, one fuel jar, tea light cans, and pot) comes out to 408 g / 14.41 oz, or a bit less than an empty Whisperlite stove (not counting the pot, fuel tank, or fuel)!

    The only thing left was to try cooking on it. I squeezed a bacon-cheddar FSR sandwich into my pot, added a bit of water, and put it on top of the stove with two candle cups of fuel.


    Nice clean blue flame, barely visible
    By the six-minute mark the water was boiling away merrily and a cloud of steam was coming up around the edge of the lid. I took the pot off around 8 minutes and removed my snack.

    Munching on my sandwich. You can't tell in this lighting, but the stove is still burning.
    For those of you who haven't eaten First Strike Rations, the sandwiches in them are kind of like Hot Pockets or Toaster Strudels, except with a very thick and dense bread rather than a fluffy, flaky one. The fats in the bread are solid at room temperature and liquefy once it gets warm. This significantly softens the texture of the bread and makes it taste a lot better, so reaching this point is generally the primary goal when cooking one.

    My sandwich was firmly over that line and tasted very good (for Army food baked two years ago). The bacon could have been a bit warmer, but the stove kept on burning until a bit after the ten-minute mark so I could easily have left it in the boiling water for another two minutes and made it even hotter.

    Once I was done eating it was time to clean up. The stove had no visible dirt (beyond what was there from my previous experiments), and the tea light canisters were clean and fairly free of soot except in one or two spots around the edges. Almost no goopy residue was left behind.

    Stove after the cook test
    The pot was quite clean as well, with no black soot and only a very thin film of discoloration that was thin enough to leave colored interference fringes. Some of this was left over from previous testing, so if this test had been run on a virgin pot there'd be even less residue.

    Bottom of the pot after the cook test


    Overall, it was a long journey with many false steps, but I now have the ability to cook for myself over a weekend trip in less than a pound of weight, so I'm pretty happy. 

    EDIT: A few people have asked to see the raw data from my temperature-vs-time cook tests, so here it is.

    Raw data (graph 1)
     
    Raw data (graph 2)

    by Andrew Zonenberg (noreply@blogger.com) at April 28, 2017 08:37 PM

    April 26, 2017

    Bunnie Studios

    Name that Ware, April 2017

    The Ware for April 2017 is shown below.

    This is a guest ware, but the contributor shall remain anonymous per request. Thank you for the contribution, you know who you are!

    by bunnie at April 26, 2017 05:46 PM

    Winner, Name that Ware March 2017

    The ware for March 2017 seems to be a Schneider ATV61 industrial variable speed drive controller. As rasz_pl pointed out, I left the sticker unredacted. I had a misgiving about hiding it fearing the ware would be unguessable, but leaving it in made it perhaps a bit too easy. Prize goes to rasz_pl for being the first to guess, email me for your prize!

    by bunnie at April 26, 2017 05:46 PM

    April 25, 2017

    Altus Metrum

    TeleMini3

    TeleMini V3.0 Dual-deploy altimeter with telemetry now available

    TeleMini v3.0 is an update to our original TeleMini v1.0 flight computer. It is a miniature (1/2 inch by 1.7 inch) dual-deploy flight computer with data logging and radio telemetry. Small enough to fit comfortably in an 18mm tube, this powerful package does everything you need on a single board:

    • 512kB on-board data logging memory, compared with 5kB in v1.

    • 40mW, 70cm ham-band digital transceiver for in-flight telemetry and on-the-ground configuration, compared to 10mW in v1.

    • Transmitted telemetry includes altitude, speed, acceleration, flight state, igniter continutity, temperature and battery voltage. Monitor the state of the rocket before, during and after flight.

    • Radio direction finding beacon transmitted during and after flight. This beacon can be received with a regular 70cm Amateur radio receiver.

    • Barometer accurate to 100k' MSL. Reliable apogee detection, independent of flight path. Barometric data recorded on-board during flight. The v1 boards could only fly to 45k'.

    • Dual-deploy with adjustable apogee delay and main altitude. Fires standard e-matches and Q2G2 igniters.

    • 0.5” x 1.7”. Fits easily in an 18mm tube. This is slightly longer than the v1 boards to provide room for two extra mounting holes past the pyro screw terminals.

    • Uses rechargeable Lithium Polymer battery technology. All-day power in a small and light-weight package.

    • Learn more at http://www.altusmetrum.org/TeleMini/

    • Purchase these at http://shop.gag.com/home-page/telemini-v3.html

    I don't have anything in these images to show just how tiny this board is—but the spacing between the screw terminals is 2.54mm (0.1in), and the whole board is only 13mm wide (1/2in).

    This was a fun board to design. As you might guess from the version number, we made a couple prototypes of a version 2 using the same CC1111 SoC/radio part as version 1 but in the EasyMini form factor (0.8 by 1.5 inches). Feed back from existing users indicated that bigger wasn't better in this case, so we shelved that design.

    With the availability of the STM32F042 ARM Cortex-M0 part in a 4mm square package, I was able to pack that, the higher power CC1200 radio part, a 512kB memory part and a beeper into the same space as the original TeleMini version 1 board. There is USB on the board, but it's only on some tiny holes, along with the cortex SWD debugging connection. I may make some kind of jig to gain access to that for configuration, data download and reprogramming.

    For those interested in an even smaller option, you could remove the screw terminals and battery connector and directly wire to the board, and replace the beeper with a shorter version. You could even cut the rear mounting holes off to make the board shorter; there are no components in that part of the board.

    by keithp's rocket blog at April 25, 2017 04:01 PM

    April 18, 2017

    Open Hardware Repository

    White Rabbit - 18-04-17: clean that fibre and SFP!

    The White Rabbit team at CERN organised a short course about fibre-optic cleaning and inspection.

    A special fibre inspection microscope that automatically analyses the image to decide if a cable or SFP passes or fails the norms was demonstrated.
    The images of some of the often-used cables and SFP modules that we picked from the development lab, showed clearly traces of grease and dust.

    The course showed undoubtedly that fibres should always be inspected and that in almost all cases they should be cleaned before plugging in.
    One should not forget to inspect and clean the SFP side either!

    The slides of this Fibre Cleaning and Inspection course are available via the OHWR Electronics Design project.

    Thanks to Amin Shoaie from CERN's EN-EL group for making this course available.
    Note that this course and the practical exercises will be repeated at CERN in the last week of April. Please contact us if you are interested.

    Click on image to see the course (pdf, 711kB)

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at April 18, 2017 05:49 PM

    April 16, 2017

    Harald Welte

    Things you find when using SCTP on Linux

    Observations on SCTP and Linux

    When I was still doing Linux kernel work with netfilter/iptables in the early 2000's, I was somebody who actually regularly had a look at the new RFCs that came out. So I saw the SCTP RFCs, SIGTRAN RFCs, SIP and RTP, etc. all released during those years. I was quite happy to see that for new protocols like SCTP and later DCCP, Linux quickly received a mainline implementation.

    Now most people won't have used SCTP so far, but it is a protocol used as transport layer in a lot of telecom protocols for more than a decade now. Virtually all protocols that have traditionally been spoken over time-division multiplex E1/T1 links have been migrated over to SCTP based protocol stackings.

    Working on various Open Source telecom related projects, i of course come into contact with SCTP every so often. Particularly some years back when implementing the Erlang SIGTAN code in erlang/osmo_ss7 and most recently now with the introduction of libosmo-sigtran with its OsmoSTP, both part of the libosmo-sccp repository.

    I've also hard to work with various proprietary telecom equipment over the years. Whether that's some eNodeB hardware from a large brand telecom supplier, or whether it's a MSC of some other vendor. And they all had one thing in common: Nobody seemed to use the Linux kernel SCTP code. They all used proprietary implementations in userspace, using RAW sockets on the kernel interface.

    I always found this quite odd, knowing that this is the route that you have to take on proprietary OSs without native SCTP support, such as Windows. But on Linux? Why? Based on rumors, people find the Linux SCTP implementation not mature enough, but hard evidence is hard to come by.

    As much as it pains me to say this, the kind of Linux SCTP bugs I have seen within the scope of our work on Osmocom seem to hint that there is at least some truth to this (see e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1308360 or https://bugzilla.redhat.com/show_bug.cgi?id=1308362).

    Sure, software always has bugs and will have bugs. But we at Osmocom are 10-15 years "late" with our implementations of higher-layer protocols compared to what the mainstream telecom industry does. So if we find something, and we find it even already during R&D of some userspace code, not even under load or in production, then that seems a bit unsettling.

    One would have expected, with all their market power and plenty of Linux-based devices in the telecom sphere, why did none of those large telecom suppliers invest in improving the mainline Linux SCTP code? I mean, they all use UDP and TCP of the kernel, so it works for most of the other network protocols in the kernel, but why not for SCTP? I guess it comes back to the fundamental lack of understanding how open source development works. That it is something that the given industry/user base must invest in jointly.

    The leatest discovered bug

    During the last months, I have been implementing SCCP, SUA, M3UA and OsmoSTP (A Signal Transfer Point). They were required for an effort to add 3GPP compliant A-over-IP to OsmoBSC and OsmoMSC.

    For quite some time I was seeing some erratic behavior when at some point the STP would not receive/process a given message sent by one of the clients (ASPs) connected. I tried to ignore the problem initially until the code matured more and more, but the problems remained.

    It became even more obvious when using Michael Tuexen's m3ua-testtool, where sometimes even the most basic test cases consisting of sending + receiving a single pair of messages like ASPUP -> ASPUP_ACK was failing. And when the test case was re-tried, the problem often disappeared.

    Also, whenever I tried to observe what was happening by meas of strace, the problem would disappear completely and never re-appear until strace was detached.

    Of course, given that I've written several thousands of lines of new code, it was clear to me that the bug must be in my code. Yesterday I was finally prepare to accept that it might actually be a Linux SCTP bug. Not being able to reproduce that problem on a FreeBSD VM also pointed clearly into this direction.

    Now I could simply have collected some information and filed a bug report (which some kernel hackers at RedHat have thankfully invited me to do!), but I thought my use case was too complex. You would have to compile a dozen of different Osmocom libraries, configure the STP, run the scheme-language m3ua-testtool in guile, etc. - I guess nobody would have bothered to go that far.

    So today I tried to implement a test case that reproduced the problem in plain C, without any external dependencies. And for many hours, I couldn't make the bug to show up. I tried to be as close as possible to what was happening in OsmoSTP: I used non-blocking mode on client and server, used the SCTP_NODELAY socket option, used the sctp_rcvmsg() library wrapper to receive events, but the bug was not reproducible.

    Some hours later, it became clear that there was one setsockopt() in OsmoSTP (actually, libosmo-netif) which enabled all existing SCTP events. I did this at the time to make sure OsmoSTP has the maximum insight possible into what's happening on the SCTP transport layer, such as address fail-overs and the like.

    As it turned out, adding that setsockopt for SCTP_FLAGS to my test code made the problem reproducible. After playing around which of the flags, it seems that enabling the SENDER_DRY_EVENT flag makes the bug appear.

    You can find my detailed report about this issue in https://bugzilla.redhat.com/show_bug.cgi?id=1442784 and a program to reproduce the issue at http://people.osmocom.org/laforge/sctp-nonblock/sctp-dry-event.c

    Inside the Osmocom world, luckily we can live without the SENDER_DRY_EVENT and a corresponding work-around has been submitted and merged as https://gerrit.osmocom.org/#/c/2386/

    With that work-around in place, suddenly all the m3ua-testtool and sua-testtool test cases are reliably green (PASSED) and OsmoSTP works more smoothly, too.

    What do we learn from this?

    Free Software in the Telecom sphere is getting too little attention. This is true even those small portions of telecom relevant protocols that ended up in the kernel like SCTP or more recently the GTP module I co-authored. They are getting too little attention in development, even more lack of attention in maintenance, and people seem to focus more on not using it, rather than fixing and maintaining what is there.

    It makes me really sad to see this. Telecoms is such a massive industry, with billions upon billions of revenue for the classic telecom equipment vendors. Surely, they would be able to co-invest in some basic infrastructure like proper and reliable testing / continuous integration for SCTP. More recently, we see millions and more millions of VC cash burned by buzzword-flinging companies doing "NFV" and "SDN". But then rather reimplement network stacks in userspace than to fix, complete and test those little telecom infrastructure components which we have so far, like the SCTP protocol :(

    Where are the contributions to open source telecom parts from Ericsson, Nokia (former NSN), Huawei and the like? I'm not even dreaming about the actual applications / network elements, but merely the maintenance of something as basic as SCTP. To be fair, Motorola was involved early on in the Linux SCTP code, and Huawei contributed a long series of fixes in 2013/2014. But that's not the kind of long-term maintenance contribution that one would normally expect from the primary interest group in SCTP.

    Finally, let me thank to the Linux SCTP maintainers. I'm not complaining about them! They're doing a great job, given the arcane code base and the fact that they are not working for a company that has SCTP based products as their core business. I'm sure the would love more support and contributions from the Telecom world, too.

    by Harald Welte at April 16, 2017 10:00 PM

    April 09, 2017

    Harald Welte

    SIGTRAN/SS7 stack in libosmo-sigtran merged to master

    As I blogged in my blog post in Fabruary, I was working towards a more fully-featured SIGTRAN stack in the Osmocom (C-language) universe.

    The trigger for this is the support of 3GPP compliant AoIP (with a BSSAP/SCCP/M3UA/SCTP protocol stacking), but it is of much more general nature.

    The code has finally matured in my development branch(es) and is now ready for mainline inclusion. It's a series of about 77 (!) patches, some of which already are the squashed results of many more incremental development steps.

    The result is as follows:

    • General SS7 core functions maintaining links, linksets and routes
    • xUA functionality for the various User Adaptations (currently SUA and M3UA supported)
      • MTP User SAP according to ITU-T Q.701 (using osmo_prim)
      • management of application servers (AS)
      • management of application server processes (ASP)
      • ASP-SM and ASP-TM state machine for ASP, AS-State Machine (using osmo_fsm)
      • server (SG) and client (ASP) side implementation
      • validated against ETSI TS 102 381 (by means of Michael Tuexen's m3ua-testtool)
      • support for dynamic registration via RKM (routing key management)
      • osmo-stp binary that can be used as Signal Transfer Point, with the usual "Cisco-style" command-line interface that all Osmocom telecom software has.
    • SCCP implementation, with strong focus on Connection Oriented SCCP (as that's what the A interface uses).
      • osmo_fsm based state machine for SCCP connection, both incoming and outgoing
      • SCCP User SAP according to ITU-T Q.711 (osmo_prim based)
      • Interfaces with underlying SS7 stack via MTP User SAP (osmo_prim based)
      • Support for SCCP Class 0 (unit data) and Class 2 (connection oriented)
      • All SCCP + SUA Address formats (Global Title, SSN, PC, IPv4 Address)
      • SCCP and SUA share one implementation, where SCCP messages are transcoded into SUA before processing, and re-encoded into SCCP after processing, as needed.

    I have already done experimental OsmoMSC and OsmoHNB-GW over to libosmo-sigtran. They're now all just M3UA clients (ASPs) which connect to osmo-stp to exchange SCCP messages back and for the between them.

    What's next on the agenda is to

    • finish my incomplete hacks to introduce IPA/SCCPlite as an alternative to SUA and M3UA (for backwards compatibility)
    • port over OsmoBSC to the SCCP User SAP of libosmo-sigtran
      • validate with SSCPlite lower layer against existing SCCPlite MSCs
    • implement BSSAP / A-interface procedures in OsmoMSC, on top of the SCCP-User SAP.

    If those steps are complete, we will have a single OsmoMSC that can talk both IuCS to the HNB-GW (or RNCs) for 3G/3.5G as well as AoIP towards OsmoBSC. We will then have fully SIGTRAN-enabled the full Osmocom stack, and are all on track to bury the OsmoNITB that was devoid of such interfaces.

    If any reader is interested in interoperability testing with other implementations, either on M3UA or on SCCP or even on A or Iu interface level, please contact me by e-mail.

    by Harald Welte at April 09, 2017 10:00 PM

    April 08, 2017

    Andrew Zonenberg, Silicon Exposed

    STARSHIPRAIDER: Preparing for high-speed I/O characterization

    In my previous post, I characterized the STARSHIPRAIDER I/O circuit for high voltage fault transient performance, but was unable to adequately characterize the high speed data performance because my DSO (Rigol DS1102D) only has 100 MHz of bandwidth.

    Although I did have some ideas on how to improve the performance of the current I/O circuit, it was already faster than I could measure so I had no way to know if my improvements were actually making it any better. Ideally I'd just buy an oscilloscope with several GHz of bandwidth, but I'm not made of money and those scopes tend to be in the "request a quote" price range.

    The obvious solution was to build one. I already had a proven high-speed sampling architecture from my TDR project so all I had to do was repackage it as an oscilloscope and make it faster still.

    The circuit was beautifully simple: an output from the FPGA drives a 50 ohm trace to a SMA connector, then a second SMA connector drives the positive input of an ADCMP572 through a 3 dB attenuator (to keep my signal within range). The negative input is driven by a cheap 12-bit I2C DAC. The comparator output is then converted from CML to LVDS and fed to the host FPGA board. Finally, a 3.3V CML output from the FPGA drives the latch enable input on the comparator.

    The "ADC" algorithm is essentially the same as on my TDR. I like to think of it as an equivalent-time version of a flash ADC: rather than 256 comparators digitizing the signal once, I digitize the signal 256 times with one comparator (and of course 256 different reference voltages). The post-processing to turn the comparator outputs into 8-bit ADC codes is the same.

    Unlike the TDR, however, I also do equivalent-time sampling in the time domain. The FPGA generates the sampling and PRBS clocks with different PLL outputs (at 250 MHz / 4 ns period), and sweeps the relative phase in 100 ps steps to produce an effective resolution of 10 Gsps / 100 ps timebase.

    Without further ado here's a picture of the board. Total BOM cost including connectors and PCB was approximately $50.

    Oscilloscope board (yes, it's PMOD form factor!)
    After some initial firmware development I was able to get some preliminary eye renders off the board. They were, to say the least, not ideal.

    250 Mbps: very bumpy rise
    500 Mbps: significant eye closure even with increased drive strength

    I spent quite a while tracking down other bugs before dealing with the signal integrity issues. For example, a low-frequency pulse train showed up with a very uneven duty cycle:

    Duty cycle distortion
    Someone suggested that I try a slow rise time pulse to show the distortion more clearly. Not having a proper arbitrary waveform generator, I made do with a squarewave and R-C lowpass filter.

    Ever seen breadboarded passives interfacing to edge-launch SMA connectors before?
    It appeared that I had jump discontinuities in my waveform every two blocks (color coding)
    I don't have an EE degree, but I can tell this looks wrong!

    Interestingly enough, two blocks (of 32 samples each) were concatenated into a single JTAG transfer. These two were read in one clock cycle and looked fine, but the junction to the next transfer seemed to be skipping samples.

    As it turned out, I had forgotten to clear a flag which led to me reading the waveform data before it was done capturing. Since the circular buffer was rotating in between packets, some samples never got sent.

    The next bug required zooming into the waveform a bit to see. The samples captured on the first few (the number seemed to vary across bitstream builds) of my 40 clock phases were showing up shifted by 4 ns (one capture clock).

    Horizontally offset samples

    I traced this issue to a synchronizer between clock domains having variable latency depending on the phase offset of the source and destination clocks. This is an inherent issue in clock domain crossing, so I think I'm just going to have to calibrate it out somehow. For the short term I'm manually measuring the number of offset phases each time I recompile the FPGA image, and then correcting the data in post-processing.

    The final issue was a hardware bug. I was terminating the incoming signal with a 50Ω resistor to ground. Although this had good AC performance, at DC the current drawn from a high-level input was quite significant (66 mA at 3.3V). Since my I/O pins can't drive this much, the line was dragged down.

    I decided to rework the input termination to replace the 50Ω terminator with split 100Ω resistors to 3.3V and ground. This should have about half the DC current draw, and is Thevenin equivalent to a 50Ω terminator to 1.65V. As a bonus, the mid-level termination will also allow me to AC-couple the incoming signal if that becomes necessary.

    Mill out trace from ground via to on-die 50Ω termination resistor

    Remove soldermask from ground via and signal trace

    Add 100Ω 0402 low-side terminator
    Add 100Ω 0402 high-side terminator, plus jumper trace to 3.3V bulk decoupling cap

    Add 10 nF high speed decoupling cap to help compensate for inductance of long feeder trace
    I cleaned off all of the flux residue and ran a second set of eye loopback tests at 250 and 500 Mbps. The results were dramatically improved:

    Post-rework at 250 Mbps
    Post-rework at 500 Mbps
    While not perfect, the new eye openings are a lot cleaner. I hope to tweak my input stage further to reduce probing artifacts, but for the time being I think I have sufficient performance to compare multiple STARSHIPRAIDER test circuits and see how they stack up at relatively high speeds.

    Next step: collect some baseline data for the current STARSHIPRAIDER characterization board, then use that to inform my v0.2 I/O circuit!

    by Andrew Zonenberg (noreply@blogger.com) at April 08, 2017 01:50 AM

    March 31, 2017

    Bunnie Studios

    Name that Ware, March 2017

    The Ware for March 2017 is shown below.

    I honestly have no idea what this one is from or what it’s for — found it in a junk pile in China. But I was amused by the comically huge QFP, so I snapped a shot of it.

    Sorry this is a little late — been ridiculously busy prepping for the launch of a line of new products for Chibitronics, going beta (hopefully) next month.

    by bunnie at March 31, 2017 05:52 PM

    Winner, Name that Ware February 2017

    The Ware for February 2017 is a Data Harvest EcoLog.

    A number of people guessed it was a datalogger of some type, but didn’t quite identify the manufacturer or model correctly. That being said, I found Josh Myer’s response an interesting read, so I’ll give the prize to him. Congrats, email me for your prize!

    by bunnie at March 31, 2017 05:52 PM

    March 30, 2017

    Open Hardware Repository

    White Rabbit - Tutorial at ICALEPCS conference in October

    The ICALEPCS organizing committee will organize a pre-conference workshop on WR in Barcelona in October.

    We intend to make this more of a "WR tutorial." and we think there will be something to learn and discuss for everybody: newcomers, casual users and even experts.

    Online registration will open on April 17. Registration for the workshop is independent of registration to the conference. If you register, it will be a great pleasure to see you there. Also, please send me comments on the program if you have any. We still have a bit of freedom to change it if need be.

    And of course, please forward this to any other people you think could be interested!

    Javier

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at March 30, 2017 01:59 PM

    March 29, 2017

    Free Electrons

    Free Electrons at the Netdev 2.1 conference

    Netdev 2.1 is the fourth edition of the technical conference on Linux networking. This conference is driven by the community and focus on both the kernel networking subsystems (device drivers, net stack, protocols) and their use in user-space.

    This edition will be held in Montreal, Canada, April 6 to 8, and the schedule has been posted recently, featuring amongst other things a talk giving an overview and the current status display of the Distributed Switch Architecture (DSA) or a workshop about how to enable drivers to cope with heavy workloads, to improve performances.

    At Free Electrons, we regularly work on networking related topics, especially as part of our Linux kernel contribution for the support of Marvell or Annapurna Labs ARM SoCs. Therefore, we decided to attend our first Netdev conference to stay up-to-date with the network subsystem and network drivers capabilities, and to learn from the community latest developments.

    Our engineer Antoine Ténart will be representing Free Electrons at this event. We’re looking forward to being there!

    by Antoine Ténart at March 29, 2017 02:35 PM

    March 26, 2017

    Harald Welte

    OsmoCon 2017 Updates: Travel Grants and Schedule

    /images/osmocon.png

    April 21st is approaching fast, so here some updates. I'm particularly happy that we now have travel grants available. So if the travel expenses were preventing you from attending so far: This excuse is no longer valid!

    Get your ticket now, before it is too late. There's a limited number of seats available.

    OsmoCon 2017 Schedule

    The list of talks for OsmoCon 2017 has been available for quite some weeks, but today we finally published the first actual schedule.

    As you can see, the day is fully packed with talks about Osmocom cellular infrastructure projects. We had to cut some talk slots short (30min instead of 45min), but I'm confident that it is good to cover a wider range of topics, while at the same time avoiding fragmenting the audience with multiple tracks.

    OsmoCon 2017 Travel Grants

    We are happy to announce that we have received donations to permit for providing travel grants!

    This means that any attendee who is otherwise not able to cover their travel to OsmoCon 2017 (e.g. because their interest in Osmocom is not related to their work, or because their employer doesn't pay the travel expenses) can now apply for such a travel grant.

    For more details see OsmoCon 2017 Travel Grants and/or contact osmocon2017@sysmocom.de.

    OsmoCon 2017 Social Event

    Tech Talks are nice and fine, but what many people enjoy even more at conferences is the informal networking combined with good food. For this, we have the social event at night, which is open to all attendees.

    See more details about it at OsmoCon 2017 Social Event.

    by Harald Welte at March 26, 2017 10:00 PM

    March 23, 2017

    Harald Welte

    Upcoming v3 of Open Hardware miniPCIe WWAN modem USB breakout board

    Back in October 2016 I designed a small open hardware breakout board for WWAN modems in mPCIe form-factor. I was thinking some other people might be interested in this, and indeed, the first manufacturing batch is already sold out by now.

    Instead of ordering more of the old (v2) design, I decided to do some improvements in the next version:

    • add mounting holes so the PCB can be mounted via M3 screws
    • add U.FL and SMA sockets, so the modems are connected via a short U.FL to U.FL cable, and external antennas or other RF components can be attached via SMA. This provides strain relief for the external antenna or cabling and avoids tearing off any of the current loose U.FL to SMA pigtails
    • flip the SIM slot to the top side of the PCB, so it can be accessed even after mounting the board to some base plate or enclosure via the mounting holes
    • more meaningful labeling of the silk screen, including the purpose of the jumpers and the input voltage.

    A software rendering of the resulting v3 PCB design files that I just sent for production looks like this:

    /images/mpcie-breakout-v3-pcb-rendering.png

    Like before, the design of the board (including schematics and PCB layout design files) is available as open hardware under CC-BY-SA license terms. For more information see http://osmocom.org/projects/mpcie-breakout/wiki

    It will take some expected three weeks until I'll see the first assembled boards.

    I'm also planning to do a M.2 / NGFF version of it, but haven't found the time to get around doing it so far.

    by Harald Welte at March 23, 2017 11:00 PM

    March 21, 2017

    Harald Welte

    Osmocom - personal thoughts

    As I just wrote in my post about TelcoSecDay, I sometimes worry about the choices I made with Osmocom, particularly when I see all the great stuff people doing in fields that I previously was working in, such as applied IT security as well as Linux Kernel development.

    History

    When people like Dieter, Holger and I started to play with what later became OpenBSC, it was just for fun. A challenge to master. A closed world to break open and which to attack with the tools, the mindset and the values that we brought with us.

    Later, Holger and I started to do freelance development for commercial users of Osmocom (initially basically only OpenBSC, but then OsmoSGSN, OsmoBSC, OsmoBTS, OsmoPCU and all the other bits on the infrastructure side). This lead to the creation of sysmocom in 2011, and ever since we are trying to use revenue from hardware sales as well as development contracts to subsidize and grow the Osmocom projects. We're investing most of our earnings directly into more staff that in turn works on Osmocom related projects.

    NOTE

    It's important to draw the distinction betewen the Osmocom cellular infrastructure projects which are mostly driven by commercial users and sysmocom these days, and all the many other pure juts-for-fun community projects under the Osmocom umbrella, like OsmocomTETRA, OsmocomGMR, rtl-sdr, etc. I'm focussing only on the cellular infrastructure projects, as they are in the center of my life during the past 6+ years.

    In order to do this, I basically gave up my previous career[s] in IT security and Linux kernel development (as well as put things like gpl-violations.org on hold). This is a big price to pay for crating more FOSS in the mobile communications world, and sometimes I'm a bit melancholic about the "old days" before.

    Financial wealth is clearly not my primary motivation, but let me be honest: I could have easily earned a shitload of money continuing to do freelance Linux kernel development, IT security or related consulting. There's a lot of demand for related skills, particularly with some experience and reputation attached. But I decided against it, and worked several years without a salary (or almost none) on Osmocom related stuff [as did Holger].

    But then, even with all the sacrifices made, and the amount of revenue we can direct from sysmocom into Osmocom development: The complexity of cellular infrastructure vs. the amount of funding and resources is always only a fraction of what one would normally want to have to do a proper implementation. So it's constant resource shortage, combined with lots of unpaid work on those areas that are on the immediate short-term feature list of customers, and that nobody else in the community feels like he wants to work on. And that can be a bit frustrating at times.

    Is it worth it?

    So after 7 years of OpenBSC, OsmocomBB and all the related projects, I'm sometimes asking myself whether it has been worth the effort, and whether it was the right choice.

    It was right from the point that cellular technology is still an area that's obscure and unknown to many, and that has very little FOSS (though Improving!). At the same time, cellular networks are becoming more and more essential to many users and applications. So on an abstract level, I think that every step in the direction of FOSS for cellular is as urgently needed as before, and we have had quite some success in implementing many different protocols and network elements. Unfortunately, in most cases incompletely, as the amount of funding and/or resources were always extremely limited.

    Satisfaction/Happiness

    On the other hand, when it comes to metrics such as personal satisfaction or professional pride, I'm not very happy or satisfied. The community remains small, the commercial interest remains limited, and as opposed to the Linux world, most players have a complete lack of understanding that FOSS is not a one-way road, but that it is important for all stakeholders to contribute to the development in terms of development resources.

    Project success?

    I think a collaborative development project (which to me is what FOSS is about) is only then truly successful, if its success is not related to a single individual, a single small group of individuals or a single entity (company). And no matter how much I would like the above to be the case, it is not true for the Osmocom cellular infrastructure projects. Take away Holger and me, or take away sysmocom, and I think it would be pretty much dead. And I don't think I'm exaggerating here. This makes me sad, and after all these years, and after knowing quite a number of commercial players using our software, I would have hoped that the project rests on many more shoulders by now.

    This is not to belittle the efforts of all the people contributing to it, whether the team of developers at sysmocom, whether those in the community that still work on it 'just for fun', or whether those commercial users that contract sysmocom for some of the work we do. Also, there are known and unknown donors/funders, like the NLnet foundation for some parts of the work. Thanks to all of you, and clearly we wouldn't be where we are now without all of that!

    But I feel it's not sufficient for the overall scope, and it's not [yet] sustainable at this point. We need more support from all sides, particularly those not currently contributing. From vendors of BTSs and related equipment that use Osmocom components. From operators that use it. From individuals. From academia.

    Yes, we're making progress. I'm happy about new developments like the Iu and Iuh support, the OsmoHLR/VLR split and 2G/3G authentication that Neels just blogged about. And there's progress on the SIMtrace2 firmware with card emulation and MITM, just as well as there's progress on libosmo-sigtran (with a more complete SUA, M3UA and connection-oriented SCCP stack), etc.

    But there are too little people working on this, and those people are mostly coming from one particular corner, while most of the [commercial] users do not contribute the way you would expect them to contribute in collaborative FOSS projects. You can argue that most people in the Linux world also don't contribute, but then the large commercial beneficiaries (like the chipset and hardware makers) mostly do, as are the large commercial users.

    All in all, I have the feeling that Osmocom is as important as it ever was, but it's not grown up yet to really walk on its own feet. It may be able to crawl, though ;)

    So for now, don't panic. I'm not suffering from burn-out, mid-life crisis and I don't plan on any big changes of where I put my energy: It will continue to be Osmocom. But I also think we have to have a more open discussion with everyone on how to move beyond the current situation. There's no point in staying quiet about it, or to claim that everything is fine the way it is. We need more commitment. Not from the people already actively involved, but from those who are not [yet].

    If that doesn't happen in the next let's say 1-2 years, I think it's fair that I might seriously re-consider in which field and in which way I'd like to dedicate my [I would think considerable] productive energy and focus.

    by Harald Welte at March 21, 2017 06:00 PM

    Returning from TelcoSecDay 2017 / General Musings

    I'm just on my way back from the Telecom Security Day 2017 <https://www.troopers.de/troopers17/telco-sec-day/>, which is an invitation-only event about telecom security issues hosted by ERNW back-to-back with their Troopers 2017 <https://www.troopers.de/troopers17/> conference.

    I've been presenting at TelcoSecDay in previous years and hence was again invited to join (as attendee). The event has really gained quite some traction. Where early on you could find lots of IT security / hacker crowds, the number of participants from the operator (and to smaller extent also equipment maker) industry has been growing.

    The quality of talks was great, and I enjoyed meeting various familiar faces. It's just a pity that it's only a single day - plus I had to head back to Berlin still today so I had to skip the dinner + social event.

    When attending events like this, and seeing the interesting hacks that people are working on, it pains me a bit that I haven't really been doing much security work in recent years. netfilter/iptables was at least somewhat security related. My work on OpenPCD / librfid was clearly RFID security oriented, as was the work on airprobe, OsmocomTETRA, or even the EasyCard payment system hack

    I have the same feeling when attending Linux kernel development related events. I have very fond memories of working in both fields, and it was a lot of fun. Also, to be honest, I believe that the work in Linux kernel land and the general IT security research was/is appreciated much more than the endless months and years I'm now spending my time with improving and extending the Osmocom cellular infrastructure stack.

    Beyond the appreciation, it's also the fact that both the IT security and the Linux kernel communities are much larger. There are more people to learn from and learn with, to engage in discussions and ping-pong ideas. In Osmocom, the community is too small (and I have the feeling, it's actually shrinking), and in many areas it rather seems like I am the "ultimate resource" to ask, whether about 3GPP specs or about Osmocom code structure. What I'm missing is the feeling of being part of a bigger community. So in essence, my current role in the "Open Source Cellular" corner can be a very lonely one.

    But hey, I don't want to sound more depressed than I am, this was supposed to be a post about TelcoSecDay. It just happens that attending IT Security and/or Linux Kernel events makes me somewhat gloomy for the above-mentioned reasons.

    Meanwhile, if you have some interesting projcets/ideas at the border between cellular protocols/systems and security, I'd of course love to hear if there's some way to get my hands dirty in that area again :)

    by Harald Welte at March 21, 2017 05:00 PM

    March 16, 2017

    Open Hardware Repository

    CERN BE-CO-HT contribution to KiCad - Support of free software in public institutions:

    At the Octave conference in Geneva the presentation Support of free software in public institutions: the KiCad case will be given by Javier Serrano and Tomasz Wlostowski from CERN.

    KiCad is a tool to help electronics designers develop Printed Circuit Boards (PCB). CERN's BE-CO-HT section has been contributing to its development since 2011. These efforts are framed in the context of CERN's activities regarding Open Source Hardware (OSHW), and are meant to provide an environment where design files for electronics can be shared in an efficient way, without the hurdles imposed by the use of proprietary formats.

    The talk will start by providing some context about OSHW and the importance of using Free Software tools for sharing design files. We will then move on to a short KiCad tutorial, and finish with some considerations about the role public institutions can play in developing and fostering the use of Free Software, and whether some of the KiCad experience can apply in other contexts.

    Access to the presentation: Support of free software in public institutions: the KiCad case

    by Erik van der Bij (Erik.van.der.Bij@cern.ch) at March 16, 2017 04:42 PM

    March 15, 2017

    Bunnie Studios

    Looking for Summer Internship in Hardware Hacking?

    Tim Ansell (mithro), who has been giving me invaluable advice and support on the NeTV2 project, just had his HDMI (plaintext) video capture project accepted into the Google Summer of Code. This summer, he’s looking for university students who have an interest in learning FPGAs, hacking on video, or designing circuits. To learn more you can check out his post at hdmi2usb.tv.

    I’ve learned a lot working with Tim. I also respect his work ethic and he is a steadfast contributor to the open source community. This would be an excellent summer opportunity for any student interested in system-level hardware hacking!

    Please note: application deadline is April 3 16:00 UTC.

    by bunnie at March 15, 2017 07:47 AM

    March 10, 2017

    Free Electrons

    Free Electrons at the Embedded Linux Conference 2017

    Last month, five engineers from Free Electrons participated to the Embedded Linux Conference in Portlan, Oregon. It was once again a great conference to learn new things about embedded Linux and the Linux kernel, and to meet developers from the open-source community.

    Free Electrons team at work at ELC 2017, with Maxime Ripard, Antoine Ténart, Mylène Josserand and Quentin Schulz

    Free Electrons talks

    Free Electrons CEO Michael Opdenacker gave a talk on Embedded Linux Size Reduction techniques, for which the slides and video are available:

    Free Electrons engineer Quentin Schulz gave a talk on Power Management Integrated Circuits: Keep the Power in Your Hands, the slides and video are also available:

    Free Electrons selection of talks

    Of course, the slides from many other talks are progressively being uploaded, and the Linux Foundation published the video recordings in a record time: they are all already available on Youtube!

    Below, each Free Electrons engineer who attended the conference has selected one talk he/she has liked, and gives a quick summary of the talk, hopefully to encourage you watch the corresponding video recording.

    Using SWupdate to Upgrade your system, Gabriel Huau

    Talk selected by Mylène Josserand.

    Gabriel Huau from Witekio did a great talk at ELC about SWUpdate, a tool created by Denx to update your system. The talk gives an overview of this tool, how it is working and how to use it. Updating your system is very important for embedded devices to fix some bugs/security fixes or add new features, but in an industrial context, it is sometimes difficult to perform an update: devices not easily accessible, large number of devices and variants, etc. A tool that can update the system automatically or even Over The Air (OTA) can be very useful. SWUpdate is one of them.

    SWUpdate allows to update different parts of an embedded system such as the bootloader, the kernel, the device tree, the root file system and also the application data.
    It handles different image types: UBI, MTD, Raw, Custom LUA, u-boot environment and even your custom one. It includes a notifier to be able to receive feedback about the updating process which can be useful in some cases. SWUPdate uses different local and OTA/remote interfaces such as USB, SD card, HTTP, etc. It is based on a simple update image format to indicate which images must be updated.

    Many customizations can be done with this tool as it is provided with the classic menuconfig configuration tool. One great thing is that this tool is supported by Yocto Project and Buildroot so it can be easily tested.

    Do not hesitate to have a look to his slides, the video of his talk or directly test SWUpdate!

    GCC/Clang Optimizations for embedded Linux, Khem Raj

    Talk selected by Michael Opdenacker.

    Khem Raj from Comcast is a frequent speaker at the Embedded Linux Conference, and one of his strong fields of expertise is C compilers, especially LLVM/Clang and Gcc. His talk at this conference can interest anyone developing code in the C language, to know about optimizations that the compilers can use to improve the performance or size of generated binaries. See the video and slides.

    Khem Raj slide about compiler optimization optionsOne noteworthy optimization is Clang’s -Oz (Gcc doesn’t have it), which goes even beyond -Os, by disabling loop vectorization. Note that Clang already performs better than Gcc in terms of code size (according to our own measurements). On the topic of bundle optimizations such as -O2 or -Os, Khem added that specific optimizations can be disabled in both compilers through the -fno- command line option preceding the name of a given optimization. The name of each optimization in a given bundle can be found through the -fverbose-asm command line option.

    Another new optimization option is -Og, which is different from the traditional -g option. It still allows to produce code that can be debugged, but in a way that provides a reasonable level of runtime performance.

    On the performance side, he also recalled the Feedback-Directed Optimizations (FDO), already covered in earlier Embedded Linux Conferences, which can be used to feed the compiler with profiler statistics about code branches. The compiler can use such information to optimize branches which are the more frequent at run-time.

    Khem’s last advise was not to optimize too early, and first make sure you do your debugging and profiling work first, as heavily optimized code can be very difficult to debug. Therefore, optimizations are for well-proven code only.

    Note that Khem also gave a similar talk in the IoT track for the conference, which was more focused on bare-metal code optimization code and portability: “Optimizing C for microcontrollers” (slides, video).

    A Journey through Upstream Atomic KMS to Achieve DP Compliance, Manasi Navare

    Talk selected by Quentin Schulz.

    This talk was about the journey of a new comer in the mainline kernel community to fix the DisplayPort support in Intel i915 DRM driver. It first presented what happens from the moment we plug a cable in a monitor until we actually see an image, then where the driver is in the kernel: in the DRM subsystem, between the hardware (an Intel Integrated Graphics device) and the libdrm userspace library on which userspace applications such as the X server rely.

    The bug to fix was that case when the driver would fail after updating to the requested resolution for a DP link. The other existing drivers usually fail before updating the resolution, so Manasi had to add a way to tell the userspace the DP link failed after updating the resolution. Such addition would be useless without applications using this new information, therefore she had to work with their developers to make the applications behave correctly when reading this important information.

    With a working set of patches, she thought she had done most of the work with only the upstreaming left and didn’t know it would take her many versions to make it upstream. She wished to have sent a first version of a driver for review earlier to save time over the whole development plus upstreaming process. She also had to make sure the changes in the userspace applications will be ready when the driver will be upstreamed.

    The talk was a good introduction on how DisplayPort works and an excellent example on why involving the community even in early stages of the development process may be a good idea to quicken the overall driver development process by avoiding complete rewriting of some code parts when upstreaming is under way.

    See also the video and slides of the talk.

    Timekeeping in the Linux Kernel, Stephen Boyd

    Talk selected by Maxime Ripard.

    Stephen did a great talk about one thing that is often overlooked, and really shouldn’t: Timekeeping. He started by explaining the various timekeeping mechanisms, both in hardware and how Linux use them. That meant covering the counters, timers, the tick, the jiffies, and the various POSIX clocks, and detailing the various frameworks using them. He also explained the various bugs that might be encountered when having a too naive counter implementation for example, or using the wrong POSIX clock from an application.

    See also the video and slides of the talk.

    Android Things, Karim Yaghmour

    Talk selected by Antoine Ténart

    Karim did a very good introduction to Android Things. His talk was a great overview of what this new OS from Google targeting embedded devices is, and where it comes from. He started by showing the history of Android, and he explained what this system brought to the embedded market. He then switched to the birth of Android Things; a reboot of Google’s strategy for connected devices. He finally gave an in depth explanation of the internals of this new OS, by comparing Android Things and Android, with lots of examples and demos.

    Android Things replaces Brillo / Weave, and unlike its predecessor is built reusing available tools and services. It’s in fact a lightweight version of Android, with many services removed and a few additions like the PIO API to drive GPIO, I2C, PWM or UART controllers. A few services were replaced as well, most notably the launcher. The result is a not so big, but not so small, system that can run on headless devices to control various sensors; with an Android API for application developers.

    See also the video and slides of the talk.

    by Thomas Petazzoni at March 10, 2017 09:01 AM

    March 07, 2017

    Harald Welte

    VMware becomes gold member of Linux Foundation: And what about the GPL?

    As we can read in recent news, VMware has become a gold member of the Linux foundation. That causes - to say the least - very mixed feelings to me.

    One thing to keep in mind: The Linux Foundation is an industry association, it exists to act in the joint interest of it's paying members. It is not a charity, and it does not act for the public good. I know and respect that, while some people sometimes appear to be confused about its function.

    However, allowing an entity like VMware to join, despite their many years long disrespect for the most basic principles of the FOSS Community (such as: Following the GPL and its copyleft principle), really is hard to understand and accept.

    I wouldn't have any issue if VMware would (prior to joining LF) have said: Ok, we had some bad policies in the past, but now we fully comply with the license of the Linux kernel, and we release all derivative/collective works in source code. This would be a positive spin: Acknowledge past issues, resolve the issues, become clean and then publicly underlining your support of Linux by (among other things) joining the Linux Foundation. I'm not one to hold grudges against people who accept their past mistakes, fix the presence and then move on. But no, they haven't fixed any issues.

    They are having one of the worst track records in terms of intentional GPL compliance issues for many years, showing outright disrespect for Linux, the GPL and ultimately the rights of the Linux developers, not resolving those issues and at the same time joining the Linux Foundation? What kind of message sends that?

    It sends the following messages:

    • you can abuse Linux, the GPL and copyleft while still being accepted amidst the Linux Foundation Members
    • it means the Linux Foundations has no ethical concerns whatsoever about accepting such entities without previously asking them to become clean
    • it also means that VMware has still not understood that Linux and FOSS is about your actions, particularly the kind of choices you make how to technically work with the community, and not against it.

    So all in all, I think this move has seriously damaged the image of both entities involved. I wouldn't have expected different of VMware, but I would have hoped the Linux Foundation had some form of standards as to which entities they permit amongst their ranks. I guess I was being overly naive :(

    It's a slap in the face of every developer who writes code not because he gets paid, but because it is rewarding to know that copyleft will continue to ensure the freedom of related code.

    UPDATE (March 8, 2017):
     I was mistaken in my original post in that VMware didn't just join, but was a Linux Foundation member already before, it is "just" their upgrade from silver to gold that made the news recently. I stand corrected. Still doesn't make it any better that the are involved inside LF while engaging in stepping over the lines of license compliance.
    UPDATE2 (March 8, 2017):
     As some people pointed out, there is no verdict against VMware. Yes, that's true. But the mere fact that they rather distribute derivative works of GPL licensed software and take this to court with an armada of lawyers (instead of simply complying with the license like everyone else) is sad enough. By the time there will be a final verdict, the product is EOL. That's probably their strategy to begin with :/

    by Harald Welte at March 07, 2017 11:00 PM

    Gory details of USIM authentication sequence numbers

    I always though I understood UMTS AKA (authentication and key agreement), including the re-synchronization procedure. It's been years since I wrote tools like osmo-sim-auth which you can use to perform UMTS AKA with a SIM card inserted into a PC reader, i.e. simulate what happens between the AUC (authentication center) in a network and the USIM card.

    However, it is only now as the sysmocom team works on 3G support of the dedicated OsmoHLR (outside of OsmoNITB!), that I seem to understand all the nasty little details.

    I always thought for re-synchronization it is sufficient to simply increment the SQN (sequence number). It turns out, it isn't as there is a MSB-portion called SEQ and a lower-bit portion called IND, used for some fancy array indexing scheme of buckets of highest-used-SEQ within that IND bucket.

    If you're interested in all the dirty details and associated spec references (the always hide the important parts in some Annex) see the discussion between Neels and me in Osmocom redmine issue 1965.

    by Harald Welte at March 07, 2017 11:00 PM

    March 05, 2017

    Harald Welte

    GTA04 project halts GTA04A5 due to OMAP3 PoP soldering issues

    For those of you who don't know what the tinkerphones/OpenPhoenux GTA04 is: It is a 'professional hobbyist' hardware project (with at least public schematics, even if not open hardware in the sense that editable schematics and PCB design files are published) creating updated mainboards that can be used to upgrade Openmoko phones. They fit into the same enclosure and can use the same display/speaker/microphone.

    What the GTA04 guys have been doing for many years is close to a miracle anyway: Trying to build a modern-day smartphone in low quantities, using off-the-shelf components available in those low quantities, and without a large company with its associated financial backing.

    Smartphones are complex because they are highly integrated devices. A seemingly unlimited amount of components is squeezed in the tiniest form-factors. This leads to complex circuit boards with many layers that take a lot of effort to design, and are expensive to build in low quantities. The fine-pitch components mandated by the integration density is another issue.

    Building the original GTA01 (Neo1937) and GTA02 (FreeRunner) devices at Openmoko, Inc. must seem like a piece of cake compared to what the GTA04 guys are up to. We had a team of engineers that were familiar at last with feature phone design before, and we had the backing of a consumer electronics company with all its manufacturing resources and expertise.

    Nevertheless, a small group of people around Dr. Nikolaus Schaller has been pushing the limits of what you can do in a small for fun project, and the have my utmost respect. Well done!

    Unfortunately, there are bad news. Manufacturing of their latest generation of phones (GTA04A5) has been stopped due to massive soldering problems with the TI OMAP3 package-on-package (PoP). Those PoPs are basically "RAM chip soldered onto the CPU, and the stack of both soldered to the PCB". This is used to save PCB footprint and to avoid having to route tons of extra (sensitive, matched) traces between the SDRAM and the CPU.

    According to the mailing list posts, it seems to be incredibly difficult to solder the PoP stack due to the way TI has designed the packaging of the DM3730. If you want more gory details, see this post and yet another post.

    It is very sad to see that what appears to be bad design choices at TI are going to bring the GTA04 project to a halt. The financial hit by having only 33% yield is already more than the small community can take, let alone unused parts that are now in stock or even thinking about further experiments related to the manufacturability of those chips.

    If there's anyone with hands-on manufacturing experience on the DM3730 (or similar) TI PoP reading this: Please reach out to the GTA04 guys and see if there's anything that can be done to help them.

    UPDATE (March 8, 2017):
     In an earlier post I was asserting that the GTA04 is open hardware (which I actually believed up to that point) until some readers have pointed out to me that it isn't. It's sad it isn't, but still it has my sympathies.

    by Harald Welte at March 05, 2017 11:00 PM

    March 04, 2017

    Free Electrons

    Buildroot 2017.02 released, Free Electrons contributions

    Buildroot LogoThe 2017.02 version of Buildroot has been released recently, and as usual Free Electrons has been a significant contributor to this release. A total of 1369 commits have gone into this release, contributed by 110 different developers.

    Before looking in more details at the contributions from Free Electrons, let’s have a look at the main improvements provided by this release:

    • The big announcement is that 2017.02 is going to be a long term support release, maintained with security and other important fixes for one year. This will allow companies, users and projects that cannot upgrade at each Buildroot release to have a stable Buildroot version to work with, coming with regular updates for security and bug fixes. A few fixes have already been collected in the 2017.02.x branch, and regular point releases will be published.
    • Several improvements have been made to support reproducible builds, i.e the capability of having two builds of the same configuration provide the exact same bit-to-bit output. These are not enough to provide reproducible builds yet, but they are a piece of the puzzle, and more patches are pending for the next releases to move forward on this topic.
    • A package infrastructure for packages using the waf build system has been added. Seven packages in Buildroot are using this infrastructure currently.
    • Support for the OpenRISC architecture has been added, as well as improvements to the support of ARM64 (selection of ARM64 cores, possibility of building an ARM 32-bit system optimized for an ARM64 core).
    • The external toolchain infrastructure, which was all implemented in a single very complicated package, has been split into one package per supported toolchain and a common infrastructure. This makes it much easier to maintain.
    • A number of updates has been made to the toolchain components and capabilities: uClibc-ng bumped to 1.0.22 and enabled for ARM64, mips32r6 and mips64r6, gdb 7.12.1 added and switched to gdb 7.11 as the default, Linaro toolchains updated to 2016.11, ARC toolchain components updated to arc-2016.09, MIPS Codescape toolchains bumped to 2016.05-06, CodeSourcery AMD64 and NIOS2 toolchains bumped.
    • Eight new defconfigs for various hardware platforms have been added, including defconfigs for the NIOSII and OpenRISC Qemu emulation.
    • Sixty new packages have been added, and countless other packages have been updated or fixed.

    Buildroot developers at work during the Buildroot Developers meeting in February 2017, after the FOSDEM conference in Brussels.

    More specifically, the contributions from Free Electrons have been:

    • Thomas Petazzoni has handled the release of the first release candidate, 2017.02-rc1, and merged 742 patches out of the 1369 commits merged in this release.
    • Thomas contributed the initial work for the external toolchain infrastructure rework, which has been taken over by Romain Naour and finally merged thanks to Romain’s work.
    • Thomas contributed the rework of the ARM64 architecture description, to allow building an ARM 32-bit system optimized for a 64-bit core, and to allow selecting specific ARM64 cores.
    • Thomas contributed the raspberrypi-usbboot package, which packages a host tool that allows to boot a RaspberryPi system over USB.
    • Thomas fixed a large number of build issues found by the project autobuilders, contributing 41 patches to this effect.
    • Mylène Josserand contributed a patch to the X.org server package, fixing an issue with the i.MX6 OpenGL acceleration.
    • Gustavo Zacarias contributed a few fixes on various packages.

    In addition, Free Electrons sponsored the participation of Thomas to the Buildroot Developers meeting that took place after the FOSDEM conference in Brussels, early February. A report of this meeting is available on the eLinux Wiki.

    The details of Free Electrons contributions:

    by Thomas Petazzoni at March 04, 2017 08:53 PM

    February 27, 2017

    Bunnie Studios

    Name that Ware, February 2017

    The ware for February 2017 is shown below:

    This is a ware contributed by an anonymous reader. Thanks for the contribution, you know who you are!

    by bunnie at February 27, 2017 08:04 AM

    Winner, Name that Ware January 2017

    The Ware for January 2017 is a Philips Norelco shaver, which recently died so I thought I’d take it apart and see what’s inside. It’s pretty similar to the previous generation shaver I was using. Hard to pick a winner — Jimmyjo got the thread on the right track, Adrian got the reference to the prior blog post…from 8 years ago. I think I’ll run with with Jimmyjo as the winner though, since it looks from the time stamps he was the first to push the thread into the general category of electric shaver. Congrats, email me to claim your prize (again)!

    by bunnie at February 27, 2017 08:04 AM

    February 24, 2017

    Free Electrons

    Linux 4.10, Free Electrons contributions

    After 8 release candidates, Linus Torvalds released the final 4.10 Linux kernel last Sunday. A total of 13029 commits were made between 4.9 and 4.10. As usual, LWN had a very nice coverage of the major new features added during the 4.10 merge window: part 1, part 2 and part 3. The KernelNewbies Wiki has an updated page about 4.10 as well.

    On the total of 13029 commits, 116 were made by Free Electrons engineers, which interestingly is exactly the same number of commits we made for the 4.9 kernel release!

    Our main contributions for this release have been:

    • For Atmel platforms, Alexandre Belloni added support for the securam block of the SAMA5D2, which is needed to implement backup mode, a deep suspend-to-RAM state for which we will be pushing patches over the next kernel releases. Alexandre also fixed some bugs in the Atmel dmaengine and USB gadget drivers.
    • For Allwinner platforms
      • Antoine Ténart enabled the 1-wire controller on the CHIP platform
      • Boris Brezillon fixed an issue in the NAND controller driver, that prevented from using ECC chunks of 512 bytes.
      • Maxime Ripard added support for the CHIP Pro platform from NextThing, together with many addition of features to the underlying SoC, the GR8 from Nextthing.
      • Maxime Ripard implemented audio capture support in the sun4i-i2s driver, bringing capture support to Allwinner A10 platforms.
      • Maxime Ripard added clock support for the Allwinner A64 to the sunxi-ng clock subsystem, and implemented numerous improvements for this subsystem.
      • Maxime Ripard reworked the pin-muxing driver on Allwinner platforms to use a new generic Device Tree binding, and deprecated the old platform-specific Device Tree binding.
      • Quentin Schulz added a MFD driver for the Allwinner A10/A13/A31 hardware block that provides ADC, touchscreen and thermal sensor functionality.
    • For the RaspberryPi platform
      • Boris Brezillon added support for the Video Encoder IP, which provides composite output. See also our recent blog post about our RaspberryPi work.
      • Boris Brezillon made a number of improvements to clock support on the RaspberryPi, which were needed for the Video Encoder IP support.
    • For the Marvell ARM platform
      • Grégory Clement enabled networking support on the Marvell Armada 3700 SoC, a Cortex-A53 based processor.
      • Grégory Clement did a large number of cleanups in the Device Tree files of Marvell platforms, fixing DTC warnings, and using node labels where possible.
      • Romain Perier contributed a brand new driver for the SPI controller of the Marvell Armada 3700, and therefore enabled SPI support on this platform.
      • Romain Perier extended the existing i2c-pxa driver to support the Marvell Armada 3700 I2C controller, and enabled I2C support on this platform.
      • Romain Perier extended the existing hardware number generator driver for OMAP to also be usable for SafeXcel EIP76 from Inside Secure. This allows to use this driver on the Marvell Armada 7K/8K SoC.
      • Romain Perier contributed support for the Globalscale EspressoBin board, a low-cost development board based on the Marvell Armada 3700.
      • Romain Perier did a number of fixes to the CESA driver, used for the cryptographic engine found on 32-bit Marvell SoCs, such as Armada 370, XP or 38x.
      • Thomas Petazzoni fixed a bug in the mvpp2 network driver, currently only used on Marvell Armada 375, but in the process of being extended to be used on Marvell Armada 7K/8K as well.
    • As the maintainer of the MTD NAND subsystem, Boris Brezillon did a few cleanups in the Tango NAND controller driver, added support for the TC58NVG2S0H NAND chip, and improved the core NAND support to accommodate controllers that have some special timing requirements.
    • As the maintainer of the RTC subsystem, Alexandre Belloni did a number of small cleanups and improvements, especially to the jz4740

    Here is the detailed list of our commits to the 4.10 release:

    by Thomas Petazzoni at February 24, 2017 09:12 AM

    February 23, 2017

    Harald Welte

    Manual testing of Linux Kernel GTP module

    In May 2016 we got the GTP-U tunnel encapsulation/decapsulation module developed by Pablo Neira, Andreas Schultz and myself merged into the 4.8.0 mainline kernel.

    During the second half of 2016, the code basically stayed untouched. In early 2017, several patch series of (at least) three authors have been published on the netdev mailing list for review and merge.

    This poses the very valid question on how do we test those (sometimes quite intrusive) changes. Setting up a complete cellular network with either GPRS/EGPRS or even UMTS/HSPA is possible using OsmoSGSN and related Osmocom components. But it's of course a luxury that not many Linux kernel networking hackers have, as it involves the availability of a supported GSM BTS or UMTS hNodeB. And even if that is available, there's still the issue of having a spectrum license, or a wired setup with coaxial cable.

    So as part of the recent discussions on netdev, I tested and described a minimal test setup using libgtpnl, OpenGGSN and sgsnemu.

    This setup will start a mobile station + SGSN emulator inside a Linux network namespace, which talks GTP-C to OpenGGSN on the host, as well as GTP-U to the Linux kernel GTP-U implementation.

    In case you're interested, feel free to check the following wiki page: https://osmocom.org/projects/linux-kernel-gtp-u/wiki/Basic_Testing

    This is of course just for manual testing, and for functional (not performance) testing only. It would be great if somebody would pick up on my recent mail containing some suggestions about an automatic regression testing setup for the kernel GTP-U code. I have way too many spare-time projects in desperate need of some attention to work on this myself. And unfortunately, none of the telecom operators (who are the ones benefiting most from a Free Software accelerated GTP-U implementation) seems to be interested in at least co-funding or otherwise contributing to this effort :/

    by Harald Welte at February 23, 2017 11:00 PM

    February 20, 2017

    Free Electrons

    Free Electrons and Raspberry Pi Linux kernel upstreaming

    Raspberry Pi logoFor a few months, Free Electrons has been helping the Raspberry Pi Foundation upstream to the Linux kernel a number of display related features for the Rasperry Pi platform.

    The main goal behind this upstreaming process is to get rid of the closed-source firmware that is used on non-upstream kernels every time you need to enable/access a specific hardware feature, and replace it by something that is both open-source and compliant with upstream Linux standards.

    Eric Anholt has been working hard to upstream display related features. His biggest contribution has certainly been the open-source driver for the VC4 GPU, but he also worked on the display controller side, and we were contracted to help him with this task.

    Our first objective was to add support for SDTV (composite) output, which appeared to be much easier than we imagined. As some of you might already know, the display controller of the Raspberry Pi already has a driver in the DRM subsystem. Our job was to add support for the SDTV encoder (also called VEC, for Video EnCoder). The driver has been submitted just before the 4.10 merge window and surprisingly made it into 4.10 (see also the patches). Eric Anholt explained on his blog:

    The Raspberry Pi Foundation recently started contracting with Free Electrons to give me some support on the display side of the stack. Last week I got to review and release their first big piece of work: Boris Brezillon’s code for SDTV support. I had suggested that we use this as the first project because it should have been small and self contained. It ended up that we had some clock bugs Boris had to fix, and a bug in my core VC4 CRTC code, but he got a working patch series together shockingly quickly. He did one respin for a couple more fixes once I had tested it, and it’s now out on the list waiting for devicetree maintainer review. If nothing goes wrong, we should have composite out support in 4.11 (we’re probably a week late for 4.10).

    Our second objective was to help Eric with HDMI audio support. The code has been submitted on the mailing list 2 weeks ago and will hopefully be queued for 4.12. This time on, we didn’t write much code, since Eric already did the bulk of the work. What we did though is debugging the implementation to make it work. Eric also explained on his blog:

    Probably the biggest news of the last two weeks is that Boris’s native HDMI audio driver is now on the mailing list for review. I’m hoping that we can get this merged for 4.12 (4.10 is about to be released, so we’re too late for 4.11). We’ve tested stereo audio so far, no compresesd audio (though I think it should Just Work), and >2 channel audio should be relatively small amounts of work from here. The next step on HDMI audio is to write the alsalib configuration snippets necessary to hide the weird details of HDMI audio (stereo IEC958 frames required) so that sound playback works normally for all existing userspace, which Boris should have a bit of time to work on still.

    On our side, it has been a great experience to work on such topics with Eric, and you should expect more contributions from Free Electrons for the Raspberry Pi platform in the next months, so stay tuned!

    by Boris Brezillon at February 20, 2017 04:23 PM

    February 15, 2017

    Harald Welte

    Cellular re-broadcast over satellite

    I've recently attended a seminar that (among other topics) also covered RF interference hunting. The speaker was talking about various real-world cases of RF interference and illustrating them in detail.

    Of course everyone who has any interest in RF or cellular will know about fundamental issues of radio frequency interference. To the biggest part, you have

    • cells of the same operator interfering with each other due to too frequent frequency re-use, adjacent channel interference, etc.
    • cells of different operators interfering with each other due to intermodulation products and the like
    • cells interfering with cable TV, terrestrial TV
    • DECT interfering with cells
    • cells or microwave links interfering with SAT-TV reception
    • all types of general EMC problems

    But what the speaker of this seminar covered was actually a cellular base-station being re-broadcast all over Europe via a commercial satellite (!).

    It is a well-known fact that most satellites in the sky are basically just "bent pipes", i.e. they consist of a RF receiver on one frequency, a mixer to shift the frequency, and a power amplifier. So basically whatever is sent up on one frequency to the satellite gets re-transmitted back down to earth on another frequency. This is abused by "satellite hijacking" or "transponder hijacking" and has been covered for decades in various publications.

    Ok, but how does cellular relate to this? Well, apparently some people are running VSAT terminals (bi-directional satellite terminals) with improperly shielded or broken cables/connectors. In that case, the RF emitted from a nearby cellular base station leaks into that cable, and will get amplified + up-converted by the block up-converter of that VSAT terminal.

    The bent-pipe satellite subsequently picks this signal up and re-transmits it all over its coverage area!

    I've tried to find some public documents about this, an there's surprisingly little public information about this phenomenon.

    However, I could find a slide set from SES, presented at a Satellite Interference Reduction Group: Identifying Rebroadcast (GSM)

    It describes a surprisingly manual and low-tech approach at hunting down the source of the interference by using an old nokia net-monitor phone to display the MCC/MNC/LAC/CID of the cell. Even in 2011 there were already open source projects such as airprobe that could have done the job based on sampled IF data. And I'm not even starting to consider proprietary tools.

    It should be relatively simple to have a SDR that you can tune to a given satellite transponder, and which then would look for any GSM/UMTS/LTE carrier within its spectrum and dump their identities in a fully automatic way.

    But then, maybe it really doesn't happen all that often after all to rectify such a development...

    by Harald Welte at February 15, 2017 11:00 PM

    February 13, 2017

    Free Electrons

    Power measurement with BayLibre’s ACME cape

    When working on optimizing the power consumption of a board we need a way to measure its consumption. We recently bought an ACME from BayLibre to do that.

    Overview of the ACME

    The ACME is an extension board for the BeagleBone Black, providing multi-channel power and temperature measurements capabilities. The cape itself has eight probe connectors allowing to do multi-channel measurements. Probes for USB, Jack or HE10 can be bought separately depending on boards you want to monitor.

    acme

    Last but not least, the ACME is fully open source, from the hardware to the software.

    First setup

    Ready to use pre-built images are available and can be flashed on an SD card. There are two different images: one acting as a standalone device and one providing an IIO capture daemon. While the later can be used in automated farms, we chose the standalone image which provides user-space tools to control the probes and is more suited to power consumption development topics.

    The standalone image userspace can also be built manually using Buildroot, a provided custom configuration and custom init scripts. The kernel should be built using a custom configuration and the device tree needs to be patched.

    Using the ACME

    To control the probes and get measured values the Sigrok software is used. There is currently no support to send data over the network. Because of this limitation we need to access the BeagleBone Black shell through SSH and run our commands there.

    We can display information about the detected probe, by running:

    # sigrok-cli --show --driver=baylibre-acme
    Driver functions:
        Continuous sampling
        Sample limit
        Time limit
        Sample rate
    baylibre-acme - BayLibre ACME with 3 channels: P1_ENRG_PWR P1_ENRG_CURR P1_ENRG_VOL
    Channel groups:
        Probe_1: channels P1_ENRG_PWR P1_ENRG_CURR P1_ENRG_VOL
    Supported configuration options across all channel groups:
        continuous: 
        limit_samples: 0 (current)
        limit_time: 0 (current)
        samplerate (1 Hz - 500 Hz in steps of 1 Hz)
    

    The driver has four parameters (continuous sampling, sample limit, time limit and sample rate) and has one probe attached with three channels (PWR, CURR and VOL). The acquisition parameters help configuring data acquisition by giving sampling limits or rates. The rates are given in Hertz, and should be within the 1 and 500Hz range when using an ACME.

    For example, to sample at 20Hz and display the power consumption measured by our probe P1:

    # sigrok-cli --driver=baylibre-acme --channels=P1_ENRG_PWR \
          --continuous --config samplerate=20
    FRAME-BEGIN
    P1_ENRG_PWR: 1.000000 W
    FRAME-END
    FRAME-BEGIN
    P1_ENRG_PWR: 1.210000 W
    FRAME-END
    FRAME-BEGIN
    P1_ENRG_PWR: 1.210000 W
    FRAME-END
    

    Of course there are many more options as shown in the Sigrok CLI manual.

    Beta image

    A new image is being developed and will change the way to use the ACME. As it’s already available in beta we tested it (and didn’t come back to the stable image). This new version aims to only use IIO to provide the probes data, instead of having a custom Sigrok driver. The main advantage is many software are IIO aware, or will be, as it’s the standard way to use this kind of sensors with the Linux kernel. Last but not least, IIO provides ways to communicate over the network.

    A new webpage is available to find information on how to use the beta image, on https://baylibre-acme.github.io. This image isn’t compatible with the current stable one, which we previously described.

    The first nice thing to notice when using the beta image is the Bonjour support which helps us communicating with the board in an effortless way:

    $ ping baylibre-acme.local
    

    A new tool, acme-cli, is provided to control the probes to switch them on or off given the needs. To switch on or off the first probe:

    $ ./acme-cli switch_on 1
    $ ./acme-cli switch_off 1
    

    We do not need any additional custom software to use the board, as the sensors data is available using the IIO interface. This means we should be able to use any IIO aware tool to gather the power consumption values:

    • Sigrok, on the laptop/machine this time as IIO is able to communicate over the network;
    • libiio/examples, which provides the iio-monitor tool;
    • iio-capture, which is a fork of iio-readdev designed by BayLibre for an integration into LAVA (automated tests);
    • and many more..

    Conclusion

    We didn’t use all the possibilities offered by the ACME cape yet but so far it helped us a lot when working on power consumption related topics. The ACME cape is simple to use and comes with a working pre-built image. The beta image offers the IIO support which improved the usability of the device, and even though it’s in a beta version we would recommend to use it.

    by Antoine Ténart at February 13, 2017 03:38 PM

    February 12, 2017

    Harald Welte

    Towards a real SIGTRAN/SS7 stack in libosmo-sigtran

    In the good old days ever since the late 1980ies - and a surprising amount even still today - telecom signaling traffic is still carried over circuit-switched SS7 with its TDM lines as physical layer, and not an IP/Ethernet based transport.

    When Holger first created OsmoBSC, the BSC-only version of OpenBSC some 7-8 years ago, he needed to implement a minimal subset of SCCP wrapped in TCP called SCCP Lite. This was due to the simple fact that the MSC to which it should operate implemented this non-standard protocol stacking that was developed + deployed before the IETF SIGTRAN WG specified M3UA or SUA came around. But even after those were specified in 2004, the 3GPP didn't specify how to carry A over IP in a standard way until the end of 2008, when a first A interface over IP study was released.

    As time passese, more modern MSCs of course still implement classic circuit-switched SS7, but appear to have dropped SCCPlite in favor of real AoIP as specified by 3GPP meanwhile. So it's time to add this to the osmocom universe and OsmoBSC.

    A couple of years ago (2010-2013) implemented both classic SS7 (MTP2/MTP3/SCCP) as well as SIGTRAN stackings (M2PA/M2UA/M3UA/SUA in Erlang. The result has been used in some production deployments, but only with a relatively limited feature set. Unfortunately, this code has nto received any contributions in the time since, and I have to say that as an open source community project, it has failed. Also, while Erlang might be fine for core network equipment, running it on a BSC really is overkill. Keep in miond that we often run OpenBSC on really small ARM926EJS based embedded systems, much more resource constrained than any single smartphone during the late decade.

    In the meantime (2015/2016) we also implemented some minimal SUA support for interfacing with UMTS femto/small cells via Iuh (see OsmoHNBGW).

    So in order to proceed to implement the required SCCP-over-M3UA-over-SCTP stacking, I originally thought well, take Holgers old SCCP code, remove it from the IPA multiplex below, stack it on top of a new M3UA codebase that is copied partially from SUA.

    However, this falls short of the goals in several ways:

    • The application shouldn't care whether it runs on top of SUA or SCCP, it should use a unified interface towards the SCCP Provider. OsmoHNBGW and the SUA code already introduce such an interface baed on the SCCP-User-SAP implemented using Osmocom primitives (osmo_prim). However, the old OsmoBSC/SCCPlite code doesn't have such abstraction.
    • The code should be modular and reusable for other SIGTRAN stackings as required in the future

    So I found myself sketching out what needs to be done and I ended up pretty much with a re-implementation of large parts. Not quite fun, but definitely worth it.

    The strategy is:

    And then finally stack all those bits on top of each other, rendering a fairly clean and modern implementation that can be used with the IuCS of the virtually unmodified OsmmoHNBGW, OsmoCSCN and OsmoSGSN for testing.

    Next steps in the direction of the AoIP are:

    • Implementation of the MTP-SAP based on the IPA transport
    • Binding the new SCCP code on top of that
    • Converting OsmoBSC code base to use the SCCP-User-SAP for its signaling connection

    From that point onwards, OsmoBSC doesn't care anymore whether it transports the BSSAP/BSSMAP messages of the A interface over SCCP/IPA/TCP/IP (SCCPlite) SCCP/M3UA/SCTP/IP (3GPP AoIP), or even something like SUA/SCTP/IP.

    However, the 3GPP AoIP specs (unlike SCCPlite) actually modify the BSSAP/BSSMAP payload. Rather than using Circuit Identifier Codes and then mapping the CICs to UDP ports based on some secret conventions, they actually encapsulate the IP address and UDP port information for the RTP streams. This is of course the cleaner and more flexible approach, but it means we'll have to do some further changes inside the actual BSC code to accommodate this.

    by Harald Welte at February 12, 2017 11:00 PM

    February 11, 2017

    Harald Welte

    Testing (not only) telecom protocols

    When implementing any kind of communication protocol, one always dreams of some existing test suite that one can simply run against the implementation to check if it performs correct in at least those use cases that matter to the given application.

    Of course in the real world, there rarely are protocols where this is true. If test specifications exist at all, they are often just very abstract texts for human consumption that you as the reader should implement yourself.

    For some (by far not all) of the protocols found in cellular networks, every so often I have seen some formal/abstract machine-parseable test specifications. Sometimes it was TTCN-2, and sometimes TTCN-3.

    If you haven't heard about TTCN-3, it is basically a way to create functional tests in an abstract description (textual + graphical), and then compile that into an actual executable tests suite that you can run against the implementation under test.

    However, when I last did some research into this several years ago, I couldn't find any Free / Open Source tools to actually use those formally specified test suites. This is not a big surprise, as even much more fundamental tools for many telecom protocols are missing, such as good/complete ASN.1 compilers, or even CSN.1 compilers.

    To my big surprise I now discovered that Ericsson had released their (formerly internal) TITAN TTCN3 Toolset as Free / Open Source Software under EPL 1.0. The project is even part of the Eclipse Foundation. Now I'm certainly not a friend of Java or Eclipse by all means, but well, for running tests I'd certainly not complain.

    The project also doesn't seem like it was a one-time code-drop but seems very active with many repositories on gitub. For example for the core module, titan.core shows plenty of activity on an almost daily basis. Also, binary releases for a variety of distributions are made available. They even have a video showing the installation ;)

    If you're curious about TTCN-3 and TITAN, Ericsson also have made available a great 200+ pages slide set about TTCN-3 and TITAN.

    I haven't yet had time to play with it, but it definitely is rather high on my TODO list to try.

    ETSI provides a couple of test suites in TTCN-3 for protocols like DIAMETER, GTP2-C, DMR, IPv6, S1AP, LTE-NAS, 6LoWPAN, SIP, and others at http://forge.etsi.org/websvn/ (It's also the first time I've seen that ETSI has a SVN server. Everyone else is using git these days, but yes, revision control systems rather than periodic ZIP files is definitely a big progress. They should do that for their reference codecs and ASN.1 files, too.

    I'm not sure once I'll get around to it. Sadly, there is no TTCN-3 for SCCP, SUA, M3UA or any SIGTRAN related stuff, otherwise I would want to try it right away. But it definitely seems like a very interesting technology (and tool).

    by Harald Welte at February 11, 2017 11:00 PM

    February 10, 2017

    Harald Welte

    FOSDEM 2017

    Last weekend I had the pleasure of attending FOSDEM 2017. For many years, it is probably the most exciting event exclusively on Free Software to attend every year.

    My personal highlights (next to meeting plenty of old and new friends) in terms of the talks were:

    I was attending but not so excited by Georg Greve's OpenPOWER talk. It was a great talk, and it is an important topic, but the engineer in me would have hoped for some actual beefy technical stuff. But well, I was just not the right audience. I had heard about OpenPOWER quite some time ago and have been following it from a distance.

    The LoRaWAN talk couldn't have been any less technical, despite stating technical, political and cultural in the topic. But then, well, just recently 33C3 had the most exciting LoRa PHY Reverse Engineering Talk by Matt Knight.

    Other talks whose recordings I still want to watch one of these days:

    by Harald Welte at February 10, 2017 11:00 PM

    February 05, 2017

    Andrew Zonenberg, Silicon Exposed

    STARSHIPRAIDER: Input buffer rev 0.1 design and characterization

    Working as an embedded systems pentester is a lot of fun, but it comes with some annoying problems. There's so many tools that I can never seem to find the right one. Need to talk to a 3.3V UART? I almost invariably have an FTDI cable configured for 5 or 1.8V on my desk instead. Need to dump a 1.8V flash chip? Most of our flash dumpers won't run below 3.3. Need to sniff a high-speed bus? Most of the Saleae Logic analyzers floating around the lab are too slow to keep up with fast signals, and the nice oscilloscopes don't have a lot of channels. And everyone's favorite jack-of-all-trades tool, the Bus Pirate, is infamous for being slow.

    As someone with no shortage of virtual razors, I decided that this yak needed to be shaved! The result was an ongoing project I call STARSHIPRAIDER. There will be more posts on the project in the coming months so stay tuned!

    The first step was to decide on a series of requirements for the project:
    • 32 bidirectional I/O ports split into four 8-pin banks.
      This is enough to sniff any commonly encountered embedded bus other than DRAM. Multiple banks are needed to support multiple voltage levels in the same target.
    • Full support for 1.2 to 5V logic levels.This is supposed to be a "Swiss Army knife" embedded systems debug/testing tool. This voltage range encompasses pretty much any signalling voltage commonly encountered in embedded devices.
    • Tolerance to +/- 12V DC levels.Test equipment needs to handle some level of abuse. When you're reverse engineering a board it's easy to hook up ground to the wrong signal, probe a power rail, or even do both at once. The device doesn't have to function in this state (shutting down for protection is OK) but needs to not suffer permanent damage. It's also OK if the protection doesn't handle AC sources - the odds of accidentally connecting a piece of digital test equipment to a big RF power amplifier are low enough that I'm not worried.
    • 500 Mbps input/output rate for each pin.This was a somewhat arbitrary choice, but preliminary math indicated it was feasible. I wanted something significantly faster than existing tools in the class.
    • Ethernet-based interface to host PC.I've become a huge fan of Ethernet and IPv6 as communications interface for my projects. It doesn't require any royalties or license fees, scales from 10 Mbps to >10 Gbps and supports bridging between different link speeds, supports multi-master topologies, and can be bridged over a WAN or VPN. USB and PCIe, the two main alternatives, can do few if any of these.
    • Large data buffer.Most USB logic analyzers have very high peak capture rates, but the back-haul interface to the host PC can't keep up with extended captures at high speed. Commodity DRAM is so cheap that there's no reason to not stick a whole SODIMM of DDR3 in the instrument to provide an extremely deep capture buffer.
    • Multiple virtual instruments connected to a crossbar.Any nontrivial embedded device contains multiple buses of interest to a reverse engineer. STARSHIPRAIDER needs to be able to connect to several at once (on arbitrary pins), bridge them out to separate TCP ports, and allow multiple testers to send test vectors to them independently.
    The brain of the system will be fairly straightforward high-speed digital. It will be a 6-8 layer PCB with an Artix-7 FPGA in FGG484 package, a SODIMM socket for 4GB of DDR3 800, a KSZ9031 Gigabit Ethernet PHY, a TLK10232 10gbit Ethernet PHY, and a SFP+ cage, plus some sort of connector (most likely a Samtec Q-strip) for talking to the I/O subsystem on a separate board.

    The challenging part of the design, from an architectural perspective, seemed to be the I/O buffer and input protection circuit, so I decided to prototype it first.

    STARSHIPRAIDER v0.1 I/O buffer design

    A block diagram of the initial buffer design is shown above. The output buffer will be discussed in a separate post once I've had a chance to test it; today we'll be focusing on the input stage (the top half of the diagram).

    During normal operation, the protection relay is closed. The series resistor has insignificant resistance compared to the input impedance of the comparator (an ADCMP607), so it can be largely ignored. The comparator checks the input signal against a threshold (chosen appropriately for the I/O standard in use) and sends a differential signal to the host board for processing. But what if something goes wrong?

    If the user accidentally connects the probe to a signal outside the acceptable voltage range, a Schottky diode connected to the +5V or ground rail will conduct and shunt the excess voltage safely into the power rails. The series resistor limits fault current to a safe level (below the diode's peak power rating). After a short time (about 150 µs with my current relay driver), the protection relay opens and breaks the circuit.

    The relay is controlled by a Silego GreenPAK4 mixed-signal FPGA, running a small design written in Verilog and compiled with my open-source toolchain. The code for the GreenPAK design is on Github.

    All well and good in theory... but does it work? I built a characterization board containing a single I/O buffer and loaded with test points and probe connectors. You can grab the KiCAD files for this on Github as well. Here's a quick pic after assembly:

    STARSHIPRAIDER I/O characterization board
    Initial test results were not encouraging. Positive overvoltage spikes were clamped to +8V and negative spikes were clamped to -1V - well outside the -0.5 to +6V absolute max range of my comparator.
    Positive transient response

    Negative transient response


    After a bit of review of the schematics, I found two errors. The "5V" ESD diode I was using to protect the high side had a poorly controlled Zener voltage and could clamp as high as 8V or 9V. The Schottky on the low side was able to survive my fault current but the forward voltage increased massively beyond the nominal value.

    I reworked the board to replace the series resistor with a larger one (39 ohms) to reduce the maximum fault current, replaced the low-side Schottky with one that could handle more current, and replaced the Zener with an identical Schottky clamping to the +5V rail.

    Testing this version gave much better results. There was still a small amount of ringing (less than five nanoseconds) a few hundred mV past the limit, but the comparator's ESD diodes should be able to safely dissipate this brief pulse.

    Positive transient response, after rework
    Negative transient response, after rework
    Now it was time to test the actual signal path. My first iteration of the test involved cobbling together a signal path from an FPGA board through the test platform and to the oscilloscope without any termination. The source of the signal was a BNC-to-minigrabber flying lead test clip! Needless to say, results were less than stellar.

    PRBS31 eye at 80 Mbps through protection circuit with flying leads and no terminator
    After ordering some proper RF test supplies (like an inline 50 ohm BNC terminator), I got much better signal quality. The eye was very sharp and clear at 100 Mbps. It was visibly rounded at 200 Mbps, but rendering a squarewave at that rate requires bandwith much higher than the 100 MHz of my oscilloscope so results were inconclusive.

    PRBS31 eye at 100 Mbps through protection circuit with proper cabling
    PRBS31 eye at 200 Mbps, limited by oscilloscope bandwidth
    I then hooked the protection circuit up to the comparator to test the entire inbound signal chain. While the eye looked pretty good at 100 Mbps (plotting one leg of the differential since my scope was out of channels), at 200 Mbps horrible jitter appeared.

    PRBS31 eye at 100 Mbps through full input buffer
    PRBS31 eye at 200 Mbps through full input buffer
    After quite a bit of scratching my head and fumbling with datasheets, I realized my oscilloscope was the problem by plotting the clock reference I was triggering on. The jitter was visible in this clock as well, suggesting that it was inherent in the oscilloscope's trigger circuit. This isn't too surprising considering I'm really pushing the limits of this scope - I need a better one to do this kind of testing properly.

    PRBS31 eye at 200 Mbps plus 200 MHz sync clock
     At this point I've done about all of the input stage testing I can do with this oscilloscope. I'm going to try and rig up a BER tester on the FPGA so I can do PRBS loopback through the protection stage and comparator at higher speeds, then repeat for the output buffer and the protection run in the opposite direction.

    I still have more work to do on the protection circuit as well... while it's fine at 100 Mbps, the 2x 10pF Schottky diode parasitic capacitance is seriously degrading my rise times (I calculated an RC filter -3dB point of around 200 MHz, so higher harmonics are being chopped off). I have some ideas on how I can cut this down much less but that will require a board respin and another blog post!

    by Andrew Zonenberg (noreply@blogger.com) at February 05, 2017 10:20 AM

    February 03, 2017

    Free Electrons

    Video and slides from Linux Conf Australia

    Linux Conf Australia took place two weeks ago in Hobart, Tasmania. For the second time, a Free Electrons engineer gave a talk at this conference: for this edition, Free Electrons CTO Thomas Petazzoni did a talk titled A tour of the ARM architecture and its Linux support. This talk was intended as an introduction-level talk to explain what is ARM, what is the concept behind the ARM architecture and ARM System-on-chip, bootloaders typically used on ARM and the Linux support for ARM with the concept of Device Tree.