copyleft hardware planet

September 26, 2016

Village Telco

SECN 4.0 Firmware Available

mp2_phone_resetThe fourth release of the Small Enterprise / Campus Network (SECN) firmware for MP02, Ubiquity and TP Link devices, designed to provide combined telephony and data network solutions is now available for download.

The major features of this update are:

  • Updated OpenWrt version to Chaos Calmer version
  • Updated stable batman-adv  mesh software to version 2016.1
  • Added factory restore function from Hardware Reset button

Unless you are running a network with some of the first generation Mesh Potatoes, you should consider upgrading to this firmware.   The new factory reset function is particularly handy in that any device can be reset to its factory firmware settings by holding down the reset button for 15 seconds.

Stable firmware is available here:

MP02 –  http://download.villagetelco.org/firmware/secn/stable/mp-02/SECN_4/
TP-Link – http://download.villagetelco.org/firmware/secn/stable/tp-link/SECN_4/
Ubiquiti – http://download.villagetelco.org/firmware/secn/stable/ubnt/SECN_4/

Please subscribe to the Village Telco community development list if you have questions or suggestions.

by steve at September 26, 2016 04:25 PM

September 25, 2016

Bunnie Studios

Name that Ware, September 2016

The Ware for September 2016 is shown below.

Thanks to J. Peterson for sharing this ware!

by bunnie at September 25, 2016 09:45 AM

Winner, Name that Ware August 2016

After reading through the extensive comments on August’s ware, I’m not convinced anyone has conclusively identified the ware. I did crack a grin at atomicthumbs’ suggestion that this was a “mainboard from a Mrs. Butterworth’s Syrup of Things sensor platform”, but I think I’ll give the prize (please email me to claim it) once again to Christian Vogel for his thoughtful analysis of the circuitry, and possibly correct guess that this might be an old school laser barcode scanner.

The ware is difficult to evaluate due to the lack of a key component — whatever it is that mounts into the pin sockets and interacts with the coil or transformer near the hole in the center of the circuit board. My feeling is the placement of that magnetic device is not accidental.

A little bit of poking around revealed this short Youtube video which purports to demonstrate an old-school laser barcode mechanism. Significantly, it has a coil of similar shape and orientation to that of this ware, as well as three trimpots, although that could be a coincidence. Either way, thanks everyone for the entertaining and thoughtful comments!

by bunnie at September 25, 2016 09:45 AM

September 19, 2016

Elphel

NC393 development progress and the future plans

Since we started to deliver first NC393 series cameras in May we were working on the cameras software – original version was rather limited. While it was capable of serving images/video over the network and recording them on the internal m.2 SSD, it did not have the advanced image acquisition control (through the GUI and programmatically) that was standard for the earlier NC353 series. Now the core functionality is operational and in a month we plan to have the remaining parts (inter-camera synchronization, working with multiple sensors per-port with 10359 multiplexer, GPS+IMU logging) online too. FPGA code is already ported, but it needs to be tested and a fair amount of troubleshooting, identifying the problems and weeding out the bugs is still left to be done.

Fig 1. Four camvc instances for four channels of NC393 camera

Fig 1. Four camvc instances for the four channels of NC393 camera

Users of earlier Elphel cameras can easily recognize familiar camvc web interface – Fig. 1 shows a screenshot of the four instances of this interface controlling 4 sensors of NC393 camera in “H” configuration.

This web application tests multiple underlaying pieces of software in the camera: FPGA code, Linux kernel drivers that control the low level of the camera operation and are handling 8 interrupts from the imaging subsystem (NC353 camera processor had just one), PHP extension to interact with the drivers, image server, histograms visualization program, autoexposure and white balance daemons as well as multiple PHP scripts and Javascript code. Luckily, the higher the level, the less changes we needed in the code from the NC353 (in most cases just a single new parameter – sensor port had to be introduced), but the debugging process included going through all the levels of code – bug chasing could start from Javascript code, go to PHP code, then to PHP extension, to kernel driver, direct FPGA control from the Python code (bypassing drivers), simulating Verilog code with Cocotb. Then, when the problem was identified and the HDL code corrected (it usually required several more iterations with simulation), the top level programs were tested again with the new FPGA bitstream. And this is the time when the integration of all the development in the same Eclipse IDE is really paying off – easy code navigation, making changes to different language programs – and the software was rebuilding and transferring the results to the target system automatically.

Camera core software

NC393 camera software aims the same goals as the previous models – allow the full speed operation of the imagers while minimizing real-time requirements to the software on the two levels:

  • kernel level (tolerate large delays when waiting for the interrupts to be served) and
  • application level – allow even scripting languages to keep up with the hardware

Interrupt latency is usually not a problem what working with full frame multi-megapixel images, but the camera can operate a small window at high FPS too. Many operations with the sensor (like changing resolution or image size) require coordinated updating sensor internal registers (usually over I²C connection), changing parameters of the sensor-to-memory FPGA channel (with appropriate latency), parameters of the memory-to-compressor channel, and parameters of the compressor itself. Additionally the camera software should provide the modified image headers (reflecting the new window size) when the acquired image will be recorded or requested over the network.

Application software just needs to tell when (at what frame number) it needs the new window size and the kernel plus FPGA code will take care of the rest. Slow software should just tell in advance so the camera code and the sensor itself will have enough time to execute the request. Multiple parameters modifications designated for a specific frame will be applied almost simultaneously even if frame sync pulses where received from the sensor while application was sending the new data.

Image-derived data remains available long after the image is acquired

Similar things happen with the data received from the sensor – image itself and histograms (they are used for the automatic exposure adjustment and white balancing). Application does not need to to read them before the next frame data arrives – compressed images are kept in a large (64MB per port) ring buffer in the system memory – it can keep record of several seconds of images. Histograms (for up to 4 different windows inside the full image for each sensor port) are preserved for 15 frames after being acquired and transferred over DMA to the system memory. Subset of essential acquisition parameters and image metadata (needed for Exif output) are preserved for 2048 and 511 frames respectively.

Fig 2. Interaction of the image sensor, FPGA, kernel drivers and user space applications

Fig 2. Interaction of the image sensor, FPGA, kernel drivers and user space applications

FPGA frame-based command sequencers

There are 2 sequencers for each of the four sensor ports on the FPGA level – they do not use any of the CPU resources:

  • I²C sequencers handle relatively slow i2c commands to be sent to the senor, usually these commands need to arrive before start of the next frame,
  • Command sequencers perform writes to the memory-mapped registers and so control the FPGA operation. These operations need to happen in guaranteed time just after the start of frame, before the corresponding subsystems begin to process the incoming image data.

Both are synchronized by the “start of frame” signals from the sensor, each sequencer has 16 frame pages, each page contains 64 command slots.

Sequencers allow absolute (modulo 16) frame address and relative (to current) frame address. Writing to the current frame (zero offset) is interpreted as “ASAP” and the commands are issued immediately, not synchronized by the start of frame. Additionally, if the commands were written too late and the frame sync arrived before they were executed, they will still be processed before the next frame slot page is activated.

Kernel support of the image frame parameters

There are many frame-related parameters that control image acquisition in the camera, including various sensor register settings, parameters that control gamma conversion, image format for recording to video memory (dedicated to FPGA DDR3 not shared with the CPU), compressor format, signal gains, color saturations, compression quality, coring parameters, histogram windows size and position. There is no such thing as the “current frame parameters” in the camera, at any given moment the sensor may be programmed for a certain image size, while its output data reflects the previous frame format, and the compressor is still not finished with even earlier image. That means that the camera should be aware of multiple sets of the same parameters, each applicable to a certain frame (identified by an absolute frame number). In that case the sensor “now” is receiving not the “current” frame parameters, but the frame parameters of a frame that will happen 2 frame intervals later.

Current implementation keeps parameters (all parameters are unsigned long) in a 16-element ring buffer, each element being a

/** Parameters block, maintained for each frame (0..15 in NC393) of each sensor channel */
struct framepars_t {
        unsigned long pars[927];      ///< parameter values (indexed by P_* constants)
        unsigned long functions;      ///< each bit specifies function to be executed (triggered by some parameters change)
        unsigned long modsince[31];   ///< parameters modified after this frame - each bit corresponds to one element in in par[960] (bit 31 is not used)
        unsigned long modsince32;     ///< parameters modified after this frame super index - non-zero elements in in mod[31]  (bit 31 is not used)
        unsigned long mod[31];        ///< modified parameters - each bit corresponds to one element in in par[960] (bit 31 is not used)
        unsigned long mod32;          ///< super index - non-zero elements in in mod[31]  (bit 31 is not used)
};

Interrupt driven processing of the parameters take CPU time (in contrast with the FPGA sequencers described before), so the processing should be efficient and not iterate through almost a thousand entries for each interrupt, It is also not practical to copy a full set of parameters from the previous frame. Parameters structure for each frame include mod[31] array where each element stores a bit field that describes modification of the 32 consecutive parameters, and a single mod32 represents each mod as a single bit. So mod32 == 0 means that there were no changes (as is true for the majority of frames) and there is nothing to do for the interrupt service routine. Additional fields modsince[31] and modsince32 mean that there were changes to the parameter after this frame. It is used to initialize a new (15 frames ahead of “now”) frame entry in the ring buffer. The buffer is modulo 16, so parameters for [this_frame + 15] share the same memory address as [this_frame-1], and if the parameter is not “modified since” (as is true for the majority of parameters), nothing has to be done for it when advancing this_frame.

There is a configurable parameter that tells parameter processing at interrupts how far to look ahead in the future (Fig.2 shows frames that are too far in the future hatched). The function starts with the current frame and proceeds in the future (up to the specified limit) looking for modified, but not yet processed parameters. Processing of the modified parameters involves calling of up to 32 “generic”(sensor-agnostic) functions and up to 32 their sensor-specific variants. Each parameter that triggers some action if modified is assigned a bitmask of functions to schedule on change, and when the parameter is written to buffer, the functions field for the frame is OR-ed, so during the interrupt only this single field has to be considered.

Processing parameters in a frame scans all the bits in functions (in defined order, starting from the LSB, generic first), the functions involve verification and calculation of derivative values, writing data to the FPGA command and I²C sequencers (deep green and blue on Fig. 2 show the new added commands to the sequencers). Additionally some actions may schedule other parameters changes to be processed at later frame.

User space applications and the frame parameters

Application see frame parameters through the character device driver that supports write, mmap, and (overloaded) lseek.

  • write operation allows to set a list of parameters and apply these changes to a particular frame as a single transaction
  • mmap provides read access to all the frame parameters for up to 15 frames in the future, parameter defines are provided through the header files under kernel include/uapi, so applications (such as PHP extension) can access them by symbolic names.
  • lseek is heavily overloaded, especially for positive offsets to SEEK_END – such commands initiate special actions in this driver, such as waiting for the specific frame. It is partially used instead of the ioctl command, because lseek is immediately supported in most languages while ioctl often requires special extensions.

Communicating image data to the user space

Similar to handling of the frame acquisition and processing parameters, that deals with the future and lets even slow applications to control the process being frame-accurate, other kernel drivers use the FPGA code features to give applications sufficient time to process acquired data before it is overwritten by the newer one. These drivers use similar character device interface with mmap for data access and lseek for control, some use write to send data to the driver.

  • circbuf driver provides access to the compressed image data in any of the four 64MB ring buffers that contain compressed by the FPGA data (FPGA also provides the microsecond-accurate timestmap and the image size). Each image is 32-byte aligned, FPGA skips additional 32 bytes after each frame. Compressor interrupt service routine (located in sensor_common.c) fills this area with some of the image acquisition metadata.
  • histograms driver handles the histograms for the acquired images. Histograms are calculated in the FPGA on the image-to-memory path and so are active even if compressor is stopped. There are 3 types of histogram data that may be needed by the applications, and only the first one (direct) is provided by the FPGA over DMA, two others (derivative) are calculated in the driver and cached, so application request for the same derivative histogram does not require re-calculation. Histograms are calculated for the pixels after gamma-conversion even if raw (2 bytes/pixel) data is recorded, so table indices are always in the range of 0 to 255.
    • direct histograms are provided by the FPGA that maintains data for 16 consecutive (last acquired) frames, for each of the 4 color channels (2 separate green ones), for each of the sensor ports and sub-channels (when multiplexers are used). Each frame data contain 256*4=1024 of the unsigned long (32 bit) values.
    • cumulative histograms contain the corresponding cumulative values, each element equals to sum of the direct histogram values from 0 to the specified index. When divided by the value at index 255 (total number of pixel of this color channel =1/4 of all pixels in WOI) the result will tell what part of all pixels have values less or equal to the current.
    • percentiles are reversed cumulative histograms, they tell what is the pixel level for which a certain fraction of all pixels has a value of equal or below it. These values refer to non-linear (gamma-converted) pixel values, so automatic exposure also uses reversed gamma tables and does interpolation between the two values in the percentile table.
  • jpeghead driver generates JPEG/JP4 headers that need to be concatenated with the compressed output from circbuf (and with the end-of-image 0xff/0xd9 marker) to make a complete image file
  • exif driver manipulates Exif data in the camera – it stores Exif frame-variable data for the last acquired frames in a 512-element ring buffer, allows to specify and set additional Exif fields, provides mmap read access to the metadata.

Camera applications

Current applications include

  • Elphel PHP extension allows multiple PHP scripts to work in the camera, providing server-side of the web applications functionality, such as camvc.
  • imgsrv is a fast image server that bypasses camera web server and transfers images and metadata avoiding any copying of extra data – network controller sends data over DMA from the same buffer where FPGA delivered compressed data (also over DMA). Each sensor port has a corresponding instance of imgsrv, serving different network ports.
  • camogm allows simultaneous recording image data from multiple channels at up to 220 MB/s
  • autoexposure is an auto exposure and white balance daemon that uses image histograms for the specified WOI to adjust exposure time, sensor analog gains and signal gain coefficients in the FPGA.
  • pnghist is a CGI program that visualizes histograms as PNG images, it supports several histogram presentation modes.

Other applications that were available in the earlier NC353 series cameras (such as RTP/RTSP video streamer) will be ported shortly.

Future plans

NC393 camera has 12 times higher performance than the earlier NC353 series, and porting of the functionality of the NC353 is much more than just tweaking of the FPGA code and the drivers – large portions had to be redesigned completely. Camera FPGA project includes provisions for advanced image processing, and that changed the foundation of the camera code. That said, it is much more exciting to move forward and implement functionality that did not exist before, but we had to finish that “boring” part first. And as now it is coming closer, I would like to share our future development plans and invite others who may be interested to cooperate.

New sensors

NC393 was designed to have maximal flexibility in the sensor interface – this we learned from our experience with 303-313-333-353 series of cameras. So far NC393 is tested with one parallel interface sensor and one with a 4-lane HiSPI interface (both have links to the circuit diagrams). Each port can use 8 lanes+clock (9 differential) pairs and several more control/clock signals. Larger/faster sensors may use multiple sensors ports and so multiply available interface lines.
It will be interesting to try high sensitivity large pixel E2V sensors and ToF technology. TI OPT8241 seems to be a good fit for NC393, but OPT8241 I²C register map is not provided.

Quadcopters flying Star Wars style

Most quadcopters use brushless DC motors (BLDC) that maybe tricky to control. Integrated motor controllers that detect rotor position using the voltage on the power coils or external sensors (and so emulate ancient physical brushes) work fine when you apply only moderate variations to the rotation speed but may fail if you need to change the output fast and in precisely calculated manner. FPGA can handle such calculations better and leave CPU resources for the high level tasks. I would imagine such motor control to include some tiny FPGA paired with the high-current MOSFET drivers attached to the motors. Then use lightweight SATA cables (such as 3m 5602 series) to connect them to the NC393 daughter board. NC393 already has dual ARM CPU so it can use existing free software to fly drones and take video/images at the same time. Making it not just fly, but do “tricks” will be really exciting.

Image processing and High Level Synthesis (HLS) alternative

NC393 FPGA design started around a 16-channel memory access optimized for 2d data. Common memory may be not the most modern approach to parallel processing, but when the bulk memory (0.5GB of the DDR3) is a single device, it has to be shared between the channels and not all the module connection can be converted to simple stream protocols. Even before we started to add image processing, we have to maintain two separate bitstreams – one for the parallel sensors, and the other – for HiSPI (serial) ones. They can not be made run-time programmable as even the voltage levels are different, to say nothing that both interfaces together will not fit into Zynq FPGA – we already balancing around 80% of the slice utilization. Theoretically NC393 can use two of the parallel and 2 serial sensors (two pairs of sensor ports use two separate I/O banks with individually programmable supply voltage), but that adds even more variants to the top level module configuration and matching constraints files, and makes the code less readable.

Things will get even more complicated when there will be more active memory channels involved in the processing, especially when the inter-synchronization of the different modules processing multi-sensor 2d data is more complex than just stream in/stream out.

When processing muti-view scenes we will start with de-warping followed by FFT to implement correlation between the 4 simultaneous images and so significantly reduce ambiguity of a stereo-pair correlation. In parallel with working on Verilog code for the new modules I plan to try to reduce the complexity of the inter-module connections, making it more flexible and easier to maintain. I would love to use something higher level, but unfortunately there is nothing for me to embrace and use.

Why I do not believe in HLS

Focusing on the algorithmic level and leaving RTL implementation to the software is definitely a good idea, but the task is much more ambitious than to try to replace GCC or GNU/Linux operating system that even most proprietary and encryption-loving companies have to use. The gap between the algorithms and RTL code is wider than between the C code and the Assembler for the CPU, regardless of some nice demos with the Sobel filter applied to the live video stream or similar simple processing.

One of the major handicaps of the existing approach is an obsession with making modern reprogrammable FPGA code mimic the fixed-function hardware integrated circuits popular in the last century. To be software-like is much more powerful than to look like some old hardware. It is sure that separation of the application levels, use of the standard APIs are important, but it is most beneficial in the mature areas. In the new ones I consider it to be a beauty of coding to be able to freely cross any implementation levels, break some good programming practices, adjust it here and there, redesign and start over, balance overall performance and structure to create something new. Features and interfaces freeze will come later.

So what to use instead?

I do not yet know what it should be exactly, but I would borrow Python decorators and functionality of Verilog generate operators. Instead of just instantiating “black boxes” with rigid interfaces – allow the wrapper code (both automatically generated and hand-crafted) to get inside the instantiated modules code and modify it for the particular instances. “Decoration” meaning generation of the modified module code for the specific instances. Something like programmatic parametrization (modifying code, not just the parameter values, even those that direct generate operators).

Elphel FPGA code is source code based, there are zero of the “black boxes” in the design. And as all the code (109579 lines of it) is available it is accessible for the software too, and “robots” can analyze it and make it easier to manage. We would like to have them as “helpers” not as “wizards” who can offer just a few choices among the pre-programmed options.

To some extend we already do have such “helpers” – our current Python code “understands” Verilog parameter definitions in the source code, including some calculations of the derivative ones. That makes it possible for the Python programs running in the camera to use the same register addresses and bit fields as defined for the FPGA code implemented in the current bitstream.

When the cameras became capable of running FPGA code controlled by the Python program and we were ready to develop kernel drivers, we added extra functionality to the existing Python code. Now it is able not just to read Verilog parameters for itself, but also to generate C code to facilitate drivers development. This converter is not a compiler-like program that takes Verilog input and generates C header files. It is still a human-coded program that retrieves the parameters values from the Verilog code and helps developer by using familiar content-assist functionality of the IDE, detects and flags misspelled parameter names in PyDev (Eclipse IDE plugin for Python), re-generates output when the Verilog source is modified.

We also used Python to generate Verilog code for AHCI implementation, it seemed more convenient than native Verilog generate. Wrapping Verilog in Python and generating clean (for human analysis) Verilog code that can be used in wave viewer and in implementation tools timing analysis. It will be quite natural to make the Python programs understand more of Verilog code and help us manage the structure, generate matching constraints files that FPGA implementation tools require in addition to the HDL code. FPGA professionals probably use TCL scripts for that, it may be a nice language but I never used it outside of the FPGA scripting, so it is always a problem for me to recall how to use it when coming back to FPGA coding after long interruptions.

I did look at MyHDL of course, but it is not exactly what I need. MyHDL tries to replace Verilog completely and the structural modeling part of it suffers from the focus on RTL. I just want Python to help me with Verilog code, not to replace it (similar to how I do not think that Verilog is the best language to simulate CPU activities). I love Cocotb more – even its gentle name (COroutine based COsimulation) tells me that it is not “instead of” but “in addition to”. Cocotb does not have a ready solution for this project either (it was never a goal of this program) so here is an interesting project to implement.

There are several specific cases that I would like to be handled by the implementation.

  • add new functionally horizontal connections in a clean way between hierarchical objects: add outputs all the way up to the common parent module, wires at the top, and then inputs all the way down to the destination. Of course it is usually better to avoid such extra connections, but their traces in module ports help to keep them under control. Such connections may be just temporary and later removed, or be a start of adding new functionality to the involved modules.
  • generate a low footprint debug network to selected hierarchical modules and generate target Python code to probe/modify registers through this network accessing data by the HDL hierarchical names.
  • control the destiny of the decorators – either keep them as separate pieces of code or merge with the original source and make the result HDL code a new “co-designed” source.

And this is what I plan to start with (in parallel to adding new Verilog code). Try to combine existing pieces of the solution and make it a complete one.

by Andrey Filippov at September 19, 2016 07:41 PM

September 13, 2016

Elphel

Reaching 220 MB/s sustained write speed with SATA-2 controller

Introduction

Elphel cameras use camogm, a user space application, for recording acquired images to a disk storage. The application is developed to use such storage devices as disk drives or USB drives mounted in the operating system. The Elphel393 model cameras have SATA-2 controller implemented in FPGA, a system driver for this controller, and they can be equipped with an SSD drive. We were interested in performing write speed tests using the SATA controller and a couple of M.2 SSDs to find out the top disk bandwidth camogm can use during image recording. Our initial approach was to try a commonly accepted method of using hdparm and dd system utilities. The first disk was SanDisk SD8SMAT128G1122. According to the manufacturer specification [pdf], this is a low power disk for embedded applications and this disk can show 182 MB/s sequential write speed in SATA-3 mode. We had the following:

~# hdparm -t /dev/sda2
/dev/sda2:
Timing buffered disk reads: 274 MB in  3.02 seconds =  90.70 MB/sec

~# time sh -c "dd if=/dev/zero of=/dev/sda2 bs=500M count=1 &amp;&amp; sync"
1+0 records in
1+0 records out

real	0m6.096s
user	0m0.000s
sys	0m5.860s

which results in total write speed around 82 MB/s.

The second disk was Crusial CT250MX200SSD6 [pdf] and its sequential write speed should be 500 MB/s in SATA-3 mode. We had the following:

~# hdparm -t /dev/sda2
/dev/sda2:
Timing buffered disk reads: 236 MB in  3.01 seconds =  78.32 MB/sec

~# time sh -c "dd if=/dev/zero of=/dev/sda2 bs=500M count=1 &amp;&amp; sync"
1+0 records in
1+0 records out

real	0m6.376s
user	0m0.010s
sys	0m5.040s

which results in total write speed around 78 MB/s. Our preliminary tests had shown that the controller can achieve 200 MB/s write speed. Taking this into consideration, the performance figures obtained were not very promising, so we decided to add one new feature in the latest version of camogm – the ability to write data to a raw storage device. Raw storage device is a disk or a disk partition with direct access to hardware bypassing any operating system caches and buffers. Such type of access can potentially improve I/O performance but requires additional efforts to implement data management in software.

First approach

We tried to bypass file system in the first attempt and used device file (/dev/sda in our case) in camogm for I/O operations. We compared CPU load and I/O wait time during write operation to a partition with ext4 file system and to a device file. dstat turned to be a very handy tool for generating system resource statistics. The statistics were collected during 3 periods of operation: in idle mode before writing, during writing, and in idle mode after writing. All these periods can be clearly seen on the figures below. We also changed the quality parameter which affects the resulting size of JPEG files. Files with quality parameter set to 80 were around 1 MB in size and files with quality parameter set to 90 were almost 2 MB in size.

sys-q80
sys-q90

As expected, the figures show that device file write operation takes less CPU time than the same operation with file system, because there no file system operations and caches involved.

wai-q80
wai-q90

CPU wait for disk IO on the figures means the amount of time in percent the CPU waits for an I/O operation to complete. Here camogm process spends more CPU time waiting for data to be written during device file operations than during file system operations, and again this could be explained by the fact that caching on the file system level in not used.

We also measured the time camogm spent on writing each individual file to device file and to files on ext4 file system.

write-q80
write-q90

The clear patterns on the figures correspond to several sensor channels used during recording and each channel produced JPEG files different in size from the other channels. As we have already seen, file system caching has its influence on the results and the difference in overall write time becomes less obvious when the size of files increases.

Although the tests had shown that writing data to file system and to device file had different overall performance, we could not achieve any significant performance gain which would narrow the gap between initial results and preliminary write speed data. We decided to try another approach: only pass commands to disk driver and write data from disk driver.

Second approach

The idea behind this approach was simple. We already have JPEG data in circular buffer in memory and disk driver only needs pointers to the data we want to write at any given moment in time. camogm was modified to pass those pointers and some meta information to driver via its sysfs interface. We modified our AHCI driver as well to add new functions. The driver accepts a command from camogm, aligns data buffers to a predefined boundary and a frame in total to a physical sector boundary, and places the command to command queue. Commands are picked from the command queue right after current disk transaction is complete. We measured the time spent by driver preparing a new command, waiting for an interrupt after a command had been issued, and waiting for a new command to arrive. Total data size per each transaction was around 9.5 MB in case of SD8SMAT128G1122 and around 3 MB in case of CT250MX200SSD6. The disks were installed in cameras with 14 Mpx and 5 Mpx sensors respectively.

write-sd
write-ct

These figures show that the time spent in the driver on command preparation is almost negligible in comparison to the time spent waiting for the write command to complete and this was exactly what we finally wanted to get. We could achieve almost 160 MB/s write speed for SD8SMAT128G1122 and around 220 MB/s for CT250MX200SSD6. Here is a summary of results obtained in different modes of writing for two test disks:

Disk write performance
Disk File system access Device file access Raw driver access
SD8SMAT128G1122 82 MB/s 90 MB/s 160 MB/s
CT250MX200SSD6 78 MB/s 220 MB/s

CT250MX200SSD6 was not tested in device file access mode as it was clear that this method did not fit our needs.

Disk access sharing

One of the problems we had to solve while working on the driver was disk access sharing from operating system and from driver during recording. The disk in camera had two partitions, one was formatted to ext4 file system and mounted in operating system and the other was used as a data buffer for camogm. It is possible that some user space application could access mounted partition when camogm is writing data to disk data buffer and this situation should be correctly processed. camogm as a top priority process should always have the full disk bandwidth and other system processes should be granted access only during periods of time when camogm is waiting for the next frame. libata has built-in command deferral mechanism and we used this mechanism in the driver to decide whether the system process should have access to disk or the command should be deferred. To use this mechanism, we added our function to ATA port operations structure:

static struct ata_port_operations ahci_elphel_ops = {
    ...
    .qc_defer       = elphel_qc_defer,
};

This function is called every time a new system command arrives and the driver can defer the command in case it is busy writing data.

by Mikhail Karpenko at September 13, 2016 10:51 PM

Free Electrons

Yocto project and OpenEmbedded training updated to Krogoth

yocto

Continuing our efforts to keep our training materials up-to-date we just refreshed our Yocto project and OpenEmbedded training course to the latest Yocto project release, Krogoth (2.1.1). In addition to adapting our training labs to the Krogoth release, we improved our training materials to cover more aspects and new features.

The most important changes are:

  • New chapter about devtool, the new utility from the Yocto project to improve the developers’ workflow to integrate a package into the build system or to make patches to existing packages.
  • Improve the distro layers slides to add configuration samples and give advice on how to use these layers.
  • Add a part about quilt to easily patch already supported packages.
  • Explain in depth how file inclusions are handled by BitBake.
  • Improve the description about tasks by adding slides on how to write them in Python.

The updated training materials are available on our training page: agenda (PDF), slides (PDF) and labs (PDF).

Join our Yocto specialist Alexandre Belloni for the first public session of this improved training in Lyon (France) on October 19-21. We are also available to deliver this training worldwide at your site, contact us!

by Antoine Ténart at September 13, 2016 12:24 PM

September 12, 2016

Free Electrons

Free Electrons at the Kernel Recipes conference

Kernel RecipesThe 2016 edition of the Kernel Recipes conference will take place from September 28th to 30th in Paris. With talks from kernel developers Jonathan Corbet, Greg Kroah-Hartmann, Daniel Vetter, Laurent Pinchart, Tejun Heo, Steven Rosdedt, Kevin Hilman, Hans Verkuil and many others, the schedule looks definitely very appealing, and indeed the event is now full.

Thomas Petazzoni, Free Electrons CTO, will be attending this event. If you’re interested in discussing business or career opportunities with Free Electrons, this event will be a great place to meet together.

by Thomas Petazzoni at September 12, 2016 12:04 PM

September 09, 2016

Elphel

A web interface for a simpler and more flexible Linux kernel dynamic debug controlling

Along with the documentation there is a number of articles explaining the dynamic debug (dyndbg) feature of the Linux kernel like this one or this. Though we haven’t found anything that would extend the basic functionality – so, we created a web interface using JavaScript and PHP on top of the dyndbg.

debugfs-webgui

Fig.1 debugfs-webgui

In most cases it all works fine – when writing a linux driver you:
1. insert pr_debug()/dev_dbg() for debug messaging.
2. compile kernel with dyndbg enabled (CONFIG_DYNAMIC_DEBUG=y)
3. then just ‘echo‘ query strings or ‘cat‘ files with commands to switch on/off the debug messages at runtime. Examples:

  • single:

echo -c 'file svcsock.c line 1603 +pfmt' > /dynamic_debug/control

  • batch file:

cat query-batch-file > /dynamic_debug/control

When it’s all small – enabling/disabling the whole file or a function is not a problem. When the driver grows big with lots of debug messages or there are a few drivers interact with each other it becomes more convenient to have multiple configurations with certain debug lines on or off. As the source code changes the lines get shifted – and so, the batch files require editing.

If the target system (embedded or not) has network and a web browser (Apache2 + PHP) a quite simple solution is to add a web interface to the dynamic debug. The one we have developed has the following features:

  • allows having multiple configurations for each file
  • displays only files of interest
  • updates debug configuration for modified files where debug lines got shifted
  • keeps/updates the current config (in json format) in tmpfs – saves to disk on button click
  • p, f, l, m, t flags are supported

Get the source code then proceed with the README.md.

by Oleg Dzhimiev at September 09, 2016 12:40 AM

September 08, 2016

Free Electrons

Free Electrons at the Embedded Linux Conference Europe

The next Embedded Linux Conference Europe will take place from October 11 to October 13 in Berlin, Germany. As usual, the entire Free Electrons engineering team will participate, which means this time 10 participants from Free Electrons!

Embedded Linux Conference Europe 2016

The schedule for the conference has been published recently, and a number of our talk proposals have been accepted, so we will present on the following topics:

Like every year, we’re looking forward to attending this conference, and meeting all the nice folks of the Embedded Linux community!

by Thomas Petazzoni at September 08, 2016 11:38 AM

September 01, 2016

Free Electrons

Free Electrons at the X Developer Conference

The next X.org Developer Conference will take place on September 21 to September 23 in Helsinki, Finland. This is a major event for Linux developers working in the graphics/display areas, not only at the X.org level, but also at the kernel level, in Mesa, and other related projects.

Free Electrons engineer Maxime Ripard will be attending this conference, with 80+ other engineers from Intel, Google, NVidia, Texas Instruments, AMD, RedHat, etc.

Maxime is the author of the DRM/KMS driver in the upstream Linux kernel for the Allwinner SoCs, which provides display support for numerous Allwinner platforms, especially Nextthing’s CHIP (with parallel LCD support, HDMI support, VGA support and composite video support). Maxime has also worked on making the 3D acceleration work on this platform with a mainline kernel, by adapting the Mali kernel driver. Most recently, Maxime has been involved in Video4Linux development, writing a driver for the camera interface of Allwinner SoCs, and supervising Florent Revest work on the Allwinner VPU that we published a few days ago.

by Thomas Petazzoni at September 01, 2016 02:58 PM

August 31, 2016

Free Electrons

Free Electrons mentioned in Linux Foundation’s report

Linux Kernel Development Report 2016Lask week, the Linux Foundation announced the publication of the 2016 edition of its usual report “Linux Kernel Development – How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It”.

This report gives a nice overview of the evolution of the Linux kernel since 3.18, especially from a contribution point of view: the rate of changes, who is contributing, are there new developers joining, etc.

Free Electrons is mentioned in several places in this report. First of all, even though Free Electrons is a consulting company, it is shown individually rather than part of the general “consultants” category. As the report explains:

The category “consultants” represents developers who contribute to the kernel as a work-for-hire effort from different companies. Some consultant companies, such as Free Electrons and Pengutronix, are shown individually as their contributions are a significant number.

Thanks to being mentioned separately from the “consultants” category, the report also shows that:

  • Free Electrons is the #15 contributing company over the 3.19 to 4.7 development period, in number of commits. Free Electrons contributed a total of 1453 commits, corresponding to 1.3% of the total commits
  • Free Electrons is ranked #13 in the list of companies by number of Signed-off-by from developers who are not the author of patches. This happens because 6 of our engineers are maintainers or co-maintainers from various areas in the kernel: they merge patches from contributors, sign-off on them, and send them to another maintainer (either arm-soc maintainers or directly Linus Torvalds, depending on the subsystem).

We’re glad to see Free Electrons mentioned in this report, which shows that we are a strong contributor to the official Linux kernel. Thanks to this contribution effort, we have tremendous experience with adding support for new hardware in the kernel, so contact us if you want your hardware supported in the official Linux kernel!

by Thomas Petazzoni at August 31, 2016 09:08 AM

August 30, 2016

Free Electrons

Support for the Allwinner VPU in the mainline Linux kernel

Over the last few years, and most recently with the support for the C.H.I.P platform, Free Electrons has been heavily involved in initiating and improving the support in the mainline Linux kernel for the Allwinner ARM processors. As of today, a large number of hardware features of the Allwinner processors, especially the older ones such as the A10 or the A13 used in the CHIP, are usable with the mainline Linux kernel, including complex functionality such as display support and 3D acceleration. However, one feature that was still lacking is proper support for the Video Processing Unit (VPU) that allows to accelerate in hardware the decoding and encoding of popular video formats.

During the past two months, Florent Revest, a 19 year old intern at Free Electrons worked on a mainline solution for this Video Processing Unit. His work followed the reverse engineering effort of the Cedrus project, and this topic was also listed as a High Priority Reverse Engineering Project by the FSF.

The internship resulted in a new sunxi-cedrus driver, a Video4Linux memory-to-memory decoder kernel driver and a corresponding VA-API backend, which allows numerous userspace applications to use the decoding capabilities. Both projects have both been published on Github:

Currently, the combination of the kernel driver and VA-API backend supports MPEG2 and MPEG4 decoding only. There is for the moment no support for encoding, and no support for H264, though we believe support for both aspects can be added within the architecture of the existing driver and VA-API backend.

A first RFC patchset of the kernel driver has been sent to the linux-media mailing list, and a complete documentation providing installation information and architecture details has been written on the linux-sunxi’s wiki.

Here is a video of VLC playing a MPEG2 demo video on top of this stack on the Next Thing’s C.H.I.P:

by Thomas Petazzoni at August 30, 2016 02:13 PM

August 18, 2016

Bunnie Studios

Name that Ware August 2016

The Ware for August 2016 is shown below.

Thanks to Adrian Tschira (notafile) for sharing this well-photographed ware! The make and model of this ware is unknown to both of us, so if an unequivocal identification isn’t made over the coming month, I’ll be searching the comments for either the most thoughtful or the most entertaining analysis of the ware.

by bunnie at August 18, 2016 04:48 PM

Winner, Name that Ware July 2016

The Ware for July 2016 was a board from a Connection Machine CM-2 variant; quite likely a CM-200.

It’s an absolutely gorgeous board, and the sort of thing I’d use as a desktop background if I used a desktop background that was’t all black. Thanks again to Mark Jessop for contributing the ware. Finally, the prize this month goes to ojn for a fine bit of sleuthing, please email me to claim your prize! I particularly loved this little comment in the analysis:

The board layout technique is different from what I’ve been able to spot from IBM, SGI, DEC. Cray used different backplanes so the connectors at the top also don’t match.

Every designer and design methodology leaves a unique fingerprint on the final product. While I can’t recognize human faces very well, I do perceive stylistic differences in a circuit board. The brain works in funny ways…

by bunnie at August 18, 2016 04:48 PM

August 16, 2016

Harald Welte

(East) European motorbike tour on 20y old BMW F650ST

For many years I've always been wanting to do some motrobike riding accross the Alps, but somehow never managed to do so. It seems when in Germany I've always been too busy - contrary to the many motorbike tours around and accross Taiwan which I did during my frequent holidays there.

This year I finally took the opportunity to combine visiting some friends in Hungary and Bavaria with a nice tour starting from Berlin over Prague and Brno (CZ), Bratislava (SK) to Tata and Budapeest (HU), further along lake Balaton (HU) towards Maribor (SI) and finally accross the Grossglockner High Alpine Road (AT) to Salzburg and Bavaria before heading back to Berlin.

It was eight fun (but sometimes long) days riding. For some strange turn of luck, not a single drop of rain was encountered during all that time, travelling accross six countries.

The most interesting parts of the tour were:

  • Along the Elbe river from Pirna (DE) to Lovosice (CZ). Beautiful scenery along the river valey, most parts of the road immediately on either side of the river. Quite touristy on the German side, much more pleaant and quiet on the Czech side.
  • From Mosonmagyarovar via Gyor to Tata (all HU). Very little traffic alongside road '1'. Beatutil scenery with lots of agriculture and forests left and right.
  • The Nothern coast of Lake Balaton, particularly from Tinany to Keszthely (HU). Way too many tourists and traffic for my taste, but still very impressive to realize how large/long that lake really is.
  • From Maribor to Dravograd (SI) alongside the Drau/Drav river valley.
  • Finally, of course, the Grossglockner High Alpine Road, which reminded me in many ways of the high mountain tours I did in Taiwan. Not a big surprise, given that both lead you up to about 2500 meters above sea level.

Finally, I have to say I've been very happy with the performancee of my 1996 model BMW F 650ST bike, who has coincidentially just celebrated its 20ieth anniversary. I know it's an odd bike design (650cc single-cylinder with two spark plugs, ignition coils and two carburetors) but consider it an acquired taste ;)

I've also published a map with a track log of the trip

In one month from now, I should be reporting from motorbike tours in Taiwan on the equally trusted small Yamaha TW-225 - which of course plays in a totally different league ;)

by Harald Welte at August 16, 2016 02:00 PM

August 03, 2016

Free Electrons

Linux 4.7 statistics: Free Electrons engineer #2 contributor

LWN.net has published yesterday an article containing statistics for the 4.7 development cycle. This article is available for LWN.net subscribers only during the coming week, and will then be available for everyone, free of charge.

It turns out that Boris Brezillon, Free Electrons engineer, is the second most active contributor to the 4.7 kernel in number of commits! The top three contributors in number of commits are: H Hartley Sweeten (208 commits), Boris Brezillon (132 commits) and Al Viro (127 commits).

LWN.net 4.7 kernel statistics

In addition to being present in the most active developers by number of commits, Boris Brezillon is also in the #11 most active contributor in terms of changed lines. As we discussed in our previous blog post, most contributions from Boris were targeted at the PWM subsystem on one side (atomic update support) and the NAND subsystem on the other side.

Another Free Electrons engineer shows up in the per-developer statistics: Maxime Ripard is the #17 most active contributor by lines changed. Indeed, Maxime contributed a brand new DRM/KMS driver for the Allwinner display controller.

As a company, Free Electrons is ranked for the 4.7 kernel as the #12 most active company by number of commits, and #10 by number of changed lines. We are glad to continue being such a contributor to the Linux kernel development, as we have been for the last four years. If you want your hardware to be supported in the official Linux kernel, contact us!

by Thomas Petazzoni at August 03, 2016 07:41 AM

August 02, 2016

Free Electrons

“Understanding D-Bus” talk at the Toulouse Embedded Linux Meetup

A few months ago, in May, Free Electrons engineer Mylène Josserand presented a talk titled Understanding D-Bus at the Toulouse Embedded Linux and Android meetup.

In this talk, Mylène shared her experience working with D-Bus, especially in conjunction with the OFono and Connman projects, to support modem and 3G connections on embedded Linux systems.

Understanding D-Bus

We are now publishing the slides of Mylène’s talk, they are available in PDF format.

by Thomas Petazzoni at August 02, 2016 09:14 AM

August 01, 2016

Free Electrons

Linux 4.7 released, Free Electrons contributions

Adelie PenguinLinux 4.7 has been released on Sunday by Linus Torvalds, with numerous new features and improvements that have been described in details on LWN: part 1, part 2 and part 3. KernelNewbies also has an updated page on the 4.7 release. We contributed a total of 222 patches to this release.

Our most significant contributions:

  • Boris Brezillon has contributed a core improvement to the PWM subsystem: a mechanism that allows to update the properties of a PWM in an atomic fashion. This is needed when a PWM has been initialized by the bootloader, and the kernel needs to take over without changing the properties of the PWM. See the main patch for more details. What prompted the creation of this patch series is a problem on Rockchip based Chromebook platforms where a PWM is used for a regulator, and the PWM properties need to be preserved across the bootloader to kernel transition. In addition to the changes of the core infrastructure, Boris contributed numerous patches to fix existing PWM users.
  • In the MTD subsystem, Boris Brezillon continued his cleanup efforts
    • Use the common Device Tree parsing code provided by nand_scan_ident() in more drivers, rather than driver-specific code.
    • Move drivers to expose their ECC/OOB layout information using the mtd_ooblayout_ops structure, and use the corresponding helper functions where appropriate. This change will allow a more flexible description of the ECC and OOB layout.
    • Document the Device Tree binding that should now be used for all NAND controllers / NAND chip, with a clear separation between the NAND controller and the NAND chip. See this commit for more details.
  • In the RTC subsystem, Mylène Josserand contributed numerous improvements to the rv3029 and m41t80 drivers, including the addition of the support for the RV3049 (the SPI variant of RV3029). See also our previous blog post on the support of Microcrystal’s RTCs/.
  • On the support of Atmel platforms
    • Boris Brezillon contributed a number of fixes and improvements to the atmel-hlcdc driver, the DRM/KMS driver for Atmel platforms
  • On the support of Allwinner platforms
    • Maxime Ripard contributed a brand new DRM/KMS driver to support the display controller found on several Allwinner platforms, with a specific focus on Allwinner A10. This new driver allows to have proper graphics support in the Nextthing Co. C.H.I.P platform, including composite output and RGB output for LCD panels. To this effect, in addition to the driver itself, numerous clock patches and Device Tree patches were made.
    • Boris Brezillon contributed a large number of improvements to the NAND controller driver used on Allwinner platforms, including performance improvements.
    • Quentin Schulz made his first kernel contribution by sending a patch fixing the error handling in a PHY USB driver used by Allwinner platforms.
  • On the support of Marvell platforms
    • Grégory Clement made some contributions to the mv_xor driver to make it 64-bits ready, as the same XOR engine is used on Armada 3700, a Cortex-A53 based SoC. Grégory then enabled the use of the XOR engines on this platform by updating the corresponding Device Tree.
    • Romain Perier did some minor updates related to the Marvell cryptographic engine support. Many more updates will be present in the upcoming 4.8, including significant performance improvements.
    • Thomas Petazzoni contributed some various fixes (cryptographic engine usage on some Armada 38x boards, HW I/O coherency related fixes).
    • Thomas also improved the support for Armada 7K and 8K, with the description of more hardware blocks, and updates to drivers.

Here are in details, the different contributions we made to this release:

by Thomas Petazzoni at August 01, 2016 03:55 PM

July 23, 2016

Harald Welte

python-inema: Python module implementing Deutsche Post 1C4A Internetmarke API

At sysmocom we maintain a webshop with various smaller items and accessories interesting to the Osmocom community as well as the wider community of people experimenting (aka 'playing') with cellular communications infrastructure. As this is primarily a service to the community and not our main business, I'm always interested in ways to reduce the amount of time our team has to use in order to operate the webshop.

In order to make the shipping process more efficient, I discovered that Deutsche Post is offering a Web API based on SOAP+WSDL which can be used to generate franking for the (registered) letters that we ship around the world with our products.

The most interesting part of this is that you can generate combined address + franking labels. As address labels need to be printed anyway, there is little impact on the shipping process beyond having to use this API to generate the right franking for the particular shipment.

Given the general usefulness of such an online franking process, I would have assumed that virtually anyone operating some kind of shop that regularly mails letters/products would use it and hence at least one of those users would have already written some free / open source software code fro it. To my big surprise, I could not find any FOSS implementation of this API.

If you know me, I'm the last person to know anything about web technology beyond HTML 4 which was the latest upcoming new thing when I last did anything web related ;)

Nevertheless, using the python-zeep module, it was fairly easy to interface the web service. The weirdest part is the custom signature algorithm that they use to generate some custom soap headers. I'm sure they have their reasons ;)

Today I hence present the python-inema project, a python module for accessing this Internetmarke API.

Please note while I'm fluent in Pascal, Perl, C and Erlang, programming in Python doesn't yet come natural to me. So if you have any comments/feedback/improvements, they're most welcome by e-mail, including any patches.

by Harald Welte at July 23, 2016 02:00 PM

Going to attend Electromagnetic Field 2016

Based on some encouragement from friends as well as my desire to find more time again to hang out at community events, I decided to attend Electromagnetic Field 2016 held in Guildford, UK from August 5th through 7th.

As I typically don't like just attending an event without contributing to it in some form, I submitted a couple of talks / workshops, all of which were accepted:

  • An overview talk about the Osmocom project
  • A Workshop on running your own cellular network using OpenBSC and related Osmocom software
  • A Workshop on tracing (U)SIM card communication using Osmocom SIMtrace

I believe the detailed schedule is still in the works, as I haven't yet been able to find any on the event website.

Looking forward to having a great time at EMF 2016. After attending Dutch and German hacker camps for almost 20 years, let's see how the Brits go about it!

by Harald Welte at July 23, 2016 02:00 PM

EC-GSM-IoT: Enhanced Coverage GSM for IoT

In private conversation, Holger mentioned EC-GSM-IoT to me, and I had to dig a bit into it. It was introduced in Release 13, but if you do a web search for it, you find surprisingly little information beyond press releases with absolutely zero information content and no "further reading".

The primary reason for this seems to be that the feature was called EC-EGPRS until the very late stages, when it was renamed for - believe it or not - marketing reasons.

So when searching for the right term, you actually find specification references and change requests in the 3GPP document archives.

I tried to get a very brief overview, and from what I could find, it is centered around GERAN extension in the following ways:

  • EC-EGPRS goal: Improve coverage by 20dB
    • New single-burst coding schemes
    • Blind Physical Layer Repetitions where bursts are repeated up to 28 times without feedback from remote end
      • transmitter maintains phase coherency
      • receiver uses processing gain (like incremental redundancy?)
    • New logical channel types (EC-BCCH, EC-PCH, EC-AGC, EC-RACH, ...)
    • New RLC/MAC layer messages for the EC-PDCH communication
  • Power Efficient Operation (PEO)
    • Introduction of eDRX (extended DRX) to allow for PCH listening intervals from minutes up to a hour
    • Relaxed Idle Mode: Important to camp on a cell, not best cell. Reduces neighbor cell monitoring requirements

In terms of required modifications to an existing GSM/EDGE implementation, there will be (at least):

  • changes to the PHY layer regarding new coding schemes, logical channels and burst scheduling / re-transmissions
  • changes to the RLC/MAC layer in the PCU to implement the new EC specific message types and procedures
  • changes to the BTS and BSC in terms of paging in eDRX

In case you're interested in more pointers on technical details, check out the links provided at https://osmocom.org/issues/1780

It remains to be seen how widely this will be adopted. Rolling this cange out on moderm base station hardware seems technicalyl simple - but it remains to be seen how many equipment makers implement it, and at what cost to the operators. But I think the key issue is whether or not the baseband chipset makers (Intel, Qualcomm, Mediatek, ...) will implement it anytime soon on the device side.

There are no plans on implementing any of this in the Osmocom stack as of now,but in case anyone was interested in working on this, feel free to contact us on the osmocom-net-gprs@lists.osmocom.org mailing list.

by Harald Welte at July 23, 2016 10:00 AM

July 21, 2016

Bunnie Studios

Why I’m Suing the US Government

Today I filed a lawsuit against the US government, challenging Section 1201 of the Digital Millennium Copyright Act. Section 1201 means that you can be sued or prosecuted for accessing, speaking about, and tinkering with digital media and technologies that you have paid for. This violates our First Amendment rights, and I am asking the court to order the federal government to stop enforcing Section 1201.

Before Section 1201, the ownership of ideas was tempered by constitutional protections. Under this law, we had the right to tinker with gadgets that we bought, we had the right to record TV shows on our VCRs, and we had the right to remix songs. Section 1201 built an extra barrier around copyrightable works, restricting our prior ability to explore and create. In order to repair a gadget, we may have to decrypt its firmware; in order to remix a video, we may have to strip HDCP. Whereas we once readily expressed feelings and new ideas through remixes and hardware modifications, now we must first pause and ask: does this violate Section 1201? Especially now that cryptography pervades every aspect of modern life, every creative spark is likewise dampened by the chill of Section 1201.

The act of creation is no longer spontaneous.

Our recent generation of Makers, hackers, and entrepreneurs have developed under the shadow of Section 1201. Like the parable of the frog in the well, their creativity has been confined to a small patch, not realizing how big and blue the sky could be if they could step outside that well. Nascent 1201-free ecosystems outside the US are leading indicators of how far behind the next generation of Americans will be if we keep with the status quo.

Our children deserve better.

I can no longer stand by as a passive witness to this situation. I was born into a 1201-free world, and our future generations deserve that same freedom of thought and expression. I am but one instrument in a large orchestra performing the symphony for freedom, but I hope my small part can remind us that once upon a time, there was a world free of such artificial barriers, and that creativity and expression go hand in hand with the ability to share without fear.

If you want to read more about the lawsuit, please check out the EFF’s press release on the matter.

by bunnie at July 21, 2016 01:01 PM

Countering Lawful Abuses of Digital Surveillance

Completely separate from the Section 1201 lawsuit against the Department of Justice, I’m working with the FPF on a project to counter lawful abuses of digital surveillance. Here’s the abstract:

Front-line journalists are high-value targets, and their enemies will spare no expense to silence them. Unfortunately, journalists can be betrayed by their own tools. Their smartphones are also the perfect tracking device. Because of the precedent set by the US’s “third-party doctrine,” which holds that metadata on such signals enjoys no meaningful legal protection, governments and powerful political institutions are gaining access to comprehensive records of phone emissions unwittingly broadcast by device owners. This leaves journalists, activists, and rights workers in a position of vulnerability. This work aims to give journalists the tools to know when their smart phones are tracking or disclosing their location when the devices are supposed to be in airplane mode. We propose to accomplish this via direct introspection of signals controlling the phone’s radio hardware. The introspection engine will be an open source, user-inspectable and field-verifiable module attached to an existing smart phone that makes no assumptions about the trustability of the phone’s operating system.

You can find out more about the project by reading the white paper at Pubpub.

by bunnie at July 21, 2016 01:00 PM

July 16, 2016

Harald Welte

Deeper ventures into Ericsson (Packet) Abis

Some topics keep coming back, even a number of years after first having worked on them. And then you start to search online using your favorite search engine - and find your old posts on that subject are the most comprehensive publicly available information on the subject ;)

Back in 2011, I was working on some very basic support for Ericsson RBS2xxx GSM BTSs in OpenBSC. The major part of this was to find out the weird dynamic detection of the signalling timeslot, as well as the fully non-standard OM2000 protocol for OML. Once it reached the state of a 'proof-of-concept', work at this ceased and remained in a state where still lots of manual steps were involved in BTS bring-up.

I've recently picked this topic up again, resulting in some work-in-progress code in http://git.osmocom.org/openbsc/log/?h=laforge/om2000-fsm

Beyond classic E1 based A-bis support, I've also been looking (again) at Ericsson Packet Abis. Packet Abis is their understanding of Abis over IP. However, it is - again - much further from the 3GPP specifications than what we're used to in the Osmocom universe. Abis/IP as we know consists of:

  • RSL and OML over TCP (inside an IPA multiplex)
  • RTP streams for the user plane (voice)
  • Gb over IP (NS over UDP/IP), as te PCU is in the BTS.

In the Ericsson world, they decided to taka a much lower-layer approach and decided to

  • start with L2TP over IP (not the L2TP over UDP that many people know from VPNs)
  • use the IETF-standardized Pseudowire type for HDLC but use a frame format in violation of the IETF RFCs
  • Talk LAPD over L2TP for RSL and OML
  • Invent a new frame format for voice codec frames called TFP and feed that over L2TP
  • Invent a new frame format for the PCU-CCU communication called P-GSL and feed that over L2TP

I'm not yet sure if we want to fully support that protocol stack from OpenBSC and related projects, but in any case I've extende wireshark to decode such protocol traces properly by

  • Extending the L2TP dissector with Ericsson specific AVPs
  • Improving my earlier pakcet-ehdlc.c with better understanding of the protocol
  • Implementing a new TFP dissector from scratch
  • Implementing a new P-GSL dissector from scratch

The resulting work can be found at http://git.osmocom.org/wireshark/log/?h=laforge/ericsson-packet-abis in case anyone is interested. I've mostly been working with protocol traces from RBS2409 so far, and they are decoded quite nicely for RSL, OML, Voice and Packet data. As far as I know, the format of the STN / SIU of other BTS models is identical.

Is anyone out there in possession of Ericsson RBS2xxx RBSs interested in collboration on either a Packet Abis implementation, or an inteface of the E1 or packet based CCU-PCU interface to OsmoPCU?

by Harald Welte at July 16, 2016 10:00 AM

July 12, 2016

Bunnie Studios

Name that Ware July 2016

The ware for July 2016 is shown below.

Thanks to Mark Jessop for contributing this wonderful ware. It’s a real work of art on the front side, but google makes it way too easy to identify with a couple part number queries. To make it a smidgen more challenging, I decided to start this month’s competition with just the back side of the board. If the photo above doesn’t give enough clues, I’ll add a photo of the front side as well…

by bunnie at July 12, 2016 09:01 AM

Winner, Name that Ware June 2016

The Ware for June 2016 is an ATS810C by ATS Automation. There’s no information on their website about this particular board, but there’s lots of good ideas in the comments as to what this could be from. However, none of them have me 100% convinced. So I’ll just go with the first answer that generally identified the ware as a stepper motor controller with RS-232 interface by notafile. Congrats, email me for your prize!

by bunnie at July 12, 2016 09:00 AM

Elphel

I will not have to learn SystemVerilog

Or at least larger (verification) part of it – interfaces, packages and a few other synthesizable features are very useful to reduce size of Verilog code and make it easier to maintain. We now are able to run production target system Python code with Cocotb simulation over BSD sockets.

Client-server simulation of NC393 with Cocotb

Client-server simulation of NC393 with Cocotb


Previous workflow

Before switching to Cocotb our FPGA-related workflow involved:

  1. Creating RTL design code
  2. Writing Verilog tests
  3. Running simulations
  4. Synthesizing and creating bitfile
  5. Re-writing test code to run on the target system in Python
  6. Developing kernel drivers to support the FPGA functionality
  7. Developing applications that access FPGA functionality through the kernel drivers

Of course the steps are not that linear, there are hundreds of loops between steps 1 and 3 (editing RTL source after finding errors at step 3), almost as many from 5 to 1 (when the problems reveal themselves during hardware testing) but few are noticed only at step 6 or 7. Steps 2, 5, 6+7 involve a gross violation of DRY principle, especially the first two. The last steps sufficiently differ from step 5 as their purpose is different – while Python tests are made to reveal the potential problems including infrequent conditions, drivers only use a subset of functionality and try to “hide” problems – perform recovering actions to maintain operation of the device after abnormal condition occurs.

We already tried to mitigate these problems – significant part of the design flexibility is achieved through parametrized modules. Parameters are used to define register map and register bit fields – they are one of the most frequently modified when new functionality is added. Python code in the camera is able to read and process Verilog parameters include files when running on the target system, and while generating C header files for the kernel drivers, so here DRY principle stands. Changes in any parameters definitions in Verilog files will be automatically propagated to both Python and C code.

But it is definitely not enough. Steps 2 and 5 may involve tens of thousands lines of code and large part of the Python code is virtually a literal translation from the Verilog original. All our FPGA-based systems (and likely it is true for most other applications) involve symbiotic operation of the FPGA and some general purpose processor. In Xilinx Zynq they are on the same chip, in our earlier designs they were connected on the PCB. Most of the volume of the Verilog test code is the simulation of the CPU running some code. This code interacts with the rest of the design through the writes/reads of the memory-mapped control/status registers as well as the system memory when FPGA is a master sending/receiving data over DMA.

This is one of the reasons I hesitated to learn verification functionality of SystemVerilog. There are tons of computer programming languages that may be a better fit to simulate program activity of the CPU (this is what they do naturally). Currently most convenient for bringing the new hardware to life seems to be Python, so I was interested in trying Cocotb. If I new it is that easy I would probably start earlier, but having rather large volume of the existing Verilog code for testing I was postponing the switch.

Trying Cocotb

Two weeks ago I gave it a try. First I prepared the instruments – integrated Cocotb into VDT, made sure that Eclipse console output is clickable for the simulator reported problems, simulator output as well as for the errors in Python code and the source links in Cocotb logs. I used the Cocotb version of the JPEG encoder that has the Python code for simulation – just added configuration options for VDT and fixed the code to reduce number of warning markers that VDT generated. Here is the version that can be imported as Eclipse+VDT project.

Converting x393 camera project to Cocotb simulation

Next was to convert simulation of our x393 camera project to use Cocotb. For that I was looking not just to replace Verilog test code with Python, but to use the same Python program that we already have running on the target hardware for simulation. The program already had a “dry run” option for development on a host computer, that part had to be modified to access simulator. I needed some way to effectively isolate the Python code that is linked to the simulator and the code of the target system, and BSD sockets provide a good match that. One part of the program that uses Cocotb modules and is subject to special requirement to the Python coroutines to work for simulation – it plays a role of the server. The other part (linked to the target system program) replaces memory accesses, sends the request parameters over the socket connection, and waits for the response from the server. The only other than memory access commands that are currently implemented are “finish” (to complete simulation and analyze the results in wave viewer – GtkWave), “flush” (flush file writes – similar to cache flushes on a real hardware) and interruptible (by the simulated system interrupt outputs) wait for specified time. Simulation time is frozen between requests from the client, so the target system has to specifically let the simulated system run for certain time (or until it will generate an interrupt).

Simulation client

Below is the example of modification to the target code memory write (full source). X393_CLIENT is True: branch is for old dry run (NOP) mode, second one (elif not X393_CLIENT is None:) is for the simulation server and the last one accesses real memory over /dev/mem.

def write_mem (self,addr, data,quiet=1):
        """
        Write 32-bit word to physical memory
        @param addr - physical byte address
        @param data - 32-bit data to write
        @param quiet - reduce output
        """
        if X393_CLIENT is True:
            print ("simulated: write_mem(0x%x,0x%x)"%(addr,data))
            return
        elif not X393_CLIENT is None:
            if quiet  1:
                print ("remote: write_mem(0x%x,0x%x)"%(addr,data))
            X393_CLIENT.write(addr, [data])
            if quiet  1:
                print ("remote: write_mem done" )
            return
        with open("/dev/mem", "r+b") as f:
            page_addr=addr & (~(self.PAGE_SIZE-1))
            page_offs=addr-page_addr
            mm = self.wrap_mm(f, page_addr)
            packedData=struct.pack(self.ENDIAN+"L",data)
            d=struct.unpack(self.ENDIAN+"L",packedData)[0]
            mm[page_offs:page_offs+4]=packedData
            if quiet 2:
                print ("0x%08x == 0x%08x (%d)"%(addr,d,d))

There is not much magic in initializing X393_CLIENT class instance:

print("Creating X393_CLIENT")
            try:
                X393_CLIENT= x393Client(host=dry_mode.split(":")[0], port=int(dry_mode.split(":")[1]))
                print("Created X393_CLIENT")
            except:
                X393_CLIENT= True
                print("Failed to create X393_CLIENT")

And all the sockets handling code is less than 100 lines (source):

import json
import socket

class SocketCommand():
    command=None
    arguments=None
    def __init__(self, command=None, arguments=None): # , debug=False):
        self.command = command
        self.arguments=arguments
    def getCommand(self):
        return self.command
    def getArgs(self):
        return self.arguments
    def getStart(self):
        return self.command == "start" 
    def getStop(self):
        return self.command == "stop" 
    def getWrite(self):
        return self.arguments if self.command == "write" else None
    def getWait(self):
        return self.arguments if self.command == "wait" else None
    def getFlush(self):
        return self.command == "flush"
    def getRead(self):
        return self.arguments if self.command == "read" else None
    def setStart(self):
        self.command = "start"
    def setStop(self):
        self.command = "stop"
    def setWrite(self,arguments):
        self.command = "write"
        self.arguments=arguments
    def setWait(self,arguments): # wait irq mask, timeout (ns)
        self.command = "wait"
        self.arguments=arguments
    def setFlush(self):         #flush memory file (use when sync_for_*
        self.command = "flush"
    def setRead(self,arguments):
        self.command = "read"
        self.arguments=arguments
    def toJSON(self,val=None):
        if val is None:
            return json.dumps({"cmd":self.command,"args":self.arguments})
        else:
            return json.dumps(val)    
    def fromJSON(self,jstr):
        d=json.loads(jstr)
        try:
            self.command=d['cmd']
        except:
            self.command=None
        try:
            self.arguments=d['args']
        except:
            self.arguments=None
        
class x393Client():
    def __init__(self, host='localhost', port=7777):
        self.PORT = port
        self.HOST = host   # Symbolic name meaning all available interfaces
        self.cmd= SocketCommand()
    def communicate(self, snd_str):
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.connect((self.HOST, self.PORT))
        sock.send(snd_str)
        reply = sock.recv(16384)  # limit reply to 16K
        sock.close()
        return reply
    def start(self):
        self.cmd.setStart()
        print("start->",self.communicate(self.cmd.toJSON()))
    def stop(self):
        self.cmd.setStop()
        print("stop->",self.communicate(self.cmd.toJSON()))
    def write(self, address, data):
        self.cmd.setWrite([address,data])
        rslt = self.communicate(self.cmd.toJSON())
    def waitIrq(self, irqMask,wait_ns):
        self.cmd.setWait([irqMask,wait_ns])
        rslt = self.communicate(self.cmd.toJSON())
    def flush(self):
        self.cmd.setFlush()
    def read(self, address):
        self.cmd.setRead(address)
        rslt = self.communicate(self.cmd.toJSON())
        return json.loads(rslt)

Simulation server

Server code is larger (it now has 360 lines) but it is rather simple too. It runs in the Cocotb environment (coroutines that yield to simulator have “@cocotb.coroutine” decorations), receives and responds to the commands over the socket. When the command involves writing, it compares requested address to the pre-defined ranges and either sends data over one of the master AXI channels (defined in x393interfaces.py) or writes to the “system memory” – just a file system file with appropriate offset corresponding to the specified address)

elif self.cmd.getWrite():
            ad = self.cmd.getWrite()
            self.dut._log.debug('Received WRITE, 0x%0x: %s'%(ad[0],hex_list(ad[1])))
            if ad[0]in self.RESERVED:
                if ad[0] == self.INTM_ADDRESS:
                    self.int_mask = ad[1][0]
                rslt = 0 
            elif (ad[0] >= self.memlow) and  (ad[0]  self.memhigh):
                addr = ad[0]
                self._memfile.seek(addr)
                for data in ad[1]: # currently only single word is supported
                    sdata=struct.pack(" 0x%08x"%(data,addr))
                    addr += 4
                rslt = 0 
            elif(ad[0] >= 0x40000000) and (ad[0]  0x80000000):
                rslt = yield self.maxigp0.axi_write(address =     ad[0],
                                                value =           ad[1],
                                                byte_enable =     None,
                                                id =              self.writeID,
                                                dsize =           2,
                                                burst =           1,
                                                address_latency = 0,
                                                data_latency =    0)
                self.dut._log.debug('maxigp0.axi_write yielded %s'%(str(rslt)))
                self.writeID = (self.writeID+1) & self.writeIDMask
            elif (ad[0] >= 0xc0000000) and (ad[0]  0xfffffffc):
                self.ps_sbus.write_reg(ad[0],ad[1][0])
                rslt = 0 
            else:
                self.dut._log.info('Write address 0x%08x is outside of maxgp0, not yet supported'%(ad[0]))
                rslt = 0
            self.dut._log.info('WRITE 0x%08x = %s'%(ad[0],hex_list(ad[1], max_items = 4)))
            self.soc_conn.send(self.cmd.toJSON(rslt)+"\n")
            self.dut._log.debug('Sent rslt to the socket')

Similarly read commands acquire data from either AXI read channel or from the same memory image file. Data is sent to this file over the AXI slave interface by the simulated device.

Top Verilog module

The remaining part of the conversion form plain Verilog to Cocotb simulation is the top Verilog file – x393_dut.v. It contains an instance of the actual synthesized module (x393_i) and Verilog simulation modules of the connected peripherals. These modules can also be replaced by Python ones (and some eventually will be), but others, like Micron DDR3 memory model, are easier to use as they are provided by the chip manufacturer.

Python modules can access hierarchical nodes in the design, but to keep things cleaner all the design inputs and outputs are routed to/from the outputs/inputs of the x393_dut module. In the case of Xilinx Zynq that involves connecting internal nodes – Zynq considers CPU interface not as I/O, but as an empty module (PS7) instantiated in the design.

Screenshot of the simulation with client (black console) and server (in Eclipse IDE)

Screenshot of the simulation with client (black console) and server (in Eclipse IDE)


Conclusions

Conversion to Python simulation was simple, considering rather large amount of project (Python+Verilog) code – about 100K lines. After preparing the tools it took just one week and now we have the same code running both on the real hardware and in the simulator.

Splitting the simulation into client/server duo makes it easy to use any other programming language on the client side – not just the Python of our choice. Unix sockets provide convenient means for that. Address decoder (which decides what interface to use for the received memory access request) is better to keep on the server (simulator) side of the socket connection, not on the client. This minimizes changes to the target code and the server is playing the role of the memory-mapped system bus, behaves as the real hardware does.

Are there any performance penalties compared to all-Verilog simulation? None visible in our designs. Simulation (and Icarus Verilog is a single-threaded application) is the most time-consuming part – for our application it is about 8,000,000 times slower than the modeled hardware. Useful simulations (all-Verilog) for the camera runs for 15-40 minutes with tiny 64×32 pixel images. If we ran normal set of 14 MPix frames it would take about a week for the first images to appear at the output. Same Python code on the target runs for a fraction of a second, so even as the simulator is stopped while Python runs, combined execution time does not noticeably change for the Python+Verilog vs. all-Verilog mode. It would be nice to try to use Verilator in addition to Icarus. While it is not a real Verilog simulator (it can not handle ‘bx and ‘bz values, just ‘0’ and ‘1’) it is much faster.

by Andrey Filippov at July 12, 2016 12:32 AM

July 08, 2016

Video Circuits

Edge of Frame DIY Space For London

















I am performing on Thursday the 14th at DIY space for London.
Edge of Frame  very kindly asked me to play. There will also be a fantastic selection of films by other artists including some of my favourite people operating in the world of experimental electronic image making.

by Chris (noreply@blogger.com) at July 08, 2016 02:21 AM

July 03, 2016

ZeptoBARS

MAX2659 - SiGe GPS/GNSS LNA : weekend die-shot

Maxim Integrated MAX2659 is a low-noise (NF 0.8dB) SiGe RF amplifier for GPS/GNSS applications.

July 03, 2016 01:06 AM

July 01, 2016

Video Circuits

Video Circuits Room at Brighton Modular Meet

Alexander Peverett and I will be looking after a Video Synthesis room at Brighton Modular Meet this weekend. Come down to chat about video synthesis and our strange love for CRT tubes :) there will be at least 6 sony PVMs on display! along with a few systems and unusual bits of video gear. I am super excited that Nitin AKA Bradford Bahamas will be bringing his incredible playable CRT rig.

brightonmodularmeet.co.uk
Invited to be part of the 2016 International Conference on Live Interfaces (http://www.liveinterfaces.org/) at the University of Sussex, this years Brighton Modular Meet takes place in the newly renovated Attenborough Centre for the Creative Arts (formerly the Gardner Arts Centre). We will be taking over a number of rooms in this Grade 2 listed building designed by Sir Basil Spence.

In addition to a large open access space where you can set up your synth and mingle with others, this year we will have a room for manufacturers and shops, a space for talks and live performances, and a video synthesis room.



by Chris (noreply@blogger.com) at July 01, 2016 03:22 AM

June 28, 2016

Bunnie Studios

Episode 4: Reinventing 35 years of Innovation

Episode 4 is out!

It’s a daunting challenge to document a phenomenon as diverse as Shenzhen, so I don’t envy the task of trying to fit it in four short episodes.

Around 6:11 I start sounding like a China promo clip. This is because as a foreigner, I’m a bit cautious about saying negative things about a country, especially when I’m a guest of that country.

I really love the part at 3:58 where Robin Wu, CEO of Meegopad, reflects on the evolution of the term Shanzhai in China:

I was one of the people who made Shanzhai products. In the past, everyone looked down on Shanzhai products. Now, I think the idea of the maker is the same as Shanzhai. Shanzhai is not about copying. Shanzhai is a spirit.

by bunnie at June 28, 2016 03:56 PM

June 26, 2016

ZeptoBARS

Microchip HCS301 KeeLoq : weekend die-shot

Microchip HCS301 is an old and popular code hopping encoder for remote controls.
Die size 2510x1533 µm, 1µm technology.



PS. Thanks Andrew for the chips!

June 26, 2016 11:34 PM

Video Circuits

Raven Row Vasulka Talk

Here is a shot from the talk, hopefully will have some more images and audio to share soon.

by Chris (noreply@blogger.com) at June 26, 2016 03:27 AM

June 21, 2016

Bunnie Studios

Episode 3: A New Breed of Intellectual Property

Episode 3 is out!

I say the darndest things on camera. O_o

Like everyone else, I see the videos when they are released. So far, this episode makes the clearest case for why Shenzhen is the up-and-coming place for hardware technology.

Most of the time my head is buried in resistors and capacitors. However, this video takes a wide-angle shot of the tech ecosystem. I’ve been visiting for over a decade, and this video is the first time I’ve seen some of the incredible things going on in Shenzhen, particularly in the corporate world.

by bunnie at June 21, 2016 03:31 PM

June 17, 2016

Free Electrons

Free Electrons contributes to KernelCI.org

The Linux kernel is well-known for its ability to run on thousands of different hardware platforms. However, it is obviously impossible for the kernel developers to test their changes on all those platforms to check that no regressions are introduced. To address this problem, the KernelCI.org project was started: it tests the latest versions of the Linux kernel from various branches on a large number of hardware plaforms and provides a centralized interface to browse the results.

KernelCI.org project

KernelCI.org project

From a physical point of view, KernelCI.org relies on labs containing a number of hardware platforms that can be remotely controlled. Those labs are provided by various organizations or individuals. When a commit in one of the Linux kernel Git branches monitored by KernelCI is detected, numerous kernel configurations are built, tests are sent to all labs and results are collected on the KernelCI.org website. This allows kernel developers and maintainers to detect and fix bugs and regressions before they reach users. As of May, 10th 2016, KernelCI stats show a pool of 185 different boards and around 1900 daily boots.

Free Electrons is a significant contributor to the Linux kernel, especially in the area of ARM hardware platform support. Several of our engineers are maintainers or co-maintainers of ARM platforms (Grégory Clement for Marvell EBU, Maxime Ripard for Allwinner, Alexandre Belloni for Atmel and Antoine Ténart for Annapurna Labs). Therefore, we have a specific interest in participating to an initiative like KernelCI, to make sure that the platforms that we maintain continue to work well, and a number of the platforms we care about were not tested by the KernelCI project.

Over the last few months, we have been building our boards lab in our offices, and we have joined the KernelCI project since April 25th. Our lab currently consists of 15 boards:

  • Atmel SAMA5D2 Xplained
  • Atmel SAMA5D3 Xplained
  • Atmel AT91SAM9X25EK
  • Atmel AT91SAM9X35EK
  • Atmel AT91SAMA5D36EK
  • Atmel AT91SAM9M10G45EK
  • Atmel AT91SAM9261EK
  • BeagleBone Black
  • Beagleboard-xM
  • Marvell Armada XP based Plathome Openblocks AX3
  • Marvell Armada 38x Solidrun ClearFog,
  • Marvell Armada 38x DB-88F6820-GP
  • Allwinner A13 Nextthing Co. C.H.I.P
  • Allwinner A33 Sinlinx SinA33
  • Freescale i.MX6 Boundary Devices Nitrogen6x

We will very soon be adding 4 more boards:

  • Atmel SAMA5D4 Xplained
  • Atmel SAMA5D34EK
  • Marvell Armada 7K 7040-DB (ARM64)
  • Marvell Armada 39x DB

Free Electrons board farm

Three of the boards we have were already tested thanks to other KernelCI labs, but the other sixteen boards were not tested at all. In total, we plan to have about 50 boards in our lab, mainly for the ARM platforms that we maintain in the official Linux kernel. The results of all boots we performed are visible on the KernelCI site. We are proud to be part of this unique effort to perform automated testing and validation of the Linux kernel!

In the coming weeks, we will publish additional articles to present the software and physical architecture of our lab and the program we developed to remotely control boards that are in our lab, so stay tuned!

by Quentin Schulz at June 17, 2016 10:07 AM

Buildroot training updated to Buildroot 2016.05

Buildroot LogoAlmost exactly one year ago, we announced the availability of our training course on Buildroot. This course received very good feedback, both from our customers, and from the community.

In our effort to continuously improve and update our training materials, we have recently updated our Buildroot training course to Buildroot 2016.05, which was released at the end of May. In addition to adapting our practical labs to use this new Buildroot version, we have also improved the training materials to cover some of the new features that have been added over the last year in Buildroot. The most important changes are:

  • Cover the graph-size functionality, which allows to generate a pie chart of the filesystem size, per package. This is a very nice feature to analyze the size of your root filesystem and see how to reduce it.
  • Improve the description about the local site method and the override source directory functionalities, that are very useful when doing active application/library development in Buildroot, or to package custom application/library code.
  • Add explanations about using genimage to create complete SD card images that are ready to be flashed.
  • Add explanations about the hash file that can be added to packages to verify the integrity of the source code that is downloaded before it gets built.

The updated training materials are available on the training page: agenda (PDF), slides (PDF) and practical labs (PDF).

Contact us if you would like to organize this training session in your company: we are available to deliver it worldwide.

by Thomas Petazzoni at June 17, 2016 07:57 AM

June 14, 2016

Bunnie Studios

Episode 2: Shenzhen and the Maker Movement

Woooooo episode 2 is out!

I wrote a post once about getting my phone’s screen fixed in Shenzhen. I’ve learned a lot from watching these phone repair guys do their thing in Shenzhen.

This video shows most of the process, from splitting the bonded LCD/digitizer assembly using a cutting wire and a heated vacuum chuck, to rebonding, to removing bubbles in the LOCA (liquid optically clear adhesive) by way of a vacuum chamber. There’s also typically a UV curing step that was probably left out of the segment for time reasons. The whole video is a good watch, but if you’re short on time, the segment on repairing a screen starts at 12:36.

by bunnie at June 14, 2016 07:23 PM

June 13, 2016

Bunnie Studios

Name that Ware, June 2016

The Ware for June 2016 is shown below.

Thanks to Liwei from TinyMOS for contributing the ware. He found it on his way to school many years ago. The function of this board is probably an easy guess, so bonus points to anyone who has a convincing idea about the larger system this was once a part of.

by bunnie at June 13, 2016 02:01 PM

Winner, Name that Ware May 2016

The Ware for May 2016 was guessed within the hour of posting — it’s an Antminer S1 (v1.4 mainboards) from BitMainTech.

Tracing through the rapid-fire guesses and picking a winner was a bit of a convoluted process. Based on my primary criteria of awarding to the first person to home in on a make/model of a ware, the winner is Wouter’s post at 10:15PM (congrats, btw email me for your prize).

However, if make/model isn’t guessed, I’d go with an alternate criteria of thoughtful analysis, which would give the prize to Richard Ames’ conclusion that it’s a cryptocurrency compute module posted at 10:06PM. However, even that decision is contracted by 0x3d’s post at 9:53PM, earlier than all the rest, that this is an ASIC cryptocoin miner — no make/model, but still the correct genre.

Also, in response to Richard Ames’ question: HDB = Housing Development Board. It’s the colloquial term in Singapore for public housing, after the government agency in charge of managing public housing.

by bunnie at June 13, 2016 02:01 PM

June 11, 2016

ZeptoBARS

OPUS Microsystems OP-6111 - MEMS 2D scanner : weekend die-shot

OPUS Microsystems OP-6111 is a sensor-less resonant 2D-tilting MEMS mirror for application requiring laser scanning (laser projectors, 3D scanning, photoresist exposure e.t.c). Electrostatic actuator.

Die size 3688x3180 µm, mirror is 1000 µm in diameter.


June 11, 2016 02:13 PM

June 10, 2016

Video Circuits

Telechrome Special Effects Generator

This is an interesting video effects system from 59, to put it into context quad video was only developed in the mid 50s, colour quad became available in 58, Telerecording/Kinescope film recording systems were still in use by the broadcast networks and the earliest machines to be discussed as "video synthesizers" were still a few years off.

Whats interesting about this system is it's relatively open modular nature, something that became less common as the technology became more advanced. The Telechrome system (not to be confused with Baird's early colour tube experiments) contains a waveform generator, a switching amplifier and a control unit for selecting effects. The waveform generator would be used to generate electronic mattes analogous to film mattes and the switching amplifier would place two distinct video sources either side of the geometric form.

Here is the control panel

 you can see each of the systems modules here




















And here is the advertisement in full.

















I also dug up a later system with a similarly modular aproach.























Sources here:
http://www.americanradiohistory.com/Archive-BC/BC-1959/1959-10-05-BC.pdf
http://saltofamerica.com/contents/displayArticle.aspx?19_393

by Chris (noreply@blogger.com) at June 10, 2016 12:20 AM

June 08, 2016

Bunnie Studios

WIRED Documentary on Shenzhen

WIRED is now running a multi-part video documentary on Shenzhen:

This shoot was a lot of fun, and it was a great pleasure working with Posy and Jim. I think their talent as producer and director really show through. They also did a great job editing my off-the-cuff narratives. The spot in the video where I’m pointing out Samsung parts isn’t matched to the b-roll of Apple parts, but in their defense I was moving so fast through the market that Jim couldn’t capture all the things I was pointing at.

I haven’t seen the whole documentary myself (I was just called in to give some tours of the market and answer a few questions in my hotel room), so I’m curious and excited to see where this is going! Especially because of the text chosen for printing during my Moore’s Law explanation at 3:13 — “ALL PROPRIETARY AND NO OPEN SOURCE MAKES INNOVATION A SLOW PROCESS.”

:)

by bunnie at June 08, 2016 06:33 AM

June 06, 2016

Harald Welte

Recent public allegations against Jacob Appelbaum

In recent days, various public allegations have been brought forward against Jacob Appelbaum. The allegations rank from plagiarism to sexual assault and rape.

I find it deeply disturbing that the alleged victims are putting up the effort of a quite slick online campaign to defame Jakes's name, using a domain name consisting of only his name and virtually any picture you can find online of him from the last decade, and - to a large extent - hide in anonymity.

I'm upset about this not because I happen to know Jake personally for many years, but because I think it is fundamentally wrong to bring up those accusations in such a form.

I have no clue what is the truth or what is not the truth. Nor does anyone else who has not experienced or witnessed the alleged events first hand. I'd hope more people would think about that before commenting on this topic one way or another on Twitter, in their blogs, on mailing lists, etc. It doesn't matter what we believe, hypothesize or project based on a personal like or dislike of either the person accused or of the accusers.

We don't live in the middle ages, and we have given up on the pillory for a long time (and the pillory was used after a judgement, not before). If there was illegal/criminal behavior, then our societies have a well-established and respected procedure to deal with such: It is based on laws, legal procedure and courts.

So if somebody has a claim, they can and should seek legal support and bring those claims forward to the competent authorities, rather than starting what very easily looks like a smear campaign (whether it is one or not).

Please don't get me wrong: I have the deepest respect and sympathies for victims of sexual assault or abuse - but I also have a deep respect for the legal foundation our societies have built over hundreds of years, and it's principles including the human right "presumption of innocence".

No matter who has committed which type of crime, everyone deserve to receive a fair trial, and they are innocent until proven guilty.

I believe nobody deserves such a public defamation campaign, nor does anyone have the authority to sentence such a verdict, not even a court of law. The Pillory was abandoned for good reasons.

by Harald Welte at June 06, 2016 10:00 AM

June 01, 2016

Harald Welte

Nuand abusing the term "Open Source" for non-free Software

Back in late April, the well-known high-quality SDR hardware company Nuand published a blog post about an Open Source Release of a VHDL ADS-B receiver.

I was quite happy at that time about this, and bookmarked it for further investigation at some later point.

Today I actually looked at the source code, and more by coincidence noticed that the LICENSE file contains a license that is anything but Open Source: The license is a "free for evaluation only" license, and it is only valid if you run the code on an actual Nuand board.

Both of the above are clearly not compatible with any of the well-known and respected definitions of Open Source, particularly not the official Open Source Definition of the Open Source Initiative.

I cannot even start how much this makes me upset. This is once again openwashing, where something that clearly is not Free or Open Source Software is labelled and marketed as such.

I don't mind if an author chooses to license his work under a proprietary license. It is his choice to do so under the law, and it generally makes such software utterly unattractive to me. If others still want to use it, it is their decision. However, if somebody produces or releases non-free or proprietary software, then they should make that very clear and not mis-represent it as something that it clearly isn't!

Open-washing only confuses everyone, and it tries to market the respective company or product in a light that it doesn't deserve. I believe the proper English proverb is to adorn oneself with borrowed plumes.

I strongly believe the community must stand up against such practise and clearly voice that this is not something generally acceptable or tolerated within the Free and Open Source software world. It's sad that this is happening more frequently, like recently with OpenAirInterface (see related blog post).

I will definitely write an e-mail to Nuand management requesting to correct this mis-representation. If you agree with my posting, I'd appreciate if you would contact them, too.

by Harald Welte at June 01, 2016 10:00 AM

May 27, 2016

Harald Welte

Keynote at Black Duck Korea Open Source Conference

I've been giving a keynote at the Black Duck Korea Open Source Conference yesterday, and I'd like to share some thoughts about it.

In terms of the content, I spoke about the fact that the ultimate goal/wish/intent of free software projects is to receive contributions and for all of the individual and organizational users to join the collaborative development process. However, that's just the intent, and it's not legally required.

Due to GPL enforcement work, a lot of attention has been created over the past ten years in the corporate legal departments on how to comply with FOSS license terms, particularly copyleft-style licenses like GPLv2 and GPLv3. However,

License compliance ensures the absolute bare legal minimum on engaging with the Free Software community. While that is legally sufficient, the community actually wants to have all developers join the collaborative development process, where the resources for development are contributed and shared among all developers.

So I think if we had more contribution and a more fair distribution of the work in developing and maintaining the related software, we would not have to worry so much about legal enforcement of licenses.

However, in the absence of companies being good open source citizens, pulling out the legal baton is all we can do to at least require them to share their modifications at the time they ship their products. That code might not be mergeable, or it might be outdated, so it's value might be less than we would hope for, but it is a beginning.

Now some people might be critical of me speaking at a Black Duck Korea event, where Black Duck is a company selling (expensive!) licenses to proprietary tools for license compliance. Thereby, speaking at such an event might be seen as an endorsement of Black Duck and/or proprietary software in general.

Honestly, I don't think so. If you've ever seen a Black Duck Korea event, then you will notice there is no marketing or sales booth, and that there is no sales pitch on the conference agenda. Rather, you have speakers with hands-on experience in license compliance either from a community point of view, or from a corporate point of view, i.e. how companies are managing license compliance processes internally.

Thus, the event is not a sales show for proprietary software, but an event that brings together various people genuinely interested in license compliance matters. The organizers very clearly understand that they have to keep that kind of separation. So it's actually more like a community event, sponsored by a commercial entity - and that in turn is true for most technology conferences.

So I have no ethical problems with speaking at their event. People who know me, know that I don't like proprietary software at all for ethical reasons, and avoid it personally as far as possible. I certainly don't promote Black Ducks products. I promote license compliance.

Let's look at it like this: If companies building products based on Free Software think they need software tools to help them with license compliance, and they don't want to develop such tools together in a collaborative Free Software project themselves, then that's their decision to take. To state using words of Rosa Luxemburg:

Freedom is always the freedom of those who think different

I may not like that others want to use proprietary software, but if they think it's good for them, it's their decision to take.

by Harald Welte at May 27, 2016 01:00 AM

May 26, 2016

Harald Welte

Osmocom.org GTP-U kernel implementation merged mainline

Have you ever used mobile data on your phone or using Tethering?

In packet-switched cellular networks (aka mobile data) from GPRS to EDGE, from UMTS to HSPA and all the way into modern LTE networks, there is a tunneling protocol called GTP (GPRS Tunneling Protocol).

This was the first cellular protocol that involved transport over TCP/IP, as opposed to all the ISDN/E1/T1/FrameRelay world with their weird protocol stacks. So it should have been something super easy to implement on and in Linux, and nobody should have had a reason to run a proprietary GGSN, ever.

However, the cellular telecom world lives in a different universe, and to this day you can be safe to assume that all production GGSNs are proprietary hardware and/or software :(

In 2002, Jens Jakobsen at Mondru AB released the initial version of OpenGGSN, a userspace implementation of this tunneling protocol and the GGSN network element. Development however ceased in 2005, and we at the Osmocom project thus adopted OpenGGSN maintenance in 2016.

Having a userspace implementation of any tunneling protocol of course only works for relatively low bandwidth, due to the scheduling and memory-copying overhead between kernel, userspace, and kernel again.

So OpenGGSN might have been useful for early GPRS networks where the maximum data rate per subscriber is in the hundreds of kilobits, but it certainly is not possible for any real operator, particularly not at today's data rates.

That's why for decades, all commonly used IP tunneling protocols have been implemented inside the Linux kernel, which has some tunneling infrastructure used with tunnels like IP-IP, SIT, GRE, PPTP, L2TP and others.

But then again, the cellular world lives in a universe where Free and Open Source Software didn't exit until OpenBTS and OpenBSC changed all o that from 2008 onwards. So nobody ever bothered to add GTP support to the in-kernel tunneling framework.

In 2012, I started an in-kernel implementation of GTP-U (the user plane with actual user IP data) as part of my work at sysmocom. My former netfilter colleague and current netfilter core team leader Pablo Neira was contracted to bring it further along, but unfortunately the customer project funding the effort was discontinued, and we didn't have time to complete it.

Luckily, in 2015 Andreas Schultz of Travelping came around and has forward-ported the old code to a more modern kernel, fixed the numerous bugs and started to test and use it. He also kept pushing Pablo and me for review and submission, thanks for that!

Finally, in May 2016, the code was merged into the mainline kernel, and now every upcoming version of the Linux kernel will have a fast and efficient in-kernel implementation of GTP-U. It is configured via netlink from userspace, where you are expected to run a corresponding daemon for the control plane, such as either OpenGGSN, or the new GGSN + PDN-GW implementation in Erlang called erGW.

You can find the kernel code at drivers/net/gtp.c, and the userspace netlink library code (libgtpnl) at git.osmocom.org.

I haven't done actual benchmarking of the performance that you can get on modern x86 hardware with this, but I would expect it to be the same of what you can also get from other similar in-kernel tunneling implementations.

Now that the cellular industry has failed for decades to realize how easy and little effort would have been needed to have a fast and inexpensive GGSN around, let's see if now that other people did it for them, there will be some adoption.

If you're interested in testing or running a GGSN or PDN-GW and become an early adopter, feel free to reach out to Andreas, Pablo and/or me. The osmocom-net-gprs mailing list might be a good way to discuss further development and/or testing.

by Harald Welte at May 26, 2016 10:00 AM

May 25, 2016

Free Electrons

Linux 4.6 released, with Free Electrons contributions

Adelie PenguinThe 4.6 version of the Linux kernel was released last Sunday by Linus Torvalds. As usual, LWN.net had a very nice coverage of this development cycle merge window, highlighting the most significant changes and improvements: part 1, part 2 and part 3. KernelNewbies is now active again, and has a very detailed page about this release.

On a total of 13517 non-merge commits, Free Electrons contributed for this release a total of 107 non-merge commits, a number slightly lower than our past contributions for previous kernel releases. That being said, there are still a few interesting contributions in those 107 patches. We are particular happy to see patches from all our eight engineers in this release, including from Mylène Josserand and Romain Perier, who just joined us mid-March! We also already have 194 patches lined-up for the next 4.7 release.

Here are the highlights of our contributions to the 4.6 release:

  • Atmel ARM processors support
    • Alexandre Belloni and Boris Brezillon contributed a number of patches to improve and cleanup the support for the PMC (Power Management and Clocks) hardware block. As expected, this involved patching both clock drivers and power management code for the Atmel platforms.
  • Annapurna Labs Alpine platforms support
    • As a newly appointed maintainer of the Annapurna Labs ARM/ARM64 Alpine platforms, Antoine Ténart contributed the base support for the ARM64 Alpine v2 platform: base platform support and Device Tree, and an interrupt controller driver to support MSI-X
  • Marvell ARM processors support
    • Grégory Clement added initial support for the Armada 3700, a new Cortex-A53 based ARM64 SoC from Marvell, as well as a first development board using this SoC. So far, the supported features are: UART, USB and SATA (as well as of course timers and interrupts).
    • Thomas Petazzoni added initial support for the Armada 7K/8K, a new Cortex-A72 based ARM64 SoC from Marvell, as well as a first development board using this SoC. So far, UART, I2C, SPI are supported. However, due to the lack of clock drivers, this initial support can’t be booted yet, the clock drivers and additional support is on its way to 4.7.
    • Thomas Petazzoni contributed an interrupt controller driver for the ODMI interrupt controller found in the Armada 7K/8K SoC.
    • Grégory Clement and Thomas Petazzoni did a few improvements to the support of Armada 38x. Thomas added support for the NAND flash used on Armada 370 DB and Armada XP DB.
    • Boris Brezillon contributed a number of fixes to the Marvell CESA driver, which is used to control the cryptographic engine found in most Marvell EBU processors.
    • Thomas Petazzoni contributed improvements to the irq-armada-370-xp interrupt controller driver, to use the new generic MSI infrastructure.
  • Allwinner ARM processors support
    • Maxime Ripard contributed a few improvements to Allwinner clock drivers, and a few other fixes.
  • MTD and NAND flash subsystem
    • As a maintainer of the NAND subsystem, Boris Brezillon did a number of contributions in this area. Most notably, he added support for the randomizer feature to the Allwinner NAND driver as well as related core NAND subsystem changes. This change is needed to support MLC NANDs on Allwinner platforms. He also contributed several patches to continue clean up and improve the NAND subsystem.
    • Thomas Petazzoni fixed an issue in the pxa3xx_nand driver used on Marvell EBU platforms that prevented using some of the ECC configurations (such as 8 bits BCH ECC on 4 KB pages). He also contributed minor improvements to the generic NAND code.
  • Networking subsystem
    • Grégory Clement contributed an extension to the core networking subsystem that allows to take advantage of hardware capable of doing HW-controlled buffer management. This new extension is used by the mvneta network driver, useful for several Marvell EBU platforms. We expect to extend this mechanism further in the future, in order to take advantage of additional hardware capabilities.
  • RTC subsystem
    • As a maintainer of the RTC subsystem, Alexandre Belloni did a number of fixes and improvements in various RTC drivers.
    • Mylène Josserand contributed a few improvements to the abx80x RTC driver.
  • Altera NIOSII support
    • Romain Perier contributed two patches to fix issues in the kernel running on the Altera NIOSII architecture. The first one, covered in a previous blog post, fixed the NIOSII-specific memset() implementation. The other patch fixes a problem in the generic futex code.

In addition, several our of engineers are maintainers of various platforms or subsystems, so they do a lot of work reviewing and merging the contributions from other kernel developers. This effort can be measured by looking at the number of patches on which they Signed-off-by, but for which they are not the author. Here are the number of patches that our engineered Signed-off-by, but for which they were not the author:

  • Alexandre Belloni, as the RTC subsystem maintainer and the Atmel ARM co-maintainer: 91 patches
  • Maxime Ripard, as the Allwinner ARM co-maintainer: 65 patches
  • Grégory Clement, as the Marvell EBU ARM co-maintainer: 45 patches
  • Thomas Petazzoni, simply resubmitting patches from others: 2 patches

Here is the detailed list of our contributions to the 4.6 kernel release:

by Thomas Petazzoni at May 25, 2016 07:10 AM

May 23, 2016

Bunnie Studios

Name that Ware, May 2016

The Ware for May 2016 is shown below.

Xobs discovered this morsel of technology sitting in the junk pile at his HDB, and brought it into the office for me to have a look at. I hadn’t seen of these first-hand until then.

Despite being basically a picture of two large hunks of metal, I’m guessing this ware will be identified within minutes of going up.

by bunnie at May 23, 2016 01:20 PM

Winner, Name that Ware April 2016

Really great participation this month in Name that Ware!

The Ware for April 2016 is a “LED-Handbrause” by miomare — in other words, a shower head with LEDs on the inside which tell you the temperature of the water. It has an integral paddlewheel that generates power for the circuitry via water flowing through the shower head, as evidenced by this more complete photo of the ware:

It looks like LW was the first to guess the function of the ware, so congrats! email me for your prize. And thanks again to Philipp Gühring for submitting a ware that sparked so much interesting discussion!

by bunnie at May 23, 2016 01:20 PM

May 22, 2016

Elphel

Tutorial 02: Eclipse-based FPGA development environment for Elphel cameras

Elphel cameras offer unique capabilities – they are high performance systems out of the box and have all the firmware and FPGA code distributed under GNU General Public Licenses making it possible for users to modify any part of the code. The project does not use any “black boxes” or encrypted modules, so it is simulated with the free software tools and user has access to every net in the design. We are trying to do our best to make this ‘hackability’ not just a theoretical possibility, but a practical one.

Current camera FPGA project contains over 400 files under version control and almost 100K lines of HDL (Verilog) code, there are also constraints files, tool configurations, so we need to provide means for convenient navigation and modification of the project by the users.

We are starting a series of tutorials to facilitate acquaintance with this project, and here is the first one that shows how to install and configure the software. This tutorial is made with a fresh Kubuntu 16.04 LTS distribution installed on a virtual machine – this flavor of GNU/Linux we use ourselves and so it is easier for us to help others in the case of problems, but it should be also easy to install it on other GNU/Linux systems.

Later we plan to show how to navigate code and view/modify tool parameters with VDT plugin, run simulation and implementation tools. Next will be a “Hello world” module added to the camera code base, then some simple module that accesses the video memory.



Video resolution is 1600×900 pixels, so full screen view is recommended.

Download links for: video and captions.

Running this software does not require to have an actual camera, so it may help our potential users to evaluate software capabilities and see if it matches their requirements before purchasing an actual hardware. We will also be able to provide remote access to the cameras in our office for experimenting with them.

by Andrey Filippov at May 22, 2016 10:20 PM

ZeptoBARS

Silicon Labs Si8641 - quad channel digital isolator : weekend die-shot

Silicon Labs Si8641 uses capacitive coupling to implement digital isolation (up to 5kV, this model 2.5kV) as speeds of up to 1 Mbps.
This particular model (Si8641AB) contains 2 identical dies, apparently configured by bonding some of the pads on the sides.




May 22, 2016 03:50 AM

May 21, 2016

Harald Welte

Slovenian student sentenced for detecting TETRA flaws using OsmocomTETRA

According to some news report, including this report at softpedia, a 26 year old student at the Faculty of Criminal Justice and Security in Maribor, Slovenia has received a suspended prison sentence for finding flaws in Slovenian police and army TETRA network using OsmocomTETRA

As the Osmocom project leader and main author of OsmocomTETRA, this is highly disturbing news to me. OsmocomTETRA was precisely developed to enable people to perform research and analysis in TETRA networks, and to audit their safe and secure configuration.

If a TETRA network (like any other network) is configured with broken security, then the people responsible for configuring and operating that network are to be blamed, and not the researcher who invests his personal time and effort into demonstrating that police radio communications safety is broken. On the outside, the court sentence really sounds like "shoot the messenger". They should instead have jailed the people responsible for deploying such an insecure network in the first place, as well as those responsible for not doing the most basic air-interface interception tests before putting such a network into production.

According to all reports, the student had shared the results of his research with the authorities and there are public detailed reports from 2015, like the report (in Slovenian) at https://podcrto.si/vdor-v-komunikacijo-policije-razkril-hude-varnostne-ranljivosti-sistema-tetra/.

The statement that he should have asked the authorities for permission before starting his research is moot. I've seen many such cases and you would normally never get permission to do this, or you would most likely get no response from the (in)competent authorities in the first place.

From my point of view, they should give the student a medal of honor, instead of sentencing him. He has provided a significant service to the security of the public sector communications in his country.

To be fair, the news report also indicates that there were other charges involved, like impersonating a police officer. I can of course not comment on those.

Please note that I do not know the student or his research first-hand, nor did I know any of his actions or was involved in them. OsmocomTETRA is a Free / Open Source Software project available to anyone in source code form. It is a vital tool in demonstrating the lack of security in many TETRA networks, whether networks for public safety or private networks.

by Harald Welte at May 21, 2016 10:00 PM

May 20, 2016

Video Circuits

Video Synthesis Techniques

So I am doing a talk at Raven Row as part of their very lovely exhibition of Steina & Woody Vasulka's work, will be demoing some of the techniques they used to make their work. Hopfully it will be interesting.

"Chris King: Video Circuits
Thursday 2 June, 6.30pm

Artist Chris King leads a live demonstration of early media art and video synthesis technologies, working with a selection of different techniques used by the Vasulkas and other video artists during the 1970s and 80s.Chris King: Video Circuits
Thursday 2 June, 6.30pm

Artist Chris King leads a live demonstration of early media art and video synthesis technologies, working with a selection of different techniques used by the Vasulkas and other video artists during the 1970s and 80s."
http://www.ravenrow.org/events/chris_king_video_circuits/

by Chris (noreply@blogger.com) at May 20, 2016 03:34 AM

DIY Colouriser

Here are some snaps of the colouriser I built for my DIY system based on the board from the Visualist






by Chris (noreply@blogger.com) at May 20, 2016 03:28 AM

Seeing Sound

Here are some shots from my talk, the video circuits screening, Alex's piece and Andrews performance at Seeing Sound, It was great fun. 








by Chris (noreply@blogger.com) at May 20, 2016 03:23 AM

May 16, 2016

Free Electrons

Linux kernel support for Microcrystal RTCs

micro-crystalThanks to Microcrystal, a Switzerland-based real-time clock vendor, Free-Electrons has contributed support for a number of new I2C and SPI based real-time clocks to the Linux kernel over the last few months. More specifically, we added or improved support for the Microcrystal RV-1805, RV-4162, RV-3029 and RV-3049. In this blog post, we detail the contributions we have done to support those real-time clocks.

RV-1805

The RV-1805 RTC is similar to the Abracon 1805 one, for which a driver already existed in the Linux kernel. Therefore, the support for the RV-1805 RTC was added in the same driver, rtc-abx80x.c. The patch which adds the support of this RTC is already upstream since v4.5 (see Free-Electrons contributions to linux 4.5). In this kernel version, the support of the alarm has also been added. In the 4.6 kernel release, the support for two additional functionalities has been contributed: oscillator selection and handling of oscillator failure.

The oscillator selection functionality allows to select between the two oscillators available in this RTC:

  • The XT oscillator, a more stable, but also more power-hungy oscillator
  • The RC oscillator, a less accurate, but also more power-efficient oscillator

This patch adds the possibility to select which oscillator the RTC should use and also, a way to configure the auto-calibration (auto-calibration is a feature to calibrate the RC oscillator using the digital XT oscillator).

To select the oscillator, a sysfs entry has been added:

cat /sys/class/rtc/rtc0/device/oscillator

To configure and activate the autocalibration, another sysfs entry has been added:

cat /sys/class/rtc/rtc0/device/autocalibration

Here is an example of using RC oscillator and an autocalibration of 512 seconds cycle.

echo rc > /sys/class/rtc/rtc0/device/oscillator
echo 512 > /sys/class/rtc/rtc0/device/autocalibration

The other functionality that was added is handling the Oscillator Failure situation (see this patch). The Oscillator Failure is detected when the XT oscillator generates ticks at less than 8 kHz for more than 32 ms. In this case, the date and time can be wrong so an error is returned when an attempt to read the date from the RTC is made. This Oscillator Failure condition is cleared when a new date/time is set into the RTC.

RV-4162

The RV-4162 RTC is similar to ST M41T80 RTC family, so the existing driver has been used as well. However, as this driver was quite old, eight patches have been contributed to update the driver to newer APIs and to add new functionalities such as oscillator failure and alarm. The patches have already been merged by RTC maintainer Alexandre Belloni and should therefore find their way into the 4.7 Linux kernel release:

See [PATCH 0/8] rtc: m41t80: update and add functionalities for the entire patch series. Thanks to this project, the RV-4162 is now supported in the Linux Kernel and the entire family of I2C-based M41T80 RTCs will benefit from these improvements.

RV-3029 / RV-3049

The RV-3029 RTC driver already existed in the Linux kernel, and the the RV-3049 is the same reference than the RV-3029 but it is an SPI-based interface instead of an I2C one. This is a typical case where the regmap mechanism of the Linux kernel is useful: it allows to abstract the register accesses, regardless of the bus being used to communicate with the hardware. Thanks to this, a single driver can easily handle two devices that are interfaced over different busses, but offering the same register set, which is the case with RV-3029 on I2C and RV-3049 on SPI.

For this driver, some updates were needed to prepare the switch to using the regmap mechanism. Once the driver had been converted to regmap and worked as before, the RV-3049 support has been added. Finally, the alarm functionality has been added and fixed. The corresponding patches have already been merged by the RTC maintainer, and should therefore also be part of Linux 4.7:

Conclusion

It is great to see hardware vendors actively engaged in having support for their hardware in the upstream Linux kernel. This way, their users can immediately use the kernel version of their choice on their platform, without having to mess with outdated out-of-tree drivers. Thanks to Microcrystal for supporting this work!

Do not hesitate to contact us if you would like to see your hardware supported in the official Linux kernel. Read more about our Linux kernel upstreaming services.

by Mylène Josserand at May 16, 2016 03:23 PM

May 11, 2016

ZeptoBARS

You can now support Zeptobars at Patreon and more


We are running this blog for more than 3 years with no monetization of any kind (advertisements, merchandise and such) but from time to time people kept asking on how to help us. From the other side - we've probably reached the limits of our resources to improve the quality of our lab/imaging setup, and we'll need your help to move further.

We've finally outlined a number of voluntary ways you can support our efforts to produce higher-quality microchip photographs for curiosity and education.

Basically, there are 4 ways - spread the word, send us few cool chips for future work, support us at our Patreon campaign (which would allow you to schedule small contribution for each new die shot we publish) or send us some Bitcoins (or use good old Paypal).

Either way content of this blog will remain free for everyone and it will continue to be licensed under permissive CC BY 3.0 license.

May 11, 2016 04:36 AM

May 10, 2016

Elphel

3D Print Your Camera Freedom

Two weeks ago we were making photos of our first production NC393 camera to post an announcement of the new product availability. We got all the mechanical parts and most of the electronic boards (14MPix version will be available shortly) and put them together. Nice looking camera, powered by a high performance SoC (dual ARM plus FPGA), packaged in a lightweight aluminum extrusion body, providing different options for various environments – indoors, outdoors, on board of the UAV or even in the open space with no air (cooling is important when you run most of the FPGA resources at full speed). Tons of potential possibilities, but the finished camera did not seem too exciting – there are so many similar looking devices available.

NC393 camera, front view

NC393 camera, back panel view. Includes DC power input (12-36V and 20-75V options), GigE, microSD card (bootable), microUSB(type B) connector for a system console with reset and boot source selection, USB/eSATA combo connector, microUSB(type A) and 2.5mm 4-contact barrel connector for external synchronization I/O

NC393 assembled boards: 10393(system board), 10385 (power supply board), 10389(interface board), 10338e (sensor board) and 103891 - synchronization adapter board, view from 10389. m.2 2242 SSD shown, bracket for the 2260 format provided. 10389 internal connectors include inter-camera synchronization and two of 3.3VDC+5.0VDC+I2C+USB ones.

NC393 assembled boards: 10393(system board), 10385 (power supply board), 10389(interface board), 10338e (sensor board) and 103891 - synchronization adapter board, view from 10385

10393 system board attached to the heat frame, view from the heat frame. There is a large aluminum heat spreader attached to the other side of the frame with thermal conductive epoxy that provides heat transfer from the CPU without the use of any spring load. Other heat dissipating components use heat pads.

10393 system board attached to the heat frame, view from the 10393 board

10393 system board, view from the processor side

An obvious reason for our dissatisfaction is that the single-sensor camera uses just one of four available sensor ports. Of course it is possible to use more of the freed FPGA resources for a single image processing, but it is not what you can use out of the box. Many of our users buy camera components and arrange them in their custom setup themselves – that does not have a single-sensor limitation and it matches our goals – make it easy to develop a custom system, or sculpture the camera to meet your ideas as stated on our web site. We would like to open the cameras to those who do not have capabilities of advanced mechanical design and manufacturing or just want to try new camera ideas immediately after receiving the product.

Why multisensor?

One simple answer can be “because we can” – the CPU+FPGA based camera system can simultaneously handle multiple small image sensors we love – sensors perfected by the cellphone industry. Of course it is also possible to connect one large (high resolution/high FPS) sensor or even to use multiple camera system for one really fast sensor – we did such trick with the NC323 camera for book scanning, but we believe that the future is with the multiple view systems that combine images from several synchronized cameras using computational photography rather than large lens/large sensor traditional cameras.

Multi-sensor systems can acquire high-resolution panoramic images in a single shot (or offer full sphere live video), they can be used for image-based 3-d reconstruction that in many cases provide much superior quality to the now traditional LIDAR-based scanners which can not output cinematographic quality 3-d scenes. They can be used to capture HDR video by combining data for the same voxels rather than pixels. Such systems can easily beat the shallow depth of field of the traditional large format cameras and offer possibility of the post-production focus distance adjustment. Applications are virtually endless, and while at Elphel we are developing such multi-sensor systems our main products are still the high-performance camera systems hackable at any imaginable level.

Prototype of the 21-sensor 3D HDR 6K cinematographic camera

Eyesis4π stereophotogrammetric camera

NC353-369-PHG3 3-sensor camera camera, view demo (mouse scroll changes disparity, shift-scroll - zoom)

Spherical view camera with two fish eye lenses

Two sensor stereo camera with two interchangeable C/CS-mount lenses

SCINI project underwater remotely operated vehicle (ROV)

A helmet-mounted panoramic camera by HomeSide 720°

Quadcopter using multisensor camera for navigation, by 'Autonomous Aerospace' team in Krasnoyarsk, Russia

Multisensor R5 camera by Google

Hackable by Design

To have all documentation open and released under free licenses such as GNU GPL and CERN OHL is a precondition, but it is not sufficient. The hackable products must be designed to be used that way and we strive to provide this functionality to our users. This is true for the products themselves and for the required tools, so we had to go as far as to develop software for FPGA tools integration with the popular Eclipse IDE and replace closed source manufacturer code that is not compatible with the free software Verilog simulators.

Same is true for the camera mechanical parts – users need to be able to reconfigure not just the firmware, FPGA code or rearrange the electronic components, but to change the physical layout of their systems. One popular solution to this challenge is to offer modular camera systems, but unfortunately this approach has its limits. It is similar to Lego® sets (where kids can assemble just one object already designed by the manufacturer) vs. Lego® bricks where the possibilities are limited by the imagination only. Often camera modularity is more about marketing (suggesting that you can start with a basic less expensive set and later buy more parts) than about the real user freedom.

We too provide modular components and try to maintain compatibility between the generations of modules – new Elphel cameras can directly interface more than a decade old sensor boards and this does not prevent them from simultaneously supporting modern sensor interfaces. Physical dimensions and shapes of the camera electronic boards also remain the same – they just pack more performance in the same volume as newer components become available. Being in the business of developing hackable cameras for 15 years, we realize that the modularity alone is not a magic bullet. Luckily now there are other possibilities.

3d printing camera parts

3d printing process offers freedom in the material world but so far we were pessimistic about its use for the camera components where microns often matter. Some of the camera modules use invar (metal alloy that has almost zero thermal expansion coefficient at normal temperatures) elements to compensate for the thermal expansion, and the PLA plastic parts seem rather alien here. Nevertheless it is possible to meet the requirements of the camera mechanical design even with this material. In some cases it is sufficient to have precise and stable sensor/lens combination – sensor front end (SFE), small fluctuations in the mutual position/orientation of the individual SFE may be compensated using image data itself in the overlapping areas. It is possible to design composite structure that combines metal elements of simple shape (such as aluminum, thin wall stainless steel tubes or even small diameter invar rods) and printed elements of complex shape. Modern fiber-reinforced material for 3d-printing promise to improve mechanical stability and reduce thermal expansion of the finished parts.

This technology perfectly fits to the hackable multi-sensor systems and fills important missing part of “sculpturing” the user camera. 3-d printing is slow and we can not print every camera, but that is not really needed. While we certainly can print some parts, we are counting that this technology is now available in  most parts of the world where we ship the products, and the parts can be manufactured by the end user. We anticipate that many of the customer designs being experimental by nature will need later modifications, building the parts by the user can save on the overseas shipments too.

We count that the users will design their own parts, but we will try to make their job easier and provide modifiable design examples and fragments that can be used in their parts. This idea of incorporating 3-d printing technology into Elphel products is just 2 weeks old and we prepared several quick design prototypes to try it – below are some examples of our first generation of such camera parts.

Panoramic camera with perfect stitching - it uses 2 center cameras to measure distances

Stereo camera with 4 sensors having 1:3:2 bases providing all integer 1 to 6 multiples of 43mm in the lens pairs

Rectangular arranged 4-sensor stereo camera, adjustable bases

Short-base (48mm form center) 4-sensor camera

Printed adapter for the SFE of the 4-sensor panoramic camera

Printed adapter for the short-base 4-sensor camera

Various 3-d printed camera parts

It takes about 3 hours to print one SFE adapter

Deliverables

“3d print your camera freedom” – we really mean that. It is not about printing of a camera or its body. You can always get a complete camera in one of the available configurations packaged in a traditional all-metal body if it matches your ideas, printing just adds freedom to the mechanical design.

We will continue to provide all the spectrum of the camera components such as assembled boards and sensor front ends as well as the complete cameras in multiple configurations. For the 3-d printed versions we will have the models and reusable design fragments posted online. We will be able to print some parts and ship the factory assembled cameras. In some cases we may be able to help with the mechanical design, but we try to avoid doing any custom design ourselves. We consider our job is done well if we are not needed to modify anything for the end user. Currently we use one of the proprietary mechanical CAD programs so we do not have fully editable models and can only provide exported STEP files of the complete parts and interface fragments that can be incorporated in the user designs.

We would like to learn how to do this in FreeCAD – then it will be possible to provide the usable source files and detailed instructions how to customize them. FreeCAD environment can be used to create custom generator scripts in Python – this powerful feature helped us to convert all our mechanical design files into x3d models that can be viewed and navigated in the browser (video tutorial). This web based system proved to be not just a good presentation tool but to be more convenient for parts navigation than the CAD program itself, we use it ourselves regularly for that purpose.

Maybe we’ll be able to find somebody who is both experienced in mechanical design in FreeCAD and interested in multi-sensor camera systems to cooperate on this project?

by Andrey Filippov at May 10, 2016 07:31 PM

ZeptoBARS

ST HCF4056 - BCD to 7 segment : weekend die-shot

ST HCF4056 is a CMOS BCD to 7 segment decoder/driver with strobed latch.


May 10, 2016 03:17 PM

May 08, 2016

Andrew Zonenberg, Silicon Exposed

Open Verilog flow for Silego GreenPak4 programmable logic devices

I've written a couple of posts in the past few months but they were all for the blog at work so I figured I'm long overdue for one on Silicon Exposed.

So what's a GreenPak?


Silego Technology is a fabless semiconductor company located in the SF Bay area, which makes (among other things) a line of programmable logic devices known as GreenPak. Their 5th generation parts were just announced, but I started this project before that happened so I'm still targeting the 4th generation.

GreenPak devices are kind of like itty bitty PSoCs - they have a mixed signal fabric with an ADC, DACs, comparators, voltage references, plus a digital LUT/FF fabric and some typical digital MCU peripherals like counters and oscillators (but no CPU).

It's actually an interesting architecture - FPGAs (including some devices marketed as CPLDs) are a 2D array of LUTs connected via wires to adjacent cells, and true (product term) CPLDs are a star topology of AND-OR arrays connected by a crossbar. GreenPak, on the other hand, is a star topology of LUTs, flipflops, and analog/digital hard IP connected to a crossbar.

Without further ado, here's a block diagram showing all the cool stuff you get in the SLG46620V:

SLG46620V block diagram (from device datasheet)
They're also tiny (the SLG46620V is a 20-pin 0.4mm pitch STQFN measuring 2x3 mm, and the lower gate count SLG46140V is a mere 1.6x2 mm) and probably the cheapest programmable logic device on the market - $0.50 in low volume and less than $0.40 in larger quantities.

The Vdd range of GreenPak4 is huge, more like what you'd expect from an MCU than an FPGA! It can run on anything from 1.8 to 5V, although performance is only specified at 1.8, 3.3, and 5V nominal voltages. There's also a dual-rail version that trades one of the GPIO pins for a second power supply pin, allowing you to interface to logic at two different voltage levels.

To support low-cost/space-constrained applications, they even have the configuration memory on die. It's one-time programmable and needs external Vpp to program (presumably Silego didn't want to waste die area on charge pumps that would only be used once) but has a SRAM programming mode for prototyping.

The best part is that the development software (GreenPak Designer) is free of charge and provided for all major operating systems including Linux! Unfortunately, the only supported design entry method is schematic entry and there's no way to write your design in a HDL.

While schematics may be fine for quick tinkering on really simple designs, they quickly get unwieldy. The nightmare of a circuit shown below is just a bunch of counters hooked up to LEDs that blink at various rates.

Schematic from hell!
As if this wasn't enough of a problem, the largest GreenPak4 device (the SLG46620V) is split into two halves with limited routing between them, and the GUI doesn't help the user manage this complexity at all - you have to draw your schematic in two halves and add "cross connections" between them.

The icing on the cake is that schematics are a pain to diff and collaborate on. Although GreenPak schematics are XML based, which is a touch better than binary, who wants to read a giant XML diff and try to figure out what's going on in the circuit?

This isn't going to be a post on the quirks of Silego's software, though - that would be boring. As it turns out, there's one more exciting feature of these chips that I didn't mention earlier: the configuration bitstream is 100% documented in the device datasheet! This is unheard of in the programmable logic world. As Nick of Arachnid Labs says, the chip is "just dying for someone to write a VHDL or Verilog compiler for it". As you can probably guess by from the title of this post, I've been busy doing exactly that.

Great! How does it work?


Rather than wasting time writing a synthesizer, I decided to write a GreenPak technology library for Clifford Wolf's excellent open source synthesis tool, Yosys, and then make a place-and-route tool to turn that into a final netlist. The post-PAR netlist can then be loaded into GreenPak Designer in order to program the device.

The first step of the process is to run the "synth_greenpak4" Yosys flow on the Verilog source. This runs a generic RTL synthesis pass, then some coarse-grained extraction passes to infer shift register and counter cells from behavioral logic, and finally maps the remaining logic to LUT/FF cells and outputs a JSON-formatted netlist.

Once the design has been synthesized, my tool (named, surprisingly, gp4par) is then launched on the netlist. It begins by parsing the JSON and constructing a directed graph of cell objects in memory. A second graph, containing all of the primitives in the device and the legal connections between them, is then created based on the device specified on the command line. (As of now only the SLG46620V is supported; the SLG46621V can be added fairly easily but the SLG46140V has a slightly different microarchitecture which will require a bit more work to support.)

After the graphs are generated, each node in the netlist graph is assigned a numeric label identifying the type of cell and each node in the device graph is assigned a list of legal labels: for example, an I/O buffer site is legal for an input buffer, output buffer, or bidirectional buffer.

Example labeling for a subset of the netlist and device graphs
The labeled nodes now need to be placed. The initial placement uses a simple greedy algorithm to create a valid (although not necessarily optimal or even routable) placement:
  1. Loop over the cells in the netlist. If any cell has a LOC constraint, which locks the cell to a specific physical site, attempt to assign the node to the specified site. If the specified node is the wrong type, doesn't exist, or is already used by another constrained node, the constraint is invalid so fail with an error.
  2. Loop over all of the unconstrained cells in the netlist and assign them to the first unused site with the right label. If none are available, the design is too big for the device so fail with an error.
Once the design is placed, the placement optimizer then loops over the design and attempts to improve it. A simulated annealing algorithm is used, where changes to the design are accepted unconditionally if they make the placement better, and with a random, gradually decreasing probability if they make it worse. The optimizer terminates when the design receives a perfect score (indicating an optimal placement) or if it stops making progress for several iterations. Each iteration does the following:
  1. Compute a score for the current design based on the number of unroutable nets, the amount of routing congestion (number of nets crossing between halves of the device), and static timing analysis (not yet implemented, always zero).
  2. Make a list of nodes that contributed to this score in some way (having some attached nets unroutable, crossing to the other half of the device, or failing timing).
  3. Remove nodes from the list that are LOC'd to a specific location since we're not allowed to move them.
  4. Remove nodes from the list that have only one legal placement in the device (for example, oscillator hard IP) since there's nowhere else for them to go.
  5. Pick a node from the remainder of the list at random. Call this our pivot.
  6. Find a list of candidate placements for the pivot:
    1. Consider all routable placements in the other half of the device.
    2. If none were found, consider all routable placements anywhere in the device.
    3. If none were found, consider all placements anywhere in the device even if they're not routable.
  7. Pick one of the candidates at random and move the pivot to that location. If another cell in the netlist is already there, put it in the vacant site left by the pivot.
  8. Re-compute the score for the design. If it's better, accept this change and start the next iteration.
  9. If the score is worse, accept it with a random probability which decreases as the iteration number goes up. If the change is not accepted, restore the previous placement.
After optimization, the design is checked for routability. If any edges in the netlist graph don't correspond to edges in the device graph, the user probably asked for something impossible (for example, trying to hook a flipflop's output to a comparator's reference voltage input) so fail with an error.

The design is then routed. This is quite simple due to the crossbar structure of the device. For each edge in the netlist:
  1. If dedicated (non-fabric) routing is used for this path, configure the destination's input mux appropriately and stop.
  2. If the source and destination are in the same half of the device, configure the destination's input mux appropriately and stop.
  3. A cross-connection must be used. Check if we already used one to bring the source signal to the other half of the device. If found, configure the destination to route from that cross-connection and stop.
  4. Check if we have any cross-connections left going in this direction. If they're all used, the design is unroutable due to congestion so fail with an error.
  5. Pick the next unused cross-connection and configure it to route from the source. Configure the destination to route from the cross-connection and stop.
Once routing is finished, run a series of post-PAR design rule checks. These currently include the following:
  • If any node has no loads, generate a warning
  • If an I/O buffer is connected to analog hard IP, fail with an error if it's not configured in analog mode.
  • Some signals (such as comparator inputs and oscillator power-down controls) are generated by a shared mux and fed to many loads. If different loads require conflicting settings for the shared mux, fail with an error.
If DRC passes with no errors, configure all of the individual cells in the netlist based on the HDL parameters. Fail with an error if an invalid configuration was requested.

Finally, generate the bitstream from all of the per-cell configuration and write it to a file.

Great, let's get started!

If you don't already have one, you'll need to buy a GreenPak4 development kit. The kit includes samples of the SLG46620V (among other devices) and a programmer/emulation board. While you're waiting for it to arrive, install GreenPak Designer.

Download and install Yosys. Although Clifford is pretty good at merging my pull requests, only my fork on Github is guaranteed to have the most up-to-date support for GreenPak devices so don't be surprised if you can't use a bleeding-edge feature with mainline Yosys.

Download and install gp4par. You can get it from the Github repository.

Write your HDL, compile with Yosys, P&R with gp4par, and import the bitstream into GreenPak Designer to program the target device. The most current gp4par manual is included in LaTeX source form in the source tree and is automatically built as part of the compile process. If you're just browsing, there's a relatively recent PDF version on my web server.

If you'd like to see the Verilog that produced the nightmare of a schematic I showed above, here it is.

Be advised that this project is still very much a work in progress and there are still a number of SLG46620V features I don't support (see the manual for exact details).

I love it / it segfaulted / there's a problem in the manual!

Hop in our IRC channel (##openfpga on Freenode) and let me know. Feedback is great, pull requests are even better,

You're competing with Silego's IDE. Have they found out and sued you yet?

Nope. They're fully aware of what I'm doing and are rolling out the red carpet for me. They love the idea of a HDL flow as an alternative to schematic entry and are pretty amazed at how fast it's coming together.

After I reported a few bugs in their datasheets they decided to skip the middleman and give me direct access to the engineer who writes their documentation so that I can get faster responses. The last time I found a problem (two different parts of the datasheet contradicted each other) an updated datasheet was in my inbox and on their website by the next day. I only wish Xilinx gave me that kind of treatment!

They've even offered me free hardware to help me add support for their latest product family, although I plan to get GreenPak4 support to a more stable state before taking them up on the offer.

So what's next?


Better testing, for starters. I have to verify functionality by hand with a DMM and oscilloscope, which is time consuming.

My contact at Silego says they're going to be giving me documentation on the SRAM emulation interface soon, so I'm going to make a hardware-in-loop test platform that connects to my desktop and the Silego ZIF socket, and lets me load new bitstreams via a scriptable interface. It'll have FPGA-based digital I/O as well as an ADC and DAC on every device pin, plus an adjustable voltage regulator for power, so I can feed in arbitrary mixed-signal test waveforms and write PC-based unit tests to verify correct behavior.

Other than that, I want to finish support for the SLG46620V in the next month or two. The SLG46621V will be an easy addition since only one pin and the relevant configuration bits have changed from the 46620 (I suspect they're the same die, just bonded out differently).

Once that's done I'll have to do some more extensive work to add the SLG46140V since the architecture is a bit different (a lot of the combinatorial logic is merged into multi-function blocks). Luckily, the 46140 has a lot in common architecturally with the GreenPak5 family, so once that's done GreenPak5 will probably be a lot easier to add support for.

My thanks go out to Clifford Wolf, whitequark, the IRC users in ##openfpga, and everyone at Silego I've worked with to help make this possible. I hope that one day this project will become mature enough that Silego will ship it as an officially supported extension to GreenPak Designer, making history by becoming the first modern programmable logic vendor to ship a fully open source synthesis and P&R suite.

by Andrew Zonenberg (noreply@blogger.com) at May 08, 2016 09:21 AM

May 07, 2016

Altus Metrum

Altos1.6.3

AltOS 1.6.3 —

Bdale and I are pleased to announce the release of AltOS version 1.6.3.

AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, STMF042, LPC11U14 and ATtiny85 based electronics and Java-based ground station software.

Version 1.6.3 adds idle mode to AltosDroid and has bug fixes for our host software on desktops, laptops an android devices along with BlueTooth support for Windows.

1.6.3 is in Beta test for Android; if you want to use the beta version, join the AltosDroid beta program

AltOS

AltOS fixes:

  • Fix hardware flow control on TeleBT v3.0. RTS/CTS is wired backwards on this board, switch from using the hardware to driving these pins with software.

AltosUI and TeleGPS Applications

AltosUI and TeleGPS New Features:

  • Add BlueTooth support for Windows operating system. This supports connections to TeleBT over BlueTooth rather than just USB.

AltosUI and TeleGPS Fixes:

  • Change Java detection and install on Windows. Detection is now done by looking for the 'javaw.exe' program, and installation by opening a browser on the java.com web site.

  • Delay polling while the Fire Igniters is visible to allow for TeleMega to report back complete status over the radio.

  • Disallow changing RF calibration numbers in the configuration UI. There's no good reason to change this from the field, and recovering is really hard if you haven't written down the right number.

  • Fix USB device discovery on Mac OS X El Capitan. This makes the connected Altus Metrum USB devices appear again.

  • Fix acceleration data presented in MonitorIdle mode for TeleMetrum v2.0 flight computers.

AltosDroid

AltosDroid new features:

  • Monitor Idle mode. Check state of flight computer while in idle mode over the radio link

  • Fire Igniters. Remotely fire ignires for recovery system ground tests.

  • Remote reboot. Cause the flight computer to reboot over the radio link. This provides a method for switching the flight computer from idle to flight mode without needing to reach the power switch.

  • Configurable frequency menu. Change the set of available frequencies and provide more descriptive names.

AltosDroid bug fixes:

  • Don't set target location if GPS hasn't locked yet.

  • Fix saving target states so they can be reloaded when the application restarts. When the application is shut down and restarted, all previous target state information will be restored (including GPS position if available).

  • Fix crash on some Android devices for offline maps when changing the map scale or location.

  • Don't require USB OTG support. This kept the latest AltosDroid from being offered on devices without USB device support, although it can work without that just fine using BlueTooth.

  • Don't require bluetooth to be enabled. This allows the application to operate with USB devices or just show old data without turning on the bluetooth radio.

  • Recover old tracker positions when restarting application. This finally allows you to safely stop and restart the application without losing the last known location of any tracker.

Documentation

  • Document TeleMega and EasyMega additional pyro channel continuity audio alert pattern.

by keithp's rocket blog at May 07, 2016 01:18 AM

May 02, 2016

ZeptoBARS

Maxim ds2401z - serial number chip : weekend die-shot

Dallas Semiconductor/Maxim DS2401 is a factory pre-programmed silicon serial number chip.
Right at the center of the die you can see 64-bit laser-trimmed ROM. Die size 1346x686 µm.


May 02, 2016 11:29 PM

May 01, 2016

Harald Welte

Developers wanted for Osmocom GSM related work

Right now I'm feeling sad. I really shouldn't, but I still do.

Many years ago I started OpenBSC and Osmocom in order to bring Free Software into an area where it barely existed before: Cellular Infrastructure. For the first few years, it was "just for fun", without any professional users. A FOSS project by enthusiasts. Then we got some commercial / professional users, and with them funding, paying for e.g. Holger and my freelance work. Still, implementing all protocol stacks, interfaces and functional elements of GSM and GPRS from the radio network to the core network is something that large corporations typically spend hundreds of man-years on. So funding for Osmocom GSM implementations was always short, and we always tried to make the best out of it.

After Holger and I started sysmocom in 2011, we had a chance to use funds from BTS sales to hire more developers, and we were growing our team of developers. We finally could pay some developers other than ourselves from working on Free Software cellular network infrastructure.

In 2014 and 2015, sysmocom got side-tracked with some projects where Osmocom and the cellular network was only one small part of a much larger scope. In Q4/2015 and in 2016, we are back on track with focussing 100% at Osmocom projects, which you can probably see by a lot more associated commits to the respective project repositories.

By now, we are in the lucky situation that the work we've done in the Osmocom project on providing Free Software implementations of cellular technologies like GSM, GPRS, EDGE and now also UMTS is receiving a lot of attention. This attention translates into companies approaching us (particularly at sysmocom) regarding funding for implementing new features, fixing existing bugs and short-comings, etc. As part of that, we can even work on much needed infrastructural changes in the software.

So now we are in the opposite situation: There's a lot of interest in funding Osmocom work, but there are few people in the Osmocom community interested and/or capable to follow-up to that. Some of the early contributors have moved into other areas, and are now working on proprietary cellular stacks at large multi-national corporations. Some others think of GSM as a fun hobby and want to keep it that way.

At sysmocom, we are trying hard to do what we can to keep up with the demand. We've been looking to add people to our staff, but right now we are struggling only to compensate for the regular fluctuation of employees (i.e. keep the team size as is), let alone actually adding new members to our team to help to move free software cellular networks ahead.

I am struggling to understand why that is. I think Free Software in cellular communications is one of the most interesting and challenging frontiers for Free Software to work on. And there are many FOSS developers who love nothing more than to conquer new areas of technology.

At sysmocom, we can now offer what would have been my personal dream job for many years:

  • paid work on Free Software that is available to the general public, rather than something only of value to the employer
  • interesting technical challenges in an area of technology where you will not find the answer to all your problems on stackoverflow or the like
  • work in a small company consisting almost entirely only of die-hard engineers, without corporate managers, marketing departments, etc.
  • work in an environment free of Microsoft and Apple software or cloud services; use exclusively Free Software to get your work done

I would hope that more developers would appreciate such an environment. If you're interested in helping FOSS cellular networks ahead, feel free to have a look at http://sysmocom.de/jobs or contact us at jobs@sysmocom.de. Together, we can try to move Free Software for mobile communications to the next level!

by Harald Welte at May 01, 2016 10:00 PM

April 30, 2016

Bunnie Studios

Circuit Classics — Sneak Peek!

My first book on electronics was Getting Started with Electronics; to this day, I still imagine electrons as oval-shaped particles with happy faces because of its illustrations. So naturally, I was thrilled to find that the book’s author, Forrest Mims III, and my good friend Star Simpson joined forces to sell kit versions of classic circuits straight off the pages of Getting Started with Electronics. This re-interpretation of a classic as an interactive kit is perfect for today’s STEM curriculum, and I hope it will inspire another generation of engineers and hackers.

I’m very lucky that Star sent me a couple early prototypes to play with. Today was a rainy Saturday afternoon, so I loaded a few tracks from Information Society’s Greatest Hits album (I am most definitely a child of the 80’s) and fired up my soldering iron for a walk down memory lane. I remembered how my dad taught me to bend the leads of resistors with pliers, to get that nice square look. I remembered how I learned to use masking tape and bent leads to hold parts in place, so I could flip the board over for soldering. I remembered doodling circuits on scraps of paper after school while watching Scooby-Doo cartoons on a massive CRT TV that took several minutes to warm up. Things were so much simpler back then …

I couldn’t help but embellish a little bit. I added a socket for the chip on my Bargraph Voltage Indicator (when I see chips in sockets, I hear a little voice in my head whispering “hack me!” “fix me!” “reuse me!”), and swapped out the red LEDs for some high-efficiency white LEDs I happened to have on the shelf.

I appreciated Star’s use of elongated pads on the DIP components, a feature not necessary for automated assembly but of great assistance to hand soldering.

It works! Here I am testing the bargraph voltage indicator with a 3V coin cell on my (very messy) keyboard desk.

Voilà! My rendition of a circuit classic. I think the photo looks kind of neat in inverse color.

I really appreciate seeing a schematic printed on a circuit board next to its circuit. It reminds me that before Open Hardware, hardware was open. Schematics like these taught me that circuits were knowable; unlike the mysteries of quantum physics and molecular biology, virtually every circuit is a product of human imagination. That another engineer designed it, means any other engineer could understand it, given sufficient documentation. As a youth, I didn’t understand what these symbols and squiggles meant; but just knowing that a map existed set me on a path toward greater comprehension.

Whether a walk down nostalgia lane or just getting started in electronics, Circuit Classics are a perfect activity for both young and old. If you want to learn more, check out Star Simpson’s crowdfunding campaign on Crowd Supply!

by bunnie at April 30, 2016 04:19 PM

Hacking Humble Bundle

I’m very honored and proud to have one of my books offered as part of the Hacking Humble Bundle. Presented by No Starch Press, the Hacking Humble Bundle is offering several eBook titles for a “pay-what-you-feel” price, including my “Hacking the Xbox”, along with “Automate the Boring Stuff with Python”, “The Linux Command Line” and “The Smart Girl’s Guide to Privacy”. Of course, you can already download Hacking the Xbox for free, but if you opt to pay at least $15 you can get 9 more fantastic titles — check out all of them at the Humble Bundle page.

One of the best parts about a humble bundle is you have a say in where your money goes.

If you click on “Choose where your money goes” near checkout area, you’re presented with a set of sliders that let you pick how much money goes to charity, how much to the publisher, and how much as a tip to the Humble Bundle. For the Hacking Humble Bundle, the default charity is the EFF (you’re free to pick others if you want). For the record, I don’t get any proceeds from the Humble Bundle; I’m in it to support the EFF and No Starch.

If you enjoyed Hacking the Xbox, this is a perfect opportunity to give back to a charitable organization that was instrumental in making it happen. Without the EFF’s counsel, I wouldn’t have known my rights. Knowledge is power, and their support gave me the courage I needed to stand up and assert my right to hack, despite imposing adversaries. To this day, the EFF continues to fight for our rights on the digital frontier, and we need their help more than ever. No Starch has also been a stalwart supporter of hackers; their founder, Bill Pollock, and his “Damn the Torpedoes, Full Speed Ahead” attitude toward publishing potentially controversial topics has enabled hackers to educate the world about relevant but edgy technical topics.

If hacking interests you, it’s probably worth the time to check out the Hacking Humble Bundle and give a thought about what it’s worth to you. After all, you can “pay what you feel” and still get eBooks in return.

by bunnie at April 30, 2016 03:49 PM

April 26, 2016

Free Electrons

How we found that the Linux nios2 memset() implementation had a bug!

NIOS II processorNiosII is a 32-bit RISC embedded processor architecture designed by Altera, for its family of FPGAs: Cyclone III, Cyclone IV, etc. Being a soft-core architecture, by using Altera’s Quartus Prime design software, you can adjust the CPU configuration to your needs and instantiate it into the FPGA. You can customize various parameters like the instruction or the data cache size, enable/disable the MMU, enable/disable an FPU, and so on. And for us embedded Linux engineers, a very interesting aspect is that both the Linux kernel and the U-Boot bootloader, in their official versions, support the NIOS II architecture.

Recently, one of our customers designed a custom NIOS II platform, and we are working on porting the mainline U-Boot bootloader and the mainline Linux kernel to this platform. The U-Boot porting went fine, and quickly allowed us to load and start a Linux kernel. However, the Linux kernel was crashing very early with:

[    0.000000] Linux version 4.5.0-00007-g1717be9-dirty (rperier@archy) (gcc version 4.9.2 (Altera 15.1 Build 185) ) #74 PREEMPT Fri Apr 22 17:43:22 CEST 2016
[    0.000000] bootconsole [early0] enabled
[    0.000000] early_console initialized at 0xe3080000
[    0.000000] BUG: failure at mm/bootmem.c:307/__free()!
[    0.000000] Kernel panic - not syncing: BUG!

This BUG() comes from the __free() function in mm/bootmem.c. The bootmem allocator is a simple page-based allocator used very early in the Linux kernel initialization for the very first allocations, even before the regular buddy page allocator and other allocators such as kmalloc are available. We were slightly surprised to hit a BUG in a generic part of the kernel, and immediately suspected some platform-specific issue, like an invalid load address for our kernel, or invalid link address, or other ideas like this. But we quickly came to the conclusion that everything was looking good on that side, and so we went on to actually understand what this BUG was all about.

The NIOS II memory initialization code in arch/nios2/kernel/setup.c does the following:

bootmap_size = init_bootmem_node(NODE_DATA(0),
                                 min_low_pfn, PFN_DOWN(PHYS_OFFSET),
                                 max_low_pfn);
[...]
free_bootmem(memory_start, memory_end - memory_start);

The first call init_bootmem_node() initializes the bootmem allocator, which primarily consists in allocating a bitmap, with one bit per page. The entire bootmem bitmap is set to 0xff via a memset() during this initialization:

static unsigned long __init init_bootmem_core(bootmem_data_t *bdata,
        unsigned long mapstart, unsigned long start, unsigned long end)
{
        [...]
        mapsize = bootmap_bytes(end - start);
        memset(bdata->node_bootmem_map, 0xff, mapsize);
        [...]
}

After doing the bootmem initialization, the NIOS II architecture code calls free_bootmem() to mark all the memory pages as available, except the ones that contain the kernel itself. To achieve this, the __free() function (which is the one triggering the BUG) clears the bits corresponding to the page to be marked as free. When clearing those bits, the function checks that the bit was previously set, and if it’s not the case, fires the BUG:

static void __init __free(bootmem_data_t *bdata,
                        unsigned long sidx, unsigned long eidx)
{
        [...]
        for (idx = sidx; idx  eidx; idx++)
                if (!test_and_clear_bit(idx, bdata->node_bootmem_map))
                        BUG();
}

So to summarize, we were in a situation where a bitmap is memset to 0xff, but almost immediately afterwards, a function that clears some bits finds that some of the bits are already cleared. Sounds odd, doesn’t it?

We started by double checking that the address of the bitmap was the same between the initialization function and the __free function, verifying that the code was not overwriting the bitmap, and other obvious issues. But everything looked alright. So we simply dumped the bitmap after it was initialized by memset to 0xff, and to our great surprise, we found that the bitmap was in fact initialized with the pattern 0xff00ff00 and not 0xffffffff. This obviously explained why we were hitting this BUG(): simply because the buffer was not properly initialized. At first, we really couldn’t believe this: how it is possible that something as essential as memset() in Linux was not doing its job properly?

On the NIOS II platform, memset() has an architecture-specific implementation, available in arch/nios2/lib/memset.c. For buffers smaller than 8 bytes, this memset implementation uses a simple naive loop, iterating byte by byte. For larger buffers, it uses a more optimized implementation, using inline assembly. This implementation copies data per blocks of 4-bytes rather than 1 byte to speed-up the memset.

We quickly tested a workaround that consisted in using the naive implementation for all buffer sizes, and it solved the problem: we had a booting kernel, all the way to the point where it mounts a root filesystem! So clearly, it’s the optimized implementation in assembly that had a bug.

After some investigation, we found out that the bug was in the very first instructions of the assembly code. The following piece of assembly is supposed to create a 4-byte value that repeats 4 times the 1-byte pattern passed as an argument to memset:

/* fill8 %3, %5 (c & 0xff) */
"       slli    %4, %5, 8\n"
"       or      %4, %4, %5\n"
"       slli    %3, %4, 16\n"
"       or      %3, %3, %4\n"

This code takes as input in %5 the one-byte pattern, and is supposed to return in %3 the 4-byte pattern. It goes through the following logic:

  • Stores in %4 the initial pattern shifted left by 8 bits. Provided an initial pattern of 0xff, %4 should now contain 0xff00
  • Does a logical or between %4 and %5, which leads to %4 containing 0xffff
  • Stores in %3 the 2-byte pattern shifted left by 16 bits. %3 should now contain 0xffff0000.
  • Does a logical or between code>%3
and %4, i.e between 0xffff0000 and 0xffff, which gives the expected 4-byte pattern 0xffffffff

When you look at the source code, it looks perfectly fine, so our source code review didn’t spot the problem. However, when looking at the actual compiled code disassembled, we got:

34:	280a923a 	slli	r5,r5,8
38:	294ab03a 	or	r5,r5,r5
3c:	2808943a 	slli	r4,r5,16
40:	2148b03a 	or	r4,r4,r5

Here r5 gets used for both %4 and %5. Due to this, the final pattern stored in r4 is 0xff00ff00 instead of the expected 0xffffffff.

Now, if we take a look at the output operands, %4 is defined with the "=r" constraint, i.e an output operand. How to prevent the compiler from re-using the corresponding register for another operand? As explained in this document, "=r" does not prevent gcc from using the same register for an output operand (%4) and input operand (%5). By adding the constrainst & (in addition to "=r"), we tell the compiler that the register associated with the given operand is an output-only register, and so, cannot be used with an input operand.

With this change, we get the following assembly output:

34:	2810923a 	slli	r8,r5,8
38:	4150b03a 	or	r8,r8,r5
3c:	400e943a 	slli	r7,r8,16
40:	3a0eb03a 	or	r7,r7,r8

Which is much better, and correctly produces the 0xffffffff pattern when 0xff is provided as the initial 1-byte pattern to memset.

In the end, the final patch only adds one character to adjust the inline assembly constraint and gets the proper behavior from gcc:

diff --git a/arch/nios2/lib/memset.c b/arch/nios2/lib/memset.c
index c2cfcb1..2fcefe7 100644
--- a/arch/nios2/lib/memset.c
+++ b/arch/nios2/lib/memset.c
@@ -68,7 +68,7 @@ void *memset(void *s, int c, size_t count)
 		  "=r" (charcnt),	/* %1  Output */
 		  "=r" (dwordcnt),	/* %2  Output */
 		  "=r" (fill8reg),	/* %3  Output */
-		  "=r" (wrkrega)	/* %4  Output */
+		  "=&r" (wrkrega)	/* %4  Output only */
 		: "r" (c),		/* %5  Input */
 		  "0" (s),		/* %0  Input/Output */
 		  "1" (count)		/* %1  Input/Output */

This patch was sent upstream to the NIOS II kernel maintainers:
[PATCH v2] nios2: memset: use the right constraint modifier for the %4 output operand, and has already been applied by the NIOS II maintainer.

We were quite surprised to find a bug in some common code for the NIOS II architecture: we were assuming it would have already been tested on enough platforms and with enough compilers/situations to not have such issues. But all in all, it was a fun debugging experience!

It is worth mentioning that in addition to this bug, we found another bug affecting NIOS II platforms, in the asm-generic implementation of the futex_atomic_cmpxchg_inatomic() function, which was causing some preemption imbalance warnings during the futex subsystem initialization. We also sent a patch for this problem, which has also been applied already.

by Romain Perier at April 26, 2016 03:03 PM

April 22, 2016

Elphel

Tutorial 01: Access to Elphel camera documentation from 3D model

We have created a short video tutorial to help our users navigate through 3D models of Elphel cameras. Cameras can be virtually taken apart and put back together which helps to understand the camera configuration and access information about every camera component. Please feel free to comment on the video quality and usefulness, as we are launching a series of tutorials about cameras, software modifications, FPGA development on 10393 camera board, etc. and we would like to receive feedback on them.



Description:

In this video we will show how the 3D model of Elphel NC393 camera can be used to view the camera, understand the components it is made of, take it apart and put back together, and get access to each part’s documentation.

The camera model is made using X3Dom technology autogenerated from STEP files used for production.

In your browser you can open the link to one of the camera assemblies from Elphel wiki page:

The buttons on the right list all camera components.

You can click on one of the buttons and the component will be selected on the model. Click again and the part will be selected without the rest of the model.
From here, using the buttons at the bottom of the screen you can open the part in a new window.
Or look for the part on Elphel wiki;
Or hide the part and see the rest of the model;
Eventually you can return to the whole model by clicking on the part button once more, or there is always a reset model button, at the top left corner.

You can also select part by clicking on the part on the model.

To deselect it click again;

Right click removes the part, so you can get access to the insides of the camera.

Once you have selected the part you can look for more information about it on Elphel wiki.

For the selected board you can type the board name in the wiki search and get access to the description about the board, circuit diagram, parts list and PCB layout.

All Elphel software is Free Software and distributed under GNU/GPL license as well as Elphel camera designs are open hardware, distributed under CERN open Hardware license.

by olga at April 22, 2016 02:08 AM

April 21, 2016

Free Electrons

Article on the CHIP in French Linux magazine

Free Electrons engineer and Allwinner platform maintainer Maxime Ripard has written a long article presenting the Nextthing C.H.I.P platform in issue #18 of French magazine OpenSilicium, dedicated to open source in embedded systems. The C.H.I.P has even been used for the front cover of the magazine!

OpenSilicium #18

In this article, Maxime presents the C.H.I.P platform, its history and the choice of the Allwinner SoC. He then details how to set up a developer-friendly environment to use the board, building and flashing from scratch U-Boot, the kernel and a Debian-based root filesystem. Finally, he describes how to use Device Tree overlays to describe additional peripherals connected to the board, with the traditional example of the LED.

OpenSilicium #18 CHIP article

In the same issue, OpenSilicium also covers numerous other topics:

  • A feedback on the FOSDEM 2016 conference
  • Uploading code to STM32 microcontrollers: the case of STM32-F401RE
  • Kernel and userspace debugging with ftrace
  • IoT prototyping with Buildroot
  • RIOT, the free operating system for the IoT world
  • Interview of Cedric Bail, working on the Enligthenment Foundation Libraries for Samsung
  • Setup of Xenomai on the Zynq Zedboard
  • Decompression of 3R data stream using a VHDL-described circuit
  • Write a userspace device driver for a FPGA using UIO

by Thomas Petazzoni at April 21, 2016 08:56 PM

April 20, 2016

Free Electrons

Slides from the Embedded Linux Conference

Two weeks ago, the entire Free Electrons engineering team (9 persons) attended the Embedded Linux Conference in San Diego. We had some really good time there, with lots of interesting talks and useful meetings and discussions.

Tim Bird opening the conferenceDiscussion between Linus Torvalds and Dirk Hohndel

In addition to attending the event, we also participated by giving 5 different talks on various topics, for which we are publishing the slides:

Boris Brezillon, the new NAND Linux subsystem maintainer, presented on Modernizing the NAND framework: The big picture.

Boris Brezillon's talk on the NAND subsystem

Antoine Ténart presented on Using DT overlays to support the C.H.I.P’s capes.

Antoine Tenart's talk on using DT overlays for the CHIP

Maxime Ripard, maintainer of the Allwinner platform support in Linux, presented on Bringing display and 3D to the C.H.I.P computer.

Maxime Ripard's talk on display and 3D for the CHIP

Alexandre Belloni and Thomas Petazzoni presented Buildroot vs. OpenEmbedded/Yocto Project: a four hands discussion.

Belloni and Petazzoni's talk on OpenEmbedded vs. Buildroot

Thomas Petazzoni presented GNU Autotools: a tutorial.

Petazzoni's tutorial on the autotools

All the other slides from the conference are available from the event page as well as from eLinux.org Wiki. All conferences have been recorded, and the videos will hopefully be posted soon by the Linux Foundation.

by Thomas Petazzoni at April 20, 2016 09:17 AM

April 19, 2016

Free Electrons

Free Electrons engineer Boris Brezillon becomes Linux NAND subsystem maintainer

Free Electrons engineer Boris Brezillon has been involved in the support for NAND flashes in the Linux kernel for quite some time. He is the author of the NAND driver for the Allwinner ARM processors, did several improvements to the NAND GPMI controller driver, has initiated a significant rework of the NAND subsystem, and is working on supporting MLC NANDs. Boris is also very active on the linux-mtd mailing list by reviewing patches from others, and making suggestions.

Hynix NAND flash

For those reasons, Boris was recently appointed by the MTD maintainer Brian Norris as a new maintainer of the NAND subsystem. NAND is considered a sub-subsystem of the MTD subsystem, and as such, Boris will be sending pull requests to Brian, who in turn is sending pull requests to Linus Torvalds. See this commit for the addition of Boris as a NAND maintainer in the MAINTAINERS file. Boris will therefore be in charge of reviewing and merging all the patches touching drivers/mtd/nand/, which consist mainly of NAND drivers. Boris has created a nand/next on Github, where he has already merged a number of patches that will be pushed to Brian Norris during the 4.7 merge window.

We are happy to see one of our engineers taking another position as a maintainer in the kernel community. Maxime Ripard was already a co-maintainer of the Allwinner ARM platform support, Alexandre Belloni a co-maintainer of the RTC subsystem and Atmel ARM platform support, Grégory Clement a co-maintainer of the Marvell EBU platform support, and Antoine Ténart a co-maintainer of the Annapurna Labs platform support.

by Thomas Petazzoni at April 19, 2016 07:59 AM

April 16, 2016

ZeptoBARS

NXP/Philips BC857BS - dual pnp BJT : weekend die-shot

SOT-363 package contains 2 separate identical transistor dies.
Size of each die is 285x259 µm.


April 16, 2016 02:35 AM

April 12, 2016

Free Electrons

Slides from Collaboration Summit talk on Linux kernel upstreaming

As we announced in a previous blog post, Free Electrons CTO Thomas Petazzoni gave a talk at the Collaboration Summit 2016 covering the topic of “Upstreaming hardware support in the Linux kernel: why and how?“.

The slides of the talk are now available in PDF format.

Upstreaming hardware support in the Linux kernel: why and how?

Upstreaming hardware support in the Linux kernel: why and how?

Upstreaming hardware support in the Linux kernel: why and how?

Through this talk, we identified a number of major reasons that should encourage hardware vendors to contribute the support for their hardware to the upstream Linux kernel, and some hints on how to achieve that. Of course, within a 25 minutes time slot, it was not possible to get into the details, but hopefully the general hints we have shared, based on our significant Linux kernel upstreaming experience, have been useful for the audience.

Unfortunately, none of the talks at the Collaboration Summit were recorded, so no video will be available for this talk.

by Thomas Petazzoni at April 12, 2016 11:35 AM

April 09, 2016

Bunnie Studios

Name that Ware, April 2016

The Ware for April 2016 is shown below.

The ware this month is courtesy Philipp Gühring. I think it should be a bit more challenging that the past couple months’ wares. If readers are struggling to guess this one by the end of this month, I’ve got a couple other photos Philipp sent which should give additional clues.

But, interested to see what people think this is, with just this photo!

by bunnie at April 09, 2016 11:22 AM

April 03, 2016

ZeptoBARS

ST TS971 : weekend die-shot

ST TS321 is a single 12 MHz R2R opamp in SOT23-5 package with low noise and low distortion.
Die size 1079x799 µm.


April 03, 2016 08:40 AM

March 30, 2016

Elphel

Synchronizing Verilog, Python and C

Elphel NC393 as all the previous camera models relies on the intimate cooperation of the FPGA programmed in Verilog HDL and the software that runs on a general purpose CPU. Just as the FPGA manufacturers increase the speed and density of their devices, so do the Elphel cameras. FPGA code consists of the hundreds of files, tens of thousand lines of code and is constantly modified during the lifetime of the product both by us and by our users to accommodate the cameras for their applications. In most cases, if it is not just a bug fix or minor improvement of the previously implemented functionality, the software (and multiple layers of it) needs to be aware of the changes. This is both the power and the challenge of such hybrid systems, and the synchronization of the changes is an important issue.

Verilog parameters

Verilog code of the camera consists of the parameterized modules, we try to use parameters and

generate
Verilog operators in most cases, but
`define
macros and
`ifdef
conditional directives are still used to switch some global options (like synthesis vs. compilation, various debug levels). Eclipse-based VDT that we use for the FPGA development is aware of the parameters, and when the code instantiates a parametrized module that has parameter-dependent widths of the ports, VDT verifies that the instance ports match the signals connected to them, and warns the developer if it is not the case. Many parameters are routed through the levels of the hierarchy so the deeper instances can be controlled from a single header file, making it obvious which parameters influence which modules operations. Some parameters are specified directly, while some have to be calculated – it is the case for the register address decoders of the same module instances for different channels. Such channels have the same relative address maps, but different base addresses. Most of the camera parameters (not counting the trivial ones where the module instance parameters are defined by the nature of the code) are contained in a single x393_parameters.vh header file. There are more than six hundred of them there and most influence the software API.

Development cycle

When implementing some new camera FPGA functionality, we start with the simulation – always. Sometimes very small changes can be applied to the code, synthesized and tested in the actual hardware, but it almost never works this way – bypassing the simulation step. So far all the simulation we use consit of the plain old Verilog test benches (such as this or that) – not even System Verilog. Most likely for simulating CPU+FPGA devices ideal would be the use the software programming language to model the CPU side of the SoC and keep Verilog (or VHDL who prefers it) to the FPGA. Something like cocotb may work, especially we are already manually translating Verilog into Python, but we are not there yet.

Translaing Verilog to Python

So the next step is as I just mentioned – manual translation of the Verilog tasks and functions used in simulation to Python that code that can run on the actual hardware. The result does not look extremely pythonian as I try to follow already tested Verilog code, but it is OK. Not all the translation is manual – we use a import_verilog_parameters.py module to “understand” the parameters defined in Verilog files (including simple arithmetic and logical operations used to generate derivative parameters/localparams in the Verilog code), get the values from the same source and so reduce the possibility to accidentally use old software with the modified FPGA implementation. As the parameters are known to the program at a run time and PyDev (running, btw, in the same Eclipse IDE as the VDT – just as a different “perspective”) can not catch the misspelled parameter names. So the program has an option to modify itself and generate pre-defines for each of the parameter. Only the top part of the vrlg module is human-generated, everything under line 120 is automatically generated (and has to be re-generated only after adding new parameters to the Verilog source).

Hardware testing with Python programs

When the Verilog code is manually translated (or while new parts of the code are being translated or developed from scratch) it is possible to operate the actual camera. The top module is still called test_mcntrl as it started with DDR3 memory calibration using Levenberg-Marquardt algorithm (luckily it needs to run just once – it takes camera 10 minutes to do the full calibration this way).

This program keeps track of the Verilog parameters and macros, exposes all the functions (with the names not beginning with the underscore character), extracts docstrings from the code and combines it with the generated list of the function parameters and their default values, provides search/help for the functions with regexp (a must when there are hundreds of such functions). Next code ran in the camera:

x393 +0.043s--> help w.*_sensor_r
=== write_sensor_reg16 ===
defined in x393_sensor.X393Sensor, /usr/local/bin/x393_sensor.py: 496)
Write i2c register in immediate mode
@param num_sensor - sensor port number (0..3), or "all" - same to all sensors
@param reg_addr16 - 16-bit register address (page+low byte, for MT9P006 high byte is an 8-bit slave address = 0x90)
@param reg_data16 - 16-bit data to write to sensor register
     Usage: write_sensor_reg16 <num_sensor> <reg_addr16> <reg_data16>
x393 +0.010s-->

And the same one in PyDev console window of Eclipse IDE – “simulated” means that the program could not detect the FPGA and so it is not the target hardware:

x393(simulated) +0.121s--> help w.*_sensor_r
=== write_sensor_reg16 ===
defined in x393_sensor.X393Sensor, /home/andrey/git/x393/py393/x393_sensor.py: 496)
Write i2c register in immediate mode
@param num_sensor - sensor port number (0..3), or "all" - same to all sensors
@param reg_addr16 - 16-bit register address (page+low byte, for MT9P006 high byte is an 8-bit slave address = 0x90)
@param reg_data16 - 16-bit data to write to sensor register
     Usage: write_sensor_reg16 <num_sensor> <reg_addr16> <reg_data16>
x393(simulated) +0.001s-->

Python program was also used for the AHCI SATA controller initial development (before adding it was possible to add is as Linux kernel platform driver, but number of parameters there is much smaller, and most of the addresses are defined by the AHCI standard.

Synchronizing parameters with the kernel drivers

Next step is to update/redesign/develop the Linux kernel drivers to support camera functionality. Learning the lessons from the previous camera models (software was growing with the hardware incrementally) we are trying to minimize manual intervention into the process of synchronizing different layers of code (including the “hardware” one). Previous camera interface to the FPGA consisted of the hand-crafted files such as x353.h. It started from the x313.h (for NC313 – our first camera based on Axis CPU and Xilinx FPGA – same was used in NC323 that scanned many billions of book pages), was modified for the NC333 and later for our previous NC353 used in car-mounted panoramic cameras that captured most of the world’s roads.

Each time the files were modified to accommodate the new hardware, it was always a challenge to add extra bits to the memory controller addresses, image frame widths and heights (they are now all 16-bit wide – enough for the multi-gigapixel sensors). With Python modules already knowing all the current values of the Verilog parameters that define software interface it was natural to generate the C files needed to interface the hardware in the same environment.

Implementation of the register access in the FPGA

The memory-mapped registers in the camera share the same access mechanism – they use MAXIGP0 (CPU master, general purpose, channel 0) AXI port available in SoC, generously mapped there to 1/4 of the whole 32-bit address range (0x40000000.0x7fffffff). While logically all the locations are 32-bit wide, some use just 1 byte or even no data at all – any write to such address causes defined action.

Internally the commands are distributed to the target modules over a tree of byte-parallel buses that tolerate register insertion, at the endpoints they are converted to the parallel format by cmd_deser.v instances. The status data from the modules (sent by status_generate.v) is routed as messages (also in byte-parallel format to reduce the required FPGA routing resources) to a single block memory that can be read over the AXI by the CPU with zero delay. The status generation by the subsystems is individually programmed to be either on demand (in response to the write operation by the CPU) or automatically when the register data changes. While this write and read mechanism is common, the nature of the registers and data may be very different as the project combines many modules designed at different time for different purposes. All the memory mapped locations in the design fall into 3 categories:

  • Read only registers that allow to read status from the various modules, DMA pointers and other small data items.
  • Read/write registers – the ones where result of writing does not depend on any context. The full write register address range has a shadow memory block in parallel, so reading from that address will return the data that was last written there.
  • Write-only registers – all other registers where write action depends on the context. Some modules include large tables exposed through a pair of address/data locations in the address map, many other have independent bit fields with the corresponding “set” bit, so internal values are modified for only the selected field.

Register access as C11 anonymous members

All the registers in the design are 32-bit wide and are aligned to 4-byte ranges, even as not all of them use all the bits. Another common feature of the used register model is that some modules exist in multiple instances, each having evenly spaced base addresses, some of them have 2-level hierarchy (channel and sub-channel), where the address is a sum of the category base address, relative register address and a linear combination of the two indices.

Individual C

typedef
is generated for each set of registers that have different meanings of the bit fields – this way it is possible to benefit from the compiler type checking. All the types used fit into the 32 bits, and as in many cases the same hardware register can accept alternative values for individual bit fields, we use unions of anonymous (to make access expressions shorter) bit-field structures.

Here is a generated example of such typedef code (full source):

// I2C contol/table data

typedef union {
    struct {
          u32        tbl_addr: 8; // [ 7: 0] (0) Address/length in 64-bit words (<<3 to get byte address)
          u32                :20;
          u32        tbl_mode: 2; // [29:28] (3) Should be 3 to select table address write mode
          u32                : 2;
    }; 
    struct {
          u32             rah: 8; // [ 7: 0] (0) High byte of the i2c register address
          u32             rnw: 1; // [    8] (0) Read/not write i2c register, should be 0 here
          u32              sa: 7; // [15: 9] (0) Slave address in write mode
          u32            nbwr: 4; // [19:16] (0) Number of bytes to write (1..10)
          u32             dly: 8; // [27:20] (0) Bit delay - number of mclk periods in 1/4 of the SCL period
          u32    /*tbl_mode*/: 2; // [29:28] (2) Should be 2 to select table data write mode
          u32                : 2;
    }; 
    struct {
          u32         /*rah*/: 8; // [ 7: 0] (0) High byte of the i2c register address
          u32         /*rnw*/: 1; // [    8] (0) Read/not write i2c register, should be 1 here
          u32                : 7;
          u32            nbrd: 3; // [18:16] (0) Number of bytes to read (1..18, 0 means '8')
          u32           nabrd: 1; // [   19] (0) Number of address bytes for read (0 - one byte, 1 - two bytes)
          u32         /*dly*/: 8; // [27:20] (0) Bit delay - number of mclk periods in 1/4 of the SCL period
          u32    /*tbl_mode*/: 2; // [29:28] (2) Should be 2 to select table data write mode
          u32                : 2;
    }; 
    struct {
          u32  sda_drive_high: 1; // [    0] (0) Actively drive SDA high during second half of SCL==1 (valid with drive_ctl)
          u32     sda_release: 1; // [    1] (0) Release SDA early if next bit ==1 (valid with drive_ctl)
          u32       drive_ctl: 1; // [    2] (0) 0 - nop, 1 - set sda_release and sda_drive_high
          u32    next_fifo_rd: 1; // [    3] (0) Advance I2C read FIFO pointer
          u32                : 8;
          u32         cmd_run: 2; // [13:12] (0) Sequencer run/stop control: 0,1 - nop, 2 - stop, 3 - run 
          u32           reset: 1; // [   14] (0) Sequencer reset all FIFO (takes 16 clock pulses), also - stops i2c until run command
          u32                :13;
          u32    /*tbl_mode*/: 2; // [29:28] (0) Should be 0 to select controls
          u32                : 2;
    }; 
    struct {
          u32             d32:32; // [31: 0] (0) cast to u32
    }; 
} x393_i2c_ctltbl_t;

Some member names in the example above are commented out (like /*tbl_mode*/ in lines 398, 408 and 420). This is done so because some bit fields (in this case bits [29:28]) have the same meaning in all alternative structures, and auto-generating complex union/structure combinations to create a valid C code with each member having unique name would produce rather clumsy code. Instead this script makes sure that same named members really designate the same bit fields, and then makes them anonymous while preserving names for a human reader. The last member (u32 d32:32;) is added to each union making it possible to address each of them as an unsigned long variable without casting.

And this is a snippet of the part of the generator code that produced it:

def _enc_i2c_tbl_wmode(self):
    dw=[]
    dw.append(("rah",      vrlg.SENSI2C_TBL_RAH,    vrlg.SENSI2C_TBL_RAH_BITS, 0, "High byte of the i2c register address"))
    dw.append(("rnw",      vrlg.SENSI2C_TBL_RNWREG,                         1, 0, "Read/not write i2c register, should be 0 here"))
    dw.append(("sa",       vrlg.SENSI2C_TBL_SA,     vrlg.SENSI2C_TBL_SA_BITS,  0, "Slave address in write mode"))
    dw.append(("nbwr",     vrlg.SENSI2C_TBL_NBWR,   vrlg.SENSI2C_TBL_NBWR_BITS,0, "Number of bytes to write (1..10)"))
    dw.append(("dly",      vrlg.SENSI2C_TBL_DLY,    vrlg.SENSI2C_TBL_DLY_BITS, 0, "Bit delay - number of mclk periods in 1/4 of the SCL period"))
    dw.append(("tbl_mode", vrlg.SENSI2C_CMD_TAND,                           2, 2, "Should be 2 to select table data write mode"))
    return dw

The vrlg.* values used above are in turn read from the x393_parameters.vh Verilog file:

//i2c page table bit fields
    parameter SENSI2C_TBL_RAH =        0, // high byte of the register address
    parameter SENSI2C_TBL_RAH_BITS =   8,
    parameter SENSI2C_TBL_RNWREG =     8, // read register (when 0 - write register
    parameter SENSI2C_TBL_SA =         9, // Slave address in write mode
    parameter SENSI2C_TBL_SA_BITS =    7,
    parameter SENSI2C_TBL_NBWR =      16, // number of bytes to write (1..10)
    parameter SENSI2C_TBL_NBWR_BITS =  4,
    parameter SENSI2C_TBL_NBRD =      16, // number of bytes to read (1 - 8) "0" means "8"
    parameter SENSI2C_TBL_NBRD_BITS =  3,
    parameter SENSI2C_TBL_NABRD =     19, // number of address bytes for read (0 - 1 byte, 1 - 2 bytes)
    parameter SENSI2C_TBL_DLY =       20, // bit delay (number of mclk periods in 1/4 of SCL period)
    parameter SENSI2C_TBL_DLY_BITS=    8,

Auto-generated files also include x393.h, it provides other constant definitions (like valid values for the bit fields) – lines 301..303, and function declarations to access registers. Names of the functions for read-only and write-only are derived from the address symbolic names by converting them to the lower case, the ones which deal with read/write registers have set_ and get_ prefixes attached.

#define X393_CMPRS_CBIT_CMODE_JPEG18           0x00000000 // Color 4:2:0
#define X393_CMPRS_CBIT_FRAMES_SINGLE          0x00000000 // Use single-frame buffer
#define X393_CMPRS_CBIT_FRAMES_MULTI           0x00000001 // Use multi-frame buffer

// Compressor control

void               x393_cmprs_control_reg (x393_cmprs_mode_t d, int cmprs_chn);  // Program compressor channel operation mode
void               set_x393_cmprs_status  (x393_status_ctrl_t d, int cmprs_chn); // Setup compressor status report mode
x393_status_ctrl_t get_x393_cmprs_status  (int cmprs_chn);

Register access functions are implemented with readl() and writel(), this is a corresponding section of the x393.c file:

// Compressor control

void               x393_cmprs_control_reg (x393_cmprs_mode_t d, int cmprs_chn)  {writel(d.d32, mmio_ptr + (0x1800 + 0x40 * cmprs_chn));} // Program compressor channel operation mode
void               set_x393_cmprs_status  (x393_status_ctrl_t d, int cmprs_chn) {writel(d.d32, mmio_ptr + (0x1804 + 0x40 * cmprs_chn));} // Setup compressor status report mode
x393_status_ctrl_t get_x393_cmprs_status  (int cmprs_chn)                       { x393_status_ctrl_t d; d.d32 = readl(mmio_ptr + (0x1804 + 0x40 * cmprs_chn)); return d; }

There are two other header files generated from the same data, one (x393_defs.h) is just an alternative way to represent register addresses – instead of the getter and setter functions it defines the preprocessor macros:

// Compressor control

#define X393_CMPRS_CONTROL_REG(cmprs_chn) (0x40001800 + 0x40 * (cmprs_chn)) // Program compressor channel operation mode, cmprs_chn = 0..3, data type: x393_cmprs_mode_t (wo)
#define X393_CMPRS_STATUS(cmprs_chn)      (0x40001804 + 0x40 * (cmprs_chn)) // Setup compressor status report mode, cmprs_chn = 0..3, data type: x393_status_ctrl_t (rw)

The last generated file – x393_map.h uses the preprocessor macro format to provide a full ordered address map of all the available registers for all channels and sub-channels. It is intended to be used just as a reference for developers, not as an actual include file.

Conclusions

The generated code for Elphel NC393 camera is definitely very hardware-specific, its main purpose is to encapsulate as much as possible of the hardware interface details and so to reduce dependence of the higher layers of software on the modifications of the HDL code. Such tasks are common to other projects that involve CPU/FPGA tandems, and similar approach to organizing software/hardware interface may be useful there too.

by Andrey Filippov at March 30, 2016 08:04 PM

March 27, 2016

Harald Welte

You can now install a GSM network using apt-get

This is great news: You can now install a GSM network using apt-get!

Thanks to the efforts of Debian developer Ruben Undheim, there's now an OpenBSC (with all its flavors like OsmoBSC, OsmoNITB, OsmoSGSN, ...) package in the official Debian repository.

Here is the link to the e-mail indicating acceptance into Debian: https://tracker.debian.org/news/755641

I think for the past many years into the OpenBSC (and wider Osmocom) projects I always assumed that distribution packaging is not really something all that important, as all the people using OpenBSC surely would be technical enough to build it from the source. And in fact, I believe that building from source brings you one step closer to actually modifying the code, and thus contribution.

Nevertheless, the project has matured to a point where it is not used only by developers anymore, and particularly also (god beware) by people with limited experience with Linux in general. That such people still exist is surprisingly hard to realize for somebody like myself who has spent more than 20 years in Linux land by now.

So all in all, today I think that having packages in a Distribution like Debian actually is important for the further adoption of the project - pretty much like I believe that more and better public documentation is.

Looking forward to seeing the first bug reports reported through bugs.debian.org rather than https://projects.osmocom.org/ . Once that happens, we know that people are actually using the official Debian packages.

As an unrelated side note, the Osmocom project now also has nightly builds available for Debian 7.0, Debian 8.0 and Ubunut 14.04 on both i586 and x86_64 architecture from https://build.opensuse.org/project/show/network:osmocom:nightly. The nightly builds are for people who want to stay on the bleeding edge of the code, but who don't want to go through building everything from scratch. See Holgers post on the openbsc mailing list for more information.

by Harald Welte at March 27, 2016 10:00 PM

March 26, 2016

ZeptoBARS

ST TS321 - generic SOT23 opamp : weekend die-shot

ST TS321 is a single opamp in SOT23-5 package designed to match and exceed industry standard LM358A and LM324 opamps.
Die size 1270x735 µm.


March 26, 2016 08:18 AM

March 24, 2016

Video Circuits

Seeing Sound

I will be giving a talk on some research I have been doing into early British video synthesis and  electronic video work at this years Seeing Sound. I will also be screening some work from some contemporary Video Circuits Regulars as part of the conference.
www.seeingsound.co.uk Sign up here!


by Chris (noreply@blogger.com) at March 24, 2016 08:59 AM

March 22, 2016

Bunnie Studios

Formlabs Form 2 Teardown

I don’t do many teardowns on this blog, as several other websites already do an excellent job of that, but when I was given the chance to take apart a Formlabs Form 2, I was more than happy to oblige. About three yeargalvos ago, I had posted a teardown of a Form 1, which I received as a Kickstarter backer reward. Today, I’m looking at a Form 2 engineering prototype. Now that the Form 2 is in full production, the prototypes are basically spare parts, so I’m going to unleash my inner child and tear this thing apart with no concern about putting it back together again.

For regular readers of this blog, this teardown takes the place of March 2016’s Name that Ware — this time, I’m the one playing Name that Ware and y’all get to follow along as I adventure through the printer. Next month I’ll resume regular Name that Ware content.

First Impressions

I gave the Form 2 a whirl before tearing it into an irreparable pile of spare parts. In short, I’m impressed; the Form 2 is a major upgrade from the Form 1. It’s an interesting contrast to Makerbot. The guts of the Makerbot Replicator 2 are basically the same architecture as previous models, inheriting all the limitations of its previous incarnation.

The Form 2 is a quantum leap forward. The product smells of experienced, seasoned engineers; a throwback to the golden days of Massachusetts Route 128 when DEC, Sun, Polaroid and Wang Laboratories cranked out quality American-designed gear. Formlabs wasn’t afraid to completely rethink, re-architect, and re-engineer the system to build a better product, making bold improvements to core technology. As a result, the most significant commonality between the Form 1 and the Form 2 is the iconic industrial design: an orange acrylic box sitting atop an aluminum base with rounded corners and a fancy edge-lit power button.

Before we slip off the cover, here’s a brief summary of the upgrades that I picked up on while doing the teardown:

  • The CPU is upgraded from a single 72MHz ST Micro STM32F103 Cortex-M3 to a 600 MHz TI Sitara AM3354 Cortex A8, with two co-processors: a STM32F030 as a signal interface processor, and a STM32F373 as a real-time DSP on the galvo driver board.
  • This massive upgrade in CPU power leapfrogs the UI from a single push button plus monochrome OLED on the Form 1, to a full-color 4.3” capacitive touch screen on the Form 2.
  • The upgraded CPU also enables the printer to have built-in wifi & ethernet, in addition to USB. Formlabs thoughtfully combines this new TCP/IP capability with a Bonjour client. Now, computers can automatically discover and enumerate Form 2’s on the local network, making setup a snap.
  • The UI also makes better use of the 4 GB of on-board FLASH by adding the ability to “replay” jobs that were previously uploaded, making the printer more suitable for low volume production.
  • The galvanometers are full custom, soup-to-nuts. We’ll dig into this more later, but presumably this means better accuracy, better print jobs, and a proprietary advantage that makes it much harder for cloners to copy the Form 2.
  • The optics pathway is fully shrouded, eliminating dust buildup problems. A beautiful and much easier to clean AR-coated glass surface protects the internal optics; internal shrouds also limit the opportunity for dust to settle on critical surfaces.
  • The resin tray now features a heater with closed-loop control, for more consistent printing performance in cold New England garages in the dead of winter.
  • The resin tray is now auto-filling from an easy to install cartridge, enabling print jobs that require more resin than could fit in a single tank while making resin top-ups convenient and spill-free.
  • The peel motion is now principally lateral, instead of vertical.
  • The resin tank now features a stirrer. On the Form 1, light scattering would create thickened pools of partially cured resin near the active print region. Presumably the stirrer helps homogenize the resin; I also remember someone once mentioning the importance of oxygen to the surface chemistry of the resin tank.
  • There are novel internal photosensor elements that hint at some sort of calibration/skew correction mechanism
  • There’s a tilt sensor and manual mechanical leveling mechanism. A level tank prevents the resin from pooling to one side.
  • There are sensors that can detect the presence of the resin tank and the level of the resin. With all these new sensors, the only way a user can bork a print is to forget to install the build platform
  • Speaking of tank detection, the printer now remembers what color resin was used on a given tank, so you don’t accidentally spoil a clear resin tank with black resin
  • The power supply is now fully embedded; goodbye PSU failures and weird ground loop issues. It’s a subtle detail, but it’s the sort of “grown-up” thing that younger companies avoid doing because it complicates safety certification and requires compliance to elevated internal wiring and plastic flame retardance standards.
  • I’m also guessing there are a number of upgrades that are less obvious from a visual inspection, such as improvements to the laser itself, or optimizations to the printing algorithm.

    These improvements indicate a significant manpower investment on the part of Formlabs, and an incredible value add to the core product, as many of the items I note above would take several man-months to bring to production-ready status.

    Test Print

    As hinted from the upgrade list, the UI has been massively improved. The touchscreen-based UI features tech-noir themed iconography and animations that would find itself at home in a movie set. This refreshing attention to detail sets the Form 2’s UI apart from the utilitarian “designed-by-programmers-for-geeks” UI typical of most digital fabrication tools.


    A UI that would seem at home on a Hollywood set. Life imitating art imitating life.

    Unfortunately, the test print didn’t go smoothly. Apparently the engineering prototype had a small design problem which caused the resin tray’s identification contacts to intermittently short against the metal case during a peel operation. This would cause the bus shared between the ID chips on the resin tank and the filler cartridge to fail. As a result, the printer paused twice on account of a bogus “missing resin cartridge” error. Thankfully, the problem would eventually fix itself, and the print would automatically resume.


    Test print from the Form 2. The red arrow indicates the location of a hairline artifact from the print pausing for a half hour due to issues with resin cartridge presence detection.

    The test print came out quite nicely, despite the long pauses in printing. There’s only a slight, hairline artifact where the printer had stopped, so that’s good – if the printer actually does run out of resin, the printer can in fact pause without a major impact on print quality.

    Significantly, this problem is fixed in my production unit – with this unit, I’ve had no problems with prints pausing due to the resin cartridge ID issue. It looks like they tweaked the design of the sheet metal around the ID contacts, giving it a bit more clearance and effectively solving the problem. It goes to show how much time and resources are required to vet a product as complex as a 3D printer – with so many sensors, moving parts, and different submodules that have to fit together perfectly throughout a service life involving a million cycles of movement, it takes a lot of discipline to chase down every last detail. So far, my production Form 2 is living up to expectations.

    Removing the Outer Shell

    I love that the Form 2, like the Form 1, uses exclusively hex and torx drive fasteners. No crappy philips or slotted screws here! They also make extensive use of socket cap style, which is a perennial favorite of mine.

    Removing the outer shell and taking a look around, we continue to see evidence of thoughtful engineering. The cable assemblies are all labeled and color-coded; there’s comprehensive detail on chassis grounding; the EMI countermeasures are largely designed-in, as opposed to band-aided at the last minute; and the mechanical engineering got kicked up a notch.

    I appreciated the inclusion of an optical limit switch on the peel drive. The previous generation’s peel mechanism relied on a mechanical clutch with a bit of overdrive, which meant every peel cycle ended with a loud clicking sound. Now, it runs much more quietly, thanks to the feedback of the limit switch.


    Backside of the Form 2 LCD + touchscreen assembly.

    The touchpanel and display are mounted on the outer shell. The display is a DLC0430EZG 480×272 pixel TFT LCD employing a 24-bit RGB interface. I was a bit surprised at the use of a 30-pin ribbon cable to transmit video data between the electronics mainboard and the display assembly, as unshielded ribbon cables are notorious for unintentional RF emissions that complicate the certification process. However, a closer examination of the electronics around the ribbon cable reveal the inclusion of a CMOS-to-LVDS serdes IC on either side of the cable. Although this increases the BOM, the use of differential signaling greatly reduces the emissions footprint of the ribbon cable while improving signal integrity over an extended length of wire.

    Significantly, the capacitive touchpanel’s glass seems to be a full custom job, as indicated by the fitted shape with hole for mounting the power button. The controller IC for the touchpanel is a Tango C44 by PIXCIR, a fabless semiconductor company based out of Suzhou, China. It’s heartening to see that the market for capacitve touchpanels has commoditized to the point where a custom panel makes sense for a relatively low volume product. I remember trying to source captouch solutions back in 2008, just a couple years after the iPhone’s debut popularized capacitive multi-touch sensors. It was hard to get any vendor to return your call if you didn’t have seven figures in your annual volume estimate, and the quoted NRE for custom glass was likewise prohibitive.

    Before leaving the touchpanel and display subsection, I have to note with a slight chuckle the two reference designators (R22 and U4) that are larger than the rest. It’s a purely cosmetic mistake which I recognize because I’ve done it myself several times. From the look of the board, I’m guessing it was designed using Altium. Automatic ECOs in Altium introduce new parts with a goofy huge default designator size, and it’s easy to miss the difference. After all, you spend most of your time editing the PCB with the silkscreen layer turned off.

    The Electronics

    As an electronics geek, my attention was first drawn to the electronics mainboard and the galvanometer driver board. The two are co-mounted on the right hand side of the printer, with a single 2×8 0.1” header spanning the gap between the boards. The mounting seems to be designed for easy swapping of the galvanometer board.

    I have a great appreciation for Formlabs’ choice of using a Variscite SOM (system-on-module). I can speak from first-hand experience, having designed the Novena laptop, that it’s a pain in the ass to integrate a high speed CPU, DDR3 memory, and power management into a single board with complex mixed-signal circuitry. Dropping down a couple BGA’s and routing the DDR3 fly-by topology while managing impedance and length matching is just the beginning of a long series of headaches. You then get to look forward to power sequencing, hardware validation, software drivers, factory testing, yield management and a hundred extra parts in your supply chain. Furthermore, many of the parts involved in the CPU design benefit from economies of scale much larger than can be achieved from this one product alone.

    Thus while it may seem attractive from a BOM standpoint to eliminate the middleman and integrate everything into a single PCB, from a system standpoint the effort may not amortize until the current version of the product has sold a few thousand units. By using a SOM, Formlabs reduces specialized engineering staff, saves months on the product schedule, and gains the option to upgrade their CPU without having to worry about amortization.

    Furthermore, the pitch of the CPU and DDR3 BGAs are optimized for compact designs and assume a 6 or 8-layer PCB with 3 or 4-mil design rules. If you think about it, only the 2 square inches around the CPU and DRAM require these design rules. If the entire design is just a couple square inches, it’s no big deal to fab the entire board using premium design rules. However, the Form 2’s main electronics board is about 30 square inches. Only 2 square inches of this would require the high-spec design rules, meaning they would effectively be fabricating 28 square inches of stepper motor drivers using an 8-layer PCB with 3-mil design rules. The cost to fabricate such a large area of PCB adds up quickly, and by reducing the technology requirement of the larger PCB they probably make up decent ground on the cost overhead of the SOM.

    Significantly, Formlabs was very selective about what they bought from Variscite: the SOM contained neither Wifi nor FLASH memory, even though the SOM itself had provisions for both. These two modules can be integrated onto the mainboard without driving up technology requirements, so Formlabs opted to self-source these components. In essence, they kept Variscite’s mark-up limited to a bare minimum set of components. The maturity to pick and choose cost battles is a hallmark of an engineering team with experience working in a startup environment. Engineers out of large, successful companies are used to working with virtually limitless development budgets and massive purchasing leverage, and typically show less discretion when allocating effort to cost reduction.


    Mainboard assembly with SOM removed; back side of SOM is photoshopped into the image for reference.

    I also like that Formlabs chose to use eMMC FLASH, instead of an SD card, for data storage. It’s probably a little more expensive, but the supply chain for eMMC is a bit more reliable than commodity SD memory. As eMMC is soldered onto the board, J3 was added to program the memory chip after assembly. It looks like the same wires going to the SOM are routed to J3, so the mainboard is probably programmed before the SOM is inserted.

    Formlabs also integrates the stepper motor drivers into the mainboard, instead of using DIP modules like the Makerbot did until at least the Replicator’s Mighty Board Rev E. I think the argument I heard for the DIP modules was serviceability; however, I have to imagine the DIP modules are problematic for thermal management. PCBs are pretty good heatsinks, particularly those with embedded ground planes. Carving up the PCB into tiny modules appreciably increases the thermal resistance between the stepper motor driver and the air around it, which might actually drive up the failure rate. The layout of the stepper motor drivers on the Formlabs mainboard show ample provisions for heat to escape the chips into the PCB through multiple vias and large copper fills.


    Mainboard assembly with annotations according to the discussion in this post.

    Overall, the mainboard was thoughtfully designed and laid out; the engineering team (or engineer) was thinking at a system-level. They managed to escape the “second system effect” by restrained prioritization of engineering effort; just because they raised a pile of money didn’t mean they had to go re-engineer all the things. I also like that the entire layout is single-sided, which simplifies assembly, inspection and testing.

    I learned a lot from reading this board. I’ve often said that reading PCBs is better than reading a textbook for learning electronics design, which is part of the reason I do a monthly Name that Ware. For example, I don’t have extensive experience in designing motor controllers, so next time I need to design a stepper motor driver, I’m probably going to have a look at this PCB for ideas and inspiration – a trivial visual inspection will inform me on what parts they used, the power architecture, trace widths, via counts, noise isolation measures and so forth. Even if the hardware isn’t Open, there’s still a lot that can be learned just by looking at the final design.

    Now, I turn my attention to the galvanometer driver board. This is a truly exciting development! The previous generation used a fully analog driver architecture which I believe is based on an off-the-shelf galvanometer driver. A quick look around this PCB reveals that they’ve abandoned closing the loop in the analog domain, and stuck a microcontroller in the signal processing path. The signal processing is done by a STM32F373 – a 72 MHz, Cortex-M4 with FPU, HW division, and DSP extensions. Further enhancing its role as a signal processing element, the MCU integrates a triplet of 16-bit sigma-delta ADCs and 12-bit DACs. The board also has a smattering of neat-looking support components, such as a MCP42010 digital potentiometer, a fairly handsome OPA4376 precision rail-to-rail op amp, and a beefy LM1876 20W audio amplifier, presumably used to drive the galvanometer voice coils.

    The power for the audio amplifier is derived from a pair of switching regulators, a TPS54336A handling the positive rail, and an LTC3704 handling the negative rail. There’s a small ECO wire on the LTC3704 which turns off burst mode operation; probably a good idea, as burst mode would greatly increase the noise on the negative rail, and in this application standby efficiency isn’t a paramount concern. I’m actually a little surprised they’re able to get the performance they need using switching regulators, but with a 20W load that may have been the only practical option. I guess the switching regulator’s frequency is also much higher than the bandwidth of the galvos, so maybe in practice the switching noise is irrelevant. There is evidence of a couple of tiny SOT-23 LDOs scattered around the PCB to clean up the supplies going to sensitive analog front-end circuitry, and there’s also this curious combination of a FQD7N10L NFET plus MPC6L02 dual op-amp. It looks like they intended the NFET to generate some heat, given the exposed solder slug on the back side, which makes me think this could be a discrete pass-FET LDO of some type. There’s one catch: the MCP6L02 can only operate at up to 6V, and power inside the Form 2 is distributed at 24V. There’s probably something clever going on here that I’m not gathering from a casual inspection of the PCBs; perhaps later I’ll break out some oscope probes to see what’s going on.

    Overall, this ground-up redesign of the galvanometer driver should give Formlabs a strong technological foundation to implement tricks in the digital domain, which sets it apart from clones that still rely upon off-the-shelf fully analog galvanometer driver solutions.

    Before leaving our analysis of the electronics, let’s not forget the main power supply. It’s a Meanwell EPS-65-24-C. The power supply itself isn’t such a big deal, but the choice to include it within the chassis is interesting. Many, if not most, consumer electronic devices prefer to use external power bricks because it greatly simplifies certification. Devices that use voltages below 60V fall into the “easy” category for UL and CE certification. By pulling the power supply into the chassis, they are running line voltages up to 240V inside, which means they have to jump through IEC 60950-1 safety testing. It ups the ante on a number of things, including the internal wiring standards and the flame retardance of any plastics used in the assembly. I’m not sure why they decided to pull the power supply into the chassis; they aren’t using any fancy point-of-load voltage feedback to cancel out IR drops on the cable. My best guess is they felt it would either be a better customer experience to not have to deal with an external power brick, or perhaps they were bitten in the previous generation by flaky power bricks or ground loop/noise issues that sometimes plague devices that use external AC power supplies.

    The Mechanical Platform

    It turns out that my first instinct to rip out the electronics was probably the wrong order for taking apart the Form 2. A closer inspection of the base reveals a set of rounded rectangles that delineate the screws belonging to each physical subsystem within the device. This handy guide makes assembly (and repair) much easier.

    The central set of screws hold down the mechanical platform. Removing those causes the whole motor and optics assembly to pop off cleanly, giving unfettered access to all the electronics.

    I’m oddly excited about the base of the Form 2. It looks like just a humble piece of injection molded plastic. But this is an injection molded piece of plastic designed to withstand the apocalypse. Extensive ribbing makes the base extremely rigid, and resistant to warpage. The base is also molded using glass-filled polymer – the same tough stuff used to make Pelican cases and automotive engine parts. I’ve had the hots for glass-filled polymers recently, and have been itching for an excuse to use it in one of my designs. Glass-filled polymer isn’t for happy-meal toys or shiny gadgets, it’s tough stuff for demanding applications, and it has an innately rugged texture. I’m guessing they went for a bomb-proof base because anything less rigid would lead to problems keeping the resin tank level. Either that, or someone in Formlabs has the same fetish I have for glass filled polymers.

    Once removed from the base, the central mechanical chassis stands upright on its own. Inside this assembly is the Z-axis leadscrew for the build platform, resin level sensor, resin heater, peel motor, resin stirrer, and the optics engine.

    Here’s a close-up of the Z-stepper motor + leadscrew, resin level & temperature sensor, and resin valve actuator. The resin valve actuator is a Vigor Precision BO-7 DC motor with gearbox, used to drive a swinging arm loaded with a spring to provide the returning force. The arm pushes on the integral resin cartridge valve, which looks uncannily like the bite valve from a Camelback.

    The resin tank valve is complimented by the resin tank’s air vent, which also looks uncannily like the top of a shampoo bottle.

    My guess is Formlabs is either buying these items directly from the existing makers of Camelback and shampoo products, in which case First Sale Doctrine means any patent claims that may exist on these has been exhausted, or they have licensed the respective IP to make their own version of each.

    The resin level and temperature sensor assembly is also worth a closer look. It’s a PCB that’s mounted directly behind the resin tank, and in front of the Z-motor leadscrew.


    Backside of the PCB mounted directly behind the resin tank.

    It looks like resin level is measured using a TI FDC1004 capacitive liquid level sensor. I would have thought that capacitive sensing would be too fussy for accurate liquid level sensing, but after reading the datasheet for the FDC1004 I’m a little less skeptical. However, I imagine the sensor is extremely sensitive to all kinds of contamination, the least of which is resin splattered or dripped onto the sensor PCB.


    Detail of the sensor PCB highlighting the non-contact thermopile temperature sensor.

    The resin temperature sense mechanism is also quite interesting. You’ll note a little silvery square, shrouded in plastic, mounted on the PCB behind the resin tank. First of all, the plastic shroud on my unit is clearly a 3D printed piece done by another Formlabs printer. You can see the nubs from the support structure and striation artifacts from the buildup process. I love that they’re dogfooding and using their own products to prototype and test; it’s a bad sign if the engineering team doesn’t believe in their own product enough to use it themselves.

    Unscrewing the 3D printed shroud reveals a curious flip-chip CSP device, which I’m guessing is a TI TMP006 or TMP007 MEMS therompile. Although there are no part numbers on the chip, a quick read through the datasheet reveals a reference layout that is a dead ringer for the pattern on the PCB around the chip. Thermopiles can do non-contact remote temperature sensing, and it looks like this product has an accuracy of about +/-1 C between 0-60C. This explains the mystery of how they’re able to report the resin temperature on the UI without any sort of probe dipping into the resin tank.

    But then how do they heat it? Look under the resin tank mount, and we find another PCB.

    When I first saw this board, I thought its only purpose was to hold the leafspring contacts for the ID chip that helps track individual resin tanks and what color resin was used in them. Flip the PCB over, and you’ll see a curious pinkish tape covering the reverse surface.

    The pinkish tape is actually a thermal gap sealer, and peeling the tape back reveals that the PCB itself has a serpentine trace throughout, which means they are using the resistivity of the copper trace on the PCB itself as a heating mechanism for the resin.

    Again, I wouldn’t have guessed this is something that would work as well as it does, but there you have it. It’s a low-cost mechanism for controlling the temperature of the resin during printing. Probably the PCB material is the most expensive component, even more than the thermopile IR sensor, and all that’s needed to drive the heating element is a beefy BUK9277 NFET.

    I’ve been to the Formlabs offices in Boston, and it does get rather chilly and dry there in the winter, so it makes sense they would consider cold temperature as a variable that could cause printing problems on the Form 2.

    Cold weather isn’t a problem here in Singapore; however, persistent 90% humidity conditions is an issue. If I didn’t use my Form 1 for several weeks, the first print would always come out badly; usually I’d have to toss the resin in the tank and pour a fresh batch for the print to come out. I managed to solve this problem by placing a large pack of desiccant next to the resin tank, as well as using the shipping lid to try to seal out moisture. However, I’m guessing they have very few users in the tropics, so humidity-related print problems are probably going to be a unique edge case I’ll have to solve on my own for some time to come.

    The Optics Pathway

    Finally, the optics – I’m saving the best for last. The optics pathway is the beating heart of the Form 2.


    The last thing uncured resin sees before it turns into plastic.

    The first thing I noticed about the optics is the inclusion of a protective glass panel underneath the resin tank. In the Form 1, if the build platform happened to drip resin while the tank was removed, or if the room was dusty, you had the unenviable task of reaching into the printer to clean the mirror. The glass panel simplifies the cleaning operation while protecting sensitive optics from dust and dirt.

    I love that the protective glass has an AR coating. You can tell there’s an AR coating from the greenish tint of the reflections off the surface of the glass. AR coatings are sexy; if I had a singles profile, you’d see “the green glint of AR-coated glasses” under turn-ons. Of course, the coating is there for functional reasons – any loss of effective laser power due to reflections off of the protective glass would reduce printing efficiency.

    The contamination-control measures don’t just stop at a protective glass cover. Formlabs also provisioned a plastic shroud around the entire optics assembly.


    Bottom view of the mechanical platform showing the protective shrouds hiding the optics.

    Immediately underneath the protective glass sheet is a U-shaped PCB which I can only assume is used for some kind of calibration. The PCB features five phtoodetectors; one mounted in “plain sight” of the laser, and four mounted in the far corners on the reverse side of the PCB, with the detectors facing into the PCB, such that the PCB is obscuring the photodetectors. A single, small pinhole located in the center of each detector allows light to fall onto the obscured photodetectors. However, the size of the pinhole and the dimensional tolerance of the PCB is probably too large for this to be an absolute calibration for the printer. My guess is this is probably used as more of a coarse diagnostic to confirm laser power and range of motion of the galvanometers.

    Popping off the shroud reveals the galvanometer and laser assembly. The galvanometers sport a prominent Formlabs logo. They are a Formlabs original design, and not simply a relabeling of an off the shelf solution. This is a really smart move, especially in the face of increasing pressure from copycats. Focusing resources into building a proprietary galvo is a trifecta for Formlabs: they get distinguished print quality, reduced cost, and a barrier to competition all in one package. Contrast this to Formlabs’ decision to use a SOM for the CPU; if Formlabs can build their own galvo & driver board, they certainly had the technical capability to integrate a CPU into the mainboard. But in terms of priorities, improving the galvo is a much better payout.

    Readers unfamiliar with galvanometers may want to review a Name that Ware I did of a typical galvanometer a while back. In a nutshell, a typical galvanometer consists of a pair of voice coils rotating a permanent magnet affixed to a shaft. The shaft’s angle is measured by an optical feedback system, where a single light source shines onto a paddle affixed to the galvo’s shaft. The paddle alternately occludes light hitting a pair of photodetectors positioned behind the paddle relative to the light source.

    Now, here’s the entire Form 2 galvo assembly laid out in pieces.


    Close-up view of the photoemitter and detector arrangement.

    Significantly, the Form 2 galvo has not two, but four photodetectors, surrounding a single central light source. Instead of a paddle, a notch is cut into the shaft; the notch modulates the light intensity reaching the photodiodes surrounding the central light source according to the angle of the shaft.


    The notched shaft above sits directly above the photoemitter when the PCB is mated to the galvo body.

    This is quite different from the simple galvanometer I had taken apart previously. I don’t know enough about galvos to recognize if this is a novel technique, or what exactly is the improvement they hoped to get by using four photodiodes instead of two. With two photodiodes, you get to subtract out the common mode of the emitter and you’re left with the error signal representing the angle of the shaft: two variables solving for two unknowns. With four photodiodes, they can solve for a couple more unknowns – but what are they? Maybe they are looking to correct for alignment errors of the light source & photodetectors relative to the shaft, wobble due to imperfections in the bearings, or perhaps they’re trying to avoid a dead-spot in the response of the photodiodes as the shaft approaches the extremes of rotation. Or perhaps the explanation is as simple as removing the light-occluding paddle reduces the mass of the shaft assembly, allowing it to rotate faster, and four photodetectors was required to produce an accurate reading out of a notch instead of the paddle. When I reached out to Formlabs to ask about this, someone in the know responded that the new design is an improvement on three issues: more signal leading to an improved SNR, reduced impact of off-axis shaft motion, and reduced thermal drift due to better symmetry.

    This is the shaft plus bearings once it’s pulled out of the body of the galvo. The gray region in the middle is the permanent magnet, and it’s very strong.

    And this is staring back into the galvo with the shaft removed. You can see the edges of the voice coils. I couldn’t remove them from the housing, as they seem to be fixed in place with some kind of epoxy.

    Epilogue
    And there you have it – the Form 2, from taking off its outer metal case down to the guts of its galvanometers. It was a lot of fun tearing down the Form2, and I learned a lot while doing it. I hope you also enjoyed reading this post, and perhaps gleaned a couple useful bits of knowledge along the way.

    If you think Formlabs is doing cool stuff and solving interesting problems, good news: they’re hiring! They have new positions for a Software Lead and an Electrical Systems Lead. Follow the links for a detailed description and application form.

    by bunnie at March 22, 2016 05:58 PM

    Winner, Name that Ware February 2016

    The Ware for February 2016 was indeed a Commodore 65 prototype. As expected, the ware was quite easy to guess, and the prize goes to Philipp Mundhenk. Congrats, email me for your prize!

    Here’s an image of the full motherboard, and its boot screen:

    by bunnie at March 22, 2016 05:50 PM

    Free Electrons

    Free Electrons contributing Linux kernel initial support for Annapurna Labs ARM64 Platform-on-Chip

    Annapurna Labs LogoWe are happy to announce that on February 8th 2016 we submitted to the mainline Linux kernel the initial support for Annapurna Labs Alpine v2 Platform-on-Chip based on the 64-bit ARMv8 architecture.

    See our patch series:

    Annapurna Labs was founded in 2011 in Israel. Annapurna Labs provides 32-bit and 64-bit ARM products including chips and subsystems under the Alpine brand for the home NAS, Gateway and WiFi router equipment, see this page for details. The 32-bit version already has support in the official Linux kernel (see alpine.dtsi), and we have started to add support for the quad core 64-bit version, called Alpine v2, which brings significant performance for the home.

    This is our initial contribution and we plan to follow it with additional Alpine v2 functionality in the near future.

    by Thomas Petazzoni at March 22, 2016 05:38 AM

    March 18, 2016

    Elphel

    NAND flash support for Xilinx Zynq in U-Boot SPL

    Overview

    • Target board: Elphel 10393 (Xilinx Zynq 7Z030) with 1GB NAND flash
    • U-Boot final image files (both support NAND flash commands):
      • boot.bin – SPL image – loaded by Xilinx Zynq BootROM into OCM, no FSBL required
      • u-boot-dtb.img – full image – loaded by boot.bin into RAM
    • Build environment and dependencies (for details see this article) :


     

    The story

    First of all, Ezynq was updated to use the mainstream U-Boot to remove an extra agent (u-boot-xlnx) from the dependency chain. But since the flash driver for Xilinx Zynq hasn’t make it to the mainstream yet it was copied to Ezynq’s source tree for U-Boot. When building the source tree is copied over U-Boot source files. We will make a patch someday.

    Full image (u-boot-dtb.img)

    Next, the support for flash and commands was added to the board configuration for the full u-boot image. Required defines:

    include/configs/elphel393.h (from zynq-common.h in u-boot-xlnx):
    #define CONFIG_NAND_ZYNQ
    #ifdef CONFIG_NAND_ZYNQ
    #define CONFIG_CMD_NAND_LOCK_UNLOCK /*zynq driver doesn't have lock/unlock commands*/
    #define CONFIG_SYS_MAX_NAND_DEVICE 1
    #define CONFIG_SYS_NAND_SELF_INIT
    #define CONFIG_SYS_NAND_ONFI_DETECTION
    #define CONFIG_MTD_DEVICE
    #endif
    #define CONFIG_MTD

    NOTE: original Zynq NAND flash driver for U-Boot (zynq_nand.c) doesn’t have Lock/Unlock commands. Same applies to pl35x_nand.c in the kernel they provide. By design, on power on the NAND flash chip on 10393 is locked (write protected). While these commands were added to both drivers there’s no need for unlock in U-Boot as all of the writing will be performed from OS boot from either flash or micro SD card. Out there some designs with NAND flash do not have flash locked on power on.

    And configs/elphel393_defconfig:

    CONFIG_CMD_NAND=y

    There are few more small modifications to add the driver to the build – see ezynq/u-boot-tree. Anyways, it worked on the board. Easy. Type “nand” in u-boot terminal for available commands.

    SPL image (boot.bin)

    Then the changes for the SPL image were made.

    Currently U-Boot runs twice to build both images. For the SPL run it sets CONFIG_SPL_BUILD, the results are found in spl/ folder. So, in general, if one would like to build U-Boot with SPL supporting NAND flash for some other board he/she should check out common/spl/spl_nand.c for the required functions, they are:

    nand_spl_load_image()
    nand_init() /*no need if drivers/mtd/nand.c is included in the SPL build*/
    nand_deselect() /*usually an empty function*/

    And drivers/mtd/nand/ – for driver examples for SPL – there are not too many of them for some reason.

    For nand_init() I included drivers/mtd/nand.c – it calls board_nand_init() which is found in the driver for the full image – zynq_nand.c.

    Defines in include/configs/elphel393.h:

    #define CONFIG_SPL_NAND_ELPHEL393
    #define CONFIG_SYS_NAND_U_BOOT_OFFS 0x100000 /*look-up in dts!*/
    #define CONFIG_SPL_NAND_SUPPORT
    #define CONFIG_SPL_NAND_DRIVERS
    #define CONFIG_SPL_NAND_INIT
    #define CONFIG_SPL_NAND_BASE
    #define CONFIG_SPL_NAND_ECC
    #define CONFIG_SPL_NAND_BBT
    #define CONFIG_SPL_NAND_IDS
    /* Load U-Boot to this address */
    #define CONFIG_SYS_NAND_U_BOOT_DST CONFIG_SYS_TEXT_BASE
    #define CONFIG_SYS_NAND_U_BOOT_START CONFIG_SYS_NAND_U_BOOT_DST

    CONFIG_SYS_NAND_U_BOOT_OFFS 0x100000 – is the offset in the flash where u-boot-dtb.img is written – this is done in OS. The flash partitions are defined in the device tree for the kernel.

    Again a few small modifications (KConfigs and makefiles) to include everything in the build – see ezynq/u-boot-tree.

    NOTES:

    • Before boot.bin was about 60K (out of 192K available). After everything was included the size is 110K. Well, it fits and so the optimization can be done some time in the future for the driver to have only what is needed – init and read.
    • drivers/mtd/nand/nand_base.c – kzalloc would hang the board – had to change it in the SPL build.
    • drivers/mtd/nand/zynq_nand.c – added timeout for some flash functions (NAND_CMD_RESET) – addresses the case when the board has flash width configured (through MIO pins) but doesn’t carry flash or the flash cannot be detected for some reason. Not having timeout hangs such boards.

    Other Notes

    • With U-Boot moving to KBuild nobody knows what will happen to the CONFIG_EXTRA_ENV_SETTINGS – multi-line define.
    • Current U-Boot uses a stripped down device tree – added to Ezynq.
    • The ideal scenario is to boot from SPL straight to OS – the falcon mode (CONFIG_SPL_OS_BOOT). Consider in future.
    • Tertiary Program Loader (TPL) – no plans.

     

    by Oleg Dzhimiev at March 18, 2016 11:40 PM

    Free FPGA: Reimplement the primitives models

    We added the AHCI SATA controller Verilog code to the rest of the camera FPGA project, together they now use 84% of the Zynq slices. Building the FPGA bitstream file requires proprietary tools, but all the simulation can be done with just the Free Software – Icarus Verilog and GTKWave. Unfortunately it is not possible to distribute a complete set of the files needed – our code instantiates a few FPGA primitives (hard-wired modules of the FPGA) that have proprietary license.

    Please help us to free the FPGA devices for developers by re-implementing the primitives as Verilog modules under GNU GPLv3+ license – in that case we’ll be able to distribute a complete self-sufficient project. The models do not need to provide accurate timing – in many cases (like in ours) just the functional simulation is quite sufficient (combined with the vendor static timing analysis). Many modules are documented in Xilinx user guides, and you may run both the original and replacement models through the simulation tests in parallel, making sure the outputs produce the same signals. It is possible that such designs can be used as student projects when studying Verilog.

    Models we are looking for

    The camera project includes more than 200 Verilog files, and it depends on just 29 primitives from the Xilinx simulation library (total number of the files there is 214):

    • BUFG.v
    • BUFH.v
    • BUFIO.v
    • BUFMR.v
    • BUFR.v
    • DCIRESET.v
    • GLBL.v
    • IBUF.v
    • IBUFDS_GTE2.v
    • IBUFDS.v
    • IDELAYCTRL.v
    • IDELAYE2_FINEDELAY.v
    • IDELAYE2.v
    • IOBUF_DCIEN.v
    • IOBUF.v
    • IOBUFDS_DCIEN.v
    • ISERDESE1.v *
    • MMCME2_ADV.v
    • OBUF.v
    • OBUFT.v
    • OBUFTDS.v
    • ODDR.v
    • ODELAYE2_FINEDELAY.v
    • OSERDESE1.v *
    • PLLE2_ADV.v
    • PS7.v
    • PULLUP.v
    • RAMB18E1.v
    • RAMB36E1.v

    This is just a raw list of the unisims modules referenced in the design, it includes PS7.v – a placeholder model of the ARM processing system, modules for AXI functionality simulation are already included in the project. The implementation is incomplete, but sufficient for the the camera simulation and can be used for other Zynq-based projects. Some primitives are very simple (like DCIRESET), some are much more complex. Two modules (ISERDESE1.v and OSERDESE1.v) in the project are the open-source replacements for the encrypted models of the enhanced hardware in Zynq (ISERDESE2.v and OSERDESE2.v) – we used a simple ifdef wrapper that selects reduced (but sufficient for us) functionality of the earlier open source model for simulation and the current “black box” for synthesis.

    The files list above includes all the files we need for our current project, as soon as the Free Software replacement will be available we will be able to distribute the self-sufficient project. Other FPGA development projects may need other primitives, so ideally we would like to see all of the primitives to have free models for simulation.

    Why is it important

    Elphel is developing high-performance products based on the FPGA desings that we believe are created for Freedom. We share all the code with our users under GNU General Public License version 3 (or later) but the project depends on proprietary tools distributed by vendors who have monopoly on the tools for their silicon.

    There are very interesting projects (like icoBOARD) that use smaller devices with completely Free toolchain (Yosys), but the work of those developers is seriously complicated by non-cooperation of the FPGA vendors. I hope that in the future there will be laws that will limit the monopoly of the device manufacturers and require complete documentation for the products they release to the public. There are advanced patent laws that can protect the FPGA manufacturers and their inventions from the competitors, there is no real need for them to fight against their users by hiding the documentation for the products.

    Otherwise this secrecy and “Security through Obscurity” will eventually (and rather soon) lead to a very insecure world where all those self-driving cars, “smart homes” will obey not us, but just the “bad guys” as the current software malware will get to even deeper hardware level. It is very naive to believe that they (the manufacturers) are ultimate masters and have the complete control of “their” devices of ever growing complexity. Unfortunately they do not realize this and are still living in the 20-th century dreams, treating their users as kids who can only play with “Lego blocks” and believe in powerful Wizards who pretend to know everything.

    We use proprietary toolchain for implementation, but exclusively Free tools – for simulation

    Our projects require devices that are more advanced than those that already can be programmed with independently designed Free Software tools, so we have to use the proprietary ones. Freeing the simulation seems to be achievable, and we made a step in this direction – made the whole project simulation possible with the Free Software. Working with the HDL code and simulating it takes most part of the FPGA design cycle, in our experience it is 2/3 – 3/4, and only the remaining part involves running the toolchain and test/troubleshoot the hardware. The last step (hardware troubleshooting) can also be done without any proprietary software – we never used any in this project that utilizes most of the Xilinx Zynq FPGA resources. Combination of the Verilog modules and extensible Python programs that run on the target devices proved to be a working and convenient solution that keeps the developer in the full control of the process. These programs read the Verilog header files with parameter definitions to synchronize register and bit fields addresses between the hardware and the software that uses them.

    Important role of the device primitives models

    Modern FPGA include many hard-wired embedded modules that supplement the uniform “sea of gates” – addition of such modules significantly increases performance of the device while preserves its flexibility. The modules include memory blocks, DSP slices, PLL circuits, serial-to-parallel and parallel-to-serial converters, programmable delays, high-speed serial transceivers, processor cores and more. Some modules can be automatically extracted by the synthesis software from the source HDL code, but in many cases we have to directly instantiate such primitives in the code, and this code now directly references the device primitives.

    The less of the primitives are directly instantiated in the project – the more portable (not tied to a particular FPGA architecture) it is, but in some cases synthesis tools (they are proprietary, so not fixable by the users) incorrectly extract the primitives, in other – the module functionality is very specific to the device and the synthesis tool will not even try to recognize it in the behavioral Verilog code.

    Even open source proprietary modules are inconvenient

    In earlier days Xilinx was providing all of their primitives models as open source code (but under non-free license), so it was possible to use Free Software tools to simulate the design. But even then it was not so convenient for both our users and ourselves.

    It is not possible to distribute the proprietary code with the projects, so our users had to register with the FPGA manufacturer, download the multi-gigabyte software distribution and agree to the specific license terms before they were able to extract those primitives models missing from our project repository. The software license includes the requirement to install mandatory spyware that you give a permission to transfer your files to the manufacturer – this may be unacceptable for many of our users.

    It is also inconvenient for ourselves. The primitives models provided by the manufacturer sometimes have problems – either do not match the actual hardware or lack full compatibility with the simulator programs we use. In such cases we were providing patches that can be applied to the code provided by the manufacturer. If Xilinx kept them in a public Git repository, we could base our patches on particular tags or commits, but it is not the case and the manufacturer/software provider preserves the right to change the distributed files at any time without notice. So we have to update the patches to maintain the simulation working even we did not change a single line in the code.

    Encrippled modules are unacceptable

    When I started working on the FPGA design for Zynq I was surprised to notice that Xilinx abandoned a practice to provide the source code for the simulation models for the device primitives. The new versions of the older primitives (such as ISERDESE2.v and OSERDESE2.v instead of the previous ISERDESE1.v and OSERDESE1.v) now come in encrippled (crippled by encryption) form while they were open-sourced before. And it is likely this alarming tendency will continue – many proprietary vendors are hiding the source code just because they are not so proud about its quality and can not resist a temptation to encrypt it instead of removing the obsolete statements and updating the code to the modern standards.

    Such code is not just inconvenient, it is completely unacceptable for our design process. The first obvious reason is that it is not compatible with the most important development tool – a simulator. Xilinx provides decryption keys to trusted vendors of proprietary simulators and I do not have plans to abandon my choice of the tool just because the FPGA manufacturer prefers a different one.

    Personally I would not use any “black boxes” even if Icarus supported them – the nature of the FPGA design is already rather complex to spend any extra time of your life on guessing – why this “black box” behaves differently than expected. And all the “black boxes” and “wizards” are always limited and do not 100% match the real hardware. That is normal, when they cover most of the cases and you have the ability to peek inside when something goes wrong, so you can isolate the bug and (if it is actually a bug of the model – not your code) report it precisely and find the solution with the manufacturer support. Reporting problems in a form “my design does not work with your black box” is rather useless even when you provide all your code – it will be a difficult task for the support team to troubleshoot a mixture of your and their code – something you could do yourself better.

    So far we used two different solutions to handle encrypted modules. In one case when the older non-crippled model was available we just used the older version for the new hardware, the other one required complete re-implementation of the GTX serial transceiver model. The current code has many limitations even with its 3000+ lines of code, but it proved to be sufficient for the SATA controller development.

    Additional permission under GNU GPL version 3 section 7

    GNU General Public License Version 3 offers a tool to apply the license in a still “grey area” of the FPGA code. When we were using earlier GPLv2 for the FPGA projects we realized that it was more a statement of intentions than a binding license – FPGA bitstream as well as the simulation inevitably combined free and proprietary components. It was OK for us as the copyright holders, but would make it impossible for others to distribute their derivative projects in a GPL-compliant way. Version 3 has a Section 7 that can be used to give the permission for distribution of the derivative projects that depend on non-free components that are still needed to:

    1. generate a bitstream (equivalent to a software “binary”) file and
    2. simulate the design with Free Software tools

    The GPL requirement to provide other components under the same license terms when distributing the combined work remains in force – it is not possible to mix this code with any other non-free code. The following is our wording of the additional permission as included in every Verilog file header in Elphel FPGA projects.

    Additional permission under GNU GPL version 3 section 7:
    If you modify this Program, or any covered work, by linking or combining it
    with independent modules provided by the FPGA vendor only (this permission
    does not extend to any 3-rd party modules, "soft cores" or macros) under
    different license terms solely for the purpose of generating binary "bitstream"
    files and/or simulating the code, the copyright holders of this Program give
    you the right to distribute the covered work without those independent modules
    as long as the source code for them is available from the FPGA vendor free of
    charge, and there is no dependence on any encrypted modules for simulating of
    the combined code. This permission applies to you if the distributed code
    contains all the components and scripts required to completely simulate it
    with at least one of the Free Software programs.

    Available documentation for Xilinx FPGA primitives

    Xilinx has User Guides files available for download on their web site, some of the following links include release version and may change in the future. These files provide valuable information needed to re-implement the simulation models.

    • UG953 Vivado Design Suite 7 Series FPGA and Zynq-7000 All Programmable SoC Libraries Guide lists all the primitives, their I/O ports and attributes
    • UG474 7 Series FPGAs Configurable Logic Block has description of the CLB primitives
    • UG473 7 Series FPGAs Memory Resources has description for Block RAM modules, ports, attributes and operation of these modules
    • UG472 7 Series FPGAs Clocking Resources provides information for the clock buffering (BUF*) primitives and clock management tiles – MMCM and PLL primitives of the library
    • UG471 7 Series FPGAs SelectIO Resources covers advanced I/O primitives, including DCI, programmable I/O delays elements and serializers/deserializers, I/O FIFO elements
    • UG476 7 Series FPGAs GTX/GTH Transceivers is dedicated to the high speed serial transceivers. Simulation models for these modules are partially re-implemented for use in AHCI SATA Controller.

    by Andrey Filippov at March 18, 2016 10:42 PM

    March 15, 2016

    Elphel

    AHCI platform driver

    AHCI PLATFORM DRIVER

    In kernels prior to 2.6.x AHCI was only supported through PCI and hence required custom patches to support platform AHCI implementation. All modern kernels have SATA support as part of AHCI framework which significantly simplifies driver development. Platform drivers follow the standard driver model convention which is described in Documentation/driver-model/platform.txt in kernel source tree and provide methods called during discovery or enumeration in their platform_driver structure. This structure is used to register platform driver and is passed to module_platform_driver() helper macro which replaces module_init() and module_exit() functions. We redefined probe() and remove() methods of platform_driver in our driver to initialize/deinitialize resources defined in device tree and allocate/deallocate memory for driver specific structure. We also opted to resource-managed function devm_kzalloc() as it seems to be preferred way of resource allocation in modern drivers. The memory allocated with resource-managed function is associated with the device and will be freed automatically after driver is unloaded.

    HARDWARE LIMITATIONS

    As Andrey has already pointed out in his post, current implementation of AHCI controller has several limitations and our platform driver is affected by two of them.
    First, there is a deviation from AHCI specification which should be considered during platform driver implementation. The specification defines that host bus adapter uses system memory for the Command List Structure, Received FIS Structure and Command Tables. The common approach in platform drivers is to allocate a block of system memory with single dmam_alloc_coherent() call, set pointers to different structures inside this block and store these pointers in port specific structure ahci_port_priv. The first two of these structures in x393_sata are stored in the FPGA RAM blocks and mapped to register memory as it was easier to make them this way. Thus we need to allocate a block of system memory for Command Tables only and set other pointers to predefined addresses.
    Second, and the most significant one from the driver’s point of view, proved to be single command slot implemented. Low level drivers assume that all 32 slots in Command List Structure are implemented and explicitly use the last slot for internal commands in ata_exec_internal_sg() function as shown in the following code snippet:
    struct ata_queued_cmd *qc;
    unsigned int tag, preempted_tag;
     
    if (ap->ops->error_handler)
        tag = ATA_TAG_INTERNAL;
    else
        tag = 0;
    qc = __ata_qc_from_tag(ap, tag);

    ATA_TAG_INTERNAL is defined in libata.h and reserved for internal commands. We wanted to keep all the code of our driver in our own sources and make as fewer changes to existing Linux drivers as possible to simplify further development and upgrade to newer kernels. So we decided that substitution of the command tag in our own code which handles command preparation would be the easiest way of fixing this issue.

    DRIVER STRUCTURES

    Proper platform driver initialization requires that several structures to be prepared and passed to platform functions during driver probing. One of them is scsi_host_template and it serves as a direct interface between middle level drivers and low level drivers. Most AHCI drivers use default AHCI_SHT macro to fill the structure with predefined values. This structure contains a field called .can_queue which is of particular interest for us. The .can_queue field sets the maximum number of simultaneous commands the host bus adapter can accept and this is the way to tell middle level drivers that our controller has only one command slot. The scsi_host_template structure was redefined in our driver as follows:
    static struct scsi_host_template ahci_platform_sht = {
        AHCI_SHT(DRV_NAME),
        .can_queue = 1,
        .sg_tablesize = AHCI_MAX_SG,
        .dma_boundary = AHCI_DMA_BOUNDARY,
        .shost_attrs = ahci_shost_attrs,
        .sdev_attrs = ahci_sdev_attrs,
    };

    Unfortunately, ATA layer driver does not take into consideration the value we set in this template and uses hard coded tag value for its internal commands as I pointed out earlier, so we had to fix this in command preparation handler.
    ata_port_operations is another important driver structure as it controls how the low level driver interfaces with upper layers. This structure is defined as follows:
    static struct ata_port_operations ahci_elphel_ops = {
        .inherits = &ahci_ops,
        .port_start = elphel_port_start,
        .qc_prep = elphel_qc_prep,
    };

    The port start and command preparation handlers were redefined to add some implementation specific code. .port_start is used to allocate memory for Command Table and set pointers to Command List Structure and Received FIS Structure. We decided to use streaming DMA mapping instead of coherent DMA mapping used in generic AHCI driver as explained in Andrey’s article. .qc_prep is used to change the tag of current command and organize proper access to DMA mapped buffer.

    PERFORMANCE CONSIDERATIONS

    We used debug code in the driver along with profiling code in the controller to estimate overall performance and found out that upper driver layers introduce significant delays in command execution sequence. The delay between last DMA transaction in a sequence of transactions and next command could be as high as 2 ms. There are various sources of overhead which could lead to delays, for instance, file system operations and context switches in the operating system. We will try to use read/write operations on a raw device to improve performance.

    LINKS

    AHCI/SATA stack under GNU GPL
    GitHub: AHCI driver source code

    by Mikhail Karpenko at March 15, 2016 02:16 AM

    March 14, 2016

    Harald Welte

    Open Source mobile communications, security research and contributions

    While preparing my presentation for the Troopers 2016 TelcoSecDay I was thinking once again about the importance of having FOSS implementations of cellular protocol stacks, interfaces and network elements in order to enable security researches (aka Hackers) to work on improving security in mobile communications.

    From the very beginning, this was the motivation of creating OpenBSC and OsmocomBB: To enable more research in this area, to make it at least in some ways easier to work in this field. To close a little bit of the massive gap on how easy it is to do applied security research (aka hacking) in the TCP/IP/Internet world vs. the cellular world.

    We have definitely succeeded in that. Many people have successfully the various Osmocom projects in order to do cellular security research, and I'm very happy about that.

    However, there is a back-side to that, which I'm less happy about. In those past eight years, we have not managed to attract significant amount of contributions to the Osmocom projects from those people that benefit most from it: Neither from those very security researchers that use it in the first place, nor from the Telecom industry as a whole.

    I can understand that the large telecom equipment suppliers may think that FOSS implementations are somewhat a competition and thus might not be particularly enthusiastic about contributing. However, the story for the cellular operators and the IT security crowd is definitely quite different. They should have no good reason not to contribute.

    So as a result of that, we still have a relatively small amount of people contributing to Osmocom projects, which is a pity. They can currently be divided into two groups:

    • the enthusiasts: People contributing because they are enthusiastic about cellular protocols and technologies.
    • the commercial users, who operate 2G/2.5G networks based on the Osmocom protocol stack and who either contribute directly or fund development work at sysmocom. They typically operate small/private networks, so if they want data, they simply use Wifi. There's thus not a big interest or need in 3G or 4G technologies.

    On the other hand, the security folks would love to have 3G and 4G implementations that they could use to talk to either mobile devices over a radio interface, or towards the wired infrastructure components in the radio access and core networks. But we don't see significant contributions from that sphere, and I wonder why that is.

    At least that part of the IT security industry that I know typically works with very comfortable budgets and profit rates, and investing in better infrastructure/tools is not charity anyway, but an actual investment into working more efficiently and/or extending the possible scope of related pen-testing or audits.

    So it seems we might want to think what we could do in order to motivate such interested potential users of FOSS 3G/4G to contribute to it by either writing code or funding associated developments...

    If you have any thoughts on that, feel free to share them with me by e-mail to laforge@gnumonks.org.

    by Harald Welte at March 14, 2016 11:00 PM

    TelcoSecDay 2016: Open Source Network Elements for Security Analysis of Mobile Networks

    Today I had the pleasure of presenting about Open Source Network Elements for Security Analysis of Mobile Networks at the Troopers 2016 TelcoSecDay.

    The main topics addressed by this presentation are:

    • Importance of Free and Open Source Software implementations of cellular network protocol stacks / interfaces / network elements for applied telecom security research
    • The progress we've made at Osmocom over the last eight years.
    • An overview about our current efforts to implement at 3G Network similar to the existing 2G/2.5G/2.75G implementations.

    There are no audio or video recordings of this session.

    Slides are available at http://git.gnumonks.org/index.html/laforge-slides/plain/2016/telcosecday/foss-gsm.html

    by Harald Welte at March 14, 2016 11:00 PM

    March 13, 2016

    Bunnie Studios

    Preparing for Production of The Essential Guide To Electronics in Shenzhen

    The crowd funding campaign for The Essential Guide to Electronics in Shenzhen is about to wrap up in a couple of days.

    I’ve already started the process of preparing the printing factory for production. Last week, I made another visit to the facility, to discuss production forecasts, lead time and review the latest iteration of the book’s prototype. It’s getting pretty close. I’m now using a heavy, laminated cardstock for the tabbed section dividers to improve their durability. The improved tabs pushes up the cost of the book, and more significantly, pushes the shipping weight of the book over 16 oz, which means I’m now paying a higher rate for postage. However, this is mostly offset by the higher print volume, so I can mitigate the unexpected extra costs.

    The printing factory has a lot of mesmerizing machines running on the floor, like this automatic cover binder for perfect-bound books:

    And this high speed two-color printing press:

    This is probably the very press that the book will be printed on. The paper moves so fast that it’s just a blur as an animated gif. I estimate it does about 150 pages per minute, and each page is about a meter across, which gives it an effective throughput of over a thousand book-sized pages per minute. Even for a run of a couple thousand books, this machine would only print for about 15 minutes before it has to stop for a printing plate swap, an operation which takes a few minutes to complete. This explains why books don’t get really cheap until the volume reaches tens of thousands of copies.

    Above is the holepunch used for building prototypes of ring-bound books. The production punch is done using a semi-automated high-volume die-cutter, but for the test prints, this is the machine used to punch out the holes.

    Sorry, your browser does not support the video tag.

    The ring binding itself is done by a fairly simple machine. The video above shows the process used to adjust the machine’s height for a single shot on the prototype book. In a production scenario, there would be a few workers on the table to the left of the binding machine aligning pages, adding the covers, and inserting the ring stock. Contrast this to the fully automated perfect binding machine shown at the top of this post — ring binding is a much more expensive binding style in this factory, since they haven’t automated the process (yet).

    I also got a chance to see the machine that gilds and debosses the book cover. It’s a bit of a different process than the edge-gilding I described in the previous post about designing the cover.

    Here, an aluminum plate is first made with the deboss pattern. It looks pretty neat — I’ve half a mind to ask the laoban if he’d save the used plates for me to keep as a souvenir, although the last thing I need in my tiny flat in Singapore is more junk.

    The plate is then glued into a huge press. This versatile machine can do debossing, die cutting, and gilding on sheets of paper as large as A0. For the gilding operation, the mounting face for the aluminum plate is heated to around 130 degrees Celsius.

    I think it’s kind of cute how they put good luck seals all over the machines. The characters say “kai gong da ji” which literally translated means “start operation, big luck”. I don’t know what’s the underlying reason — maybe it’s to wish good luck on the machine, the factory, or the operator; or maybe fix its feng shui, or some kind of voodoo to keep the darned thing from breaking down again. I’ll have to remember to ask what’s the reason for the sticker next time I visit.

    Once at temperature, the gilding foil is drawn over the plate, and the alignment of the plate is determined by doing a test shot onto a transparent plastic sheet. The blank cover is then slid under the sheet, taped in place, and the clear sheet removed.

    The actual pressing step is very fast — so fast I didn’t have a chance to turn my camera into video mode, so I only have a series of three photos to show the before, pressing, and after states.

    And here’s a photo of me with the factory laoban (boss), showing off the latest prototype. I’ve often said that if you can’t meet the laoban, the factory’s too big for you. Having a direct relationship with the laoban has been helpful for this project; he’s very patiently addressed all my strange customization requests, and as a side bonus he seems to know all the good restaurants in the area so the after-work meals are usually pretty delicious.

    I’m looking forward to getting production started on the book, and getting all the pledge rewards delivered on-time. Now’s the last chance to back the crowd funding campaign and get the book at a discounted price. I will order some extra copies of the book, but it’s been hard to estimate demand, so there’s a risk the book could sell out soon after the campaign concludes.

    by bunnie at March 13, 2016 02:11 PM

    March 12, 2016

    Elphel

    AHCI/SATA stack under GNU GPL

    Implementation includes AHCI SATA host adapter in Verilog under GNU GPLv3+ and a software driver for GNU/Linux running on Xilinx Zynq. Complete project is simulated with Icarus Verilog, no encrypted modules are required.

    This concludes the last major FPGA development step in our race against finished camera parts and boards already arriving to Elphel facility before the NC393 can be shipped to our customers.

    Fig. 1. AHCI Host Adapter block diagram

    Fig. 1. AHCI Host Adapter block diagram


    Why did we need SATA?

    Elphel cameras started as network cameras – devices attached to and controlled over the Ethernet, the previous generations used 100Mbps connection (limited by the SoC hardware), and NC393 uses GigE. But this bandwidth is still not sufficient as many camera applications require high image quality (compared to “raw”) without compression artifacts that are always present (even if not noticeable by the human viewer) with the video codecs. Recording video/images to some storage media is definitely an option and we used it in the older camera too, but the SoC IDE controller limited the recording speed to just 16MB/s. It was about twice more than the 100Mb/s network, but still was a bottleneck for the system in many cases. The NC393 can generate 12 times the pixel rate (4 simultaneous channels instead of a single one, each running 3 times faster) of the NC353 so we need 200MB/s recording speed to keep the same compression quality at the increased maximal frame rate, higher recording rate that the modern SSD are capable of is very desirable.

    Fig.2. SATA routing

    Fig.2. SATA routing: a) Camera records data to the internal SSD; b) Host computer connects directly to the internal SSD; c) Camera records to the external mass storage device

    The most universal ways to attach mass storage device to the camera would be USB, SATA and PCIe. USB-2 is too slow, USB-3 is not available in Xilinx Zynq that we use. So what remains are SATA and PCIe. Both interfaces are possible to implement in Zynq, but PCIe (being faster as it uses multiple lanes) is good for the internal storage while SATA (in the form of eSATA) can be used to connect external storage devices too. We may consider adding PCIe capability to boost recording speed, but for initial implementation the SATA seems to be more universal, especially when using a trick we tested in Eyesis series of cameras for fast unloading of the recorded data.

    Routing SATA in the camera

    It is a solution similar to USB On-The-Go (similar term for SATA is used for unrelated devices), where the same connector is used to interface a smartphone to the host PC (PC is a host, a smartphone – a device) and to connect a keyboard or other device when a phone becomes a host. In contrast to the USB cables the eSATA ones always had identical connectors on both ends so nothing prevented to physically link two computers or two external drives together. As eSATA does not carry power it is safe to do, but nothing will work – two computers will not talk to each other and the storage devices will not be able to copy data between them. One of the reasons is that two signal pairs in SATA cable are uni-directional – pair A is output for the host and input for device, pair B – the opposite.

    Camera uses Vitesse (now Microsemi) VSC3304 crosspoint switch (Eyesis uses larger VSC3312) that has a very useful feature – it has reversible I/O ports, so the same physical pins can be configured as inputs or outputs, making it possible to use a single eSATA connector in both host and device mode. Additionally VSC3304 allows to change the output signal level (eSATA requires higher swing than the internal SATA) and perform analog signal correction on both inputs and outputs facilitating maintaining signal integrity between attached SATA devices.

    Aren’t SATA implementations for Xilinx Zynq already available?

    Yes and no. When starting the NC393 development I contacted Ashwin Mendon who already had SATA-2 working on Xilinx Virtex. The code is available on OpenCores under GNU GPL license. There is an article published by IEEE . The article turned out to be very useful for our work, but the code itself had to be mostly re-written – it was still for different hardware and were not able to simulate the core as it depends on Xilinx proprietary encrypted primitives – a feature not compatible with the free software simulators we use.

    Other implementations we could find (including complete commercial solution for Xilinx Zynq) have licenses not compatible with the GNU GPLv3+, and as the FPGA code is “compiled” to a single “binary” (bitstream file) it is not possible to mix free and proprietary code in the same design.

    Implementation

    The SATA host adapter is implemented for Elphel NC393 camera, 10393 system board documentation is on our wiki page. The Verilog code is hosted at GitHub, the GNU/Linux driver ahci_elphel.c is also there (it is the only hardware-specific driver file required). The repository contains a complete setup for simulation with Icarus Verilog and synthesis/implementation with Xilinx tools as a VDT (plugin for Eclipse IDE) project.

    Current limitations

    The current project was designed to be a minimal useful implementation with provisions to future enhancements. Here is the list of what is not yet done:

    • It is only SATA2 (3GHz) while the hardware is SATA3(6GHz) capable. We will definitely work on the SATA3 after we will complete migration to the new camera platform. Most of the project modules are already designed for the higher data rate.
    • No scrambling of outgoing primitives, only recognizing incoming ones. Generation of CONTp is optional by SATA standard, but we will definitely add this as it reduces EMI and we already implemented multiple hardware measures in this direction. Most likely we will need it for the CE certification.
    • No FIS-based switching for port multipliers.
    • Single command slot, and no NCQ. This functionality is optional in AHCI, but it will be added – not much is missing in the current design.
    • No power management. We will look for the best way to handle it as some of the hardware control (like DevSleep) requires i2c communication with the interface board, not directly under FPGA control. Same with the crosspoint switch.

    There is also a deviation from the AHCI standard that I first considered temporary but now think it will stay this way. AHCI specifies that a Command list structure (array of 32 8-DWORD command headers) and a 256-byte Received FIS structure are stored in the system memory. On the other hand these structures need non-paged memory, are rather small and require access from both CPU and the hardware. In x393_sata these structures are mapped to the register memory (stored in the FPGA RAM blocks) – not to the regular system memory. When working with the AHCI driver we noticed that it is even simpler to do it that way. The command tables themselves that involve more data passing form the software to device (especially PRDT – physical region descriptor tables generated from the scatter-gather lists of allocated data memory) are stored in the system memory as required and are read to the hardware by the DMA engine of the controller.

    As of today the code is still not yet cleaned up from temporary debug additions. It will all be done in the next couple weeks as we need to combine this code with the large camera-specific code – SATA controller (~6% of the FPGA resources) was developed separately from the rest of the code (~80% resources) as it makes both simulation and synthesis iterations much faster.

    Extras

    This implementation includes some additions functionality controlled by Verilog `ifdef directives. Two full block RAM primitives as used for capturing data in the controller. One of these “datascopes” captures incoming data right after 10b/8b decoder – it can store either 1024 samples of the incoming data combined of 16 bit of data plus attributes or the compact form when each 32-bit primitive is decoded and the result is a 5-bit primitive/error number. In that case 6*1024 primitives are recorded – 3 times longer than the longest FIS.

    Another 4KB memory block is used for profiling – the controller timestamps and records first 5 DWORDs of each each incoming and outgoing FIS, additionally it timestamps software writes to the specific location allowing mixed software/hardware profiling.

    This project implements run-time access to the primitive attributes using Xilinx DRP port of the GTX elements, same interface is used to programmatically change the logical values of the configuration inputs, making it significantly simpler to guess how the partially documented attributes change the device functionality. We will definitely need it when upgrading to SATA3.

    Code description

    Top connections

    The controller uses 3 differential I/O pads of the device – one input pair (RX on Fig.1) and one output pair (TX) make up a SATA port, additional dedicated input pair (CLK) provides 150MHz that synchronizes most of the controller and the transmit channel of the Zynq GTX module. In the 10393 board uses SI53338 spread-spectrum capable programmable clock to drive this input.

    Xilinx conventions tell that the top level module should instantiate the SoC Processing System PS7 (I would consider connections to the PS7 as I/O ports), so the top module does exactly that and connects to AXI ports of the actual design top module to the MAXIGP1 and SAXIHP3 ports of the PS7, IRQF2P[0] provides interrupt signal to the CPU. MAXIGP1 is one of the two 32-bit AXI ports where CPU is master – it is used for PIO access to the controller register memory (and read out debug information), SAXIHP3 is one of the 4 “high performance” 64-bit wide paths, this port is used by the controller DMA engine to transfer command tables and data to/from the device. The port numbers are selected to match ones unused in the camera specific code, other designs may have different assignments.

    Clocks and clock domains

    Current SATA2 implementation uses 4 different clock domains, some may be shared with other unrelated modules or have the same source.

    1. aclk is used in MAXIGP1 channel and part of the MAXI REGISTERS module synchronizing AXI-pointing port of the dual-port block RAM that implements controller registers. 150 MHz (maximal permitted frequency) is used, it is generated from one of the PS7 FPGA clocks
    2. hclk is used in AXI HP3 channel, DMA Control and parts of the H2D CCD FIFO (host-to-device cross clock domain FIFO ), D2H CCD FIFO and AFI ABORT modules synchronizing. 150 MHz (maximal permitted frequency) is used, same as the aclk
    3. mclk is used throughout most of the other modules of the controller except parts of the GTX, COMMA, 10b8 and input parts of the ELASTIC. For current SATA2 implementation it is 75MHz, this clock is derived from the external clock input and is not synchronous with the first two
    4. xclk – source-synchronous clock extracted from the incoming SATA data. It drives COMMA and 10b8 modules, ELASTIC allows data to cross clock boundaries by adding/removing ALIGNp primitives

    ahci_sata_layers

    The two lower layers of the stack (phy and link) that are independent of the controller system interface (AHCI) are instantiated in ahci_sata_layers.v module together with the 2 FIFO buffers for D2H (incoming) and H2D outgoing data.

    SATA PHY

    SATA PHY layer Contains the OOB (Out Of Band) state machine responsible for handling COMRESET,COMINIT and COMWAKE signals, the rest is just a wrapper for the functionality of the Xilinx GTX transceiver. This device includes both high-speed elements and some blocks that can be synthesized using FPGA fabric. Xilinx does not provide the source code for the GTX simulation module and we were not able to match the hardware operation to the documentation, so in the current design we use only those parts of the GTXE2_CHANNEL primitive that can not be replaced by the fabric. Other modules are implemented as regular Verilog code included in the x393_sata project. There is a gtx_wrap module in the design that has the same input/output ports as the primitive allowing to select which features are handled by the primitive and which – by the Verilog code without changing the rest of the design.
    The GTX primitive itself can not be simulated with the tools we use, so the simulation module was replaced, and Verilog `ifdef directive switches between the simulation model and non-free primitive for synthesis. The same approach we used earlier with other Xilinx proprietary primitives.

    Link

    Link module implements SATA link state machine, scrambling/descrambling of the data, calculates CRC for transmitted data and verifies CRC for the received one. SATA does not transmit and receive data simultaneously (only control primitives), so both CRC and scrambler modules have a single instance each providing dual functionality. This module required most troubleshooting and modifications during testing the hardware with different SSD – at some stages controller worked with some of them, but not with others.

    ahci_top

    Other modules of the design are included in the ahci_top. Of them the largest is the DMA engine shown as a separate block on the Fig.1.

    DMA

    DMA engine makes use of one of the Zynq 64-bit AXI HP ports. This channel includes FIFO buffers on the data and address subchannels (4 total) – that makes interfacing rather simple. The hard task is resetting the channels after failed communication of the controller with the device – even reloading bitsteam and resetting the FPGA would not help (actually it makes things even worse). I searched Xilinx support forum and found that similar questions where only discussed between the users, there was no authoritative recommendation from Xilinx staff. I added axi_hp_abort module that watches over the I/O transactions and keeps track of what was sent to the FIFO buffers, being able to complete transactions and drain buffers when requested.

    The DMA module reads command table, saves command data in the memory block to be later read by the FIS TRANSMIT module, it then reads the scatter-gather memory descriptors (PRDT) (supporting pre-fetch if enabled) and reads/writes the data itself combining the fragments.

    On the controller side data that comes out towards the device (H2D CCD FIFO) and coming from device(D2H CCD FIFO) needs to cross the clock boundary between hclk and mclk, and handle alignment issues. AXI HP operates in 64-bit mode, data to/from the link layer is 32-bit wide and AHCI allows alignment to the even number of bytes (16bits). When reading from the device the cross-clock domain FIFO module does it in a single step, combining 32-bit incoming DWORDs into 64-bit ones and using a barrel shifter (with 16-bit granularity) to align data to the 64-bit memory QWORDs – the AXI HP channel provides per-byte write mask that makes it rather easy. The H2D data is converted in 2 steps: First it crosses the clock domain boundary being simultaneously transformed to 32-bit with a 2-bit word mask that tells which of the two words in each DWORD are valid. Additional module WORD STUFFER operates in mclk domain and consolidates incoming sparse DWORDs into full outgoing DWORDs to be sent to the link layer.

    AHCI

    The rest of the ahci_top module is shown as AHCI block. AHCI standard specifies multiple registers and register groups that HBA has. It is intended to be used for PCI devices, but the same registers can be used even when no PCI bus is physically present. The base address is programmed differently, but the relative register addressing is still the same.

    MAXI REGISTERS

    MAXI REGISTERS module provides the register functionality and allows data to cross the clock domain boundary. The register memory is made of a dual-port block RAM module, additional block RAM (used as ROM) is pre-initialized to make each bit field of the register bank RW (read/write), RO (read only), RWC (read, write 1 to clear) or RW1 (read, write 1 to set) as specified by the AHCI. Such initialization is handled by the Python program create_ahci_registers.py that also generates ahci_localparams.vh include file that provides symbolic names for addressing register fields in Verilog code of other modules and in simulation test benches. The same file runs in the camera to allow access to the hardware registers by names.

    Each write access to the register space generates write event that crosses the clock boundary and reaches the HBA logic, it is also used to start the AHCI FSM even if it is in reset state.

    The second port of the register memory operates in mclk domain and allows register reads and writes by other AHCI submodules (FIS RECEIVE – writes registers, FIS TRANSMIT and CONTROL STATUS)

    The same module also provides access to debug registers and allows reading of the “datascope” acquired data.

    CONTROL STATUS

    The control/status module maintains “live” registers/bits that the controller need to react when they are changed by the software and react on various events in the different parts of the controller. The updated register values are written to the software accessible register bank.

    This module generates interrupt request to the processor as specified in the AHCI standard. It uses one of the interrupt lines from the FPGA to the CPU (IRQF2P) available in Zynq.

    AHCI FSM

    The AHCI state machine implements the AHCI layer using programmable sequencer. Each state traverses the following two stages: actions and conditions. The first stage triggers single-cycle pulses that are distributed to appropriate modules (currently 52 total). Some actions require just one cycle, others wait for “done” response from the destination. Conditions phase involves freezing logical conditions (now 44 total) and then going through them in the order specified in AHCI documentation. The state description for the machine is provided in the Assembler-like format inside the Python program ahci_fsm_sequence.py it generates Verilog code for the action_decoder.v and condition_mux.v modules that are instantiated in the ahci_fsm.v.

    The output listing of the FSM generator is saved to ahci_fsm_sequence.lst. Debug output registers include address of the last FSM transition, so this listing can be used to locate problems during hardware testing. It is possible to update the generated FSM sequence at run time using designated as vendor-specific registers in the controller I/O space.

    FIS RECEIVE

    The FIS RECEIVE module processes incoming FIS (DMA Setup FIS, PIO Setup FIS, D2H register FIS, Set device bits FIS, unknown FIS), updates required registers and saves them in the appropriate areas of received FIS structure. For incoming data FIS it consumes just the header DWORD and redirects the rest to the D2H CCD FIFO of the DMA module. This module also implements the word counters (PRD byte count and decrementing transfer counter), these counters are shared with the transmit channel.

    FIS TRANSMIT

    FIS TRANSMIT module recognizes the following commands received from the AHCI FSM: fetch_cmd, cfis_xmit, atapi_xmit and dx_xmit, following the prefetch condition bit. The first command (fetch_cmd) requests DMA engine to read in the command table and optionally to prefetch PRD memory descriptors. The command data is read from the DMA module memory after one of the cfis_xmit or atapi_xmit comamnds, it is then transmitted to the link layer to be sent to device. When processing the dx_xmit this module sends just the header DWORD and transfers control to the DMA engine, continuing to count PRD byte count and decrementing transfer counter.

    FPGA resources used

    According to the “report_utilization” Xilinx Vivado command, current design uses:

    • 1358 (6.91%) slices
    • 9.5 (3.58%) Block RAM tiles
    • 7 (21.88%) BUFGCTRL
    • 2 (40%) PLLE2_ADV

    The resource usage will be reduced as there are debug features not yet disabled. One of the PLLE2_ADV uses clock already available in the rest of the x393 code (150MHz for MAXIGP1 and SXAHIHP3), the other PLL that produces 75MHz transmit-synchronous clock can probably be eliminated too. Two of the block RAM tiles are capturing incoming primitives and profiling data, this functionality is not needed in the production version. More the resources may be saved if we’ll be able to use the hard-wired 10b/8b decoder, 8b/10b encoder, comma alignment and elastic buffer primitives of the Xilinx GTXE2_CHANNEL.

    Update: eliminated use of the PLLE2_ADV in the SATA controller (one left is just to generate AXI clock, it is not needed with proper setting of the PS output clock), reduced number of slices (datascope functionality preserved) to 1304 (6.64%). PLLs are valuable resource for multi-sensor camera as we keep possibility to use different sensors/clocks on each sensor port.

    Testing the hardware

    Testing with Python programs

    All the initial work with the actual hardware was done with the Python script that started with reimplementation of the same functionality used when simulating the project. Most is in x393sata.py that imports x393_vsc3304.py to control the VSC3304 crosspoint switch. This option turned out very useful for troubleshooting starting from initial testing of the SSD connection (switch can route the SSD to the desktop computer), then for verifying the OOB exchange (the only what is visible on my oscilloscope) – switch was set to connect SSD to Zynq, and use eSATA connector pins to duplicate signals between devices, so probing did not change the electrical characteristics of the active lines. Python program allowed to detect communication errors, modify GTX attributes over DRP, capture incoming data to reproduce similar conditions with the simulator. Step-by-step it was possible to receive signature FIS, then get then run the identify command. In these tests I used large area of the system memory that was reserved as a video ring buffer set up as “coherent” DMA memory. We were not able to make it really “coherent” – the command data transmitted to the device (controller reads it from the system memory as a master) often contained just zeros as the real data written by the CPU got stuck in either one of the caches or in the DDR memory controller write buffer. These errors only went away when we abandoned the use of the coherent memory allocation and switched to the stream DMA with explicit synchronization with dma_sync_*_for_cpu/dma_sync_*_for_device.

    AHCI driver for GNU/Linux

    Mikhail Karpenko is preparing post about the software driver, and as expected this development stage revealed new controller errors that were not detected with just manual launching commands through the Python program. When we mounted the SSD and started to copy gigabyte files, the controller reported some fake CRC errors. And it happened with one SSD, but not with the other. Using data capturing modules it was not so difficult to catch the conditions that caused errors and then reproduce them with the simulator – one of the last bugs detected was that link layer incorrectly handled single incoming HOLD primitives (rather unlikely condition).

    Performance results

    First performance testing turned out to be rather discouraging – ‘dd’ reported under 100 MB/s rate. At that point I added profiling code to the controller, and the data rate for the raw transfers (I tried command that involved reading of 24 of the 8KB FISes), measured from the sending of the command FIS to the receiving of the D2H register FIS confirming the transfer was 198MB/s – about 80% of the maximal for the SATA2. Profiling the higher levels of the software we noticed that there is virtually no overlap between the hardware and software operation. It is definitely possible to improve the result, but the fact that the software slowed twice the operation tells that it if the requests and their processing were done in parallel, it will consume 100% of the CPU power. Yes, there are two cores and the clock frequency can be increased (the current boards use the speed grade 2 Zynq, while the software still thinks it is speed grade 1 for compatibility with the first prototype), it still may be a big waste in the camera. So we will likely bypass the file system for sequential recording video/images and use the second partition of the SSD for raw recording, especially as we will record directly from the video buffer of the system memory, so no dealing with scatter-gather descriptors, and no need to synchronize system memory as no cache is involved. The memory controller is documented as being self-coherent, so reading the same memory while it is being written to through a different channel should cause write operation to be performed first.

    Conclusions and future plans

    We’ve achieved the useful functionality of the camera SATA controller allowing recording to the internal high capacity m.2 SSD, so all the hardware is tested and cameras can be shipped to the users. The future upgrades (including SATA3) will be released in the same way as other camera software. On the software side we will first need to upgrade our camogm recorder to reduce CPU usage during recording and provide 100% load to the SATA controller (rather easy when recording continuous memory buffer). Later (it will be more important after SATA3 implementation) we may optimize controller even more try to short-cut the video compressors outputs directly to the SATA controller, using the system memory as a buffer only when the SSD is not ready to receive data (they do take “timeouts”).

    We hope that this project will be useful for other developers who are interested in Free Software solutions and prefer the Real Verilog Code (RVC) to all those “wizards”, “black boxes” and “IP”.

    Software tools used (and not)

    Elphel designs and builds high performance cameras striving to provide our users/developers with the design freedom at every possible level. We do not use any binary-only modules or other hidden information in our designs – all what we know ourselves is posted online – usually on GitHub and Elphel Wiki. When developing FPGA, and that unfortunately still depends on proprietary tools, we limit ourselves to use only free for download tools to be exactly in the same position as many of our users. We can not make it necessary for the users (and consider it immoral) to purchase expensive tools to be able to modify the free software code for the hardware they purchased from Elphel, so no “Chipscopes” or other fancy proprietary tools were used in this project development.

    Keeping information free is a precondition, but it is not sufficient alone for many users to be able to effectively develop new functionality to the products, there needs to be ease of doing that. In the area of the FPGA design (and it is a very powerful tool resulting in high performance that is not possible with just software applications) we think of our users as smart people, but not necessarily professional FPGA developers. Like ourselves.

    Fig.3 FPGA development with VDT

    Fig.3 FPGA development with VDT

    We learned a lesson from our previous FPGA projects that depended too much on particular releases of Xilinx tools and were difficult to maintain even for ourselves. Our current code is easier to use, port and support, we tried to minimize dependence on particular tools used what we think is a better development environment. I believe that the “Lego blocks” style is not the the most productive way to develop the FPGA projects, and it is definitely not the only one possible.

    Treating HDL code similar to the software one is not less powerful paradigm, and to my opinion the development tools should not pretend to be “wizards” who know better than me what I am allowed (or not allowed) to do, but more like gentle secretaries or helpers who can take over much of routine work, remind about important events and provide some appropriate suggestions (when asked for). Such behavior is even more important if the particular activity is not the only one you do and you may come back to it after a long break. A good IDE should be like that – help you to navigate the code, catch problems early, be useful with default settings but provide capabilities to fine tune the functionality according to the personal preferences. It is also important to provide familiar environment, this is why we use the same Eclipse IDE for Verilog, Python, C/C++ and Java and more. All our projects come with the initial project settings files that can be imported in this IDE (supplemented by the appropriate plugins) so you can immediately start development from the point we currently left it.

    For FPGA development Elphel provides VDT – a powerful tool that includes deep Verilog support and integrates free software simulator Icarus Verilog with the Github repository and a popular GTKWave for visualizing simulation results. It comes with the precofigured support of FPGA vendors proprietary synthesis and implementation tools and allows addition of other tools without requirement to modify the plugin code. The SATA project uses Xilinx Vivado command line tools (not Vivado GUI), support for several other FPGA tools is also available.

    by Andrey Filippov at March 12, 2016 11:14 PM

    ZeptoBARS

    GD32F103CBT6 - Cortex-M3 with serial flash : weekend die-shot

    Giga Devices GD32F103CBT6 really surprised us:



    Giga Devices was a serial flash manufacturer for quite some time. When they launched their ARM Cortex M3 lineup (with some level of binary compatibility to STM32) - instead of going conventional route of making numerous dies with different flash and SRAM sizes they went for SRAM&logic die and separate serial flash die. How this could work fast enough? Keep reading :-) At least ESP8266 already taught us that executing code from serial flash and reaching acceptable speed is not impossible.

    Use of serial flash allows Giga Devices to increase maximum flash size in their microcontrollers quite a bit (currently they have up to 3MiB) and to save quite a bit on ARM licensing fees (if they are paying "per die design").


    Die has 110 pads, 9 of which are used by a flash die. GD32F103CBT6 is in TQFP48 package - which again suggests that this die is universal and also used in higher pin count models. Die size 2889x3039 µm.

    Logo:


    ADC capacitor bank:


    After etching to poly level we clearly see that there is no flash on the die:


    SRAM sizes are 32KiB in each largest block (128 KiB total) - stores code, which means first 128KiB could be accessed faster than typical flash. GD32 chips with 20Kb of SRAM or less have no more than 128KiB of flash, so all flash content is served from SRAM. This might also mean that startup time is slower than one would expect. With this SRAM mirroring it is not surprising that GD32 is beating STM32 in performance even on the same frequency and loosing in idle & sleep power consumption. Consumption at full load is lower than STM32 due to better (smaller) manufacturing technology.

    2 smaller blocks are 10KiB each and are likely to be user-accessible SRAM.
    4 smallest blocks closest to the synthesized logic are 512B each.

    SRAM has cell size 2.04 µm², which is ~110nm. Scale 1px = 57nm:


    Standard cells:


    Low power standard cells:


    Flash die:

    Flash die size: 1565x1378 µm.

    PS. Thanks for the chips go to dongs from irc.

    March 12, 2016 12:00 PM

    March 08, 2016

    Harald Welte

    Linaro Connect BKK16 Keynote on GPL Compliance

    Today I had the pleasure of co-presenting with Shane Coughlan the Linaro Connect BKK16 Keynote on GPL compliance about GPL compliance.

    The main topics addressed by this presentation are:

    • Brief history about GPL enforcement and how it has impacted the industry
    • Ultimate Goal of GPL enforcement is compliance
    • The license is not an end in itself, but rather to facilitate collaborative development
    • GPL compliance should be more engineering and business driven, not so much legal (compliance) driven.

    The video recording is available at https://www.youtube.com/watch?v=b4Bli8h0V-Q

    Slides are available at http://git.gnumonks.org/index.html/laforge-slides/plain/2016/linaroconnect/compliance.html

    The video of a corresponding interview is available from https://www.youtube.com/watch?v=I6IgjCyO-iQ

    by Harald Welte at March 08, 2016 11:00 PM

    March 03, 2016

    Free Electrons

    Free Electrons at the Embedded Linux Conference 2016

    Like every year for about 10 years, the entire Free Electrons engineering team will participate to the next Embedded Linux Conference, taking place on April 4-6 in San Diego, California. For us, participating to such conferences is very important, as it allows to remain up to date with the latest developments in the embedded Linux world, create contacts with other members of the embedded Linux community, and meet the community members we already know and work with on a daily basis via the mailing lists or IRC.

    Embedded Linux Conference 2016

    Over the years, our engineering team has grown, and with the arrival of two more engineers on March 14, our engineering team now gathers 9 persons, all of whom are going to participate to the Embedded Linux Conference.

    As usual, in addition to attending, we also proposed a number of talks, and some of them have been accepted and are visible in the conference schedule:

    As usual, our talks are centered around our areas of expertise: hardware support in the Linux kernel, especially for ARM platforms, and build system related topics (Buildroot, Yocto, autotools).

    We are looking forward to attending this event, and see many other talks from various speakers: the proposed schedule contains a wide range of topics, many of which look really interesting!

    by Thomas Petazzoni at March 03, 2016 01:49 PM

    February 24, 2016

    Harald Welte

    Report from the VMware GPL court hearing

    Today, I took some time off to attend the court hearing in the GPL violation/infringement case that Christoph Hellwig has brought against VMware.

    I am not in any way legally involved in the lawsuit. However, as a fellow (former) Linux kernel developer myself, and a long-term Free Software community member who strongly believes in the copyleft model, I of course am very interested in this case - and of course in an outcome in favor of the plaintiff. Nevertheless, the below report tries to provide an un-biased account of what happened at the hearing today, and does not contain my own opinions on the matter. I can always write another blog post about that :)

    I blogged about this case before briefly, and there is a lot of information publicly discussed about the case, including the information published by the Software Freedom Conservancy (see the link above, the announcement and the associated FAQ.

    Still, let's quickly summarize the facts:

    • VMware is using parts of the Linux kernel in their proprietary ESXi product, including the entire SCSI mid-layer, USB support, radix tree and many, many device drivers.
    • as is generally known, Linux is licensed under GNU GPLv2, a copyleft-style license.
    • VMware has modified all the code they took from the Linux kernel and integrated them into something they call vmklinux.
    • VMware has modified their proprietary virtualization OS kernel vmkernel with specific API/symbol to interact with vmklinux
    • at least in earlier versions of ESXi, virtually any block device access has to go through vmklinux and thus the portions of Linux they took
    • vmklinux and vmkernel are dynamically linked object files that are linked together at run-time
    • the Linux code they took runs in the same execution context (address space, stack, control flow) like the vmkernel.

    Ok, now enter the court hearing of today.

    Christoph Hellwig was represented by his two German Lawyers, Dr. Till Jaeger and Dr. Miriam Ballhausen. VMware was represented by three German lawyers lead by Matthias Koch, as well as a US attorney, Michael Jacobs (by means of two simultaneous interpreters). There were also several members of the in-house US legal team of VMware present, but not formally representing the defendant in court.

    As is unusual for copyright disputes, there was quite some audience following the court. Next to the VMware entourage, there were also a couple of fellow Linux kernel developers as well as some German IT press representatives following the hearing.

    General Introduction of the presiding judge

    After some formalities (like the question whether or not a ',' is missing after the "Inc." in the way it is phrased in the lawsuit), the presiding judge started with some general remarks

    • the court is well aware of the public (and even international public) interest in this case
    • the court understands there are novel fundamental legal questions raised that no court - at least no German court - had so far to decide upon.
    • the court also is well aware that the judges on the panel are not technical experts and thus not well-versed in software development or computer science. Rather, they are a court specialized on all sorts of copyright matters, not particularly related to software.
    • the court further understands that Linux is a collaborative, community-developed operating system, and that the development process is incremental and involves many authors.
    • the court understands there is a lot of discussion about interfaces between different programs or parts of a program, and that there are a variety of different definitions and many interpretations of what interfaces are

    Presentation about the courts understanding of the subject matter

    The presiding judge continued to explain what was their understanding of the subject matter. They understood VMware ESXi serves to virtualize a computer hardware in order to run multiple copies of the same or of different versions of operating systems on it. They also understand that vmkernel is at the core of that virtualization system, and that it contains something called vmkapi which is an interface towards Linux device drivers.

    However, they misunderstood that this case was somehow an interface between a Linux guest OS being virtualized on top of vmkernel. It took both defendant and plaintiff some time to illustrate that in fact this is not the subject of the lawsuit, and that you can still have portions of Linux running linked into vmkernel while exclusively only virtualizing Windows guests on top of vmkernel.

    The court went on to share their understanding of the GPLv2 and its underlying copyleft principle, that it is not about abandoning the authors' rights but to the contrary exercising copyright. They understood the license has implications on derivative works and demonstrated that they had been working with both the German translation a well as the English language original text of GPLv2. At least I was sort-of impressed by the way they grasped it - much better than some of the other courts that I had to deal with in the various cases I was bringing forward during my gpl-violations.org work before.

    They also illustrated that they understood that Christoph Hellwig has been developing parts of the Linux kernel, and that modified parts of Linux were now being used in some form in VMware ESXi.

    After this general introduction, there was the question of whether or not both parties would still want to settle before going further. The court already expected that this would be very unlikely, as it understood that the dispute serves to resolve fundamental legal question, and there is hardly any compromise in the middle between using or not using the Linux code, or between licensing vmkernel under a GPL compatible license or not. And as expected, there was no indication from either side that they could see an out-of-court settlement of the dispute at this point.

    Right to sue / sufficient copyrighted works of the plaintiff

    There was quite some debate about the question whether or not the plaintiff has shown that he actually holds a sufficient amount of copyrighted materials.

    The question here is not, whether Christoph has sufficient copyrightable contributions on Linux as a whole, but for the matter of this legal case it is relevant which of his copyrighted works end up in the disputed product VMware ESXi.

    Due to the nature of the development process where lots of developers make intermittent and incremental changes, it is not as straight-forward to demonstrate this, as one would hope. You cannot simply print an entire C file from the source code and mark large portions as being written by Christoph himself. Rather, lines have been edited again and again, were shifted, re-structured, re-factored. For a non-developer like the judges, it is therefore not obvious to decide on this question.

    This situation is used by the VMware defense in claiming that overall, they could only find very few functions that could be attributed to Christoph, and that this may altogether be only 1% of the Linux code they use in VMware ESXi.

    The court recognized this as difficult, as in German copyright law there is the concept of fading. If the original work by one author has been edited to an extent that it is barely recognizable, his original work has faded and so have his rights. The court did not state whether it believed that this has happened. To the contrary, the indicated that it may very well be that only very few lines of code can actually make a significant impact on the work as a whole. However, it is problematic for them to decide, as they don't understand source code and software development.

    So if (after further briefs from both sides and deliberation of the court) this is still an open question, it might very well be the case that the court would request a techncial expert report to clarify this to the court.

    Are vmklinux + vmkernel one program/work or multiple programs/works?

    Finally, there was some deliberation about the very key question of whether or not vmkernel and vmklinux were separate programs / works or one program / work in the sense of copyright law. Unfortunately only the very surface of this topic could be touched in the hearing, and the actual technical and legal arguments of both sides could not be heard.

    The court clarified that if vmkernel and vmklinux would be considered as one program, then indeed their use outside of the terms of the GPL would be an intrusion into the rights of the plaintiff.

    The difficulty is how to actually venture into the legal implications of certain technical software architecture, when the people involved have no technical knowledge on operating system theory, system-level software development and compilers/linkers/toolchains.

    A lot is thus left to how good and 'believable' the parties can present their case. It was very clear from the VMware side that they wanted to down-play the role and proportion of vmkernel and its Linux heritage. At times their lawyers made statements like linux is this small yellow box in the left bottom corner (of our diagram). So of course already the diagrams are drawn in a way to twist the facts according to their view on reality.

    Summary

    • The court seems very much interested in the case and wants to understand the details
    • The court recognizes the general importance of the case and the public interest in it
    • There were some fundamental misunderstandings on the technical architecture of the software under dispute that could be clarified
    • There are actually not that many facts that are disputed between both sides, except the (key, and difficult) questions on
      • does Christoph hold sufficient rights on the code to bring forward the legal case?
      • are vmkernel and vmklinux one work or two separate works?

    The remainder of this dispute will thus be centered on the latter two questions - whether in this court or in any higher courts that may have to re-visit this subject after either of the parties takes this further, if the outcome is not in their favor.

    In terms of next steps,

    • both parties have until April 15, 2016 to file further briefs to follow-up the discussions in the hearing today
    • the court scheduled May 19, 2016 as date of promulgation. However, this would of course only hold true if the court would reach a clear decision based on the briefs by then. If there is a need for an expert, or any witnesses need to be called, then it is likely there will be further hearings and no verdict will be reached by then.

    by Harald Welte at February 24, 2016 11:00 PM

    February 23, 2016

    Harald Welte

    Software under OSA Public License is neither Open Source nor Free Software

    It seems my recent concerns on the OpenAirInterface re-licensing were not unjustified.

    I contacted various legal experts on Free Software legal community about this, and the response was unanimous: In all feedback I received, the general opinion was that software under the OSA Public License V1.0 is neither Free Software nor Open Source Software.

    The rational is, that it does not fulfill the criteria of

    • the FSF Free Software definition, as the license does not fulfill freedom 0: The freedom to run the program as you wish, for any purpose (which obviously includes commercial use)
    • the Open Source Initiatives Open Source Definition, as the license must not discriminate against fields of endeavor, such as commercial use.
    • the Debian Free Software Guidelines, as the DFSG also require no discrimination against fields of endeavor, such as commercial use.

    I think we as the community need to be very clear about this. We should not easily tolerate that people put software under restrictive licenses but still call that software open source. This creates a bad impression to those not familiar with the culture and spirit of both Free Software and Open Source. It creates the impression that people can call something Open Source but then still ask royalties for it, if used commercially.

    It is a shame that entities like Eurecom and the OpenAirInterface Software Association are open-washing their software by calling it Open Source when in fact it isn't. This attitude frankly makes me sick.

    That's just like green-washing when companies like BP are claiming they're now an environmental friendly company just because they put some solar panels on the roof of some building.

    by Harald Welte at February 23, 2016 11:00 PM

    Bunnie Studios

    The Story Behind the Cover for The Essential Guide to Electronics in Shenzhen

    First, I want to say wow! I did not expect such a response to this book. When preparing for the crowdfunding campaign, I modeled several scenarios, and none of them predicted an outcome like this.

    The Internet has provided fairly positive feedback on the cover of the book. I’m genuinely flattered that people like how it turned out. There’s actually an interesting story behind the origins of the book cover, which is the topic of this post.

    It starts with part of a blog post series I did a while back, “The Factory Floor, Part 3 of 4: Industrial Design for Startups”. In that post, I outline a methodology for factory-aware design, and I applied these methods when designing my book cover. In particular, step 3 & 4 read:

    3. Visit the facility, and take note of what is actually running down the production lines. … Practice makes perfect, and from the operators to the engineers they will do a better job of executing things they are doing on a daily basis than reaching deep and exercising an arcane capability.

    4. Re-evaluate the design based on a new understanding of what’s possible, and iterate.

    My original cover design was going to be fairly conventional – your typical cardboard laminated in four color printing, or perhaps even a soft cover, and the illustration was to be done by the same fellow who did the cute bunny pictures that preface each chapter, Miran Lipovača.

    But, as a matter of practicing what I preach, I made a visit to the printing factory to see what was running down its lines. They had all manners of processes going on in the factory, from spine stitching to die cutting and lamination.


    Chibitronics’ Circuit Sticker Sketchbook is also printed at this factory

    One process in particular caught my eye – in the back, there was a room full of men using belt sanders with varying grits of sand paper to work the edges of books until they were silky smooth. Next to that was a hot foil transfer machine – through heat and pressure, it can apply a gold (or any other color) foil to the surface of paper. In this case, they were gilding the edges of books, in a style similar to that found on fancy bibles and prayer books. They could also use the same process to do a foil deboss on cardboard.


    Beltsanding the edges of a stack of books until they are silky smooth


    Closeup of the hot foil transfer mechanism


    Stacks of books with gleaming, gilded edges

    This is when I got the idea for the cover. These gilded books looked beautiful – and because the process is done in-house, I knew I could get it for a really good price. So, I went back to the drawing board and thought about what would look good using this process. The first idea was to take the bunny picture, and adapt it for the gold foil process. Unfortunately, the bunny illustrations relied heavily upon halftone grays, something which wouldn’t translate well into a gold foil process. Someone else suggested that perhaps I should do a map of China, with Shenzhen marked and some pictures of components around it. I didn’t like it for a number of reasons, the first one being the headache of securing the copyright to a decent map of China that was both geographically accurate and politically correct.

    So I did a Google image search for “gold leaf covers” just to see what’s out there. The typical motif I observed was some kind of filigree, typically with at least left/right symmetry, if not also up/down symmetry.

    I thought maybe I’d go and fire up Adobe Illustrator and start sketching some filigree patterns, but quickly gave up on that idea – it was a lot of work, and I’m not entirely comfortable with that tool. Then it hit upon me that individual PCB layers have the same sort of intricacy as a filligree – and I live and breathe PCB design.

    So, I started up my favorite PCB design package, Altium. I tried playing around a bit with the polygon fill function, using its hashing feature and adjusting the design rules to see if I couldn’t make a decent filigree with it. The effect seemed reasonable, especially when I used a fairly coarse fill and an additional design rule that caused polygon fills to keep a wide berth around any traces.

    Then I had to come up with some circuitry to fill the cover. I looked at a few of my circuit boards, and in reality, few practical circuits had the extreme level of symmetry I was looking for. So I went ahead and cocked up a fake circuit on the fly. I made a QFN footprint based on fictional design rules that would look good, and sorted through my library of connector footprints for ones that had large enough pads to print reasonably well using the foil transfer process. I found a 2.4GHz antenna and some large-ish connectors.

    I then decided upon a theme – generally, I wanted the book to go from RF on the bottom to digital on the top. So I started by drawing the outline of an A5 page, and putting a couple lines of symmetry down. In the lower left, I placed the 2.4 GHz antenna, and then coupled it to a QFN in a semi-realistic fashion, throwing a couple of capacitors in for effect. I added an SMA connector that spanned the central symmetry line, and then an HRS DF-11 connector footprint above it. I decided in the RF section I’d make extensive use of arcs in the routing, calling upon a motif quite common in RF design and visually distinct from digital routing. Next I added a SATA connector off to the middle edge, and routed a set of differential pairs to the TX/RX pads, to which I applied the trace length equalization feature of the PCB tool to make them wavy – just for added aesthetic effect.

    Then I started from the top left and designed the digital section. Nothing says “old school digital” to me louder than a DB-9 connector (and yes, you pedants, it’s technically a DE-9, but in my heart it will always be a DB-9), so I plopped one of those down up top. I decided I’d spice things up a bit by throwing series termination resistors between the connector and a fake QFN IC; yes, in reality, not all pins would have these, but I thought it looked more aesthetic to put it on all the pins. Then, I routed signals from the QFN as a bus, this time using 45 degree angles, to a 14-pin JTAG connector which I placed in the heart of the book. Everything starts and ends with the JTAG connector these days, so why not?

    The design now occupied just the left half of the board. I copied it, flipped it, and pasted it to create a perfect 2-fold symmetry around the vertical axis.

    Around all of this, I put a border with fiducials and gutters, the same as you would find in a PCB destined for production in an automated SMT line. You’ll notice I break symmetry by making the top right fiducial a square, not a circle; this is a hallmark feature of fiducials, since their purpose is to both align the vision recognition systems and determine if the PCB has been loaded into the machine correctly.

    Finally, I added the book title and author using Altium’s TrueType string facility, and ran an automated fill of the empty space to create the filigree.

    I actually designed the whole cover while I was on the long flight from Hong Kong to Amsterdam for 32C3. I find that airplane flights are excellent for doing PCB routing and design work like this, free of any distractions from the Internet. As a bonus, every now and then someone comes along and feeds you and tops up your glass of wine, allowing your creative streak to be unbroken by concerns about hunger or sobriety.

    When viewed in black and white, the book cover honestly looks a little “meh” – when I first saw it, I was thought, “well, at least maybe the geeks will appreciate it”. But after seeing the faux-linen with gold foil transfer sample, I knew this was the design I would run with for production.

    The next difficult challenge was to not paint legs on the metaphorical snake. As an engineer, I disliked how over-simplified the design was. There really should be bypass capacitors around the digital components. And SATA requires series DC blocking caps. But I had to let all that go, set it aside, and stop looking at it as a design, and let it live its own life as the cover of a book.

    And so there you have it – the story behind perhaps the only book cover designed using Altium (if you have a gerber viewer, you can check out the gerber files). The design went from a .PcbDoc file, to a .DXF, to .AI, and finally placed in a .INDD – not your typical progression of file formats, but in the end, it was fun and worthwhile figuring it all out.

    Thanks again to everyone who helped promote and fund my book. I’m really excited to get started on the print run. The problem I’m facing now is I don’t know how many to print. Originally, I was fairly certain no matter what, I would just barely hit the minimum order quantity (MOQ) of 1,000 books. Now that the campaign has blown past that, I have to wait until the campaign finishes in 23 days before I know what to put on the purchase order to the manufacturer. And, shameless plug – if you’re interested in the book, it’s $5 cheaper if you back during the campaign, so consider getting your order in before the prices go up.

    by bunnie at February 23, 2016 06:07 PM

    February 22, 2016

    Bunnie Studios

    Name that Ware, February 2016

    The Ware for February 2016 is shown below.

    I couldn’t bring myself to blemish this beautiful ware by pixelating all of the part numbers necessary to make this month’s game a real challenge. Instead, I just relied upon a strategic cropping to remove the make and model number from the lower left corner of the board.

    Remember the TMS4464? Yah, back when TI’s thing was making DRAM, not voltage regulators, and when Foxconn made connectors, not iPhones. Somewhere along the way, some business guy coined the term “pivot” to describe such changes in business models.

    Thanks to Michael Steil for sharing this beautiful piece of history with me at 32C3!

    by bunnie at February 22, 2016 08:02 AM

    Winner, Name that Ware January 2016

    The Ware for January 2016 was a TPI model 342 water resistant, dual-input type K&J thermocouple thermometer. Picking a winner was tough. Eric Hill was extremely close on guessing the model number — probably the only difference between the TPI 343 and the 342 is a firmware change and perhaps the button that lets you pick between K/J type thermocouples, neither of which would be obvious from the image shown.

    However, I do have to give kudos to CzajNick for pointing out that the MCU in this is a 4-bit microcontroller. Holy shit, I didn’t know they made those anymore, much less be useful for anything beyond a calculator. This is probably the only functional 4-bit machine that I have in my lab. All of a sudden this thermometer got a little bit cooler in my mind. He also correctly identified the ware as some type of double-input thermocouple thermometer in the course of his analysis.

    Despite not citing a specific make/model, I really appreciated the analysis, especially the factoid about this having a 4-bit microcontroller, so I’ll declare CzajNick the winner. Congrats and email me for your prize!

    Also, I’ll have to say, after tearing apart numerous pieces of shoddy Chinese test equipment to fix stupid problems in them, it was a real sight for sore eyes to see such a clean design with high quality, brand-name components. I guess this is 90’s-vintage Korean engineering for you — a foreshadowing of the smartphone onslaught to come out of the same region a decade later.

    by bunnie at February 22, 2016 08:02 AM

    February 20, 2016

    Harald Welte

    Osmocom.org migrating to redmine

    In 2008, we started bs11-abis, which was shortly after renamed to OpenBSC. At the time it seemed like a good idea to use trac as the project management system, to have a wiki and an issue tracker.

    When further Osmocom projects like OsmocomBB, OsmocomTETRA etc. came around, we simply replicated that infrastructure: Another trac instance with the same theme, and a shared password file.

    The problem with this (and possibly the way we used it) is:

    • it doesn't scale, as creating projects is manual, requires a sysadmin and is time-consuming. This meant e.g. SIMtrace was just a wiki page in the OsmocomBB trac installation + associated http redirect, causing some confusion.
    • issues can not easily be moved from one project to another, or have cross-project relationships (like, depend on an issue in another project)
    • we had to use an external planet in order to aggregate the blog of each of the trac instances
    • user account management the way we did it required shell access to the machine, meaning user account applications got dropped due to the effort involved. My apologies for that.

    Especially the lack of being able to move pages and tickets between trac's has resulted in a suboptimal use of the tools. If we first write code as part of OpenBSC and then move it to libosmocore, the associated issues + wiki pages should be moved to a new project.

    At the same time, for the last 5 years we've been successfully using redmine inside sysmocom to keep track of many dozens of internal projects.

    So now, finally, we (zecke, tnt, myself) have taken up the task to migrate the osmocom.org projects into redmine. You can see the current status at http://projects.osmocom.org/. We could create a more comprehensive project hierarchy, and give libosmocore, SIMtrace, OsmoSGSN and many others their own project.

    Thanks to zecke for taking care of the installation/sysadmin part and the initial conversion!

    Unfortunately the conversion from trac to redmine wiki syntax (and structure) was not as automatic and straight-forward as one would have hoped. But after spending one entire day going through the most important wiki pages, things are looking much better now. As a side effect, I have had a more comprehensive look into the history of all of our projects than ever before :)

    Still, a lot of clean-up and improvement is needed until I'm happy, particularly splitting the OpenBSC wiki into separate OsmoBSC, OsmoNITB, OsmoBTS, OsmoPCU and OsmoSGSN wiki's is probably still going to take some time.

    If you would like to help out, feel free to register an account on projects.osmocom.org (if you don't already have one from the old trac projects) and mail me for write access to the project(s) of your choice.

    Possible tasks include

    • putting pages into a more hierarchic structure (there's a parent/child relationship in redmine wikis)
    • fixing broken links due to page renames / wiki page moves
    • creating a new redmine 'Project' for your favorite tool that has a git repo on http://git.osmocom.org/ and writing some (at least initial) documentation about it.

    You don't need to be a software developer for that!

    by Harald Welte at February 20, 2016 11:00 PM

    February 19, 2016

    Harald Welte

    Some update on recent OsmoBTS changes

    After quite some time of gradual bug fixing and improvement, there have been quite some significant changes being made in OsmoBTS over the last months.

    Just a quick reminder: In Fall 2015 we finally merged the long-pending L1SAP changes originally developed by Jolly, introducing a new intermediate common interface between the generic part of OsmoBTS, and the hardware/PHY specific part. This enabled a clean structure between osmo-bts-sysmo (what we use on the sysmoBTS) and osmo-bts-trx (what people with general-purpose SDR hardware use).

    The L1SAP changes had some fall-out that needed to be fixed, not a big surprise with any change that big.

    More recently however, three larger changes were introduced:

    proper Multi-TRX support

    Based on the above phy_link/phy_instance infrastructure, one can map each phy_instance to one TRX by means of the VTY / configuration file.

    The core of OsmoBTS now supports any number of TRXs, leading to flexible Multi-TRX support.

    OCTPHY support

    A Canadian company called Octasic has been developing a custom GSM PHY for their custom multi-core DSP architecture (OCTDSP). Rather than re-inventing the wheel for everything on top of the PHY, they chose to integrate OsmoBTS on top of it. I've been working at sysmocom on integrating their initial code into OsmoBTS, rendering a new osmo-bts-octphy backend.

    This back-end has also recently been ported to the phy_link/phy_instance API and is Multi-TRX ready. You can both run multiple TRX in one DSP, as well as have multiple DSPs in one BTS, paving the road for scalability.

    osmo-bts-octphy is now part of OsmoBTS master.

    Corresponding changes to OsmoPCU (for full GPRS support on OCTPHY) are currently been worked on by Max at sysmocom.

    Litecell 1.5 PHY support

    Another Canadian company (Nutaq/Nuran) has been building a new BTS called Litecell 1.5. They also implemented OsmoBTS support, based on the osmo-bts-sysmo code. We've been able to integrate that code with the above-mentioned phy_link/phy_interface in order to support the MultiTRX capability of this hardware.

    Litecell 1.5 MultiTRX capability has also been integrated with OsmoPCU.

    osmo-bts-litecell15 is now part of OsmoBTS master.

    Summary

    • 2016 starts as the OsmoBTS year of MultiTRX.
    • 2016 also starts as a year of many more hardware choices for OsmoBTS
    • we see more commercial adoption of OsmoBTS outside of the traditional options of sysmocom and Fairwaves

    by Harald Welte at February 19, 2016 11:00 PM

    February 18, 2016

    Free Electrons

    Free Electrons speaking at the Linux Collaboration Summit

    Free Electrons engineers are regular speakers at the Embedded Linux Conference and Embedded Linux Conference Europe events from the Linux Foundation, to which our entire engineering team participates each year.

    In 2016, for the first time, we will also be speaking at the Collaboration Summit, an invitation-only event where, as the Linux Foundation presents it, “the world’s thought leaders in open source software and collaborative development convene to share best practices and learn how to manage the largest shared technology investments of our time”.

    Collaboration Summit 2016

    This event will take place on March 29-31 in Lake Tahoe, California, and the event schedule has been published recently. Free Electrons CTO Thomas Petazzoni will be giving a talk

    Upstreaming hardware support in the Linux kernel: why and how?, during which we will share our experience working with HW manufacturers to bring the support for their hardware to the upstream Linux kernel, discuss the benefits of upstreaming, and best practices to work with upstream.

    With a small team of engineers, Free Electrons has merged over the last few years thousands of patches in the official Linux kernel, and has several of its engineers having maintainer positions in the Linux kernel community. We are happy to take the opportunity of the Collaboration Summit to share some of our experience, and hopefully encourage and help other companies to participate upstream.

    by Thomas Petazzoni at February 18, 2016 04:16 PM

    February 15, 2016

    Free Electrons

    Initial support for ARM64 Marvell Armada 7K/8K platform

    Two weeks ago, we submitted the initial support for the Marvell Armada 3700, which was the first ARM64 platform that Free Electrons engineers contributed to the upstream Linux kernel.

    Today, we submitted initial support for another Marvell ARM64 platform, the Armada 7K and Armada 8K platform. Compared to the Armada 3700, the Armada 7K and 8K are much more on the high-end side: they use a dual Cortex-A72 or a quad Cortex-A72, as opposed to the Cortex-A53 for the Armada 3700.

    Marvell Armada 7KMarvell Armada 8K

    The Armada 7K and 8K also use a fairly unique architecture, internally they are composed of several components:

    • One AP (Application Processor), which contains the processor itself and a few core hardware blocks. The AP used in the Armada 7K and 8K is called AP806, and is available in two configurations: dual Cortex-A72 and quad Cortex-A72.
    • One or two CP (Communication Processor), which contain most of the I/O interfaces (SATA, PCIe, Ethernet, etc.). The 7K family chips have one CP, while the 8K family chips integrate two CPs, providing two times the number of I/O interfaces available in the CP. The CP used in the 7K and 8K is called CP110.

    All in all, this gives the following combinations:

    • Armada 7020, which is a dual Cortex-A72 with one CP
    • Armada 7040, which is a quad Cortex-A72 with one CP
    • Armada 8020, which is a dual Cortex-A72 with two CPs
    • Armada 8040, which is a quad Cortex-A72 with two CPs

    So far, we submitted initial support only for the AP806 part of the chip, with the following patch series:

    We will continue to submit more and more patches to support other features of the Armada 7K and 8K processors in the near future.

    by Thomas Petazzoni at February 15, 2016 11:02 AM

    Factory flashing with U-Boot and fastboot on Freescale i.MX6

    Introduction

    For one of our customers building a product based on i.MX6 with a fairly low-volume, we had to design a mechanism to perform the factory flashing of each product. The goal is to be able to take a freshly produced device from the state of a brick to a state where it has a working embedded Linux system flashed on it. This specific product is using an eMMC as its main storage, and our solution only needs a USB connection with the platform, which makes it a lot simpler than solutions based on network (TFTP, NFS, etc.).

    In order to achieve this goal, we have combined the imx-usb-loader tool with the fastboot support in U-Boot and some scripting. Thanks to this combination of a tool, running a single script is sufficient to perform the factory flashing, or even restore an already flashed device back to a known state.

    The overall flow of our solution, executed by a shell script, is:

    1. imx-usb-loader pushes over USB a U-Boot bootloader into the i.MX6 RAM, and runs it;
    2. This U-Boot automatically enters fastboot mode;
    3. Using the fastboot protocol and its support in U-Boot, we send and flash each part of the system: partition table, bootloader, bootloader environment and root filesystem (which contains the kernel image).
    The SECO uQ7 i.MX6 platform used for our project.

    The SECO uQ7 i.MX6 platform used for our project.

    imx-usb-loader

    imx-usb-loader is a tool written by Boundary Devices that leverages the Serial Download Procotol (SDP) available in Freescale i.MX5/i.MX6 processors. Implemented in the ROM code of the Freescale SoCs, this protocol allows to send some code over USB or UART to a Freescale processor, even on a platform that has nothing flashed (no bootloader, no operating system). It is therefore a very handy tool to recover i.MX6 platforms, or as an initial step for factory flashing: you can send a U-Boot image over USB and have it run on your platform.

    This tool already existed, we only created a package for it in the Buildroot build system, since Buildroot is used for this particular project.

    Fastboot

    Fastboot is a protocol originally created for Android, which is used primarily to modify the flash filesystem via a USB connection from a host computer. Most Android systems run a bootloader that implements the fastboot protocol, and therefore can be reflashed from a host computer running the corresponding fastboot tool. It sounded like a good candidate for the second step of our factory flashing process, to actually flash the different parts of our system.

    Setting up fastboot on the device side

    The well known U-Boot bootloader has limited support for this protocol:

    The fastboot documentation in U-Boot can be found in the source code, in the doc/README.android-fastboot file. A description of the available fastboot options in U-Boot can be found in this documentation as well as examples. This gives us the device side of the protocol.

    In order to make fastboot work in U-Boot, we modified the board configuration file to add the following configuration options:

    #define CONFIG_CMD_FASTBOOT
    #define CONFIG_USB_FASTBOOT_BUF_ADDR       CONFIG_SYS_LOAD_ADDR
    #define CONFIG_USB_FASTBOOT_BUF_SIZE          0x10000000
    #define CONFIG_FASTBOOT_FLASH
    #define CONFIG_FASTBOOT_FLASH_MMC_DEV    0
    

    Other options have to be selected, depending on the platform to fullfil the fastboot dependencies, such as USB Gadget support, GPT partition support, partitions UUID support or the USB download gadget. They aren’t explicitly defined anywhere, but have to be enabled for the build to succeed.

    You can find the patch enabling fastboot on the Seco MX6Q uQ7 here: 0002-secomx6quq7-enable-fastboot.patch.

    U-Boot enters the fastboot mode on demand: it has to be explicitly started from the U-Boot command line:

    U-Boot> fastboot
    

    From now on, U-Boot waits over USB for the host computer to send fastboot commands.

    Using fastboot on the host computer side

    Fastboot needs a user-space program on the host computer side to talk to the board. This tool can be found in the Android SDK and is often available through packages in many Linux distributions. However, to make things easier and like we did for imx-usb-loader, we sent a patch to add the Android tools such as fastboot and adb to the Buildroot build system. As of this writing, our patch is still waiting to be applied by the Buildroot maintainers.

    Thanks to this, we can use the fastboot tool to list the available fastboot devices connected:

    # fastboot devices
    

    Flashing eMMC partitions

    For its flashing feature, fastboot identifies the different parts of the system by names. U-Boot maps those names to the name of GPT partitions, so your eMMC normally requires to be partitioned using a GPT partition table and not an old MBR partition table. For example, provided your eMMC has a GPT partition called rootfs, you can do:

    # fastboot flash rootfs rootfs.ext4
    

    To reflash the contents of the rootfs partition with the rootfs.ext4 image.

    However, while using GPT partitioning is fine in most cases, i.MX6 has a constraint that the bootloader needs to be at a specific location on the eMMC that conflicts with the location of the GPT partition table.

    To work around this problem, we patched U-Boot to allow the fastboot flash command to use an absolute offset in the eMMC instead of a partition name. Instead of displaying an error if a partition does not exists, fastboot tries to use the name as an absolute offset. This allowed us to use MBR partitions and to flash at defined offset our images, including U-Boot. For example, to flash U-Boot, we use:

    # fastboot flash 0x400 u-boot.imx
    

    The patch adding this work around in U-Boot can be found at 0001-fastboot-allow-to-flash-at-a-given-address.patch. We are working on implementing a better solution that can potentially be accepted upstream.

    Automatically starting fastboot

    The fastboot command must be explicitly called from the U-Boot prompt in order to enter fastboot mode. This is an issue for our use case, because the flashing process can’t be fully automated and required a human interaction. Using imx-usb-loader, we want to send a U-Boot image that automatically enters fastmode mode.

    To achieve this, we modified the U-Boot configuration, to start the fastboot command at boot time:

    #define CONFIG_BOOTCOMMAND "fastboot"
    #define CONFIG_BOOTDELAY 0
    

    Of course, this configuration is only used for the U-Boot sent using imx-usb-loader. The final U-Boot flashed on the device will not have the same configuration. To distinguish the two images, we named the U-Boot image dedicated to fastboot uboot_DO_NOT_TOUCH.

    Putting it all together

    We wrote a shell script to automatically launch the modified U-Boot image on the board, and then flash the different images on the eMMC (U-Boot and the root filesystem). We also added an option to flash an MBR partition table as well as flashing a zeroed file to wipe the U-Boot environment. In our project, Buildroot is being used, so our tool makes some assumptions about the location of the tools and image files.

    Our script can be found here: flash.sh. To flash the entire system:

    # ./flash.sh -a
    

    To flash only certain parts, like the bootloader:

    # ./flash.sh -b 
    

    By default, our script expects the Buildroot output directory to be in buildroot/output, but this can be overridden using the BUILDROOT environment variable.

    Conclusion

    By assembling existing tools and mechanisms, we have been able to quickly create a factory flashing process for i.MX6 platforms that is really simple and efficient. It is worth mentioning that we have re-used the same idea for the factory flashing process of the C.H.I.P computer. On the C.H.I.P, instead of using imx-usb-loader, we have used FEL based booting: the C.H.I.P indeed uses an Allwinner ARM processor, providing a different recovery mechanism than the one available on i.MX6.

    by Antoine Ténart at February 15, 2016 09:55 AM

    February 14, 2016

    Harald Welte

    Back from netdevconf 1.1 in Seville

    I've had the pleasure of being invited to netdevconf 1.1 in Seville, spain.

    After about a decade of absence in the Linux kernel networking community, it was great to meet lots of former colleagues again, as well as to see what kind of topics are currently being worked on and under discussion.

    The conference had a really nice spirit to it. I like the fact that it is run by the community itself. Organized by respected members of the community. It feels like Linux-Kongress or OLS or UKUUG or many others felt in the past. There's just something that got lost when the Linux Foundation took over (or pushed aside) virtually any other Linux kernel related event on the planet in the past :/ So thanks to Jamal for starting netdevconf, and thanks to Pablo and his team for running this particular instance of it.

    I never really wanted to leave netfilter and the Linux kernel network stack behind - but then my problem appears to be that there are simply way too many things of interest to me, and I had to venture first into RFID (OpenPCD, OpenPICC), then into smartphone hardware and software (Openmoko) and finally embark on a journey of applied telecoms archeology by starting OpenBSC, OsmocomBB and various other Osmocom projects.

    Staying in Linux kernel networking land was simply not an option with a scope that can only be defined as wide as wanting to implement any possible protocol on any possible interface of any possible generation of cellular network.

    At times like attending netdevconf I wonder if I made the right choice back then. Linux kernel networking is a lot of fun and hard challenges, too - and it is definitely an area that's much more used by many more organizations and individuals: The code I wrote on netfilter/iptables is probably running on billions of devices by now. Compare that to the Osmocom code, which is probably running on a few thousands of devices, if at all. Working on Open Source telecom protocols is sometimes a lonely fight. Not that I wouldn't value the entire team of developers involved in it. to the contrary. But lonely in the context that 99.999% of that world is a proprietary world, and FOSS cellular infrastructure is just the 0.001% at the margin of all of that.

    One the Linux kernel side, you have virtually every IT company putting in their weight these days, and properly funded development is not that hard to come by. In cellular, reasonable funding for anything (compared to the scope and complexity of the tasks) is rather the exception than the norm.

    But no, I don't have any regrets. It has been an interesting journey and I probably had the chance to learn many more things than if I had stayed in TCP/IP-land.

    If only each day had 48 hours and I could work both on Osmocom and on the Linux kernel...

    by Harald Welte at February 14, 2016 11:00 PM

    February 12, 2016

    Video Circuits

    Glass House (1983)




    "Video by G.G. Aries
    Music by Emerald Web
    from California Images: Hi Fi For The Eyes"

    by Chris (noreply@blogger.com) at February 12, 2016 03:16 AM

    February 11, 2016

    Elphel

    NC393 camera is fit for flight

    The components for 10393 and other related circuit boards for the new NC393 camera series have been ordered and contract manufacturing (CM) is ready to assemble the first batch of camera boards.

    In the meantime, the extruded parts that will be made into NC393 camera body have been received at Elphel. The extrusion looks very slick with thin, 1mm walls made out of strong 6061-T6 aluminium, and weighs only 55g. The camera’s new lightweight design is suitable for use on a small aircraft. The heat frame responsible for cooling the powerful processor has also been extruded.

    We are very pleased with the performance of Profile Precision Extrusions located in Phoenix, Arizona, which have delivered a very accurate product ahead of the proposed schedule. Now we can proudly engrave “Made in USA” on the camera, as now even the camera body parts are made in the United States.

    Of course, we have tried to order the extrusion in China, but the intricately detailed profile is difficult to extrude and tolerances were hard to match, so when Profile Precision was recommended to us by local extrusion facilities we were happy to discover the outstanding quality this company offers.

     

    extrusion_393 extrusion_393_heatFrame 4extrusions_393

     

    While waiting for the extruded parts we have been playing with another new toy: the 3D printer. We have been creating prototypes of various camera models of the NC393 series. The cameras are designed and modelled in a 3D virtual environment, and can viewed and even taken apart by mouse click thanks to X3dom technology. The next step is to build actual parts on the 3D printer and physically assemble the camera prototypes, which will allow us to start using the prototypes in the physical world: finding what features are missing, and correcting and finalizing the design. For example, when the mini-panoramic NC393-4PI4 camera prototype was assembled it was clear that it needs the 4 fins (now seen on the final model) to protect the lenses from touching the surfaces as well as to provide shade from the sun. NC393-4PI4 and NC393-4PI4-IMU-GPS are small 360 degree panoramic cameras assembled with 4 fish-eye lenses especially suitable for interior panoramic applications.

    The prototypes are not as slick as the actual aluminium bodies, but they give a very good example of what the actual cameras will look like.

     

    NC393_parts_prototype NC393-M2242-CS_prototype1 NC393-4PI4-IMU-GPS_prototype2

     

    As of today, the 10393 and other boards are in production, the prototypes are being built and tested for design functionality, and the aluminium extrusions have been received. With all this taken care of, we are now less than one month away from the NC393 being offered for sale; the first cameras will be distributed to the loyal Elphel customers who have placed and pre-paid orders several weeks ago.

    by olga at February 11, 2016 10:49 PM

    February 09, 2016

    Harald Welte

    netdevconf 1.1: Running cellular infrastructure on Linux

    Today I had the pleasure of presenting at netdevconf 1.1 a tutorial about Running cellular infrastructure on Linux. The tutorial is intended to guide you through the process of setting up + configuring yur own minimal private GSM+GPRS network.

    The video recording is available from https://www.youtube.com/watch?v=I4i2Gy4JhDo

    Slides are available at http://git.gnumonks.org/index.html/laforge-slides/plain/2016/netdevconf-osmocom/running-foss-gsm.html

    by Harald Welte at February 09, 2016 11:00 PM

    February 04, 2016

    osPID

    Brand New Shinning Website

    We’ve been working hard over the last month or so getting our old website sorted out. Out of date software running on the site, an enormous amount of spam on the forum, and software update mishaps lead us to completely redo everything.  The new website runs completely on WordPress, removing the wiki software (Mediawiki) and the forum software (phpbb). Now, both the forum and wiki are served through WordPress using bbPress and custom posts respectively. We did our best to migrate all content over from the old platforms.  The wiki content came over perfectly, and we were even able to add some updates.  The forum was also ported (posts/topics/accounts)  but we were unable to bring over account passwords.  As a result you will need to do a password reset before using the new forum. We’re sorry about the inconvenience.

    We hope that this  new website will help us better serve the osPID community. Please let us know if there are any broken links or other issues with the website.

    Take care!

    by rocketscream at February 04, 2016 01:55 PM

    February 03, 2016

    Bunnie Studios

    Help Make “The Essential Guide to Electronics in Shenzhen” a Reality

    Readers of my blog know I’ve been going to Shenzhen for some time now. I’ve taken my past decade of experience and created a tool, in the form of a book, that can help makers, hackers, and entrepreneurs unlock the potential of the electronics markets in Shenzhen. I’m looking for your help to enable a print run of this book, and so today I’m launching a campaign to print “The Essential Guide to Electronics in Shenzhen”.

    As a maker and a writer, the process of creating the book is a pleasure, but I’ve come to dread the funding process. Today is like judgment day; after spending many months writing, I get to find out if my efforts are deemed worthy of your wallet. It’s compounded by the fact that funding a book is a chicken-and-egg problem; even though the manuscript is finished, no copies exist, so I can’t send it to reviewers for validating opinions. Writing the book consumes only time; but printing even a few bound copies for review is expensive.

    In this case, the minimum print run is 1,000 copies. I’m realistic about the market for this book – it’s most useful for people who have immediate plans to visit Shenzhen, and so over the next 45 days I think I’d be lucky if I got a hundred backers. However, I don’t have the cash to finance the minimum print run, so I’m hoping I can convince you to purchase a copy or two of the book in the off-chance you think you may need it someday. If I can hit the campaign’s minimum target of $10,000 (about 350 copies of the book), I’ll still be in debt, but at least I’ll have a hope of eventually recovering the printing and distribution costs.

    The book itself is the guide I wish I had a decade ago; you can have a brief look inside here. It’s designed to help English speakers make better use of the market. The bulk of the book consists of dozens of point-to-translate guides relating to electronic components, tools, and purchasing. It also contains supplemental chapters to give a little background on the market, getting around, and basic survival. It’s not meant to replace a travel guide; its primary focus is on electronics and enabling the user to achieve better and more reliable results despite the language barriers.

    Below is an example of a point-to-translate page:

    For example, the above page focuses on packaging. Once you’ve found a good component vendor, sometimes you find your parts are coming in bulk bags, instead of tape and reel. Or maybe you just need the whole thing put in a shipping box for easy transportation. This page helps you specify these details.

    I’ve put several pages of the guide plus the whole sales pitch on Crowd Supply’s site; I won’t repeat that here. Instead, over the coming month, I plan to post a couple stories about the “making of” the book.

    The reality is that products cost money to make. Normally, a publisher takes the financial risk to print and market a book, but I decided to self-publish because I wanted to add a number of custom features that turn the book into a tool and an experience, rather than just a novel.

    The most notable, and expensive, feature I added are the pages of blank maps interleaved with business card and sample holders.

    Note that in the pre-print prototype above, the card holder pages are all in one section, but the final version will have one card holder per map.

    When comparison shopping in the market, it’s really hard to keep all the samples and vendors straight. After the sixth straight shop negotiating in Chinese over the price of switches or cables, it’s pretty common that I’ll swap a business card, or a receipt will get mangled or lost. These pages enable me to mark the location of a vendor, associate it with a business card and pricing quotation, and if the samples are small (like the LEDs in the picture above) keep the sample with the whole set. I plan on using a copy of the book for every project, so a couple years down the road if someone asks me for another production run, I can quickly look up my suppliers. Keeping the hand-written original receipts is essential, because suppliers will often honor the pricing given on the receipt, even a couple years later, if you can produce it. The book is designed to give the best experience for sourcing components in the Shenzhen electronic markets.

    In order to accommodate the extra thickness of samples, receipts and business cards, the book is spiral-bound. The spiral binding is also convenient for holding a pen to take notes. Finally, the spiral binding also allows you to fold the book flat to a page of interest, allowing both the vendor and the buyer to stare at the same page without fighting to keep the book open. I added an elastic strap in the back cover that can be used as a bookmark, or to help keep the book closed if it starts to get particularly full.

    I also added tabbed pages at the beginning of every major section, to help with quickly finding pages of interest. Physical print books enable a fluidity in human interaction that smartphone apps and eBooks often fail to achieve. Staring at a phone to translate breaks eye contact, and the vendor immediately loses interest; momentum escapes as you scroll, scroll, scroll to the page of interest, struggle with auto-correction on a tiny on-screen keyboard, or worse yet stare at an hourglass as pages load from the cloud. But pull out the book and start thumbing through the pages, the vendor can also see and interact with the translation guide. They become a part of the experience; it’s different, interesting, and keeps their attention. Momentum is preserved as both of you point at various terms on the page to help clarify the transaction.

    Thus, I spent a fair bit of time customizing the physical design of the book to make it into a tool and an experience. I considered the human factors of the Shenzhen electronics market; this book is not just a dictionary. This sort of tweaking can only be done by working with the printer directly; we had to do a bit of creative problem solving to figure out a process that works to bring all these elements together that can also pump out books at a rate fast enough to keep it in the realm of affordability. Of course, the cost of these extra features are reflected in the book’s $35 cover price (discounted to $30 if you back the campaign now), but I think the book’s value as a sourcing and translation tool makes up for its price, especially compared to the cost of plane tickets. Or worse yet, getting the wrong part because of a failure to communicate, or losing track of a good vendor because a receipt got lost in a jumble of samples.

    This all bring me back to the point of this post. Printing the book is going to cost money, and I don’t have the cash to print and inventory the book on my own. If you think someday you might go to Shenzhen, or maybe you just like reading what I write or how the cover looks, please consider backing the campaign. If I can hit the minimum funding target in the next 45 days, it will enable a print run of 1,000 books and help keep it in stock at Crowd Supply.

    Thanks, and happy hacking!

    by bunnie at February 03, 2016 04:13 PM

    ZeptoBARS

    Noname TL431 : weekend die-shot

    Yet another noname TL431.
    Die size 730x571 µm.


    February 03, 2016 05:50 AM

    January 31, 2016

    Harald Welte

    On the OpenAirInterface re-licensing

    In the recent FOSDEM 2016 SDR Devroom, the Q&A session following a presentation on OpenAirInterface touched the topic of its controversial licensing. As I happen to be involved deeply with Free Software licensing and Free Software telecom topics, I thought I might have some things to say about this topic. Unfortunately the Q&A session was short, hence this blog post.

    As a side note, the presentation was actually certainly the least technical presentation in all of the FOSDEM SDR track, and that with a deeply technical audience. And probably the only presentation at all at FOSDEM talking a lot about "Strategic Industry Partners".

    Let me also state that I actually have respect for what OAI/OSA has been and still is doing. I just don't think it is attractive to the Free Software community - and it might actually not be Free Software at all.

    OpenAirInterface / History

    Within EURECOM, a group around Prof. Raymond Knopp has been working on a Free Software implementation of all layers of the LTE (4G) system known as OpenAirInterface. It includes the physical layer and goes through to the core network.

    The OpenAirInterface code was for many years under GPL license (GPLv2, other parts GPLv3). Initially the SVN repositories were not public (despite the license), but after some friendly mails one (at least I) could get access.

    I've read through the code at several points in the past, it often seemed much more like a (quick and dirty?) proof of concept implementation to me, than anything more general-purpose. But then, that might have been a wrong impression on my behalf, or it might be that this was simply sufficient for the kind of research they wanted to do. After all, scientific research and FOSS often have a complicated relationship. Researchers naturally have their papers as primary output of their work, and software implementations often are more like a necessary evil than the actual goal. But then, I digress.

    Now at some point in 2014, a new organization the OpenAirInterface Software Association (OSA) was established. The idea apparently was to get involved with the tier-1 telecom suppliers (like Alcatel, Huawei, Ericsson, ...) and work together on an implementation of Free Software for future mobile data, so-called 5G technologies.

    Telecom Industry and Patents

    In case you don't know, the classic telecom industry loves patents. Pretty much anything and everything is patented, and the patents are heavily enforced. And not just between Samsung and Apple, or more recently also Nokia and Samsung - but basically all the time.

    One of the big reasons why even the most simple UMTS/3G capable phones are so much more expensive than GSM/2G is the extensive (and expensive) list of patents Qualcomm requires every device maker to license. In the past, this was not even a fixed per-unit royalty, but the license depended on the actual overall price of the phone itself.

    So wanting to work on a Free Software implementation of future telecom standards with active support and involvement of the telecom industry obviously means contention in terms of patents.

    Re-Licensing

    The existing GPLv2/GPLv3 license of the OpenAirInterface code of course would have meant that contributions from the patent-holding telecom industry would have to come with appropriate royalty-free patent licenses. After all, of what use is it if the software is free in terms of copyright licensing, but then you still have the patents that make it non-free.

    Now the big industry of course wouldn't want to do that, so the OSA decided to re-license the code-base under a new license.

    As we apparently don't yet have sufficient existing Free Software licenses, they decided to create a new license. That new license (the OSA Public License V1.0 not only does away with copyleft, but also does away with a normal patent grant.

    This is very sad in several ways:

    • license proliferation is always bad. Major experts and basically all major entities in the Free Software world (FSF, FSFE, OSI, ...) are opposed to it and see it as a problem. Even companies like Intel and Google have publicly raised concern about license Proliferation.
    • abandoning copyleft. Many people particularly from a GNU/Linux background would agree that copyleft is a fair deal. It ensures that everyone modifying the software will have to share such modifications with other users in a fair way. Nobody can create proprietary derivatives.
    • taking away the patent grant. Even the non-copyleft Apache 2.0 License the OSA used as template has a broad patent grant, even for commercial applications. The OSA Public License has only a patent grant for use in research context

    In addition to this license change, the OSA also requires a copyright assignment from all contributors.

    Consequences

    What kind of effect does this have in case I want to contribute?

    • I have to sign away my copyright. The OSA can at any given point in time grant anyone whatever license they want to this code.
    • I have to agree to a permissive license without copyleft, i.e. everyone else can create proprietary derivatives of my work
    • I do not even get a patent grant from the other contributors (like the large Telecom companies).

    So basically, I have to sign away my copyright, and I get nothing in return. No copyleft that ensures other people's modifications will be available under the same license, no patent grant, and I don't even keep my own copyright to be able to veto any future license changes.

    My personal opinion (and apparently those of other FOSDEM attendees) is thus that the OAI / OSA invitation to contributions from the community is not a very attractive one. It might all be well and fine for large industry and research institutes. But I don't think the Free Software community has much to gain in all of this.

    Now OSA will claim that the above is not true, and that all contributors (including the Telecom vendors) have agreed to license their patents under FRAND conditions to all other contributors. It even seemed to me that the speaker at FOSDEM believed this was something positive in any way. I can only laugh at that ;)

    FRAND

    FRAND (Fair, Reasonable and Non-Discriminatory) is a frequently invoked buzzword for patent licensing schemes. It isn't actually defined anywhere, and is most likely just meant to sound nice to people who don't understand what it really means. Like, let's say, political decision makers.

    In practise, it is a disaster for individuals and small/medium sized companies. I can tell you first hand from having tried to obtain patent licenses from FRAND schemes before. While they might have reasonable per-unit royalties and they might offer those royalties to everyone, they typically come with ridiculous minimum annual fees.

    For example let's say they state in their FRAND license conditions you have to pay 1 USD per device, but a minimum of USD 100,000 per year. Or a similarly large one-time fee at the time of signing the contract.

    That's of course very fair to the large corporations, but it makes it impossible for a small company who sells maybe 10 to 100 devices per year, as the 100,000 / 10 then equals to USD 10k per device in terms of royalties. Does that sound fair and Non-Discriminatory to you?

    Summary

    OAI/OSA are trying to get a non-commercial / research-oriented foot into the design and specification process of future mobile telecom network standardization. That's a big and difficult challenge.

    However, the decisions they have taken in terms of licensing show that they are primarily interested in aligning with the large corporate telecom industry, and have thus created something that isn't really Free Software (missing non-research patent grant) and might in the end only help the large telecom vendors to uni-directionally consume contributions from academic research, small/medium sized companies and individual hackers.

    by Harald Welte at January 31, 2016 11:00 PM

    January 27, 2016

    January 26, 2016

    Michele's GNSS blog

    uBlox: Galileo, anti-jamming and anti-spoofing firmware

    Just downloaded the firmware upgrade for flash-based M8 modules from uBlox.
    Flashed it in no time.
    The result of UBX-MON-VER is now:



    So checked Galileo in CFG-GNSS:



    Result :)



    Incidentally, there is a "spoofing" flag now as well :O



    Don't dare trying this on M8T...

    by noreply@blogger.com (Michele Bavaro) at January 26, 2016 10:42 PM

    January 22, 2016

    Bunnie Studios

    Novena on the Ben Heck Show

    I love seeing the hacks people do with Novena! Thanks to Ben & Felix for sharing their series of adventures! The custom case they built looks totally awesome, check it out.

    by bunnie at January 22, 2016 04:37 PM

    January 21, 2016

    Bunnie Studios

    Name that Ware January 2016

    The Ware for January 2016 is shown below.

    I just had to replace the batteries on this one, so while it was open I tossed it in the scanner and figured it would make a fun and easy name that ware to start off the new year.

    by bunnie at January 21, 2016 03:37 PM

    Winner, Name that Ware December 2015

    The ware for December 2015 was a Thurlby LA160 logic analyzer. Congrats to Cody Wheeland for nailing it! email me for your prize. Also, thanks to everyone for sharing insights as to why the PCBs developed ripples of solder underneath the soldermask. Fascinating stuff, and now I understand why in PCB processing there’s a step of stripping the tin plate before applying the soldermask.

    by bunnie at January 21, 2016 03:37 PM

    January 19, 2016

    Free Electrons

    ELCE 2015 conference videos available

    ELC Europe 2015 logoAs often in the recent years, the Linux Foundation has shot videos of most of the talks at the Embedded Linux Conference Europe 2015, in Dublin last October.

    These videos are now available on YouTube, and individual links are provided on the elinux.org wiki page that keeps track of presentation materials as well. You can also find them all through the Embedded Linux Conference Europe 2015 playlist on YouTube.

    All this is of course a priceless addition to the on-line slides. We hope these talks will incite you to participate to the next editions of the Embedded Linux Conference, like in San Diego in April, or in Berlin in October this year.

    In particular, here are the videos from the presentations from Free Electrons engineers.

    Alexandre Belloni, Supporting multi-function devices in the Linux kernel

    Kernel maintainership: an oral tradition

    Tutorial: learning the basics of Buildroot

    Our CTO Thomas Petazzoni also gave a keynote (Linux kernel SoC mainlining: Some success factors), which was well attended. Unfortunately, like for some of the other keynotes, no video is available.

    by Michael Opdenacker at January 19, 2016 01:06 PM

    January 15, 2016

    Bunnie Studios

    Making of the Novena Heirloom

    Make is hosting a wonderfully detailed article written by Kurt Mottweiler about his experience making the Novena Heirloom laptop. Check it out!


    by bunnie at January 15, 2016 05:39 PM

    Free Electrons

    Device Tree on ARM article in French OpenSilicium magazine

    Our French readers are most likely aware of the existence of a magazine called OpenSilicium, a magazine dedicated to embedded technologies, with frequent articles on platforms like the Raspberry Pi, the BeagleBone Black, topics like real-time, FPGA, Android and many others.

    Open Silicium #17

    Issue #17 of the magazine has been published recently, and features a 14-pages long article Introduction to the Device Tree on ARM, written by Free Electrons engineer Thomas Petazzoni.

    Open Silicium #17

    Besides Thomas article, many other topics are covered in this issue:

    • A summary of the Embedded Linux Conference Europe 2015 in Dublin
    • Icestorm, a free development toolset for FPGA
    • Using the Armadeus APF27 board with Yocto
    • Set up an embedded Linux system on the Zynq ZedBoard
    • Debugging with OpenOCD and JTAG
    • Usage of the mbed SDK on a small microcontroller, the LPC810
    • From Javascript to VHDL, the art of writing synthetizable code using an imperative language
    • Optimization of the 3R strems decompression algorithm

    by Thomas Petazzoni at January 15, 2016 09:16 AM

    Free Electrons at FOSDEM and the Buildroot Developers Meeting

    FOSDEM 2016The FOSDEM conference will take place on January 30-31 in Brussels, Belgium. Like every year, there are lots of interesting talks for embedded developers, starting from the Embedded, Mobile and Automotive Devroom, but also the Hardware track, the Graphics track. Some talks of the IoT and Security devrooms may also be interesting to embedded developers.

    Thomas Petazzoni, embedded Linux engineer and CTO at Free Electrons, will be present during the FOSDEM conference. Thomas will also participate to the Buildroot Developers Meeting that will take place on February 1-2 in Brussels, hosted by Google.

    by Thomas Petazzoni at January 15, 2016 08:52 AM

    January 14, 2016

    Free Electrons

    Linux 4.4, Free Electrons contributions

    Linux 4.4 is the latest releaseLinux 4.4 has been released, a week later than the normal schedule in order to allow kernel developers to recover from the Christmas/New Year period. As usual, LWN has covered the 4.4 cycle merge window, in two articles: part 1 and part 2. This time around, KernelNewbies has a nice overview of the Linux 4.4 changes. With 112 patches merged, we are the 20th contributing company by number of patches according to the statistics.

    Besides our contributions in terms of patches, some of our engineers have also become over time maintainers of specific areas of the Linux kernel. Recently, LWN.net conducted a study of how the patches merged in 4.4 went into the kernel, which shows the chain of maintainers who pushed the patches up to Linus Torvalds. Free Electrons engineers had the following role in this chain of maintainers:

    • As a co-maintainer of the Allwinner (sunxi) ARM support, Maxime Ripard has submitted a pull request with one patch to the clock maintainers, and pull requests with a total of 124 patches to the ARM SoC maintainers.
    • As a maintainer of the RTC subsystem, Alexandre Belloni has submitted pull requests with 30 patches directly to Linus Torvalds.
    • As a co-maintainer of the AT91 ARM support, Alexandre Belloni has submitted pull requests with 46 patches to the ARM SoC maintainers.
    • As a co-maintainer of the Marvell EBU ARM support, Gregory Clement has submitted pull requests with a total of 33 patches to the ARM SoC maintainers.

    Our contributions for the 4.4 kernel were centered around the following topics:

    • Alexandre Belloni continued some general improvements to support for the AT91 ARM processors, with fixes and cleanups in the at91-reset, at91-poweroff, at91_udc, atmel-st, at91_can drivers and some clock driver improvements.
    • Alexandre Belloni also wrote a driver for the RV8803 RTC from Microcrystal.
    • Antoine Ténart added PWM support for the Marvell Berlin platform and enabled the use of cpufreq on this platform.
    • Antoine Ténart did some improvements in the pxa3xx_nand driver, still in preparation to the addition of support for the Marvell Berlin NAND controller.
    • Boris Brezillon did a number of improvements to the sunxi_nand driver, used for the NAND controller found on the Allwinner SoCs. Boris also merged a few patches doing cleanups and improvements to the MTD subsystem itself.
    • Boris Brezillon enabled the cryptographic accelerator on more Marvell EBU platforms by submitting the corresponding Device Tree descriptions, and he also fixed a few bugs found in the driver
    • Maxime Ripard reworked the interrupt handling of per-CPU interrupts on Marvell EBU platforms especially in the mvneta network driver. This was done in preparation to enable RSS support in the mvneta driver.
    • Maxime Ripard added support for the Allwinner R8 and the popular C.H.I.P platform.
    • Maxime Ripard enabled audio support on a number of Allwinner platforms, by adding the necessary clock code and Device Tree descriptions, and also several fixes/improvements to the ALSA driver.

    The details of our contributions for 4.4:

    by Thomas Petazzoni at January 14, 2016 02:32 PM

    January 13, 2016

    Michele's GNSS blog

    NT1065 review

    So I finally came about testing the NT1065… apologies for the lack of detail but I have done this in my very little spare time. Also, I would like to clarify that I am in no way affiliated to NTLab.

    Chip overview

    A picture speaks more than a thousand words.
    Figure 1: NT1065 architecture
    Things worth noting above are:
    • Four independent input channels with variable RF gain, so up to 4 distinct antennas can be connected;
    • Two LOs controlled by integer synthesizers, one per pair of channels, tuned respectively for the high and low RNSS band, but one can choose to route the upper LO to the lower pair and have 4 phase coherent channels
    • ADC sample rate derived from either LO through integer division
    • 4 independent image-reject mixers, IF filters and variable gain (with AGC) paths
    • Four independent outputs, either as a CMOS two bit ADC or analogue differential so one could
      • connect his/her own ADC or
      • phase-combine the IF outputs in a CRPA fashion prior to digitisation
    • standard SPI port control
    Another important point for a hardware designer (I used to be a little bit of that) is this: