#milkymist IRC log for Thursday, 2011-10-20

lekernellekernel productions presents... an almost decent M1 unboxing video09:41
lekernelor http://www.youtube.com/watch?v=0k080nzA_z409:42
lekernelwpwrak, what do you think about a video input "color transformation matrix" controllable from the patch?12:06
lekernelalso, I used LED lighting... doesn't help12:07
wpwrakyeah, video input modification could be very interesting. maybe even go one step further, and have a few FPUs that just work on that ? easy to parallelize, shouldn't need a lot of instructions/registers, and the existing FPU design is already perfect for SIMD12:14
lekernelwhat kind of transforms would you like to be able to do?12:19
lekernelor, more particularly, what data?12:19
lekernelone pixel? squares of 2x2 pixels? more? overlapping/non-overlapping?12:19
wpwraki was thinking of just pixels. dunno if that's too technical a view, though12:21
wpwrakif you have >1 pixel, you could of course to edge detection and such12:22
lekerneldo you have a color transform in mind that would look significantly better than with using a matrix?12:22
wpwrakno. i'm just thinking of what could give us a maximum of flexibility with the available technology12:23
lekernelalso, the color transform could also generate an alpha channel12:23
lekernelthere's already alpha support down the pipeline, but it's only a global alpha12:24
lekernelthat could be nice to make parts of the included images transparent12:24
wpwrakoh yes, that would be cool12:25
lekernelwe'll need to overhaul the PFPU register allocator a bit... 4x3 matrices will eat registers like crazy12:26
wpwrak3x3 ? 27 registers if you have one for r, g, b each12:27
lekernelthe matrix does RGB->RGBA12:28
lekernelit's 4x3, 12 coefficients12:28
wpwrakaah, i was thinking of an PFU (array) that works on each pixel.12:29
wpwrake.g., with a 3x3 RGB matrix on input, one RGBA output12:30
lekernelprocessing multiple pixels at once is more difficult... you can't just simply insert a stage in the TMU pipeline12:31
wpwrakjust in the video in path, before it hits anything else ?12:32
lekernelthe rendering system uses the TMU to blit the camera picture into the texture buffer12:34
lekernelif we put this matrix thing in the TMU, we have more flexibility and we factor things (e.g. supersedes decay, makes it possible to apply the transform on still pictures)12:35
wpwrakso you'd put it in the frame buffer feedback loop ?12:36
lekernelall camera pictures go through the TMU atm12:36
lekernelexcept for video-in preview in the GUI (and that's why it's slow)12:37
wpwraki haven't looked at that part in detail yet, but is the camera->TMU interface a pixel stream ? or something more complicated ?12:40
wpwrakmy FPU idea was to put an array of FPUs in the camera pixel stream, where they transform each pixel before it goes to the TMU for further processing12:40
lekernelthe video-in core receives the signal, does the YUV->RGB transform and just DMA's the result12:41
lekernelyou get raw interlaced framebuffers at the output12:41
wpwrakinterlaced is like the Terminator. damn hard to kill.12:42
lekernelthere are two framebuffers, one for each field. you can also tell the video-in core that you want only one field (and this it what it does atm)12:42
lekernelwell, we simply discard one field and scale it up vertically with the TMU :-)12:42
wpwrakaah, hence the low resolution ! :)12:43
lekernelyes, but deinterlacing without major losses of vertical resolution is a massive pain12:45
wpwrakokay, so at least with the current half-frame approach, such a pixel processor array ought to work. for a NxN matrix input, you'd need a (N-1)/2 lines + (N-1)/2 pixels buffer before the pixel FPU array12:45
lekerneldifficult, easy to get wrong, and a memory bandwidth pig12:45
wpwrakoh yes, deinterlacing sucks :)12:45
wpwrakwhy a pig ?12:45
lekernelbecause deinterlacing algos can make a lot of memory accesses for things like motion estimation12:46
wpwrakaah, deinterlacing. okay. thought you were talking about the FPU array :)12:46
lekernelif you implement this processor array in the TMU, you can't choose the pixel access order12:49
lekernelyou can implement it as a separate unit that operates in framebuffers, and then you are free to use any access order12:49
lekernelbut it increases memory bandwidth consumption12:50
lekernelif you do it in the video-in core, you lose flexibility12:50
wpwrakand if it's between camera YUV->RGB and TMU ? or maybe even replacing YUB-RGB ?12:50
lekernelor, you can break away from the DMA paradigm and implement "pixel stream buses" :)12:51
wpwrakthat sounds nice :)12:51
lekernelyou have this pixel processing unit sitting somewhere on the chip, and you can route it between the video-in core and its DMA write unit12:52
lekernelor between a DMA-read and a DMA-write unit12:52
wpwrakthat sounds very nice, yes12:52
wpwrakor maybe even have two of them. not sure how fat they'd get. the DMA-to-DMA one would have to be bigger than a camera-only unit, due to the larger amount of pixels.12:53
lekernelwell, there are a lot of details. here's one :)12:54
lekerneland you'll need flow control on the buses anyway12:54
wpwrakfor the DMA-to-DMA path, yes, sounds reasonable12:57
lekerneleven at the video-in output on at the DMA-write input13:04
lekernelyou cannot predict the memory access latency13:04
lekernelbecause of DRAM refreshes, switching DRAM rows, and bus sharing13:04
wpwrakhmm, but you'd already have to have a mechanism of this sort in place13:04
wpwrakthe FPU array would just add a fixed delay to the camera pixel pipeline13:05
wpwrak"fixed" as in either worst-case (i.e., size of program space) or as in always the same for the same program, but variable across programs13:06
zumbihi guys! do you have a pointer for the copyright license for milkymist hardware?15:28
lekernelwhat hardware?15:31
lekernelFPGA? PCB? box?15:31
zumbilekernel: uhm.. PCB15:40
kristianpaulfirst time i read cpu and club in the same tittle :)18:46
lekernel_yes, I'm trying to write noticeable headlines ;)18:48
lekernel_ok it should be in soon :)19:28
lekernel_there! http://hardware.slashdot.org/story/11/10/20/193245/open-source-cpus-coming-to-a-club-near-you20:06
Action: lekernel_ is getting better at PR20:06
kristianpaulplace and route? :-)20:09
wpwrakplace the news and route the readers ;-)20:10
rigidjust saw the project. nice! (i love the transparent case :)20:33
rigidwhat s/w are you using? is this somehow related to milkdrop?20:33
Action: rigid crawls the webpage20:33
rigidah.. yep20:34
lekernel_hi rigid22:12
lekernel_own software, and yes, lots of milkdrop inspirations22:13
rigidlekernel_: looks very nice... and great its open22:25
rigidi'm looking for open audiovisualization software to use them with LEDs22:26
juliusbnice slashdot article23:41
juliusblekernel_: I like your presentation about OS FPGA toolchains23:44
juliusbare you doing anything like that in the UK any time soon? I've moved to Cambridge, btw, and am planning on doing things with the OSHUG here - if you're not too far away you might like to come and get some of those guys interested23:45
--- Fri Oct 21 201100:00

Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!