#milkymist IRC log for Wednesday, 2011-11-09

wpwrakwell, you probably don't need it in this case. bah. getting tired. time for dinner.00:04
wpwrakah, what was the other problem - shutdown not working ?00:31
wpwrakreboot00:31
wpwrakindeed, that's still brokebn00:32
wpwrakit sits in yaffs_flush_whole_cache -> yaffs_flush_file_cache00:33
wpwrakwild guess: yaffs is being shut down after the NOR driver ?00:34
wpwrakhmm. though ...00:34
wpwrak yaffs_flush_file_cache (obj=0xffffffff) at yaffs_guts.c:140300:34
wpwrakdev->cache == NULL00:36
wpwrakthis system really wants an inaccessible zero page. badly.00:36
wpwrakgrmbl. so watchpoints have a size limit :-(00:43
wpwrak(gdb) awatch *(struct yaffs_obj_hdr *) 000:43
wpwrakExpression cannot be implemented with read/access watchpoint.00:43
wpwrakinteresting ... (gdb) p *dev00:44
wpwrakCannot access memory at address 0x418d80d800:44
wpwrak(gdb) p dev->cache00:45
wpwrak$2 = (struct yaffs_cache *) 0x000:45
wpwrakwhat now ? :)00:45
wpwrakseems that gdb is too lazy to transfer such a big structure00:47
wpwrakbah, a meager 3684 bytes !00:47
wpwrakmaybe that's also the reason why stack traces always terminate so early00:49
wpwraktypically after 2-3 frames, with a complaint "Backtrace stopped: previous frame inner to this frame (corrupt stack?)"00:49
wpwrakah. that's obviously wrong. but in this case, i don't know what to do about it00:56
wpwraktentative patch sent as well. at least reboot seems to work properly.01:21
mumptaihi06:47
GitHub142[flickernoise] sbourdeauducq pushed 1 new commit to master: http://git.io/X7ORdg09:51
GitHub142[flickernoise/master] yaffs.c: ycb_fsunmount already cleans up, don't do it again in unmount_handler - Werner Almesberger09:51
wpwrakhmm, i kinda wonder what would happen if i just replaced that nasty linked list code with the one from linux ...11:05
terpstraquick question: if you strip rtems down to the minimum. how big does the rom end up as on the lm32?11:05
terpstraball park11:06
wpwraki guess lekernel  might know. else, try and see ? :)11:14
kristianpaulterpstra: minimun means no shell and just a printf hello world?11:19
lekernelhttp://www.rtems.com/wiki/index.php/TinyRTEMS#Configuring_RTEMS_for_Size11:20
lekernelbut I have not really checked... we have the luxury of SDRAM here11:21
kristianpaulah true11:22
terpstraso 44k-ish11:22
terpstrawe currently have only 64k of memory =D11:22
lekernelmore 57k-ish, LM32 is closer to sparc than 68k11:22
lekernelyou want to run RTEMS on White Rabbit?11:23
terpstrai am evaluating our options atm11:23
terpstracurrently the lm32 is just running PTP over ethernet11:24
terpstrabut we need it also to run arp and dhcp11:24
terpstraand maybe snmp11:24
terpstraso that starts to sound like an OS would be helpful11:24
terpstrabut we are very resource constrained11:24
lekernelmaybe this? http://www.sics.se/~adam/lwip/11:25
terpstrayes, i am aware of that project11:26
terpstrawe probably need task switching, though11:26
qi-botThe Firmware build was successfull, see images here: http://fidelio.qi-hardware.com/~xiangfu/build-milkymist/milkymist-firmware-11092011-1114/12:05
wpwrakinteresting. now it's easier to trigger the hang. am i just quicker in the morning ? :)12:12
wpwrakhmpf. nice. I: Booting...  assertion "!the_message" failed:13:05
wpwrakthat's even before i get to grab the thing with gdb ...13:06
wpwrakattaching when it's there worked, though. phew :)13:06
wpwrakmwalle: how do you see the possibility of overcoming gdb's apparently very small transfer size ?13:07
wpwrakmwalle: (see also my posting with subject "gdb data transfer limitation ?")13:08
wpwraki can work around it reasonably well in the case of data, but there doesn't seem to be a sane way for processing stack frames piecewise (sane = without prying it apart manually)13:09
lekernelhttp://www.ted.com/talks/a_robot_that_flies_like_a_bird.html16:16
lekernellong version: http://www.festo.com/cms/en_corp/11369_11468.htm#id_1146816:27
kristianpaulah, festo, yes it looked very like that :)16:28
mumptaimoin16:29
kristianpaulcool fly16:30
kristianpaulhi mumptai16:30
kristianpauli guess battery still a problem for long flys?, also will be cool see it fly with some more wind around16:40
kristianpaulwondering also the kind of materials they used16:40
Action: kristianpaul check second link16:40
mumptaiwhat is flying?16:44
kristianpaulmumptai: http://www.festo.com/cms/en_corp/11369_11468.htm#id_1146816:52
Action: kristianpaul see how usefull a CNC is :-)16:52
mumptainice16:56
wpwrakhmm, is it safe to increase the size of the memory regions in milkymist/software/gdbstub/linker.ld ?17:15
wpwraki.e., where does the rom+ram = 8 kB limit come from ?17:15
lekernelit's on-chip memory, which is scarce17:26
lekernelalso, if you change the gdbstub, you'll need to rebuild the bitstream17:27
wpwrakshit :-(17:27
lekernel(you can increase a bit the amount of memory at the same occasion, though. but try not to - it's an expensive resource)17:27
wpwrakhmm. let's see if i can fix the problem on the gdb side instead then17:28
lekernelwell... the main problem with bitstream builds is the 10GB of bloatware you need to set up. once you have ISE installed, it's just one command.17:28
wpwrakand years of processing etc.17:28
wpwrakhow is llhdl coming along anyway ? ;-)17:29
lekernelonly 15-30 min on a modern computer17:29
wpwrakgdb-7.3.1 should be good, right ? i.e., supporting lm32 out of the box ?17:30
lekernelyes17:30
lekernelif you're using the flterm pass-through, any GDB version with lm32 support should be OK17:30
wpwrakkewl, thanks17:30
lekernelthe gdb 7.3+ requirement is only when NOT using flterm17:31
wpwrakhehe, sometimes just reading the source helps :)17:42
wpwrakthe magic word is  set remote memory-read-packet-size 50017:42
wpwrak(or something like this)17:42
wpwrakhmm .. maybe17:43
wpwrakthis lets me read big variables. no luck with the stack, though17:44
wpwrakseems that gdb doesn't understand lm32 stack frames. hmm.17:51
wpwraki guess things are compiled with omit-frame-pointer ?17:54
lekernelFN isn't, unless it's the compiler default17:57
lekernelRTEMS I don't know17:57
wpwraki don't see it there either17:57
wpwrakand we do have a very nice frame pointer. hmm.17:59
wpwrakdown18:00
wpwrakoops18:00
mwallewpwrak: yeah there is a qSupported query18:48
mwallewhich returns PacketSize=..18:48
mwalle#define BUFMAX 80018:49
mwalle#define BUFMAX_HEX 320  /* keep this in sync with BUFMAX */18:49
mwallei fixed this some time ago, and iirc a quick test showed that everything was working ;)18:51
mwallecommit id 02cdac90da618:51
wpwrakhmm, that's old enough that it should be in the SDK18:57
wpwraka test would be something like this (on flickernoise):  p *( struct yaffs_dev *) 0x408d85c018:58
wpwrak(any reasonably valid address will do)18:59
mwallecould you please show remote memory-read-packet-size before setting it?19:02
wpwrakThe memory-read-packet-size is 0. Packets are limited to 800 bytes.19:02
mwallemh is 750 working?19:02
mwalle799?19:03
mwalle798? :)19:03
wpwrakup to 799 yes ;-)19:04
wpwraklength < (sizeof(remcom_out_buffer) / 2))  maybe s/</<=/ ?19:04
wpwrakand then find a way to sneak in the \0 ? :)19:05
mwallebtw do you have binary downloads enabled?19:06
wpwrakah, that's already there19:06
wpwrakno, it's all text19:06
mwallemh theres a checksum at the end of the packet19:07
wpwrakaah, right, the checksum19:07
mwallebut this not written to the buffer19:08
wpwrakyup. generated on the fly19:08
mwalleare you getting E22?19:08
wpwrakso maybe it's really just < vs. <=19:08
wpwrakyes19:08
wpwrakbtw, you don't need parentheses around sizeof(...)/219:09
Action: wpwrak wonders how many bugs are overlooked in reviews due to excessive parentheses19:10
wpwrakmaybe we need a -Wredundant-parentheses ideally enabled by default, along with -Werror :)19:12
mwallewpwrak: mh nice compiler switch :)19:13
mwallewpwrak: thats not my code ;)19:13
wpwrakheh :) many possible culprits19:14
wpwraknow, what's really troubling me at the moment is that stack traces don't work19:16
mwalleyeah <= makes sense19:16
mwallewhats the last stack frame you see?19:18
mwallewpwrak: do you submit a patch for gdbstub? rebuild it, copy gdbstub.rom to the proper location and submit that patch too? :)19:19
wpwraki don't have the bitstream build process set up, so i couldn't test changes to gdbstub19:22
wpwrak*snivel* all i wanted are some nice colorful effects. and now i'm there, fixing RTEMS, poking around in the depths of gdb, ...19:24
wpwrak(fixing RTEMS) as in, i already have two more bugs in the queue. one fixed, the other one in need of better debugging support (hence the stack traces). the 2nd bug may be in FN/MM, though19:26
wpwrak(it's something trying to read from an empty queue right after the system says "Booting". sometimes, a small assert can work wonders ...)19:26
mwallewpwrak: it almost never works like that :b19:27
wpwrakyeah, but usually the ride is a little less scenic :)19:27
lekernelwpwrak, and you're doing an outstanding job at it :) keep up the good stuff *g*19:28
mwallewpwrak: lekernel: ok so i'll fix and test it19:28
wpwrakin fact, the only thing that saves the system from total collapse is that most of the RTEMS infrastructure is used very lightly. put a bit more pressure on it, and all those little oversights will go up like new year's eve fireworks19:28
wpwraklekernel: thanks for commiserating ;-)19:29
wpwrakmwalle: thanks !19:29
mwallei like wpwrak pictured metaphors ;)19:30
wpwrakwhat scares me a bit is that some of the more dubious constructs (like the doubly linked list structure) can have subtle failure modes. e.g., if you just use the list in the "old-fashioned" way, i.e., for (p = list_first(); p; p = p->next) ..., things will almost work. the only problem will be that you get what looks like a corrupt last list element. but the pointer will still be valid, so you wouldn't even trip over that.19:32
wpwrakand there's a pair of examples of mixing up the paradigms right in chain.h, so the issue is less hypothetical than it may seem ...19:33
lekernelhttp://www.rtems.com/node/7019:34
lekernelthis code is used in those :-)19:34
wpwrake.g., if you did the above in linux, where doubly linked lists are somewhat similar, you'd loop forever. that ought to get noticed :)19:35
wpwrakthat casts an interesting light on certain "accidents" ... :)19:35
wpwrakerr, i meant chain.inl19:36
wpwrakjust posted it19:54
wpwraki should start using quilt. the collection of patches is getting a bit messy :)19:54
lekernelyou should post them to the RTEMS mailing list too19:55
wpwraki know .. but i'm already suffering a bit of mailing list overload. so i'm trying to avoid adding more that i don't read either :)20:02
wpwrakbesides, i very much hope we can end this misery before too long ;)20:04
mwallewpwrak: could you send me a flickernoise binary with debug symbols?20:10
wpwraklike http://www.milkymist.org/updates/2011-07-13/flickernoise ?20:14
wpwrakwith the flashable binary http://www.milkymist.org/updates/2011-07-13/flickernoise.fbi20:14
Fallenou(reading rtems ML) Ralf really is a pain.20:34
wpwrak"I don't see what add a cast to void* would fix." ? :)20:39
FallenouI was looking at the chat between Joel and Ralf about RTEMS being dependent on GNU C compiler20:40
wpwrakand, towards what are they leaning ?20:43
FallenouJoel is just saying it would be cool to reduce step by step dependencies on gcc in order to someday be able to compile with something else20:44
FallenouRalf is just answering like an ass as usual20:44
Fallenousarcasm etc20:44
wpwrakhmm. moderate gcc dependencies shouldn't be a big issue. you need a major compiler for such a project anyway. and the likely candidates seem to try to be compatible with gcc anyway.20:45
wpwrakah, very good. at least two patches are in.20:46
Fallenousdcc pcc clang were quoted20:46
FallenouMetrowerks and GreenHill as well20:47
wpwrakdoesn't clang track gcc ?20:47
kristianpaulsdcc :)20:47
wpwraksdcc ... i wouldn't try that ;-)20:47
kristianpauli will hold my hope for while20:48
kristianpaul:_)20:48
Fallenouvery nice catch about rtems linked lists20:48
Fallenoureally crazy that such errors are still in rtems code base ...20:48
Fallenouand that nobody spotted it20:48
wpwrakyes :)20:48
Fallenoucrazy crazy20:48
kristianpaullinux linux ? :)20:48
wpwrakalso all the macro use issues20:49
wpwrakkristianpaul: indeed. most of this wouldn't survive first review :)20:49
FallenouIs VxWorks better than rtems ? (someone knows ?)20:50
Fallenoubetter in term of code quality, memory footprint, features etc ?20:51
lekernelit's proprietary, no?20:51
FallenouI think yes20:51
kristianpaulyes it is20:51
Fallenoubut I mean, NASA uses RTEMS, I don't think NASA hates proprietary software20:51
FallenouSo I wonder why they use RTEMS and not some proprietary OS20:51
kristianpaulmay be is not NASA it self, may be a thir party contract? but yes is cool see rtems in NASA20:52
Action: Fallenou is hoping a linked list related crash won't make a missile crash into his home20:53
lekernelwpwrak, btw, I've had a good share of linux problems, and I think mwalle and lars_ too. don't imagine linux is a little paradise :)20:53
kristianpaulFallenou: ;)20:53
Fallenoulekernel: you mean linux in general ? or lm32-linux or some-arch-linux ?20:53
lekernellm3220:53
Fallenouok ok20:53
wpwraklekernel: dunno. it's usually been kind to me the last ~20 years :)20:54
wpwraknot that i hadnt't had my moments of despair searching for flaws in linked lists. didn't find any, though :)20:55
lars_thats because your linux code wasn't written by theobroma20:55
wpwrakwho's that ?20:56
Fallenouguys who wrote lm32 port of linux20:57
lars_the inital lm32 port ;)20:57
wpwrakah, i see20:57
Fallenouah yes initial20:57
wpwraklars_: i suppose you and your pitchfork had some fun there ?20:58
lekernelhow's the current code btw? "only" missing 90% of the drivers?20:58
wpwrakand the mmu :)20:59
lars_wpwrak: took me a while to get fork() stable20:59
wpwrakthat sounds ugly20:59
lekernelno other niceties like shared libs not working, random crashes, linker problems etc.?21:00
lars_lekernel: basically yes. there hasn't been much work done lately. starting a job probably wasn't one of my brightest ideas ;)21:01
lars_except for the random crashes, that should be fixed by now21:01
wpwrakan mmu also helps a bit with these things :) particularly the random crashes - by catching these NULL pointers before they mess up things at weird places21:02
lekernelhe, catching NULL pointers should be about 2 lines of verilog21:03
lekernelassuming the gdb system handles bus errors correctly21:03
wpwrakshow us ! (-:C21:03
lars_hehe. adding a watchpoint for 0x00 is usually the first thing i do when i want to debug weird problems on lm3221:04
wpwrakright now, my "null pointer catcher" consists of awatch *(uint32_t *) 0 ... awatch *(uint32_t *) 1221:05
lekernelwe can easily generate a bus error on the first 256kb or so21:05
wpwrakthis gets me at least things those pesky linked lists. it's not good for much else, though21:05
wpwraks/things/things with/21:05
wpwraklars_: i just wish watchpoints could be larger than 4 bytes. or there could be more than 4 of them.21:06
lars_i tink you can use expressions like <= 0x1024, but gdb will fallback to singlestepping when you use it21:07
wpwraklekernel: even the first 4 kB would be great. or special-case a watchpoint at address 0 to cover a larger area. whatever.21:08
lars_but, like lekernel generating a bus error would probably be easier21:08
wpwraksingle-stepping is very un-fun :)21:08
wpwrakof course, it means i don't have to hurry to get my MIDI/mouse hang. but getting to the overflow, with even the most moderate amount of single-stepping (conditional breakpoint), took something like half an hour ...21:09
wpwrakmwalle: btw, any ideas about those stack traces ? :)21:10
mwallewpwrak: mom just had dinner ;)21:11
mwalle(gdb) p *( struct yaffs_dev *) 0x408d85c021:11
mwalleNo struct type named yaffs_dev.21:11
mwallewith the flickernoise binary21:11
mwallearent there the structures embedded into the binary with debug symbols?21:11
wpwrakhmm. checking ...21:11
wpwrakhah, indeed21:13
mwallelars_: the fork bug is fixed?21:15
wpwrakmwalle: seems that yaffs was built without debug symbols :-(21:16
wpwraklet me find something else ...21:16
mwalleok :)21:16
lars_mwalle: at least for me ;)21:17
wpwrakstruct compiler_sc shuold do nicely. 87036 bytes :)21:18
mwalleyep21:25
wpwrakoh. stupid me :) reversed the assertion. freud must have been whispering BUG_ON() ;-)21:28
GitHub179[rtems-yaffs2] sbourdeauducq pushed 1 new commit to master: https://github.com/milkymist/rtems-yaffs2/commit/07a447def19f511fcd229886ec0ee88bf1fa884f21:28
GitHub179[rtems-yaffs2/master] Build with debug symbols - Sebastien Bourdeauducq21:28
wpwrakah, thanks21:29
wpwrakphew. two more stack frames.21:43
wpwraklet's switch to code review. this has worked in the past ...21:56
mwallenaa timing not met22:07
mwallelekernel: is there any other milkymist branch for flickernoise stable?22:18
mwalleother than master, which has the new uart22:19
lekernelyou mean soc stable?22:27
lekernelnot yet, but i'll probably create one soonish with the bus error on null pointers22:27
wpwrakwe don't use _CORE_message_queue_Flush_support (or any of its gazillion aliases) in any way, do we ?22:28
wpwrakor _CORE_message_queue_Broadcast22:28
lekernelno, I don't think so22:29
lekernelat least FN and the MM driver code do not use flush/broadcast22:29
wpwrakgood. i suspect they may all have races. but i feel way too lazy to fix these, too :)22:29
lekernelhaha22:29
wpwrakand they may actually be hard to fix. because the code assumes you can do those things atomically. of course, you CAN but then, you need to do a few other things with interrupts off. such as copying messages.22:32
wpwrakwell, all this in the absence of anything ensuring exclusive access22:34
wpwraklet's see if there's anything ...22:36
mwallemh i guess gdb is completely broken on the master branch :(22:37
wpwraknice :)22:41
wpwrakhmm, are you even supposed to use things like rtems_message_queue_send from an interrupt ?22:46
wpwrakah, and you actually do use rtems_message_queue_flush ;-)22:53
wpwrakbut it's in midi_open. should be harmless22:53
mwallewpwrak: mh instead of submitting a patch i filed a bug ;)22:54
wpwrakor, rather, if there's trouble with it, it will affect the system for its entire lifetime22:54
mwallemaybe i find more time to look into this this weekend or next week22:54
wpwrakmwalle: that was for the stack trace ?22:54
mwallenope the off-by-one length22:55
wpwrakah, so it's not gdbstub's fault ?22:55
mwallewpwrak: no its a gdbstub bug, https://github.com/milkymist/bugs/issues/29 ;)22:57
mwallenot on the gdb bugtracker22:57
wpwrakah, so it's the resolution we talked about. why bug and not patch ? lack of time for testing ?22:58
mwalleyeah, and on master the uart seems to be broken22:59
wpwrakoh, i see22:59
mwallei have to wait for a branch for the stable soc23:00
mwallelekernel: btw with a non autogenerated gdbstub.rom, its impossible to cherry-pick / merge patches between two branches23:01
Action: wpwrak needs a shower after so much RTEMS23:51
--- Thu Nov 10 201100:00

Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!