#qi-hardware IRC log for Monday, 2011-08-22

kristianpaulwpwrak: what was the file you pusblished somwhere.. to update usb id list to qi-hardware related products?02:50
kristianpaulso lsusb is not just the ID and no description..02:50
wpwrakah, lemme see ..02:53
wpwrakhttp://lists.en.qi-hardware.com/pipermail/discussion/2011-February/007210.html02:55
kristianpaulThat one!, thanks02:56
rjeffriesrejon how long will you stay in China?04:11
qi-botThe build has FAILED, see log here: http://fidelio.qi-hardware.com/~xiangfu/compile-log/openwrt-xburst.full_system-08212011-1734/05:26
qi-bot[commit] Xiangfu Liu: work around on python (master) http://qi-hw.com/p/openwrt-packages/0df4b6d06:32
qi-bot[commit] Xiangfu Liu: nanonote speedup powerup, thanks Bas (master) http://qi-hw.com/p/openwrt-packages/0ffd3d006:42
qi-bot[commit] Xiangfu Liu: gmenu2x wallpaper only support .png (master) http://qi-hw.com/p/gmenu2x/46d680907:13
qi-bot[commit] Xiangfu Liu: gmenu2x update (master) http://qi-hw.com/p/openwrt-packages/c609bba07:24
qi-bot[commit] Xiangfu Liu: 4th update to 2.61.2 (master) http://qi-hw.com/p/openwrt-packages/c4d594a07:24
qi-bot[commit] Xiangfu Liu: ben nanonote: forward patches to linux-3.1 (next) http://qi-hw.com/p/openwrt-xburst/2d22f1f08:20
qi-bot[commit] Xiangfu Liu: nanonote mtd.nn: disable kernel log when normal boot (master) http://qi-hw.com/p/openwrt-packages/d95cfe108:52
kyakxiangfu: since you're modifying cmdline, how about adding "consoleblank=30"?09:35
kyakby default, it is 600 (i.e. 10 minutes)09:36
xiangfukyak, yes. sure why not ;-)09:37
qi-bot[commit] Xiangfu Liu: change the console blank to 30 seconds, thanks kyak (master) http://qi-hw.com/p/openwrt-packages/70ce35e09:38
xiangfukyak, ^ :) don't know this option before. thanks.09:39
xiangfukyak, I use the workaround for now. hope I can get a new image recently two days.  09:40
xiangfukyak, I pushed one branch 'next' which I already rebase on last openwrt svn trunk.09:41
xiangfualso I have committed my small work on linux 3.1 , just a start. we will jump to this recently , just after this release .09:42
kyaksounds great :)09:58
kyakand the changes i wanted are all inside09:58
kyakthe problem is however, that ffmpeg is updated upstream and now some packages (like mocp) fail to build09:59
kyakhttps://dev.openwrt.org/changeset/2806309:59
kyakthis is the changeset that causes trouble09:59
kyakperhaps we can override and revert it for this build10:00
kyakand then see how it goes on openwrt side10:00
kyakjow said he was going to talk to the committer :)10:01
xiangfukyak, (mocp failed) yes. notice that so I have modify the feeds.conf to @28054 :)  http://fidelio.qi-hardware.com/~xiangfu/openwrt-xburst.full_system/feeds.conf  stop update upstream packages, :)10:53
xiangfukyak, (jow said he was going to talk to the committer ) great.10:53
wpwrakkristianpaul: just catching up with some really old stuff ...  216.239.32.2: i can ping it but not connect to port 80. neither from home nor via a server in the US.11:13
kristianpaulhe, okay :)11:16
kristianpauli need to remenber was was the porpuse of my question, but first finish breadfast ;)11:17
wpwrakkristianpaul: and the .2 doesn't work because it's .21. can reach .21 without problems from home :)11:18
kristianpaulhe ;)11:21
qi-bot[commit] Xiangfu Liu: nanonote: add 4th entry (master) http://qi-hw.com/p/gmenu2x/9dc513f11:29
`antonio`wpwrak, here's the video http://www.vimeo.com/2792400411:30
qi-bot[commit] Xiangfu Liu: gmenu2x update (master) http://qi-hw.com/p/openwrt-packages/58392ee11:30
`antonio`:) and here's the wiki if you want to know about the project http://theclashingrocks.org/wiki/doku.php?id=tcr11:31
wpwrak`antonio`: (video) ah yes, zedstar mentioned it. very nice ! i can't quite figure out what it shows, but it looks (and sounds) cool ;-)11:31
wpwrak`antonio`: btw, did you solve the duplicate ping response problem ?11:31
`antonio`i minimized it but didn't completely solve it yet 11:34
wpwrak`antonio`: what did you do ti minimize it ?11:37
wpwraks/ti/to/11:37
`antonio`well changed the T_REASS_MS and T_ACK_MS on dirtpan.c and instead of having a real mesh i just connected each device to the coordinator11:41
wpwrakwhat does "real mesh" mean in this context ?11:41
`antonio`i'll work on it again this week and let you know how that goes11:41
`antonio`instead of having all devices connected to each other, i connect each device only to the coordinator 11:42
wpwrakhmm. strange. shouldn't make a difference if there's a connection to an IP you don't use.11:43
`antonio`I know, I will make some more experiments and let you know 11:44
wpwrakkewl :) thanks !11:44
`antonio`:)11:45
qi-bot[commit] Xiangfu Liu: nanonote: add python lua guile icons (master) http://qi-hw.com/p/gmenu2x/41e5f3111:55
qi-bot[commit] Xiangfu Liu: gmenu2x update (master) http://qi-hw.com/p/openwrt-packages/6baf30d11:56
wpwrakDocScrutinizer: by the way, if you like puzzles, here's a nice one (from milkymist rc3): http://downloads.qi-hardware.com/people/adam/m1/pic/rc3_0x77_ch1-TP36_ch2-NOR-pin34-DQ8_500ns.JPG.JPG13:31
wpwrakDocScrutinizer: CH1 shows a line that's normally pulled high with 10 kOhm and bypassed with 220 pF to ground. could be pulled low by an open collector, but we have reason to believe this isn't happening in this case.13:35
wpwrakDocScrutinizer: CH2 shows a quite unrelated line driven by a push-pull output. one thing they have in common is that they're on adjacent balls of the FPGA.13:35
wpwrakDocScrutinizer: when measuring the resistance between the two, we get about 10 kOhm in one direction, about 120 kOhm in the other. the 10 kOhm seems low and if i model this, i get roughly the right curve with similar values: http://downloads.qi-hardware.com/people/werner/m1/tmp/pin34.ps13:37
wpwrakDocScrutinizer: however, what kind of failure would really explain this ? (this is something we've only observed on two out of ~80 boards so far, and the other one may have a higher resistance between the two signals)13:40
DocScrutinizersounds odd13:48
wpwrakDocScrutinizer: to say the least ;-)13:48
DocScrutinizeryou considered clamping diodes when you did your measuring?13:48
wpwraki thought of them but can't quite picture a credible failure scenario that involves them. they could explain any sneak current when measuring in-circuit and with power off, though.13:49
DocScrutinizeryes, sure they do, and it comes out as 10k on one meter, and as 307k on another, and yet another may tell it's 600R13:50
wpwrak;-)13:51
wpwrakbut then, when the system is powered, they shouldn't get in the way13:51
DocScrutinizernever probe outputs (or inputs) for resistance against ground or against another pin without proper voltage on clamping diodes13:52
DocScrutinizeryes13:52
DocScrutinizerexactly13:52
wpwraki wonder if a diode test would yield more useful results. kinda tricky, though. some meters go up to relatively high voltages and i'm not sure adam has a good one.13:52
wpwrakif the system is powered, we can't really measure much resistance anyway. so it's an attempt to make do with what little we have13:53
DocScrutinizerunless your probing voltage is of wrong polarity (ground clamp diode kicks in, 0V7) or of higher voltage than VDD (VDD clamp diode kicks in, result random)13:53
DocScrutinizeranother way to probe such things is with really good meter that has probing voltage <0.3V13:54
DocScrutinizerno non-linear component should spoil your results then - usually13:54
DocScrutinizeryet another way to probe is comparative probing between known good and DUT13:55
DocScrutinizerneeds some experience to read the results of all those probings13:56
DocScrutinizerbasically on a device without VDD applied, each pin is connected to each other pin via 2 diodes in series, no matter which is + and which -13:58
DocScrutinizerat least initially, until your meter has charged the PSU buffer capacitors ;-P13:59
wpwrakyeah ;-)13:59
wpwrakon DUTs, we got about 60/120 kOhm, compared to the 10/120 kOhm on the suspicious device13:59
wpwraks/DUTs/good DUTs/13:59
DocScrutinizerI think we called this reverse alien powering a circuit13:59
wpwrak;-))14:00
DocScrutinizerwell, 60/120 vs 10/120 sounds like chip defect14:01
wpwrakinteresting .. my fluke 8845A goes up to 10 V when measuring resistance (10 MOhm). the picotest m3500a goes up to 5 V. my handheld UT60G -230 mV14:02
DocScrutinizeror a short to some other pin you haven't considered14:02
DocScrutinizeryeah voltage on R test, a way underrated specifications detail of meters14:03
wpwrakthere are no other paths that really seem to make sense. already this one (the adjacent but otherwise unrelated balls) was a lucky guess14:04
DocScrutinizerworst I've seen was some 40V, though on a electron valve heathkit device built for probing up to 100GR14:04
DocScrutinizeror 10GR, can't recall14:04
DocScrutinizerwpwrak: a soldering short rarely acts as diode14:05
wpwrakwow, 40 V not bad. 10 GOhm is even better, though :)14:05
DocScrutinizerso your reading had to be 10/10 rather than 10/12014:06
wpwrakyes, accidental creation of a PN barrier somehow seems improbable :)14:06
DocScrutinizerpossible but unlikely, yes14:06
DocScrutinizeresp on TWO devices14:06
DocScrutinizerthat's when I came up with silicon defect14:07
wpwrakof course, if we can do that just out of thin air, perhaps we should try our luck with cold fusion next ;-)14:07
wpwrakthe boards had some light rework (addition of a 0402 component, replacement of another 0402 component, some wires) in the vicinity of those pads, including on traces leading to them14:08
DocScrutinizerOTOH on-chip defects that create shorts to a random PN structure nearby aren't uncommon14:08
DocScrutinizeroften triggered by ESD14:08
wpwrakif a defect was caused by this rework, what are our chances to see anything useful on x-ray ? likewise, in case of ESD ?14:09
DocScrutinizerzero14:09
wpwraksigh14:09
DocScrutinizerI'd guess14:09
DocScrutinizerI'd not bet a penny on it14:09
wpwrakon xray or on xray being useless ?14:10
DocScrutinizeryou don't see silicon defects of that class in usual optical examination14:10
DocScrutinizerbefore sth shows up in xray even microscopy you need a severe burnout in the die14:10
wolfspraulthe xray is an smt xray, not a foundry xray :-)14:11
wpwraki was wondering is there could perhaps be a zone that looks abnormal, like lighter/darker than the rest or such14:11
DocScrutinizera electromigration-alike defect is usually invisible unless you use electron microscopy I think14:11
wpwrakhmm14:11
wpwrakmaybe we should just fire a few neutrinos at it and check how they get deflected :)14:12
DocScrutinizerlet me put it this way: do your se the structures of a single transistor/diode on your xray? then you got chances to see the defect as well14:12
wpwrakanyone got the number of CERN ? it's about time they do something useful with their LHC :)14:12
DocScrutinizerhehe14:13
wpwrak(see transistors) i wouldn't count on that ;-) probably a few orders of magnitude too low-res14:13
DocScrutinizeryes, that's what I meant14:13
wpwrakare there other symptoms we could probe for that could reveal this sort of damage ? besides the "resistance"14:14
DocScrutinizer1st approach: unsolder the chip and then probe again, without adjacent circuitry14:15
wolfspraulma-tek could do it, but no way we will go in that direction http://www.ma-tek.com/14:15
wpwrakphew. that a ~450 balls FPGA.14:16
wolfsprauljust replace the chip, and/or write off the board14:16
DocScrutinizer:shrug:14:16
DocScrutinizeranyway rework next to those pins smells like ESD damage14:17
DocScrutinizersymptoms are matching14:17
DocScrutinizerI've seen similar symptoms before14:17
wpwrakwolfspraul: let's see if we get the same on the other board. what troubles me is that 0x3c didn't show anything suspicious on the resistance test. 14:18
wpwrakwolfspraul: if he have this sort of failures, we need at least a way to detect them. even if it means the boards in question can't be salvaged.14:19
wpwraks/he/we/14:19
wolfspraulI think Adam can smell them from 10m distance now.14:20
wolfspraulnormally the decisions are made much faster and more radical, just replace/discard, all the way to happiness :-)14:21
wpwrakDocScrutinizer: yes, it all points in that direction. there are a few more things on those nets, but i can't really come up with a good theory how they would cause such effects. not without some very long failure chains. of course, if you assume triple- or quadruple failures even at different points, then anything is possible ;-)14:21
wolfspraulbut of course it's good that we study a bit, a little knowledge never hurts. And Adam can learn a lot too I think.14:21
wolfspraulunless I feel there is something to be learnt for other boards, I feel pretty confident even I know how to 'fix' those boards :-) just throw away and replace with another one :-)14:22
wpwrakwolfspraul: naw, he can't smell these yet. he does occasionally see the boards go crazy, but even that comes and goes. maybe he catches all the afternoon boards but the morning boards are a bit colder and some escape. it's really weird.14:22
wpwrakwolfspraul: yes, if it's a defect in the chip, your fix is the only thing that's even remotely reliable ;-)14:23
wolfspraulI feel we've learnt 98% of what we can learn.14:24
wolfspraulI wouldn't want to take the chips to ma-tek, pay 20,000 USD, and finally have the final ultimate proof that it's an ESD damage because Adam touched the board in a certain way during some rework.14:25
wolfspraulthat would be horrible waste14:25
DocScrutinizerI managed to "repair" such chips with a second OV treatment14:25
wpwrakwolfspraul: on 0x77, it seems we're pretty much exhausted our possibilities, yes14:25
DocScrutinizer;-D14:25
wpwrakwolfspraul: 0x3c still bothers me, though. maybe it's TP36 plus something else than pin 34.14:26
wolfspraullet's see how the fix2b goes on other boards now14:26
wolfspraulI thought we had mostly exhausted what we can learn on 32/3c/77 etc. so far I haven't been proven wrong.14:27
DocScrutinizerI figured what component structure on the die might have been damaged initialy and with wild handwaving came up with a burn out the bug procedure that fused away another component thus "fixing" the short14:27
wolfspraulof course if there would be something it would be very significant, so it's good to have a little paranoia14:27
DocScrutinizeramazingly it actually worked on one of the three chips14:27
DocScrutinizerthose were pretty simple chips though, where you actually could get even structure photos14:28
DocScrutinizerin the very early days of integrated circuits14:28
wpwrakwolfspraul: what i'm looking for is a non-statistical test for a defect there. so far, TP36-pin34 marks 0x77 as a clear outlier. but 0x3c proves that it doesn't show all the outliers. but if we could have a reasonably small set of pins to probe, that would help to catch damaged boards before they fail in the field.14:28
DocScrutinizeron sth like SN74xx series components14:28
wpwrakwolfspraul: also, the other NOR problems we've observed could be linked to this14:29
wolfspraulI don't think any of the boards made it very far14:29
wolfspraulI will look at that again, but not so worried14:29
wolfspraulwe can also make the test a little stronger and catch them that way :-)14:29
wolfspraulit seems to fail quite early14:29
DocScrutinizerare you using JTAG boundary scan?14:30
wpwrakDocScrutinizer: the day wolfgang comes across some money, he'll hire you as the magic ESD healer for the fab ;-)14:30
DocScrutinizerlol, no thanks14:30
wolfspraulwhat drove me crazy before fix2b was boards failing on render cycle 2, 5, 9 etc. that meant even if we would have increase the render cycles to 20, 50, etc. we would have ignored something major (and driven up the fail rate sky high in the process)14:31
wolfspraulthis is all far far better now, all under control14:31
wpwrakwolfspraul: yes, failures that late would be very bad14:32
wolfspraulI think tomorrow Adam should really move 80% or more of his time to fix2b and getting 30 full units ready for sales14:32
DocScrutinizerare you using JTAG boundary scan?14:32
wolfspraulafter that day the time pressure goes away too14:32
wpwrakDocScrutinizer: (boundary scan) i don't think there's any test of that sort14:32
DocScrutinizerrunning "normal" programs and deciding go/no-go isn'T a proper QA14:33
wpwrak(atben/atusb production test has sort of a boundary scan ;-)14:34
DocScrutinizerproper QA consists of putting DUT into pathological operation modes to see how it reacts. The results of these operation modes are not like works/doesn't-work but more like "this error pattern is expected, so the board is ok" vs "duh it refuses to fail, so it might be defect"14:35
wpwrak(not that it actually caught anything - all three defects where outside the scope of these tests. but the tests are there, in all their cryptic beauty :)14:35
wpwrakwolfspraul: let's get the 0x3c signals and measure resistance on a few more pins on it then. that shouldn't take long. then adam can finish the fix2b and testing, and then we'll also see if more boards to decide to join the cluster.14:38
DocScrutinizere.g you reduce VDD to a level where you expect a certain error to occur. If you got flaky output loads the error pattern will be different14:38
wpwrakDocScrutinizer: hmm, that seems tricky. lots and lots of undocumented behaviour.14:38
DocScrutinizerindeed it's undocumented, but you always do your own documentation on a known-good device anyway14:39
DocScrutinizerlots of QA are mere comparative tests between DUT and reference device14:40
wolfspraulagreed, some 3c pins and that's it14:40
wpwrakDocScrutinizer: what i do is pull up/down all pins, then set one to the opposite level. see if all the rest still reads as set. etc.14:40
wolfspraulDocScrutinizer: yes :-) comparative testing, lots and fast14:40
DocScrutinizeryup, when you also precisely log the VDD current etc then that's a good way14:40
wolfsprauland then even a dumb guy like me can make a thumb up/down decision :-)14:41
wpwrakDocScrutinizer: i skipped current this time (atben/atusb). too hard to pull it off (would have had to make a test rig, etc.)14:41
DocScrutinizergood way to follow and do more alike that14:42
DocScrutinizeryes, without a test rig you can not do decent QA14:42
wolfspraulcalling it a day, n814:42
DocScrutinizerthat's the whole purpose of all those test pads after all14:42
wpwrakDocScrutinizer: (current test) of course, with my lab it would have been easy. maybe a simple passive board, but that would have been all. alas, tuxbrain wouldn't have had the instruments to make use of this ...14:42
DocScrutinizernight wolfspraul 14:43
wpwrakwolfspraul: untroubled dreams ! :)14:43
wpwrakhmm, i think i'll see if i can find some food. government sneaked in some public holiday today. the things that happen in election years under a government that sees blatant populism as its strongest card :-(14:47
wpwrakwell, if all else fails, i can still live on crackers and cheese for a week or so ...14:48
wpwrakDocScrutinizer: thanks for your help ! so, no urgent xray session for adam :)14:52
DocScrutinizeryw14:54
DocScrutinizertime for 12648430    14:55
DocScrutinizerhttp://www.thehackernews.com/2011/08/nokia-website-hacked-by-pr0tect0r-aka.html14:55
wpwrakDocScrutinizer: have you been a bad boy ? ;-)14:58
DocScrutinizernah14:58
urandom__why did he hack nokia? for fun?15:21
wpwrakurandom__: maybe to show solidarity with elop ? :)15:30
urandom__wpwrak well he doesn't write anything about solidarity in his message15:49
larscprobably he was just bored15:51
urandom__if he is bored he should do something productive and not hack other peoples websites just to show off how cool he is15:58
DocScrutinizerI appreciate he kicked develper.nokia.com webmins' butts, as that website had several issues with authentication etc before he came16:05
DocScrutinizere.g it dropped a session-end cookie that made subsequent logins always fail16:06
DocScrutinizerdoesn't show up if you use firefox with standard configuration for cookie management :-P16:09
DocScrutinizerfor those users prefering a different browser and/or more sophisticated cookie management, it failed boldly16:11
DocScrutinizerI'd prefer he had found a way to snitch Nokia's root signing key used in OMAP3630 "TPM"16:17
DocScrutinizerbut I guess iven if you find that file, it'd be secured with a passphrase that needs special treatment of Elop to get hold of it16:19
wpwrakDocScrutinizer: competent security ? don't count on it ;-)17:03
rohhm.. john wasnt seen here lately eh?17:39
roh.s17:39
rohmeh17:39
DocScrutinizerhah17:40
DocScrutinizerjust thought of you, while closing a 'mail.om.org: connection timed out' requester17:40
wpwrakroh: isn't it nice when people always think of you in their moments of distress ? you must feel like some superhero, better than superman ;-)17:43
DocScrutinizerwell, I'm almost used to it. This warning requester poping up every 5 min, for the best part of last 2 weeks17:48
DocScrutinizerand each time I have to close it by clicking "OK", just to find it popping up 3min later again. Alas it's a modal requester which sucks big time17:51
DocScrutinizerptobably I should stop polling of that account completely17:53
DocScrutinizer:-/17:53
rohmeh18:12
rohtried re-setting the box again 2 times today but it doesnt look good. maybe we really need a human on the other end now to get it up again18:12
rohjust entered a  verbose 'support-ticket'18:20
rohthe will check the machine for visible damage now and if there is none run some tests18:20
DocScrutinizerroh: is it colocated?19:29
DocScrutinizerroh: ...or Hetzner hw?19:29
wpwrakrented, i think19:33
DocScrutinizerif it's rented then they shall damn replace it for a working iron there19:36
DocScrutinizerWhen I rent a car, I won't accept extended repair times in the garage. If we rented a server, that was meant to be a working one19:37
DocScrutinizerI thought they even guarantee a certain uptime percentage19:39
DocScrutinizerwhich should be better than 95%19:39
qi-botThe build has FAILED, see log here: http://fidelio.qi-hardware.com/~xiangfu/compile-log/openwrt-xburst.full_system-08222011-0827/20:21
rohDocScrutinizer: hetzner hw, really old one20:49
DocScrutinizerroh: I know a Hetzer service guy, if that can help in any way20:56
DocScrutinizerhe offered to move the server on one of his boxes20:56
DocScrutinizerI said "thanks, but no thanks"20:56
DocScrutinizermaybe that was a bit unthought20:57
jow_laptopsounds offending somehow :)20:57
jow_laptophowever hetzner is also the hoster that will write you a mail stating that they move your server across germany tomorrow ...20:58
DocScrutinizerActually I said "ich glaub wir wollen da root rechte behalten"20:58
jow_laptopah I see20:59
rohdas eh.20:59
DocScrutinizerI can ask him to swap the friggin iron21:00
rohthe hw seems just instable and crashing. i guess psu or mainboard are broken/caps burnt or so... i already added a support ticket21:00
DocScrutinizergimme the ticket number, and I'll ping him to have an eye on it21:00
rohalso we seem to have gotten some kind of filtering on the vm master ip from and to the outside, just the vm-subnet was working so i ignored it21:00
rohhm. no clue.. havent gotten the mail for it. need to ask gismo tomorrow.. maybe i should add myself to that list sometime21:02
rohthe machine is #17402 if that helps somehow21:03
DocScrutinizer/queried him21:05
DocScrutinizerroh: anyway thanks for taking care21:08
--- Tue Aug 23 201100:00

Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!