Here's my thread:
viewtopic.php?f=50&t=2100
Statistics: Posted by rowan194 — Fri Jan 16, 2015 8:23 am
]]>
Sounds as if you have a more consistently fubar-ed board than I do
My bad board would sometimes lock up on the e_rest test so I had a peek at what resetting the Epiphany entails in the library source and although at the time I only had a quick look I did notice it was more involved than you might imagine (well, as I imagined!).
I have got to the point now of stepping back and setting up a more robust development environment, which when I get it up and running I hope will allow me to investigate your "How can I debug the issue?" question in some depth as I too would like to get a better handle on where things are going wrong.
I seem to remember that there are other posts that touch on debugging in some form or other and will probably be trying to track them down myself in a while.
Regards
Ralph
Statistics: Posted by ralphmcardell — Sun Aug 24, 2014 5:07 pm
]]>
On your suggestion, I have disconnected the fan and run the Zync up to 68 degC for a few hours to ensure the board and epiphany chip are both warm and up to temperature.
Still same issue in that any example causes a lockup of the board.
I also notice that running e-reset causes a lockup.
I am running this command from:
/opt/adapteva/esdk.5.13.09.10/tools/host/bin/e-reset
It looks like I can not reset the epiphany chip. I think all of the examples reset the device before running the code, so there is perhaps a reset issue which causes all of the examples to fail and lock up.
Is this a problem in the FPGA code? How can I debug the issue?
Statistics: Posted by mtimms2 — Thu Aug 21, 2014 10:14 pm
]]>
As far as I can tell (just did a quick compare with the bad board and one of the others) my A101040s are the same.
You could give it a go with a different PSU - certainly seems that "your power supply is not up to the job" is the default response to most Parallella problems and seems to have worked for some.
However, I have tried 3 PSUs so far:
- a small PC PSU which offers 5V @ 12A as part of a mini-cluster (4 Parallellas, one network switch)
- the PC PSU as above but only powering the cluster network switch and a single Parallella
- a TDK Lambda 5V 10A PSU powering just one Parallella
(experimented with setting voltages to ~4.76V and ~5.2V as well as ~5.02V)
- and recently, via the barrel connector, one of the Adapteva supplied wall-wart PSUs
And it did not in my case make any difference. So YMMV as they say.
I agree that running at a high temperature is not ideal long term - and in my case tricky to arrange with 3 other boards being cooled to more sensible temperatures in the same case - which is why I gave up in the end. However it might be a useful experiment to see if you can coax better behaviour out of your Parallella when using the Epiphany cores.
Statistics: Posted by ralphmcardell — Fri Aug 15, 2014 4:53 pm
]]>
Maybe the 4A Walmart type adapters are not stable enough a supply? I might try a bench PSU.
I have not tried running at a higher temperature yet, but this seems non ideal long term.
I too think I would have problems with an RMA as I have already had to pay duty once on import to the UK.
Statistics: Posted by mtimms2 — Fri Aug 15, 2014 4:18 pm
]]>
Sorry to hear you also have a board that fails when using the Epiphany cores. On the other hand it seems it is not quite such an isolated incident as just my one board.
Have you tried allowing the board to run warmer as I mention in a previous post:
viewtopic.php?f=50&t=1438&sid=8f1a61350979fc73303884fb460b8ff2#p9488
?
I did manage to get my really dodgy board to execute tests for just over 24 hours by keeping the temperature reported for the Zynq chip above 65 degrees C but mostly below 70 degrees C (I think it spiked briefly at 73 degrees in the warm late afternoon before I could adjust the airflow!).
It still locked up eventually however. My brother,who many years ago used to work repairing micro computers, mentioned that in his experience when a board works better when warm it usually indicated a bad joint which became less bad (as it were) under heat expansion. If this is the case then who knows if such a fault occurred at manufacture, transit or installation. On the other hand it could be some other fault - maybe signalling or timing. No one else, you might have noticed, has come forward to offer any possibilities and things that might be looked into to help track down the problem
I have now removed this board from my mini-cluster and am in the process of replacing it with a new 7010 based P1601 (its the 2nd - the 1st lasted about 24 hours before being dead and was replaced by RS - I am doing burn-in tests on the replacement before installing it in the cluster so if it fails I will not have made any changes such as bridging J15 for power via mounting pads).
The bad 7020 A101040 board I will use for non-Epiphany development and experiments - learning a bit about FPGA programming would seem an appropriate use. I did not try to RMA this boards as firstly I would have to get it back to Adapteva in the USA from the UK, and secondly in trying to sort out what was wrong I blew the 4A fuse on the board while trying to check the 5V test point (which is surrounded by grounded things) and had to bridge the fuse - not as expertly as some I just made a solder bridge.
However, all my boards, including the new 7010 P1601 incumbent, exhibit the 'soft' failures with matmul-16 and fft2d. The new board is being powered during 'acceptance testing' via the barrel connector using one of the wall-wart power supplies supplied by Adapteva as part of my mini-cluster reward - maybe it will improve when I connect it to the 5V, 12A supply shared by all mini cluster boards and their network switch, but I'll not be holding my breath...
Statistics: Posted by ralphmcardell — Fri Aug 15, 2014 3:44 pm
]]>
S/N 0001717
Everytime I run the Epiphany or Parallella examples, approximately one second after starting the code, the system locks up on both serial port, XWindows terminal or ssh connection. The lockup occurs whenever the code tries to interact with the Adapteva processor. I have tried running the examples with X Windows disabled and see the same result.
I have loaded the current Linux firmware and have tried both the 7020 HDMI and headless Zync image. I wonder if the 7020 FPGA code is missing any fixed which are included in the 7010 version??
I have a fan with airflow across the width of the board which keeps both the Adapteva and Zync chips at a low temperature. I am using a 5V 4A supply connected to the barrel connector, so there should be enough supply current to ensure that there are no voltage drops across the board whilst the Adapteva cores are being loaded with code.
When only using the Zync chip, the system and OS is stable and X Windows, Linux and other applications work ok. My intention however was to to use the Parallella for a project that requires full use of the 16 Adapteva parallel cores.
Statistics: Posted by mtimms2 — Fri Aug 15, 2014 2:16 pm
]]>
Hello all,
I am still having problems with my mini-cluster of Parallellas, however I have made some progress.
To make sure that power was not the issue I have been powering Parallellas one board at a time and have also tried a different PSU to power a single Parallella - a TDK Lambda 5V/10A affair - with no change in stability for any of the boards.
Finally I acted on a vague observation that the most stable runs seemed to be on very hot days (for the UK!) in the mid to late afternoon when it was hottest (note: Parallellas are in a quite small room with only a door and windows for air conditioning).
I let a board get hot by restricting the air flow around it. I have so far concentrated on the board that locks up the system (or possibly just crashes the network) when it fails as this is the more serious problem.
I find that if I let the board warm up so that the ztemp.sh script reports temperatures above 60 degrees C (but below the 70 degrees C recommended maximum) then this board magically starts to repeatedly pass all the Epiphany tests listed in the LIST.E16 file run by /home/linaro/epiphany-examples/scripts/TestEpiphany.pl.
I have been running this board hot over that last few days - and cooler to get an idea of the range of temperatures involved). Today I have been running the board at 64 degrees C to 67 degrees C for over four and a half hours and 265+ iterations of the tests with no lock ups and no failures or interruptible getting stuck incidents. Note that I had no luck with an earlier setup that cooled the Zynq chip to ~57 degrees C.
I am not convinced it is the Zynq chip that needs to be this hot nor necessarily the Epiphany chip, but might well be some other component or components that of course get warm as well.
Any ideas as to what could cause such behaviour - i.e. Running Epiphany tests and examples causes board to fail unless something - or somethings - are warm enough - would be very much appreciated. As would any suggestions as to other things I could check such as voltage levels or frequencies (I have a multi-meter and an old analogue CRT 20MHz oscilloscope available).
Regards
Ralph
Statistics: Posted by ralphmcardell — Tue Jul 22, 2014 2:36 pm
]]>
]]>
]]>
]]>
]]>
]]>
]]>
]]>