Critical issue with non-conforming USB powered hubs.

[aolofsson]

Hi all,

I was messing around with the board Sunday night and I managed to fry a board! Fortunately, I was then able to repeat the sequence and fry another two boards!

Forbidden operation:
-Remove the 5V DC power connector while keeping the USB peripheral connector (J6) connected to a non-conforming powered USB hub.

Result:
-USB port of the Parallella board gets fried if left on long enough.

Why it happens:
(this is based on the last 24 hrs, so feedback is welcome)
-When the 5V DC connector is removed, the SYS_5P0V signal drops to 3.9V which is just enough to allow the Intersil regulators to stay up.
-Since the rails stay up there is no reset triggered and the EN signal for the bidirectional switch (U13) stays up allowing significant current to flow back from the VBUS to SYS_5P0V keeping the circuit powered.
-The U13 circuit and the rest of this path was not designed to draw 1A+, so eventually the circuit gets fried.

Potential Immediate Workarounds:

RECOMMENDED:
-Connect a powered USB hub and Parallella to a common power strip to power both devices on/off at the same time.(tested)

OTHER OPTIONS:
-Only use un-powered USB hubs(tested)
-Use a conforming USB powered hub that prevents power flowing back to the Parallella board(untested)
-Modify the USB cable to the powered hub to remove the VBUS line(untested)
-Modify a USB cable to put a low forward-voltage-drop diode on USB VBUS line, preventing current from flowing back.(untested)

Long Term Fixes:
-We are working on a bullet proof board fix for to be implemented as soon as possible but we will not hold up production.
-Parallella Kickstarter backers will need to use one of the workarounds.

Any feedback regarding the severity of this issue would be much appreciated, especially if you have experience with similar issues and workarounds from other projects.

Andreas

[tnt]

I have a few follow up questions:

- Did you confirm U13 is the only thing fried ? Fron the schematic I don't see what else would have an issue and U13 is SO8, so easy to replace.

- I assume you mean schottky and not zener ? In any case, it's not ideal because for powered devices this will get a ~0.3v drop min at 500mA, which added to other tolerances (in the cable / in the input / ...) might get below the 4.5v spec at the device.

- What about soldering a schottky from U13 output to it's input. Reverse current flow would go through that path, bypassing U13. Sure the powered device might have an issue with drawing 1A from it but the parallella would be OK and this is a workaround easy enough to do with a bit of soldering experience.

- I'm still a bit surprised it would die ... datasheet for MIC2025 states "Reverse current flow blocking (no “body diode”)" and "Short-circuit protection with thermal shutdown". You'd think either the reverse blocking or thermal shutdown would have prevented it from dying.

[shodruk]

-Modify the USB cable to the powered hub to remove the VBUS line

Is that safe when the hub is not powered?

[aolofsson]

shodruk, You should not remove the VBUS line if you are connecting an un-powered USB hub.
(note that unpowered hubs are completely safe for the Parallella).
Andreas

[aolofsson]

- Did you confirm U13 is the only thing fried ? Fron the schematic I don't see what else would have an issue and U13 is SO8, so easy to replace.

There may be other issues, but we did confirm that the USB on a board came back once we replaced the U13 with a new device. Note that this whole section was not designed to draw 1-1.2A so all bets are off until we have had time to review in depth.

- I assume you mean schottky and not zener ? In any case, it's not ideal because for powered devices this will get a ~0.3v drop min at 500mA, which added to other tolerances (in the cable / in the input / ...) might get below the 4.5v spec at the device.

Yes, thanks! I corrected original post. Agree that this method needs more testing. Saw some people using the technique for other boards.

- What about soldering a schottky from U13 output to it's input. Reverse current flow would go through that path, bypassing U13. Sure the powered device might have an issue with drawing 1A from it but the parallella would be OK and this is a workaround easy enough to do with a bit of soldering experience.

Until further exploration,my assumption is that the Micrel (U13) may not be the only part with an issue in this area (even though the datasheets suggest that they all "should" be fine.

- I'm still a bit surprised it would die ... datasheet for MIC2025 states "Reverse current flow blocking (no “body diode”)" and "Short-circuit protection with thermal shutdown". You'd think either the reverse blocking or thermal shutdown would have prevented it from dying.

Yes, still exploring the exact failure mode, but clearly the Micrel device is not shutting down until it's too late..

[tnt]

Well there isn't that many possibly failure modes for this condition. The MIC2025 only relevant connection are to the VBUS and SYS_5P0V.

VBUS only other connections are caps (which won't do anything), and the VBUS detection through R86 which will limit any current to 5mA, not enough to damage anything. So this leaves only the MIC2025 itself.

Sending voltage from the MIC input through to SYS_5P0V is equivalent to powering the board normally, the only difference is that the copper trace might not be up to support 1/1.2A. However it has to be designed for 500mA, so this is "only" a 100% overload. Sure the trace is going to heat more than what it was designed for, but not enough to burn off or damage the FR4 or anything in the zone.

Upon reading the complete MIC datasheet : "If V_OUT is greater than V_IN, current will flow from OUT to IN since the switch bidirectional when enabled."
So at least it explains why the current flowed. Doesn't explain why the thermal shutdown didn't engage though. Also doesn't explain the 1.1v drop (which would mean like 1 ohm of path resistance, which is fairly high. On the rev0 that trace is 0.05 ohm + 0.15 ohm from rds_on on the MIC2025. Of course things heat and their resistance could increase. The MIC2025 is also under-powered from a lower voltage which might lead to a higher rds_on, all of this in a self-reinforcing cycle (higher rds -> more voltage drop -> more current drawn by the swithing reg -> more heat -> higher rds).

I guess the only way to be sure is buy a bunch of MIC2025 and torture them.

[mlfreeman]

2 questions:

First, is there any way to tell a non-conforming powered hub from a conforming one?
Maybe someone can eventually post a list of recommended makes/models.

Second, what does "long enough" mean? Seconds? Minutes?

[tnt]

Only way to know if it's conforming or not is to measure it. Connect a 100R resistor (or so) between VBUS and GND on the upstream port of the hub and measure voltage across it. Should be 0. Note that you probably need the resistor because even if they have something to prevent reverse current flow, with only the high impedance of the multimeter you could see some ghost voltage in there.

[ticso]

It is amazing what crap is on the market.
A self powered hub feeding uplink is a desaster, not just for parallella.
A lot of systems effectively have USB power directly connected to their supplies.
A hub wiring their supply directly with the uplink supply calls for problems in any way.
The propper way to deal with it is not chopping the VCC on USB-cable, it is to hand over the wannabe HUB to recycling (after axing it to give a clear mark of broken).
I don't think it is unfair to show pictures of this device, so others have a chance to avoid this product and some other rebrands.
Fortunately I've never seen this brokenness myself - at least to my knowledge.

[richardlich]

Since we are the first one to back you up.

I don't think, "-Parallella Kickstarter backers will need to use one of the workarounds." is acceptable.

You should send a good product to backers.