Fix for pco camera crashing/capturing black frames/being stuck#1620
Fix for pco camera crashing/capturing black frames/being stuck#1620Yujin-Bao wants to merge 5 commits into
Conversation
| @property | ||
| def _timeout(self): | ||
| return int(max(2*100*self.GetCycleTime(), 100)) | ||
| # set the time longer for all hardware check |
There was a problem hiding this comment.
This timeout should be specific to the camera (other hardware and hardware checks are completely unimportant here). It's used in the tight loop which just serves the camera. Setting this too long is likely to be undesirable (makes it more likely that the camera buffer is not kept full).
| # Bad buffer -> recycle it, don't deliver | ||
| logger.warning(f"PCO: error {status} during check of buffer {_curr_buf}. Recycling.") | ||
| self._consecutive_errors += 1 | ||
| self._buffers_to_queue.put(_curr_buf) |
There was a problem hiding this comment.
We might not want to re-queue bad buffers unless we are in continuous mode (if we want to match the current logic, i.e.:
For non-continuous modes:
- A single buffer is queued in TriggerAq
- We wait on this buffer, and return once full.
Re-queueing buffers that are bad risks letting the acquisition get out of sync with the triggers (the re-queued buffer might end up waiting on the next trigger)
|
FWIW, I recently used the pco. code as-is (prior to this PR) on an edge 4.2 LT and it worked perfectly. All of the timeouts/transfer issues we experienced ended up being due to the cable connecting the camera to the computer. We tried several off the shelf options that in theory matched the required spec with no success. We finally ordered a far too expensive cable directly from Excelitas and the camera buffering worked. |
|
Thank you @zacsimile ! That's good to know swithing a cable solved the problem. Honestly I'm a bit surprised only their expensive cables work as I also tried troubleshooting some hardware before this PR but wasn't able to completely solve the issue. I updated the pco usb 3.0 driver and the pco firmware and inserted a PCIe board to provide a usb 3.0 controller only for the camera. This solved the crashing issue but there are still some timeouts and black frames. I therefore tried tweeking the software. |
3 participants
Addresses issue of pco edge 4.2 LT used on the OI-DIC microscope.
Is this a bugfix or an enhancement?
This is a bugfix.
Proposed changes:
Recycle buffer when encoutering timeouts or other errors. Flag errors in
pco_sdk_cam.pyforframewrangler.pyto restart the camera. Add a logging to report which hardware is not ready, which is useful for debugging what causes the delay in the single-shot mode. There might be some redundancy in the type of errors might be encountered because I only have the timeout error in my logging based on my test, but I still keep the code as is to be safe.Checklist:
The below is a list of things what will be considered when reviewing PRs. It is not prescriptive, and does not
imply that PRs which touch any of these will be rejected but gives a rough indication of where there is a potential
for hangups (i.e. factors which could turn a 5 min review into a half hour or longer and shunt it to the bottom
of the TODO list).
much simpler if this is kept separate from functional changes. The auto-formatting performed by some editors is particulaly egregious and can lead to files with thousands
of non-functional changes with a few functional changes scattered amoungst them]
If an enhancement (or non-trivial bugfix):