Skip to content

raster scan camera shim#1551

Draft
barentine wants to merge 37 commits into
python-microscopy:masterfrom
barentine:rastercam
Draft

raster scan camera shim#1551
barentine wants to merge 37 commits into
python-microscopy:masterfrom
barentine:rastercam

Conversation

@barentine
Copy link
Copy Markdown
Member

Along the lines of #1463 , I am currently working on a system with both a camera and a point detector. Ideally, I would like to be able to shim a stage-scanned image in as a Camera, so I can acquire images with either "camera" and/or run one from another instance for simultaneous imaging.

What I'm draft PR'ing for discussion here is a close-to-minimum working example for the approach. In my init script, I'm wrapping each axis of my 3-axis stage in a base_piezo.SingleAxisWrapper and then a offsetPiezoREST.TargetOwningOffsetPiezo so that I can leave the global microscope position the same, and scan about that position in a RasterscanCameraShim.scan call.

At the moment I've tried to set things up to accept a variety of "axes" to scan. Generally, I like the idea of using state handlers so we can scan more than just positions (i.e. scanning positions of a filter wheel would also be viable).

Is this a bugfix or an enhancement?
enhancement
Proposed changes:

  • add voxelsignalprovider base class, a test version which simply records the scan positions at each location, and a dual-phase lock in version which logs both the X and Y channels at each scan position
  • add a StageScanner class, with the idea of having it on somewhat equal footing with a mirror scanning approach at a later date
  • add a RasterScanCameraShim class to mimic a camera while actually raster-scanning a point detector

My init script looks like the following:

init script
    from PYME.Acquire.Hardware.Piezos import piezo_pipython_gcs, base_piezo
    from PYME.Acquire.Hardware.Piezos.offsetPiezoREST import TargetOwningOffsetPiezo

    nanocube_desc = 'PI E-727 Controller SN 0121089077'
    scope.stage = piezo_pipython_gcs.GCSPiezoThreaded(nanocube_desc, axes=('1', '2', '3'))
    x = base_piezo.SingleAxisWrapper(scope.stage, 0)
    x = TargetOwningOffsetPiezo(x)
    y = base_piezo.SingleAxisWrapper(scope.stage, 2)
    y = TargetOwningOffsetPiezo(y)
    z = base_piezo.SingleAxisWrapper(scope.stage, 1)
    z = TargetOwningOffsetPiezo(z)

    scope.register_piezo(x, 'x', needCamRestart=False, multiplier=-1)
    scope.register_piezo(y, 'y', needCamRestart=False, multiplier=1)
    scope.register_piezo(z, 'z', needCamRestart=False)
    scope.pifoc = z

    from PYME.Acquire.Hardware import raster_scan_shim
    scope.raster_scanner = raster_scan_shim.StageScanner(scope.positioning)
    
    from pymeasure.adapters import VISAAdapter
    from pymeasure.instruments.srs import sr844
    adapter = VISAAdapter("GPIB0::8::INSTR", write_termination='\n')
    scope.lockin = sr844.SR844(adapter=adapter)
    scope.lockin.reference_source = 'External'
    scope.lockin.time_constant = 0.1
    scope.lockin.sensitivity = 300e-6  # [V]
    scope.lockin.phase = 0

    scope.lockin_voxel_provider = raster_scan_shim.LockInSignalProvider(scope.lockin)
    scope.raster_cam = raster_scan_shim.RasterscanCameraShim(scope.raster_scanner, scope.lockin_voxel_provider,
                                        n_pixels_x=9, n_pixels_y=9,
                                        pixel_size_x=0.2, pixel_size_y=0.2)  # [um]

@David-Baddeley
Copy link
Copy Markdown
Contributor

Hi Andrew,

I'm really sorry I didn't respond to the original issue in a timely fashion - it came in when I was on holiday and then dropped out of my attention ... I'll follow up with more detail shortly (probably on the original issue), but the main thrust of my thoughts are as follows:

  • you'll hit the limitations of pure-software point-scanning pretty quickly, and it would be good to structure things so that the architecture continues to work even when you have more deterministic scanning (the simplest here would likely be to have position command voltages and sensor reads running from a constant clock in an DAC card, arduino, or something else with deterministic timing). It's very likely in this case that you would want to buffer an entire frame on the ADC card and read that buffer in one go.
  • my gut feeling is that going through the scope.state mechanism might have some undesired wrinkles (notably automatic camera restarting when changing some state variables)
  • there seems to be some duplication in the motion logic with the existing point_scanner class currently used for tiling (this was originally written to do half-assed confocal using a small ROI on camera and summing the pixels).

Best wishes,
David

Comment thread PYME/Acquire/Hardware/raster_scan_camera.py Outdated
@barentine
Copy link
Copy Markdown
Member Author

Working towards a more sane/complete implementation. As it stands now the BaseScanner class outlines an interface for a mirror/stage scanner to hold a buffer of frames, and a PointscanCamera class acts like a ~normal camera class pulling frames from the BaseScanner instead of e.g. a camera FPGA.
There are two init_sim_X.py scripts in a pointscan_shim folder for testing/debugging.

Followed your suggestions, @David-Baddeley , for multichannel, signaling 1 frame for each channel to be read in by the frameWrangler. The Preview in PYMEAcquire is a bit odd, but for now this is a really nice improvement for me to be able to use the spooling/saving framework.

Image

I modified PYME/Acquire/SpoolController.py and protocol_acquisition.py to check for the number of channels of the PointscanCamera and mofidy the shape accordingly. If there's one thing to review early / provide some feedback on, @David-Baddeley , it'd be nice to know how bad I hacked it there ;)

Comment thread PYME/Acquire/SpoolController.py Outdated
elif self.spoolType == 'File':
backend_kwargs['complevel'] = settings.get('hdf_compression_level', self.hdf_compression_level)

# set dimensions and shape
Copy link
Copy Markdown
Contributor

@David-Baddeley David-Baddeley Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be revisited to make sure it plays well with XYZTCAcquisition and the like. XYZTCAcquisition creates it's backend with backend(..., shape=shape, dim_order=dim_order, **backend_kwargs) meaning that you will get repeated dim_order and shape keyword arguments to the backend and an error.

@David-Baddeley
Copy link
Copy Markdown
Contributor

@barentine - as noted above, SpoolController changes will need revisiting. Need to think a bit about best way to handle this, but is likely to depend on your exact requirements, specifically whether your colour acquisitions need to be ProtocolAcquisition instances (or subclasses), or if they would be better as an XYZTCAcquisition subclass. To help answer this there are 2 questions:

  • Do you know the sequence length in advance?
  • Do you need the protocol features (e.g. modifying laser powers during acquisition etc ...)?

If the answers are yes and no respectively it would likely be better to use XYZTCAcquisition as a base class. Using ProtocolAcquisition might also bring some interesting complexity linking events to frame numbers with the emitting multiple frames paradigm.

@David-Baddeley
Copy link
Copy Markdown
Contributor

If you still think that ProtocolAcquisition is the correct acquisition type, a short term workaround (although possibly still not a final one) would be to move the logic you have in SpoolController into ProtocolAcquisition. This would avoid the XYZTC breakage, but would not, e.g., let you do colour z stacks.

@barentine
Copy link
Copy Markdown
Member Author

Really appreciate the help, @David-Baddeley !

I don't have the best overview of the acquisition backends, so I appreciate your patience.

In answer to your questions,

  • at some point it would be nice to image until bleach without knowing ahead of time how long that will be. This is a less immediate need, and possibly not urgent to have undetermined series lengths at the same time as Z stepping
  • I would like to use protocols for laser shuttering/unshuttering and possibly triggering an acquisition on an ~overview camera in a separate instance

It seems like maybe a reasonable thing to do is move logic out of SpoolController into ProtocolAcquisition, at least for short term, because I'm comfortable with protocols and might want to use them.

I think I see now how to register my own acquisition type for the subclass XYZTCAcquisition approach (will try and remember to call some of this out in configuring PYMEAcquire doc at some point). That said, it seems odd for me to subclass just to use an argument, channel_settings that already exists in XYZTCAcquisition, but as far as I can tell doesn't get used anywhere. How/where should one be presenting channel_settings so they'll get used appropriately?

@David-Baddeley
Copy link
Copy Markdown
Contributor

For now channel_settings is just a stub, but we should really flesh it out a bit. It's stubbed in the base class to allow the maximum flexibility for how channels could be implemented, and I haven't put any effort into fleshing it out so far. There are at least 4 different legitimate ways to implement multi-channel acquisitions:

  • Monochrome camera + filter wheel and/or control of lasers / LEDs
  • Colour camera
  • A combination of the above two
  • Something else entirely (e.g. OIDIC phase shifts - if you still have access to the BewersdorfLab github, there is an example XYZTC subclass in the pyme-oidic repo).

In the reasonably short term we should probably add a channel implementation to the default XYZTC acquisition, but maybe by making a subclass the default that gets used for z stacks etc rather than modifying the base.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants