TEM vision software
From MMVLWiki
![]() Prototype using Distributed Ruby (http://www.ruby-doc.org/core/classes/DRb.html) for vision-based closed-loop control |
As part of the Nanorobotics project a TEM vision software was developed. The software makes use of a JEOL 3010 (http://www.jeolusa.com/PRODUCTS/ElectronOptics/TransmissionElectronMicroscopesTEM/300kV/JEM3010/tabid/213/Default.aspx) transmission electron microscope with a TVIPS FastScan-F114 camera (http://tvips.com/Prod_F114.php) which is an IIDC/DCAM-compatible firewire camera. The nano-indenter is controlled by a Nanomagnetics SPM controller (http://web.nanomagnetics-inst.com/product_detail.php?product_link=SPM) (the old version of the controller can be accessed with a PCI-DIO24 card (http://www.mccdaq.com/pci-data-acquisition/PCI-DIO24.aspx)).
The software runs under GNU/Linux and it makes use of Damien Douxchamps' libdc1394 (http://damien.douxchamps.net/ieee1394/libdc1394/) to access the camera and Warren Jasper's PCI-DIO24 driver (ftp://lx10.tx.ncsu.edu/pub/Linux/drivers/) to access the PCI-card which interfaces with the SPM controller.
The software was implemented in Ruby (http://www.ruby-lang.org/) using Qt4-QtRuby (http://rubyforge.org/projects/korundum/), HornetsEye, libJIT (http://dotgnu.org/libjit-doc/libjit.html), and a custom Ruby-extension to access the SPM controller via the PCI-DIO24 card. Distributed Ruby (http://www.ruby-doc.org/core/classes/DRb.html) and multiple processes were used to work around the problem that Ruby-1.8 does not offer native threads.
The vision algorithms are configured using a separate program and the configuration is saved in a file using Ruby marshalling (http://www.ruby-doc.org/core/classes/Marshal.html). A plugin-based architecture, which accepts plugins for recognition and tracking, was implemented which allows one to select and configure Normalised Cross-Correlation, Lucas-Kanade tracking, or Connected Component Analysis.
Table of contents |
Demonstration
Videos
Demonstration of TEM vision software including telemanipulation as well as closed-loop control using machine-vision feedback (also available as DivX3 videos configuration.avi (64 MByte), closed-loop.avi (44 MByte), and interaction.avi (19 MByte)) |
Setup procedure
Download
The software can be downloaded here: visiongui-0.2.tar.bz2 (http://vision.eng.shu.ac.uk/jan/visiongui-0.2.tar.bz2)
The software is implemented in Ruby (http://www.ruby-lang.org/). It uses HornetsEye (current development of version 0.32) and Qt4-QtRuby (http://rubyforge.org/projects/korundum/).
Future Work
Possible future work is
- port to Ruby-1.9 which has native threads
- integrate serial-port interface of JEOL TEM
- access USB-controls for shutter and gain of the TVIPS camera
- feature-based recognition and tracking (less sensitive to brightness changes)
- offset- and gain-compensation for camera image
See Also
External Links
- Hardware
- JEOL 3010 (http://www.jeolusa.com/PRODUCTS/ElectronOptics/TransmissionElectronMicroscopesTEM/300kV/JEM3010/tabid/213/Default.aspx)
- TVIPS FastScan-F114 IIDC/DCAM-compatible firewire camera (http://tvips.com/Prod_F114.php)
- Nanomagnetics SPM controller (http://web.nanomagnetics-inst.com/product_detail.php?product_link=SPM)
- PCI-DIO24 card (http://www.mccdaq.com/pci-data-acquisition/PCI-DIO24.aspx)
- Software
- Ruby (http://www.ruby-lang.org/)
- Warren Jasper's PCI-DIO24 driver (ftp://lx10.tx.ncsu.edu/pub/Linux/drivers/)
- Qt4-QtRuby (http://rubyforge.org/projects/korundum/)
- Related publications
- Jung-Me Park, C. G. Looney, Hui-Chuan Chen: Fast connected component labeling algorithm using a divide and conquer technique (http://cs.ua.edu/research/TechnicalReports/TR-2000-04.pdf), 15th International Conference on Computers and their Applications, March, 2000, pp. 373-6
- J. P. Lewis: Fast Normalized Cross-Correlation (http://www.idiom.com/~zilla/Work/nvisionInterface/nip.html), Industrial Light & Magic
- S. Baker, I. Matthews: Lucas-Kanade 20 Years On: A Unifying Framework (http://www.ri.cmu.edu/pub_files/pub3/baker_simon_2004_1/baker_simon_2004_1.pdf), International Journal of Computer Vision, Vol. 56, No. 3, March, 2004, pp. 221-255.