Difference between revisions of "Hacked Optical Mouse"

From Hackteria Wiki
Jump to: navigation, search
 
(42 intermediate revisions by 2 users not shown)
Line 1: Line 1:
http://www.dusseiller.ch/labs/?p=998
+
== Optical Mouse Sensor ==
 
 
http://www.dusseiller.ch/labs/?p=950
 
 
 
http://www.dusseiller.ch/mis_wiki/index.php?title=%CE%9C-wetPONG#Projekt_1_.7C_Mouse_Flowers
 
 
 
=== Optical Mouse Sensor ===
 
  
 
''Can we detect and measure flow in microchannels with that?''
 
''Can we detect and measure flow in microchannels with that?''
Line 11: Line 5:
 
[[File:optical_mouse_sensor.png|none|180px]]
 
[[File:optical_mouse_sensor.png|none|180px]]
  
[http://www.bidouille.org/hack/mousecam/index.en.php Optical Mouse Cam] Bidouille.org | Electronics for the people
+
this post explains how to hack an optical mouse, using an [http://www.arduino.cc arduino] to connect directly to the Avago optical flow sensor, ADNS-2051, [[File:ADNS2051.pdf]]. It includes a java program to see the video output of the optical mouse sensor, the program needs a special rxtx library for serial communication: [http://users.frii.com/jarvi/rxtx/download.html RXTX]
  
this post explains how to hack an optical mouse, using an [http://www.arduino.cc arduino] to connect directly to the Avago optical flow sensor, ADNS-2051, [[File:ADNS2051.pdf]]. It includes a java program to see the video output of the optical mouse sensor, the program needs a special rxtx library for serial communication: [http://users.frii.com/jarvi/rxtx/download.html RXTX]
 
  
[http://www.techbriefs.com/component/content/216?task=view Insect-Inspired Optical-Flow Navigation Sensors] NASA techbriefs
+
=== more info ===
  
This NASA paper describes how the optical mouse sensor can be used to measure the optical flow and be used for navigation of flying robots. [[File:NASA-techbrief_optical_mouse.pdf]]
+
[[File:MouseAndChipCoordinates.png|none|350px|thumb|chip in the mouse]]
  
 +
[[File:OpticalAssembly.png|none|350px|thumb|optical assembly]]
  
 
=== different chips ===
 
=== different chips ===
  
====Optical Mouse====
+
[[File:Avagochip.jpg|100px|thumb|Optischer Sensor einer Maus Avago 2051]]
 
 
[[File:Avagochip.jpg|100px|thumb|Optischer Sensor einer Maus Avago 2610]][[File:Optical_mouse_sensor.png|100px|thumb|Optischer Maus Sensor]]
 
  
 
Verwendet wird eine alte optische Maus USB/PS2 hierbei ist es egal ob sie defekte Schalter oder Scrollräder hat. Wichtig ist nur, dass der optische Chip noch funktionstüchtig ist.  Dieser wird als einzigstes Bauteil benötigt. Der optische Chip sollte vom Typ '''PAN3101, ADNS-2610, ADNS-2083''' oder '''ADNS-2051''' sein. Die meisten handelsüblichen optischen Computer Mäuse haben so eine Chip. Dieser Chip hat etwa 18x18 Pixel die wir auslesen wollen und auf dem Display des Computers darstellen. Die Werte delta X und delta Y die für eine Fluss Messung nötig sind werden on-Chip berechnet auch die wollen wir auslesen um die Fluss Geschwindigkeit zu ermitteln.
 
Verwendet wird eine alte optische Maus USB/PS2 hierbei ist es egal ob sie defekte Schalter oder Scrollräder hat. Wichtig ist nur, dass der optische Chip noch funktionstüchtig ist.  Dieser wird als einzigstes Bauteil benötigt. Der optische Chip sollte vom Typ '''PAN3101, ADNS-2610, ADNS-2083''' oder '''ADNS-2051''' sein. Die meisten handelsüblichen optischen Computer Mäuse haben so eine Chip. Dieser Chip hat etwa 18x18 Pixel die wir auslesen wollen und auf dem Display des Computers darstellen. Die Werte delta X und delta Y die für eine Fluss Messung nötig sind werden on-Chip berechnet auch die wollen wir auslesen um die Fluss Geschwindigkeit zu ermitteln.
Line 30: Line 22:
  
 
Um den verwendeten Chip in die gesamte Produktionspallete einzuordnen, vergleicht man die Seriennummer mit denen auf dem [http://www.avagotech.com/pages/en/navigation_interface_devices/navigation_sensors/led-based_sensors Datasheet der Avago Chips]
 
Um den verwendeten Chip in die gesamte Produktionspallete einzuordnen, vergleicht man die Seriennummer mit denen auf dem [http://www.avagotech.com/pages/en/navigation_interface_devices/navigation_sensors/led-based_sensors Datasheet der Avago Chips]
 +
 +
[[File:Pinout.png|100px|thumb|PAN 3101 Chip Pins]]
 +
Der Avago Chip muss nun über 4 Leitungen mit der Arduino Plattform verbunden werden.
 +
 +
+5V auf Pin +5V bei Power Pins von Arduino
 +
 +
GND auf Pin GND bei Power Pins von Arduino
 +
 +
SDIO auf Pin 3 von Arduino
 +
 +
SCLK auf Pin 4 von Arduino
 +
 +
Dazu kann mit einfachen kleinen Kabeln und Lötkolben eine Verbindung Gelötet werden. Um Störungen über den Maus eigenen Controller auf dem Optischen Chip zu unterbinden, können die Leitungen auf dem PCB von SCLK und SDIO durch kratzen mit einem Cutter getrennt werden.
  
 
==== Serial comm ====
 
==== Serial comm ====
 +
 +
PAN-3101
  
 
PAN-3102
 
PAN-3102
 +
[[File:PAN3101.pdf]]
  
 
ADNS-2051
 
ADNS-2051
 +
[[File:ADNS2051.pdf]]
  
 
ADNS-2610
 
ADNS-2610
 +
[[File:ADNS2610.pdf]]
  
 
==== USB comm ====
 
==== USB comm ====
  
[www.yaxun-cn.com/UploadFiles/201067152243925.pdf C2165]
+
[http://www.yaxun-cn.com/UploadFiles/201067152243925.pdf C2165]
 +
 
 +
==== Chip Data sheets ====
 +
 
 +
[[File:OverviewMouseChips.png|none|350px|thumb|none|mouse chip photo sensor array]]
 +
 
 +
Check the number of pins and the hole pattern to recognize your chip.
 +
 
 +
 
 +
[http://hackteria.org/wiki/images/6/62/ChipsOverview.pdf Chips Overview in PDF]
 +
 
 +
 
 +
Download a collection of data sheets here:
 +
http://hackteria.org/wp-content/uploads/2010/12/MouseChipDatasheets.zip
 +
 
 +
=== Acknowledgements ===
 +
 
 +
[http://www.bidouille.org/hack/mousecam/index.en.php Optical Mouse Cam] Bidouille.org | Electronics for the people
 +
 
 +
[http://www.martijnthe.nl/2009/07/interfacing-an-optical-mouse-sensor-to-your-arduino/ Martijn Thé's Tutorial] Connecting a mouse to an arduino
 +
 
 +
[http://spritesmods.com/?art=mouseeye Ghetto Scanner] by SpritesMods
 +
 
 +
[http://www.techbriefs.com/component/content/216?task=view Insect-Inspired Optical-Flow Navigation Sensors] NASA techbriefs
 +
 
 +
This NASA paper describes how the optical mouse sensor can be used to measure the optical flow and be used for navigation of flying robots. [[File:NASA-techbrief_optical_mouse.pdf]]
  
 
=== First experiments by dusjagr ===
 
=== First experiments by dusjagr ===
  
Ihr findet einige infos zu meinen kürzlichen experimenten auf meiner website:
+
Just done further experiments with the hacked optical mouse sensor, in this case a Avago ADNS2610, from a cheap wired logitec optical mouse. So i managed to connect the optical sensor to the arduino and send all the data, including the image from the 18×18pixel sensor, through a serial connection to puredata (pd). The sensor is a real high-end device, it calculates the optical flow, thus gets out motion and the speed of the mouse, it detects focus by a SQL-value, which is proportional to the number of features the sensor detects on a surface. and other values such as maximum and average pixel and shutter speed can be read from the registers.
 +
 
 +
More stuff on my other website:
 +
 
 +
http://www.dusseiller.ch/labs/?p=998
 +
 
 +
http://www.dusseiller.ch/labs/?p=950
  
 
http://www.dusseiller.ch/labs/?p=941
 
http://www.dusseiller.ch/labs/?p=941
  
{{#widget:Vimeo|id=11785980}}
+
Stuff I have tried with my students:
 +
 
 +
http://www.dusseiller.ch/mis_wiki/index.php?title=%CE%9C-wetPONG#Projekt_1_.7C_Mouse_Flowers
 +
 
 +
=== experiments by Gaudi ===
 +
 
 +
[[File:MouseSensor.jpg|none|350px|thumb|none|mouse chip photo sensor array]]
 +
 
 +
[[File:MouseSensorChip.jpg|none|350px|thumb|disassembling the chip]]
 +
 
 +
High magnification:
 +
[[File:MouseAsAMicroscope.JPG|none|350px|thumb|Mouse as a microscope]]
 +
 
 +
Lower magnification:
 +
[[File:MouseAsAWebCam.JPG|none|350px|thumb|Mouse as a web cam]]
 +
 
 +
 
 +
==== Interface Application for Mac OSX ====
 +
 
 +
[[File:MacMouseHack_ScreenShot.jpg|none|300px|thumb|alt text]]
 +
 
 +
Cocoa application hack for interfacing a mouse sensor with a Mac. The mouse chip is connected to an arduino board and read out through the serial port.
 +
 
 +
The compiled application and the c code (to be programmed on the arduino):
 +
 
 +
http://hackteria.org/wp-content/uploads/2010/12/MouseHackTest.zip
 +
 
 +
Video here:
 +
http://www.youtube.com/watch?v=CAqWehKD_z0&feature=youtube_gdata_player
 +
 
 +
<br style="clear: both" />
 +
 
 +
== Detection of Microorganisms ==
 +
 
 +
[[File:blepharisma_overview.jpg|600px]]
 +
 
 +
The frame rate of the transmitted image is sadly relatively low, cos the ADNS chip is not meant to present this info at higher speed. still its reasonable, when reading all 18×18 pixel a framerate of roughly 5 fps is feasible, if you only dump the first couple of lines you can increase it to higher framerates.
 +
 
 +
After building a patch to visualize all these data from pd, i started experimenting with placing microorganisms on top of the sensor, keeping the optics intact, but replacing the LED with an RBG, so i can control illumination.
 +
 
 +
I got some really amazing results, it seems i can detect single cell microorganisms using just the mouse sensor and an arduino. see the movie below. while the sensor only detects motion if the whole image is shifted, it seems that when a microorganism (Blepharisma Japonicum) passes the field of view, there is a substantial change in the SQL-value.
 +
 
 +
[[File:worm_frames_full.jpg|800px]]
 +
 
 +
Later i used the hacked optical mouse to detect the motion of nematodes, anguila aceti, and got some really clear results, both in the quality of the image and a substantial change in the SQL value. due to some grabbing issues when recording the screen the values are a bit delayed in the image above.
 +
 
 +
[http://www.dusseiller.ch/labs/?p=950 see movie]
 +
 
 +
=== DIY simple microfluidic device ===
 +
 
 +
to make sure that the fluid containing the microorganisms are always in the right focal plane a simple microfluidic device was made by using two pieces of glass coverslip with some double sided adhesive tape in between. the channels can easily be filled by capillary action. they tend to dry out quite quickly though, but by regular refilling the concentration of nematode even increases.
 +
 
 +
== Worm is a VJ ==
 +
 
 +
[[File:worm_is_a_VJ_blue.jpg|320px]]
 +
 
 +
[[File:worm_is_a_VJ_inabox.jpg|320px]]
 +
 
 +
== Workshop at H.A.C.K ==
 +
 
 +
Tuesday 7.12.2010, [http://hspbp.org/tiki-index.php H.A.C.K.] Budapest
  
{{#widget:Vimeo|id=11607692}}
+
'''Participants'''

Latest revision as of 16:35, 6 December 2010

Optical Mouse Sensor

Can we detect and measure flow in microchannels with that?

Optical mouse sensor.png

this post explains how to hack an optical mouse, using an arduino to connect directly to the Avago optical flow sensor, ADNS-2051, File:ADNS2051.pdf. It includes a java program to see the video output of the optical mouse sensor, the program needs a special rxtx library for serial communication: RXTX


more info

chip in the mouse
optical assembly

different chips

Optischer Sensor einer Maus Avago 2051

Verwendet wird eine alte optische Maus USB/PS2 hierbei ist es egal ob sie defekte Schalter oder Scrollräder hat. Wichtig ist nur, dass der optische Chip noch funktionstüchtig ist. Dieser wird als einzigstes Bauteil benötigt. Der optische Chip sollte vom Typ PAN3101, ADNS-2610, ADNS-2083 oder ADNS-2051 sein. Die meisten handelsüblichen optischen Computer Mäuse haben so eine Chip. Dieser Chip hat etwa 18x18 Pixel die wir auslesen wollen und auf dem Display des Computers darstellen. Die Werte delta X und delta Y die für eine Fluss Messung nötig sind werden on-Chip berechnet auch die wollen wir auslesen um die Fluss Geschwindigkeit zu ermitteln.


Um den verwendeten Chip in die gesamte Produktionspallete einzuordnen, vergleicht man die Seriennummer mit denen auf dem Datasheet der Avago Chips

PAN 3101 Chip Pins

Der Avago Chip muss nun über 4 Leitungen mit der Arduino Plattform verbunden werden.

+5V auf Pin +5V bei Power Pins von Arduino

GND auf Pin GND bei Power Pins von Arduino

SDIO auf Pin 3 von Arduino

SCLK auf Pin 4 von Arduino

Dazu kann mit einfachen kleinen Kabeln und Lötkolben eine Verbindung Gelötet werden. Um Störungen über den Maus eigenen Controller auf dem Optischen Chip zu unterbinden, können die Leitungen auf dem PCB von SCLK und SDIO durch kratzen mit einem Cutter getrennt werden.

Serial comm

PAN-3101

PAN-3102 File:PAN3101.pdf

ADNS-2051 File:ADNS2051.pdf

ADNS-2610 File:ADNS2610.pdf

USB comm

C2165

Chip Data sheets

mouse chip photo sensor array

Check the number of pins and the hole pattern to recognize your chip.


Chips Overview in PDF


Download a collection of data sheets here: http://hackteria.org/wp-content/uploads/2010/12/MouseChipDatasheets.zip

Acknowledgements

Optical Mouse Cam Bidouille.org | Electronics for the people

Martijn Thé's Tutorial Connecting a mouse to an arduino

Ghetto Scanner by SpritesMods

Insect-Inspired Optical-Flow Navigation Sensors NASA techbriefs

This NASA paper describes how the optical mouse sensor can be used to measure the optical flow and be used for navigation of flying robots. File:NASA-techbrief optical mouse.pdf

First experiments by dusjagr

Just done further experiments with the hacked optical mouse sensor, in this case a Avago ADNS2610, from a cheap wired logitec optical mouse. So i managed to connect the optical sensor to the arduino and send all the data, including the image from the 18×18pixel sensor, through a serial connection to puredata (pd). The sensor is a real high-end device, it calculates the optical flow, thus gets out motion and the speed of the mouse, it detects focus by a SQL-value, which is proportional to the number of features the sensor detects on a surface. and other values such as maximum and average pixel and shutter speed can be read from the registers.

More stuff on my other website:

http://www.dusseiller.ch/labs/?p=998

http://www.dusseiller.ch/labs/?p=950

http://www.dusseiller.ch/labs/?p=941

Stuff I have tried with my students:

http://www.dusseiller.ch/mis_wiki/index.php?title=%CE%9C-wetPONG#Projekt_1_.7C_Mouse_Flowers

experiments by Gaudi

mouse chip photo sensor array
disassembling the chip

High magnification:

Mouse as a microscope

Lower magnification:

Mouse as a web cam


Interface Application for Mac OSX

alt text

Cocoa application hack for interfacing a mouse sensor with a Mac. The mouse chip is connected to an arduino board and read out through the serial port.

The compiled application and the c code (to be programmed on the arduino):

http://hackteria.org/wp-content/uploads/2010/12/MouseHackTest.zip

Video here: http://www.youtube.com/watch?v=CAqWehKD_z0&feature=youtube_gdata_player


Detection of Microorganisms

Blepharisma overview.jpg

The frame rate of the transmitted image is sadly relatively low, cos the ADNS chip is not meant to present this info at higher speed. still its reasonable, when reading all 18×18 pixel a framerate of roughly 5 fps is feasible, if you only dump the first couple of lines you can increase it to higher framerates.

After building a patch to visualize all these data from pd, i started experimenting with placing microorganisms on top of the sensor, keeping the optics intact, but replacing the LED with an RBG, so i can control illumination.

I got some really amazing results, it seems i can detect single cell microorganisms using just the mouse sensor and an arduino. see the movie below. while the sensor only detects motion if the whole image is shifted, it seems that when a microorganism (Blepharisma Japonicum) passes the field of view, there is a substantial change in the SQL-value.

Worm frames full.jpg

Later i used the hacked optical mouse to detect the motion of nematodes, anguila aceti, and got some really clear results, both in the quality of the image and a substantial change in the SQL value. due to some grabbing issues when recording the screen the values are a bit delayed in the image above.

see movie

DIY simple microfluidic device

to make sure that the fluid containing the microorganisms are always in the right focal plane a simple microfluidic device was made by using two pieces of glass coverslip with some double sided adhesive tape in between. the channels can easily be filled by capillary action. they tend to dry out quite quickly though, but by regular refilling the concentration of nematode even increases.

Worm is a VJ

Worm is a VJ blue.jpg

Worm is a VJ inabox.jpg

Workshop at H.A.C.K

Tuesday 7.12.2010, H.A.C.K. Budapest

Participants