Difference between revisions of "UBISS2024"

From Sketching with Hardware at LMU Wiki
Jump to navigation Jump to search
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Random Links =
 
https://docs.arduino.cc/micropython/basics/board-examples/
 
https://www.sketching-with-hardware.org/wiki/Jupyter
 
https://docs.arduino.cc/resources/pinouts/ABX00083-full-pinout.pdf
 
https://towardsdatascience.com/micropython-on-esp-using-jupyter-6f366ff5ed9
 
https://www.codemotion.com/magazine/backend/getting-started-with-micropython-on-arduino-nano-rp2040-connect/
 
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-openmv-setup/
 
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-data-logger/
 
https://www.penguintutor.com/programming/arduino-python
 
https://micropython.org/download/ARDUINO_NANO_RP2040_CONNECT/
 
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-ap-web-server-rgb/
 
https://www.datacamp.com/tutorial/markdown-in-jupyter-notebook
 
https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html
 
https://docs.micropython.org/en/latest/library/socket.html
 
https://ubicomp.net/sw/db1/var2db.php?
 
http://localhost:8888/notebooks/ArduinoNanoRP2040_v01.ipynb
 
https://labs.arduino.cc/en/labs/micropython
 
https://labs.arduino.cc/en/labs/micropython-installer
 
https://micropython.org/
 
https://www.arduino.cc/en/software
 
https://www.linkedin.com/pulse/arduino-truly-tiny-machine-learning-simone-salerno
 
https://eloquentarduino.com/posts/micropython-machine-learning
 
https://github.com/eloquentarduino/everywhereml
 
https://github.com/mocleiri/tensorflow-micropython-examples
 
https://dev.to/tkeyo/tinyml-machine-learning-on-esp32-with-micropython-38a6
 
https://saturncloud.io/blog/how-to-import-python-file-as-module-in-jupyter-notebook/
 
https://github.com/jposada202020/MicroPython_LSM6DSOX
 
 
 
 
= Tasks =
 
= Tasks =
  
Line 52: Line 23:
 
** it was moved "quickly"
 
** it was moved "quickly"
  
 +
== Task 3: ML on Arduino Nano Connect RP2040 ==
 +
* use https://github.com/eloquentarduino/everywhereml to detect the same gestures as in 2.2
  
== Task 3: connect both boards to WIFI ==
+
== Task 4: connect both boards to WIFI ==
 
* connect both boards to WIFI using [[Tutorial_Network]]
 
* connect both boards to WIFI using [[Tutorial_Network]]
 
* use the Arduino Nano ESP32 as output (showing a color)
 
* use the Arduino Nano ESP32 as output (showing a color)
 
* use the Arduino Nano Connect RP2040 as input (recognize with rules 3 gestures)
 
* use the Arduino Nano Connect RP2040 as input (recognize with rules 3 gestures)
 +
 +
 +
= Links =
 +
 +
== Micropython Basics ==
 +
https://docs.arduino.cc/micropython/#micropython-101
 +
https://docs.arduino.cc/micropython/basics/board-examples/
 +
https://www.codemotion.com/magazine/backend/getting-started-with-micropython-on-arduino-nano-rp2040-connect/
 +
https://www.penguintutor.com/programming/arduino-python
 +
https://micropython.org/
 +
https://docs.arduino.cc/micropython/micropython-course/course/installation/
 +
https://docs.arduino.cc/micropython/micropython-course/course/examples/
 +
https://wellys.com/posts/rp2040_micropython_1/
 +
https://micropython.org/download/RPI_PICO_W/
 +
 +
== Python / Jupyter Notebooks for Hardware ==
 +
https://www.sketching-with-hardware.org/wiki/Jupyter
 +
https://towardsdatascience.com/micropython-on-esp-using-jupyter-6f366ff5ed9
 +
https://www.datacamp.com/tutorial/markdown-in-jupyter-notebook
 +
https://saturncloud.io/blog/how-to-import-python-file-as-module-in-jupyter-notebook/
 +
https://jupyter.org/install
 +
https://www.geeksforgeeks.org/install-jupyter-notebook-in-windows
 +
 +
== Development environments ==
 +
https://labs.arduino.cc/en/labs/micropython
 +
https://labs.arduino.cc/en/labs/micropython-installer
 +
https://www.arduino.cc/en/software
 +
 +
== Libraries ==
 +
https://github.com/jposada202020/MicroPython_LSM6DSOX
 +
 +
== Data sheets and resources ==
 +
https://micropython.org/download/ARDUINO_NANO_RP2040_CONNECT/
 +
https://docs.arduino.cc/resources/pinouts/ABX00083-full-pinout.pdf
 +
 +
 +
== Tutorials ==
 +
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-openmv-setup/
 +
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-data-logger/
 +
 +
== Machine Learning Basics ==
 +
https://github.com/eloquentarduino/everywhereml
 +
https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html
 +
https://www.linkedin.com/pulse/arduino-truly-tiny-machine-learning-simone-salerno
 +
https://eloquentarduino.com/posts/micropython-machine-learning
 +
https://github.com/mocleiri/tensorflow-micropython-examples
 +
https://dev.to/tkeyo/tinyml-machine-learning-on-esp32-with-micropython-38a6
 +
 +
== Networking Basics ==
 +
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-ap-web-server-rgb/
 +
https://docs.micropython.org/en/latest/library/socket.html
 +
 +
== Local Links ==
 +
https://ubicomp.net/sw/db1/var2db.php?
 +
http://localhost:8888/notebooks/ArduinoNanoRP2040_v01.ipynb
 +
http://localhost:8888/doc/tree/create-ML-model01.ipynb
 +
 +
= Reading =
 +
== Required Reading before the course ==
 +
Albrecht Schmidt. 2020. Interactive Human Centered Artificial Intelligence: A Definition and Research Challenges. In Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). Association for Computing Machinery, New York, NY, USA, Article 3, 1–4. https://doi.org/10.1145/3399715.3400873 (4p) https://uni.ubicomp.net/as/iHCAI2020.pdf
 +
 +
A. Schmidt and K. van Laerhoven, "How to build smart appliances?," in IEEE Personal Communications, vol. 8, no. 4, pp. 66-71, Aug. 2001, doi: 10.1109/98.944006. (6p)
 +
https://www.eti.uni-siegen.de/ubicomp/papers/sl_ieeepc2001.pdf
 +
 +
Schmidt, Albrecht. "Understanding and researching through making: a plea for functional prototypes." interactions 24.3 (2017): 78-81. (4p)
 +
https://www.sketching-with-hardware.org/files/functional3058498.pdf
 +
 +
Huy Viet Le, Sven Mayer, and Niels Henze. 2020. Deep learning for human-computer interaction. interactions 28, 1 (January - February 2021), 78–82. (5p)
 +
https://doi.org/10.1145/3436958 https://sven-mayer.com/wp-content/uploads/2021/01/huy2021deep.pdf
 +
 +
Le, Huy Viet; Mayer, Sven; Weiß, Max; Vogelsang, Jonas; Weingärtner, Henrike; Henze, Niels (2020) Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones. In: ACM Trans. Comput.-Hum. Interact., vol. 27, no. 5, pp. 38, 2020, ISSN: 1073-0516. (38p)
 +
https://sven-mayer.com/wp-content/uploads/2020/09/le2020shortcuts.pdf
 +
 +
Hurwitz, Judith, and Daniel Kirsch. "Machine learning for dummies." IBM Limited Edition 75 (2018): 9780429196645-6. https://www.ibm.com/downloads/cas/GB8ZMQZ3 (Pages 3-18 and 29-47, this is Chapters 1 and 3) (35p)
 +
 +
Chris Garrett. MicroPython: An Intro to Programming Hardware in Python https://realpython.com/micropython/ (14 pages)
 +
 +
MicroPython Basics https://docs.arduino.cc/micropython/basics/micropython-basics/ (5 pages)
 +
 +
== Recommended Reading before the course: ==
 +
John D. Kelleher, Deep Learning, https://mitpress.mit.edu/9780262537551/deep-learning/
 +
 +
Yuli Vasiliev, Python for Data Science: A Hands-On Introduction, https://nostarch.com/python-data-science
 +
 +
Tutorial on Jupyter Notebooks: https://www.datacamp.com/tutorial/tutorial-jupyter-notebook
 +
 +
Smola, Alex, and S. V. N. Vishwanathan. "Introduction to machine learning." Cambridge University, UK 32.34 (2008): 2008. https://alex.smola.org/drafts/thebook.pdf 
 +
 +
 +
= Random Commands =
 +
pip install micropython-lsm6dsox
 +
 +
picotool.exe load -x C:\Users\ru42qak\AppData\Roaming\OpenMV\openmvide\firmware\ARDUINO_NANO_RP2040_CONNECT\firmware.bin
 +
 +
pip install jupyterlab 
 +
 +
pip install everywhereml     
 +
 +
python -m pip install jupyter
 +
 +
git clone https://github.com/goatchurchprime/jupyter_micropython_kernel.git
 +
 +
pip install -e jupyter_micropython_kernel
 +
 +
python -m notebook
 +
 +
python -m jupyter kernelspec list
 +
 +
 +
C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>pip install -e jupyter_micropython_kernel
 +
 +
C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>python -m notebook

Latest revision as of 01:41, 23 May 2024

Tasks[edit]

Task 1: connect a Arduino Nano ESP32 board[edit]

  • connect the board
  • control the internal RGB LED


Task 2: connect a Arduino Nano Connect RP2040[edit]

  • connect the board
  • read the accelerometer and the gyro and show it in the notebook

Task 2.1: is it moved?[edit]

  • read acceleration and gyro
  • calculate the differences between values
  • show an ouput when it is move
  • create a file on the device that logs, when it is moved

Task 2.2: it was turned upside down?[edit]

  • read acceleration and gyro
  • make a rule based "AI" that records
    • it was put upside down
    • it was turned 360
    • it was moved "quickly"

Task 3: ML on Arduino Nano Connect RP2040[edit]

Task 4: connect both boards to WIFI[edit]

  • connect both boards to WIFI using Tutorial_Network
  • use the Arduino Nano ESP32 as output (showing a color)
  • use the Arduino Nano Connect RP2040 as input (recognize with rules 3 gestures)


Links[edit]

Micropython Basics[edit]

https://docs.arduino.cc/micropython/#micropython-101 https://docs.arduino.cc/micropython/basics/board-examples/ https://www.codemotion.com/magazine/backend/getting-started-with-micropython-on-arduino-nano-rp2040-connect/ https://www.penguintutor.com/programming/arduino-python https://micropython.org/ https://docs.arduino.cc/micropython/micropython-course/course/installation/ https://docs.arduino.cc/micropython/micropython-course/course/examples/ https://wellys.com/posts/rp2040_micropython_1/ https://micropython.org/download/RPI_PICO_W/

Python / Jupyter Notebooks for Hardware[edit]

https://www.sketching-with-hardware.org/wiki/Jupyter https://towardsdatascience.com/micropython-on-esp-using-jupyter-6f366ff5ed9 https://www.datacamp.com/tutorial/markdown-in-jupyter-notebook https://saturncloud.io/blog/how-to-import-python-file-as-module-in-jupyter-notebook/ https://jupyter.org/install https://www.geeksforgeeks.org/install-jupyter-notebook-in-windows

Development environments[edit]

https://labs.arduino.cc/en/labs/micropython https://labs.arduino.cc/en/labs/micropython-installer https://www.arduino.cc/en/software

Libraries[edit]

https://github.com/jposada202020/MicroPython_LSM6DSOX

Data sheets and resources[edit]

https://micropython.org/download/ARDUINO_NANO_RP2040_CONNECT/ https://docs.arduino.cc/resources/pinouts/ABX00083-full-pinout.pdf


Tutorials[edit]

https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-openmv-setup/ https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-data-logger/

Machine Learning Basics[edit]

https://github.com/eloquentarduino/everywhereml https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html https://www.linkedin.com/pulse/arduino-truly-tiny-machine-learning-simone-salerno https://eloquentarduino.com/posts/micropython-machine-learning https://github.com/mocleiri/tensorflow-micropython-examples https://dev.to/tkeyo/tinyml-machine-learning-on-esp32-with-micropython-38a6

Networking Basics[edit]

https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-ap-web-server-rgb/ https://docs.micropython.org/en/latest/library/socket.html

Local Links[edit]

https://ubicomp.net/sw/db1/var2db.php? http://localhost:8888/notebooks/ArduinoNanoRP2040_v01.ipynb http://localhost:8888/doc/tree/create-ML-model01.ipynb

Reading[edit]

Required Reading before the course[edit]

Albrecht Schmidt. 2020. Interactive Human Centered Artificial Intelligence: A Definition and Research Challenges. In Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). Association for Computing Machinery, New York, NY, USA, Article 3, 1–4. https://doi.org/10.1145/3399715.3400873 (4p) https://uni.ubicomp.net/as/iHCAI2020.pdf

A. Schmidt and K. van Laerhoven, "How to build smart appliances?," in IEEE Personal Communications, vol. 8, no. 4, pp. 66-71, Aug. 2001, doi: 10.1109/98.944006. (6p) https://www.eti.uni-siegen.de/ubicomp/papers/sl_ieeepc2001.pdf

Schmidt, Albrecht. "Understanding and researching through making: a plea for functional prototypes." interactions 24.3 (2017): 78-81. (4p) https://www.sketching-with-hardware.org/files/functional3058498.pdf

Huy Viet Le, Sven Mayer, and Niels Henze. 2020. Deep learning for human-computer interaction. interactions 28, 1 (January - February 2021), 78–82. (5p) https://doi.org/10.1145/3436958 https://sven-mayer.com/wp-content/uploads/2021/01/huy2021deep.pdf

Le, Huy Viet; Mayer, Sven; Weiß, Max; Vogelsang, Jonas; Weingärtner, Henrike; Henze, Niels (2020) Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones. In: ACM Trans. Comput.-Hum. Interact., vol. 27, no. 5, pp. 38, 2020, ISSN: 1073-0516. (38p) https://sven-mayer.com/wp-content/uploads/2020/09/le2020shortcuts.pdf

Hurwitz, Judith, and Daniel Kirsch. "Machine learning for dummies." IBM Limited Edition 75 (2018): 9780429196645-6. https://www.ibm.com/downloads/cas/GB8ZMQZ3 (Pages 3-18 and 29-47, this is Chapters 1 and 3) (35p)

Chris Garrett. MicroPython: An Intro to Programming Hardware in Python https://realpython.com/micropython/ (14 pages)

MicroPython Basics https://docs.arduino.cc/micropython/basics/micropython-basics/ (5 pages)

Recommended Reading before the course:[edit]

John D. Kelleher, Deep Learning, https://mitpress.mit.edu/9780262537551/deep-learning/

Yuli Vasiliev, Python for Data Science: A Hands-On Introduction, https://nostarch.com/python-data-science

Tutorial on Jupyter Notebooks: https://www.datacamp.com/tutorial/tutorial-jupyter-notebook

Smola, Alex, and S. V. N. Vishwanathan. "Introduction to machine learning." Cambridge University, UK 32.34 (2008): 2008. https://alex.smola.org/drafts/thebook.pdf


Random Commands[edit]

pip install micropython-lsm6dsox

picotool.exe load -x C:\Users\ru42qak\AppData\Roaming\OpenMV\openmvide\firmware\ARDUINO_NANO_RP2040_CONNECT\firmware.bin

pip install jupyterlab

pip install everywhereml

python -m pip install jupyter

git clone https://github.com/goatchurchprime/jupyter_micropython_kernel.git

pip install -e jupyter_micropython_kernel

python -m notebook

python -m jupyter kernelspec list


C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>pip install -e jupyter_micropython_kernel

C:\Users\ru42qak\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\jupyterlab>python -m notebook