Embodied AI - direct applications to tennis/table tennis robotics

Meditations

10:09 AM

Started my day off right - meditated for 30 minutes. Set the frame of reference correct.

Embodied robotics, flow states.

Workspace

Is the embodied AI, robotics experience similar to that of humans? Especially when inputs are raw sensory encoding for movements, we would theoretically need greater scale of compute right?

Robots can be deployed on a greater scale, while optimally strategizing actions - for better or wose with humans.

Black swan event with NVIDIA robotics.

Can robotics learn to beat a human in tennis, 100 times out of 100? How would you theoretically build a robot who can beat a human in tennis

[DeepMind’s Quest for Self-Improving Table Tennis Agents How robots can learn new skills by challenging each other](

Google DeepMind vs Human - Table Tennis

ArXiv Paper

People to Reach Out:

Pannag Sanketi
Laura Graesser

Related Research by this Team

Robotic Table Tennis: A Case Study into a High Speed Learning System

i-Sim2Real: Reinforcement Learning of Robotic Policies in Tight Human-Robot Interaction Loops

Kinesthetic Learning

GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot

Agile Catching with Whole-Body MPC and Blackbox Policy Learning

Robotic Table Tennis with Model-Free Reinforcement Learning

Workspace

Install tkinter first so python mounts correctly

brew install python-tk@3.11

this is the pain of using open source, robotics on python. I keep on having these version, independency issues, especially with having to reinstall pip versus brew, worrying about the architecture of my operating system (arm64 vs x86), and the cascading number of modules independencies running from trimesh, igl, tkinter… the list goes on.

I need to figure out how Docker works so we can just have a deployable version that is containerized.

Workspace

Install tkinter first so python mounts correctly

brew install python-tk@3.11

I need to figure out how Docker works so we can just have a deployable version that is containerized.

11:44 PM

I reinstalled, reconfigured my environment so hierarchy of dependencies and paths are solved

# 0. outside any venv ---------------------------------------------------------
brew install tcl-tk glfw  # glfw is needed system-wide

# 1. create env ---------------------------------------------------------------
python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip wheel setuptools

# 2. torch (CPU/MPS) ---------------------------------------------------------
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

# 3. genesis + viewer deps ----------------------------------------------------
pip install genesis-world
pip install taichi==1.6.1 PyGEL3D==0.1.0         # quiet the duplicate-glfw warning
pip install "trimesh[easy]" pyrender PyOpenGL glfw

# 4. sanity test --------------------------------------------------------------
python - <<'PY'
import genesis as gs
gs.init(backend=gs.cpu)
scene = gs.Scene(show_viewer=False)  # first run headless
print("Genesis core OK")
PY

I ran a monkey path on the main .py script here, which enabled a 3 component vs 4 component mismatch to be addressed.

# 1_helloWorld.py  (top of file)

import igl as _igl          # grab the module first
_orig_sd = _igl.signed_distance

def _sd(*args, **kwargs):
    out = _orig_sd(*args, **kwargs)   # call the real function
    # PyIGL ≥2.3.0 returns 4 values; older Genesis expects 3
    return out[:3] if isinstance(out, tuple) and len(out) == 4 else out

_igl.signed_distance = _sd   # monkey-patch in place

Also, implementing “sanity check” code without running main 1_helloWorld.py script allows for more modular debugging - my Genesis has been installed properly, but the actual simulation rendering is running into several MacOS specific edge cases.

python - <<'PY'
import genesis as gs
gs.init(backend=gs.cpu)
scene = gs.Scene(show_viewer=False)  # first run headless
print("Genesis core OK")
PY